Skip to main content

Before you rent 8 GPUs

Two things decide whether a multi-GPU node suits your job. Check both in the first two minutes of a rental.

  1. How the GPUs are wired. Tensor parallelism and FSDP need NVLink. On PCIe they run several times slower, and without peer-to-peer (P2P) NCCL may not start at all.
  2. How fast the node downloads. The same 756 GB checkpoint took 12 minutes on one 8-GPU node and 75 minutes on another.

Jobs with one independent process per GPU (batch inference, sweeps) run fine on any wiring.

Check the wiring​

Run on the pod

nvidia-smi topo -m
nvidia-smi topo -p2p r      # OK = P2P reads work, NS = not supported

Or from your machine: lium exec <pod> "nvidia-smi topo -m && nvidia-smi topo -p2p r".

Read the result

  • NV# in every GPU pair (e.g. NV18): NVLink. Good for tensor parallelism.
  • PIX, PXB, PHB, NODE or SYS: PCIe. Slower.
  • NS in the P2P matrix: no peer-to-peer. NCCL won't work with its defaults.

If it's not what you need

Delete the pod and pick another node.

Check the download speed​

Use the disk you will download to (/workspace, not /root):

# a real Hugging Face object, ~550 MB  (pip install -U huggingface_hub if `hf` is missing)
cd /workspace && time hf download openai-community/gpt2 model.safetensors --local-dir /workspace/probe

# a 50 MB CDN object; prints the HTTP code and bytes per second
curl -o /dev/null -sS -w 'ingress HTTP %{http_code} %{speed_download} B/s\n' 'https://speed.cloudflare.com/__down?bytes=50000000'

At 45 MB/s a 750 GB checkpoint takes 4.6 hours; at 1 GB/s it takes 12.5 minutes. If it doesn't fit your budget, delete the pod now.

The Download figure in Browse Pods does not predict Hugging Face or S3 speed.

Filter before you rent​

In Browse Pods, an NVLink badge marks a node where every GPU pair is on NVLink. NVLink 2/6 pairs means only some pairs are bridged. Hover the badge for the link bandwidth.

GET /executors shows each node's wiring and filters on nvlink=true and min_download_mbps. The lium ls Link column shows the same. See Interconnect and download speed.

Last updated on