GPU profiling with ncu

Profile CUDA kernels with NVIDIA Nsight Compute (ncu) in a Lium pod. ncu needs GPU performance counters, which most machines keep closed, so you need a profiling machine.

Rent a profiling machine

Find one

In Browse Pods on lium.io, turn on GPU Profiling (ncu).

Rent it

Rent as usual. You always get the whole machine, so the price covers all its GPUs.

Run ncu

Get the toolkit

Use the CUDA devel (nvcc, full toolkit) template, or install the CUDA toolkit for the machine's driver (Max CUDA driver in the create-pod summary):

apt-get update && apt-get install -y cuda-toolkit-12-6

Profile

ncu --version                          # sanity check
ncu -o profile ./my_kernel_binary      # profile a binary, write profile.ncu-rep
ncu --set full -o profile python train.py

Open the report

Copy the .ncu-rep file to your computer with scp or rsync and open it in Nsight Compute.