Skip to main content

Docker Storage Setup

Lium expects every node's Docker daemon to use the overlay2 storage driver backed by an xfs filesystem mounted with pquota and ftype=1. The lium CLI inspects the host, formats the target device, and migrates /var/lib/docker for you.

When you need this
  • Required today if you want to enable GPU splitting — preflight blocks the portal toggle until the storage driver and backing filesystem match.
  • Required for every node soon — Lium will extend this requirement to all nodes regardless of GPU splitting, so completing this setup now future-proofs the host.

The rest of this page covers the host-side setup and verification. Once the host is ready, GPU-splitting providers finish the portal-side step from Managing Nodes → Setting Minimum GPU Count for Rental.

Before You Run the CLI​

The --device argument is the storage target Lium will use for Docker storage.

You do not need to:

  • pre-format it as XFS
  • mount it at /var/lib/docker
  • manually configure overlay2

Lium handles partition creation, XFS formatting, mount setup, and Docker storage migration after the target passes safety checks.

Safe Device States​

Lium will use the device when it is one of these states:

  • a blank whole disk
  • a whole disk with one unambiguous free-space region
  • an unused partition with no filesystem
  • an empty XFS partition with ftype=1

Device States That Will Be Rejected​

Lium will skip or reject the device when it is:

  • the root disk
  • the current Docker backing device
  • mounted or otherwise in use
  • a partition with an existing non-XFS filesystem
  • a non-empty XFS partition
  • a disk with an ambiguous layout
  • a loop or device-mapper target

Prepare the Device​

The safest path is to attach a fresh extra SSD/NVMe and leave it unused and unmounted before you run the CLI.

  1. Attach a new extra disk.
  2. Make sure it is not your root disk.
  3. Make sure it is not the device currently backing /var/lib/docker.
  4. Pass that whole-disk path to the CLI.

Quick Checks​

Use these commands before running setup:

# See the candidate device, filesystem, and mount state
lsblk -o NAME,PATH,TYPE,FSTYPE,SIZE,MOUNTPOINTS

# See which device backs /
findmnt /

# See which device currently backs Docker storage
findmnt /var/lib/docker

If you want to reuse an existing XFS partition, confirm it is XFS with ftype=1:

sudo xfs_info /dev/<partition>

Common Remediation​

  • If the device is mounted, unmount it first and confirm nothing important is using it.
  • If the device has old filesystems or old data, the safest option is to use a different empty disk.
  • If you want to reuse the same disk and it is safe to erase, reset it to a clean disk or unused partition before rerunning the CLI.
Single-disk hosts

If you only have one drive, there is no supported single-disk path. The lium gpu-splitting CLI explicitly rejects the root disk and the disk currently backing /var/lib/docker, and this page does not document a manual workaround. Reusing the only drive on the host would mean repartitioning the live system — that is risky, unsupported, and not covered by the CLI safety checks. The only supported options today are: attach a second disk, or reinstall the host onto an XFS root before bringing the node back online.

Example: reset a dedicated extra disk so Lium can partition and format it itself:

# Warning: destructive. Replace /dev/nvme1n1 with the extra disk you want to erase.
sudo umount /dev/nvme1n1p1 2>/dev/null || true
sudo parted -s /dev/nvme1n1 mklabel gpt

After that, rerun:

lium gpu-splitting check --device /dev/nvme1n1

Run the Storage Setup​

Use the CLI in this order:

# Inspect the host and confirm the device is eligible
lium gpu-splitting check --device /dev/<device>

# Apply the storage migration
sudo lium gpu-splitting setup --device /dev/<device>

# Verify the final state
lium gpu-splitting verify

What each command does:

  • check inspects the host and prints the exact storage plan without changing anything
  • setup performs the Docker storage migration
  • verify confirms the final Docker storage state
CLI naming

The commands live under lium gpu-splitting because GPU splitting is the first feature that depends on this storage layout. The migration itself is a generic Docker storage setup and the same commands are used regardless of whether you plan to enable GPU splitting.

Destructive operation

setup stops Docker, formats the target partition as XFS, and remounts /var/lib/docker. It backs up existing Docker data first, but you should have an independent backup before running it on a host with live containers.

Verify the Host​

After setup, confirm these requirements are satisfied:

  • Docker storage driver is overlay2
  • Docker backing filesystem is xfs
  • Supports d_type is true
  • /var/lib/docker is mounted with pquota or prjquota

Quick verification:

docker info
findmnt /var/lib/docker
lium gpu-splitting verify

Manual setup​

If you can't or don't want to use the CLI, the manual sequence is: stop Docker → format the target partition as XFS with ftype=1 → mount it at /var/lib/docker with pquota → restore the previous contents → restart Docker.

# Stop Docker
sudo systemctl stop docker docker.socket

# Make sure /etc/docker/daemon.json sets the overlay2 storage driver
# {
#   "storage-driver": "overlay2"
# }

# Format the new partition (replace $new_partition)
new_partition=<your-new-partition-device-name>
sudo rsync -aXS /var/lib/docker/ /tmp/docker-backup/
sudo mkfs.xfs -n ftype=1 "$new_partition" -f
sudo mount -t xfs -o defaults,inode64,pquota "$new_partition" /var/lib/docker
sudo rsync -aXS /tmp/docker-backup/ /var/lib/docker/

# Make the mount permanent
UUID=$(sudo blkid -s UUID -o value "$new_partition")
echo "UUID=${UUID}  /var/lib/docker  xfs  defaults,inode64,pquota  0  2" | sudo tee -a /etc/fstab

# Start Docker
sudo systemctl start docker

Then run docker info and confirm Storage Driver: overlay2, Backing Filesystem: xfs, Supports d_type: true.

Next step (GPU splitting only)​

If you're enabling GPU splitting, finish the portal-side configuration:

  1. Open the Managing Nodes guide.
  2. Follow the Setting Minimum GPU Count for Rental steps.

For nodes that aren't using GPU splitting, no further action is needed — the host is ready and stays ready when the requirement extends to all providers.

Troubleshooting​

If the CLI does not use the device you passed, check these first:

  • it is not the root or current Docker device
  • it is not mounted
  • it does not contain an existing non-XFS filesystem
  • it is not a non-empty XFS partition
  • the disk layout is simple enough for Lium to classify safely

If you are unsure, the most reliable fix is to use a fresh extra disk and rerun lium gpu-splitting check --device /dev/<device>.