---
sidebar_position: 4
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lium.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Docker Storage Setup

Lium expects every node's Docker daemon to use the **`overlay2`** storage driver backed by an **`xfs`** filesystem mounted with **`pquota`** and `ftype=1`. The [`lium`](https://github.com/Datura-ai/lium) CLI inspects the host, formats the target device, and migrates `/var/lib/docker` for you.

:::info When you need this
- **Required today** if you want to enable [GPU splitting](./gpu-splitting.md) — preflight blocks the portal toggle until the storage driver and backing filesystem match.
- **Required for every node soon** — Lium will extend this requirement to all nodes regardless of GPU splitting, so completing this setup now future-proofs the host.
:::

The rest of this page covers the host-side setup and verification. Once the host is ready, GPU-splitting providers finish the portal-side step from [Managing Nodes → Setting Minimum GPU Count for Rental](../portal/managing-nodes.md#setting-minimum-gpu-count-for-rental).

## Before You Run the CLI

The `--device` argument is the storage target Lium will use for Docker storage.

You do **not** need to:

- pre-format it as XFS
- mount it at `/var/lib/docker`
- manually configure `overlay2`

Lium handles partition creation, XFS formatting, mount setup, and Docker storage migration after the target passes safety checks.

### Safe Device States

Lium will use the device when it is one of these states:

- a blank whole disk
- a whole disk with one unambiguous free-space region
- an unused partition with no filesystem
- an empty XFS partition with `ftype=1`

### Device States That Will Be Rejected

Lium will skip or reject the device when it is:

- the root disk
- the current Docker backing device
- mounted or otherwise in use
- a partition with an existing non-XFS filesystem
- a non-empty XFS partition
- a disk with an ambiguous layout
- a loop or device-mapper target

## Prepare the Device

### Recommended Beginner Path

The safest path is to attach a fresh extra SSD/NVMe and leave it unused and unmounted before you run the CLI.

1. Attach a new extra disk.
2. Make sure it is not your root disk.
3. Make sure it is not the device currently backing `/var/lib/docker`.
4. Pass that whole-disk path to the CLI.

### Quick Checks

Use these commands before running setup:

```bash
# See the candidate device, filesystem, and mount state
lsblk -o NAME,PATH,TYPE,FSTYPE,SIZE,MOUNTPOINTS

# See which device backs /
findmnt /

# See which device currently backs Docker storage
findmnt /var/lib/docker
```

If you want to reuse an existing XFS partition, confirm it is XFS with `ftype=1`:

```bash
sudo xfs_info /dev/<partition>
```

### Common Remediation

- If the device is mounted, unmount it first and confirm nothing important is using it.
- If the device has old filesystems or old data, the safest option is to use a different empty disk.
- If you want to reuse the same disk and it is safe to erase, reset it to a clean disk or unused partition before rerunning the CLI.

:::warning Single-disk hosts
If you only have one drive, there is **no supported single-disk path**. The `lium gpu-splitting` CLI explicitly rejects the root disk and the disk currently backing `/var/lib/docker`, and this page does not document a manual workaround. Reusing the only drive on the host would mean repartitioning the live system — that is risky, unsupported, and not covered by the CLI safety checks. The only supported options today are: attach a second disk, or reinstall the host onto an XFS root before bringing the node back online.
:::

Example: reset a dedicated extra disk so Lium can partition and format it itself:

```bash
# Warning: destructive. Replace /dev/nvme1n1 with the extra disk you want to erase.
sudo umount /dev/nvme1n1p1 2>/dev/null || true
sudo parted -s /dev/nvme1n1 mklabel gpt
```

After that, rerun:

```bash
lium gpu-splitting check --device /dev/nvme1n1
```

## Run the Storage Setup

Use the CLI in this order:

```bash
# Inspect the host and confirm the device is eligible
lium gpu-splitting check --device /dev/<device>

# Apply the storage migration
sudo lium gpu-splitting setup --device /dev/<device>

# Verify the final state
lium gpu-splitting verify
```

What each command does:

- `check` inspects the host and prints the exact storage plan without changing anything
- `setup` performs the Docker storage migration
- `verify` confirms the final Docker storage state

:::note CLI naming
The commands live under `lium gpu-splitting` because GPU splitting is the first feature that depends on this storage layout. The migration itself is a generic Docker storage setup and the same commands are used regardless of whether you plan to enable GPU splitting.
:::

:::warning Destructive operation
`setup` stops Docker, formats the target partition as XFS, and remounts `/var/lib/docker`. It backs up existing Docker data first, but you should have an independent backup before running it on a host with live containers.
:::

## Verify the Host

After setup, confirm these requirements are satisfied:

- Docker storage driver is `overlay2`
- Docker backing filesystem is `xfs`
- `Supports d_type` is `true`
- `/var/lib/docker` is mounted with `pquota` or `prjquota`

Quick verification:

```bash
docker info
findmnt /var/lib/docker
lium gpu-splitting verify
```

## Manual setup

If you can't or don't want to use the CLI, the manual sequence is: stop Docker → format the target partition as XFS with `ftype=1` → mount it at `/var/lib/docker` with `pquota` → restore the previous contents → restart Docker.

```bash
# Stop Docker
sudo systemctl stop docker docker.socket

# Make sure /etc/docker/daemon.json sets the overlay2 storage driver
# {
#   "storage-driver": "overlay2"
# }

# Format the new partition (replace $new_partition)
new_partition=<your-new-partition-device-name>
sudo rsync -aXS /var/lib/docker/ /tmp/docker-backup/
sudo mkfs.xfs -n ftype=1 "$new_partition" -f
sudo mount -t xfs -o defaults,inode64,pquota "$new_partition" /var/lib/docker
sudo rsync -aXS /tmp/docker-backup/ /var/lib/docker/

# Make the mount permanent
UUID=$(sudo blkid -s UUID -o value "$new_partition")
echo "UUID=${UUID}  /var/lib/docker  xfs  defaults,inode64,pquota  0  2" | sudo tee -a /etc/fstab

# Start Docker
sudo systemctl start docker
```

Then run `docker info` and confirm `Storage Driver: overlay2`, `Backing Filesystem: xfs`, `Supports d_type: true`.

## Next step (GPU splitting only)

If you're enabling GPU splitting, finish the portal-side configuration:

1. Open the [Managing Nodes guide](../portal/managing-nodes.md#setting-minimum-gpu-count-for-rental).
2. Follow the **Setting Minimum GPU Count for Rental** steps.

For nodes that aren't using GPU splitting, no further action is needed — the host is ready and stays ready when the requirement extends to all providers.

## Troubleshooting

If the CLI does not use the device you passed, check these first:

- it is not the root or current Docker device
- it is not mounted
- it does not contain an existing non-XFS filesystem
- it is not a non-empty XFS partition
- the disk layout is simple enough for Lium to classify safely

If you are unsure, the most reliable fix is to use a fresh extra disk and rerun `lium gpu-splitting check --device /dev/<device>`.