---
sidebar_position: 2.5
title: SDK
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lium.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Lium SDK

The `lium.io` package ships both the [CLI](./cli/overview) and a **Python SDK** for managing GPU pods programmatically. Install it once and use whichever interface fits the job.

:::info SDK Reference
The generated Python SDK reference covers the public client, data models, exceptions, configuration, and decorators.

**[Open SDK Reference](./sdk/reference)**
:::

## Installation

```bash
pip install lium.io
```

## Authentication

For local development, authenticate once with the CLI:

```bash
lium init
```

This saves your API key to `~/.lium/config.ini`, which is shared by both the CLI and SDK. After that, `Lium()` can authenticate automatically:

```python
from lium.sdk import Lium

lium = Lium()
```

For CI, scripts, or temporary overrides, set `LIUM_API_KEY` instead:

```bash
export LIUM_API_KEY="sk_..."
```

`LIUM_API_KEY` takes precedence over the saved config file.

## Two Entry Points

The SDK exposes two ways to run work on Lium GPUs:

- **`@lium.machine` decorator** — annotate a Python function and offload it to a GPU pod. Best for quickly running isolated workloads.
- **`Lium()` client** — a direct client for long-lived orchestration code that manages pod lifecycles.

### `@lium.machine` decorator

Annotate a function with the machine and its dependencies, then call it like a normal Python function. The SDK rents the cheapest matching node, ships the function, installs the requirements once, streams its output, returns the result or re-raises its exception, and removes the pod — or keeps it warm for the next call.

```python
import lium

@lium.machine(machine="RTX4090", requirements=["transformers", "accelerate"], keep_warm=300)
def infer(prompt: str) -> str:
    import torch
    from transformers import AutoTokenizer, AutoModelForCausalLM
    model_id = "HuggingFaceTB/SmolLM2-135M-Instruct"
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")
    text = tokenizer.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to("cuda")
    out = model.generate(**inputs, max_new_tokens=64, do_sample=False, pad_token_id=tokenizer.eos_token_id)
    return tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True).strip()

print(infer("Who discovered penicillin?"))      # cold: rent + boot + install, ~1-2 min
print(infer("And who first mass-produced it?"))  # warm pod, cached environment: ~20 s
infer.close()                                   # remove the warm pod (otherwise: 300 s + 2 min later)
```

This snippet and the description below are the behaviour of the `lium` package **since release 0.0.40** (lium#208). The decorator in 0.0.37–0.0.39 takes `machine`, `template_id`, `cleanup` and `requirements` only — no `keep_warm`, `timeout`, `.map()` or `.close()` — rents the first node whose name contains the string (for `"A100"` that is an 8×A100 today), installs `requirements` into a venv that does not see the image's torch, and returns JSON-serialisable results only. Check `pip show lium` before copying it.

`machine` is `"<count>x<gpu>"` or `"<gpu>"` (`"1xH200"`, `"RTX4090"`, `"2xA100"`; count defaults to 1). Only the function's own `def` travels, so import inside it and pass everything else as arguments; arguments are pickled, the result comes back as JSON plus an `.npz` for numpy arrays (plain types, the stdlib value types, numpy — the [examples page](./sdk/examples/machine-functions#what-can-and-cannot-travel) lists exactly what round-trips). A remote builtin exception (`ValueError`, `OSError`, …) is re-raised with its own type, with `lium.RemoteExecutionError` (remote traceback, exit code, output) as its cause; any other exception class, a timeout or a killed process surfaces as `lium.RemoteExecutionError` itself. `timeout` (default 1 h) bounds the run, and the pod is scheduled for removal at `timeout + 15 min` from the moment it is rented. `f.map(items)` runs a batch on one pod; `f.local(...)` or `local=True` runs in-process.

For the full walk-through — what a call costs, three copy-paste examples, what can travel — see [Serverless functions with `@lium.machine`](./sdk/examples/machine-functions).

### `Lium()` client

The client mirrors the CLI's pod lifecycle — list nodes, bring a pod up, wait until it's ready, execute commands, and tear it down.

```python
from lium.sdk import Lium

lium = Lium()
ready = None

try:
    executor = lium.ls(gpu_type="A100", gpu_count=1)[0]
    pod = lium.up(executor_id=executor.id, name="demo")
    ready = lium.wait_ready(pod, timeout=600)
    if ready is None:
        raise RuntimeError("Pod did not become ready before the timeout")
    print(lium.exec(ready, command="nvidia-smi")["stdout"])
finally:
    if ready is not None:
        lium.down(ready)
```

For the complete CLI-equivalent workflow, see [Pod Lifecycle with `Lium()`](./sdk/examples/pod-lifecycle).

## Waiting for a pod: slow versus dead

:::note Since 0.0.35
`PodStartError`, the terminal-status detection in `wait_ready` and `lium up --ready-timeout` shipped in lium 0.0.35 (lium#214). On 0.0.33 and 0.0.34 `wait_ready` returns `None` at the timeout whatever happened to the pod.
:::

`wait_ready(pod, timeout=…)` returns the pod once it is `RUNNING` with an SSH command, and `None` only when the timeout passes while the pod is **still starting**. A pod that will never become ready raises `PodStartError` (a `LiumError`) as soon as that is known: when it reports a terminal status (`TERMINAL_POD_STATUSES` — `FAILED`, `STOPPED`, `ERROR`, `TERMINATED`, `DELETED`, `REMOVED`, `CANCELLED`, and the backend's own names `CREATION_FAILED`, `BROKEN`, `REBOOT_FAILED`, `DELETING`), when it disappears from the pod list after having been seen, or when an id is never listed for 20 s after the first poll (`Lium.MISSING_GRACE_SECONDS`, below). The exception carries `.pod_id`, `.pod`, `.status` and `.history` (the statuses seen while waiting); `.pod` and `.status` are `None` when the id was never listed. `.cause` is the failure the backend recorded for the pod (for example `Container creation failed due to Failed create_container (failure_step: ssh_connect)`), `None` when it recorded nothing readable. So the caller can clean up instead of renting another pod.

```python
from lium.sdk import Lium, PodStartError

lium = Lium()
executor = lium.ls(gpu_type="H100", gpu_count=1)[0]
pod = lium.up(executor_id=executor.id)
try:
    ready = lium.wait_ready(pod, timeout=600)   # None only if still starting after 600 s
except PodStartError as exc:
    label = exc.pod.huid if exc.pod is not None else exc.pod_id   # .pod is None if never listed
    print(f"{label} ended as {exc.status}: {exc.history}")
    raise
```

The CLI equivalent is [`lium up --ready-timeout`](./cli/reference/up.md#waiting).

### How often `wait_ready` polls

:::note Since 0.0.35
Released in lium 0.0.35 (lium#214). On 0.0.33 and 0.0.34 `wait_ready` called the pod list every `poll_interval` seconds (default 10) and returned `None` at the timeout whatever happened to the pod.
:::

`wait_ready(pod, timeout=…, poll_interval=None)` — `None`, the new default, polls every 2 s for the first 90 s of the wait and every 10 s after that (`Lium.FAST_POLL_SECONDS`, `Lium.FAST_POLL_WINDOW_SECONDS`, `Lium.SLOW_POLL_SECONDS`; `Lium.poll_delay(elapsed, poll_interval=None)` returns the delay for a given point in the wait). A pod the backend marks `RUNNING` at ~22 s is returned within 2 s instead of up to 10 s later; the cost is about 13 `GET /pods` calls instead of 3 during a 25 s start (the first poll is immediate). Pass a number to keep a fixed interval — `poll_interval=10` sleeps 10 s between polls as 0.0.34 did. The never-listed rule is a time budget, not a poll count: a pod that is not in the list 20 s after the first poll raises `PodStartError` (`Lium.MISSING_GRACE_SECONDS`) — also when `timeout` ends at that same moment — so an empty listing before the pod is first seen at the 2 s cadence does not fail a pod that is already billing; at a fixed 10 s interval that is the third check, 20 s after the first.

## Next Steps

Ready to go deeper? Open the **[generated SDK reference](./sdk/reference)** for signatures, arguments, return types, data models, and exceptions, or follow the **[SDK examples](./sdk/examples)** for complete workflows.

## Related

- [SDK Examples](./sdk/examples) — complete SDK workflows
- [CLI Installation](./cli/installation) — install the `lium.io` package
- [CLI Reference](./cli/reference) — per-command reference, grouped by category
- [CLI Quickstart](./cli/quickstart) — get started with the CLI