---
title: "Serverless functions with @lium.machine"
sidebar_label: "@lium.machine functions"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lium.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Serverless functions with `@lium.machine`

`@lium.machine` turns a Python function into one that runs on a rented GPU pod: rent, boot, ship the function, run it, bring the result back, remove the pod. You call it like any other function.

```python
import lium

@lium.machine(machine="RTX4090")
def matmul(n: int) -> float:
    import torch
    a = torch.randn(n, n, device="cuda")
    return float((a @ a).sum())

print(matmul(4096))
```

```text
[lium] matmul: renting 1xRTX4090 $0.30/h (swift-fox-c8, Romania), removal in 1.2h
[lium] matmul: pod ready in 38s
[lium] matmul: running
[lium] matmul: done in 71s (~$0.0059)
[lium] matmul: pod removed
-1342.5
```

Authenticate once with `lium init` (or set `LIUM_API_KEY`). The behaviour described here is that of the `lium` package **since release 0.0.40** (lium#208). The decorator in 0.0.37–0.0.39 takes `machine`, `template_id`, `cleanup` and `requirements` only — no `timeout`, `keep_warm`, `.map()` or `.close()` — rents the first node whose name contains the string, installs `requirements` into a venv that does not see the image's torch, and returns JSON-serialisable results only.

## How a call works

1. **Pick a node.** `machine` is `"<count>x<gpu>"` or `"<gpu>"` — `"1xH200"`, `"RTX4090"`, `"2xA100"`; the count defaults to 1. The cheapest available node with exactly that many GPUs of that type is rented. If none matches you get the list of what exists (`No node found matching machine type: 2xA100. Available: 1xA100 $1.20/h, 8xA100 $3.60/h.`).
2. **Bound the cost.** The pod is scheduled for removal at `timeout + 15 min` the moment it is rented (`timeout` defaults to one hour; with `timeout=None` the removal is set 24 hours out), so a caller that crashes or is killed cannot leave a pod billing past that point — the bound is the `timeout` you chose, plus `keep_warm` when set. `cleanup=False` does not cancel it (see [Parameters](#parameters)).
3. **Ship the function.** Only the function's own `def` is sent — decorators and annotations stripped. Arguments are pickled and go with it (your own bytes, loaded on your own pod).
4. **Prepare the environment.** A venv that sees the image's own packages (the default PyTorch template ships torch and CUDA) is created once per pod and per `requirements` list; `pip install` runs once, later calls find it ready.
5. **Run and relay.** The function runs under `timeout`; everything it prints reaches your terminal while it runs.
6. **Return or raise.** The result comes back as a JSON envelope plus an `.npz` sidecar for numpy arrays, read with `allow_pickle=False` — nothing the pod writes is unpickled on your machine. If the function raised a builtin exception, you get the same type here (`except ValueError` works); its `__cause__` is a `lium.RemoteExecutionError` with `remote_traceback`, `exit_code`, `stdout` and `stderr`.
7. **Release the pod.** Removed when the call returns — or kept for the next call with `keep_warm`.

## What a call costs

Measured on 6 Sep 2026 from a laptop to a 1×RTX 4090 node at $0.30/h:

| | wall time | of which the function | cost |
|---|---|---|---|
| cold call (rent, boot, venv, run, return) | ~70 s | ~5 s | ~$0.006 |
| second call on a warm pod | ~19 s | ~4 s | ~$0.002 |
| `requirements=["transformers", "accelerate"]`, first call on a pod | +32 s | | |
| same requirements, later call on that pod | +1 s | | |
| idle on a warm pod (`keep_warm=600`) | 10 min per window | — | ~$0.05 per window (~$0.06 after the last call, removal comes 2 min later) |

The fixed overhead is one SSH connection (~3 s to a distant node), the Python start-up on the pod, and the result transfer. `keep_warm` is what turns a 70 s call into a 19 s one — and the pod bills through the whole warm window whether or not a call arrives: ten idle minutes at $0.30/h cost 25× the warm call itself. Size `keep_warm` to the gap between your calls, and `close()` when you are done.

## Example 1 — a GPU operation on the image's torch

No `requirements`: the default template already has torch with CUDA.

```python
import lium

@lium.machine(machine="RTX4090")
def bench(n: int) -> dict:
    import time
    import torch
    a = torch.randn(n, n, device="cuda", dtype=torch.float16)
    torch.cuda.synchronize()
    t = time.perf_counter()
    for _ in range(10):
        a @ a
    torch.cuda.synchronize()
    ms = (time.perf_counter() - t) * 100
    tflops = 2 * n ** 3 / (ms / 1000) / 1e12
    print(f"{torch.cuda.get_device_name(0)}: {ms:.1f} ms per {n}x{n} fp16 matmul")
    return {"device": torch.cuda.get_device_name(0), "ms": round(ms, 1), "tflops": round(tflops, 1)}

print(bench(8192))
```

Return plain Python types. `torch.__version__` is a `TorchVersion` object and a tensor is a tensor: neither travels back from the pod (the call ends with a `lium.ResultEncodingError` naming the type) — use `str(...)`, `.tolist()` or `.cpu().numpy()`.

## Example 2 — transformers inference, with `requirements`

Torch comes from the image; `transformers` and `accelerate` are installed once per pod.

```python
import lium

@lium.machine(machine="RTX4090", requirements=["transformers", "accelerate"], timeout=600)
def generate(prompt: str, max_new_tokens: int = 64) -> str:
    import torch
    from transformers import AutoModelForCausalLM, AutoTokenizer

    model_id = "HuggingFaceTB/SmolLM2-135M-Instruct"
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")
    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to("cuda")
    out = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False,
                         pad_token_id=tokenizer.eos_token_id)
    return tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True).strip()

try:
    print(generate("In one sentence, who discovered penicillin?"))
except lium.RemoteExecutionError as e:      # a timeout, a killed process, or a non-builtin remote exception
    print(e, e.remote_traceback)
except Exception as e:                      # a builtin remote exception, e.g. OSError for a bad model id
    print(type(e).__name__, e)
    if isinstance(e.__cause__, lium.RemoteExecutionError):
        print(e.__cause__.remote_traceback)  # what happened on the pod
```

While it runs you see the model download and `Loading weights: 100%|██████████|` progress from the pod, then the answer.

## Example 3 — a warm pod, a batch, and a local escape hatch

`keep_warm=600` keeps the pod ten minutes after each call. The next call — from this loop, or from the next run of the script while you iterate — reuses it. `.map()` runs every item on one pod. `.close()` removes the pod when you are done; if you forget, the server removes it `keep_warm + 2 min` after the last call.

```python
import lium

@lium.machine(machine="RTX4090", requirements=["sentence-transformers"], keep_warm=600)
def embed(texts: list[str]) -> list[list[float]]:
    from sentence_transformers import SentenceTransformer
    model = SentenceTransformer("all-MiniLM-L6-v2", device="cuda")
    return model.encode(texts, normalize_embeddings=True).tolist()

batches = [["a red apple", "a green pear"], ["a fast car"], ["an old boat", "a new ship"]]
vectors = embed.map(batches)                 # one pod, three calls
print(len(vectors), len(vectors[0][0]))      # 3 384

embed.close()                                # remove the warm pod now
```

For tests and offline work, `embed.local(texts)` runs the original function here, and `@lium.machine(..., local=True)` (or `LIUM_MACHINE_LOCAL=1` in the environment) makes every call local without editing the code — all three arrived in 0.0.40 (lium#208); 0.0.37–0.0.39 have none of them.

## What can and cannot travel

- **Arguments** are pickled — anything both sides can import; an argument that cannot be pickled is refused before anything is rented.
- **Results** are not pickled (the file comes from provider hardware). What round-trips, exactly, each as its own type: `None`, `bool`, `int`, `float`, `str`, `bytes`; `list`, `tuple`, `set`, `frozenset` and `dict` of those, nested; `datetime`/`date`/`time`/`timedelta`, `Decimal`, `pathlib.Path`, `uuid.UUID`; `numpy.ndarray` of any dtype without Python objects (any shape, structured and datetime64 included) and numpy scalars, carried in an `.npz` read with `allow_pickle=False`. Anything else — a dataclass, an `Enum`, an `OrderedDict`, a tensor, an ndarray subclass — is a `lium.ResultEncodingError` raised on the pod that names the type and where it sits in the result; return `.tolist()`, `dict(x)`, `x.value` instead.
- **Only the function's `def` travels.** Import inside the body. A module-level constant, import or helper used inside the function is refused when the module loads, with the names listed — because on the pod it would be a `NameError` a minute and a few cents later. A closure over a local variable is refused the same way; pass it as an argument.
- **Nested functions and `async def`** work (`async` functions are awaited on the pod). A method works only when `self`'s class is importable on the pod — a class defined in the script itself is not. Lambdas do not.
- **Output**: `print()` and anything written to stdout/stderr on the pod appears in your terminal as it happens, and stays on `RemoteExecutionError.stdout/.stderr` if the call fails. `quiet=True` silences only the `[lium]` progress lines.
- **Errors**: a builtin remote exception (`ValueError`, `OSError`, …) is re-raised with its type and `lium.RemoteExecutionError` as its `__cause__`; any other exception class (a library's own, say `torch.OutOfMemoryError`), or a remote `sys.exit()` surfaces as `lium.RemoteExecutionError` itself; `timeout` raises `RemoteExecutionError: <fn> exceeded timeout=…s and was killed`; a process killed on the pod (out of memory, for instance) says so.
- **Ctrl-C** removes the pod (or leaves it warm under `keep_warm`). A caller that dies without running its cleanup leaves the pod to its scheduled removal.

## Parameters

| parameter | default | meaning |
|---|---|---|
| `machine` | — | `"<count>x<gpu>"` or `"<gpu>"`; cheapest matching node |
| `requirements` | `None` | pip packages, installed once per pod into a venv that also sees the image's packages |
| `template_id` | node default | Docker template to rent with |
| `timeout` | `3600` | seconds the function may run; `None` for no limit. Pod removal is scheduled at `timeout + 15 min` (24 h with `None`) |
| `keep_warm` | `0` | seconds the pod stays after a call for the next one, **billed while idle**; removal re-armed to `keep_warm + 2 min` after each call |
| `cleanup` | `True` | `False` leaves the pod running when the call returns (remove it with `lium rm`). It also turns off `keep_warm` and warm-pod reuse, so `.map()` rents a separate pod per item. The removal scheduled at rent time still stands — cancel it with [`lium schedules rm`](/developers/cli/reference/schedules) if the pod must outlive it |
| `local` | `False` | run in this process; `LIUM_MACHINE_LOCAL=1` does the same for every function (since 0.0.40, lium#208) |
| `quiet` | `False` | no `[lium]` progress lines on stderr |

Methods on the decorated function: `f.remote(*args)` (same as `f(*args)`), `f.local(*args)`, `f.map(iterable)`, `f.close()`.