Serverless functions with `@lium.machine`

@lium.machine turns a Python function into one that runs on a rented GPU pod: rent, boot, ship the function, run it, bring the result back, remove the pod. You call it like any other function.

import lium

@lium.machine(machine="RTX4090")
def matmul(n: int) -> float:
    import torch
    a = torch.randn(n, n, device="cuda")
    return float((a @ a).sum())

print(matmul(4096))

[lium] matmul: renting 1xRTX4090 $0.30/h (swift-fox-c8, Romania), removal in 1.2h
[lium] matmul: pod ready in 38s
[lium] matmul: running
[lium] matmul: done in 71s (~$0.0059)
[lium] matmul: pod removed
-1342.5

Authenticate once with lium init (or set LIUM_API_KEY). The behavior described here is that of the lium package since release 0.0.40 (lium#208). The decorator in 0.0.37–0.0.39 takes machine, template_id, cleanup and requirements only — no timeout, keep_warm, .map() or .close() — rents the first node whose name contains the string, installs requirements into a venv that does not see the image's torch, and returns JSON-serializable results only.

How a call works

Pick a node. machine is "<count>x<gpu>" or "<gpu>" — "1xH200", "RTX4090", "2xA100"; the count defaults to 1. The cheapest available node with exactly that many GPUs of that type is rented. If none matches you get the list of what exists (No node found matching machine type: 2xA100. Available: 1xA100 $1.20/h, 8xA100 $3.60/h.).
Bound the cost. The pod is scheduled for removal at timeout + 15 min the moment it is rented (timeout defaults to one hour; with timeout=None the removal is set 24 hours out), so a caller that crashes or is killed cannot leave a pod billing past that point — the bound is the timeout you chose, plus keep_warm when set. cleanup=False does not cancel it (see Parameters).
Ship the function. Only the function's own def is sent — decorators and annotations stripped. Arguments are pickled and go with it (your own bytes, loaded on your own pod).
Prepare the environment. A venv that sees the image's own packages (the default PyTorch template ships torch and CUDA) is created once per pod and per requirements list; pip install runs once, later calls find it ready.
Run and relay. The function runs under timeout; everything it prints reaches your terminal while it runs.
Return or raise. The result comes back as a JSON envelope plus an .npz sidecar for numpy arrays, read with allow_pickle=False — nothing the pod writes is unpickled on your machine. If the function raised a builtin exception, you get the same type here (except ValueError works); its __cause__ is a lium.RemoteExecutionError with remote_traceback, exit_code, stdout and stderr.
Release the pod. Removed when the call returns — or kept for the next call with keep_warm.

What a call costs

Measured on 6 Sep 2026 from a laptop to a 1×RTX 4090 node at $0.30/h:

	wall time	of which the function	cost
cold call (rent, boot, venv, run, return)	~70 s	~5 s	~$0.006
second call on a warm pod	~19 s	~4 s	~$0.002
`requirements=["transformers", "accelerate"]`, first call on a pod	+32 s
same requirements, later call on that pod	+1 s
idle on a warm pod (`keep_warm=600`)	10 min per window	—	~~$0.05 per window (~~$0.06 after the last call, removal comes 2 min later)

The fixed overhead is one SSH connection (~3 s to a distant node), the Python start-up on the pod, and the result transfer. keep_warm is what turns a 70 s call into a 19 s one — and the pod bills through the whole warm window whether or not a call arrives: ten idle minutes at $0.30/h cost 25× the warm call itself. Size keep_warm to the gap between your calls, and close() when you are done.

Example 1 — a GPU operation on the image's torch

No requirements: the default template already has torch with CUDA.

import lium

@lium.machine(machine="RTX4090")
def bench(n: int) -> dict:
    import time
    import torch
    a = torch.randn(n, n, device="cuda", dtype=torch.float16)
    torch.cuda.synchronize()
    t = time.perf_counter()
    for _ in range(10):
        a @ a
    torch.cuda.synchronize()
    ms = (time.perf_counter() - t) * 100
    tflops = 2 * n ** 3 / (ms / 1000) / 1e12
    print(f"{torch.cuda.get_device_name(0)}: {ms:.1f} ms per {n}x{n} fp16 matmul")
    return {"device": torch.cuda.get_device_name(0), "ms": round(ms, 1), "tflops": round(tflops, 1)}

print(bench(8192))

Return plain Python types. torch.__version__ is a TorchVersion object and a tensor is a tensor: neither travels back from the pod (the call ends with a lium.ResultEncodingError naming the type) — use str(...), .tolist() or .cpu().numpy().

Example 2 — transformers inference, with `requirements`

Torch comes from the image; transformers and accelerate are installed once per pod.

import lium

@lium.machine(machine="RTX4090", requirements=["transformers", "accelerate"], timeout=600)
def generate(prompt: str, max_new_tokens: int = 64) -> str:
    import torch
    from transformers import AutoModelForCausalLM, AutoTokenizer

    model_id = "HuggingFaceTB/SmolLM2-135M-Instruct"
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")
    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to("cuda")
    out = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False,
                         pad_token_id=tokenizer.eos_token_id)
    return tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True).strip()

try:
    print(generate("In one sentence, who discovered penicillin?"))
except lium.RemoteExecutionError as e:      # a timeout, a killed process, or a non-builtin remote exception
    print(e, e.remote_traceback)
except Exception as e:                      # a builtin remote exception, e.g. OSError for a bad model id
    print(type(e).__name__, e)
    if isinstance(e.__cause__, lium.RemoteExecutionError):
        print(e.__cause__.remote_traceback)  # what happened on the pod

While it runs you see the model download and Loading weights: 100%|██████████| progress from the pod, then the answer.

Example 3 — a warm pod, a batch, and a local escape hatch

keep_warm=600 keeps the pod ten minutes after each call. The next call — from this loop, or from the next run of the script while you iterate — reuses it. .map() runs every item on one pod. .close() removes the pod when you are done; if you forget, the server removes it keep_warm + 2 min after the last call.

import lium

@lium.machine(machine="RTX4090", requirements=["sentence-transformers"], keep_warm=600)
def embed(texts: list[str]) -> list[list[float]]:
    from sentence_transformers import SentenceTransformer
    model = SentenceTransformer("all-MiniLM-L6-v2", device="cuda")
    return model.encode(texts, normalize_embeddings=True).tolist()

batches = [["a red apple", "a green pear"], ["a fast car"], ["an old boat", "a new ship"]]
vectors = embed.map(batches)                 # one pod, three calls
print(len(vectors), len(vectors[0][0]))      # 3 384

embed.close()                                # remove the warm pod now

For tests and offline work, embed.local(texts) runs the original function here, and @lium.machine(..., local=True) (or LIUM_MACHINE_LOCAL=1 in the environment) makes every call local without editing the code — all three arrived in 0.0.40 (lium#208); 0.0.37–0.0.39 have none of them.

What can and cannot travel

Arguments are pickled — anything both sides can import; an argument that cannot be pickled is refused before anything is rented.
Results are not pickled (the file comes from provider hardware). What round-trips, exactly, each as its own type: None, bool, int, float, str, bytes; list, tuple, set, frozenset and dict of those, nested; datetime/date/time/timedelta, Decimal, pathlib.Path, uuid.UUID; numpy.ndarray of any dtype without Python objects (any shape, structured and datetime64 included) and numpy scalars, carried in an .npz read with allow_pickle=False. Anything else — a dataclass, an Enum, an OrderedDict, a tensor, an ndarray subclass — is a lium.ResultEncodingError raised on the pod that names the type and where it sits in the result; return .tolist(), dict(x), x.value instead.
Only the function's def travels. Import inside the body. A module-level constant, import or helper used inside the function is refused when the module loads, with the names listed — because on the pod it would be a NameError a minute and a few cents later. A closure over a local variable is refused the same way; pass it as an argument.
Nested functions and async def work (async functions are awaited on the pod). A method works only when self's class is importable on the pod — a class defined in the script itself is not. Lambdas do not.
Output: print() and anything written to stdout/stderr on the pod appears in your terminal as it happens, and stays on RemoteExecutionError.stdout/.stderr if the call fails. quiet=True silences only the [lium] progress lines.
Errors: a builtin remote exception (ValueError, OSError, …) is re-raised with its type and lium.RemoteExecutionError as its __cause__; any other exception class (a library's own, say torch.OutOfMemoryError), or a remote sys.exit() surfaces as lium.RemoteExecutionError itself; timeout raises RemoteExecutionError: <fn> exceeded timeout=…s and was killed; a process killed on the pod (out of memory, for instance) says so.
Ctrl-C removes the pod (or leaves it warm under keep_warm). A caller that dies without running its cleanup leaves the pod to its scheduled removal.

Parameters

parameter	default	meaning
`machine`	—	`"<count>x<gpu>"` or `"<gpu>"`; cheapest matching node
`requirements`	`None`	pip packages, installed once per pod into a venv that also sees the image's packages
`template_id`	node default	Docker template to rent with
`timeout`	`3600`	seconds the function may run; `None` for no limit. Pod removal is scheduled at `timeout + 15 min` (24 h with `None`)
`keep_warm`	`0`	seconds the pod stays after a call for the next one, billed while idle; removal re-armed to `keep_warm + 2 min` after each call
`cleanup`	`True`	`False` leaves the pod running when the call returns (remove it with `lium rm`). It also turns off `keep_warm` and warm-pod reuse, so `.map()` rents a separate pod per item. The removal scheduled at rent time still stands — cancel it with `lium schedules rm` if the pod must outlive it
`local`	`False`	run in this process; `LIUM_MACHINE_LOCAL=1` does the same for every function (since 0.0.40, lium#208)
`quiet`	`False`	no `[lium]` progress lines on stderr

Methods on the decorated function: f.remote(*args) (same as f(*args)), f.local(*args), f.map(iterable), f.close().

How a call works​

What a call costs​

Example 1 — a GPU operation on the image's torch​

Example 2 — transformers inference, with requirements​

Example 3 — a warm pod, a batch, and a local escape hatch​

What can and cannot travel​

Parameters​