Choose GPUs and check availability

You describe the hardware a run needs; Nodus finds capacity that fits and finishes it for the lowest expected cost. This guide shows how to see what is available, read an offering, and write a GPU request that says exactly what you mean.

See what is available

$ nodus get gpus
NAME                  TYPE       COUNT   VCPU   MEMORY   REGION    PRICE/H   AVAILABILITY   AGE
a100-sxm-80g-x1-any   A100-80G   1       30     200Gi    unknown   $1.27     Available      14s
h100-sxm-80g-x8-us    H100-SXM   8       208    1800Gi   us        $18.29    Available      14s
l4-24g-x1-eu-int      L4         1       8      32Gi     eu        $0.31     Limited        14s

PRICE/H is the from rate per machine-hour and AGE is how long ago the price and availability were confirmed.

Filter with flags; each one narrows the list:

$ nodus get gpus --gpu H100 --count 8 --region us
$ nodus get gpus --interruptible
$ nodus get gpus --gpu L4 -o wide        # typical and list rates, startup time and interruption rate

The console’s GPU picker shows the same list.

Read an offering

An offering is a class of capacity: accelerator × count × region class, and whether it is interruptible. Its name spells that out:

Name	Meaning
h100-sxm-80g-x8-us	Eight H100 SXM 80 GB GPUs per machine, in the us region class
l4-24g-x1-eu-int	One L4 per machine in eu, interruptible
a100-sxm-80g-x1-any	Capacity with no guaranteed location (any)
cpu-8c-32g-us	An 8 vCPU, 32 GiB CPU machine in us

Each offering shows:

• From and typical rates per hour. The from rate is that of the cheapest machine free now, the one a run starts on by default, or of the cheapest machine when none is free; the host shape shown is that machine’s. Your rate is shown in the estimate before launch and frozen for the run. No rate is ever above the list price.
• Availability: Available, Limited (a few machines, or a count that is not reported) or Unavailable.
• Startup: how long a machine usually takes to be ready (p50 and p90), from what Nodus has measured.
• Interruption rate for interruptible offerings: how often such capacity is taken back per hour.

Prices and availability refresh continuously. The public pricing page shows the same from rates.

Region classes

placement.regions takes region classes, not data centers: us, ca, eu, uk, in, apac, me and latam. Capacity whose location is not reported has the class any in its name and never satisfies a non-empty placement.regions, so a run that must stay in a region never lands there.

spec:
  placement:
    regions: [eu, uk]

Ask for the hardware you need

resources.gpu.type lists every accelerator you accept. Order does not matter: the scheduler picks among them by expected cost to finish.

You write	Nodus may use
H100	Any H100 variant: SXM, PCIe or NVL
[H100, H200]	Any H100 or H200, never another family
H100-SXM	Only the H100 SXM
H100! or exact: true	Only the family’s primary variant (H100 SXM)
minMemory: 80Gi with no type	Any accelerator with at least 80 GiB per GPU

Modal spellings work as written: A100-40GB, A100-80GB, A10G, L40S, H100, H200, B200, T4, L4.

spec:
  resources:
    gpu: {type: [H100, H200], count: 8}
  placement:
    interruptible: Allow          # use interruptible capacity only when it is cheaper to finish
    maxRateUSDPerHour: "30.00"

count is per machine (1, 2, 4 or 8). For more GPUs than one machine holds, see multi-node training (Beta).

Use every GPU of one machine

--gpu A6000:2 (or count: 2) gives your container both GPUs of one machine, numbered 0 and 1. torchrun starts one process per GPU without flags, because Nodus sets PET_NPROC_PER_NODE to the GPU count; a --nproc-per-node you pass yourself wins.

train.py

"""A DDP smoke run on every GPU of one machine. torchrun starts one process per GPU (PET_NPROC_PER_NODE)."""
import os

import torch
import torch.distributed as dist
from torch.nn.parallel import DistributedDataParallel as DDP

dist.init_process_group("nccl")
local_rank = int(os.environ["LOCAL_RANK"])
device = torch.device("cuda", local_rank)
torch.cuda.set_device(device)

# Every process contributes 1, so the sum is the world size.
one = torch.ones(1, device=device)
dist.all_reduce(one)

model = DDP(torch.nn.Linear(16, 1).to(device), device_ids=[local_rank])
opt = torch.optim.SGD(model.parameters(), lr=0.1)
for step in range(20):
    x = torch.randn(32, 16, device=device)
    loss = (model(x) - x.sum(dim=1, keepdim=True)).pow(2).mean()
    opt.zero_grad()
    loss.backward()
    opt.step()

if dist.get_rank() == 0:
    print(f"world_size={int(one.item())} gpus={torch.cuda.device_count()} "
          f"device={torch.cuda.get_device_name(device)} loss={loss.item():.4f}")
dist.destroy_process_group()

run.sh

nodus run --name torchrun-multi-gpu --gpu A6000:2 --max-cost 1.00 -- torchrun train.py

The run prints world_size=2 gpus=2 from the first process. The processes reach each other over NCCL on the machine itself; /dev/shm is sized for it (half the container’s memory), so you need no --shm-size or --ipc=host.

Check before you launch

A dry run returns the estimate without starting anything:

$ nodus apply -f job.yaml --dry-run=server -o estimate

It shows the offering Nodus would use, the expected cost p50 and p90, the startup time, the first hold and the minimum charge, and how long the estimate stays valid. When nothing fits, it says why, for example CapacityUnavailable, NoListPrice, ExceedsRemainingBudget or MissesDeadline, with a fix.

When a run waits in Queued

A run with no matching capacity stays Queued and is placed as soon as capacity appears, until placement.queueTimeout (24 hours by default). nodus describe shows the offerings that were considered and why each was rejected. To start sooner, widen resources.gpu.type or placement.regions, allow interruptible capacity, or raise placement.maxRateUSDPerHour.

How Nodus chooses among offerings is explained in placement and scheduling profiles.