nodus.checkpoint
View MarkdownThe checkpoint handshake and gang checkpoints (resources.md §8, distributed-training.md §10.3).
Nodus owns checkpoint cadence and storage; programs write their own files into nodus.state_dir(). Before a
snapshot the node sends checkpoint.request: on_request(callback) saves, then ack() tells the node the files are
consistent. Restoring files never restores process memory: a restarted program loads what it saved.
ack(seq: str | None = None) -> NoneTell the node the checkpoint files are complete for request seq (the latest one by default).
handle
Section titled “handle”handle(message: dict[str, Any]) -> NoneProcess one message from the events socket.
on_request
Section titled “on_request”on_request(callback: Callable[[], None]) -> NoneCall callback (then ack()) whenever the node asks for a checkpoint; a no-op outside Nodus.
requested
Section titled “requested”requested() -> boolTrue while a checkpoint request is pending; poll it at safe points in a training loop.
subscribe
Section titled “subscribe”subscribe() -> NoneListen for checkpoint requests without a callback: poll requested() and call ack() after saving.