Skip to content

Node

A Datatailr compute node (dt node).

A node is a machine that runs jobs. The fields match the JSON rows of dt node ls. Construct by name or instance id to load from the platform ::

node = Node("compute-node-1")
node = Node("i-08666efd2a24f10f8")

Rows built with :meth:from_dict (for example from :meth:ls) reflect the JSON as returned by the API and do not trigger an extra fetch.

Read-only methods (:meth:ls, :meth:get, :meth:exists, :meth:ps, :meth:drain_status, :meth:ls_tags, :meth:get_tag, :meth:instance_types) are open to every user. Lifecycle methods (:meth:new, :meth:start, :meth:stop, :meth:restart, :meth:terminate, :meth:change_type) need the admin or node_admin group. Draining and tag changes need the admin group.

Example::

from datatailr import Node

for node in Node.ls(filter="environment = dev"):
    print(node.name, node.status, node.cpu["free_cpus"])

node = Node("compute-node-1")
node.drain()                 # stop scheduling new jobs on this node
print(node.drain_status())   # True
node.drain(stop=True)        # accept new jobs again

Parameters:

  • name_or_id

    (str) –

    Non-empty node name (the Name tag) or cloud instance id.

Raises:

  • ValueError –

    If name_or_id is empty or no such node exists.

architecture property

CPU architecture, for example x86_64 or arm64.

availability_zone property

Cloud availability zone of the instance.

cpu property

CPU usage: allocated_cpus, free_cpus and total_cpus.

Empty when the node is not registered with the controller.

created_at property

Creation time as a timezone-aware UTC datetime.

environment property

Environment the node serves: dev, pre, prod or all.

host property

Address of the node daemon. None when the node is not registered.

id property

Cloud instance id (for example i-08666efd2a24f10f8).

image_id property

Cloud machine image the instance was started from.

jobs property

Jobs that run on the node. Each entry has name, environment, id, state, run_id, num_cpus and memory_limit.

memory property

Memory usage in bytes: allocated_memory, free_memory, max_memory, unallocated_memory, used_memory and wasted_memory.

Empty when the node is not registered with the controller.

name property

Node name (the Name tag of the instance).

node_class property

Node class, for example compute or gpu.

port property

Port of the node daemon. None when the node is not registered.

private_ip property

Private IP address of the node.

state property

State as reported by the cloud provider, for example running.

status property

Status as seen by the controller, for example running.

tags property

Cloud provider tags of the instance, as returned by dt node ls.

For the tags stored on the node daemon, use :meth:ls_tags.

type property

Cloud instance type, for example r6a.large.

change_type staticmethod

Change the instance type of nodes (dt node change-type). Admin or node_admin only.

The nodes restart with the new type. Jobs that run on them are interrupted.

Parameters:

  • instance_type

    (str) –

    The new cloud instance type, see :meth:instance_types.

  • *nodes

    (str | Node, default: () ) –

    One or more instance ids or :class:Node instances.

Raises:

  • ValueError –

    If instance_type is empty or no node is given.

drain

Toggle drain mode on this node (dt node drain). Admin only.

A draining node accepts no new jobs. Jobs that already run continue until they finish.

Parameters:

  • stop

    (bool, default: False ) –

    If True, stop draining so the node accepts new jobs again.

drain_status

Return whether this node is draining (dt node drain-status).

exists staticmethod

Return whether a node with this name or instance id exists.

Parameters:

  • name_or_id

    (str) –

    Node name or cloud instance id.

from_dict classmethod

Build a :class:Node from a dt node ls row without an extra API call.

Parameters:

  • data

    (dict[str, Any]) –

    One row of dt node ls --json. Must contain id and name.

Raises:

  • ValueError –

    If id or name is missing from data.

get staticmethod

Load one node by name or instance id.

Parameters:

  • name_or_id

    (str) –

    Node name or cloud instance id.

Raises:

  • ValueError –

    If no such node exists.

get_tag

Get one tag of this node (dt node get-tag).

Parameters:

  • key

    (str) –

    The tag key.

Returns:

  • Any –

    The tag value. JSON values are parsed, other values are returned

  • Any –

    as str.

Raises:

  • ValueError –

    If key is empty.

instance_types staticmethod

List the cloud instance types (dt node instance-types).

Returns:

  • list[dict[str, Any]] –

    A list of dicts with keys name, node_class, family,

  • list[dict[str, Any]] –

    cpu, memory, architecture, has_gpu, storage_gib

  • list[dict[str, Any]] –

    and price.

Raises:

  • TypeError –

    If the API response is not a list.

instance_types_by_class staticmethod

Cloud instance types grouped by node class (dt node instance-types-by-class).

Raises:

  • TypeError –

    If the API response is not a JSON object.

instance_types_by_family staticmethod

Cloud instance types grouped by family (dt node instance-types-by-family).

Raises:

  • TypeError –

    If the API response is not a JSON object.

ls staticmethod

List nodes (dt node ls).

Parameters:

  • filter

    (str | None, default: None ) –

    Filter expression, for example "environment = dev and status = running". Filter fields: id, name, environment, status, state, node_class, type, architecture, availability_zone, private_ip, host, image_id, subnet_id, created_at, cpu_total, cpu_free, cpu_allocated, memory_max, memory_free, memory_allocated, memory_used.

  • sort

    (str | None, default: None ) –

    Sort expression: +field for ascending, -field for descending, for example "-created_at".

  • limit

    (int | None, default: None ) –

    Return at most this many nodes.

  • offset

    (int | None, default: None ) –

    Skip this many nodes before the first result.

Returns:

  • list[Node] –

    A list of :class:Node instances.

Raises:

  • TypeError –

    If the API response is not a list.

ls_tags

List the tags stored on this node (dt node ls-tags).

These are the node daemon tags (for example node_name or node_ip), not the cloud provider tags in :attr:tags.

Raises:

  • TypeError –

    If the API response is not a JSON object.

new staticmethod

Create new nodes (dt node new). Admin or node_admin only.

This starts cloud instances and causes cloud charges.

Pass either a ready configuration (node_configuration) or the keyword arguments for one node ::

Node.new(instance_type="r6a.large")
Node.new(instance_type="g5.xlarge", node_class="gpu", environment="prod")
Node.new({"type": "r6a.large", "node_class": "compute", "environment": "dev", "num_instances": 1})

Parameters:

  • node_configuration

    (dict[str, Any] | list[dict[str, Any]] | None, default: None ) –

    One configuration dict or a list of them, sent as is. Each dict has the keys type (instance type, see :meth:instance_types), node_class (compute or gpu), environment (dev, pre or prod) and num_instances, and may set disk_size (root disk size in GB, default 10; ignored on Hetzner). Not allowed together with instance_type.

  • instance_type

    (str | None, default: None ) –

    Cloud instance type of the node to create, see :meth:instance_types. Required when node_configuration is omitted.

  • node_class

    (str, default: 'compute' ) –

    compute (default) or gpu.

  • environment

    (Environment | str, default: DEV ) –

    dev (default), pre or prod, as a string or an :class:~datatailr.utils.Environment.

  • num_instances

    (int, default: 1 ) –

    Number of instances to create. Default 1.

Returns:

  • Any –

    The response of dt node new: the instances that were started.

Raises:

  • ValueError –

    If neither or both of node_configuration and instance_type are given, if node_configuration is empty, or if a keyword argument is invalid.

  • TypeError –

    If an entry of node_configuration is not a dict.

ps

List the containers that run on this node (dt node ps).

Returns:

  • list[dict[str, Any]] –

    A list of dicts with keys id, name, image,

  • list[dict[str, Any]] –

    entrypoint, network_mode and creation_time.

Raises:

  • TypeError –

    If the API response is not a list.

restart staticmethod

Restart nodes (dt node restart). Admin or node_admin only.

Jobs that run on the nodes are interrupted.

Parameters:

  • *nodes

    (str | Node, default: () ) –

    One or more instance ids or :class:Node instances.

Raises:

  • ValueError –

    If no node is given.

rm_tag

Remove a tag from this node (dt node rm-tag). Admin only.

Parameters:

  • key

    (str) –

    The tag key.

Raises:

  • ValueError –

    If key is empty.

set_tag

Set a tag on this node (dt node set-tag). Admin only.

Parameters:

  • key

    (str) –

    The tag key.

  • value

    (str) –

    The tag value.

Raises:

  • ValueError –

    If key is empty.

start staticmethod

Start stopped nodes (dt node start). Admin or node_admin only.

Parameters:

  • *nodes

    (str | Node, default: () ) –

    One or more instance ids or :class:Node instances.

Raises:

  • ValueError –

    If no node is given.

stop staticmethod

Stop running nodes (dt node stop). Admin or node_admin only.

Jobs that run on the nodes are interrupted.

Parameters:

  • *nodes

    (str | Node, default: () ) –

    One or more instance ids or :class:Node instances.

Raises:

  • ValueError –

    If no node is given.

terminate staticmethod

Terminate nodes (dt node terminate). Admin or node_admin only.

This permanently destroys the cloud instances and all jobs that run on them. It cannot be undone.

Parameters:

  • *nodes

    (str | Node, default: () ) –

    One or more instance ids or :class:Node instances.

Raises:

  • ValueError –

    If no node is given.

to_dict

Return the raw dt node ls row for this node.