Skip to content

Task

Decorator that turns a function into a workflow task.

Apply this via the public alias @task(). When the decorated function is called inside a workflow-decorated function, the call returns a Task object that becomes a node in the workflow DAG. Dependencies between tasks are inferred automatically from the data flow between calls.

Tasks are only registered for deployment when they are invoked from within the body of a @workflow-decorated function, which builds a Workflow. Calling a @task-decorated function outside that context executes it locally as a normal Python function instead.

See Task below for methods available on the returned object (for example alias() and set_resources()).

Parameters:

  • memory

    (str | None, default: None ) –

    Memory limit for this task's container (e.g. "256m", "1g"). When omitted, the workflow's memory is used.

  • cpu

    (float | None, default: None ) –

    Number of CPU cores to allocate for this task (e.g. 1, 0.5). When omitted, the workflow's CPU count is used.

  • vram

    (str | None, default: None ) –

    GPU memory limit for this task's container (e.g. "256m", "1g"), the same form as memory. When omitted, the workflow value is used. If that value is also omitted, the task JSON does not include a vram field.

Returns:

  • Task ( Callable[[Callable[..., Any]], Callable[..., Any]] ) –

    The decorated function. When that function is called inside a @workflow-decorated function, the call returns a Task object that becomes a node in the workflow DAG.

Example 

# tasks.py
from datatailr import task

@task(memory="512m", cpu=2)
def heavy_computation(x, y):
    return x ** y

@task()
def prepare():
    return 2
Then to deploy the workflow, use the following code: 
# deploy.py
from datatailr import workflow
from tasks import heavy_computation, prepare

@workflow("My Workflow")
def my_workflow():
    base = prepare()
    heavy_computation(base, 10)

if __name__ == "__main__":
    my_workflow()  # deploy to Datatailr

Task

Task

A single task (node) within a Workflow DAG.

Task instances are usually created by calling a @task()-decorated function inside a @workflow-decorated function. You can then chain configuration methods such as alias and set_resources.

Parameters:

  • name

    (str) –

    Display name for the task.

  • entrypoint

    (EntryPoint) –

    Entrypoint that runs when the task executes.

  • resources

    (Resources | None, default: None ) –

    CPU, memory, and optional GPU memory. When None, the parent workflow's resources are used as defaults. If vram is omitted, the task JSON omits the vram field.

  • dependencies

    (Sequence[str | Task] | None, default: None ) –

    Upstream tasks this task depends on, as task names or Task instances.

  • dag

    (Workflow | None, default: None ) –

    Parent Workflow. When None, the current workflow manager is used.

  • argument_mapping

    (dict[str, str] | None, default: None ) –

    Maps upstream task names to this task's argument names for dependency wiring.

  • env_vars

    (dict[str, str | int | float | bool] | None, default: None ) –

    Environment variables for this task. Merged on top of variables inherited from the parent workflow.

  • id

    (int | None, default: None ) –

    Explicit task ID. When None, the next available ID from the parent workflow is assigned.

args property writable

Arguments bound to this task.

Returns:

  • dict[str, Any] –

    A dictionary of argument names to values.

id property

Unique identifier of this task within its workflow.

Returns:

  • int –

    The task ID.

internal_name property

Internal platform name for this task.

Returns:

  • str | None –

    {workflow_internal_name}-{task_id}, or None if the task has

  • str | None –

    no parent workflow.

__call__

Allow the task to be called like a function.

Returns:

  • Task –

    The same Task instance, for

  • Task –

    chaining or functional-style usage.

alias

Set a display-name alias for this task.

Parameters:

  • name

    (str) –

    New alias name. Must be unique within the parent workflow.

Returns:

from_dict classmethod

Create a Task from a dictionary representation.

Parameters:

  • job_dict

    (dict) –

    Dictionary produced by to_dict or returned by the platform.

  • batch

    (Workflow) –

    Parent Workflow to attach the task to.

Returns:

run

Execute this task's entrypoint.

Sets DATATAILR_BATCH_ID, DATATAILR_JOB_ID, and DATATAILR_JOB_NAME in the entrypoint environment before running.

set_resources

Set CPU, memory, and GPU memory resources for this task.

Pass either a Resources instance via resources, or set memory, cpu, and/or vram individually. When resources is omitted, a new Resources is built from memory, cpu, and vram. memory and cpu fall back to the default task limits when None. vram is omitted unless given.

Parameters:

  • resources

    (Resources | None, default: None ) –

    Complete resource specification. When provided, memory, cpu, and vram are ignored.

  • memory

    (str | None, default: None ) –

    Memory limit (e.g. "256m", "1g"). Used only when resources is None.

  • cpu

    (float | None, default: None ) –

    Number of CPU cores (e.g. 1, 0.5). Used only when resources is None.

  • vram

    (str | None, default: None ) –

    GPU memory limit (e.g. "256m", "1g"), the same form as memory. Used only when resources is None. If omitted, the task JSON does not include a vram field.

Returns:

to_dict

Convert this task to a dictionary representation.

Returns:

  • –

    A dictionary suitable for serialization and platform submission.

to_json

Convert this task to a JSON string.

Returns:

  • –

    JSON serialization of to_dict.

translate_dependencies

Resolve dependency names/tasks to upstream task IDs.

Returns:

  • set[int] –

    A set of integer task IDs this task depends on.

update_env_vars

Update environment variables for this task.

Existing variables are preserved; keys in env_vars overwrite matching entries.

Parameters:

  • env_vars

    (dict[str, str | int | float | bool]) –

    Environment variables to merge into this task's env.

Returns: