Full API Reference

Datatailr - Python SDK

The datatailr package provides a Python SDK for interacting with the Datatailr platform.

For a quickstart guide please launch the default workstation on your installation and open the README file.

Datatailr provides a simple interface for scheduling and managing data tasks, handling user access control, working with data blobs and secrets, and more.

App

Deploy a web application or dashboard to Datatailr.

This supports Streamlit, Dash, or any framework that can be started via a standard entrypoint (for example, streamlit run app.py).

Parameters:

• name

  (str) –

  App display name.

• entrypoint

  (Callable | str) –

  Callable that starts the app server.

• environment

  (Environment | None, default: DEV ) –

  Target environment for the job.

• image

  (Image | None, default: None ) –

  Container image to use for execution.

• run_as

  (str | User | None, default: None ) –

  User to execute the job as.

• resources

  (Resources, default: Resources() ) –

  Resource requirements for the job.

• acl

  (ACL | None, default: None ) –

  Access control settings for the job.

• framework

  (str | None, default: None ) –

  Framework to use for the app.

• python_version

  (str, default: '3.12' ) –

  Python version to use.

• python_requirements

  (str | List[str], default: '' ) –

  Requirements to install (string or list).

• build_script_pre

  (str, default: '' ) –

  Shell script to run before build.

• build_script_post

  (str, default: '' ) –

  Shell script to run after build.

• env_vars

  (dict[str, str | int | float | bool] | None, default: None ) –

  Environment variables to set.

• get_existing

  (bool, default: False ) –

  Update an existing job with the same name.

Examples:

Minimal Streamlit app.

# app.py
import streamlit as st

def main():
    st.title("Hello Datatailr App")

if __name__ == "__main__":
    main()

Test locally.

streamlit run app.py

Deploy to Datatailr.

from app import main
from datatailr import App

app = App(
    name="Simple Dashboard App",
    entrypoint=main,
    python_requirements="streamlit")

app.run()

Parameters:

• name

  (str) –

  Display name for the app.

• entrypoint

  (Callable | str) –

  The callable (function) that starts the application.

• environment

  (Environment | None, default: DEV ) –

  Target environment for the deployment.

• image

  (Image | None, default: None ) –

  Pre-configured container Image. When None, an image is built from python_version, python_requirements, and the build scripts.

• run_as

  (str | User | None, default: None ) –

  User or username to run the app as. Defaults to the currently signed-in user.

• resources

  (Resources, default: Resources() ) –

  CPU and memory resources for the container.

• acl

  (ACL | None, default: None ) –

  Access control list. Defaults to standard permissions for the current user.

• framework

  (str | None, default: None ) –

  Optional framework name (e.g., 'streamlit', 'dash', 'flask'). If not provided, the framework is inferred from the entrypoint.

• python_version

  (str, default: '3.12' ) –

  Python version for the container image.

• python_requirements

  (str | List[str], default: '' ) –

  Python dependencies (see Image).

• build_script_pre

  (str, default: '' ) –

  Dockerfile commands to run before installing requirements.

• build_script_post

  (str, default: '' ) –

  Dockerfile commands to run after installing requirements.

• env_vars

  (dict[str, str | int | float | bool] | None, default: None ) –

  Environment variables passed to the running container.

• get_existing

  (bool, default: False ) –

  If True, load and update an existing job definition with the same name instead of creating a new one.

• app_section

  (str, default: '' ) –

  The section to which the app belongs. If not provided, the app will be assigned to the default section. This affects the app launcher page.

• budget

  (Budget | str | None, default: None ) –

  Optional spend budget as Budget("name") or a name str. Will be set to the 'default' budget if not provided.

Blob

Interface for Datatailr's binary large object (blob) storage.

Blob storage lets you persist and retrieve arbitrary files and binary data. Each blob is identified by a unique name (key/path).

Notes

This class wraps Datatailr blob operations (list/copy/read/write/delete). Large reads/writes use temporary files internally.

Examples:

Basic file copy to/from blob storage:

>>> from datatailr import Blob
>>> b = Blob()
>>> _ = open("/tmp/data.txt", "wb").write(b"qwerty123")
>>> b.put_file("mybucket/data.txt", "/tmp/data.txt")
>>> b.get_file("mybucket/data.txt", "/tmp/data_copy.txt")

Read/write blob contents as bytes:

>>> payload = b"hello world"
>>> b.put("mybucket/hello.bin", payload)
>>> data = b.get("mybucket/hello.bin")
>>> data == payload
True

List and delete:

>>> sorted(b.ls("mybucket/"), key=lambda x: x["name"])
[{'is_file': True, 'last_modified': ..., 'name': 'mybucket/data.txt', 'size': 9}, {'is_file': True, 'last_modified': ..., 'name': 'mybucket/hello.bin', 'size': 11}]
>>> b.delete("mybucket/hello.bin")
>>> b.exists("mybucket/hello.bin")
False

acls

Retrieve the access control list (ACL) for a specific blob path.

Parameters:

• path

  (str) –

  The blob path for which to retrieve the ACL.

Returns:

• ACL ( ACL ) –

  The ACL for the blob path.

delete

Delete a blob.

Parameters:

• name

  (str) –

  The name of the blob to delete.

exists

Check if a blob exists.

Parameters:

• name

  (str) –

  The name of the blob to check.

Returns:

• bool ( bool ) –

  True if the blob exists, False otherwise.

get

Get a blob object.

Parameters:

• name

  (str) –

  The name of the blob to retrieve.

Returns:

• Blob ( bytes ) –

  The blob object.

get_file

Copy a blob file to a local file.

Parameters:

• name

  (str) –

  The name of the blob to retrieve.

• path

  (str) –

  The path to store the blob as a file.

ls

List files in the specified prefix. If recursive is True, list files in subdirectories recursively.

Parameters:

• prefix

  (str) –

  The prefix to list files from.

• recursive

  (bool, default: False ) –

  Whether to list files in subdirectories recursively.

• filter_by

  (str, default: '' ) –

  The filter expression to apply to the list of files.

• limit

  (int, default: -1 ) –

  The maximum number of files to return.

• offset

  (int, default: 0 ) –

  The offset to start returning files from.

Returns:

• list ( list[dict] ) –

  A list of file dictionaries in the specified path. If recursive is True, the list will contain files in subdirectories recursively.

• list[dict] –

  Each dictionary contains the following keys:

• list[dict] –
  • name: The name of the file.
• list[dict] –
  • size: The size of the file.
• list[dict] –
  • last_modified: The last modified time of the file.
• list[dict] –
  • is_file: Whether the file is a file.

put

Put a blob object into the blob storage.

Parameters:

• name

  (str) –

  The name of the blob to create.

• blob

  (bytes | str) –

  The blob object to store.

• acl

  (ACL, default: None ) –

  The access control list for the blob.

put_file

Copy a local file to a blob.

Parameters:

• name

  (str) –

  The name of the blob to create.

• path

  (str) –

  The path of the local file to copy.

• acl

  (ACL, default: None ) –

  The access control list for the blob.

set_acls

Set the access control list (ACL) for a specific blob path.

Parameters:

• path

  (str) –

  The blob path for which to set the ACL.

• acls

  (ACL) –

  The ACL to apply.

Budget

A Datatailr spend budget (cap, usage, prevent_overflow, ACL).

Fields match dt cost list-budgets / dt cost get-budget JSON (see internal-docs/budgets.md). Optional fields may be omitted when the caller only has operate visibility on a budget row.

Construct by name to load from the platform ::

b = Budget("my_budget")

The numeric :attr:id is the database budget id used in job definitions as budget_id. Rows built via :meth:from_dict (e.g. from list-budgets) reflect the JSON as returned by the API and do not trigger an extra fetch.

Parameters:

• name

  (str) –

  Non-empty budget name.

Raises:

• ValueError –

  If name is empty or the API response is invalid.

acl property

The budget ACL. Will be None if user does not have read permissions.

budget_usd property

Cap in USD when visible to the caller.

id property

Numeric budget id (budget_id in job JSON).

name property

Budget name.

prevent_overflow property

Whether the platform blocks spend past the limit when visible.

usage_percentage property

Usage as a percent of cap when available.

usage_usd property

Current-month usage in USD when visible.

add staticmethod

Create a budget. Admin only.

add_acl staticmethod

Merge ACL entries. Admin only.

from_dict classmethod

Build a :class:Budget from CLI/JSON dict without an extra API call.

get staticmethod

Load one budget by name.

get_acl staticmethod

Returns the budget ACL. Admin only.

ls staticmethod

Get a list of budgets visible to the current user.

remove staticmethod

Remove a budget. Admin only.

remove_acl staticmethod

Remove ACL entries. Admin only.

set_acl staticmethod

Replace the budget ACL. Admin only.

update staticmethod

Update budget cap and/or prevent_overflow. Admin only.

Environment

Deployment stage for Datatailr compute processes.

Each environment is a fully isolated stage in the development and release lifecycle. Workflows, apps, services, and other jobs run in Docker containers scoped to the selected environment, so changes in one stage do not affect the others.

New jobs default to DEV. After validation, promote them through PRE (staging) to PROD with Job.promote().

When a job runs inside the platform, the active environment is read from the DATATAILR_JOB_ENVIRONMENT variable (see get_dt_env()).

Attributes:

• DEV –

  Development environment for building and testing changes.

• PRE –

  Pre-production (staging) environment for final validation.

• PROD –

  Production environment for live workloads.

Examples:

>>> from datatailr import Environment
>>> from datatailr.scheduler import App
>>> app = App(name="my_app", environment=Environment.DEV)
>>> app.promote(from_environment=Environment.DEV)

ExcelAddin

Represents an Excel add-in deployment on Datatailr.

An Excel add-in exposes Python functions as Excel worksheet functions, allowing users to call server-side computations directly from Excel spreadsheets.

Example

from datatailr import ExcelAddin
from datatailr.excel import Addin

addin_def = Addin("Options Pricer", "Option pricing functions")

@addin_def.expose(description="Black-Scholes price")
def price_option(spot, strike, vol, rate, expiry):
    ...

def __excel_main__(port=8080, ws_port=8000):
    addin_def.run(port, ws_port)

addin = ExcelAddin(
    name="Options Pricer",
    entrypoint=__excel_main__,
    python_requirements=["numpy", "scipy"],
)
addin.run()

Parameters:

• name

  (str) –

  Display name for the add-in.

• entrypoint

  (Callable) –

  The callable (function) that starts the add-in server.

• environment

  (Environment | None, default: DEV ) –

  Target environment for the deployment.

• image

  (Image | None, default: None ) –

  Pre-configured container Image.

• run_as

  (str | User | None, default: None ) –

  User or username to run the add-in as.

• resources

  (Resources, default: Resources() ) –

  CPU and memory resources for the container.

• acl

  (ACL | None, default: None ) –

  Access control list.

• python_version

  (str, default: '3.12' ) –

  Python version for the container image.

• python_requirements

  (str | List[str], default: '' ) –

  Python dependencies (see Image).

• build_script_pre

  (str, default: '' ) –

  Dockerfile commands to run before installing requirements.

• build_script_post

  (str, default: '' ) –

  Dockerfile commands to run after installing requirements.

• env_vars

  (dict[str, str | int | float | bool] | None, default: None ) –

  Environment variables passed to the running container.

• get_existing

  (bool, default: False ) –

  If True, update an existing job definition.

• version

  (str | int | None, default: None ) –

  The version of the job to get.

• app_section

  (str, default: '' ) –

  The section to which the app belongs. If not provided, the app will be assigned to the default section. This affects the app launcher page.

• budget

  (Budget | str | None, default: None ) –

  Optional spend budget as Budget("name") or a name str. Will be set to the 'default' budget if not provided.

Group

Representing a Datatailr Group.

This class provides methods to interact with the Datatailr Group API. It allows you to create, update, delete, and manage groups within the Datatailr platform.

Attributes:

• name (str) –

  The name of the group.

• members (list) –

  A list of members in the group.

• group_id (int) –

  The unique identifier for the group.

Parameters:

• name

  (str) –

  The name of an existing group.

• members

  (optional, default: None ) –

  A list of members in the group.

• group_id

  (optional, default: None ) –

  The unique identifier for the group.

Raises:

• ValueError –

  If a group with the given name does not exist.

group_id property

The unique numeric identifier of the group.

members property

List of user IDs that are members of this group.

name property

The name of the group.

add staticmethod

Create a new group with the specified name.

Parameters:

• name

  (str) –

  The name for the new group.

Returns:

• 'Group' | None –

  The newly created Group, or None if the operation failed.

add_users

Add users to the group.

Parameters:

• usernames

  (list) –

  A list of usernames to add.

Raises:

• ValueError –

  If the group name is not set.

exists staticmethod

Check if a group exists by its name.

Parameters:

• name

  (str) –

  The group name to look up.

Returns:

• bool –

  True if the group exists, False otherwise.

from_dict classmethod

Construct a Group instance from a dictionary.

Parameters:

• data

  (dict) –

  A dictionary with keys "name", "members", and "group_id".

Returns:

• 'Group' –

  A Group instance populated from the dictionary.

Raises:

• ValueError –

  If required keys are missing from data.

get staticmethod

Retrieve an existing group by name or numeric ID.

Parameters:

• name_or_id

  (str | int) –

  The group name (str) or group ID (int).

Returns:

• 'Group' –

  The matching Group instance.

Raises:

• ValueError –

  If no group with the given name or ID exists.

is_member

Check whether a user belongs to this group.

Parameters:

• user

  (User) –

  A User instance or a user ID (int).

Returns:

• bool –

  True if the user is a member, False otherwise.

Raises:

• ValueError –

  If the group name is not set.

ls staticmethod

List all groups available in the Datatailr platform.

Parameters:

• filter

  (str | None, default: None ) –

  A filter expression to apply to the groups.

Returns:

• list –

  A list of Group instances.

remove staticmethod

Remove a group by its name.

Parameters:

• name

  (str) –

  The name of the group to remove.

remove_users

Remove users from the group.

Parameters:

• usernames

  (list) –

  A list of usernames to remove.

Raises:

• ValueError –

  If the group name is not set.

Image

Represents a container image for a job.

The image is defined based on Python dependencies and two build scripts expressed as Dockerfile commands.

Parameters:

• python_version

  (str, default: 'auto' ) –

  Python version string or "auto".

• python_requirements

  (str | list[str], default: '' ) –

  Requirements spec, file path, list, or "auto".

• build_script_pre

  (str, default: '' ) –

  Pre-build Dockerfile commands or file path.

• build_script_post

  (str, default: '' ) –

  Post-build Dockerfile commands or file path.

• branch_name

  (str | None, default: None ) –

  Git branch name for the repo, if applicable.

• commit_hash

  (str | None, default: None ) –

  Git commit hash for the repo, if applicable.

• path_to_repo

  (str | None, default: None ) –

  Path to the repository root, if applicable.

• path_to_module

  (str | None, default: None ) –

  Path to the Python module, if applicable.

Attributes:

• python_version (str) –

  Resolved Python version string.

• python_requirements (str) –

  Requirements string (newline-delimited).

• build_script_pre (str) –

  Pre-build Dockerfile commands.

• build_script_post (str) –

  Post-build Dockerfile commands.

• branch_name (str | None) –

  Git branch name.

• commit_hash (str | None) –

  Git commit hash.

• path_to_repo (str | None) –

  Path to the repository root.

• path_to_module (str | None) –

  Path to the Python module.

• build_stdout (str | None) –

  Stdout captured from the last build, if any.

• build_stderr (str | None) –

  Stderr captured from the last build, if any.

• name (str | None) –

  Built image name, if any.

• last_build_succeeded (bool | None) –

  Whether the last build succeeded.

• bundle_name (str | None) –

  Bundle name, if any.

• registry (str | None) –

  Registry URL or name, if any.

• run_after_build (bool | None) –

  Whether to run after build.

• tag (str | None) –

  Image tag, if any.

Parameters:

• python_version

  (str, default: 'auto' ) –

  Python version for the container (e.g. "3.12"). Use "auto" to match the current interpreter version.

• python_requirements

  (str | list[str], default: '' ) –

  Python dependencies. Accepts a pip-freeze string, a list of package specifiers, a path to a requirements.txt file, or "auto" to run pip freeze.

• build_script_pre

  (str, default: '' ) –

  Dockerfile commands to execute before installing Python requirements. Can be a string or a file path.

• build_script_post

  (str, default: '' ) –

  Dockerfile commands to execute after installing Python requirements. Can be a string or a file path.

• branch_name

  (str | None, default: None ) –

  Git branch name (populated automatically during build).

• commit_hash

  (str | None, default: None ) –

  Git commit hash (populated automatically during build).

• path_to_repo

  (str | None, default: None ) –

  Absolute path to the source repository root.

• path_to_module

  (str | None, default: None ) –

  Relative path from the repository root to the Python module containing the entrypoint.

from_dict

Create an Image instance from a dictionary representation.

to_dict

Convert the Image instance to a dictionary representation.

to_json

Convert the Image instance to a JSON string representation.

update

Update one or more Image attributes.

Parameters:

• **kwargs

  (Any, default: {} ) –

  Attribute names and their new values.

Raises:

• TypeError –

  If a string-only attribute receives a non-string value.

• AttributeError –

  If an attribute name does not exist on the Image.

KV

A distributed key-value store for storing and retrieving configuration data, application settings, and other key-value pairs.

Note that different environments (dev/pre/prod) have separate key-value stores.

Each key-value pair can have its own access control list (ACL) to manage permissions for reading and writing.

Example:

from datatailr import KV, ACL, User, Group, Permission
kv = KV()
kv.put("db_url", "postgresql://user:password@host:port/dbname")
acls = ACL({Permission.READ: [User.get("user1"), Group.get("group1")],
            Permission.WRITE: [User.get("user2")]})
kv.add_acl("db_url", acls)
db_url = kv.get("db_url")
all_keys = kv.ls()
kv.delete("db_url")

acls

Retrieve the access control list (ACL) for a specific key in the key-value store.

Parameters:

• key

  (str) –

  The key for which to retrieve the ACL.

Returns:

• ACL –

  dict[str, list[str]]: A dictionary containing lists of users/groups with 'read' and 'write' permissions.

Example:

from datatailr import KV
kv = KV()
acl = kv.acls("db_url")

add_acl

Add an entry to the access control list (ACL) for a specific key in the key-value store.

Parameters:

• key

  (str) –

  The key for which to add the ACL entry.

• acls

  (ACL) –

  The ACL to add.

Returns:

• None –

  None

Example:

from datatailr import KV, ACL, Permission, User, Group
kv = KV()
acls = ACL({Permission.READ: [User("user1"), Group("group1")],
            Permission.WRITE: [User("user2")]})
kv.add_acl("db_url", acls)

delete

Delete a key-value pair from the key-value store by its key.

Parameters:

• key

  (str) –

  The key of the value to delete.

Returns: None Example:

from datatailr import KV
kv = KV()
kv.delete("db_url")  # Deletes the key-value pair with key "db_url"

get

Retrieve a value by its key from the key-value store. Note that different environments (dev/pre/prod) have separate key-value stores.

Parameters:

• key

  (str) –

  The key of the value to retrieve.

Returns:

• str ( str ) –

  The value associated with the key.

Example:

from datatailr import KV
kv = KV()
db_url = kv.get("db_url")

ls

List all keys stored in the key-value store for the current environment (dev/pre/prod).

Returns:

• list[dict[str, str | ACL]] –

  list[dict[str, str|ACL]]: A list of all keys in the key-value store along with their ACLs.

Example:

from datatailr import KV
kv = KV()
all_keys = kv.ls()

put

Store a key-value pair in the key-value store. Note that different environments (dev/pre/prod) have separate key-value stores.

Parameters:

• key

  (str) –

  The key under which to store the value.

• value

  (str) –

  The value to store.

• acl

  (ACL, default: None ) –

  The access control list for the key-value pair.

Returns:

• None –

  None

Example:

from datatailr import KV, ACL, Permission, User, Group
kv = KV()
acls = ACL({Permission.READ: [User.get("user1"), Group.get("group1")],
            Permission.WRITE: [User.get("user2")]})
kv.put("db_url", "postgresql://user:password@host:port/dbname", acls)

set_acls

Set the access control list (ACL) for a specific key in the key-value store, replacing any existing ACL entries.

Parameters:

• key

  (str) –

  The key for which to set the ACL.

• acls

  (ACL) –

  The ACL to set.

Returns:

• None –

  None

Example:

from datatailr import KV, ACL, Permission, User, Group
kv = KV()
acls = ACL({Permission.READ: [User("user1"), Group("group1")],
            Permission.WRITE: [User("user2")]})
kv.set_acls("db_url", acls)

Permission

Enumeration of permission types used in Access Control Lists.

Each permission controls a different aspect of access to a Datatailr resource.

Attributes:

• READ –

  Allows reading or viewing the resource.

• WRITE –

  Allows modifying or updating the resource.

• OPERATE –

  Allows executing or operating the resource (e.g., running a job).

• ACCESS –

  Allows accessing the resource (e.g., opening an app).

• PROMOTE –

  Allows promoting the resource to the next environment.

Resources dataclass

Represents the compute resources allocated to a job container.

Attributes:

• memory (str) –

  Memory limit as a string (e.g. "256m", "1g").

• cpu (float) –

  Number of CPU cores to allocate (e.g. 1, 0.5).

Schedule

Build a schedule object for batch/workflow jobs using friendly fields.

Examples:

schedule = Schedule(at_hours=[0])
schedule = Schedule(at_minutes=[0, 30], weekdays=["Mon", "Wed", "Fri"])

You can either provide a raw cron_expression or use the human-readable helper parameters (which are compiled into cron syntax automatically).

Parameters:

• cron_expression

  (str, default: '' ) –

  A raw cron string (e.g. "0 */2 * * *"). If provided alongside helper parameters, the helpers take precedence.

• at_minutes

  (list[int] | None, default: None ) –

  Specific minutes past the hour to run (e.g. [0, 30] for ":00" and ":30").

• every_minute

  (int | None, default: None ) –

  Run every N minutes (e.g. 5 for every 5 minutes).

• at_hours

  (list[int] | None, default: None ) –

  Specific hours of the day to run (0 -- 23).

• every_hour

  (int | None, default: None ) –

  Run every N hours.

• weekdays

  (list[str] | None, default: None ) –

  Days of the week to run (e.g. ["mon", "wed", "fri"]).

• day_of_month

  (int | None, default: None ) –

  Day of the month to run (1 -- 31).

• in_month

  (list[str] | None, default: None ) –

  Months to run in (e.g. ["jan", "apr", "jul", "oct"]).

• every_month

  (int | None, default: None ) –

  Run every N months.

• timezone

  (str | None, default: None ) –

  IANA timezone name (e.g. "America/New_York").

• run_after_job_uuid

  (str | None, default: None ) –

  UUID of a job that must complete before this schedule triggers.

• run_after_job_name

  (str | None, default: None ) –

  Name of a job that must complete before this schedule triggers.

• run_after_job_condition

  (str | None, default: None ) –

  Required completion status of the predecessor job (e.g. "success").

Example

from datatailr import Schedule
# Every weekday at 08:00 and 16:00 UTC
s = Schedule(at_hours=[8, 16], weekdays=["mon","tue","wed","thu","fri"])

get_cron_string

Return the compiled cron string.

Returns:

• str –

  Cron string (str).

Examples:

>>> Schedule(
...     at_minutes=[0, 15, 30, 45],
...     at_hours=[0, 12],
...     weekdays=["Mon", "Wed", "Fri"],
...     day_of_month=15,
...     in_month=["Jan", "Jul"],
... ).get_cron_string()
'0 0,15,30,45 0,12 15 1,7 1,3,5'

Secrets

Interface for managing secrets stored in the Datatailr platform.

Secrets are sensitive key-value pairs (passwords, API keys, tokens, etc.) that are stored securely and scoped to the current environment (dev/pre/prod). Access is controlled by per-secret ACLs.

Example

from datatailr import Secrets
secrets = Secrets()
db_password = secrets.get("database_password")
all_secrets = secrets.ls()

get

Retrieve a secret value by its key. Only secrets that are available at the current environment (dev/pre/prod) can be accessed. The user must have the appropriate permissions to access the secret.

Parameters:

• key

  (str) –

  The key of the secret to retrieve.

Returns:

• str ( str ) –

  The value of the secret.

Example:

from datatailr import Secrets
secrets = Secrets()
db_password = secrets.get("database_password")

ls

List all available secret keys in the current environment (dev/pre/prod). The user must have the appropriate permissions to list the secrets.

Returns:

• list[dict[str, str | ACL]] –

  list[dict[str, str|ACL]]: A list of all secret keys along with their ACLs.

Example:

from datatailr import Secrets
secrets = Secrets()
all_secrets = secrets.ls()

set_env

Set the environment variable for the secret.

Note: Environment variables can be read by subprocesses and may be surfaced in logs or crash dumps; only use this for secrets you are comfortable exposing to the current process environment.

If env_var_name is not provided, it will be derived from the secret name by replacing slashes with underscores and converting to uppercase. Args: secret_name (str): The name of the secret to set the environment variable for. env_var_name (str | None): The name of the environment variable to set. overwrite (bool): Whether to overwrite the environment variable if it already exists.

Returns:

• str ( str ) –

  The name of the environment variable that was set.

Example:

>>> from datatailr import Secrets
>>> secrets = Secrets()
>>> env_var_name = secrets.set_env("db/password", "DB_PASSWORD") # doctest: +SKIP
>>> print(env_var_name) # doctest: +SKIP
DB_PASSWORD

Service

Represents a long-running background service deployment on Datatailr.

A service runs continuously (e.g., an API server, a message consumer, or any always-on process). It is restarted automatically if it exits.

Example

from datatailr import Service

# service.py
from flask import Flask

app = Flask(__name__)

@app.route("/health")
def health_check():
    return "OK"

# Service entrypoints receive the port from Datatailr.
def run_server(port):
    app.run("0.0.0.0", port=int(port), debug=False)

svc = Service(
    name="Simple Service",
    entrypoint=run_server,
    python_requirements=["flask"],
)
svc.run()

Parameters:

• name

  (str) –

  Display name for the service.

• entrypoint

  (Callable) –

  The callable (function) that starts the service.

• environment

  (Environment | None, default: DEV ) –

  Target environment for the deployment.

• image

  (Image | None, default: None ) –

  Pre-configured container Image.

• run_as

  (str | User | None, default: None ) –

  User or username to run the service as.

• resources

  (Resources, default: Resources() ) –

  CPU and memory resources for the container.

• acl

  (ACL | None, default: None ) –

  Access control list.

• python_version

  (str, default: '3.12' ) –

  Python version for the container image.

• python_requirements

  (str | List[str], default: '' ) –

  Python dependencies (see Image).

• build_script_pre

  (str, default: '' ) –

  Dockerfile commands to run before installing requirements.

• build_script_post

  (str, default: '' ) –

  Dockerfile commands to run after installing requirements.

• env_vars

  (dict[str, str | int | float | bool] | None, default: None ) –

  Environment variables passed to the running container.

• get_existing

  (bool, default: False ) –

  If True, update an existing job definition.

• version

  (str | int | None, default: None ) –

  The version of the job to get.

• budget

  (Budget | str | None, default: None ) –

  Optional spend budget as Budget("name") or a name str. Will be set to the 'default' budget if not provided.

User

Representing a Datatailr User.

This class provides methods to interact with the Datatailr User API. It allows you to create, update, delete, and manage users within the Datatailr platform.

Attributes:

• first_name (str) –

  The first name of the user.

• last_name (str) –

  The last name of the user.

• name (str) –

  The username of the user.

• email (str) –

  The email address of the user.

• user_id (int) –

  The unique identifier for the user.

• groups (list) –

  List of groups the user belongs to.

• is_system_user (bool) –

  Indicates if the user is a system user.

At least one of name or id must be provided. When both are given, name takes precedence.

Parameters:

• name

  (str | None, default: None ) –

  The username to look up.

• id

  (int | None, default: None ) –

  The numeric user ID to look up.

• first_name

  (str | None, default: None ) –

  The first name of the user.

• last_name

  (str | None, default: None ) –

  The last name of the user.

• email

  (str | None, default: None ) –

  The email address of the user.

• groups

  (list[str] | None, default: None ) –

  The list of groups the user belongs to.

• is_system_user

  (bool, default: False ) –

  Whether the user is a system user.

Raises:

• ValueError –

  If neither name nor id is provided.

email property

The email address of the user.

first_name property

The first name of the user.

groups property

List of group names the user belongs to.

is_system_user property

Whether this user is a system (non-interactive) user.

last_name property

The last name of the user.

name property

The username.

user_id property

The unique numeric identifier of the user.

add staticmethod

Create a new user on the Datatailr platform.

Parameters:

• name

  (str) –

  The username for the new user.

• first_name

  (str) –

  The user's first name.

• last_name

  (str) –

  The user's last name.

• email

  (str) –

  The user's email address.

• password

  (str) –

  The login password. Ignored for system users.

• groups

  (list[str] | None, default: None ) –

  Optional list of group names to add the user to.

• is_system_user

  (bool, default: False ) –

  If True, create a non-interactive system user.

Returns:

• 'User' | None –

  The newly created User, or None if the operation failed.

Raises:

• Warning –

  If a password is provided for a system user.

Notes

Passwords are ignored for system users.

exists staticmethod

Check whether a user with the given username exists.

Parameters:

• name

  (str) –

  The username to look up.

Returns:

• bool –

  True if the user exists, False otherwise.

from_dict classmethod

Construct a User instance from a dictionary.

get staticmethod

Retrieve an existing user by username or numeric ID.

Parameters:

• name_or_id

  (str | int) –

  The username (str) or user ID (int).

Returns:

• User –

  The matching User instance.

Raises:

• ValueError –

  If no user with the given name or ID exists.

ls staticmethod

List all users available in the Datatailr platform.

Parameters:

• filter

  (str | None, default: None ) –

  A filter expression to apply to the users.

Examples:

>>> from datatailr import User
>>> User.ls()
[...User(name=root, first_name=Super User, last_name=, email=, user_id=0, groups=[''], is_system_user=True), ...]

Returns:

• list –

  A list of User instances.

remove staticmethod

Remove (delete) a user from the platform.

Parameters:

• name

  (str) –

  The username of the user to remove.

signed_user staticmethod

Retrieve the currently signed-in (authenticated) user.

Returns:

• User –

  The User instance of the currently authenticated user.

Raises:

• PermissionError –

  If no signed-in user is found.

to_cookie

Return a cookie string for the user.

verify

Verify the user's authentication signature with the platform.

This refreshes and validates the user's credentials against the Datatailr authentication service.

construct_remote_base_url

Construct the base URL of the remote Datatailr instance for a given service name.

Parameters:

• service_name

  (str) –

  The name of the service.

• environment

  (str | None, default: None ) –

  The environment of the service.

Returns:

• str –

  The base URL of the remote Datatailr instance for the given service name.

Raises:

• RuntimeError –

  If not logged in or config is invalid.

get_dt_env

Get the current environment.

get_remote_base_url

Return the base URL of the remote Datatailr instance.

Raises:

• RuntimeError –

  If not logged in or config is invalid.

get_remote_http_headers

Return default HTTP headers for calling the Datatailr API as the remote CLI does (Cookie, Accept, User-Agent).

Raises:

• RuntimeError –

  If not logged in or config is invalid.

get_remote_oidc_cookie_line

Return a single X-Datatailr-Oidc-Data=<jwt> string (same line as datatailr export-auth prints when run without --shell or --fish).

Raises:

• RuntimeError –

  If not logged in or config is invalid.

get_remote_oidc_cookie_pair

Return (cookie_name, jwt) for building custom headers or cookies.

Raises:

• RuntimeError –

  If not logged in or config is invalid.

get_remote_oidc_jwt

Return the OIDC JWT cookie value used for authenticated HTTP requests.

Raises:

• RuntimeError –

  If not logged in or config is invalid.

is_dt_installed

Check if the native Datatailr CLI is available on PATH.

Returns:

• bool –

  True if the resolved dt binary is /opt/datatailr/bin/dt; otherwise False.

is_remote_logged_in

Return whether the remote CLI has a usable authenticated session.

Parameters:

• validate_server

  (bool, default: True ) –

  When true, also call the API to confirm the token is still valid. When false, only check that local config exists and looks valid.

• timeout_s

  (int, default: 5 ) –

  Timeout for server validation requests.

load_remote_client_config

Load remote client configuration from disk.

Returns:

• RemoteClientConfig –

  Validated config with base_url and jwt_cookie.

Raises:

• RuntimeError –

  If the config file is missing or invalid (run datatailr login).

task

Decorator that turns a function into a workflow task.

Apply this via the public alias @task(). When the decorated function is called inside a workflow-decorated function, the call returns a Task object that becomes a node in the workflow DAG. Dependencies between tasks are inferred automatically from the data flow between calls.

Tasks are only registered for deployment when they are invoked from within the body of a @workflow-decorated function, which builds a Workflow. Calling a @task-decorated function outside that context executes it locally as a normal Python function instead.

See Task below for methods available on the returned object (for example alias() and set_resources()).

Parameters:

• memory

  (str, default: DEFAULT_TASK_MEMORY ) –

  Memory limit for this task's container (e.g. "256m", "1g").

• cpu

  (float, default: DEFAULT_TASK_CPU ) –

  Number of CPU cores to allocate for this task (e.g. 1, 0.5).

Returns:

• Task ( Callable[[Callable[..., Any]], Callable[..., Any]] ) –

  The decorated function. When that function is called inside a @workflow-decorated function, the call returns a Task object that becomes a node in the workflow DAG.

Example

# tasks.py
from datatailr import task

@task(memory="512m", cpu=2)
def heavy_computation(x, y):
    return x ** y

@task()
def prepare():
    return 2

Then to deploy the workflow, use the following code:

# deploy.py
from datatailr import workflow
from tasks import heavy_computation, prepare

@workflow("My Workflow")
def my_workflow():
    base = prepare()
    heavy_computation(base, 10)

if __name__ == "__main__":
    my_workflow()  # deploy to Datatailr