Index
Distributed joblib on Datatailr.
Run scikit-learn (or any joblib-parallel code) on an ephemeral pool of Datatailr worker containers with no manual setup::
import joblib
from datatailr.joblib import register_datatailr
register_datatailr()
with joblib.parallel_backend("datatailr", n_jobs=16):
search.fit(X, y) # e.g. RandomizedSearchCV
# Optional: pack more fits per broker round-trip (default initial_batch_size=128)
# with joblib.parallel_backend("datatailr", n_jobs=16, initial_batch_size=256):
# search.fit(X, y)
See :class:datatailr.joblib.DatatailrBackend for the available backend
options (worker resources, environment, embedded modules, timeouts).