Skip to main content

ClearML Agent Environment Variables

This page lists the available environment variables for configuring ClearML Agent.

In addition to the environment variables listed below, ClearML also supports dynamic environment variables to override any configuration option that appears in the agent section of the clearml.conf. For more information, see Dynamic Environment Variables.

note

ClearML's environment variables override the clearml.conf file, SDK, and configuration vault, but can be overridden by command-line arguments.

NameDescription
CLEARML_DOCKER_IMAGESets the default docker image to use when running an agent in Docker mode
CLEARML_WORKER_NAMESets the Worker's name
CLEARML_WORKER_IDSets the Worker ID
CLEARML_CUDA_VERSIONSets the CUDA version to be used
CLEARML_CUDNN_VERSIONSets the CUDNN version to be used
CLEARML_CPU_ONLYForce CPU only mode
CLEARML_DOCKER_SKIP_GPUS_FLAGSkips the GPUs flag (support for docker V18)
CLEARML_AGENT_ABORT_CALLBACK_CMDPath to a callback script to execute when a Task is aborted. The script will run for up to the maximum time set with CLEARML_AGENT_ABORT_CALLBACK_TIMEOUT. Set Task callback script using -e CLEARML_AGENT_ABORT_CALLBACK_CMD=path/script.sh.
CLEARML_AGENT_ABORT_CALLBACK_TIMEOUTMaximum time (in seconds) the abort callback script (set with CLEARML_AGENT_ABORT_CALLBACK_CMD) is allowed to run. Set abort callback timeout using -e CLEARML_AGENT_ABORT_CALLBACK_TIMEOUT=30.
CLEARML_AGENT_DOCKER_ARGS_FILTERSSet a whitelist of allowed Docker arguments. Only arguments matching the specified patterns can be used when running a task. Use shlex.split whitespace-separated format. For example: CLEARML_AGENT_DOCKER_ARGS_FILTERS="^--env$ ^-e$"
CLEARML_AGENT_DOCKER_ARGS_HIDE_ENVHide Docker environment variables containing secrets when printing out the Docker command. When printed, the variable values will be replaced by ********. See agent.hide_docker_command_env_vars
CLEARML_AGENT_DISABLE_SSH_MOUNTDisables the auto .ssh mount into the docker
CLEARML_AGENT_FORCE_CODE_DIRBypass repository cloning and/or applying any changes to the code. The agent will execute the specified entry point from the provided path.
CLEARML_AGENT_FORCE_UVIf set to 1, force the agent to use UV as the package manager. Overrides the default manager set in the clearml.conf under agent.package_manager.type
CLEARML_AGENT_FORCE_EXEC_SCRIPTAllows overriding the remote execution script to bypass repository cloning and execute code already available where the remote agent is running. Use module:file.py format to specify a module and a script to execute (e.g. .:main.py to run main.py from the working dir)
CLEARML_AGENT_FORCE_TASK_INITIf set to 1, ClearML Agent adds Task.init() to scripts that do not have the call, creating a Task to capture code execution information and output, which is then sent to the ClearML Server. If set to 0 and the script does not include Task.init(), the agent will capture only the output streams and console output, without tracking code execution details, metrics, or models.
CLEARML_AGENT_FORCE_SYSTEM_SITE_PACKAGESIf set to 1, overrides default agent.package_manager.system_site_packages: true behavior when running tasks in containers (docker mode and k8s-glue)
CLEARML_AGENT_GIT_CLONE_VERBOSEIf set to 1, git clone calls will report progress verbosely
CLEARML_AGENT_GIT_USERSets the Git user for ClearML Agent
CLEARML_AGENT_GIT_PASSSets the Git password for ClearML Agent
CLEARML_AGENT_GIT_HOSTSets Git host (only sending login to this host)
CLEARML_AGENT_GIT_USE_MS_ENTRA_TOKENIf set to 1, enables authentication to Azure DevOps repositories using a Microsoft Entra token. The Azure token will be taken from the git password setting (CLEARML_AGENT_GIT_PASS or agent.git_host). CLEARML_AGENT_GIT_USE_MS_ENTRA_TOKEN and CLEARML_AGENT_GIT_USE_AZURE_PAT are mutually exclusive, only one can be set to 1. For more information, see the Azure DevOps authentication guide.
CLEARML_AGENT_GIT_USE_AZURE_PATIf set to 1, enables authentication to Azure DevOps repositories using a Personal Access Token (PAT). The PAT will be taken from the git password setting (CLEARML_AGENT_GIT_PASS or agent.git_host). CLEARML_AGENT_GIT_USE_MS_ENTRA_TOKEN and CLEARML_AGENT_GIT_USE_AZURE_PAT are mutually exclusive, only one can be set to 1. For more information, see the Azure DevOps authentication guide
CLEARML_AGENT_EXEC_USERUser for Agent executing tasks (root by default)
CLEARML_AGENT_EXTRA_DOCKER_ARGSOverrides extra docker args configuration
CLEARML_AGENT_EXTRA_DOCKER_LABELSList of labels to add to docker container. See Docker documentation.
CLEARML_EXTRA_PIP_INSTALL_FLAGSList of additional flags to use when the agent installs packages. For example: CLEARML_EXTRA_PIP_INSTALL_FLAGS=--use-deprecated=legacy-resolver for a single flag or CLEARML_EXTRA_PIP_INSTALL_FLAGS="--use-deprecated=legacy-resolver --no-warn-conflicts" for multiple flags
CLEARML_AGENT_EXTRA_PYTHON_PATHSets extra Python path
CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDEOverrides initial server connection behavior (true by default), allows explicit number to specify number of connect retries)
CLEARML_AGENT_NO_UPDATEBoolean. Set to 1 to skip agent update in the k8s pod container before the agent executes the task
CLEARML_AGENT_K8S_HOST_MOUNT / CLEARML_AGENT_DOCKER_HOST_MOUNTSpecifies Agent's mount point for Docker / K8s
CLEARML_AGENT_TEMP_STDOUT_FILE_DIRAllows overriding the default /tmp location for agent temporary files
CLEARML_K8S_GLUE_START_AGENT_SCRIPT_PATHProvide an alternate path to place the agent startup script generated inside a k8s task pod (instead of the default ~/~/__start_agent__.sh)
CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVESets the PyTorch resolving mode. The options are:
  • none - No resolving. Install PyTorch like any other package
  • pip (default) - Sets extra index based on cuda and lets pip resolve
  • direct - Resolve a direct link to the PyTorch wheel by parsing the pytorch.org pip repository, and matching the automatically detected cuda version with the required PyTorch wheel. If the exact cuda version is not found for the required PyTorch wheel, it will try a lower cuda version until a match is found
CLEARML_AGENT_DEBUG_INFOProvide additional debug information for a specific context (currently only the docker value is supported)
CLEARML_AGENT_CHILD_AGENTS_COUNT_CMDProvide an alternate bash command to list child agents while working in services mode
CLEARML_AGENT_SKIP_PIP_VENV_INSTALLInstead of creating a new virtual environment inheriting from the system packages, use an existing virtual environment and install missing packages directly to it. Specify the Python binary of the existing virtual environment. For example: CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/home/venv/bin/python.

Note: By default, ClearML Agent will still install any missing packages into the specified virtual environment. To ensure no additional packages will be installed, also set CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=1.
CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALLIf set to 1, the agent will not install any required Python packages and will just use the preexisting Python environment to run the task.
CLEARML_AGENT_VENV_CACHE_PATHOverrides venv cache folder configuration
CLEARML_MULTI_NODE_SINGLE_TASKControl how multi-node resource monitoring is reported. The options are:
  • -1 - Only master node's (rank zero) console/resources are reported
  • 1 - Graph per node i.e. machine/GPU graph for every node (console output prefixed with RANK)
  • 2 - Series per node under a unified machine resource graph, graph per type of resource e.g. CPU/GPU utilization (console output prefixed with RANK)