The script initializes a main Task and spawns subprocesses, each for an instance of that Task. The Task in each subprocess trains a neural network over a partitioned dataset (the torchvision built-in MNIST dataset), and reports (uploads) the following to the main Task:
- Artifacts - A dictionary containing different key-value pairs.
- Scalars - Loss reported as a scalar during training in each Task in a subprocess.
- Hyperparameters - Hyperparameters created in each Task are added to the hyperparameters in the main Task.
Each Task in a subprocess references the main Task by calling Task.current_task, which always returns the main Task.
When the script runs, it creates an experiment named
test torch distributed, which is associated with the
in the ClearML Web UI.
The example uploads a dictionary as an artifact in the main Task by calling the Task.upload_artifact
Task.current_task (the main Task). The dictionary contains the
of the subprocess, making each unique.
All of these artifacts appear in the main Task under ARTIFACTS > OTHER.
Loss is reported to the main Task by calling the Logger.report_scalar
Task.current_task().get_logger, which is the logger for the main Task. Since
Logger.report_scalar is called
with the same title (
loss), but a different series name (containing the subprocess'
rank), all loss scalar series are
The single scalar plot for loss appears in RESULTS > SCALARS.
ClearML automatically logs the argparse command line options. Since the Task.connect
method is called on
Task.current_task, they are logged in the main Task. A different hyperparameter key is used in each
subprocess, so they do not overwrite each other in the main Task.
All the hyperparameters appear in CONFIGURATIONS > HYPER PARAMETERS.
Output to the console, including the text messages printed from the main Task object and each subprocess appear in RESULTS > CONSOLE.