In this tutorial, you will use
clearml-task to execute this script
on a remote or local machine, from the remote repository and from a local script.
clearmlPython package installed
clearml-agentrunning on at least one machine (to execute the experiment) and assigned to listen to default queue
allegroai/events repository cloned (for local script execution)
clearml-task with the following arguments:
--project keras_examples --name remote_test- The project and experiment name. If the project entered doesn't exist, a new project will be created with the selected name.
--repo https://github.com/allegroai/events.git- The chosen repository's URL. By default,
clearml-taskwill use the latest commit from the master branch.
--script /webinar-0620/keras_mnist.py- The script to be executed.
--args batch_size=64 epochs=1- Arguments passed to the script.
This uses the
argparseobject to get CLI parameters.
--queue default- Selected queue to send the experiment to.
clearml-task does the rest of the heavy-lifting!
- It creates a new Task on the ClearML Server.
- Then, the Task is enqueued in the selected execution queue, where it will be executed by an available
clearml-agentassigned to that queue.
Your output should look something like this:
clearml-task automatically finds the requirements.txt file in remote repositories.
If a remote repo does not have such a file, make sure to either add one with all the required Python packages,
or add the
--packages '<package_name> flag to the command.
clearml-task to execute a local script is very similar to using it with a remote repo.
For this example, we will be using a local version of this script.
- Go to the root folder of the cloned allegroai/events repository
Notice that the command is almost identical to executing code from a git repository. The only differences are:
--script webinar-0620/keras_mnist.py- Pointing
clearml-taskto a local script.
--requirements webinar-0620/requirements.txt- Manually specifying a requirements.txt file.
clearml-task, a Task will be created according to the parameters entered. The Task will
be sent to a queue for execution.