Skip to main content

Keras

tip

If you are not already using ClearML, see Getting Started for setup instructions.

ClearML integrates with Keras out-of-the-box, automatically logging its models, scalars, TensorFlow definitions, and TensorBoard outputs.

All you have to do is simply add two lines of code to your Keras script:

from clearml import Task

task = Task.init(task_name="<task_name>", project_name="<project_name>")

And that's it! This creates a ClearML Task which captures:

  • Source code and uncommitted changes
  • Installed packages
  • Keras models
  • Scalars (e.g. accuracy, loss)
  • TensorFlow definitions
  • TensorBoard outputs
  • Console output
  • General details such as machine details, runtime, creation date etc.
  • And more

You can view all the task details in the WebApp.

WebApp Gif

Automatic Logging Control​

By default, when ClearML is integrated into your Keras script, it captures Keras models and scalars, as well as TensorFlow definitions and TensorBoard outputs. But, you may want to have more control over what your experiment logs.

To control a task's framework logging, use the auto_connect_frameworks parameter of Task.init(). Completely disable all automatic logging by setting the parameter to False. For finer grained control of logged frameworks, input a dictionary, with framework-boolean pairs.

For example:

auto_connect_frameworks={
   'tensorflow': False, 'tensorboard': False, 'matplotlib': True, 'pytorch': True,
   'xgboost': False, 'scikit': True, 'fastai': True, 'lightgbm': False,
   'hydra': True, 'detect_repository': True, 'tfdefines': True, 'joblib': True,
   'megengine': True, 'catboost': True
}

To control Keras logging, use the tensorflow and tensorboard keys.

You can also input wildcards as dictionary values, so ClearML will log a model created by a framework only if its local path matches at least one wildcard.

For example, in the code below, ClearML will log TensorFlow (and/or keras) models only if their paths have the .keras extension. The unspecified frameworks' values default to true so all their models are automatically logged.

auto_connect_frameworks={'tensorflow' : '*.keras'}

Manual Logging​

To augment its automatic logging, ClearML also provides an explicit logging interface.

See more information about explicitly logging information to a ClearML Task:

See Explicit Reporting Tutorial.

Examples​

Take a look at ClearML's Keras examples. The examples use Keras and ClearML in different configurations with additional tools like TensorBoard and Matplotlib:

  • Keras with Tensorboard - Demonstrates ClearML logging a Keras model, and plots and scalars logged to TensorBoard
  • Keras with Matplotlib - Demonstrates ClearML logging a Keras model, Matplotlib plots, and debug samples, plots, and scalars logged to TensorBoard

Remote Execution​

ClearML logs all the information required to reproduce an experiment on a different machine (installed packages, uncommitted changes etc.). The ClearML Agent listens to designated queues and when a task is enqueued, the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the experiment manager.

Deploy a ClearML Agent onto any machine (e.g. a cloud VM, a local GPU machine, your own laptop) by simply running the following command on it:

clearml-agent daemon --queue <queues_to_listen_to> [--docker]

Use the ClearML Autoscalers to help you manage cloud workloads in the cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up and shuts down instances as needed, according to a resource budget that you set.

Cloning, Editing, and Enqueuing​

Cloning, editing, enqueuing gif

Use ClearML's web interface to edit task details, like configuration parameters or input models, then execute the task with the new configuration on a remote machine:

  • Clone the experiment
  • Edit the hyperparameters and/or other details
  • Enqueue the task

The ClearML Agent executing the task will use the new values to override any hard coded values.

Executing a Task Remotely​

You can set a task to be executed remotely programmatically by adding Task.execute_remotely() to your script. This method stops the current local execution of the task, and then enqueues it to a specified queue to re-run it on a remote machine.

# If executed locally, process will terminate, and a copy will be executed by an agent instead
task.execute_remotely(queue_name='default', exit_process=True)

Hyperparameter Optimization​

Use ClearML's HyperParameterOptimizer class to find the hyperparameter values that yield the best performing models. See Hyperparameter Optimization for more information.