Google Colab is a common development environment for data scientists. It supports a convenient IDE as well as compute provided by Google.
Users can transform a Google Colab instance into an available resource in ClearML using ClearML Agent.
This tutorial goes over how to create a ClearML worker node in a Google Colab notebook. Once the worker is up and running, users can send Tasks to be executed on Google Colab's hardware.
- Be signed up for ClearML (or have a server deployed).
- Have a Google account to access Google Colab
Open up this Google Colab notebook.
Run the first cell, which installs all the necessary packages:
!pip install git+https://github.com/allegroai/clearml
!pip install clearml-agent
Run the second cell, which exports this environment variable:
! export MPLBACKEND=TkAg
This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen.
Create new credentials. Go to your Settings page > WORKSPACE section. Under App Credentials, click + Create new credentials, and copy the information that pops up.
Set the credentials. In the third cell, enter your own credentials:
from clearml import Task
In the fourth cell, launch a
clearml-agentthat will listen to the
!clearml-agent daemon --queue default
For additional options for running
clearml-agent, see the clearml-agent reference.
After executing cell 4, the worker appears in the Orchestration page of your server. Clone experiments and enqueue them to your hearts content! The
clearml-agentwill fetch experiments and execute them using the Google Colab hardware.