In this tutorial we will learn how to launch a remote interactive session on Jupyter Notebook using
We will be using two machines. A local one, where we will be using an interactive session of Jupyter, and a remote machine,
clearml-agent will run and spin an instance of the remote session.
clearml-sessionpackage installed (
pip install clearml-session)
- At least one
clearml-agentrunning on a remote host. See installation details. Configure the
clearml-agentto listen to the
clearml-agent daemon --queue default)
- An SSH client installed on machine being used. To verify, open terminal, execute
ssh, and if no error is received, it should be good to go.
Step 1: Launch
clearml-session command with the following command line options:
- Enter a docker image
- Enter required python packages
--packages "clearml" "tensorflow>=2.2" "keras"
- Specify the resource queue
There is an option to enter a project name using
--project <name>. If no project is input, the default project
name is "DevOps"
After launching the command, the
clearml-agent listening to the
default queue spins a remote Jupyter environment with
the specifications. It will automatically connect to the docker on the remote machine.
The terminal should return output with the session's configuration details, which should look something like this:
When the CLI asks whether to
Launch interactive session [Y]/n?, press 'Y' or 'Enter'.
The terminal should output information regarding the status of the environment-building process, which should look something like this:
Then the CLI will output a link to the ready environment:
Click on the JupyterLab link, which will open the remote session
Now, let's execute some code in the remote session!
Open up a new Notebook.
In the first cell of the notebook, clone the ClearML Repo.!git clone https://github.com/allegroai/clearml.git
In the second cell of the notebook, we are going to run this script from the repository that we cloned.%run clearml/examples/frameworks/keras/keras_tensorboard.py
Look in the script, and notice that it makes use of ClearML, Keras, and TensorFlow, but we don't need to install these packages in Jupyter, because we specified them in the
To shut down the remote session, which will free the
clearml-agent and close the CLI, enter "Shutdown".