Python versioning 101 in ClearML

What is in a version ? It’s fair to say, that with most things in life, a later version of a product is usually better. Nowhere is this more true than Python. Recent versions have brought us such nice things as dictionary comprehensions, AsyncIO and others. Lots of goodness. However, what happens with ClearML ? Which version is used and when ?

This is a common question for those who don’t run clearml-agent with the ‘–docker’ flag. Docker really does remove a lot of weirdness at runtime. The price is some extra complexity upfront. I think it’s a fair trade but, I could be alone here and I admit, I maybe biased.

ClearML will use the Python version any task is first executed with. In most cases, this is your local Python. Jupyter Notebooks allow you to, obviously, specify the kernel as well. The ClearML Agent will then try to locate the exact same Python version. This means 3.8.x isn’t good enough if you are running 3.8.6. One way to get around this is to specify the python_binary setting in your clearml.conf.

Another more Pythonic approach, is to use pyenv or similar. For everyone’s sanity though, I would advise against this.

Of course, the better and cleaner way would be to use Docker. I did say I was biased *wink*.

Scroll to Top