Skip to main content

Version 0.14

info

Trains is now ClearML.

Version 0.14.3​

Trains​

Features

  • Add the Logger.report_histogram method mode parameter. The values are group (the default), stack, and relative.
  • Add PEP610 support for the new pip version 20.1 and newer Git reference feature (see pypa/pip#609) (GitHub Issue #62).
  • Improve support for PyTorch Ignite integration.
  • Support # and ? in uploaded image file names.

Bug Fixes

  • Support ArgumentParser.parse_arg use before initializing a Task when auto_connect_arg_parser=False.
  • Fix upload_object_via_stream in the Azure storage driver.
  • Fix the Task.get_tasks method return type.

Version 0.14.2​

Trains​

Features

  • Add media (audio) support for both Logger and TensorBoard binding using Logger.report_media (GitHub Issue #120).
  • Add autokeras example (GitHub Issue #125).
  • Add caching of extracted zip artifacts.
  • Add Task.get_tasks filtering support.
  • Add trains.storage.StorageManager.
  • Add nicer stdout log flush.
  • Add text logging to TF V1 example.
  • Improve SSL retries and error handling.
  • Improve shutdown behavior in case of a user abort (Ctrl-C).
  • Improve repository and module detection:
    • Improve resource monitor.
    • Report memory usage for process (and sub-processes).
    • Report GPU memory usage for process (and sub-processes).
    • Stability
  • Improve Jupyter support:
    • Make sure trains is included in Jupyter requirements.
    • Ignore IPython directives in converted Python script (like % and ! lines).
  • Update PyTorch / TensorboardX examples.

Bug Fixes

  • Fix renaming / deleting model file right after saving will break asynchronous upload (GitHub Issue #123).
  • Fix shutdown behavior:
    • Logger flush delay and termination.
    • Repository / package detection termination.
    • Threads termination.
  • Fix text encoding utf-8 and pr_curve broken in Tensorboard support.
  • Fix single log request exceeds max packet size.
  • Fix upload debug assets with series / title including , or / did not add iteration index to the filename.
  • Fix scikit-learn module detection.
  • Use UID in message if username cannot be retrieved.
  • Fix cast in Tensorflow v2 example.

Trains Server​

User survey

  • Add survey invitation popup.

Bug fixes

  • Fix misaligned titles in experiment results, scalars, and plots (RESULTS tab, SCALARS and PLOTS sub-tabs).
  • Fix parallel coordinate comparison in comparing experiments. Adding dimension with no values removes all lines.
  • Fix resizing the experiments tables window. The preferences wheel jumped over the titles.
  • Fix the browser zoom (CTRL +, CTRL -, and mouse wheel) in experiment RESULTS > SCALARS tab. It broke labels (GitHub Issue #127).

Version 0.14.1​

Trains​

Features and Bug Fixes

  • Add OpenMPI / Slurm support, support TRAINS_FORCE_MASTER_NODE environment variable to allow forcefully specifying a node is master (not based on the MPI Rank).
  • Add Task.get_models and Task.models for retrieving stored models for previously executed tasks
  • Support multiple EventWriter in TensorFlow eager mode (TF 2.0+) (GitHub Issue #26).
  • Support reusing models.
  • Fix TensorFlow not a number (NaN) and infinity (Inf) values support (GitHub Issue #118).
  • Fix joblib binding.
  • Fix metrics / files / models upload on shutdown.
  • Fix support for Task init / close multiple times.
  • Update manual_reporting.py example with Logger.report_table example.
  • Deprecate Task.set_model_config, Task.get_model_config_text, and Task.get_model_config_dict.

Trains Server​

Features and Bug Fixes

  • Support default Trains Server Kubernetes Helm ports in the Trains Web-App (UI).
  • Fix issue switching scalar plots between linear and logarithmic views (GitHub Issue #75).
  • Do not reset view filtering when switching tabs in experiment view.

Trains Agent​

Features and Bug Fixes

  • Add daemon detached mode (--detached, -d) that runs the agent as daemon in the background and returns immediately.
  • Auto mount ~/.git-credentials into a Docker container, if the file exists.
  • Add TRAINS_AGENT_EXTRA_PYTHON_PATH environment variable to allow the adding of an additional Python path during experiment execution (helpful when using extra un-tracked modules).
  • Fix "run as user" feature (using TRAINS_AGENT_EXEC_USER environment variable).
  • Fix PyTorch support to ignore minor versions when looking for package to install / download.
  • Fix experiment execution output handling.

Version 0.14.0​

Trains​

Features and Bug Fixes

  • Add support for reporting tables (pandas.DataFrame, CSV files, URL to CSV file) using the Logger.report_table method.
  • Add support for setting a Task's initial iteration to allow continuing previous runs using the Task.set_initial_iteration and Task.get_initial_iteration methods.
  • Add multiprocess task support (GitHub Issue #96).
  • Add trains-init support for config file override using the TRAINS_CONFIG_FILE environment variable or --file command line argument (GitHub Issue #105).
  • Add initial slurm support.
  • Allow disabling repository detection when calling Task.init.
  • Add Jupyter Notebook server execution example.
  • Update examples.
  • Improve warning messages for storage errors and missing storage packages.
  • Fix HTTP link quoting in stored links.

Trains Server​

Features and Bug Fixes

  • Add debug image viewing for an unlimited number of iterations, filtering by metric, and comparing debug images for different iterations.
  • Add debug image view zoom, fit to screen, resize, hover coordinates (GitHub Issue #68).
  • Add switching scalar plots between linear and logarithmic views (GitHub Issue #75).
  • Add scalar minimums and maximums to comparing experiments.
  • Add opening HTML artifact files in the browser.
  • Add Web-App notification for version updates.
  • Add toggle button to switch between full screen and standard views in the RESULTS tab.
  • Add a load more button to the global search results.
  • Add automatic detection of debug video type from the file extension.
  • Add Task.get_parameters_as_dict and Task.set_parameters_as_dict.
  • Allow editing model description.
  • Allow editing experiment uncommitted changes and installed packages separately.
  • Allow deleting an input model from an experiment whose status is Draft.
  • Fix dialog elements which are now accessible in non-100% screen scale. (GitHub Issue #89).
  • Fix custom columns in the experiments table not properly loaded from browser store (GitHub Issue #106).
  • Fix issue with showing 2D graphs containing sub-plots (trains #slack)
  • Fix issues with vertical scrolling of experiment logs.
  • Fix issue with image plots sometimes not visible in experiments comparison.

Trains Agent​

Features and Bug Fixes

  • Add support for trains-agent execute --id <experiment-id> --docker that allows executing a specific experiment inside a docker container.
  • Add support for trains-agent execute --id <template-experiment-id> --clone that clones the provided experiment and executes the cloned experiment.
  • Add support for APIClient.models.delete to allow programmatically deleting a model (GitHub trains-server Issue #32).
  • Add daemon support for passing storage-related OS environment variables to experiments executed inside a Docker container (supported by trains>=0.13.3):
    • AWS: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION
    • Azure: AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY
    • Google: GOOGLE_APPLICATION_CREDENTIALS
  • Fix Git checkout with submodules (GitHub Issue #112).
  • Docker image in command line takes precedence over the Docker image specified in an experiment.