ClearML Hyper-Datasets
Build Data-Centric AI workflows
Hyper-Dataset workflow
Leverage unstructured data
Explore
Organize, explore, and analyze a searchable catalog of your datasets with fully searchable version trees.
Query
Slice, split, and sort by meta-data matching, allowing to oversample and debias datasets on-the-fly.
Store your queries in a fully reproducible entity, DataView, in conjunction with your code for full provenance and model reproducibility.
Visualize
Gain insights into your data using the meta-data search capabilities. Visualize any data and edit meta-data directly from the UI or programmatically and instantly view object distribution statistics for any dataset version for additional visibility.
Orchestrate
Decouple training and testing code, enabling users to test & validate models directly from the web UI & use a programmatic interface available through a flexible query mechanism with any data distribution and on multiple weighted queries.
Data-Centric model development
- Upload or register existing files to any supported storage solution including object-storage / http / shared-folder
- Add ground-truth annotations and structured meta-data (json)
- Create snapshots (a.k.a versions)
- Query and explore your data through the web UI
- Slice, split, and sort based on meta-data as part of your training code
- Refine using different slicing and dataset mixtures
- Introduce automation on any dataset version changes
Learn how Neural Guard developed a methodology to build datasets for DL models that optimize ROI and TTM to increase competitiveness in their market.
Hyper-Datasets and beyond
ClearML Hyper-Datasets integrates seamlessly with ClearML Experiment and ClearML Orchestrate, leveraging end-to-end cross-department visibility in your research, development, and production.
Get Started with ClearML for free
Trusted by thousands of teams around the world, ClearMl installs in 2 minutes.