By Chen Lugasi, Senior MLOps Engineer, Trax
Powered by leading technology and proprietary data, Trax connects brands, retailers, and shoppers, setting the standard for retail excellence. Trax’s AI-powered platform uniquely combines solutions that provide unparalleled data-driven signals to leading global CPGs and retailers to deliver real-time data, retail execution, and consumer engagement to increase ROI.
Twenty-nine of the world’s top 50 CPG companies, along with leading retailers and emerging brands, use Trax’s shelf monitoring, analytics, merchandising, activation, and shopper engagement solutions at scale to drive positive shopper experiences and unlock revenue opportunities at all points of sale.
Trax is a global company with hubs in the United States, Singapore, France, Hungary, China, Mexico, Brazil, and Israel, serving customers in more than 80 countries worldwide. To learn more, visit http://www.traxretail.com.
This is Trax’s story of how they use ClearML in their machine learning process:
In modern machine learning systems, managing models can become increasingly complex. Handling multiple variations of models and ensuring compatibility between them is crucial for maintaining a well-organized model repository. In this blog post, we’ll explore how we built a robust model repository on top of ClearML’s ModelStore, a feature that simplifies model management and deployment, allowing users to interface with any model using the queryable model catalog, add performance metrics, and trace lineage.
Why Trax Needed a Model Repository
At our organization, we deal with multiple use cases where each different use case requires a dedicated model. Each model consists of different components. A pipeline in our system might include these steps:
- Detector
- Classifier
- Embedder
Each step uses its own model and these models need to be compatible with each other for the pipeline to work seamlessly. These connections between models inputs and outputs highlights the challenge of managing different versions and variations for each model, and ensuring not only its accuracy but also its interoperability.
As Trax’s application runs on different resources (Both on edge devices like smartphones, and on the cloud), each supporting its own model format, we need to track model formats such as:
- Raw models (e.g., PyTorch or TensorFlow)
- Converted models (e.g., TensorFlow Lite or CoreML)
The way we at Trax use ClearML to solve this is we track these variations and their relationships via unique identifiers such as:
- Original model id (for the raw model)
- Previous model id (for models generated during the conversion pipeline)
We store all this metadata in ClearML ModelStore, which allows us to effectively manage the relationships between different models and their versions.
Model Metadata in ClearML
ClearML’s ModelStore offers a flexible way to store and query model metadata, allowing us to track essential attributes for each model:
- Name (e.g., `classifier_name`, `detector_v1`)
- Published (whether the model is ready for production use)
- Archived (indicating whether the model is no longer in use)
- Framework (e.g., CoreML, TensorFlow Lite)
To categorize and find models easily, we use tags like `<ENCRYPTED>` and custom tags like the model’s family. This helps us group and filter models based on their characteristics, which is crucial for scalability and searching.
Building a REST API for Model Management
To manage the lifecycle of models, we built and hosted our own REST API server that integrates with ClearML ModelStore. This REST API server serves 2 main functions:
- It acts as a buffer between ClearML’s API and the outside world (mainly for security reasons).
- It contains the extra logic we put in for managing the models the way we want.
Here are the key endpoints that power our model repository:
Publish New Models
- POST /model/{model_id}/publish: This endpoint publishes a model and archives its previous version if necessary.
Create and Update Model Resources
- POST /model/{model_id}/resource: Creates storage resources for model components like thresholds and mappings and provides storage URLs.
- PATCH /model/{model_id}: Updates mutable model attributes like custom fields and metadata.
Search and Retrieve Models
- POST /model: Allows for querying models by attributes such as model type, name, framework, and custom metadata fields.
- GET /model/{model_id}: Retrieves detailed information about a model, including storage URLs and vendor information.
Check Model State
Since a full working pipeline requires a synergy between multiple models, we also built a process to verify if all necessary model resources are present on disk. This ensures that models and their associated components are available for use in production. It checks whether all model resources for a given context (e.g., a pipeline or workflow) are available locally. A context is considered loaded only when all resources of all models are present.
Tags and Access Control
As mentioned earlier, an important part of our model repository is the tagging system. Tags are used to track each step in the model’s lifecycle. For example, tags can indicate whether a model is in production or is still part of a specific pipeline step.
Models are only accessible through the production API if they are in “Published” status and have the correct tags. This ensures that only models that are fully processed and encrypted are available for consumption in production, preventing unprocessed or insecure models from being used.
The Result: A Robust Model Repository
By using ClearML ModelStore, we’ve created a flexible and scalable model repository that allows us to:
- Track the different versions and variations of models easily.
- Ensure compatibility between model components (e.g., detector, classifier, embedder).
- Manage model access and security, such as ensuring only encrypted models are available for production.
This approach has helped us simplify model management, reduce version control complexity, and streamline the deployment of machine learning models at scale.
Editor’s Note:
Trax uses ClearML’s AI Development Center, a comprehensive, open-source platform designed to accelerate AI/ML development from lab to production. With just two lines of code, developers can seamlessly build, train, and deploy models at scale, leveraging automation, orchestration, and optimized compute resources. The AI Development Center integrates with ClearML’s Infrastructure Control Plane, ensuring secure and efficient workload execution across on-prem, cloud, and hybrid environments. It streamlines AI workflows through collaborative experiment management, federated data handling, and simplified model deployment via CLI, UI, or API. ClearML’s robust MLOps and LLMOps capabilities help teams iterate faster, automate CI/CD pipelines, and maximize efficiency while reducing infrastructure costs. Designed for flexibility, scalability, and enterprise-grade security, the AI Development Center empowers AI builders to focus on innovation, not infrastructure. For more information about ClearML, please contact us or request a demo.