By Adam Wolf
Dell Technologies has published a validated integration of ClearML with the Dell AI Data Platform (AIDP), pairing ClearML’s AI infrastructure capabilities with Dell’s enterprise-managed storage and search engines. The result is a reference architecture that lets AI teams keep moving fast while platform teams keep the data foundation enterprise-grade. Here is what the integration does, why it matters, and where it fits.
ClearML is an enterprise-ready AI platform. It is used by hundreds of organizations that self-host and manages it by themselves. And while ClearML is a complete-package, fully integrated with databases for persistency and an internal storage solution, the burden of management lays on the IT, Devops or AI team that deployed it in the first place. The interesting question at enterprise scale is not whether ClearML works; it is who carries the operational load of the data infrastructure underneath: the search indexing, the replication, the high availability, the backups, the access controls on storage and indices, and the compliance posture that wraps all of it. By default, it is handled by the platform admins, which means their responsibilities extend from management of the platform to management of underlying storage layers. Dell Technologies recently published a validated integration that addresses exactly this. It validates ClearML running on the Dell AI Data Platform (AIDP), with Dell’s managed engines underneath the ClearML control plane. The integration is interesting precisely because it does not add new AI capabilities. What it adds is a lot less for your AI team to worry about.
What Each Side Brings
ClearML provides the AI infrastructure layer: orchestration of training and inference workloads, experiment tracking, dataset versioning, model lifecycle management, compute governance, and the policies that make all of it auditable. It is the operational control plane that AI teams interact with every day.
Dell AIDP provides the enterprise data foundation underneath: storage that scales into petabytes, a search engine designed for billions of objects, governance and access controls aligned to enterprise compliance frameworks, and the engineering rigor that comes with infrastructure Dell has been shipping for decades.
The validated integration plugs them together. ClearML keeps doing what it does best. AIDP carries the storage and search load.
How the Architecture Decomposes
ClearML Server is the centralized control plane for the platform. It runs the backend services, the UI, and the metadata layer for experiments, models, datasets, and infrastructure state. By default, the server depends on three components storing state and metadata: MongoDB for structured state, Elasticsearch for search and analytics over experiment metadata and time series data, and a built-in file server for artifacts like models, datasets, and debug images.
In the Dell-validated configuration, two of those three change. MongoDB stays where it is, handling task state, configuration, and the relationships between entities. The other two get externalized:
- Dell Data Search Engine (DDSE) replaces the bundled Elasticsearch and becomes the indexing and search layer for experiment metadata, parameters, scalar metrics, and execution logs. DDSE is the Dell Data Engine developed in collaboration with Elastic, designed to scale search across billions of files with enterprise-grade indexing.
- Dell AIDP S3A-compatible object storage replaces the built-in ClearML file server and becomes the durable system of record for models, datasets, artifacts, plots, and debug samples. This is the Dell storage engine layer, typically backed by Dell ObjectScale or Dell PowerScale.
The architectural shift is what the white paper calls “separating intelligence from persistence.” ClearML manages the workflow logic and the experiment intelligence. AIDP carries the data weight underneath. ClearML’s documentation explicitly covers Dell PowerScale as a supported S3-compatible storage backend, and the new Dell Data Search Engine brings the same level of integration to the search layer.
Why This Matters for Enterprise AI Teams
There is a moment in the life of any AI platform where the team running it stops being able to treat infrastructure as background. Storage capacity becomes a planning conversation. Search performance becomes a complaint queue. Backup and recovery become a board-level question. The pattern is predictable, and most platform teams discover it at the worst possible time.
The Dell integration matters because it lets organizations avoid that moment by separating the two halves of the problem. The team running AI workflows on top of ClearML keeps a familiar interface. The platform team running the underlying data infrastructure gets the storage, search, and governance tooling they already trust at the rest of the enterprise stack.
A few concrete payoffs the integration enables:
- Search that scales independently. DDSE handles experiment metadata indexing without coupling search performance to whatever else is happening on the ClearML server. Index growth, sharding, and retention become a data infrastructure concern handled by the team and tooling already running Dell’s data engines, decoupled from the AI workflow itself.
- Storage that scales independently. Object storage capacity and throughput grow on their own curve, separate from compute and metadata services. Petabyte-scale dataset catalogs do not force you to resize the ClearML server.
- Governance that lives at the storage layer. Bucket and prefix-level access controls, audit logs, and policy enforcement all happen in AIDP, which already supports the compliance frameworks the rest of the enterprise relies on. ClearML’s project-based and group-based access controls layer on top, giving you defense in depth across both planes.
- A familiar ClearML experience on top. Users still create experiments, version datasets, run pipelines, manage models, and serve endpoints through the same ClearML interfaces. The fact that artifacts are landing in Dell ObjectScale and metadata is being indexed by DDSE is invisible to the people doing the work.
Storage Management Best Practices
The solution doesn’t cover only the integration benefits but also offers data management best practices. A few worth highlighting:
- Keep development, validation, and production isolated at both layers. Distinct storage buckets, distinct DDSE indices, distinct ClearML projects. A misconfiguration in dev should never reach prod, which is the same principle ClearML enforces.
- Plan for index growth. As experiment volume grows, monitor DDSE index size, adjust sharding and retention policies, and keep logical separation between projects, teams, or business units to support governance boundaries.
- Set storage lifecycle policies. AI artifacts accumulate fast. Archive or expire obsolete models and intermediate outputs based on project status, age, or usage frequency. Use versioning and immutability where compliance requires it.
None of this is unique to Dell or ClearML, but it is the operational maturity any enterprise AI platform needs. What the integration provides is a reference architecture where the tooling on both sides truly supports these practices, rather than getting in their way.
Where This Fits
The Dell AI Data Platform is part of Dell’s broader AI Factory strategy, a portfolio that spans infrastructure, software, and services designed to move AI from pilots to production. Dell positions AIDP as the data foundation layer of that stack. The ClearML integration extends the foundation upward into the AI infrastructure layer, where teams orchestrate workloads, track experiments, govern compute, and serve models.
For organizations already running Dell storage and looking at scaling their AI operations, this is a way to bring AI workflows onto infrastructure their platform teams already operate. For organizations already running ClearML and looking at enterprise data foundations, this is a way to get the storage and search engineering they need without rebuilding what they already have on the AI side.
In neither direction does anyone have to compromise on what they came for.
“By replacing application-embedded backend services with Dell AIDP engines, the solution demonstrates how organizations can decouple AI workflow orchestration from data persistence and metadata intelligence, while maintaining performance, governance, and operational consistency.”
Closing
The interesting question for any platform pairing is not whether the components technically work together. It is whether the combination addresses a problem the people running it actually have. The ClearML and Dell AIDP integration addresses a specific, recognizable problem: the moment when managing the data infrastructure underneath an AI platform should be decoupled from running the AI platform itself and handed to the teams and tools already running enterprise data infrastructure for the rest of the business. Solving that as a deliberate architectural pattern, rather than as a retrofit later, is the difference between a platform that grows with you and one that quietly accumulates engineering debt on the way there.
Learn More and Get in Touch
For the full technical detail, including configuration steps, validation output, and operational considerations, read the Dell Technologies white paper directly. If you would like to discuss how ClearML can fit into your enterprise AI infrastructure, whether on Dell AIDP or another data foundation, get in touch.