By Adam Wolf
NERSC, the mission high-performance computing center for the U.S. Department of Energy Office of Science, is using ClearML as part of the AI infrastructure stack for Perlmutter, the upcoming Doudna supercomputer, and the broader American Science Cloud. Here is a look at what they are exploring and why it matters for AI for science at scale.
When AI for science outgrows the script-and-cluster workflow
Most stories about supercomputing focus on hardware: the chips, the racks, the floor space. But the people running the world’s most demanding scientific workloads will tell you that hardware is only part of the story. Once a center is operating thousands of GPUs and supporting thousands of researchers across simulation, AI training, and experimental data analysis, a new layer of complexity emerges: how efficiently and accessibly that compute gets used.
This is the story of how one of the largest open-science computing facilities in the United States has been thinking about that problem, and how it is using ClearML as part of its software stack for the next decade of AI for science.
A primer on NERSC and Berkeley Lab
The National Energy Research Scientific Computing Center (NERSC) is a U.S. Department of Energy Office of Science user facility located at Lawrence Berkeley National Laboratory (Berkeley Lab). It serves as the mission high-performance computing center for the Office of Science, which is the largest funder of physical science research in the country.
A few numbers help convey the scale:
- Approximately 12,642 annual users from 850 institutions across all 50 states, Washington D.C., Puerto Rico, and 46 countries.
- 1,303 projects supported in 2025.
- Over 40 million compute hours used in 2025.
- More than 2,600 refereed publications citing NERSC each year.
- ~520 petabytes of data stored.
- 7,168 GPUs
NERSC’s flagship system today is Perlmutter, an HPE Cray EX supercomputer with NVIDIA A100 GPUs. Its next flagship, Doudna (NERSC-10), is a Dell Technologies system powered by the NVIDIA Vera Rubin platform, scheduled for delivery in late 2026 with early user access planned for 2027. Doudna is designed to provide more than 10x the performance of Perlmutter and to support, in NERSC’s own framing, complex, integrated workflows that span simulation, AI training and inference, and experimental data analysis. NERSC has also taken delivery of the NERSC-10 Pilot Early Access System, called Cech, which uses NVIDIA Grace CPUs and Blackwell GPUs as precursors to the Vera Rubin platform.
The challenge: a workload that no longer looks like batch simulation
A decade ago, an HPC workload was relatively easy to describe. A researcher submitted a simulation job, the scheduler ran it, results came back. Today, NERSC’s workload looks fundamentally different.
According to NERSC’s own workload analysis and ML@NERSC user survey stats, transformer architectures have become a major fraction of the AI work running on the system. The number of unique users importing PyTorch and TensorFlow has grown more than 30x since 2017. Roughly 32% of NERSC projects now have “learning” in their description, up from 4% in 2018. And 37% of survey respondents reported a fully developed ML workflow used in scientific production, with many more refining or developing one.
This shift creates new pressure on the software environment. AI for science workloads tend to couple training, simulation, and experimentation, often in loops. They require hyperparameter sweeps across many GPUs. They train models at parameter scales that look more like commercial foundation model work than traditional HPC. And they need observability, reproducibility, and provenance for publication and peer review.
For a facility that has to provide a productive environment for 12,000 researchers across dozens of scientific domains, none of whom should have to become infrastructure engineers, the gap between “we have GPUs” and “scientists can seamlessly run modern AI workflows on them” is real.
“Our work requires orchestrating AI workloads across HPC environments, including Slurm-managed resources, massive job arrays, LLM serving and agentic workloads, and support for heterogeneous resources across systems such as Perlmutter and the American Science Cloud (AmSC), with future expansion to additional computing resources.”- Steven Farrell, Data & AI Services Lead, NERSC
What NERSC is building toward with Doudna
NERSC’s plans for Doudna include what they describe as advanced workflow capabilities, grouped into five themes: intelligent resourcing, AI, portability, beyond-HPC capabilities, and seamless workflow. The AI theme specifically calls out two goals: seamless end-to-end high-performance AI training workflows, and hybrid simulation, AI, and data workflows. That is essentially what an AI infrastructure platform is supposed to deliver, applied to scientific computing. It is also where the AI infrastructure layer enters the picture.
Training orchestration that fits inside HPC
As discussed above, ClearML is a vertically integrated platform spanning from the orchestration layer, all the way to the actual workloads. The clearest signal of fit is how ClearML slots into the tools NERSC already runs on. ClearML integrates with NERSC’s existing Slurm cluster, allowing researchers to understand resource usage (what’s used and what’s not), send jobs for execution without handling underlying infrastructure, and get a modern AI workbench all without giving up the scheduler the center is built around.
This shows up across the different research activities: parameter sweeps that get placed across nodes through ClearML’s abstraction layer, training jobs, sweeps, and pipelines all visualized in a single platform, and multi-node training workflows using ClearML’s Slurm Glue and Podman containers. Multi-node still involves real configuration work on the platform side, but the researcher does not have to rebuild that plumbing for every project.
From LLM serving to agentic research
NERSC can leverage ClearML’s native LLM provisioning capabilities to deploy LLMs as well as chat UIs to communicate with them. Moreover, NERSC built its own MCP (Model Context Protocol) server for ClearML, which exposes ClearML SDK functionality as a set of tools an agent can call: project and task discovery, metrics and artifacts, workflow execution, and model management. Paired with LangChain, the result is a system where a researcher can describe a workflow in natural language, and an agent retrieves, clones, modifies, and submits ClearML tasks, then monitors them and produces a summary. NERSC did not just adopt ClearML; they’ve extended its agentic capabilities to meet their needs for advanced research.
Why this matters for AI in scientific research
NERSC’s roadmap captures a transition that many organizations running serious AI infrastructure are facing, scaled up to the level of a national lab. The shift from “run a script on a node” to “run a reproducible, observable, multi-step workflow that integrates training, simulation, and inference” is happening everywhere from biotech to autonomous systems. NERSC is doing it on Perlmutter today, planning for it on Doudna, and stitching it into the broader American Science Cloud through programs like the Genesis Mission and the Integrated Research Infrastructure initiative.
ClearML’s role, in NERSC’s hands, is the connective tissue across that stack. It runs training on top of Slurm, scales HPO across the cluster, manages pipeline orchestration, serves LLMs with a chat interface scientists can actually use, and hosts the MCP server that lets a NERSC-built agent drive ClearML itself. That last piece is what separates this from a standard infrastructure deployment: NERSC extended the platform rather than just deploying it.
This is the kind of environment ClearML is built for: Slurm-native orchestration, HPO and pipelines at HPC scale, LLM serving through a UI scientists already know, and the surface area to be extended into something agentic, applied not to a demo but to real workloads on one of the country’s largest open-science systems.
What is next
Doudna‘s early access window opens in 2027, with full user access following. Between now and then, NERSC is continuing to build out the AI infrastructure environment that ClearML is part of, scaling what works on Perlmutter today into the environment that 12,000 researchers will use tomorrow. The open question is not whether the platform works; it is how to scale it to the size, environment, and user base of a national lab.
Those are the same questions any organization running production AI infrastructure should be asking. The answers, in NERSC’s case, will be visible in the science that gets done on Doudna over the next decade, and ClearML will be part of how that science gets done.
Get in touch to discuss how ClearML can support your AI infrastructure requirements.