By Adam Wolf
Running production LLM inference on a new accelerator family is a layered problem. The model matters. The runtime that exists for the GPU you have matters at least as much. So does the precision mode that works without losing accuracy, the inference engine that hits your throughput targets, and the secure endpoint the rest of your stack can actually call. The entire stack underneath the model is where most of the real engineering work lives and where the cost of getting it wrong shows up first. AMD’s response is the AMD Inference Microservice (AIM), a packaged inference container engineered to make that everything-around-the-model layer disappear. This post unpacks what AIMs actually are, what is distinctive about their design, and how ClearML operationalizes them at the platform layer.
The inference problem that AIMs are built to solve
Production inference involves a long chain of decisions. Which inference engine should run this model on this GPU? What precision mode delivers the right speed and quality tradeoff? How many GPUs should the model occupy? What about batching, KV cache layout, and tensor parallelism? Every one of those decisions is hardware-specific, model-specific, and increasingly version-specific as both models and runtimes evolve. The cost of getting them wrong is real. A misconfigured deployment can leave throughput on the table, blow through GPU memory, or quietly degrade output quality.
AMD’s answer is to package the answers. An AMD Inference Microservice (AIM) is a Docker container that bundles a model and a serving runtime, pre-optimized for AMD Instinct GPUs, with the engine and configuration choices already made and validated. Precision mode is selected at the image level (AMD ships separate AIMs per precision and can auto-detect the best-fitting one for the current setup), and once deployed, the runtime selects engine and runtime parameters based on the hardware the container lands on.
What is inside AIMs
AIMs are part of the open-source AMD Enterprise AI Suite. They run on ROCm 7. They expose an OpenAI-compatible API, which means existing applications and tools that already talk to OpenAI endpoints can talk to an AIM with little to no changes.
The current AIM catalog covers the foundation models most enterprises are deploying today: Meta’s Llama family from 1B parameters up through 405B, DeepSeek’s R1 and V3.1, Mistral’s Ministral, Mixtral, Mistral Small, and Mistral Large, Alibaba’s Qwen3; and Cohere Labs’ Command A Reasoning. AMD extends the catalog with each Enterprise AI Suite release.
The design choice that matters: Profiles
The most interesting thing about AIMs, from an enterprise operations perspective, is not that they ship as containers. It is that they configure themselves through profiles.
A profile is a tested, validated combination of GPU, precision mode (typically fp16 or fp8), inference engine, and optimization target (throughput or latency). When you deploy an AIM, the runtime picks the profile that matches the hardware the container has landed on. If you want to override the automatic selection, you can specify a profile directly through an environment variable. Every published profile is validated and tested on the target hardware.
What that does is move a category of decisions out of the deployment process and into the container itself. The platform team that runs the inference service no longer has to know which engine or which tensor parallel configuration to use for a given model on a given GPU. AMD has done that work. What remains is the platform layer above the container: how AIMs get scheduled, scaled, secured, routed, and governed across a real production fleet. That is where ClearML comes in.
Where ClearML fits
ClearML has been building deep integration with AMD for some time, across both the Instinct accelerator family and the ROCm software ecosystem. The same control plane that already orchestrates AMD training workloads orchestrates AMD inference workloads, and AIMs sit on top of that foundation cleanly.
Because AIMs are Docker containers with OpenAI-compatible APIs, they fit ClearML’s container-based deployment model directly. Through ClearML, you can deploy an AIM container onto the AMD Instinct resources that ClearML manages, whether they are on bare metal, in a VM, or inside a Kubernetes cluster, and expose its endpoint through ClearML’s App Gateway.
What ClearML adds at that point is everything that turns a running container into an enterprise-grade inference service:
- Endpoint networking. Managed ingress and egress for every deployed endpoint.
- Identity-aware access control. RBAC rules at the endpoint level, with tenant-aware scoping.
- Authentication. Endpoint access is secured against the same identity system the rest of the platform uses.
- Autoscaling. Horizontal scaling driven by demand, not by a static manifest.
- Multi-tenancy. Shared AMD Instinct-based infrastructure that isolates tenants from each other at the platform layer.
- Observability. Continuous monitoring of every deployed endpoint, surfaced in a single dashboard.
- Resource scheduling. Integration with AMD’s compute and memory partitioning so AIMs can land on fractional GPUs when full-GPU exclusivity is not needed.
None of that is unique to AIMs. It is the platform layer that any production inference service needs, applied here to AMD’s pre-packaged inference containers.
Two layers of optimization, one operational result
The combination is worth thinking about as two layers of optimization. AMD’s AIMs handle the optimization that lives inside the container: the right engine, the right precision, and the right configuration for the hardware. ClearML handles the optimization that lives outside the container: access controls, autoscaling behavior, multi-tenant boundaries, and observability. Neither layer replaces the other, and neither is interesting on its own. Together, they remove most of the work between “I want to serve Llama 3.3 70B on our AMD cluster” and “the application team can call a stable, governed endpoint and stop worrying about it.”
If you would like to see how AIMs and ClearML fit together on your AMD infrastructure, request a demo.
About ClearML
ClearML is the infrastructure platform for unleashing AI in organizations worldwide. More than 1,600 customers use ClearML to manage GPU clusters, optimize utilization, streamline AI/ML workflows, and deploy GenAI models. ClearML is silicon-agnostic, with integrations across the AMD Instinct accelerator family, the ROCm software ecosystem, and other accelerated computing platforms. We are trusted by more than 250,000 AI builders and IT teams at Fortune 500 companies, enterprises, academia, public sector agencies, and start-ups worldwide.