Trial
Update 07/2026 (assess → trial): Docker Model Runner is generally available since October 2025.
- Vulkan support widens GPU coverage beyond NVIDIA/Apple Silicon.
- Models are pulled as OCI artifacts from Docker Hub or Hugging Face - model distribution fits into existing container registries and supply-chain tooling.
For teams already standardized on Docker this is now a serious alternative to Ollama for local inference and CI use cases.
Updated link: Docker Model Runner documentation
Assess
Docker has released Model Runner, a CLI tool that standardizes local inference of AI models. (Any maybe also for production in the future)
With a single docker model run <<modelname>> "Hi" command, developers can launch a model as simple as launching a container.
Simelar to Ollama, it uses llama.cpp
Model Runner is ideal for:
- Local testing before deployment.
- Working with OpenAI-compatible API.