Features
Flama is open source and installs from PyPI, with native wheels for Python on Linux, macOS and Windows.
One line to a production API
Point Flama at a packaged model and it serves it over HTTP, with routing, validation and error handling already in place.
Any framework, one format
Package scikit-learn, TensorFlow, PyTorch or an LLM into a single portable .flm artifact, so every model looks the same to your API no matter where it came from.
Generative AI serving
Serve LLMs through OpenAI-, Anthropic- and Ollama-compatible endpoints, side by side with your predictive models.
A chat UI with every model
Every served model ships a streaming chat interface at /chat/, rendering Markdown, LaTeX and Mermaid.
Native MCP
Expose tools, resources and prompts to AI agents with a single decorator. The schemas come from your type hints.
A Rust-powered core
Routing, JSON encoding, request parsing and compression compiled to native code, shipped as plain wheels with no Rust toolchain to install.
Capabilities
- Portable .flm model format
- HuggingFace Hub packaging
- OpenAI-compatible endpoints
- Anthropic-compatible endpoints
- Ollama-compatible endpoints
- Streaming chat UI
- Model Context Protocol
- Schema validation
- SQLAlchemy-backed resources
- CLI, Python, spec file or container
- Latest Python
- Linux, macOS and Windows
How it works
Three commands take you from nothing to a running service. flama get downloads a model from the HuggingFace Hub and packages it into a portable .flm file. flama model runs it straight from your terminal, so you can check it behaves before going further. flama serve puts it behind an HTTP API with the chat UI attached. The same thing can be done from Python, from a spec file, or inside a container.
Enterprise
Forward deployed
Getting a framework into production inside an existing organisation is rarely a documentation problem. It is a question of how your models, data and infrastructure actually fit together, so we send engineers to work that out with your team and adapt Flama where your setup needs it.
In your codebase
Our engineers work alongside your team, in your repositories, with your data and your constraints. They learn how your systems actually fit together, which is usually the part no documentation covers.
Adapted where it needs to be
If your setup needs something Flama does not do yet, we build it rather than route you around it. We wrote the framework, so extending it is ordinary work for us instead of a workaround bolted on from outside.
We stay after it ships
The people who set it up are the people you talk to when it needs to change. Models get retrained, traffic shifts and requirements move, and someone who already knows your deployment can absorb that quickly.
Resources
Documentation and guides
Quickstart
Install Flama and go from a packaged model to a running API.
Packaging models
How to turn a trained model from any framework into a portable .flm artifact.
Serving LLMs
Run generative models behind OpenAI-, Anthropic- and Ollama-compatible endpoints.
Model Context Protocol
Expose your tools, resources and prompts to AI agents over MCP.
Flama CLI
Reference for get, model, serve, run, start and upgrade.
Source on GitHub
The framework itself, its issue tracker, and the discussions around it.