LocalAI

LocalAI runs models on hardware you already own and speaks the APIs your code already calls. Text, vision, voice, image and video all sit behind a single endpoint, and a GPU is optional rather than assumed, which removes the usual reason for sending private data to somebody else's server.

The design is a small core rather than a bundle. Each backend wraps an established engine such as llama.cpp, vLLM, whisper.cpp, stable-diffusion or MLX in its own image, pulled only when a model needs it, so you install nothing you do not use. LocalAI detects your GPU capability and fetches the matching backend on its own.

Several things follow from that structure.

• Drop-in API compatibility: the OpenAI, Anthropic and ElevenLabs APIs are served across every backend.
• Any hardware: NVIDIA, AMD, Intel, Apple Silicon, Vulkan and CPU-only each have their own container tag.
• Model sources: run one from the gallery, Hugging Face, the Ollama registry, a YAML config or any OCI registry.
• Multi-user ready: API key authentication, per-user quotas and role-based access come built in.
• Built-in agents: autonomous agents with tool use, RAG, MCP and skills ship with the engine.
• Terminal agent: a chat client reads your files and runs commands, asking approval before anything changes state.

macOS has a disk image download, unsigned by Apple, that needs a quarantine attribute cleared after install; everywhere else a single docker run starts the server.