Open Source Machine Learning Infrastructure using GitHub

A curated collection of the best free and open source runtimes, servers and observability for machine learning models, including everything needed to run one on your own hardware using GitHub.

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Ollama

A local runner for open models, offering a CLI, a REST API and one-command launching of coding agents on your own machine.

Alternative to:

Run open-source LLMs locally on your own machine

Stars
185,051
Forks
18,667
Last commit
20 hours ago

Jan

Desktop app for downloading and running open source LLMs locally, with optional cloud model access and an OpenAI compatible local server.

Alternative to:

Run AI models locally or connect to cloud, privately

Stars
45,123

Ad

OpenLLMetry

OpenTelemetry extensions that trace LLM applications, for teams who want model calls in the observability stack they already run.

Alternative to:

Monitor LLM performance with open-source observability

Stars
7,512

Helicone

An AI gateway and observability platform that logs LLM requests, tracks cost and latency, and routes across providers for AI engineers.

Alternative to:

Build reliable AI apps with comprehensive observability

Stars
6,242

Latitude

Observability platform for AI agents that traces every run, groups failures into tracked signals, and dispatches coding agents to fix them.

Alternative to:

Full observability for AI agents in production

Stars
4,738

LangWatch

Platform for testing, simulating, evaluating and monitoring LLM-powered agents, aimed at teams needing regression testing and production observability.

Alternative to:

Simulation-based testing and evaluation for AI agents

Stars
4,996

Deepnote

Open source data notebook that swaps the ipynb format for a readable YAML file and runs inside VS Code, Cursor or Windsurf.

Alternative to:

Mode

Python, SQL, and AI in a collaborative data notebook

Stars
3,025

Spanlens

LLM observability that logs cost, tokens, latency and agent traces across eleven providers, for developers who need per-feature spend visibility.

Alternative to:

LLM observability: cost, latency, and traces in one place

Stars
13

Grepture

An LLM proxy that redacts PII, blocks sensitive content and tokenizes fields before requests reach a provider, for teams with data rules.

Alternative to:

Trace, evaluate, and protect every LLM call in one SDK

Stars
12