Open Source Machine Learning Infrastructure using Node.js

A curated collection of the best free and open source runtimes, servers and observability for machine learning models, including everything needed to run one on your own hardware using Node.js.

Ollama

A local runner for open models, offering a CLI, a REST API and one-command launching of coding agents on your own machine.

Alternative to:

LM Studio

Run open-source LLMs locally on your own machine

Stars
185,051
Forks
18,667
Last commit
20 hours ago

llama.cpp

A dependency-free C and C++ inference engine that runs language and vision models locally across CPUs, GPUs and specialised accelerators.

Alternative to:

LM Studio

Run LLMs locally with minimal setup, maximum hardware support

Stars
132,584

LocalAI

A self-hosted AI engine that runs text, vision, voice, image and video models on any hardware, with no GPU required.

Alternative to:

OpenRouter

Self-hosted AI runtime for text, voice, vision, and agents

Stars
50,014

Jan

Desktop app for downloading and running open source LLMs locally, with optional cloud model access and an OpenAI compatible local server.

Alternative to:

ChatGPT

Run AI models locally or connect to cloud, privately

Stars
45,123

Open-Notebook

A self-hosted research notebook and Notebook LM alternative for people who want to pick their own AI models and keep sources private.

Alternative to:

NotebookLM

AI-powered research notebook with full privacy control

Stars
40,648

Langfuse

LLM engineering platform where teams trace, evaluate, debug and version-control the prompts and runs behind an AI application.

Alternative to:

LangSmith

Open source LLM engineering platform for AI-powered applications

Stars
36,624

Arize Phoenix

An open source AI observability platform for engineers who need to trace, evaluate, and debug LLM applications.

Alternative to:

LangSmith

Open-source LLM tracing & evaluation for AI optimization

Stars
12,038

OpenLLMetry

OpenTelemetry extensions that trace LLM applications, for teams who want model calls in the observability stack they already run.

Alternative to:

LangSmith

Monitor LLM performance with open-source observability

Stars
7,512

Helicone

An AI gateway and observability platform that logs LLM requests, tracks cost and latency, and routes across providers for AI engineers.

Alternative to:

LangSmith

Build reliable AI apps with comprehensive observability

Stars
6,242

Agenta

Open-source workspace where teams build agents by chatting with them, connect them to apps, and run them in the background.

Alternative to:

Lindy

Build, automate, and improve AI agents with your team

Stars
4,866

Latitude

Observability platform for AI agents that traces every run, groups failures into tracked signals, and dispatches coding agents to fix them.

Alternative to:

LangSmith

Full observability for AI agents in production

Stars
4,738

LangWatch

Platform for testing, simulating, evaluating and monitoring LLM-powered agents, aimed at teams needing regression testing and production observability.

Alternative to:

LangSmith

Simulation-based testing and evaluation for AI agents

Stars
4,996

OpenLIT

An OpenTelemetry-native platform for AI engineering, covering observability, evaluations, prompt versioning and secret storage for LLM applications.

Alternative to:

LangSmith

Monitor, debug, and scale LLM applications with ease

Stars
2,890

Deepnote

Open source data notebook that swaps the ipynb format for a readable YAML file and runs inside VS Code, Cursor or Windsurf.

Alternative to:

Mode

Python, SQL, and AI in a collaborative data notebook

Stars
3,025

dstack

Control plane for GPU provisioning and orchestration across clouds, Kubernetes and on-prem clusters, for teams training and serving models.

Alternative to:

Modal

GPU orchestration across clouds, Kubernetes, and on-prem

Stars
2,292

mlop

Self-hostable experiment tracking for machine learning engineers who want high logging throughput from about five lines of Python.

Alternative to:

Weights & Biases

Experiment tracking platform for ML teams

Stars
394

Spanlens

LLM observability that logs cost, tokens, latency and agent traces across eleven providers, for developers who need per-feature spend visibility.

Alternative to:

LangSmith

LLM observability: cost, latency, and traces in one place

Stars
13

Grepture

An LLM proxy that redacts PII, blocks sensitive content and tokenizes fields before requests reach a provider, for teams with data rules.

Alternative to:

LangSmith

Trace, evaluate, and protect every LLM call in one SDK

Stars
12

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Open Source Machine Learning Infrastructure using Node.js

A curated collection of the best free and open source runtimes, servers and observability for machine learning models, including everything needed to run one on your own hardware using Node.js.

Ollama

llama.cpp

LocalAI

Jan

Open-Notebook

Langfuse

Arize Phoenix

OpenLLMetry

Helicone

Agenta

Latitude

LangWatch

OpenLIT

Deepnote

dstack

mlop

Spanlens

Grepture

Ollama

llama.cpp

LocalAI

Jan

Open-Notebook

Langfuse

Arize Phoenix

OpenLLMetry

Helicone

Agenta

Latitude

LangWatch

OpenLIT

Deepnote

dstack

mlop

Spanlens

Grepture

GPT4All