Open Source AVEVA PI System Alternatives

TDengine stores the data that billions of sensors, meters and vehicles produce every second. Generic databases buckle under that shape of workload, where writes never stop, queries are almost always bounded by time, and distinct collection points run into the billions.

The engine is written in C and organizes readings through super tables, partitioning by time interval and separating storage from compute. Caching, stream processing, data subscription and an AI agent named TDgpt are built into the same server, so the stack needs no separate cache, queue and stream processor alongside it.

Several parts of the design are worth knowing before you commit to it.

• High cardinality: the engine addresses the cardinality limit that constrains other time series databases, supporting billions of data collection points.
• Cloud native distribution: sharding, partitioning, RAFT and Kubernetes deployment let one cluster run on public, private or hybrid infrastructure.
• TDgpt: the built in AI agent connects time series foundation models, large language models and classical algorithms for forecasting, anomaly detection, imputation and classification.
• Simplified operations: administrators deploy and maintain less machinery, while developers get a simple interface and integrations with third party tools.
• Open core: the cluster feature and the AI agent are released under open source licenses, with over 730,000 instances reported running.

Builds are tested primarily on Linux, with macOS supported for local development and Windows limited in the open source tree. Install from a container, a package or Kubernetes, or build from source with CMake 3.21. Four gigabytes of RAM and two of free disk are the recommended floor.

One SQL engine spans ingestion, stream processing and tiered storage in QuestDB, a low-latency time-series database built on open formats. That removes the usual split between a hot store for recent events and somewhere else for everything older.

The engine is zero-garbage-collection Java with C++ and Rust on the hot paths, and data lives in memory-mapped, time-partitioned columns. Queries run across all cores with SIMD and JIT-compiled filters, and nothing third-party sits on the data path. Storage is tiered across a parallel write-ahead log, native columnar partitions and Apache Parquet.

One engine covers the whole life of a series.

• Capture: ordered and out-of-order events in the millions per second, with deduplication and out-of-order correction built in.
• Compute: materialized views that aggregate by time slice, and live views running window functions on an in-memory tier.
• Query: time-series SQL extensions including SAMPLE BY, LATEST ON, ASOF JOIN, WINDOW JOIN and n-dimensional arrays.
• Retain: native partitions and Parquet queried through the same SQL, with Iceberg and Arrow keeping the data open to other tools.
• QWP: a binary columnar protocol over WebSocket that writes rows in and streams results back out as Arrow columns.

Docker starts an instance in one command, and Homebrew covers Apple silicon, though the bundled native libraries mean Intel Macs should use the image instead. Published benchmarks measure 19 million rows per second on ingestion and 220 million rows per second streamed into Arrow.

Collection, storage and analysis of time series data sit in one system with Apache IoTDB, a database shaped by industrial internet of things workloads, where millions of low-power devices write continuously and the accumulated dataset still has to answer complex queries.

Storage builds on TsFile, a columnar file format designed for time series, which is how the project reaches a high compression ratio on disk and keeps hardware cost down. The system runs on Windows, macOS and Linux and requires Java 17 or newer, with versions up to 25 verified.

The design answers a handful of recurring industrial problems.

• Flexible deployment: a one-click installer for cloud platforms or terminal devices, plus a tool that synchronizes data between the two.
• Directory structure: time series from complex device hierarchies organize efficiently, with fuzzy search across a large catalog of series.
• Query semantics: time alignment across devices and measurements, frequency domain transformation and aggregation over the time dimension.
• Familiar interfaces: a SQL-like language, the standard JDBC API, a command-line client and CSV import and export tools.
• Ecosystem integration: Hadoop and Spark for analysis, Grafana for visualization.

Before starting a node, raise the maximum open file count to 65535 and, on Linux, set somaxconn to the same value, or the server may fail to start or log connection resets. Docker users apply those limits on the host machine. Building from source additionally needs Maven 3.6 or later. The project is developed at the Apache Software Foundation, with issues tracked in Jira.