Open Source GE Proficy Historian Alternatives

TDengine stores the data that billions of sensors, meters and vehicles produce every second. Generic databases buckle under that shape of workload, where writes never stop, queries are almost always bounded by time, and distinct collection points run into the billions.

The engine is written in C and organizes readings through super tables, partitioning by time interval and separating storage from compute. Caching, stream processing, data subscription and an AI agent named TDgpt are built into the same server, so the stack needs no separate cache, queue and stream processor alongside it.

Several parts of the design are worth knowing before you commit to it.

• High cardinality: the engine addresses the cardinality limit that constrains other time series databases, supporting billions of data collection points.
• Cloud native distribution: sharding, partitioning, RAFT and Kubernetes deployment let one cluster run on public, private or hybrid infrastructure.
• TDgpt: the built in AI agent connects time series foundation models, large language models and classical algorithms for forecasting, anomaly detection, imputation and classification.
• Simplified operations: administrators deploy and maintain less machinery, while developers get a simple interface and integrations with third party tools.
• Open core: the cluster feature and the AI agent are released under open source licenses, with over 730,000 instances reported running.

Builds are tested primarily on Linux, with macOS supported for local development and Windows limited in the open source tree. Install from a container, a package or Kubernetes, or build from source with CMake 3.21. Four gigabytes of RAM and two of free disk are the recommended floor.

Collection, storage and analysis of time series data sit in one system with Apache IoTDB, a database shaped by industrial internet of things workloads, where millions of low-power devices write continuously and the accumulated dataset still has to answer complex queries.

Storage builds on TsFile, a columnar file format designed for time series, which is how the project reaches a high compression ratio on disk and keeps hardware cost down. The system runs on Windows, macOS and Linux and requires Java 17 or newer, with versions up to 25 verified.

The design answers a handful of recurring industrial problems.

• Flexible deployment: a one-click installer for cloud platforms or terminal devices, plus a tool that synchronizes data between the two.
• Directory structure: time series from complex device hierarchies organize efficiently, with fuzzy search across a large catalog of series.
• Query semantics: time alignment across devices and measurements, frequency domain transformation and aggregation over the time dimension.
• Familiar interfaces: a SQL-like language, the standard JDBC API, a command-line client and CSV import and export tools.
• Ecosystem integration: Hadoop and Spark for analysis, Grafana for visualization.

Before starting a node, raise the maximum open file count to 65535 and, on Linux, set somaxconn to the same value, or the server may fail to start or log connection resets. Docker users apply those limits on the host machine. Building from source additionally needs Maven 3.6 or later. The project is developed at the Apache Software Foundation, with issues tracked in Jira.