Quickwit

Quickwit is a search engine for observability data, logs and traces today with metrics on the roadmap. Its distinguishing decision is architectural. The index lives on cloud object storage and search runs against it directly, instead of against a fleet of stateful nodes holding local disks.

Getting there meant rebuilding the index data structures and IO paths. Compute and storage are decoupled, indexers and searchers are stateless, and queries return in under a second against Amazon S3, Azure Blob Storage or Google Cloud Storage. The project estimates this runs up to ten times cheaper than Elastic.

Compatibility is where Quickwit tries hardest to be undisruptive.

• Elasticsearch API: a large subset of the query DSL, endpoints and aggregations, usable from existing clients.
• OTEL native: OpenTelemetry ingestion for both logs and traces with no translation layer in front.
• Jaeger native: traces plug straight into Jaeger for distributed tracing workflows.
• Grafana data source: a dedicated plugin for querying Quickwit from existing dashboards.
• Schemaless or strict: index without declaring a schema or enforce one, with schemaless analytics either way.

Operationally it ingests natively from Kafka, Kinesis and Pulsar, supports multi-tenancy through many indexes and partitioning, applies retention policies, and runs delete tasks for GDPR requests. A Helm chart covers Kubernetes and everything is reachable over a REST API.

Migrating off Elasticsearch or OpenSearch is eased by an ES-compatible ingest API, so shippers such as Vector, Fluent Bit and Syslog keep working. A client that balks at a version check is satisfied with extra headers in the node configuration.