SeekStorm

SeekStorm ships in two shapes, an in-process library you embed and a multi-tenancy server you run, both written in Rust. Development started in 2015, the engine went into production in 2020, was ported to Rust in 2023, and was open sourced in 2024. It remains work in progress.

Rather than bending one index to do two jobs, SeekStorm runs two native index architectures under a single query planner. An inverted index handles lexical relevance, an approximate nearest neighbor index handles vector similarity, and results are combined with reciprocal rank fusion. Callers choose lexical, vector, or hybrid search per query and are shielded from the split entirely.

Capabilities span both index types.

• Lexical ranking: BM25F and BM25F proximity scoring, six tokenizers including Chinese word segmentation, and stemming for 38 languages.
• Vector indexing: multiple vectors per field and per chunk, with embeddings generated in process through Model2Vec or imported from outside.
• Real-time search: indexing is incremental and new documents are searchable immediately, with negligible performance impact.
• Query features: faceted counting and filtering, geo proximity, typo tolerance, synonyms, autocompletion, and highlighted snippets.
• Hardware acceleration: SIMD support on x86-64 and on AArch64, which covers Apple Silicon.

The server exposes a REST API with CORS, ingests CSV, JSON, newline-delimited JSON and PDF files by console command, and manages indices and API keys per tenant. REST clients exist for Rust, Python, TypeScript, C# and Java, and an adapter redirects an existing Algolia InstantSearch.js frontend at a SeekStorm backend with no UI rewrite.