A favicon of SlateDB

SlateDB

Embedded key-value storage engine in Rust that writes its log-structured merge-tree straight to object storage instead of local disk.

Screenshot of SlateDB websiteVisit SlateDB

SlateDB is an embedded storage engine built as a log-structured merge-tree. What separates it from a traditional LSM engine is where the data lands: object storage such as S3, GCS or MinIO rather than a local disk, which buys bottomless capacity, high durability and replication that comes for free.

The trade-off is latency and per-request cost, and the design answers both directly. Writes are batched, so a single write does not become a single request; MemTables flush to object storage as sorted string tables on a configurable interval, and each write returns a handle you can await for durability or flush explicitly. Reads lean on familiar LSM techniques, in-memory block caches, compression, bloom filters and local disk caches for tables.

The engine covers more ground than a minimal key-value store.

  • Core API: get, put, delete, plus range scans and seeks over bound and unbound ranges.
  • Transactions and merge operators: both implemented, alongside clones and database split and merge.
  • Change data capture: changes can be tracked as they are written.
  • Any object store: it builds on the object_store crate and re-exports it, so anything implementing that trait works.
  • Bindings: official Go, Java, Node.js and Python bindings, with community ones for .NET and Ruby.

Adding the crate next to tokio is enough to open a database from Rust. Releases follow semantic versioning roughly every two months, with storage format compatibility guaranteed between adjacent versions, though the compile-time API may still break. SlateDB is a Commonhaus Foundation member and already in production at Dropbox, Prisma, Gadget and others.

Categories:
Share:
Details:

Auto-fetched from GitHub .

Ad

 

 
 
 
 

Open source alternatives similar to SlateDB:

 

 
 
  • Stars


  • Forks


  • Last commit


 

 
 
  • Stars


  • Forks


  • Last commit


 

 
 
  • Stars


  • Forks


  • Last commit


SlateDB: Embedded storage engine built on object storage – Unvendored