RocksDB is an open-source, non-distributed, highly-performant, key-value store. As a key-value store, it doesn’t itself provide any query-language; it just persistently associates a value (array of bytes) with a key (array of bytes). While a key-value store sounds trivial, implementing one with high performance without losing data is not so easy. RocksDB uses the log-structured merge tree algorithm which is also applied by HBase. This algorithm stores a large data in multiple files and (not surprisingly) merges these files together in a smart way.
A key-value store can be useful directly, or can be used by higher-level applications which provide query language and more structure for the data (eg columns or documents). The API that RocksDB itself offers is similar to the HBase API: GET key, PUT key, DELETE key, SCAN key-range.
RocksDB is not very widely known because it is a library rather than a standalone application, but is embedded in many other interesting projects, including:
- Apache Kafka Streams - for persistent stream state
- Apache Samza - for persistent stream state
- MyRocks - a RocksDB storage engine for MySQL
- CockroachDB (open-core software)
Some interesting points about RocksDB:
- stores data in files that are immutable (except for background merging tasks)
- is particulary SSD-friendly (primarily due to using immutable files - no “write amplification”)
- sorts entries within files by primary key
- has bloom-filters for rowkeys
- has several forms of basic transactionality, ie reads or updates of multiple (key, value) pairs
- is not (itself) distributed (though many distributed storage projects use RocksDB on each node)
- is implemented in C++
- is proven in production (Facebook, LinkedIn, etc)