this refers to RocksDB.

Yahoo! Cloud Serving Benchmark (YCSB) is popular method to benchmark database, but now, this don’t supported RocksDB.

I hvae looked for new method to benchmark RocksDB. and I decided to arrange information related to RocksDB.

the content is as follows.

1-1. What is RocksDB??

- RocksDB is an embeddable persistent key-value store for fast storage.
- RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fase storage, 
to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation

For more background on RocksDB, see Dhruba Borthakur’s introductory talk from the Data @ Scale 2013 conference.

  • write amplication

    • SSD은 overwrite가 호스트에서 쓰기 작업을 요청을 할때 SSD에서 추가적인 작업이 진행이 된다. 그 작업은 garbage collection, 새로운 장소에 쓰고, 지우는 과정과 같은 작업이 내부적으로 늘어나는 이러한 현상을 write amplication라고 한다.

the following is settings to test RocksDB’s performance on facebook

All of the benchmarks are run on the same machine. Here are the details of the test setup:

    12 CPUs, HT enabled -> 24 vCPUs reported, 2 sockets X 6 cores/socket with X5650 @ 2.67GHz
    
    2 FusionIO devices in SW RAID 0 that can do ~200k 4kb read/second at peak
    
    Fusion IO devices were about 50% to 70% full for each of the benchmark runs
    
    Machine has 144 GB of RAM
    
    Operating System Linux 2.6.38.4
    
    1G rocksdb block cache
    
    1 Billion keys; each key is of size 10 bytes, each value is of size 800 bytes
    
    total database size is 800GB, stored on XFS filesystem with TRIM support
    
    jemalloc memory allocator

The following benchmark results compare the performance of rocksdb compared to leveldb. This is an IO bound workload where the database is 800GB while the machine has only 144GB of RAM. These results are obtained with a release build of db_bench created via make release.

the following refers to infulxdata

위의 글을 봐도 요즘 성능 이슈로 인해 거의 database는 in memory 방식으로 하여 데이터를 저장 및 갱신, 삭제를 하는 것 같다.

아래 글을 보자.

RocksDB가 어떻게 reboot시 data를 보존을 하는 지는 이 사이트에 잘 나와 았다.

how to persisit in-memory RocksDB database이 사이틀 참조한 내용이다.

Every update to RocksDB is written to two places – one is an in-memory data structure called memtable and second is write-ahead log. Write-ahead log can be used to completely recover the data in memtable. By default, when we flush the memtable to table file, we also delete the current log, since we don’t need it anymore for recovery (the data from the log is “persisted” in the table file — we say that the log file is obsolete). However, if your table file is stored in in-memory file system, you may need the obsolete write-ahead log to recover the data after the machine reboots. Here’s how you can do that.