“We hope that software programmers and database
developers will use, enhance, and customise RocksDB for their own use
cases. We’re looking forward to hearing your feedback and continuing to
improve RocksDB and add more features,” he said.
The RocksDB tool is a key-value store that builds on the LevelDB,
released by Google. Facebook said it has taken this and extended its
capabilities to help it store and serve data direct from its servers.
This helps improve the speed with which information is served to users.
“Applications traditionally access their data
via remote procedure calls over a network connection, but that can be
slow – especially when we need to power user-facing products in real
time,” explained Borthakur.
“With the advent of flash storage, we are
starting to see newer applications that can access data quickly by
managing their own dataset on flash instead of accessing data over a
network. These new applications are using what we call an embedded
database.”
Borthakur said moving to the use of embedded
databases had several benefits. “When database requests are frequently
served from memory or from very fast flash storage, network latency can
slow the query response time," he said.
“Accessing the network within a data center can
take about 50 microseconds, as can fast-flash access latency. This means
that accessing data over a network could potentially be twice as slow
as an application accessing data locally.”
Facebook said it had to build on LevelDB as the
Google platform did not work well for the scale of input/output
operations per second (IOPS) on the site.
"We found that LevelDB's single-threaded
compaction process did not work well for certain server workloads, and
that RocksDB outperforms LevelDB for these IO-bound workloads. We saw
that applications were not able to derive the full potential of the
flash hardware because of data bandwidth bottleneck[s] caused by
LevelDB's high write amplification," Borthakur wrote.
"By increasing write rate and lowering write
amplification, we were able to avoid many of these issues and improve
performance with RocksDB."
Facebook said with the improvement it has made,
RocksDB can be used for a variety of key functions, such as storing and
retrieving user history on a service to present relevant content (as
Facebook does), spam detection, graph searches and Hadoop-based queries.
The move is yet another example of Facebook taking its technologies open source, as it continues to push its Open Compute Project, which focuses on improving the efficiencies of servers used in data centres.
Earlier this week the firm urged European firms
to back this goal, claiming so far interest and input has been far
behind that seen in the US and Asia.
No comments:
Post a Comment