Module riak_ensemble_storage

This module implements a central storage manager for riak_ensemble.

Behaviours: gen_server.

Description

This module implements a central storage manager for riak_ensemble. Previously, individual ensembles as well as the ensemble manager would independently save their own state to disk. However, such an approach scaled poorly as the number of independent ensembles increased. It was not uncommon to see thousands of synchronous writes issued to disk per second, overwhelming the I/O subsystem. To solve this issue, this storage manager was created.

Rather than storing data independently, the storage manager combines the state from multiple ensembles as well as the ensemble manager into a single entity that is stored together in a single file. Since this file is now a critical single point of failure, the storage manager uses the new riak_ensemble_save logic to save this data to disk such that there are four redundant copies to recover from.

This manager is also responsible for coalescing multiple writes together to reduce disk traffic. Individual writes are staged in an ETS table and then flushed to disk after a delay (eg. 50ms).

There are two ways to save data to disk that are used by other components in riak_ensemble: synchronous and asynchronous.

For synchronous writes components use the sequence: riak_ensemble_storage:put(Key, Data), riak_ensemble_storage:sync(). The sync() call than blocks until the data has successfully been written, to disk.

For asynchronous writes, components simply use put() without sync(). The data will then be written to disk either when another component calls sync, or after next storage manager tick (eg. every 5 seconds).

Function Index

code_change/3
get/1
handle_call/3
handle_cast/2
handle_info/2
init/1
put/2
start_link/0
sync/0
terminate/2

Function Details

code_change/3

code_change(OldVsn, State, Extra) -> any()

get/1

get(Key::term()) -> {ok, term()} | not_found

handle_call/3

handle_call(Request, From, State) -> any()

handle_cast/2

handle_cast(Msg, State) -> any()

handle_info/2

handle_info(Info, State) -> any()

init/1

init(X1) -> any()

put/2

put(Key::term(), Value::term()) -> true

start_link/0

start_link() -> any()

sync/0

sync() -> ok

terminate/2

terminate(Reason, State) -> any()


Generated by EDoc