rafter (in progress) is a pipelined asynchronous multi-raft group library with an in-memory key-value replicated state machine based on seastar.
rafter is deeply inspired by
- prerequisite
- Compiler with C++20 support, tested with clang-12
- Seastar, the installation of Seastar is not trivial, please refer to this post
- GoogleTest, taken care of by rafter's cmake configuration
TBD
- finalize all modules with extensive tests
- provide detail docs and notes
- prepare a docker image for build and demo
- chaos tests (maybe jepsen tests)
- benchmark and performance optimization
The execution module manages and schedules all raft groups, storage, transport and replicated state machines.
TBD
The core module implements the raft algorithm based on the etcd's design.
- ⭕design doc
⚠️ election with prevote⚠️ leader lease⚠️ leadership transfer⚠️ log compaction⚠️ snapshot⚠️ single membership change- ⭕joint consensus
⚠️ observer⚠️ witness⚠️ quiesce⚠️ linearizable semantics
The storage module works as the WAL of the raft groups, managing the persistent log entries as well as the snapshots and the configurations of the clusters.
- ✔️design doc (lack in-memory log explanation)
- ✔️segment, a WAL unit
- ✔️segment test
- ✔️index, for indexing raft logs (lack dump&load)
⚠️ index test (lack group index test)- ✔️segment_manager, for managing raft persistent logs
⚠️ segment_manager test
The transport module based on the seastar's rpc framework works as a messaging service for clusters.
⚠️ design doc- ✔️exchanger, for exchanging raft messages
⚠️ exchanger test- ✔️registry, for group discovery and peer addresses management
- ⭕registry test
⚠️ express, for sending/receiving replicated state machine's snapshot- ⭕express test
The rsm module bridges the user-defined replicated state machine (e.g. an in-memory key-value store) and the Raft Core State Machine (also Storage, Transport), managing the lookup, update operations as well as snapshot taking, recovering actions of the user's state machine.
A naive in-memory key-value storage is provided to demonstrate the interactions between a user defined replicated state machine and the underlying framework.
- ⭕design doc
⚠️ session, for idempotent operation⚠️ session manager, for managing sessions⚠️ snapshotter, for managing snapshot save/load operations⚠️ statemachine_manager, for bridging the execution engine and the user's statemachine
Various utilities to support different modules of the rafter.
- ✔️backoff, retry in coroutine style
⚠️ circuit_breaker- ✔️fragmented_temporary_buffer, for aligned (de)serialization
⚠️ rate limiter- ✔️serializer, for raft message's (de)serialization
- ✔️worker, double-buffering multi-producer single-consumer worker coroutine, building block for pipelined service in rafter
The test module hacks the interface of the GoogleTest to support testing an asynchronous application with GoogleTest's style.
- ✔️base, for integrating the coroutine and the gtest macros, and the Seastar's reactor engine management
⚠️ unit tests- ⭕integration tests
- ⭕benchmark
Some tips available.
in progress
TBD