Decided snapshot, add example and remove runtime

Cargo.toml

@@ -1,9 +1,8 @@
 [workspace]
 members = [
  "omnipaxos_core",
- "omnipaxos_runtime",
  "omnipaxos_storage",
- "examples"
+ "examples/kv_store"
 ]
 
 [profile.release]

docs/src/SUMMARY.md

@@ -4,13 +4,14 @@
 [Getting Started](getting-started.md)
 
 - [Introduction](introduction/index.md)
-- [SequencePaxos](sequencepaxos/index.md)
- - [Storage](sequencepaxos/storage.md)
- - [Communication](sequencepaxos/communication.md)
- - [Reading and Writing](sequencepaxos/log.md)
- - [Compaction](sequencepaxos/compaction.md)
- - [Reconfiguration](sequencepaxos/reconfiguration.md)
- - [Logging](sequencepaxos/logging.md)
+- [OmniPaxos](omnipaxos/index.md)
+ - [Storage](omnipaxos/storage.md)
+ - [Communication](omnipaxos/communication.md)
+ - [Reading and Writing](omnipaxos/log.md)
+ - [Leader Election](omnipaxos/leader_election.md)
+ - [Compaction](omnipaxos/compaction.md)
+ - [Reconfiguration](omnipaxos/reconfiguration.md)
+ - [Logging](omnipaxos/logging.md)
 - [Ballot Leader Election](ble/index.md)
 - [Runtime](runtime/index.md)
 

docs/src/foreword.md

@@ -3,6 +3,6 @@ OmniPaxos is an in-development replicated log library implemented in Rust. OmniP
 
 Similar to Raft, OmniPaxos can be used to build strongly consistent services such as replicated state machines. Additionally, the leader election of OmniPaxos offers better resilience to partial connectivity and more flexible and efficient reconfiguration compared to Raft.
 
-The library consist of three parts: `omnipaxos_core`, `omnipaxos_runtime` and `omnipaxos_storage`. The `omnipaxos_core` implements the algorithms of OmniPaxos as plain Rust structs and is suitable for integration with systems that already have an async runtime or are implemented in an actor framework. It requires users to implement the interaction between the different structs themselves as we describe [here](ble/index.md). If you just want a replicated log out of the box, we suggest using `omnipaxos_runtime` instead, which hides all the interactions from the user by using [Tokio](https://tokio.rs/). You can provide your own implementation for storing the log and state of OmniPaxos, but we also provide both in-memory and persistent storage implementations that work out of the box in `omnipaxos_storage`.
+The library consist of two workspaces: `omnipaxos_core` and `omnipaxos_storage`. The `omnipaxos_core` implements the algorithms of OmniPaxos as plain Rust structs and you need to implement the actual networking yourself (we describe how to send and handle messages [here](omnipaxos/communication.md)). You can provide your own implementation for storing the log and state of OmniPaxos, but we also provide both in-memory and persistent storage implementations that work out of the box in `omnipaxos_storage`.
 
 All the code from the tutorial can be found in [examples](https://github.com/haraldng/omnipaxos/tree/master/examples). In addition to the tutorial style presentation in this book, examples of usages of OmniPaxos can be found in the [tests](https://github.com/haraldng/omnipaxos/tree/master/tests).

docs/src/getting-started.md

@@ -27,11 +27,4 @@ To do so add the following to your Cargo.toml:
 omnipaxos_core = { git = "https://github.com/haraldng/omnipaxos" }
 ```
 
-or 
-
-```toml
-[dependencies]
-omnipaxos_runtime = { git = "https://github.com/haraldng/omnipaxos" }
-```
-
-Depending on if you want to use just the core structs or the runtime (see [here](./foreword.md)). For the first chapters of this tutorial we will use ``omnipaxos_core``.
+In ``omnipaxos_core/examples/kv_store``, we show a minimal example of how to use OmniPaxos to replicate KV operations using tokio.

docs/src/introduction/index.md

@@ -1,4 +1,4 @@
 # Introduction
 
-The OmniPaxos library is mainly driven by the [**SequencePaxos**](../sequencepaxos/index.md) and [**Ballot Leader Election**](../ble/index.md) (*BLE*) structs. These are plain Rust structs and the user therefore needs to provide a network implementation themselves to actually send and receive messages. In this tutorial we will show how a user should interact with these structs in order to implement a strongly consistent, replicated log. This tutorial will focus on how to use the library and showcase its features. 
+The OmniPaxos library is mainly driven by the [**OmniPaxos**](../omnipaxos/index.md)struct. It is a plain Rust struct and the user therefore needs to provide a network implementation themselves to actually send and receive messages. In this tutorial we will show how a user should interact with this struct in order to implement a strongly consistent, replicated log. This tutorial will focus on how to use the library and showcase its features. 
 <!-- For the properties and advantages of OmniPaxos in comparison to other similar protocols, we refer to the Omni-Paxos paper. -->

docs/src/sequencepaxos/communication.md → docs/src/omnipaxos/communication.md

@@ -1,34 +1,34 @@
 # Communication
-As previously mentioned, the user has to send/receive messages between servers themselves. In this section, we show how the user should interact with `SequencePaxos` and its incoming and outgoing messages.
+As previously mentioned, the user has to send/receive messages between servers themselves. In this section, we show how the user should interact with `OmniPaxos` and its incoming and outgoing messages.
 
 ## Incoming and Outgoing
-When a message is received from the network layer intended for our node, we need to handle it in `SequencePaxos`.
+When a message is received from the network layer intended for our node, we need to handle it in `OmniPaxos`.
 
 ```rust,edition2018,no_run,noplaypen
 use omnipaxos_core::messages::Message;
 
 // handle incoming message from network layer
 let msg: Message<KeyValue, KVSnapshot> = ...; // message to this node e.g. `msg.to = 2`
-seq_paxos.handle(msg);
+omni_paxos.handle(msg);
 ```
 
-By handling incoming messages and local calls such as `append()`, our local `seq_paxos` will produce outgoing messages for its peers. Thus, we must periodically send the outgoing messages on the network layer.
+By handling incoming messages and local calls such as `append()`, our local `omni_paxos` will produce outgoing messages for its peers. Thus, we must periodically send the outgoing messages on the network layer.
 
 ```rust,edition2018,no_run,noplaypen
 // send outgoing messages. This should be called periodically, e.g. every ms
-for out_msg in seq_paxos.get_outgoings_msgs() {
- let receiver = out_msg.to;
+for out_msg in omni_paxos.outgoings_msgs() {
+ let receiver = out_msg.get_receiver();
  // send out_msg to receiver on network layer
 }
 ```
 
-> **Note:** The networking i.e. how to actually send and receive messages needs to be implemented by you, the user. Similarly, you have to periodically fetch these outgoing messages from `SequencePaxos`. 
+> **Note:** The networking i.e. how to actually send and receive messages needs to be implemented by you, the user. Similarly, you have to periodically fetch these outgoing messages from `OmniPaxos`. 
 
 ## Handling Disconnections
 One of the main advantages of Omni-Paxos is its resilience to partial connectivity. If one node loses connection to another and then reconnects (e.g. after a TCP-session drop), make sure to call ``reconnected(pid)`` before handling any incoming messages from that peer.
 
 ```rust,edition2018,no_run,noplaypen
 // network layer notifies of reconnecting to peer with pid = 3
-seq_paxos.reconnected(3);
+omni_paxos.reconnected(3);
 ...
 ```

docs/src/sequencepaxos/compaction.md → docs/src/omnipaxos/compaction.md

@@ -1,5 +1,5 @@
 # Compaction
-As time passes, the replicated log in `SequencePaxos` will grow large. To avoid letting the log growing infinitely large, we support two ways of compaction that can be initiated by users:
+As time passes, the replicated log in `OmniPaxos` will grow large. To avoid letting the log growing infinitely large, we support two ways of compaction that can be initiated by users:
 
 ## Trim
 Trimming the log removes all entries up to a certain index. Since the entries are deleted from the log, a trim operation can only be done if **ALL** nodes in the cluster have decided up to that index. Example:
@@ -9,15 +9,15 @@ use omnipaxos_core::sequence_paxos::CompactionErr;
 
 // we will try trimming the first 100 entries of the log.
 let trim_idx = Some(100); // using `None` will use the highest trimmable index
-match seq_paxos.trim(trim_idx) {
+match omni_paxos.trim(trim_idx) {
  Ok(_) => {
- // later, we can see that the trim succeeded with `seq_paxos.get_compacted_idx()`
+ // later, we can see that the trim succeeded with `omni_paxos.get_compacted_idx()`
  }
  Err(e) => {
  match e {
  CompactionErr::NotAllDecided(idx) => {
  // Our provided trim index was not decided by all servers yet. All servers have currently only decided up to `idx`.
- // If desired, users can retry with seq_paxos.trim(Some(idx)) which will then succeed.
+ // If desired, users can retry with omni_paxos.trim(Some(idx)) which will then succeed.
  }
  ...
  }
@@ -28,7 +28,7 @@ match seq_paxos.trim(trim_idx) {
 > **Note:** Make sure your application really does not need the data that will be trimmed anymore. Once it is succeeded, the trimmed entries are lost and cannot be read or recovered.
 
 ## Snapshot
-The disadvantage of trimming is that the data is lost after the trim and it requires all servers to have decided the trim index. If the entries in the log are such that they can be compacted into a snapshot, `SequencePaxos` supports snapshotting decided entries of the log. For instance, in our kv-store example application, we don't need to keep every log entry that changes the kv-pairs. Instead, if we want to snapshot the log, it is sufficient to keep the latest value for every key. We implement our snapshot as a struct called `KVSnapshot` which is just a wrapper for a `HashMap` that will hold the latest value for every key in the log. To make it work with `SequencePaxos`, we need to implement the trait `Snapshot` for `KVSnapshot`:
+Trimming compacts the log and discards any data preceding the trim index. For safety, it therefore requires all servers to have decided the trim index. If you don't want to discard any data and the entries in the log are such that they can be compacted into a snapshot, `OmniPaxos` supports snapshotting decided entries of the log. For instance, in our kv-store example, we don't need to keep every log entry that changes the kv-pairs. Instead, if we want to snapshot the log, it is sufficient to keep the latest value for every key. We implement our snapshot as a struct called `KVSnapshot` which is just a wrapper for a `HashMap` that will hold the latest value for every key in the log. To make it work with `OmniPaxos`, we need to implement the trait `Snapshot` for `KVSnapshot`:
 
 ```rust,edition2018,no_run,noplaypen
 use omnipaxos_core::storage::Snapshot;
@@ -60,22 +60,22 @@ impl Snapshot<KeyValue> for KVSnapshot {
 }
 ``` 
 
-The ``create_entries()`` function tells `SequencePaxos` how to create a snapshot given a slice of entries of our `KeyValue` type. In our case, we simply want to insert the kv-pair into the hashmap. The `merge()` function defines how we can merge two snapshots. In our case, we will just insert/update the kv-pairs from the other snapshot. The `use_snapshots()` function simply tells `SequencePaxos` if snapshots should be used in the protocol. This is needed for users that does not want to use snapshots.
+The ``create()`` function tells `OmniPaxos` how to create a snapshot given a slice of entries of our `KeyValue` type. In our case, we simply want to insert the kv-pair into the hashmap. The `merge()` function defines how we can merge two snapshots. In our case, we will just insert/update the kv-pairs from the other snapshot. The `use_snapshots()` function tells `OmniPaxos` if snapshots should be used in the protocol. 
 
-With ``KVSnapshot``, we would have instead created our `SequencePaxos` node as follows:
+With ``KVSnapshot``, we would have instead created our `OmniPaxos` node as follows:
 ```rust,edition2018,no_run,noplaypen
-// ...same as shown before in the `SequencePaxos` chapter.
+// ...same as shown before in the `OmniPaxos` chapter.
 let storage = MemoryStorage::<KeyValue, KVSnapshot)>::default(); // use KVSnapshot as type argument instead of ()
-let mut sp = SequencePaxos::with(sp_config, storage);
+let mut omni_paxos = omni_paxos_config.build(storage);
 ```
-We can now create snapshots and read snapshots from `SequencePaxos`. Furthermore, snapshotting allows us to either just do the snapshot locally or request all nodes in the cluster to do it with the boolean parameter `local_only`.
+We can now create snapshots and read snapshots from `OmniPaxos`. Furthermore, snapshotting allows us to either just do the snapshot locally or request all nodes in the cluster to do it with the boolean parameter `local_only`.
 ```rust,edition2018,no_run,noplaypen
 // we will try snapshotting the first 100 entries of the log.
 let snapshot_idx = Some(100); // using `None` will use the highest snapshottable index
 let local_only = false; // snapshots will be taken by all nodes.
-match seq_paxos.trim(snapshot_idx, local_only) {
+match omni_paxos.snapshot(snapshot_idx, local_only) {
  Ok(_) => {
- // later, we can see that the snapshot succeeded with `seq_paxos.get_compacted_idx()`
+ // later, we can see that the snapshot succeeded with `omni_paxos.get_compacted_idx()`
  }
  Err(e) => {
  match e {
@@ -88,7 +88,7 @@ match seq_paxos.trim(snapshot_idx, local_only) {
 }
 
 // reading a snapshotted entry
-if let Some(e) = seq_paxos.read(20) {
+if let Some(e) = omni_paxos.read(20) {
  match e {
  LogEntry::Snapshotted(s) => {
  // entry at idx 20 is snapshotted since we snapshotted idx 100
@@ -100,4 +100,4 @@ if let Some(e) = seq_paxos.read(20) {
 }
 ```
 
-> **Note:** If your `Entry` type is not snapshottable, simply use `()` as type argument for `Snapshot`.
+> **Note:** If your `Entry` type is not snapshottable, simply use `()` as the type argument for `Snapshot`.