+

Chapter 4 --- Encoding and Evolution.ipynb

@@ -92,16 +92,14 @@
  " * Moreover, remote calls are basically always way less durable and have more variable latency, because they rely on potentially flakey, bandwidth-limited network.\n",
  " * If you retry an operation you may duplicate an action on the endpoint. This can be worked around but requires additional thought and design.\n",
  " * Networks have a much higher fixed cost than local functions. Youy need to encode all of the data necessary and send it out. Additional serialization is necessary, and large objects are a problem.\n",
- "* At the end of the day, costs of space in memory is much lower than it is in-network. A good term: good local calls are fine, network calls are coarse. This is a big reason why monoliths are hard to refactor into microservice things!"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# "
+ "* At the end of the day, costs of space in memory is much lower than it is in-network. A good term: good local calls are fine, network calls are coarse. This is a big reason why monoliths are hard to refactor into microservice things!\n",
+ "\n",
+ "\n",
+ "* The third type is the message bus pattern.\n",
+ "* Your applications can communicate with one another using streams of messages that get passed through a **message broker**.\n",
+ "* Example message brokers are ZeroMQ, Apache Kafka, and Google PubSub.\n",
+ "* When using this pattern, your application parts no longer need to be aware of one another. Message emitters can emit messages without caring about who's listening, and message listeners can consume messages without being aware of who's emitting them.\n",
+ "* This pattern is good for separation of concerns. The trade-off is that there is one more service you have to maintain. Message brokers are not a lightweight solution overhead-wise."
  ]
  }
  ],
@@ -121,7 +119,7 @@
  "name": "python",
  "nbconvert_exporter": "python",
  "pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.6.5"
  }
  },
  "nbformat": 4,

Chapter 5 --- Replication.ipynb

@@ -0,0 +1,85 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "* **Replication** is one of the strategies for distributing data processes across multiple nodes (the other is **partitioning**, the subject of a later chapter).\n",
+ "* The difference between replication and partitioning is that whilst replication has you keep multiple copies of the data in several different places, partioning has you split that one dataset amongst multiple partitions.\n",
+ "* Both strategies are useful, but obviously there are tradeoffs either way.\n",
+ "\n",
+ "\n",
+ "* Replication strategies fall into three categories:\n",
+ " * **Single-leader** — one service is deemed the leader. All other replicas follow.\n",
+ " * **Multi-leader** — multiple services are deemed leaders. All other replicas follow.\n",
+ " * **Client-driver** — not sure.\n",
+ " \n",
+ "## Single-leader replication\n",
+ "\n",
+ "* The first important choice that needs to be made is whether replication is synchronous or asynchronous.\n",
+ "* Almost all replication is done asynchronously, because synchronous replication introduces unbounded latency to the application.\n",
+ "* User-facing applications generally want to maintain the illusion of being synchronous, even when the underlying infrastructure is not.\n",
+ "* Asynchronous replication introduces a huge range of problems that you have to contend with if you try to maintain this illusion.\n",
+ "* (personal note) GFS is an interesting example. GFS was explicitly designed to be weakly consistent. That unlocked huge system power, and the additional application layer work that was required to deal with the architecture was deemed \"worth it\" because it was on the engineer's hands.\n",
+ "\n",
+ "\n",
+ "* The precise configuration of concerns with a single-leader replication strategy differs. At a minimum the leader handles writes, and communicates those writes to the follower, and then the followers provide reads.\n",
+ "\n",
+ "\n",
+ "* If a follower fails, you perform catch-up recovery. This is relatively easy (example, Redis).\n",
+ "* If a leader fails you have to perform a **failover**. This is very hard:\n",
+ " * If asynchronous replication is used the replica that is elected the leader may be \"missing\" some transaction history which occurred in the leader. Totally ordered consistency goes wrong.\n",
+ " * You can discard writes in this case, but this introduces additional complexity onto any coordinating services that are also aware of those writes (such as cache invalidation).\n",
+ " * It is possible for multiple nodes to believe they are the leader (Byzantine generals). This is dangerous as it allows concurrent writes to different parts of the system. E.g. **split brain**.\n",
+ " * Care must be taken in setting the timeout. Timeouts that are too long will result in application delay for users. Timeouts that are too short will cause unnecessary failovers during high load periods.\n",
+ "* Every solution to these problems requires making trade-offs!\n",
+ "\n",
+ "\n",
+ "## Replication streams\n",
+ "\n",
+ "* How do you implement the replication streams?\n",
+ "\n",
+ "\n",
+ "* The immediately obvious solution is **statement-based replication**. Every write request results in a follower update.\n",
+ "* This is a conceptually simple architecture and likely the most compact and human-readable one. But it's one that's fallen out of favor due to problems with edge cases.\n",
+ "* Statement-based replication requires pure functions. Non-deterministic functions (`RAND`, `NOW`), auto-incrementation (`UPDATE WHERE`), and side effects (triggers and so on) create difficulties.\n",
+ "\n",
+ "\n",
+ "* **Write-ahead log shipping** is an alternative where you ship your write-ahead log (if you're a database or something else with a WAL).\n",
+ "* This is nice because it doesn't require any additional work by the service. A WAL already exists.\n",
+ "* This deals with the statement-based replication problems because WALs contain record dumps by design. To update a follower, push that record dump to it.\n",
+ "* The tradeoff is that record dumps are a data storage implementation detail. Data storage may change at any times, and so the logical contents of the WAL may differ between service versions.\n",
+ "* The write-ahead logs of different versions of a service are generally incompatible! But your system will contain many different versions.\n",
+ "* Generally WALs are not designed to be version-portable, so upgrading distributed services using WAL shipping for replication requires application downtime.\n",
+ "\n",
+ "\n",
+ "* **Logical log replication** is the use of an alternative log format for replication. E.g. replication is handled by its own distinct logs that are only used for that one specific purpose.\n",
+ "* This is more system that you have to maintain, but it's naturally backwards and forward compatible if you design it right (using e.g. a data interchange format like Protobufs) and works well in practice.\n",
+ "\n",
+ "\n",
+ "* Post-script: you may want partial replication. In that case you generally need to have application-level replication. You can do this in databases, for example, by using triggers and stored procedures to move data that fits your criteria in specific ways."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.5"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}