Build Your Own Redis: Ping <-> Pong [2/4]

This is the second article in a series where we’ll build a toy Redis clone in Ruby. If you’d like to code-along, try the Build your own Redis challenge!

Previous article: Barebones TCP Server

Next article: Concurrent Clients

Sections in this article:

The Redis Protocol

In the previous article, we look at how to create a TCP server that can accept and handle clients. Redis clients talk to Redis using the Redis Protocol.

The Redis Protocol (we’ll refer to it as RESP from now on) is a serialization format that supports multiple data types like strings, integers and arrays.

The first byte in a reply denotes what data type we’re dealing with.

In this tutorial, we’ll only look at the following data types:

Simple Strings

For simple strings, the first byte is sent as +. This is followed by the actual string, and then a CLRF (\r\n).

As an example, here’s what the string ‘OK’ looks like when encoded in RESP:

+OK\r\n
  1. + - signifies that the data type is a simple string
  2. OK - this is the string being sent
  3. \r\n (CLRF) - this denotes the end of the string

Similarly, “PONG” would be encoded as:

+PONG\r\n

Bulk Strings

One problem with simple strings is that they aren’t binary-safe. Simple strings use CLRF to denote the end of the string. What if the string you meant to send contained a CLRF?

+THIS HAS A \r\n INSIDE IT\r\n

RESP will decode the above as ‘THIS HAS A ‘, instead of ‘THIS HAS A \r\n INSIDE IT’. When RESP encounters the \r\n, it thinks the string has ended!

This is where bulk strings come in handy. In bulk strings, the caller specifies the number of bytes they intend to send before sending the string. This way, we just read a fixed number of bytes, instead of searching for a CLRF.

Here’s the string “HEY” encoded as a bulk string:

$3\r\nHEY\r\n

Because we’re sending the number of bytes upfront, we can embed a CLRF in the response too, if needed.

The string “THIS CONTAINS A \r\n INSIDE IT” (28 bytes in length) can be encoded as so:

$30\r\nTHIS CONTAINS A \r\n INSIDE IT\r\n

Arrays

Arrays in RESP can be arrays of any data type RESP supports, including arrays themselves. You can encode arrays of strings, arrays of arrays etc.

Arrays start with *, followed by the number of elements in the array.

Here’s how the array ["hey", "there"] is encoded in RESP:

*2\r\n$3\r\nhey\r\n$5\r\nthere\r\n

Redis Commands via RESP

Redis commands are sent as arrays of bulk strings in RESP.

The command PING is sent as an array with one element:

["PING"]

The command GET key is sent as so:

["GET", "key"]

Encoded in RESP, this’ll look like:

*2\r\n$3\r\nGET\r\n$3\r\nkey\r\n

The Redis server responds with any valid RESP data type.

Let’s take the PING command as an example.

Here’s what it looks like when you execute PING in redis-cli:

➜  redis-cli
127.0.0.1:6379> PING
PONG

Under the hood, here’s what the client sent to the server:

*1\r\n$4\r\nPING\r\n

(That’s ["PING"], encoded as an array of Bulk Strings in RESP)

And here’s what the server sent back:

+PONG\r\n

(That’s "PONG", encoded as a Simple String in RESP)

Integrated Test

Let’s start out with an integrated test for the PING command. We’ll use Ruby’s redis-rb for this, since it already knows to speak RESP.

require "redis"
require "minitest/autorun"

# 6379 for official redis, 6380 for ours
SERVER_PORT = ENV["SERVER_PORT"]

class TestRedisServer < Minitest::Test
  def test_responds_to_ping
    r = Redis.new(port: SERVER_PORT)
    assert_equal "PONG", r.ping
  end
end

When I run this against the official Redis, tests pass:

➜  SERVER_PORT=6379 ruby tests/test_main.rb
Run options: --seed 20154

# Running:

.

1 runs, 1 assertions, 0 failures, 0 errors, 0 skips

When I run this against our Redis (switched the port from 6379 to 6380), tests fail:

➜  SERVER_PORT=6380 ruby tests/test_main.rb
Run options: --seed 17862

# Running:

E

  1) Error:
TestMain#test_ping:
Redis::TimeoutError: Connection timed out
    /home/rohitpaulk/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:71:in `rescue in _read_from_socket'

    ...

    /home/rohitpaulk/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/redis-4.1.3/lib/redis.rb:152:in `ping'
    tests/test_main.rb:11:in `test_ping'

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

The failure makes sense! Our server accepts connections, but it doesn’t send back replies to the client. The redis client hit a time-out when waiting for a reply.

Implementing PING

Now that we have a failing test setup, it’s time to change our server implementation so that the test passes.

(For the uninitiated, we’re following TDD here)

We need to send PONG back to a client when they send us the PING command.

To check what command the user has sent us, we’ll need to parse RESP and check if the command we’ve received is indeed PING.

Do we need to parse RESP though? To make our current set of tests pass, we could just ignore what the user sent us, and send back PONG anyway.

  require "socket"
  
  class RedisServer
    def initialize(port)
      @server = TCPServer.new(port)
    end
  
    def listen
      loop do
        client = @server.accept
-       # TODO: Serve client
+       # TODO: Handle commands other than PING
+       client.write("+PONG\r\n")
      end
    end
  end

Running the tests to check, and…

➜  SERVER_PORT=6380 ruby tests/test_main.rb
Run options: --seed 20154

# Running:

.

1 runs, 1 assertions, 0 failures, 0 errors, 0 skips

Voila!

Since our server only understands PING at the moment, it was easy to just ignore what the client sent and always send PONG back.

If a client sends us ECHO hey, we’ll still send PONG!

➜  redis-cli
127.0.0.1:6379> PING
PONG
127.0.0.1:6379> ECHO hey
PONG

This’ll work for now (yes, we cheated on the test). We’ll clean this up when we work on handling other commands like ECHO.

In the next article, we’ll look at how our current implementation is flawed when it comes to handling multiple clients.