Skip to content

Latest commit

 

History

History

redis-demo

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 

Demo to show Redis Load balancing with NGINX Plus

This demo is based on docker and spins up the following containers:

  • Redis which is a popular open source in-memory database. We create 1 master & 2 slave containers
  • Redis Sentinel which is used to provide HA and clustering for Redis
  • and of course NGINX Plus which will be used as a TCP load balancer for the Redis nodes

The demo is based off the work described in this blog post: [Load Balancing Redis with NGINX Plus](Add Link later)

Manual Install

Prerequisites and Required Software

The following software needs to be installed:

Setting up the demo

  1. Clone demo repo

    $ git clone https://github.com/nginxinc/NGINX-Demos.git

  2. Copy nginx-repo.key and nginx-repo.crt files for your account to ~/NGINX-Demos/redis-demo/nginxplus/

  3. Move into the demo directory

    $ cd ~/NGINX-Demos/redis-demo
    
  4. If you have run this demo previously or have any docker containers running, start with a clean slate by running

    $ ./clean-containers.sh
    
  5. Spin up Redis, Sentinel and NGINX Plus containers

    $ docker-compose up -d
    
  6. Now follow the steps under section 'Running the demo'

Running the demo

  1. You should have a bunch of containers up and running now:

    $ docker ps
    CONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS              PORTS                                                             NAMES
    578f9e662203        redisdemo_sentinel    "sentinel-entrypoi..."   4 seconds ago       Up 3 seconds        6379/tcp, 26379/tcp                              redisdemo_sentinel_1
    7aeee69035e7        redis:3               "docker-entrypoint..."   5 seconds ago       Up 4 seconds        0.0.0.0:63793->6379/tcp                          redis-slave2
    77530c277fdd        redis:3               "docker-entrypoint..."   5 seconds ago       Up 4 seconds        0.0.0.0:63792->6379/tcp                          redis-slave1
    4c06e6236587        redisdemo_nginxplus   "nginx -g 'daemon ..."   6 seconds ago       Up 5 seconds        0.0.0.0:6379->6379/tcp, 0.0.0.0:8080->8080/tcp   nginxplus
    0ab4d54bd353        redis:3               "docker-entrypoint..."   6 seconds ago       Up 5 seconds        0.0.0.0:63791->6379/tcp                          redis-master
    

    NGINX Plus is listening on TCP port 6379 on the docker host which is the standard Redis port and load balance the traffic across the 3 Redis containers (one Master & two Slave). One Redis Sentinel container which is used to set up Clustering and for providing High Availability.

  2. Go to the NGINX Plus Monitoring dashboard by going to http:https://127.0.0.1:8080 in your favourite web browser if you used Docker for Mac. If you set this up on a Linux VM, replace 127.0.0.1 with the IP address of your Linux VM. You will see all 3 Redis containers under TCP/UDP Upstreams

  3. Now run redis-cli -h <DOCKER-HOST-IP> and execute couple of different Redis commands. After executing each redis command, notice the number of 'Active' & 'Total' connections getting incremented in the Dashboard for each upstream Redis node in a round-robin manner. This is because of the default round-robin LB

    $ redis-cli -h 127.0.0.1
    127.0.0.1:6379> info replication
    # Replication  
    
    role:master
    connected_slaves:2
    slave0:ip=172.17.0.2,port=6379,state=online,offset=901913,lag=0
    slave1:ip=172.17.0.4,port=6379,state=online,offset=901778,lag=1
    master_repl_offset:901913
    repl_backlog_active:1
    repl_backlog_size:1048576
    repl_backlog_first_byte_offset:2
    repl_backlog_histlen:901912
    127.0.0.1:6379>
    127.0.0.1:6379> set foo abc
    (error) READONLY You can't write against a read only slave.
    127.0.0.1:6379> set foo abc
    OK
    127.0.0.1:6379>
    127.0.0.1:6379> get foo
    "abc"
    127.0.0.1:6379> config get maxclients
    1) "maxclients"
    2) "10000"
    
  4. Notice above that the 'set foo abc' command fails once since it lands on the Redis slave which cannot handle SET. It succeeds on the 2nd attempt when it lands on the Redis master. You could do GET/SET splits by configuring 2 virtual servers listening on 2 different ports (one for GET & another for SET) & then proxy_pass to just the Redis master in case of SET and to all Redis nodes in case of GET