WorkGenius Assignment is a Flask Backend application that listens to POST requests from Mandrill and send Websockets data to connected clients. In fact, Mandrill sends a standard HTTP post request with the parameter mandrill_events which is a JSON Array of webhooks events, messages events in particular. The Backend now listens and process this JSON data. First, it stores into a Redis cache and then sends this information to the connected clients via Websockets.

To resolve this problem, we have used flask, flask-socketio, redis , socketIO , black , isort , mypy , flake8 , pytest and mock.

• flask: among the best python web frameworks, it perfectly suits our situation because it is lightweight and for small projects.
• flask-socketio: our backend server needs to support Websockets. Flask does, by the use of this extension.
• redis: we are using redis for caching. It helps us to store all our messages.
• socketIO: through this library, the client sends and receives data to/from the server.
• pytest: helps us to test our routes and socket events.
• mock: patchs redis store methods to avoid persistence while testing.
• black: our python code formatter tool.
• isort: sorts our imports alphabetically, and automatically separates into section and by type.
• flake8: our python linting tool.
• mypy: our python type checker tool.

To run my Backend solution, you must have python, pip, redis-server installed in your system and configure the redis server with our flask application.

To clone my code, run the command below in the CLI

git clone "https://github.com/adrienTchounkeu/workgenius.git"

You can also download the project by clicking the link WorkGenius webhook

After downloading the code, open the CLI in the root directory and execute the command :

pip install -r requirements.txt

NB: "requirements.txt is a file which contains all the project dependencies"

After all the project dependencies are installed, run the command

python run.py # on Windows

or

python3 run.py # on Linux and Mac

To run the Redis server, follow this link Install & Run Redis

NB: The server generally starts on the port 5000

• The CLient is a simple index.html page that is served by the flask backend from the endpoint /. You can access through the link CLient App

Before starting coding, we have to understand the problem and think of the solution. We have structured our project as follow :

• Choose a great backend framework that supports websockets : Flask for instance
• Create customs events to dispatch upon connection and send data action
• Mandrill is sending an array of events. We need to send less and readable information to the client. See this function count_messages_by_event
• A simple cache server : Redis for instance

To solve the problem, we did some hypothesis:

• The JSON data sent by Mandrill has this format mandrill_events

We used flask, socketIO and redis to:

• listen for connection with Client and log successfully connection messages
• create POST endoint /message-events to process data sent from Mandrill
• store all the messages events in redis(check db/cache.py). For each message, the redis key associated is the _id of the message
• compute the JSON array and regroup it by event type. See the count_messages_by_event function in helpers/messages.py
• send the obtained dictionnary to the client
• display on client

• After running the server with python run.py and having the serving listening on 5000, you can walk through the endpoint / and you will see the client

• A connection is then established between client and the server. The server receives data from the client upon connection. You can check the logs on server and you will see : helpers.logger - INFO - Client is connected with message : {'data': "Client's connected!"}
• To simulate a POST request from Mandrill, I will send the following payload to the API

• Two messages are logs by the server, respectively upon redis persistence confirmation and notification events sent to client. helpers.logger - INFO - Mandrills Events successfully stored on Redis and helpers.logger - INFO - Events sent to Client
• The client then displays the number of emails for each type of event

Tests were performed in the project. Please see under tests/. You need to have pytest installed in your environment before running tests.

pytest tests/

We can add more security to this application with these two options :

• Add CORS on the POST route to only allow receiving requests from mandrillapp.com. I intentionnaly omit it for testing purposes
• Mandrill recommends authenticating webhook requests by comparing signatures. See the link below Authenticating WebHook Requests

Even though my code is solving the problem, I have some performance and resources used issues. To optimize my solution, I think

• implement connection Pool to handle multiple connections
• In case, we have different types of information to share among different types of users/clients: implement rooms to group certain types of users
• control the amoung of data store on redis as Redis is having some size limit
• Add Celery to process redis storage. In fact, when storing messages in redis, it is better to pass this task to a seperate worker.