MessageCorps is a wrapper around kafka-python, forcing the use of google protobuffers https://developers.google.com/protocol-buffers/docs/proto3
Later integration with GRPC for non batch operations is planned (also using the GRPC <-> Rest compatability for creating RESTful apis)
You require protoc to be installed to build the protobuffers, otherwise everything else is in requirements.txt
A very short send and recieve example is in the examples folder, you'll require docker-compose / docker for setting up your kafka instance.
You can add new folders or sub folders depending on the hierachy you require.
-
Add new .proto file and fill in (see the link above for documentation)
-
Build the protos in to python files (top level run build_protos.sh)
-
Add the import path and channel: object mapping in MessageCorps/pb_handler.py so you have the correct associations
-
Install via your favourite method (either setup.py install or bump the version number, commit to git and install your projects dependancies again using pip)