β οΈ Still under development, it didn't reach v1.0.0 and therefore is not suitable for production use yet.
AsyncAPI Event Gateway (name is subject to change) is the Event Gateway solution by excellence.
Based on traditional API Gateways, it intercepts all incoming messages moving them into a pipeline of middlewares and handlers such as:
- Message validation
- Message manipulation
- Message aggregation
- Message filtering
- Authentication
- Throttling
- Routing
- Monitoring (including tracing)
It supports all the protocols AsyncAPI supports through bindings.
This Event Gateway is also compatible with the HTTP protocol, natively or through an external provider like Krakend.io.
The Event Gateway it's a stateless solution that ensures messages are delivered as fast as possible using a minimal resource footprint. Delivering messages as a top priority means no data loss should happen.
No change in the user's code is needed. The service acts as a proxy between the client and the final broker(s). Messages infer the protocol based on the shape of the input network packet.
The service is entirely configurable, and the user can specify the settings for all protocols as well. For example, consumers' and producers' settings.
The service provides an API for uploading AsyncAPI specs, allowing the user to update their message validation, among others, very quickly. It could even be an automated task whenever you update your specs.
The Event Gateway can extend its functionality via middlewares written by the community. A catalog of middlewares made by the community is also available.
The idea is to keep iterating and support all the protocols AsyncAPI supports through bindings.
However, we reduced the scope for the first versions, so we can give support to the most used protocols.
For the first version, only Kafka protocol will be supported.
A demo of the Event Gateway has been deployed and it is completely available to the users. It is a very limited demo environment, but it is a good starting point for the users to familiarize on the concept.
Note that as per today, only the Kafka protocol is supported. More info at docs/config/kafka.md. The AsyncAPI file used for the demo is available here. You can also open it with Studio.
This video introduces this demo application:
Main things you can do now:
-
Produce Kafka messages to the Kafka broker
event-gateway-demo.asyncapi.com:20472
and topicevent-gateway-demo
. The expected message payload (lightMeasured
) is:type: object properties: lumens: type: integer minimum: 0 description: Light intensity measured in lumens. sentAt: type: string format: date-time description: Date and time when the message was sent.
You can use kcat as the Kafka producer (headers are totally optional):
echo '{"lumens": 100}' | kcat -H test=true -H origin=readme -H time=(date) -b event-gateway-demo.asyncapi.com:20472 -t event-gateway-demo -P
-
Consume Kafka messages from the Kafka broker
event-gateway-demo.asyncapi.com:20472
and topicevent-gateway-demo
. You can use kcat as the Kafka consumer (headers are totally optional):kcat -b event-gateway-demo.asyncapi.com:20472 -t event-gateway-demo -C
-
Watch for the messages via the websocket endpoint at
ws:https://event-gateway-demo.asyncapi.com:5000/ws
. You can use Websocat to connect to the websocket:websocat -v ws:https://event-gateway-demo.asyncapi.com:5000/ws
This websocket endpoint consumes from another Kafka topic where the original produced messages are forwarded.
In the case of producing messages with an invalid payload (payloads that don't validate against the schema above), an extra metadata field (Kafka header under the hood) named
_asyncapi_eg_validation_error
will be included in the message. For example:"_asyncapi_eg_validation_error": "{\"ts\":\"2021-12-20T11:33:26.583143572Z\",\"errors\":[\"lumens: Invalid type. Expected: integer, given: boolean\"]}",
TBD
TBD
This project is built with Go, and it uses Go Modules for managing dependencies.
The Minimum required version of Go is set in go.mod file.
- Clone this repository.
- Run
make build
. The binary will be placed atbin/out/event-gateway
.
Please refer to config reference.
Read CONTRIBUTING guide.
Thanks goes to these wonderful people (emoji key):
Sergio Moya π¬ π π» π π€ π π§ π π¬ π |
Fran MΓ©ndez π€ π |
Maciej UrbaΕczyk π |
Lukasz Gornicki π |
Paul B. π |
Jonas Lagoni π |
This project follows the all-contributors specification. Contributions of any kind welcome!