Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support expressions in redis output stream field #1283

Conversation

bmarinov
Copy link
Contributor

@bmarinov bmarinov commented Jun 7, 2022

PR fixes #1083

This is a quick fix for the lack of expression support in the stream configuration on the redis stream output output.redis_streams.stream config field.

I tried to follow the integration test setup for redis, but in the end I had to add a new test that allows for additional metadata to be set on messages.

I have a couple of questions and some todo's - adding comments on the PR.

@@ -137,7 +142,7 @@ func (r *redisStreamsWriter) WriteWithContext(ctx context.Context, msg *message.
if msg.Len() == 1 {
if err := client.XAdd(&redis.XAddArgs{
ID: "*",
Stream: r.conf.Stream,
Stream: r.interpolatedStreamName.String(0, msg),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This magic value (index 0) is due to the fact that there is a single message in the batch. Is it obvious enough and have I understood the message loop for the expression handling correctly?

@bmarinov bmarinov marked this pull request as ready for review June 8, 2022 16:25
@Jeffail
Copy link
Collaborator

Jeffail commented Jun 19, 2022

Thanks @bmarinov, looks great. I'll fix merge conflicts on my end as I've been refactoring constructor signatures.

@Jeffail Jeffail merged commit 7fb9f03 into redpanda-data:main Jun 19, 2022
@bmarinov bmarinov deleted the 1083-support-interpolated-redis-output-stream branch June 19, 2022 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Interpolation/dynamic routing support for redis streams
3 participants