refactor(events): populate object identifiers in outgoing webhooks analytics events during retries #5067
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Type of Change
Description
As of now, the analytics events raised during outgoing webhook retries do not include object identifiers such as the payment ID, refund ID, etc., and in turn, they are not populated in ClickHouse. The object identifiers are only available in the events raised for the initial webhook delivery attempts.
This PR updates the outgoing webhooks code to include the object identifiers in the analytics events for all retries. This is being done by storing the object identifiers in a new column in the
events
table, calledmetadata
.In addition, this PR extracts out common code to implement
FromSql
andToSql
traits on types converted to/fromJSON
/JSONB
into a declarative macro.Additional Changes
This PR adds a
metadata
column to theevents
table.Motivation and Context
Ensuring that all webhook events including retries are correctly associated with the object in question (payment, refund, etc.), and that they are correctly displayed on the control center when viewing events for that resource.
How did you test it?
Configure an incorrect webhook URL in the business profile so that outgoing webhooks are retried.
Perform an action that triggers an outgoing webhook, such as creating a successful payment.
Monitor the events raised for the payment ID in Kafka and ClickHouse:
Kafka: Searching for messages under the
hyperswitch-outgoing-webhook-events
topic with the payment ID has multiple events, including retries:ClickHouse: Searching for rows by payment ID in the
outgoing_webhook_events
table returns multiple rows, all having thepayment_id
populated, and only the initial attempt having thecontent
field populated.The analytics API to obtain outgoing webhook event logs returns all of the events, and each of them contain the payment ID:
The control center should also be ideally displaying all the events in the payment details page, that does not seem to be case however in control center version
v1.30.1
.Verify similar behavior for the other webhooks with their corresponding identifiers: refunds, payouts, etc.
Checklist
cargo +nightly fmt --all
cargo clippy