feat [sc 106247]: Add a caching layer to prevent SDK from sending the same payloads too frequently upstream

[FourSigma]

What this PR does / why we need it:

Caching middleware helps rate limits intentional or unintentional spamming of endpoints.

• Add caching middleware to endpoints
• Prevents identical payloads from being send repeatedly upstream to replicated-app while preserving the client side UX

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Steps to reproduce

Does this PR introduce a user-facing change?

Does this PR require documentation?

[marccampbell]

Is there a reason we don't apply this to other methods in the API? If we do, this should be a middleware in the same implementation that gin middlewares work.

It's weird to me that we are returning "Served From Cache" when we rate limit in this method. I do like that we are including a custom header to indicate that the request was rate limited, but the served from cache is intended to mean that the API was disconnected. Will reusing this same header create a more difficult to debug scenario where the user can't tell if the API was down or they are sending too frequently? Continuing to make this return the same status code is great, don't 429 and push the issue to the developer.

[FourSigma]

Is there a reason we don't apply this to other methods in the API? If we do, this should be a middleware in the same implementation that gin middlewares work.

It's weird to me that we are returning "Served From Cache" when we rate limit in this method. I do like that we are including a custom header to indicate that the request was rate limited, but the served from cache is intended to mean that the API was disconnected. Will reusing this same header create a more difficult to debug scenario where the user can't tell if the API was down or they are sending too frequently? Continuing to make this return the same status code is great, don't 429 and push the issue to the developer.

Good thought process @marccampbell . My initial reading of the issue was that we had a few delinquent endpoints (e.g custom-metrics) that needed to be "rate-limited" via a caching mechanism. The current closure implementation of the middleware allows each endpoint to have its own independent cache. Typing it out makes me thing that this might be unnecessary.

I can definitely change the cache header to be more unique. Do you some meaningful name suggestions already in mind? Fundamentally, we are serving cached data so I re-used the header. There is stdout logging to indicate that is being "rate limited", if that helps?

[divolgin]

we should consider making this a digest too because even without historical record this map can get big

[FourSigma]

@divolgin (CC: @marccampbell ) I made a cached sub-router and added a X-Replicated-Rate-Limited header. Let me know what you all think.

[sgalsaleh]

why not just use bytes.Equal?

[FourSigma]

@sgalsaleh JSON key ordering differences when you marshal from map[string]interface{} results in different byte slices for the same payload. Also raw JSON payloads might have different white space with same underlying content.