Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't cache Source requests #29

Closed
faultyserver opened this issue Sep 17, 2016 · 2 comments
Closed

Don't cache Source requests #29

faultyserver opened this issue Sep 17, 2016 · 2 comments
Milestone

Comments

@faultyserver
Copy link
Member

It looks like responses from sources are being cached and re-sending stale data. In other words, instead of getting the most up-to-date information on every request, the sources return the same, stale data until the cache expires (determined by the underlying HTTP library).

Fixing this will probably involve updating the doublemap_api and tripspark_api libraries to take a cache: parameter of sorts, and specifying that caching should not be done on most requests.

This is likely part of the cause of #24.

@faultyserver
Copy link
Member Author

Did some testing by looking at events coming through the system and reloading http:https://citybus.doublemap.com/map/v2/buses at the same time.

Caching is definitely happening at some point. Either that, or attributes aren't being updated as they should be.

faultyserver added a commit that referenced this issue Sep 22, 2016
This was an interesting find: The same memory for station names (and likely many other attributes) was being used through the entire stack (from the Source, to Agency, and up the Middleware stack). This means that when `Normalizer` did an in-place `sub!` on those strings, the changes cascaded all the way backwards through to the reference kept by the Source instance. Then, when the Source went through its update cycle (which seems to be having caching issues; likely what made this an issue in the first place. See #29), it would have the already-Normalized strings, causing some parse errors where certain formats are expected (e.g., the station name includes it's `stop_code`, which Normalizer removes).

This commit simply changes `sub!` to `sub` to avoid this cascading effect entirely, and resets some previous changes to `config/doublemap_source.rb` from attempting to deal with this earlier.
@faultyserver faultyserver added this to the v1.0 milestone Sep 22, 2016
@faultyserver
Copy link
Member Author

See propershark/doublemap_api#1 (comment). Fixing this should be as simple as adding api.flush_cache at the top of each refresh block.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant