Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the database updates more performant #15

Closed
SnirShechter opened this issue Jan 1, 2021 · 1 comment · May be fixed by #195
Closed

Make the database updates more performant #15

SnirShechter opened this issue Jan 1, 2021 · 1 comment · May be fixed by #195
Assignees
Labels
enhancement New feature or request
Projects

Comments

@SnirShechter
Copy link
Member

SnirShechter commented Jan 1, 2021

Problem

The database is being accessed many times, which slows down the database and server quite heavily.

Solution

Make a list of all the non-live updates - incVisitCount, incInfoCount etc.

Design an asynchronous batch updating functionality for the frequent non-live updates. e.g incInfoCount should keep an in-memory count inside the driver and flush them all every 5-10 seconds.

NOTE Before the design, check if there's a better way to handle it in the connector libraries (e.g Knex) instead of implementing our own.

EDIT

After checking, there's no such functionality in Knex. The PostgresStorageDriver should implement a batch-update mechanism for incInfoCount and incVisitCount. each of these should save its own cache and flush it every 10 seconds. Their usage in routes should also be non-blocking.

@SnirShechter SnirShechter created this issue from a note in Roadmap (Future) Jan 1, 2021
@SnirShechter SnirShechter added the infra Infrastructure label Jan 1, 2021
@SnirShechter SnirShechter moved this from Future to v1.0.0 (Apr. 2021) - Configuration & Frontend in Roadmap Feb 27, 2021
@SnirShechter SnirShechter added enhancement New feature or request and removed infra Infrastructure labels Mar 12, 2021
@Man-RP
Copy link
Collaborator

Man-RP commented Jun 23, 2021

I'll do this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Roadmap
v1.0.0 (Apr. 2021) - Configuration &a...
Development

Successfully merging a pull request may close this issue.

2 participants