Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching analysis results with memcached #241

Open
osma opened this issue Jan 21, 2019 · 1 comment
Open

Caching analysis results with memcached #241

osma opened this issue Jan 21, 2019 · 1 comment
Milestone

Comments

@osma
Copy link
Member

osma commented Jan 21, 2019

When testing different settings and training ensembles, often the same documents are analyzed over and over with the same backend. This is needlessly slow. It would help a lot if the analysis results could be cached.

Ideally the cache should be persistent across separate invocations of Annif, sharable by multiple Annif processes working in parallel, and automatically expire cached results after some TTL. memcached seems ideal for this purpose.

The cache keys have to be carefully chosen based on e.g. project configuration and timestamps on trained model files, to avoid stale entries being retrieved from the cache.

@osma osma added this to the Long term milestone Jan 21, 2019
@kinow
Copy link
Collaborator

kinow commented Feb 1, 2019

Might be easier if we use containers and/or VM's for test/developing it I think. The docs are great already for development, I could set up Annif quite quickly. But nothing beats being able to run one command (and also being able to quickly have access to all commands used for the installation)

@osma osma modified the milestones: Long term, Blue Sky Feb 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants