-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Caching analysis results with memcached #241
Labels
Milestone
Comments
Might be easier if we use containers and/or VM's for test/developing it I think. The docs are great already for development, I could set up Annif quite quickly. But nothing beats being able to run one command (and also being able to quickly have access to all commands used for the installation) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When testing different settings and training ensembles, often the same documents are analyzed over and over with the same backend. This is needlessly slow. It would help a lot if the analysis results could be cached.
Ideally the cache should be persistent across separate invocations of Annif, sharable by multiple Annif processes working in parallel, and automatically expire cached results after some TTL. memcached seems ideal for this purpose.
The cache keys have to be carefully chosen based on e.g. project configuration and timestamps on trained model files, to avoid stale entries being retrieved from the cache.
The text was updated successfully, but these errors were encountered: