Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an optional limit to downloaded files #113

Open
ju-gu opened this issue Sep 25, 2023 · 1 comment
Open

Add an optional limit to downloaded files #113

ju-gu opened this issue Sep 25, 2023 · 1 comment

Comments

@ju-gu
Copy link
Member

ju-gu commented Sep 25, 2023

User Story
The current implementation of download files (which is already really nice to have) always downloads a whole corpus. Though sometimes one wants to download just a smaller sample of the corpus to do experiments with. Therefore it would be nice to have the option to set a maximum amount of files to download.

Behaviour:
CLI:

deepset-cloud download --limit 100

SDK

download(limit=100) 

to download just a subset of files

@ArzelaAscoIi
Copy link
Member

As a quick fix you could just run deepset-cloud download and cancel the request after a couple of seconds/minutes etc. But thats definitely just a quick fix!

Will add this as a feature request :)

@ArzelaAscoIi ArzelaAscoIi changed the title option to download just a certain amount of files Add an optional limit to downloaded files Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants