Skip to content

Issues: datalad/datalad-crawler

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
36 Open 23 Closed
36 Open 23 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

crawler pipeline for 'indexes' (ftp/http) with specs for where to break into submodules
#78 opened Jun 24, 2016 by yarikoptic
@glalteva
2
openfmri: crawl derivatives into submodule(s)
#75 opened Aug 12, 2016 by yarikoptic
6
crawl http index(es) helper
#76 opened Jan 24, 2018 by yarikoptic
Provide S3 credentials
#2 opened May 8, 2018 by mih
Confusing crawl-init error UX
#3 opened May 10, 2018 by mih
Needs Debian package
#7 opened Jun 12, 2018 by yarikoptic
Needs conda recipe/package
#8 opened Jun 12, 2018 by yarikoptic
Running crawl-init in a non-dataset brings confusing and irrlevant error message
#12 opened Aug 8, 2018 by yarikoptic
2
Running datalad crawl import module error
#14 opened Aug 8, 2018 by um4r12
7
crawling sample stanford dataset failed - they have incomplete .tar external-resource-databug
#18 opened Sep 14, 2018 by yarikoptic
8
Describe basic structure of of crawler pipelines
#77 opened Sep 17, 2018 by mih
1
RF: move crawl-init into crawl making crawl accept additional params to be passed into pipeline good first issue Good for newcomers
#19 opened Oct 28, 2018 by mih
@debanjum
figshare crawler
#20 opened Nov 1, 2018 by yarikoptic
"Reproducible" crawled datasets
#21 opened Nov 1, 2018 by yarikoptic
Generic framework for crawling data providers with versions enhancement New feature or request
#22 opened Nov 1, 2018 by yarikoptic
3
using the datalad crawler for LORIS
#34 opened Jan 21, 2019 by thomasbeaudry
3
Support split archives
#39 opened Mar 4, 2019 by yarikoptic
Credential lookup failure crashes s3 crawler
#46 opened Jul 11, 2019 by TobiasKadelka
2
datalad crawl: Changing behaviour between HCP900/1200
#48 opened Jul 16, 2019 by TobiasKadelka
1
--is-pipeline relative import error bug Something isn't working
#52 opened Aug 27, 2019 by joeyzhou98
3
Crawling multiple web pages
#54 opened Aug 28, 2019 by joeyzhou98
2
NDA crawler fails with error
#56 opened Sep 24, 2019 by loj
8
Add digester to enrich git annex metadata for every key enhancement New feature or request
#65 opened Nov 15, 2019 by yarikoptic
1
Support (store) ETag if remote (http) resource provides it
#69 opened Feb 11, 2020 by yarikoptic
s3 crawler -- doesn't handle "gracefully" situation of completely removed keys in a versioned bucket
#73 opened Mar 24, 2020 by yarikoptic
ProTip! What’s not been updated in a month: updated:<2024-10-11.