Adding REST-API links to OpenNeuro datasets using NeuroJSON.io database interface

[fangq]

What would you like to see added?

I was suggested by @robertoostenveld to submit a ticket on this new feature request for OpenNeuro's web interface - if this is not the correct repo to submit this request, please feel free to move it or let me know where to post.

I am working on an NIH U24 project named NeuroJSON (https://neurojson.org), with a goal of developing a scalable neuroimaging dataset dissemination/metadata search platform using JSON and NoSQL database as the common interface. @robertoostenveld and @guiomar have been serving on our steering committee.

Over the last year, we have developed a new data portal, NeuroJSON.io (https://neurojson.io) to implement these concepts and start disseminating free/public neuroimaging datasets via our web site and scalable CouchDB database backend. Currently, we are focusing on BIDS-compatible datasets due to the simplicity of extracting needed metadata (due to the predictable data structures). We have been synchronizing OpenNeuro datasets via the datalad mirror. A number of video tutorials showing the key functionalities of our platform can be found at https://neurojson.org/Doc/Start/User

Currently, the "Download" tab of every dataset on OpenNeuro shows 4 ways to download: S3, Node.js, Datalad, and shell script.

I would like to request to add a new way of accessing OpenNeuro datasets programmably via REST-APIs. our NeuroJSON.io runs on a CouchDB database, which provides REST-API access to both the full metadata (including folder/file hierarchy and most human-readable content - .json/.tsv etc) as well as search/filter/revision functions. The format of the REST-API is quite straightforward

https://neurojson.io:7777/openneuro/dataset_ID

for example

URL	Meaning
https://neurojson.io:7777/openneuro/ds000001	REST-API for downloading the full metadata for ds000001
https://neurojson.org/db/openneuro/ds000001	URL for browsing the metadata for ds000001 and previewing data in browser

I would like to see if OpenNeuro developers can add the REST-API link above for each of the datasets to allow codes/libraries to access OpenNeuro data programmably. If you see a value, you can also add the online browsing features as we have implemented various 3D/4D data rendering features.

A convenient use-case of these REST-API is the incremental data query and download (i.e. on-demand), using our MATLAB or Python helper functions - user can query the desired data files and only download the needed files. A demo on how to do this in Python and MATLAB can be found in the "additional screenshots" section.

I am happy to provide more information regarding these API interfaces if needed.

PS: there are many other database-level REST-APIs that can facilitate full-database search, listing, versioning, and key-based queries such as _all_docs, _find, _design, _change etc, if you see a need to add these links to facilitate programmably accessing OpenNeuro datasets, I am happy to create a separate ticket.

Alternatives

Currently, the provided download options do not support REST API based data access.

Do you have any interest in helping implement the feature?

Yes

Additional information / screenshots

Demo on using REAT-API and incremental/on-demand data download/caching in Python

neurojson_python_rest_api.mp4

similarly, one can also perform the same operations in MATLAB/Octave

jcache1.mp4

jcache2.mp4