This repository provides an overview of labeled datasets used for Systematic Reviews. The datasets are available under an open licence and can be used for text mining and machine learning purposes. This repository contains scripts to collect, preprocess and clean the systematic review datasets.

The datasets are alphabetically ordered. See index.csv for all available properties.

For publishing either your data, we recommend using the Open Science frame (OSF). OSF is part of the Center for Open Science (COS), which aims at increasing openness, integrity, and reproducibility of research (OSF, 2020). How to share your data using OSF: A step-by-step guide.

Another platform to publish your data open access is provided by Zenodo. Zenodo is a platform which encourages scientists to share all materials (including data) that are necessary to understand the scholarly process (Zenodo, 2020).

When uploading your dataset to OSF or Zenodo, make sure to provide all relevant information about the dataset, by filling out all available fields. The data to be put on Zenodo or OSF can be documented as extensively as you would like (flowcharts, explanation of certain decisions, etc.). This can include a link to the systematic review itself, if it has been published elsewhere.

When sharing your dataset or a link to your already published systematic review, we recommend using a CC-BY or CC0 license for both Zenodo and OSF. By adding a Creative Commons license, everybody from individual creators to large institutions are given a standardized way to allow use of their creative work under copyright law (Creative Commons, 2020).

In short, the CC-BY license means that reusers are allowed to distribute, remix, adapt, and build upon the material in any medium or format, so long as attribution is given to the creator. The license allows for commercial use. The CC0 license releases data in the public domain, allowing reuse in any form without any conditions. This can be appropriate when sharing (meta)data only. With both OSF (see step-by-step guide) and Zenodo you can easily add the license to your project after creating a project in either platform.

The folder datasets/ has subfolders for the different systematic reviews datasets. In each of these subfolders, the .ipynb script retrieves a dataset from OSF or Zenodo, and preprocesses it by adding customized labels and marking duplicates. The script also reports the inclusion rate, and missing patterns and word clouds of titles and abstracts. After preprocessing, an ASReview-compatible dataset in .csv format is generated in the output/ folder. Extensions .csv, .xlsx, and .xls. CSV files should be comma-separated and UTF-8 encoded. To indicate labeling decisions, one can use "included" or "label_included". This label should be filled with all 0’s and 1’s, where 0 means that the record is not included and 1 means included.

The scripts in the current project are MIT licensed. The datasets (should) have a permissive license.

Open data on systematic reviews is important to advance research on machine learning enhanced systematic reviews. We thank the following people for publising their datasets openly.

Ananiadou, S., Appenzeller-Herzog , C., Bannach-Brown, A., Beecham, S., Bekkers, V., Bos, D., Bowes, D., Budgen, D., Cohen, A. M., Counsell, S., Darweesh, S. K. L., Depaoli, S., Ewald H., Gray, D., Hagenaars, M. A., Hall, T., Heeres, M. L. S., Heričko, M., Hersh, W. R., Hofman, A., Houwen, R. H. J., Ikram, M. A., Kitchenham, B., Koopmans, M. P. G., Kwok, K. T. T., Liao, J., Linkman, S., Macleod, M. R., Mathes, T., Nagtegaal, R., Niazi, M., Nieuwenhuijse, D. F., Noordegraaf, M., Pearl Brereton, O., Peterson, K., Phan, M. V. T., Pretorius, R., Przybyla, P., Radjenović, D., Rice, A. S. C., Sabayan, B., Sedaghat, S., Segufa, R. A., Sijbrandij, M., Thomas, J., Torkar, R., Tummers, L., Turner, M., Vermunt, J. K., Vernooij, M. W., Wahono, R. S., Weiss, K. H., Winter, S. D., Wolters, F. J., Yen, P., de Wolf, F., et al., van Dis, E. A. M., van Veen, S. C., van de Schoot, R., Živkovič, A.

Contact details can be found at the ASReview project page.