Skip to content

Home

Jump to bottom
Jim Mullen edited this page Dec 6, 2019 · 18 revisions

REDCap-ETL External Module Roadmap

Features Currently Being Implemented

  • SQL Server Support - add MS SQL Server as a supported database system type for loading data extracted from REDCap.
  • External Module Logging - Enhanced logging to the REDCap external module logging tables with an admin user interface to access the results. Currently, some high-level logging is done to the standard REDCap events log, and detailed logging can be done to the file system of REDCap-ETL servers. However, REDCap admins will not necessarily have access to the file system of REDCap-ETL servers.

Features Currently Planned for Future

  • ETL Workflows - support for combining multiple ETL configurations into a single unit of work that can be run interactively or scheduled to run on a regular basis. The primary motivation for this is to provide support for loading of data extracted from multiple REDCap projects into a single database
  • Loading Extracted Data to Files - currently, the standalone REDCap-ETL application supports the loading of data extracted from REDCap into a SQLite database file and CSV (Comma Separated Values) files. However, the REDCap-ETL external module does not support this, because there is an issue as to where to store the files. We are looking at the following options:
    • HTTP File Download - support download of SQLite database file or CSV zip file using HTTP file download. This will only work for the embedded server when running interactively (i.e., it will not work for remote REDCap-ETL servers and will not work for ETL processes that are scheduled to run).
    • REDCap Project File Repository - support storage of files in the REDCap project file repository. Currently we cannot implement this due to lack of support within REDCap. For us to implement this generally, an API method would need to be added to REDCap that supports this. We have requested this: https://community.projectredcap.org/idea/70810/provide-api-method-for-importing-a-file-to-a-proje.html. In addition, an API token with import/update rights, in addition to export rights, be created.
    • REDCap Project File Field - support storage of files in fields in the REDCap project. This would require that a file field be created in the project to store the file, and an API token with import/update rights, in addition to export rights, be created.
    • Cloud Storage - support for storing files in cloud storage, for example, Google Drive.
Clone this wiki locally