R COBOL DI (Data Integration) Package: A package for importing COBOL CopyBook data directly into the R Project as data frames.
The RCOBOLDI Package (COBOL Data Integration Package for the R Project for Statistical Computing) is an R package that facilitates the importation of COBOL CopyBook data directly into R as properly structured data frames.
Note that not all copybook files can be converted into CSV -- for example single-record type files can be converted to CSV however complicated multi-record type files will NOT map to CSV.
Example One: Install the RCOBOLDI package locally and then convert COBOL data files into data frames.
In this simple example the R COBOL Data Integration package has been installed locally and several data files are converted into data frames using the ReadCopyBookAsDataFrame function; the CobolToCSV function is also demonstrated.
This example also how includes a call to CobolToCSV
.
The following should be executed prior to attempting to run the R script, below.
- R version 3.6.3
- Java 11
- install.packages(c("drat", "RJSONIO", "rJava"))
If you have trouble with rJava and Java 11 then you might need to execute the following:
R CMD javareconf
The following example should work with the only change needed being the path to the files.
Test files can be found here.
library(drat)
drat::addRepo("thospfuller")
# The following should work for Mac OSX (El Capitan) and on Linux.
install.packages("RCOBOLDI")
# Or install via source (for Windows users, specifically).
install.packages("RCOBOLDI", type = "source")
library("RCOBOLDI")
RCOBOLDI::Initialize()
#
# Substitute the directory path below with one which points to the test files being used.
#
# The test files below can be found here:
#
# https://github.com/thospfuller/rcoboldi/tree/master/java/rcoboldi-core/src/test/resources
#
result <- RCOBOLDI::ReadCopyBookAsDataFrame(".../rcoboldi/java/rcoboldi-core/src/test/resources/example1/DTAR020.cbl", "/Users/thospfuller/development/projects/rcoboldi/java/rcoboldi-core/src/test/resources/example1/DTAR020.bin", "Fixed Length Binary", "cp037")
head(result)
result <- RCOBOLDI::ReadCopyBookAsDataFrame(".../rcoboldi/java/rcoboldi-core/src/test/resources/example2/DTAR107.cbl", "/Users/thospfuller/development/projects/rcoboldi/java/rcoboldi-core/src/test/resources/example2/DTAR107.bin", "Fixed Length Binary", "cp037")
head(result)
result <- RCOBOLDI::ReadCopyBookAsDataFrame(".../rcoboldi/java/rcoboldi-core/src/test/resources/example3/AmsLocation.cbl", "/Users/thospfuller/development/projects/rcoboldi/java/rcoboldi-core/src/test/resources/example3/Ams_LocDownload_20041228.txt", "Text", "cp1252")
cobrix_test1_result <- RCOBOLDI::ReadCopyBookAsDataFrame(".../rcoboldi-gh/rcoboldi/java/rcoboldi-core/src/test/resources/example4/absaoss_cobrix_test1_copybook.cob", "/Users/thospfuller/development/projects/rcoboldi-gh/rcoboldi/java/rcoboldi-core/src/test/resources/example4/absaoss_cobrix_test1_example.bin", "**TBD**", "**TBD**")
head(cobrix_test1_result)
result <- RCOBOLDI::ReadCopyBookAsDataFrame(".../rcoboldi-gh/rcoboldi/java/rcoboldi-core/src/test/resources/example4/absaoss_cobrix_test1_copybook.cob", "/Users/thospfuller/development/projects/rcoboldi-gh/rcoboldi/java/rcoboldi-core/src/test/resources/example4/absaoss_cobrix_test1_example.bin", "Fixed Length Binary", "cp037")
#
# The following line will convert the absaoss_cobrix_test1 data file into a CSV file.
#
RCOBOLDI::CobolToCSV("-I .../rcoboldi/java/rcoboldi-core/src/test/resources/example4/absaoss_cobrix_test1_example.bin -C .../rcoboldi/java/rcoboldi-core/src/test/resources/example4/absaoss_cobrix_test1_copybook.cob -FS Fixed_Length -IC cp037 -O .../temp/absaoss_cobrix_test1.csv")
Try the R COBOL Data Integration (RCOBOLDI) package via a browser using a preconfigured Docker image available on DockerHub
Below we include two examples for working with images in Docker along with a demonstration video showing the package in action. If you just want to try the package on some test data, start here.
The RCOBOLDI Rocker/RStudio image is available on DockerHub. This image is built using dependencies which have been deployed in the thospfuller/drat repository.
The RCOBOLDI Rocker/Verse image is a complete build using source code cloned from the GitHub project repository.
The following is a fully working example based on the Rocker/RStudio image.
docker pull thospfuller/rcoboldi-rockstar-rstudio:[latest or 1.0.3 or some other tag]
then
docker image ls
should show something like this:
> REPOSITORY TAG IMAGE ID CREATED SIZE
> thospfuller/rcoboldi-rockstar-rstudio 1.0.3 3f8c1c269940 37 minutes ago 2.42GB
then
docker run -d -p 8787:8787 -e PASSWORD=password --name rstudio -i -t 3f8c1c269940
The next is to test this in R so point your browser to https://localhost:8787 and use "rstudio" and "password" to login and then execute the following:
library(RCOBOLDI)
RCOBOLDI::Initialize()
result <- RCOBOLDI::ReadCopyBookAsDataFrame("/home/rstudio/rcoboldi/java/rcoboldi-core/src/test/resources/example1/DTAR020.cbl", "/home/rstudio/rcoboldi/java/rcoboldi-core/src/test/resources/example1/DTAR020.bin", "Fixed Length Binary", "cp037")
head(result)
docker build -t rcoboldi/rocker-rstudio:1.0.1 https://raw.githubusercontent.com/thospfuller/rcoboldi/master/docker/rocker/rstudio/Dockerfile
[some image id]
Step 2.) Run the R COBOL Data Integration (RCOBOLDI) container based on the image created in the previous step
docker run -d -p 8787:8787 -e PASSWORD=password --name rstudio -i -t [some image id]
[some container id]
Step 3.) Browse to https://localhost:8787 and enter the username & password combination rstudio & password.
The next three steps appear in the video.
Steps 4-7.) From the R CLI execute:
#### Step 4.)
library(RCOBOLDI)
#### Step 5.)
RCOBOLDI::Initialize()
#### Step 6.)
result <- RCOBOLDI::ReadCopyBookAsDataFrame("DTAR020.cbl", "DTAR020.bin", "Fixed Length Binary", "cp037")
#### Step 7.)
head(result)
If you're looking to load COBOL data files into the R Project for Statistical Computing then you came to the right place.
The Dockerfile in this example is based on Rocker / RStudio and will be running locally on port 8787.
Below is a video which demonstrates the R COBOL Data Integration package running in Docker.
The Java API uses Log4J and writes files to the ~/rcoboldi-package-logs/ directory. The Log4J configuration file can be found here.
- JRecord: Read Cobol data files in Java on SourceForge
- JRecord: Read Cobol data files in Java on GitHub
- AbsaOSS cobrix: A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
- (DataBricks) Cobrix: A Mainframe Data Source for Spark SQL and Streaming
- (YouTube) Cobrix: A Mainframe Data Source for Spark SQL and Streaming
- ProLeap ANTLR4-based parser for COBOL
- zenaptix-lab / copybookStreams
- EBCDIC on Wikipedia
- Cobol hits 50 and keeps counting
- ProLeap ANTLR4-based parser for COBOL
- EBCDIC Character Format - A Guide
- PyPI EBCDIC
- Brush up your COBOL: Why is a 60 year old language suddenly in demand?
- COBOL Is Everywhere. Who Will Maintain It?
- COBOL Copybook Example on Oracle.com
- COBOL copy books on IBM.com
- COBOL Copybook Format on MuleSoft.com