The RMLMapper executes RML rules to generate Linked Data. It is a Java library, which is available via the command line (API docs online). The RMLMapper loads all data in memory, so be aware when working with big datasets.

Want to get started quickly? Check out Releases on where to find the latest CLI build as a jar, and see Usage on how to use the commandline interface!

• Features
  • Supported
  • Future
• Releases
• Build
• Usage
  • CLI
  • Library
  • Docker
  • Including functions
  • Generating metadata
• Testing
  • RDBs
• Dependencies
• Commercial Support
• Remarks
  • Typed spreadsheet files
  • XML file parsing performance
  • Language tag support
  • Duplicate removal and serialization format
  • I have a question! Where can I get help?
• Documentation
  • UML Diagrams

• local data sources:
  • Excel (.xlsx)
  • LibreOffice (.ods)
  • CSV files (including CSVW)
  • JSON files (JSONPath)
  • XML files (XPath)
• remote data sources:
  • relational databases (MySQL, PostgreSQL, Oracle, and SQLServer)
  • Web APIs with W3C Web of Things
  • SPARQL endpoints
  • files via HTTP urls (via GET)
    • CSV files
    • JSON files (JSONPath (@ can be used to select the current object.))
    • XML files (XPath)
• functions (most cases)
  • For examples on how to use functions within RML mapping documents, you can have a look at the RML+FnO test cases
• configuration file
• metadata generation
• output formats: nquads (default), turtle, trig, trix, jsonld, hdt
• join conditions
• targets:
  • local file
  • VoID dataset
  • SPARQL endpoint with SPARQL UPDATE

• functions (all cases)
• conditions (all cases)
• data sources:
  • NoSQL databases
  • TPF servers

The standalone jar file (that has a commandline interface) for every release can be found on the release's page on GitHub. You can find the latest release here. This is the recommended way to get started with RMLMapper. Do you want to build from source yourself? Check Build.

The RMLMapper is built using Maven. As it is also tested against Oracle (check here for details), it needs a specific set-up to run all tests. That's why we recommend to build without testing: mvn install -DskipTests=true. If you want, you can install with tests, and just skip the Oracle tests: mvn test -Dtest=!Mapper_OracleDB_Test.

A standalone jar can be found in /target.

Two jars are found in /target: a slim jar without bundled dependencies, and a standalone jar (suffixed with -all.jar) with all dependencies bundled.

The following options are most common.

• -m, --mapping <arg>: one or more mapping file paths and/or strings (multiple values are concatenated).
• -o, --output <arg>: path to output file
• -s,--serialization <arg>: serialization format (nquads (default), trig, trix, jsonld, hdt)

All options can be found when executing java -jar rmlmapper.jar --help, that output is found below.

usage: java -jar mapper.jar <options>
options:
 -c,--configfile <arg>               path to configuration file
 -d,--duplicates                     remove duplicates in the output
 -dsn,--r2rml-jdbcDSN <arg>          DSN of the database when using R2RML
                                     rules
 -e,--metadatafile <arg>             path to output metadata file
 -f,--functionfile <arg>             one or more function file paths (dynamic
                                     functions with relative paths are found
                                     relative to the cwd)
 -h,--help                           show help info
 -l,--metadataDetailLevel <arg>      generate metadata on given detail level
                                     (dataset - triple - term)
 -m,--mappingfile <arg>              one or more mapping file paths and/or
                                     strings (multiple values are
                                     concatenated). r2rml is converted to rml
                                     if needed using the r2rml arguments.
 -psd,--privatesecuritydata <arg>    one or more private security files 
                                     containing all private security 
                                     information such as usernames, passwords, 
                                     certificates, etc.
 -o,--outputfile <arg>               path to output file (default: stdout)
 -p,--r2rml-password <arg>           password of the database when using
                                     R2RML rules
 -s,--serialization <arg>            serialization format (nquads (default),
                                     turtle, trig, trix, jsonld, hdt)
 -t,--triplesmaps <arg>              IRIs of the triplesmaps that should be
                                     executed in order, split by ',' (default
                                     is all triplesmaps)
 -u,--r2rml-username <arg>           username of the database when using
                                     R2RML rules
 -v,--verbose                        show more details in debugging output
 --strict                            Enable strict mode. In strict mode, the 
                                     mapper will fail on invalid IRIs instead 
                                     of skipping them.
 -b --base-IRI <arg>                 base IRI used to expand relative IRIs in 
                                     mapped terms. If not set and not in --strict 
                                     mode, will default to the @base directive 
                                     inside the provided mapping file.
                                                                 

The W3C Web of Things Security Ontology is used to describe how Web APIs authentication should be performed but does not include the necessary credentials to access the Web API. These credentials can be supplied using the -psd <PATH> CLI argument. The PATH argument must point to one or more private security files which contain the necessary credentials to access the Web API.

An example can be found in the test cases src/test/resources/web-of-things.

An example of how you can use the RMLMapper as an external library can be found at ./src/test/java/be/ugent/rml/readme/ReadmeTest.java

We publish our Docker images automatically on Dockerhub for every release. You can find our images here: rmlio/rmlmapper-java.

You can use Docker to run the RMLMapper by following these steps:

• Build the Docker image: docker build -t rmlmapper ..
• Run a Docker container: docker run --rm -v $(pwd):/data rmlmapper -m mapping.ttl.

The same parameters are available as via the CLI. The RMLMapper is executed in the /data folder in the Docker container.

There are three ways to include (new) functions within the RML Mapper

• dynamic loading: you add links to java files or jar files, and those files are loaded dynamically at runtime
• preloading: you register functionality via code, and you need to rebuild the mapper to use that functionality
• add as dependency

Registration of functions is done using a Turtle file, which you can find in src/main/resources/functions.ttl

The snippet below for example links an fno:function to a library, provided by a jar-file (GrelFunctions.jar).

@prefix dcterms: <http:https://purl.org/dc/terms/> .
@prefix doap:    <http:https://usefulinc.com/ns/doap#> .
@prefix fno:     <https://w3id.org/function/ontology#> .
@prefix fnoi:    <https://w3id.org/function/vocabulary/implementation#> .
@prefix fnom:    <https://w3id.org/function/vocabulary/mapping#> .
@prefix grel:    <http:https://users.ugent.be/~bjdmeest/function/grel.ttl#> .
@prefix grelm:   <http:https://fno.io/grel/rmlmapping#> .
@prefix rdfs:    <http:https://www.w3.org/2000/01/rdf-schema#> .

grel:toUpperCase a fno:Function ;
  fno:name "to Uppercase" ;
  rdfs:label "to Uppercase" ;
  dcterms:description "Returns the input with all letters in upper case." ;
  fno:expects ( grel:valueParam ) ;
  fno:returns ( grel:stringOut ) .

grelm:javaString
    a                  fnoi:JavaClass ;
    doap:download-page "GrelFunctions.jar" ;
    fnoi:class-name    "io.fno.grel.StringFunctions" .

grelm:uppercaseMapping
    a                    fno:Mapping ;
    fno:function         grel:toUpperCase ;
    fno:implementation   grelm:javaString ;
    fno:methodMapping    [ a                fnom:StringMethodMapping ;
                           fnom:method-name "toUppercase" ] .

Just put the java or jar-file in the resources folder, at the root folder of the jar-location, or the parent folder of the jar-location, it will be found dynamically.

Note: the java or jar-files are found relative to the cwd. You can change the functions.ttl path (or use multiple functions.ttl paths) using a commandline-option (-f).

This overrides the dynamic loading. An example of how you can use Preload a custom function can be found at ./src/test/java/be/ugent/rml/readme/ReadmeFunctionTest.java

This is most interesting if you use RMLMapper as a library in your own project. Just add the dependency to the function library you want to use in your project.

You can also add a function library as a Maven dependency in pom.xml of RMLMapper. You'll have to rebuild RMLMapper to use it.

Conform to how it is described in the scientific paper [1], the RMLMapper allows to automatically generate PROV-O metadata. Specifically, you need the CLI arguments below. You can specify in which output file the metadata should be stored, and up to which level metadata should be stored (dataset, triple, or term level metadata).

 -e,--metadatafile <arg>          path to output metadata file
 -l,--metadataDetailLevel <arg>   generate metadata on given detail level
                                  (dataset - triple - term)

Run the tests via test.sh.

Right-click src/test/java directory and select "Run 'All tests'".

Some tests (Excel, ODS) are derived from other tests (CSV) using a script (./generate_spreadsheet_test_cases.sh)

Make sure you have Docker running. On Unix, others read-write permission (006) is required on /var/run/docker.sock in order to run the tests. The tests will fail otherwise, as Testcontainers can't spin up the container.

• A problem with Docker (can't start the container) causes the SQLServer tests to fail locally. These tests will always succeed locally.
• A problem with Docker (can't start the container) causes the PostgreSQL tests to fail locally on Windows 7 machines.

Do you need...

• training?
• specific features?
• different integrations?
• bugfixes, on your timeline?
• custom code, built by experts?
• commercial support and licensing?

You're welcome to contact us regarding on-premise, enterprise, and internal installations, integrations, and deployments.

We have commercial support available.

We also offer consulting for all-things-RML.

All spreadsheet files are as of yet regarded as plain CSV files. No type information like Currency, Date... is used.

The RMLMapper's XML parsing implementation (javax.xml.parsers) has been chosen to support full XPath. This implementation causes a large memory consumption (up to ten times larger than the original XML file size). However, the RMLMapper can be easily adapted to use a different XML parsing implementation that might be better suited for a specific use case.

The processor checks whether correct language tags are not, using a regular expression. The regex has no support for languages of length 5-8, but this currently only applies to 'qaa..qtz'.

Performance depends on the serialization format (--serialization <format>) and if duplicate removal is enabled (--duplicates). Experimenting with various configurations may lead to better performance for your use case.

Do you have any question related to writing RML mapping rules, the RML specification, etc., feel free to ask them here: https://github.com/kg-construct/rml-questions ! If you have found a bug or need a feature for the RMLMapper itself, you can make an issue in this repository.

Generate static files at /docs/apidocs with:

mvn javadoc:javadoc

(Requires Ultimate edition)

• Right click on package: "be.ugent.rml"
• Diagrams > Show Diagram > Java Class Diagrams
• Choose what properties of the classes you want to show in the upper left corner
• Export to file > .png | Save diagram > .uml

• Go to draw.io
• Click on 'Open Existing Diagram' and choose the .html file

[1]: A. Dimou, T. De Nies, R. Verborgh, E. Mannens, P. Mechant, and R. Van de Walle, “Automated metadata generation for linked data generation and publishing workflows,” in Proceedings of the 9th Workshop on Linked Data on the Web, Montreal, Canada, 2016, pp. 1–10. PDF