Skip to content

Subscribes to a Kafka topic and deposits messages to the US DOT Situation Data Warehouse (SDW).

Notifications You must be signed in to change notification settings

usdot-jpo-ode/jpo-sdw-depositor

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

47 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

jpo-sdw-depositor Build Status Quality Gate Status

Subscribes to a Kafka topic and deposits messages to the Situation Data Warehouse (SDW).

Overview

This is a submodule of the jpo-ode repository. It subscribes to a Kafka topic and listens for incoming messages. Upon message arrival, this application deposits it over REST to the SDW.

Installation and Operation

Requirements

  • Docker

Option 1: Standalone

Use this option when you want to run the depositor by itself. This will listen to any Kafka topic you specify and deposit messages to the Situation Data Warehouse.

  1. Configure your desired properties. See Configuration Reference at the bottom of this README.
  2. Rename your sample.env file to .env if you haven't already done so
  3. Execute the run.sh script OR execute these commands:
docker build -t jpo-sdw-depositor . 
docker run --rm  --env-file .env jpo-sdw-depositor:latest

Option 2: As ODE submodule

** IN PROGRESS! Further instructions pending ODE compatibility. **

Use this option when you want to run this module in conjuction with the jpo-ode. The only action you must take here is to set the configuration properties in the env file. See the bottom of this README for a reference.

Configuration Reference

SOME OF THESE PROPERTIES ARE SENSITIVE. DO NOT PUBLISH THEM TO VERSION CONTROL

You may configure these values in jpo-sdw-depositor/src/main/resources/application.properties or by editing them in the sample.env file.

IMPORTANT When using the env file method, you must You must rename or duplicate the sample.env file to .env

Value in application.properties Value as env var (in sample.env) Description Example Value
sdw.kafkaBrokers DOCKER_HOST_IP Host IP (instructions) 10.1.2.3
sdw.kafkaPort SDW_KAFKA_PORT Port of the Kafka instance 9092
sdw.subscriptionTopic SDW_SUBSCRIPTION_TOPIC Kafka topic to listen to topic.J2735TimBroadcastJson
sdw.destinationUrl SDW_DESTINATION_URL Full path of the SDW server address 127.0.0.1
sdw.username SDW_USERNAME SDW HTTP basic auth username (n/a)
sdw.password SDW_PASSWORD SDW HTTP basic auth password (n/a)