Implemented code would be a nice example of IoT Hub WebAPIs and simulated SmartHome sensors. This project is an assignment.
IoT Hub provides:
-
Virtual IoT sensors.
-
Virtual IoT Gateway that communicate using light-weight MQTT protocol.
-
Sensors data pipeline, Sensor → MQTT Broker → Kafka → InfluxDB.
-
WebAPIs to query metrics.
-
Authentication and authorization is not taking into consideration.
-
kafka-connect-influxdb-sinkcould be replaced with open-source Kafka connector. -
WebAPI in
SensorDataQueryAPIis highly opinionated. Need better API design.
-
kafka-connect-influxdb-sink: Kafka InfluxDB Sink. -
virtual-device-simulator: Simulated IoT sensors and Gateway. -
metrics-query-service: Query on stored InfluxDB metrics.
-
Improve architectural design, with more added code, pattern should appear.
-
Code improvements:
-
Used
ObserverandObservableare deprecated, need to find alternative. -
MinValueQueryServiceis not implemented yet, but on purpose, just have to followMinValueQueryService. -
Need to use more specific exception class, instead of
RuntimeException. -
Add end-to-end tests. Cover more unit tests.
-
Improve logging.
-
-
Generate and check OWASP report to find vulnerable component.
-
Write Gatling performance tests.
-
virtual-device-simulator: Virtual sensors and gateway, produces MQTT message and post to Mosquitto. -
mosquitto: MQTT Broker. -
kafka-zookeeper: Kafka zookeeper. -
kafka: Event / Message bus. -
kafdrop: UI to administer Kafka. -
simple-kafka-mqtt-connector: MQTT to Kafka Connector. -
influxdb: Time series database. -
chronograf: Time-Series Data Visualization. -
kafka-connect-influxdb-sink: Kafka to InfluxDB Connector.
$ docker-compose up # Make sure all services are running
$ telnet localhost 8086 # Verify InfluxDB is running.$ telnet localhost 8086 # Verify InfluxDB is running.
$ java -jar metrics-query-service/target/metrics-query-service.jaror
$ cd metrics-query-service
$ mvn spring-boot:run$ telnet localhost 8080 # Verify metrics-query-service is running.-
Should able to see Kafka topics at
http:https://docker-ip:9000/. -
Should able to see InfluxDB data at
http:https://docker-ip:8888/sources/0/chronograf/data-explorer.
Sample request:
curl --location --request POST 'localhost:8080/api/devices/metrics/aggregate?aggregateOperation=max&aggregateField=deviceData.temperature&deviceGroup=TemperatureSensor&startTime=1183495454&endTime=2583495454'example response:
{
"query": {
"operation": "max",
"deviceId": null,
"deviceGroup": "TemperatureSensor",
"startTime": 1183495454,
"endTime": 2583495454,
"additionalParams": {
"aggregateField": "deviceData.temperature"
}
},
"response": {
"max": 44.903558920361355
}
}To run the unit tests, execute the following commands
mvn clean test-compile testTo run the integration tests, execute the following commands
mvn clean test-compile verify -DskipTests=trueTo run the integration tests, execute the following commands
mvn clean test-compile verifyTo run the mutation tests, execute the following commands
mvn clean install test-compile test
mvn org.pitest:pitest-maven:mutationCoverage$ cd kafka-connect-influxdb-sink
$ mvn clean compile package
$ docker build -t mmahmood/kafka-connect-influxdb-sink:latest -f Dockerfile .
$ docker push mmahmood/kafka-connect-influxdb-sink:latestLicensed under the MIT License, see the LICENSE file for details.