Skip to content
This repository has been archived by the owner on Nov 15, 2019. It is now read-only.

🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python

License

Notifications You must be signed in to change notification settings

florimondmanca/kafka-fraud-detector

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Kafka Fraud Detector

Kafka Docker Images Python

This is the supporting repository for my blog post: Building A Streaming Fraud Detection System With Kafka And Python.

Install

This fraud detection system is fully containerised. You will need Docker and Docker Compose to run it.

You simply need to create a Docker network called kafka-network to enable communication between the Kafka cluster and the apps:

$ docker network create kafka-network

All set!

Quickstart

  • Spin up the local single-node Kafka cluster (will run in the background):
$ docker-compose -f docker-compose.kafka.yml up -d
  • Check the cluster is up and running (wait for "started" to show up):
$ docker-compose -f docker-compose.kafka.yml logs -f broker | grep "started"
  • Start the transaction generator and the fraud detector (will run in the background):
$ docker-compose up -d

Usage

Show a stream of transactions in the topic T (optionally add --from-beginning):

$ docker-compose -f docker-compose.kafka.yml exec broker kafka-console-consumer --bootstrap-server localhost:9092 --topic T

Topics:

  • queuing.transactions: raw generated transactions
  • streaming.transactions.legit: legit transactions
  • streaming.transactions.fraud: suspicious transactions

Example transaction message:

{"source": "yGfZ1Xa6k1r0", "target": "N5RvY7RO5sQF", "amount": 217.46, "currency": "EUR"}

Teardown

To stop the transaction generator and fraud detector:

$ docker-compose down

To stop the Kafka cluster (use down instead to also remove contents of the topics):

$ docker-compose -f docker-compose.kafka.yml stop

To remove the Docker network:

$ docker network rm kafka-network

About

🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published