Flock loads real Twitter streams into Dgraph to make use of graph travesals.
Use Discuss Issues for reporting issues about this repository.
Flock has two parts,
- Tweet loader - Connects to realtime Tweets via the Twitter Developer API and loads a graph model of Twitter into Dgraph via mutations.
- Query client - Runs interesting graph queries on the Tweets data stored in Dgraph.
Here is the graph schema of Flock:
We need to create a Twitter developer account and an app to be able to fetch stream of Tweets using their APIs. Let's start with how to create a Twitter developer account.
- Apply for a Twitter developer account here and follow the instructions. The series of steps would end with your email verification.
- Create a Twitter app from this link.
All fields are
not
required. - You'll be redirected to the App details tab after creating the app. Go to the
Keys and tokens
tab and create new access and secret tokens. - Create a copy of the credentials template.
cp credentials-template.json credentials.json
- Open the
crendentials.json
file and replace the placeholders with the keys from the Twitter app'sKeys and token
tab.
- Clone the repository.
$ git clone https://github.com/dgraph-io/flock.git
$ cd flock
- Export the persistent data directory. Since Dgraph is run using Docker containers, it is nice to mount a directory on the host machine to persist the data across multiple runs.
$ mkdir ./data
$ export DATA_DIR=$(pwd)/data
- If you're running Linux, you can add the current user to the
docker
group to use Docker as a non-root user.newgrp
creates a new terminal session. It is necessary after the user modification to see the effects.
$ sudo usermod -aG docker $USER
$ newgrp docker
- Start the Dgraph servers and Ratel with Docker Compose. Visit https://localhost:8000 on your browser to view the UI.
$ docker-compose up
Currently, we have Flock implemention for Go and JavaScript.
Go to the specific Flock documentation for instructions: