Skip to content

In this project you will scrape as much data as you can get about the *actual* sightings of Pokemons. As it turns out, players all around the world started reporting sightings of Pokemons and are logging them into a central repository (i.e. a database). We want to get this data so we can train our machine learning models. You will of course nee…

License

Notifications You must be signed in to change notification settings

PokemonGoers/PokeData

Repository files navigation

PokeData

Build StatusJoin the chat at https://gitter.im/pokemongoers/PokeData

API Documentation

A detailed documentation of the API is available here.

API Demo

A demo of our API can also be found here.

Data Sources

Data accessible through the API has been extracted from these sources:

Build pokemon database and listen for pokemon sightings

To run locally without docker

  • Clone this repo
  • To run locally, mongodb is required, use 'mongod' to do so (standard port 27017, no credentials)
  • change the script part of package.json to the following
  • then start listeners to fill the DB, build the pokemon collection (see below) and run "npm start" to expose the API on localhost:8080
"scripts": {
    "start" : "node app.js",
    "listen": "node scripts/listen.js",
    "build" : "node scripts/build.js",
    "test"  : "node test/main.js"
  }

Build pokemon database

npm run build --collection=pokemon

Listen for PokemonSightings

  • Listen for pokemon sightings
// data-source-name - rarePokemon, pokeRadar, skiplagged, pokecrew, fastpokemap, pokezz, pokedexs, pokemap
npm run listen -collection=<data-source-name>
  • Listen for Twitter pokemon sightings:
CONSUMER_KEY=<CONSUMER_KEY> CONSUMER_SECRET=<CONSUMER_SECRET> ACCESS_TOKEN=<ACCESS_TOKEN> ACCESS_TOKEN_SECRET=<ACCESS_TOKEN_SECRET> NODE_ENV=<NODE_ENV> npm run listen -collection=twitter

To run in production

  • To use mlab as a database, get the username, password, uri and the collection name.

Build pokemon database

MLAB_USERNAME=<MLAB_USERNAME> MLAB_PASSWORD=<MLAB_PASSWORD> MLAB_URI=<MLAB_URI> MLAB_COLLECTION=<MLAB_COLLECTION> npm run build -collection=pokemon

Listen for PokemonSightings

  • Listen for pokemon sightings
// data-source-name - rarePokemon, pokeRadar, skiplagged, pokecrew, fastpokemap, pokezz, pokedexs, pokemap
MLAB_USERNAME=<MLAB_USERNAME> MLAB_PASSWORD=<MLAB_PASSWORD> MLAB_URI=<MLAB_URI> MLAB_COLLECTION=<MLAB_COLLECTION> npm run listen -collection=<data-source-name>
  • Listen for Twitter pokemon sightings:
MLAB_USERNAME=<MLAB_USERNAME> MLAB_PASSWORD=<MLAB_PASSWORD> MLAB_URI=<MLAB_URI> MLAB_COLLECTION=<MLAB_COLLECTION> CONSUMER_KEY=<CONSUMER_KEY> CONSUMER_SECRET=<CONSUMER_SECRET> ACCESS_TOKEN=<ACCESS_TOKEN> ACCESS_TOKEN_SECRET=<ACCESS_TOKEN_SECRET> NODE_ENV=<NODE_ENV> npm run listen -collection=twitter

Docker Build

To build the project using Docker, follow these steps.

  1. Execute the following commands in the terminal.
#!/bin/bash

# Stop all containers
docker stop $(docker ps -a -q)

# Delete all containers
docker rm $(docker ps -a -q)

# Delete all images
docker rmi $(docker images -q)

# Run the latest build from the develop branch
docker run --env-file <PATH_TO_ENV_FILE> -p 49160:8080 --name <CONTAINER_NAME> 
           -d pokemongoers/pokedata:develop npm start

# Check the status of the container
docker ps
  1. The application can be accessed at the URL HOST_IP:49160.

##Config There is a configuration file config.js in the root folder. If you don't want to pass parameters for DB connection and twitter credentials everytime you can adapt it. Just replace the process.env.* with the desired value. The "database" key is development mode, "shared_database" for production mode and "twitter" holds the twitter information.

##Contributors

  • swathi-ssunder
  • samitsv
  • vivek-sethia
  • jonas-he
  • fabe85

About

In this project you will scrape as much data as you can get about the *actual* sightings of Pokemons. As it turns out, players all around the world started reporting sightings of Pokemons and are logging them into a central repository (i.e. a database). We want to get this data so we can train our machine learning models. You will of course nee…

Resources

License

Stars

Watchers

Forks

Packages

No packages published