SYE

My Stat/CS SYE involving Clash Royale...
- The CS part is all about data collection and warehousing.
- The Stat component concerns data analysis.

CS

Refer to er_diagram.pdf for ER diagram of the PostgreSQL database.
create_tables.sql creates all tables involved (should be already created on host 10.32.95.90).
- If not yet created, run psql -U clashuser -h 10.32.95.90 -d clash -f create_tables.sql
Inspect the database with psql -U clashuser -h 10.32.95.90 -d clash

The goal is to migrate our original table (with less columns) to this new database.
The original table Clash resides in database Hotlogs (on host 10.32.95.90). I made a copy of it to database Clash.
cd data_migration
- data_migration_batch.sql is the migration script.
- It commits by batches of size 1 million rows.
- Run it with psql -U clashuser -h 10.32.95.90 -d clash -f data_migration_batch.sql

Open helpers.py
- In function email_admin, set variables sender_email and receiver_email to the email you want to use to be notified of any bugs.
- Preferably Gmail, created specifically for development.
- Turn on "allow less secure apps" in account settings.
cd credentials
- Create file psql-pass.txt
  - Contains the password for clashuser.
  - Just the password, no empty line.
- Create file dev-gmail-pass.txt
  - Contains the password for the email used to notify admin of any bugs.
  - Just the password, no empty line.
cd .. (back to directory CS/)
- Install dependencies: pip install -r requirements.txt
- Run collection script with python main.py
- If it is the first time you run this, it is expected to fail - in which case:
  - You will receive an email in your dev gmail you provided in a previou step.
  - It should include the following message: API key does not allow access from IP x.x.x.x
  - Log into Clash Royale API (https://developer.clashroyale.com/).
  - Create a new API key using the appropriate IP address x.x.x.x; copy the token.
  - Go to directory credentials cd credentials
  - Open api_keys.json
  - Add in the new token you just created; make sure it is the first one in the JSON file.
  - cd .. and python main.py again. It should work now.

Name		Name	Last commit message	Last commit date
Latest commit History 97 Commits
CS		CS
Stat		Stat
lee_email_nov29		lee_email_nov29
.gitignore		.gitignore
README.md		README.md