Skip to content

tommy3531/PythonDataScience

Repository files navigation

Overview

This project is centered around data mining and parse json. I am using data from propublica, NewsAPI, Opensecrets, Trade.gov, CKan. I make a request to each API create a model from the json response and then create a python object from the parsed Json Response. I will use pandas and numpy to perform calculations on the data and persent the data to the use.

Third Party Libraries

  1. NewsAPI - get news articles
  2. JsonTraverseParser - parse json easier
  3. scrapy - creating spiders
  4. pyfolio - stock risk analysis
  5. requests - REST Calls
  6. gensim - NLP
  7. scapy - NLP
  8. NLTK- NPL toolkit
  9. newspaper - extract articles and information from website
  10. Civic Information - political data
  11. Political - political data
  12. Youtube - videos

Documentation

  1. NewsAPI - https://newsapi.org/docs
  2. UnData - https://data.un.org/ws/
  3. Opensecrets - https://www.opensecrets.org/api/admin/index.php?function=user_api_list
  4. Scrapy - https://docs.scrapy.org/en/latest/
  5. pyfolio - https://quantopian.github.io/pyfolio/
  6. gensim - https://radimrehurek.com/gensim/tutorial.html
  7. scapy - https://spacy.io/api/doc
  8. newspaper - https://github.com/codelucas/newspaper
  9. Civic Information - https://developers.google.com/civic-information/docs/v2/
  10. Youtube - https://developers.google.com/youtube/v3/code_samples/python

Helpful Links

  1. Open Data - https://project-open-data.cio.gov/

Pyenv

source ~/.bashrc set pycharm to use 3.5.2, checking setting in pycharm and make sure it is using pyenv version. check local version of python, which python.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published