Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scraped data --> to --> aws #243

Closed
sashamaryl opened this issue Apr 19, 2022 · 2 comments
Closed

scraped data --> to --> aws #243

sashamaryl opened this issue Apr 19, 2022 · 2 comments
Assignees
@mikeyavorsky @sashamaryl

Comments

@sashamaryl
Copy link
Collaborator

sashamaryl commented Apr 19, 2022 •
edited
Loading

run the scrapers and load the data into aws

// @jeff & @mikeyavorsky please fill in the details from the convo in the meeting just now...
@mikeyavorsky
Copy link
Collaborator

We need to populate the database in the dev environment.

The scraper might be able to run locally with ENV variables pointing to AWS database.

@jigsawlabs-student
Copy link

jigsawlabs-student commented Apr 20, 2022 •
edited
Loading

@sashamaryl

  1. Create an RDS database in AWS
  2. Create .env file to connect RDS database to the psycopg2 client
  3. db_url = f"postgresql:https://{DB_USER}:{DB_PASSWORD}@{DB_HOST}/{DB_NAME}"
  4. Run scraper to populate RDS database

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mikeyavorsky mikeyavorsky

@sashamaryl sashamaryl

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mikeyavorsky @DMalone87 @sashamaryl @jigsawlabs-student