Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/add geolocation #764

Merged
merged 13 commits into from
Sep 23, 2022
Merged

Feature/add geolocation #764

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
3122dc1
Add lat long fields
ubergesundheit Sep 23, 2022
faaea63
Merge branch 'master' into add-geo-fields
TobiRoby Sep 23, 2022
31d8c70
add location parsing via geopy
TobiRoby Sep 23, 2022
5a3f81b
add dependencies and document installation
top-on Sep 23, 2022
e28c34e
fix path
top-on Sep 23, 2022
fc785ac
Merge branch 'master' into feature/python-dependency-management
top-on Sep 23, 2022
74bf1d3
Merge branch 'master' into feature/add_geolocation
top-on Sep 23, 2022
b1a33c6
Merge branch 'feature/python-dependency-management' into feature/add_…
top-on Sep 23, 2022
3e33634
apply black
top-on Sep 23, 2022
dfcf01e
Merge branch 'master' into feature/add_geolocation
top-on Sep 23, 2022
c3d724c
fix flake8 error
top-on Sep 23, 2022
da3ea12
only geocode if location is not in DB yet
top-on Sep 23, 2022
bf4156d
Update backend/scraping/spiders/__init__.py
top-on Sep 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
apply black
  • Loading branch information
@top-on
top-on committed Sep 23, 2022
commit 3e336341b9b40222f9e23274adb64813ebeaff58
4 changes: 3 additions & 1 deletion backend/scraping/spiders/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ def process_item(self, item, spider): # skipcq: PYL-W0613


from geopy.geocoders import Nominatim


class DatabaseExportPipeline:
def process_item(self, item, spider):
with transaction.atomic():
Expand All @@ -76,7 +78,7 @@ def process_item(self, item, spider):
values["location"], _ = Location.objects.get_or_create(
description=item["location"],
lat=location.latitude,
lon=location.longitude
lon=location.longitude,
)
if item["organizer"]:
values["organizer"], _ = Organizer.objects.get_or_create(
Expand Down