civic-scraper helps download agendas, minutes and other documents produced by government. It includes a command-line tool and reusable Python code to scrape a growing number of public agency websites.
- Documentation: https://civic-scraper.readthedocs.io/en/latest/
- GitHub: https://github.com/biglocalnews/civic-scraper
- PyPI: https://pypi.python.org/pypi/civic-scraper
- Free and open source software: Apache license
Install civic-scraper:
pip install civic-scraper
...and start scraping from the command line:
# Scrape today's agendas and minutes from a CivicPlus site civic-scraper scrape --download --url https://nc-nashcounty.civicplus.com/AgendaCenter
Or in a script:
# Scrape today's agendas and minutes from a CivicPlus site from civic_scraper.platforms import CivicPlusSite url = "https://nc-nashcounty.civicplus.com/AgendaCenter" site = CivicPlusSite(url) site.scrape(download=True)
Note
There are many more options for customizing scrapes, especially by date range. Check out the :ref:`usage` docs for details.
See the :ref:`install` docs to configure the download location.