import_br.py works locally, but fails in Github Actions

[augusto-herrmann]

Job "Update data from sources (br)" fails with the following error:

Traceback (most recent call last):
  File "scripts/import/br/import_br.py", line 384, in <module>
    import_br_data(URL, args.output)
  File "scripts/import/br/import_br.py", line 251, in import_br_data
    municipios_response = session.get(URL_MUNICIPIOS)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='servicodados.ibge.gov.br', port=443): Max retries exceeded with url: /api/v1/localidades/municipios (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1131)')))

The script works locally, though. Also if one is to run the following code locally, the download from ibge.gov.br will proceed normally, without raising an exception:

import requests

USER_AGENT = 'PublicBodiesBot (https://github.com/okfn/publicbodies)'
URL_MUNICIPIOS = 'https://servicodados.ibge.gov.br/api/v1/localidades/municipios'

session = requests.Session()
session.headers.update({'User-Agent': USER_AGENT})

municipios_response = session.get(URL_MUNICIPIOS)

It seems like it's common for the Github Actions environment to give SSL errors when connecting to government websites, as evidenced by these Stack Overflow questions:

• SSL Error in Python Requests run from Github's Actions (www.psp.cz – Czech Republic's parliament)
• SSL error unsafe legacy renegotiation disabled (ssd.jpl.nasa.gov – Jet Propulsion Lab, NASA)

From there, we could:

1. Implement the CustomHttpAdapter workaround from Stack Overflow; or
2. some commenter suggests using wget; or
3. stop updating the municipality list and remove this call from the script.

For now, I'm pending towards solution 3, since the list of municipalities in Brazil does not update that often.

[augusto-herrmann]

The IBGE data is needed because the SIORG API, which returns Brazilian Federal Government public bodies information, provides an address but uses only municipality codes, not municipality names.

So if we are going to use option 3 we need to cache somehow (perhaps a small csv in this repository) a mapping of municipality codes to names.

[augusto-herrmann]

Perhaps this csv could be fetched from Github and used instead.