Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests suite #120

Merged
merged 18 commits into from
Apr 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,7 @@ mypy
flake8
black
pre-commit

pytest==7.1.1
pytest-recording==0.12.0
python-decouple==3.6
57 changes: 57 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Test directory -- file structure

Directories:

1. `cassettes/`: Location of vcrpy and pytest-recording cassettes.

Files:

1. `conftest.py`: Location of pytest global and configuration fixtures.
2. `utils.py`: Location of global Python helper objects (i.e. constants and `Ozone` instance) to use in tests.
3. `test_*.py`: Test files, each file is testing one public method.

# Setting up and running tests

After setting development environment as pointed out in CONTRIBUTING.md, you should already have all necessary testing packages installed. To run all tests, invoke this command from the root directory.

```sh
pytest
```

# Updating tests

Generally, tests should correspond to the necessary specification/expectation of Ozone users. Tests will help us identify if our code is still in line with such expectations.

Tests should be updated when e.g.:
- There is a new functionality. In this case, add necessary tests accordingly.
- There is a change in existing functionality that is in line with the expectation of Ozone users. In this case, the tests become outdated and need to be updated.
- There is a new bug or previously unencountered or undocumented behavior. Add them to the existing tests to make sure the same bug will never slip past again in the future.

# About the cassettes and pytest-recording

This test suite is testing Ozone's functionality, so interaction with outside sources need to be mocked. Ozone uses pytest-recording plugin that uses vcrpy under the hood to record request-response pairs. These request-response pairs are stored as `.yaml` files in `tests/cassettes` directory.

The `test/cassettes` directory is organized as follows:

- Each folder per one test file.
- Each `.yaml` file per one test function.

For the purposes of this test suite, these request-response pairs are **assumed** to be all correct. In the event that WAQI API changes their specifications (unlikely), these request-response pairs need to be re-recorded. See the next section.

# Adding or updating cassettes

By default, pytest-recording will only use existing cassettes for testing, and will raise error if there's a new interaction that is not contained in existing cassettes. This prevents "accidentally" making a request that is not already been mocked.

This default behavior can be overridden by giving a `record-mode` argument when invoking `pytest`:

- `pytest --record-mode=once` to write new cassettes but not new request-response pairs within existing cassettes.
- `pytest --record-mode=new_episodes` to write new interactions, even when there's an existing cassette.
- `pytest --record-mode=rewrite` to rewrite all existing cassettes.
- `pytest --record-mode=all` to record all interactions again.
- `pytest --record-mode=none`(default) to only use existing cassettes and raise error if there's a new request that is not recorded in the existing cassettes.

> To have more confidence that the tests will not go over the wire, the `--block-network` flag can also be passed to block all network access.

For more information about pytest-recording:
- [pytest-recording homepage](https://github.com/kiwicom/pytest-recording)
- [VCRpy documentation about record modes](https://vcrpy.readthedocs.io/en/latest/usage.html#record-modes)
76 changes: 76 additions & 0 deletions tests/cassettes/ozone_init.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
interactions:
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed//london/?token=DUMMY_TOKEN
response:
body:
string: ''
headers:
Connection:
- keep-alive
Content-Length:
- '0'
Date:
- Thu, 21 Apr 2022 01:06:54 GMT
Location:
- /feed/london/?token=DUMMY_TOKEN
Server:
- nginx
status:
code: 301
message: Moved Permanently
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed/london/?token=DUMMY_TOKEN
response:
body:
string: '{"status":"ok","data":{"aqi":57,"idx":5724,"attributions":[{"url":"http:https://uk-air.defra.gov.uk/","name":"UK-AIR,
air quality information resource - Defra, UK","logo":"UK-Department-for-environment-food-and-rural-affairs.png"},{"url":"https://londonair.org.uk/","name":"London
Air Quality Network - Environmental Research Group, King''s College London","logo":"UK-London-Kings-College.png"},{"url":"https://waqi.info/","name":"World
Air Quality Index Project"}],"city":{"geo":[51.5073509,-0.1277583],"name":"London","url":"https://aqicn.org/city/london","location":""},"dominentpol":"pm25","iaqi":{"co":{"v":2.6},"h":{"v":78},"no2":{"v":14.8},"o3":{"v":26.6},"p":{"v":1015},"pm10":{"v":26},"pm25":{"v":57},"so2":{"v":0.6},"t":{"v":8.3},"w":{"v":3.6}},"time":{"s":"2022-04-21
00:00:00","tz":"+01:00","v":1650499200,"iso":"2022-04-21T00:00:00+01:00"},"forecast":{"daily":{"o3":[{"avg":19,"day":"2022-04-18","max":33,"min":8},{"avg":17,"day":"2022-04-19","max":34,"min":2},{"avg":25,"day":"2022-04-20","max":39,"min":14},{"avg":27,"day":"2022-04-21","max":36,"min":14},{"avg":23,"day":"2022-04-22","max":28,"min":17},{"avg":29,"day":"2022-04-23","max":37,"min":21},{"avg":36,"day":"2022-04-24","max":36,"min":32}],"pm10":[{"avg":18,"day":"2022-04-18","max":26,"min":10},{"avg":23,"day":"2022-04-19","max":31,"min":13},{"avg":19,"day":"2022-04-20","max":36,"min":10},{"avg":19,"day":"2022-04-21","max":28,"min":11},{"avg":18,"day":"2022-04-22","max":20,"min":14},{"avg":20,"day":"2022-04-23","max":23,"min":17},{"avg":16,"day":"2022-04-24","max":17,"min":16}],"pm25":[{"avg":47,"day":"2022-04-18","max":74,"min":23},{"avg":64,"day":"2022-04-19","max":81,"min":45},{"avg":59,"day":"2022-04-20","max":94,"min":33},{"avg":58,"day":"2022-04-21","max":79,"min":37},{"avg":56,"day":"2022-04-22","max":62,"min":46},{"avg":62,"day":"2022-04-23","max":70,"min":52},{"avg":53,"day":"2022-04-24","max":53,"min":52}],"uvi":[{"avg":1,"day":"2022-04-19","max":3,"min":0},{"avg":1,"day":"2022-04-20","max":4,"min":0},{"avg":1,"day":"2022-04-21","max":4,"min":0},{"avg":1,"day":"2022-04-22","max":3,"min":0},{"avg":1,"day":"2022-04-23","max":3,"min":0},{"avg":0,"day":"2022-04-24","max":2,"min":0},{"avg":0,"day":"2022-04-25","max":1,"min":0}]}},"debug":{"sync":"2022-04-21T09:24:45+09:00"}}}'
headers:
Access-Control-Allow-Origin:
- '*'
Connection:
- keep-alive
Content-Type:
- application/json; charset=UTF-8
Date:
- Thu, 21 Apr 2022 01:06:54 GMT
Server:
- nginx
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
X-Gen-Time:
- "123.812\xC2\xB5s"
X-Powered-By:
- rxstreamer-waqi/1.3
content-length:
- '2269'
status:
code: 200
message: OK
version: 1
138 changes: 138 additions & 0 deletions tests/cassettes/test_get_city_air/test_bad_city.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
interactions:
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed//a%20definitely%20nonexistent%20city/?token=DUMMY_TOKEN
response:
body:
string: ''
headers:
Connection:
- keep-alive
Content-Length:
- '0'
Date:
- Thu, 21 Apr 2022 01:07:03 GMT
Location:
- /feed/a%20definitely%20nonexistent%20city/?token=DUMMY_TOKEN
Server:
- nginx
status:
code: 301
message: Moved Permanently
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed/a%20definitely%20nonexistent%20city/?token=DUMMY_TOKEN
response:
body:
string: '{"status":"error","data":"Unknown station"}'
headers:
Access-Control-Allow-Origin:
- '*'
Connection:
- keep-alive
Content-Type:
- application/json; charset=UTF-8
Date:
- Thu, 21 Apr 2022 01:07:03 GMT
Server:
- nginx
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
X-Gen-Time:
- "137.712\xC2\xB5s"
X-Powered-By:
- rxstreamer-waqi/1.3
content-length:
- '43'
status:
code: 200
message: OK
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed///?token=DUMMY_TOKEN
response:
body:
string: ''
headers:
Connection:
- keep-alive
Content-Length:
- '0'
Date:
- Thu, 21 Apr 2022 01:07:04 GMT
Location:
- /feed/?token=DUMMY_TOKEN
Server:
- nginx
status:
code: 301
message: Moved Permanently
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed/?token=DUMMY_TOKEN
response:
body:
string: '{"status":"error","message":"404"}'
headers:
Access-Control-Allow-Origin:
- '*'
Connection:
- keep-alive
Content-Type:
- application/json; charset=UTF-8
Date:
- Thu, 21 Apr 2022 01:07:04 GMT
Server:
- nginx
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
content-length:
- '34'
status:
code: 200
message: OK
version: 1
76 changes: 76 additions & 0 deletions tests/cassettes/test_get_city_air/test_bad_data_format.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
interactions:
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed//london/?token=DUMMY_TOKEN
response:
body:
string: ''
headers:
Connection:
- keep-alive
Content-Length:
- '0'
Date:
- Thu, 21 Apr 2022 01:07:05 GMT
Location:
- /feed/london/?token=DUMMY_TOKEN
Server:
- nginx
status:
code: 301
message: Moved Permanently
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- python-requests/2.27.1
method: GET
uri: https://api.waqi.info/feed/london/?token=DUMMY_TOKEN
response:
body:
string: '{"status":"ok","data":{"aqi":57,"idx":5724,"attributions":[{"url":"http:https://uk-air.defra.gov.uk/","name":"UK-AIR,
air quality information resource - Defra, UK","logo":"UK-Department-for-environment-food-and-rural-affairs.png"},{"url":"https://londonair.org.uk/","name":"London
Air Quality Network - Environmental Research Group, King''s College London","logo":"UK-London-Kings-College.png"},{"url":"https://waqi.info/","name":"World
Air Quality Index Project"}],"city":{"geo":[51.5073509,-0.1277583],"name":"London","url":"https://aqicn.org/city/london","location":""},"dominentpol":"pm25","iaqi":{"co":{"v":2.6},"h":{"v":78},"no2":{"v":14.8},"o3":{"v":26.6},"p":{"v":1015},"pm10":{"v":26},"pm25":{"v":57},"so2":{"v":0.6},"t":{"v":8.3},"w":{"v":3.6}},"time":{"s":"2022-04-21
00:00:00","tz":"+01:00","v":1650499200,"iso":"2022-04-21T00:00:00+01:00"},"forecast":{"daily":{"o3":[{"avg":19,"day":"2022-04-18","max":33,"min":8},{"avg":17,"day":"2022-04-19","max":34,"min":2},{"avg":25,"day":"2022-04-20","max":39,"min":14},{"avg":27,"day":"2022-04-21","max":36,"min":14},{"avg":23,"day":"2022-04-22","max":28,"min":17},{"avg":29,"day":"2022-04-23","max":37,"min":21},{"avg":36,"day":"2022-04-24","max":36,"min":32}],"pm10":[{"avg":18,"day":"2022-04-18","max":26,"min":10},{"avg":23,"day":"2022-04-19","max":31,"min":13},{"avg":19,"day":"2022-04-20","max":36,"min":10},{"avg":19,"day":"2022-04-21","max":28,"min":11},{"avg":18,"day":"2022-04-22","max":20,"min":14},{"avg":20,"day":"2022-04-23","max":23,"min":17},{"avg":16,"day":"2022-04-24","max":17,"min":16}],"pm25":[{"avg":47,"day":"2022-04-18","max":74,"min":23},{"avg":64,"day":"2022-04-19","max":81,"min":45},{"avg":59,"day":"2022-04-20","max":94,"min":33},{"avg":58,"day":"2022-04-21","max":79,"min":37},{"avg":56,"day":"2022-04-22","max":62,"min":46},{"avg":62,"day":"2022-04-23","max":70,"min":52},{"avg":53,"day":"2022-04-24","max":53,"min":52}],"uvi":[{"avg":1,"day":"2022-04-19","max":3,"min":0},{"avg":1,"day":"2022-04-20","max":4,"min":0},{"avg":1,"day":"2022-04-21","max":4,"min":0},{"avg":1,"day":"2022-04-22","max":3,"min":0},{"avg":1,"day":"2022-04-23","max":3,"min":0},{"avg":0,"day":"2022-04-24","max":2,"min":0},{"avg":0,"day":"2022-04-25","max":1,"min":0}]}},"debug":{"sync":"2022-04-21T09:24:45+09:00"}}}'
headers:
Access-Control-Allow-Origin:
- '*'
Connection:
- keep-alive
Content-Type:
- application/json; charset=UTF-8
Date:
- Thu, 21 Apr 2022 01:07:06 GMT
Server:
- nginx
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
X-Gen-Time:
- "156.044\xC2\xB5s"
X-Powered-By:
- rxstreamer-waqi/1.3
content-length:
- '2269'
status:
code: 200
message: OK
version: 1
Loading