-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Continuous Testing. Integrate Ozone's test suite with a continuous testing feature to indicate if build is passing or not #130
Comments
maybe you can also use codecov.io to show test coverage |
Note: with #132 and #133 I think the tests should pass. I've tested the two branches separately on my device -- and I think, combined they shall pass all tests (I'm not 100% sure). You may want to check locally to confirm this after they got merged later @Milind220. |
@lahdjirayhan OH awesome! that's excellent :) I'll check them out soon! EDIT: Hey, so I ran the tests again, and I'm getting 2 failed still. Here's the summary that I see: |
@Milind220 Yeah I've got them too. They fail not because of the tests, but because a request in the test is not found in the test's corresponding cassette file. By default, pytest-recording uses Solution is simple: run the following commands in your device: # Check that the tests pass if pytest-recording is not used at all, i.e. the test uses live connection to WAQI
pytest tests/test_get_city_forecast.py tests/test_get_coordinate_air.py --disable-recording # Run the tests using live connection to WAQI, and rewrite the cassettes
pytest tests/test_get_city_forecast.py tests/test_get_coordinate_air.py --record-mode=rewrite # Save the new cassettes into version control
git commit tests/cassettes -m "test: Re-record cassette" Running the first command gives me this in my device (all the previously failing tests pass):
|
Unrelated to the issue but maybe later we need to rethink how to design the tests so that fixing to pass a failing test won't require re-recording a vcrpy cassette (because it's a hassle and frankly not intuitive for starters). Re-recording a cassette also shouldn't happen too often. EDIT: Nope after more consideration and careful thoughts I came to conclude we can't just design away our way from re-recording every time a test changes from failing to passing. Or maybe I didn't think hard enough. |
@lahdjirayhan I'm not too well versed with this actually, so I'll do some research too! |
Do you still want to implement continuous testing/CI as this post suggests. This is new to me with Github, but I was having a read around Github actions which seems to be a good feature to implement CI/CD, have you considered this? |
Yes, continuous testing is still a priority for Ozone. I've never implemented it before though, so I'm not sure what the best way to proceed is. I did notice that GitHub Actions seems to have some options for this. If you'd like to give it a try, that'd be really awesome! Even just some research into this would be super helpful! |
I think GitHub actions and something like codecov.io for showing test coverage and badge in readme will be a good option. This pipeline should be complicated but I'm not sure how cassette works. |
@ShootGan codecov + GitHub Actions sounds perfect. I don't think we need pylint in the continuous testing pipeline. Just the tests should be plenty. |
Is anyone working on this? @ShootGan @Miller2014 @Milind220 I want to work on this if no one does yet. |
Hi, sorry right now i don't have enough time to do this but I can send a similar pipeline yaml. |
@ShootGan No worries! We'll always have more issues to work on hahaha. Thanks for your help :) |
Sorry for the lack of response on this, glad to see you implemented this, great job!. I will keep an eye out for any new issues. |
Current situation
Ozone now has a test suite. The tests can be manually run by running pytest in the root directory of Ozone.
Desired situation
Ozone should have a continuous testing feature, and a badge/shield to go with it in the README. This way, with a single glance, we can tell that Ozone's working perfectly and without errors.
The text was updated successfully, but these errors were encountered: