Skip to content
This repository has been archived by the owner on Mar 13, 2024. It is now read-only.

Adding label for PR smoke tests #5689

Merged
merged 4 commits into from
Jun 10, 2020

Conversation

prapti
Copy link
Contributor

@prapti prapti commented Jun 9, 2020

Adding @smoke tags to 7 pre-existing E2E tests for PR smoke test, which takes roughly 5 minutes to run.
This PR covers smoke tests in the following E2E test files and were selected based on preliminary review of the Smoke Tests doc.

  • User can create a new team -- create_a_team_spec
  • User can create a channel -- accessibility_sidebar_spec
  • Users can join an existing channel -- add_users_to_channel_spec
  • User can post in a channel -- message_spec
  • Basic search function -- search_user_post
  • System administrator can access system console -- demoted_user_spec (changed this from cluster_spec to add a more consistently passing test)
  • User can log out -- tutorial_navigation_and_links_spec

To run the tests, please run the following:
node run_tests.js --group='@smoke'

@saturninoabril
Copy link
Member

@prapti Looks good to me. Just few requests:

  • add more details in the description like how many test cases were initially selected and the duration it takes to run the test
  • can't see the "Smoke Tests" doc mentioned in the description. Can you link it here for reference? The only reference I have is the one you posted in our GM
  • please add follow up documentation to https://developers.mattermost.com/contribute/webapp/end-to-end-tests/ on how it's selected, what are the criteria and how to run like should it be with args --group="@smoke" only or in combination with prod --stage="@prod" --group="@smoke"
  • can you confirm that all are consistently passing? The last time I checked was marketplace_spec still have failing tests that need to attend to.

@prapti prapti changed the title Adding label for PR smoke tests WIP - Adding label for PR smoke tests Jun 9, 2020
@prapti
Copy link
Contributor Author

prapti commented Jun 9, 2020

Thanks for a quick check @saturninoabril! I'll add make those changes and update the PR description. Meanwhile, I've changed it to a WIP PR for now as I'm running some tests from the smoke group to gather some data upfront. I'll update the PR soon. And thanks for reminding that the marketplace tests aren't 100%. Will need to remove them now until fixed.

Comment on lines 10 to 11
// Stage: @prod @smoke
// Group: @accessibility
// Group: @accessibility @smoke
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused how do you plan to run this.
If @smoke will be on "Group" metadata which I think makes sense, then the @smoke tags on "Stage" can be removed. With that, it could run as node run_tests.js --stage='@prod' --group'@smoke'

Suggested change
// Stage: @prod @smoke
// Group: @accessibility
// Group: @accessibility @smoke
// Stage: @prod
// Group: @accessibility @smoke

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I just read your updated description with node run_tests.js --stage='@smoke' --group='@smoke'. However, passing @smoke on both args feels redundant to me.

@prapti prapti changed the title WIP - Adding label for PR smoke tests Adding label for PR smoke tests Jun 10, 2020
@prapti
Copy link
Contributor Author

prapti commented Jun 10, 2020

Removed '@smoke' tag from all other specs not intended for smoke tests. Now that the tag has been added as "Group", the tests can be run simply with --group='@smoke' flag for running the smoke tests, or with --stage='@prod' --group='@smoke' to run it within the production tests.

…nfusion in running tests with that tag inside the 'stage' suite
Copy link
Member

@saturninoabril saturninoabril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks @prapti!

@saturninoabril saturninoabril added the 3: QA Review Requires review by a QA tester label Jun 10, 2020
Copy link
Contributor

@srkgupta srkgupta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Approving the PR. Thanks.

Copy link
Contributor

@josephbaylon josephbaylon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@josephbaylon josephbaylon added 4: Reviews Complete All reviewers have approved the pull request and removed 3: QA Review Requires review by a QA tester labels Jun 10, 2020
@josephbaylon josephbaylon merged commit afd8199 into mattermost:master Jun 10, 2020
@amyblais amyblais added Changelog/Not Needed Does not require a changelog entry Docs/Not Needed Does not require documentation labels Jun 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
4: Reviews Complete All reviewers have approved the pull request Changelog/Not Needed Does not require a changelog entry Docs/Not Needed Does not require documentation
Projects
None yet
5 participants