Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: fixes uc24 nested tests - part 3 #14114

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

sergiocazzolato
Copy link
Collaborator

@sergiocazzolato sergiocazzolato commented Jun 22, 2024

This change includes more fixes for nested tests:
. Missing models added
. Changes preparing initramfs
. New validation sets added
. Changes in tests
. Change in remote.pull to make it work in all the systems properly

These are the tests results before this change:
https://paste.ubuntu.com/p/K4DrBxw45b/

This is the list of identified errors to fix in following prs:
https://paste.ubuntu.com/p/5dnjmVmb7t/

…b77e1..1e309f41c6

1e309f41c6 change how legacy parameter is determine in remote.pull
c43c35f7e3 run remote refresh and wait-for for xenial (skip bionic)
5262d30da7 make sure the test jobs are executed in runners with the spread label
cb74259b7a add openstack systems
0b41fd40d3 fix tests.pkgs on arch-linux
558e109793 run fedora-40 spread tests in openstack
6f6187416d fix list implementation
b4a5439c9b added more type annotatios to log_helper
58da1e36c3 mypy cleaned
1ff651e680 update wording of remote.pull
18615b1667 just usc scp -O when ssh version is -ge 9
cc68c9868b Added type annotations for log-filter
66f90d10cd Adding -O to scp command to make it compatible in uc24 tests
496cb7b5b3 removing support for centos-8
f2eef30db4 Updated the log helper and log parser
5a375ebf73 Formatting for python utils
d3eed3faa5 fix codespell in CODE_OF_CONDUCT.md
18bcca6b14 new log helper
d60381fcd9 add run number to filtered filename
5dde2d67b8 consider the tests execution in main
6b9a3aabcc change filtered log name
b2756aa579 default file is .filtered.log
500b9dace4 Fix tests workflow
45db26a3d2 fix shellcheck error in log-filter
fe45c27b7d create a var to store filter params
5a9b66d7dc filter spread results
51f9b055af New tool used to filter the spread.log output
b8d20c1d5b fix snaps.name test with correct siffix spelling
f640ac72e3 Add missing test details
f0754df304 Filter the error y debug output in log-parser
fc10196efd Add suggestions to details
94ac5ffe58 Add details on tests
501578c719 add more checks in os.query to check is-core_xx
e8929207ff fix os-query for ubuntu comparing with core
226114641f os.query won't check SPREAD_SYSTEM anymore to compare core systems
b89ec98b23 use local variables in os.query tool
dacfd81de9 fix is_core functions
1db5214d5f Improve the remote docs (snapcore#36)
2e4a3153a2 1 more comment
3a0fc57e1e add explanation about why we check for ( Do | Doing )
4cf8e635bf fix os.query test after merge
b89b4f8647 fix artifacts name
d30cee6da0 Merge remote-tracking branch 'upstream/main'
5ef5dcbe8f Tests use artifacts in spread tests (snapcore#51)
555c43d2ab Support auto-refresh with Do instead of Doing
96c2b0c19c remove tests support for ubuntu 23.04 (EoL)
74082c0c34 Tests improve remote wait (snapcore#49)
5121bfb659 remove support for opensuse leap 15.4 (snapcore#48)
30df700d08 Add new systems support (snapcore#47)
1f08938925 Support check amazon linux version (snapcore#46)
43533bdd97 Change the exit value checking for test formats (snapcore#45)
3c88244c04 Update check-test-format to support a dir and a list of files (snapcore#44)
510d95f429 add extra check for error in auto-refresh detection function
3289d4031b Try open the log with latin-1 encoding when utf-8 is not working
9db785499f improved how the tools are waiting for system reboot
2a5c4414a3 fix shellcheck errors
5e7b63883d Fixes for osquery and tests pkgs (snapcore#43)
4c9145e2ac support reboot waiting for auto-refresh
45768f5188 show changes in unknown status after refresh
8013c30c2a Remove support for ubuntu 22.10
b32b80bf54 Fix remote.rait-for test in bionic
5675c625e9 Enable fedora 38
55f4471957 Support for new oss
f2e88b357c New tool used to query spread json reports
cacd35ede0 utils/spread-shellcheck: explain disabled warnings (snapcore#42)
c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs
b84eea92e2 spread-shellcheck: fix quotes in environment variables (snapcore#41)
ab1e51c29f New comparison in os-query for core systems (snapcore#40)
e5ae22a5d4 systemd units can be overwritten
63540b845a Fix error messages in remote pull and push
75e8a426a5 make sure the unit is removed in tests.systemd test
9089ff5c02 Update tests to use the new tests.systemd stop-unit
44ecd5e56a Move tests.systemd stop-units to stop-unit
01a2a83b4b Update tests.systemd to have stop units as systemd.sh
162e93bd35 update tests.systemd CLI options to be the same than retry command
14aa43a405 new feature to re-run failed spread tests (snapcore#39)
604cb782db Fix shellcheck in systemd tool
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 1e309f41c6f18a359ebc01652dac31643ddb2708
…re-fixes-nested-tests

tests/lib/external/snapd-testing-tools/.github/workflows/tests.yaml
tests/lib/external/snapd-testing-tools/tests/remote.retry/task.yaml
tests/lib/external/snapd-testing-tools/tools/tests.pkgs.pacman.sh
@sergiocazzolato sergiocazzolato added the Run nested The PR also runs tests inluded in nested suite label Jun 22, 2024
@github-actions github-actions bot added the Run Nested -auto- Label automatically added in case nested tests need to be executed label Jun 22, 2024
@@ -558,9 +558,10 @@ pkg_dependencies_ubuntu_classic(){
python3-gi
python3-yaml
upower
weston
weston
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove extra whitespaces

Suggested change
weston
weston

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

udapted. Thanks for the review

@Meulengracht
Copy link
Member

Thank you for posting the pre-PR test results

@sergiocazzolato
Copy link
Collaborator Author

These are the current results
https://paste.ubuntu.com/p/nn5Rc3jYCd/

@@ -3,17 +3,6 @@ volumes:
# bootloader configuration is shipped and managed by snapd
bootloader: grub
structure:
- name: mbr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still have this one on uc24 for amd64. Not sure why it is removed here.

Copy link
Collaborator Author

@sergiocazzolato sergiocazzolato Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@valentindavid, I found this problem
error: cannot pack "test-snapd-remodel-pc": structure #0 ("mbr"): content "mbr.img": stat test-snapd-remodel-pc/mbr.img: no such file or directory

Also discussed that with @alfonsosanchezbeato
Perhaps we could sync to define which should be the final solution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Run Nested -auto- Label automatically added in case nested tests need to be executed Run nested The PR also runs tests inluded in nested suite
Projects
None yet
3 participants