Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Randomly failing tests on MacOS #237

Open
Isty001 opened this issue Jan 12, 2021 · 5 comments
Open

Randomly failing tests on MacOS #237

Isty001 opened this issue Jan 12, 2021 · 5 comments

Comments

@Isty001
Copy link
Member

Isty001 commented Jan 12, 2021

Taking a look at the actions results it seems some tests fail on Mac, but not always.

Running clib(1) tests

./test/gh-35-exit-codes.sh: line 8: 21111 Segmentation fault: 11  clib install stephenmathieson/rot132.c > /dev/null 2>&1
  (✓) test/gh-35-exit-codes.sh
./test/help.sh: line 3: 21113 Segmentation fault: 11  clib help 2> /dev/null
Expected `clib help` to fail
  (✖) test/help.sh
./test/install-binary-dependencies.sh: line 6: 21115 Segmentation fault: 11  clib install -c stephenmathieson/[email protected] -P tmp > /dev/null
Failed to install stephenmathieson/tabs-to-spaces
  (✖) test/install-binary-dependencies.sh
./test/install-brace-expansion.sh: line 9: 21117 Segmentation fault: 11  clib install -c -o tmp stephenmathieson/trim.c stephenmathieson/case.c > /dev/null
expecting successful exit code
  (✖) test/install-brace-expansion.sh
./test/install-deps-from-package-json.sh: line 23: 21122 Segmentation fault: 11  clib install > /dev/null 2>&1
expecting exit code of 1
  (✖) test/install-deps-from-package-json.sh
./test/install-multiple-clibs-libs.sh: line 9: 21124 Segmentation fault: 11  clib install -c -o tmp ms file hash > /dev/null
expecting successful exit code
  (✖) test/install-multiple-clibs-libs.sh
./test/install-multiple-libs.sh: line 10: 21126 Segmentation fault: 11  clib install -c -o tmp stephenmathieson/case.c stephenmathieson/trim.c > /dev/null
expecting successful exit code
  (✖) test/install-multiple-libs.sh

Edit: I can't reproduce any of it on Linux now.

@Isty001 Isty001 added the bug label Jan 12, 2021
@Isty001 Isty001 changed the title Randomly failing tests Randomly failing tests with executables Jan 12, 2021
@Isty001 Isty001 changed the title Randomly failing tests with executables Randomly failing tests on MacOS Jan 19, 2021
@andy5995
Copy link
Contributor

andy5995 commented Jun 6, 2021

This log file might give some hints. It's from when I built with gcc 11.1.0 on Debian Bullseye and added the -fanalyzer flag.

clib2.log

@andy5995
Copy link
Contributor

andy5995 commented Jun 8, 2021

I had some problems recently with tests randomly failing on MacOS using Travis CI. Turned out to be a cache issue.

Btw, I've found Coverity to be a useful tool for testing. Although it sometimes gives false positives, it's pretty good at spotting potential resource leaks or buffer overflows.

Isty001 added a commit that referenced this issue Mar 23, 2022
Intialize the cache in test where it's used
Isty001 added a commit that referenced this issue Mar 23, 2022
Intialize the cache in test where it's used
@Isty001 Isty001 reopened this Mar 29, 2022
@diasbruno
Copy link
Member

Haven't seen this in a long time. Should we close this, @Isty001?

@jwerle
Copy link
Member

jwerle commented Aug 15, 2024

@Isty001 what do you think?

@Isty001
Copy link
Member Author

Isty001 commented Aug 19, 2024

Yeah, I think we can close it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants