Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress starts Xvfb even when running with Chrome Headless #19868

Open
just-jeb opened this issue Jan 25, 2022 · 30 comments
Open

Cypress starts Xvfb even when running with Chrome Headless #19868

just-jeb opened this issue Jan 25, 2022 · 30 comments
Labels
E2E Issue related to end-to-end testing Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team. type: enhancement Requested enhancement of existing feature

Comments

@just-jeb
Copy link

just-jeb commented Jan 25, 2022

Current behavior

Cypress starts Xvfb even when running with Chrome Headless, which causes display already in use error when running multiple Cypress processes.

Desired behavior

Cypress shouldn't run Xvfb if run with Chrome Headless.

Test code to reproduce

Cypress Version

9.3.1

Other

I'm aware of the possibility of running Xvfb on my own or spawning multiple machines, but this issue shouldn't be happening in first place, because Chrome Headless doesn't require Xvfb.

@cypress-app-bot
Copy link
Collaborator

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

@cypress-app-bot cypress-app-bot added the stale no activity on this issue for a long period label May 15, 2023
@just-jeb
Copy link
Author

Activity bump.

@MikeMcC399
Copy link
Contributor

@just-jeb

Have you referred to the documentation https://docs.cypress.io/guides/continuous-integration/introduction#Running-headless-tests-without-Xvfb and tried out the workaround there?

Are you still using the old Cypress 9.3.1 version or have you updated in the meantime?

Image

@just-jeb
Copy link
Author

I will check it out, thanks!

@nagash77
Copy link
Contributor

@just-jeb have you confirmed this is still happening on the latest version of Cypress?

@nagash77 nagash77 self-assigned this May 17, 2023
@cypress-app-bot cypress-app-bot removed the stale no activity on this issue for a long period label May 17, 2023
@rooby
Copy link

rooby commented May 19, 2023

@MikeMcC399 As noted in the documentation, that flag is experimental.
In my experience I've never been able to get it to work.
See
#18224
#17412
#23636

@nagash77
Copy link
Contributor

Unfortunately we have to close this issue due to inactivity. Please comment if there is new information to provide concerning the original issue and we can reopen.

@nagash77 nagash77 closed this as not planned Won't fix, can't repro, duplicate, stale May 22, 2023
@nagash77 nagash77 removed their assignment May 22, 2023
@rooby
Copy link

rooby commented May 22, 2023

@nagash77 inactivity might be the wrong reason here as there has been multiple replies in the last week.
There seems to be a lot of issues being closed recently that are still issues, which is does not inspire confidence in the direction of the project from a user’s perspective.

@nagash77
Copy link
Contributor

@rooby this ticket is for a quite old version of Cypress. Since v9 of Cypress A LOT has changed in the app. As such we ask that users verify the behavior is still present in the latest version of Cypress. If you have a reproducible example of this behavior you can share in a version >12.0 please comment back here and we will be happy to investigate.

@nagash77 nagash77 added Needs Reproduction and removed stage: needs investigating Someone from Cypress needs to look at this labels May 23, 2023
@rooby
Copy link

rooby commented May 24, 2023

@nagash77 Yes this is still an issue in the newer versions.
I just tested with cypress 12.12, using the cypress/included:12.13.0 Docker image and cypress-example-kitchensink (which was using cypress 12.12).

The command was explicitly headless: cypress run --browser chrome --headless

And with DEBUG=cypress:*,xvfb it logs

> [email protected] cy:run
> cypress run --browser chrome --headless

  cypress:cli:cli cli starts with arguments ["/usr/local/bin/node","/localDebugRepo/node_modules/.bin/cypress","run","--browser"
,"chrome","--headless"] +0ms
  cypress:cli NODE_OPTIONS is not set +0ms
  cypress:cli:cli program parsing arguments +1ms
  cypress:cli:cli running Cypress with args ... +78ms
  cypress:cli parsed cli options { browser: 'chrome', headless: true } +81ms
  cypress:cli verifying Cypress app +0ms
  cypress:cli Using CYPRESS_CACHE_FOLDER from environment variable +0ms
  cypress:cli Using CYPRESS_CACHE_FOLDER from environment variable +0ms
  cypress:cli using environment variable CYPRESS_CACHE_FOLDER /root/.cache/Cypress +0ms
  cypress:cli checking environment variables +1ms
  cypress:cli checking if executable exists /root/.cache/Cypress/12.12.0/Cypress/Cypress +2ms
  cypress:cli Binary is executable? : true +2ms
  cypress:cli binaryDir is  /root/.cache/Cypress/12.12.0/Cypress +0ms
  cypress:cli Reading binary package.json from: /root/.cache/Cypress/12.12.0/Cypress/resources/app/package.json +4ms
  cypress:cli Found binary version 12.12.0 installed in: /root/.cache/Cypress/12.12.0/Cypress +3ms
  cypress:cli { verified: true } +4ms
  cypress:cli is Verified ? true +1ms
  cypress:cli:run processing run options { browser: 'chrome', headless: true, key: null, spec: null, reporter: null, reporterOpt
ions: null, project: '/localDebugRepo' } +0ms
  cypress:cli:run --key is not set, looking up environment variable CYPRESS_RECORD_KEY +1ms
  cypress:cli:run run to spawn.start args ["--run-project","/localDebugRepo","--browser","chrome","--headed",false] +0ms
  cypress:cli undefined DISPLAY environment variable +0ms
  cypress:cli Cypress will spawn its own Xvfb +0ms
  cypress:cli Using CYPRESS_CACHE_FOLDER from environment variable +10ms
  cypress:cli Using CYPRESS_CACHE_FOLDER from environment variable +0ms
  cypress:cli using environment variable CYPRESS_CACHE_FOLDER /root/.cache/Cypress +1ms
  cypress:cli needs to start own Xvfb? true +0ms
  cypress:cli Starting Xvfb +1ms
  xvfb lock filename /tmp/.X99-lock +0ms
  xvfb lock filename /tmp/.X99-lock +0ms
  xvfb setting DISPLAY :99 +0ms
  xvfb all Xvfb arguments [ ':99', '-screen', '0', '1280x1024x24' ] +1ms
  xvfb checking if started by looking for the lock file /tmp/.X99-lock +3ms
  xvfb checking if started by looking for the lock file /tmp/.X99-lock +10ms
  xvfb lock file /tmp/.X99-lock found after 10 ms +1ms
  cypress:cli spawning, should retry on display problem? false +16ms
  cypress:cli passing DISPLAY :99 +4ms
  cypress:cli spawn args [ '--no-sandbox', '--', '--run-project', '/localDebugRepo', '--browser', 'chrome', '--headed', false, '
--cwd', '/localDebugRepo', '--userNodePath', '/usr/local/bin/node', '--userNodeVersion', '18.16.0' ] { detached: false, stdio: [
 'inherit', 'inherit', 'pipe' ] } +0ms
  cypress:cli spawning Cypress with executable: /root/.cache/Cypress/12.12.0/Cypress/Cypress +0ms
  cypress:cli piping child STDERR to process STDERR +3ms
  cypress:snapshot:info Caching 3791, defining 4730 modules! Using cache +0ms
  cypress:snapshot:debug initializing packherd require +0ms
  cypress:server:performance-benchmark elapsed time at v8-snapshot-startup-time: 151.056ms +0ms
  cypress:server:appdata path: /root/.config/Cypress/cy/production/browsers +0ms
  cypress:server:cypress starting cypress with argv
  ...

Notably:

  cypress:cli needs to start own Xvfb? true +0ms
  cypress:cli Starting Xvfb +1ms

@AtofStryker
Copy link
Contributor

@rooby Do you know where I can find information that states chrome headless doesn't need xvfb?

@MikeMcC399
Copy link
Contributor

@AtofStryker

Do you know where I can find information that states chrome headless doesn't need xvfb?

See https://developer.chrome.com/blog/headless-chrome/#faq

image

@AtofStryker
Copy link
Contributor

@MikeMcC399 I wonder if we can just pass in the browser to the isNeeded function and return false if chrome and headless? Could be a good first PR for a contributor.

@MikeMcC399
Copy link
Contributor

@AtofStryker

I wonder if we can just pass in the browser to the isNeeded function and return false if chrome and headless? Could be a good first PR for a contributor.

That would definitely be worth investigating although @rooby stated that ELECTRON_RUN_AS_NODE=1, which is also used in that code, never worked for him. Perhaps this is also dependent on the environment docker / no docker and OS in use?

@AtofStryker
Copy link
Contributor

@AtofStryker

I wonder if we can just pass in the browser to the isNeeded function and return false if chrome and headless? Could be a good first PR for a contributor.

That would definitely be worth investigating although @rooby stated that ELECTRON_RUN_AS_NODE=1, which is also used in that code, never worked for him. Perhaps this is also dependent on the environment docker / no docker and OS in use?

That or maybe the variable isn't spawned correctly in the process. Something that should be verified either way.

@AtofStryker AtofStryker added Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team. and removed Needs Reproduction labels May 24, 2023
@MikeMcC399
Copy link
Contributor

MikeMcC399 commented Jun 6, 2023

I confirmed this issue with Cypress 12.13.0 under Ubuntu 20.04 and Node.js 18.16.0

See repo https://github.com/MikeMcC399/cy-xvfb-test
Workflow run 5187602242

Edit: Still failing in Cypress 12.16.0

@AtofStryker AtofStryker removed their assignment Jun 6, 2023
@warrensplayer warrensplayer added the E2E Issue related to end-to-end testing label Jun 15, 2023
@navedanjum
Copy link

navedanjum commented Aug 8, 2023

I confirm that it fails in Cypress 12.17.2 as well. We are using Jenkins CI and Docker. The issue was more frequent in Cypress 10.10 and we recently migrated from v10 to v12. Running with ELECTRON_RUN_AS_NODE=1 and explicitly specifying --headless as in the node script below seemed to have worked for while with v10.10 however in v12.17.2 , this has re-surfaced.

`

  • "clean": "rm -rf ./results/*",
  • "test": "yarn run clean && HTTP_PROXY=http:https://localhost:8000 && ELECTRON_RUN_AS_NODE=1 && cypress run --headless --browser chrome",

`

  • `Jenkins CI.
  • Platform: linux-x64 (Debian - 11.6).
  • Cypress Version: 12.17.2
  • Docker Image: FROM cypress/included:12.17.2 `

@navedanjum
Update: In my case, the issue occurred only on Debian 11.6 and the main reason, agent(slave) was running other UI tests that uses xvfb. Removing the --network="host" from Jenkins Docker run command solved the issue.
In few pipelines we needed the host network so that didn't help, we simply had to ignore the executor to pickup the bad agent ( Debian - 11.6) by using Jenkins label : https://www.jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#node-allocate-node

  • We could not use - recommended suggestion to pkill Xvfb because that might disrupt the execution of other tests on same agent, more so over that is not permitted on CI agents.

if using docker-compose.yml , removing network_mode: host will solve the issue.

@nevercast
Copy link

I am also seeing this issue on Cypress 13 with Angular 17.

Current Behavior

When running Cypress tests through Angular CLI on a Linux CI environment (GitHub Actions), Cypress is being launched twice, once with Chrome and once with Electron. This is causing unexpected behaviour:

  1. Doubles test run-time
  2. Tests fail because Electron is not a supported browser for our project
  3. May be inadvertently causing other issues (distrust in the E2E)

Command Used

ng e2e --configuration ci --browser chrome

What's Happening

  1. The Angular CLI initiates a Cypress run with Chrome as specified.
  2. A second Cypress instance is launched, running with Electron.

Expected Behavior

Cypress should only run once with the specified browser (Chrome in this case).

Environment Details

  • OS: Linux (CI environment)
  • Cypress Version: 13.13.0
  • Node Version:
    • First run: v20.15.0
    • Second run: v20.13.1
  • Angular Version: 17.3.11
  • Angular CLI Version: 17.3.8 (note: does not match Angular version)

Configuration

package.json script

"e2eci": "ng e2e --configuration ci --browser chrome"

angular.json configuration

"e2e": {
  "builder": "@cypress/schematic:cypress",
  "options": {
    "devServerTarget": "app:serve",
    "watch": true,
    "headless": false,
    "browser": ""
  },
  "configurations": {
    "ci": {
      "baseUrl": "http:https://localhost:4204",
      "headless": true,
      "ssl": false
    }
  }
}

angular.json::e2e contains a definition of browser to trample the defaults in Cypress code, this obviously doesn't resolve it.

"browser": ""

Relevant Log Output

2024-07-05T00:00:36.152Z cypress:cli:run processing run options { project: '/home/runner/work/[PROJECT]/[PROJECT]/frontend', browser: 'chrome', headless: true, record: false, spec: null, devServerTarget: 'app:serve', watch: true, baseUrl: 'http:https://localhost:4204', ssl: false, configFile: undefined, quiet: false, e2e: true, component: false, env: '{}', exit: true, key: null, parallel: false, reporter: null, 'reporter-options': '{}', tsConfig: undefined, testingType: undefined, projectPath: '/home/runner/work/[PROJECT]/[PROJECT]/frontend/cypress', devServerBaseUrl: 'http:https://localhost:4204', dev: false, config: '{"e2e":{"baseUrl":"http:https://localhost:4204"}}', outputPath: '/tmp/tmp-1914-WtGMBrcDqLGb', reporterOptions: null }

...

2024-07-05T00:00:36.189Z cypress:cli spawning Cypress with executable: /home/runner/.cache/Cypress/13.13.0/Cypress/Cypress

...

Browser: Chrome 126 (headless)

...

2024-07-05T00:00:36.956Z cypress:cli:run processing run options { headed: false, record: false, parallel: false, quiet: false, component: false, outputPath: '/tmp/tmp-1771-tNzq5Bxu5uXl', key: null, spec: null, reporter: null, reporterOptions: null, project: '/home/runner/work/[PROJECT]/[PROJECT]/frontend' }

...

Browser: Electron 118 (headless)

Additional Notes

  1. Xvfb is being used for the headless runs:

    cypress:cli undefined DISPLAY environment variable
    cypress:cli Cypress will spawn its own Xvfb
    
  2. The double run seems to be triggered by the interaction between Angular CLI and the Cypress schematic.

  3. This behavior is not observed on Windows environments, suggesting it might be related to the Linux-specific setup or the use of Xvfb.

I would appreciate any insights into why this double run is occurring and how it can be prevented while still using the Angular CLI for running Cypress tests.

Workarounds attempted:

ELECTRON_RUN_AS_NODE

I attempted to use ELECTRON_RUN_AS_NODE=1 to inhibit the xvfb behaviour but that just caused #18224 which points back to this issue. No dice.

/opt/hostedtoolcache/node/20.15.0/x64/bin/npx cypress verify
2024-07-05T00:31:25.695Z cypress:cli:cli cli starts with arguments ["/opt/hostedtoolcache/node/20.15.0/x64/bin/node","/home/runner/work/[PROJECT]/[PROJECT]/frontend/node_modules/.bin/cypress","verify"]
2024-07-05T00:31:25.696Z cypress:cli NODE_OPTIONS is not set
2024-07-05T00:31:25.696Z cypress:cli:cli program parsing arguments
2024-07-05T00:31:25.699Z cypress:cli parsed cli options {}
2024-07-05T00:31:25.783Z cypress:cli verifying Cypress app
2024-07-05T00:31:25.783Z cypress:cli Using CYPRESS_CACHE_FOLDER from environment variable
2024-07-05T00:31:25.783Z cypress:cli Using CYPRESS_CACHE_FOLDER from environment variable
2024-07-05T00:31:25.784Z cypress:cli using environment variable CYPRESS_CACHE_FOLDER /home/runner/.cache/Cypress
2024-07-05T00:31:25.784Z cypress:cli checking environment variables
2024-07-05T00:31:25.791Z cypress:cli checking if executable exists /home/runner/.cache/Cypress/13.13.0/Cypress/Cypress
2024-07-05T00:31:25.793Z cypress:cli Binary is executable? : true
2024-07-05T00:31:25.793Z cypress:cli binaryDir is  /home/runner/.cache/Cypress/13.13.0/Cypress
2024-07-05T00:31:25.793Z cypress:cli Reading binary package.json from: /home/runner/.cache/Cypress/13.13.0/Cypress/resources/app/package.json
2024-07-05T00:31:25.798Z cypress:cli Found binary version 13.13.0 installed in: /home/runner/.cache/Cypress/13.13.0/Cypress
2024-07-05T00:31:25.799Z cypress:cli could not read binary_state.json file at "/home/runner/.cache/Cypress/13.13.0/binary_state.json"
2024-07-05T00:31:25.799Z cypress:cli {}
2024-07-05T00:31:25.800Z cypress:cli is Verified ? undefined
2024-07-05T00:31:25.800Z cypress:cli force verify
2024-07-05T00:31:25.800Z cypress:cli running binary verification check 13.13.0
[STARTED] Task without title.
2024-07-05T00:31:25.803Z cypress:cli clearing out the verified version
2024-07-05T00:31:25.805Z cypress:cli Environment variable ELECTRON_RUN_AS_NODE detected, xvfb is not needed
2024-07-05T00:31:25.805Z cypress:cli needs Xvfb? false
2024-07-05T00:31:25.805Z cypress:cli spawning, should retry on display problem? false
2024-07-05T00:31:25.805Z cypress:cli disabling Electron sandbox
2024-07-05T00:31:25.805Z cypress:cli running smoke test
2024-07-05T00:31:25.805Z cypress:cli using Cypress executable /home/runner/.cache/Cypress/13.13.0/Cypress/Cypress
2024-07-05T00:31:25.805Z cypress:cli smoke test command: /home/runner/.cache/Cypress/13.13.0/Cypress/Cypress --no-sandbox --smoke-test --ping=776
2024-07-05T00:31:25.805Z cypress:cli smoke test timeout 30000 ms
2024-07-05T00:31:26.[48](https://github.com/[PROJECT]/[PROJECT]/actions/runs/9801235292/job/27064340185#step:5:49)9Z cypress:cli Smoke test failed: Error: Command failed with exit code 9: /home/runner/.cache/Cypress/13.13.0/Cypress/Cypress --no-sandbox --smoke-test --ping=776
/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --no-sandbox
/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --smoke-test
/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --ping=776
    at makeError (/home/runner/work/[PROJECT]/[PROJECT]/frontend/node_modules/cypress/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/home/runner/work/[PROJECT]/[PROJECT]/frontend/node_modules/cypress/node_modules/execa/index.js:114:26)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  shortMessage: 'Command failed with exit code 9: /home/runner/.cache/Cypress/13.13.0/Cypress/Cypress --no-sandbox --smoke-test --ping=776',
  command: '/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress --no-sandbox --smoke-test --ping=776',
  exitCode: 9,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: '/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --no-sandbox\n' +
    '/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --smoke-test\n' +
    '/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --ping=776',
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false
}
2024-07-05T00:31:26.[49](https://github.com/[PROJECT]/[PROJECT]/actions/runs/9801235292/job/27064340185#step:5:50)0Z cypress:cli error message: /home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --no-sandbox
/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --smoke-test
/home/runner/.cache/Cypress/13.13.0/Cypress/Cypress: bad option: --ping=776
2024-07-05T00:31:26.491Z cypress:cli detecting arch { osPlatform: 'linux', osArch: 'x64' }
2024-07-05T00:31:26.499Z cypress:cli arm uname -m result: { stdout: 'x86_64' } 
[FAILED] Cypress failed to start.

@MikeMcC399
Copy link
Contributor

@nevercast

This should probably be a new issue.

@nevercast
Copy link

@nevercast

This should probably be a new issue.

Seemed related. Felt the context may help. I shall move it to a new issue and reference this one.

@navedanjum
Copy link

navedanjum commented Jul 5, 2024

@nevercast ! Any other instance of Cypress running on machine/server ? , if you're using the shared infrastructure resources. Xvfb issues we had resolved on the past because we had number of cypress jobs ran on the same linux server and we had to ensure the dockerized cypress tests do not share resource with the hosts to get rid of it. Your issue does not seem like one I am mentioned but often other job runs leave out xvfb process running on the machine.

@gweesin
Copy link

gweesin commented Jul 5, 2024

If we run with chrome, can we remove xvfb dependency in the future?

@MikeMcC399
Copy link
Contributor

@gweesin

If we run with chrome, can we remove xvfb dependency in the future?

That is the request in this issue. The issue is however still open, so there is no answer to your question at the moment.

@gweesin
Copy link

gweesin commented Jul 5, 2024

@gweesin

If we run with chrome, can we remove xvfb dependency in the future?

That is the request in this issue. The issue is however still open, so there is no answer to your question at the moment.

Thank you for your reply, I'll pay attention to the progress. 😀

@kris-fluency
Copy link

I'm experiencing this same issue, only way for us to resolve it is to go in and manually kill Xvfb processes and then rerun the tests.

@nevercast
Copy link

@nevercast ! Any other instance of Cypress running on machine/server ? , if you're using the shared infrastructure resources. Xvfb issues we had resolved on the past because we had number of cypress jobs ran on the same linux server and we had to ensure the dockerized cypress tests do not share resource with the hosts to get rid of it. Your issue does not seem like one I am mentioned but often other job runs leave out xvfb process running on the machine.

Nope, no other processes on the server. GitHub Actions is spawning a fresh runner every time.

@MikeMcC399
Copy link
Contributor

@nevercast

Did you ever open a separate issue? There is something wrong, apart from xvfb, if Cypress is being run twice. If you haven't done so already, I suggest again to open a new issue and post your GitHub Actions workflow.

@MikeMcC399
Copy link
Contributor

@nevercast

@nevercast
Copy link

@nevercast

Thanks! Yep. That was the issue. npm run e2e was specified in start: 🤦‍♂️

@MikeMcC399
Copy link
Contributor

@nevercast

It was added to the github-action documentation start-server a while back:

Caution: use the start parameter only to start a server, not to run Cypress, otherwise tests may be run twice. The action runs Cypress tests by default, unless the parameter runTests is set to false.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2E Issue related to end-to-end testing Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team. type: enhancement Requested enhancement of existing feature
Projects
None yet
Development

No branches or pull requests