[Feature] Run Playwright tests on real mobile device browsers?

[Cosmin-Gramada]

Is there any way to connect to a device, start a browser (e.g. chromium instance on Android) and run the tests?

If not, what is missing at this point to be able to use it on both Android and iOS?

---

Note from maintainers:

Please refer to the https://playwright.dev/docs/next/api/class-android for
instructions on how to use Playwright with Chrome for Android, Android
WebViews and Android itself

---

[pavelfeldman]

There is no way to attach to Chromium on Android or MobileSafari on iOS. Playwright aims at running on your desktop and in your CI/CD, targeting device farms is outside its scope.

We will emulate device metrics, geolocation, orientation, etc on Chromium on Android and MobileSafari though. Could you share more about your use case?

[Cosmin-Gramada]

We took into account the possibility of moving away from Selenium WebDriver based tools and switching to DevTools API based tools.

A really big portion of our tests are running on mobile web browsers, where we tests our own player that makes use of browser related technologies (e.g. what types of medias is able to render, what's the appropriate format, etc) and sends specific tracking events.

Our concern is that there will be differences between a real device and an emulated one, as we're able to emulate any device resolution, but only the viewport size is guaranteed to be accurate.

[pavelfeldman]

Our concern is that there will be differences between a real device and an emulated one

Yes, there will be

as we're able to emulate any device resolution, but only the viewport size is guaranteed to be accurate

More than the viewport size, but yes, if you need mobile-specific media to be accurate, you need to attach to the device.

This is technically feasible, but is outside of the scope for v1.

[tanaypratap]

This could be useful for us too. We are a fully fledged PWA email app. Testing on actual devices would catch bugs. We have also faced this issue with non-existent media queries on some phones which @Cosmin-Gramada mentioned. Especially older phones.

[Filipoliko]

Since Playwright is trying to be more testing-friendly, it would be really great to have real mobile device support. More and more users are coming to our websites from mobile devices, it is nearly 50:50 for us. Most of our bugs on mobile devices are not discoverable via simple mobile emulation. It is really important for us to be able to test on real mobile devices, or at least on emulated Android/iOS systems.

[arjunattam]

@Filipoliko - thank you for sharing. Can you elaborate on which bugs are not discoverable via emulation? Examples will really help!

[Filipoliko]

@arjun27 - They are usually some very specific (and kinda weird) use cases, typically on iOS devices. One current bug on our website, that is discoverable only on real iOS device (safari, or chrome) can be currently found on this page. (The fix will be probably released during next week, hopefully you can check it before that time).

If you click on Domácí (see the picture below), then with chrome mobile emulation, you will be redirected to a section page Domácí (correct behaviour). But if you are running chrome, or safari on real iOS device, then the click will open the mobile menu. This is fixed by setting pointer-events: none on the hidden mobile menu element. Android/Windows/macOS works fine, but iOS requires this css rule for some reason.

Probably most typical use cases (as far as I can think of) are related to some invisible elements overlapping other elements and making it impossible to click on the overlapped element. This behaviour is usually iOS specific, other systems interpret these css rules in a bit different way.

[arjunattam]

Thanks @Filipoliko. I just tried running this case with iPhone emulation on Playwright WebKit and I'm able to repro this issue. Chrome mobile emulation is not reproducing it, as you've rightly pointed it out. Also, you're right about an invisible overlapping element (which is why I had to use force: true to issue the click). Sharing code below in case you'd like to try it out.

const { webkit, devices } = require('playwright');

(async() => {
  const browser = await webkit.launch({headless: false});
  const context = await browser.newContext({...devices['iPhone 11 Pro']});
  const page = await context.newPage();
  await page.goto('https://www.seznamzpravy.cz/clanek/v-korupcni-kauze-stoka-obvinila-policie-dalsi-27-lidi-111820');
  await page.click('[data-dot="ogm-breadcrumb-navigation"] >> text=Domácí', {force: true});
})();

[Kradirhamik]

Hi,

I would like to also give support for this idea. Our use cases requires our Angular site to get input from other apps (e.g. BankID for authentication). So the real flow can only be properly tested on a real device.

[alejandroiglesias]

Hi,
I'm curious to know what the workflow is on this project for an issue like this to move from feature request to development by the team? Does it need to reach a certain amount of upvotes? Does it need any funding? Does it take a certain amount of time to pass, like waiting for the next major release planning? It would be good for users to know and be able to set expectations on the development of feature requests. Thanks!

[pavelfeldman]

We have already seen enough interest in it, so we will invest into this area. There are several questions about the use cases and the API, so we would appreciate your feedback, the more details the better. I'll post some initial questions below as comments, please upvote them in case they apply!

[pavelfeldman]

Please 👍 if you'd like to

Use Playwright with the Chrome Browser (WebView integration is not essential for you).

[pavelfeldman]

Please 👍 if you'd like to

Use Playwright against your WebView and can bring WebView into a testable state over adb.

[pavelfeldman]

Please 👍 if you'd like to

Use Playwright against AVD (Emulator) as opposed to the real device.

[pavelfeldman]

Please 👍 if you'd like to

Use Playwright against the physical device as opposed to AVD (Emulator).

[pavelfeldman]

Please 👍 if

Your testing setup requires Android w/ Play Store

[pavelfeldman]

Please 👍 if

Your testing setup does not require Android w/ Play Store

[pavelfeldman]

Please 👍 if

You intend to run tests in the cloud.

[pavelfeldman]

See https://playwright.dev/docs/api/class-android for experimental support.

[pavelfeldman]

Folks, npm i playwright-android is now live and it has a first cut of the support for Android's Chrome browser as well as WebViews. It has not been covered with the tests, neither it runs on the bots. But it does work for simple scenarios. Please let us know if it works for you or if something is missing. As a drive-by it includes uiautomator-alike driver for the Android itself.

Edit: Please reference to the guide here: https://playwright.dev/docs/api/class-android

[alejandroiglesias]

@pavelfeldman that's great! Thanks for the initial implementation efforts. Are there any plans to do the same for iOS Safari? That would be awesome to get the main mobile contenders covered. Also, do you know how/if this could work with @web/test-runner-browserstack (a test runner that supports Playwright)?

[vania-pooh]

@alejandroiglesias so far as I know BrowserStack now supports Selenium protocol only.

[alejandroiglesias]

@vania-pooh BrowserStack also supports Cypress and has an HTTP-based API, and if @web/test-runner-browserstack does what it promises and can be combined with @web/test-runner-playwright, I would imagine that Playwright it should work in BrowserStack... in theory and unless I'm missing something. Alternatively, SauceLabs supports Playwright as it's stated in their docs.

[wahreChrist]

I would be content if we at least would have an option of having all the touch-based gestures on mobile viewports because right now there is only a built-in tap, which was added in version 1.8, and nothing besides that(no swipes, pinches, zooms, etc.).

[deepnaj]

support on real mobile devices

[pranavj1001]

Please make it possible to test via browser stack on mobile devices as well. Browserstack is already supported, but only for desktop OS. I don't understand the problem, why it should not be possible on mobile devices on Browserstack.

vania-pooh is the browserstack integration something you would consider?

Hello everyone, looking at this issue and some other issues (here on github and on playwright slack channel), we can see that people are requesting support for running Playwright tests on cloud mobile devices (similar to the desktop Playwright tests).

We have raised a PR and a Feature Request for this. We are happy to collaborate on this.

[mehrad-rafigh]

I talked to Browserstack and they are currenlty working on the Android part. Its in a private alpha and I hope it will soon be in beta.
For our development efforts, it would be great to able to test on real iOS devices, as the majority of our users come from iOS devices.

[GrayedFox]

@pavelfeldman ahoy, I understand good things take time and am not asking for a specific timeline or anything, but I wanted to clarify a few things if that's alright:

1. Do the playwright team plan on implementing iOS real mobile device support?
2. Do the playwright team plan on implementing iOS real mobile cloud device testing support?

I tried accessing the beta mobile docs link but has moved (or been removed):

See https://github.com/microsoft/playwright/blob/master/docs/mobile.md for experimental support.

If experimental support has been added, or there is a nightly or dev build with such support, could you provide an up to date link?

Thanks all your hard work!

[sgnl]

I talked to Browserstack and they are currenlty working on the Android part. Its in a private alpha and I hope it will soon be in beta. For our development efforts, it would be great to able to test on real iOS devices, as the majority of our users come from iOS devices.

https://www.browserstack.com/blog/introducing-android-playwright/ in beta

[MRamonLeon]

See https://github.com/microsoft/playwright/blob/master/docs/mobile.md for experimental support.

This link is missed.
We want to test real gestures wit webkit and android-like browser (pinching, swiping, ...). The desktop browsers are not 100% reliable for this purpose. Thanks!

[flakey-bit]

This is possibly a dumb question, but would it be technically possible for Playwright to support the Webdriver protocol?

Or is this explicitly a non-goal for the Playwright project?

Naively, it seems like supporting Webdriver would unlock lots of possibilities (for example, running on real Safari on OSX)

[dantman]

This is possibly a dumb question, but would it be technically possible for Playwright to support the Webdriver protocol?

I'd like to add to this question. Even if the full capabilities of Playwright are not possible, would it be possible to have partial support for it.

One of the things I like about TestCafe is how it supports basically any browser in a tiered structure. It has native support for a number of browsers. But can also run in many others like native mobile browsers, just with some features unavailable.

The ability to take a Playwright test and run it on any Webdriver compatible browser would be game changing. Even if it didn't have the full Playwright functionality.

e.g. It would be lovely if I could develop a robust set of feature tests in Playwright for the major browsers. Then run a partial set of those as smoke tests in a bunch of other browsers just to check for any browser specific bugs. Without needing to rewrite tests in a completely different framework.

[refactoreric]

This is possibly a dumb question, but would it be technically possible for Playwright to support the Webdriver protocol?

A Playwright fan's answer: From my understanding, a lot of Playwright's strengths like the efficient auto-waiting are relying on the protocol to be bi-directional. The current Webdriver protocol is too restricting and fundamentally different.
To enable Playwright-like features, the Webdriver team is working on the Webdriver BiDi protocol, but it still seems miles away.
If the Webdriver BiDi protocol (and support for it in browsers) catches up, then the Playwright dev team might start thinking about adding support for that protocol.

[andreagubellini]

is there any update on this?

[lauvers]

is there any update on this?

I would like to know this too. Where I work, we are looking into using Playwright for our test automation, but being able to test on real devices with iOS (e.g. using Browserstack) is a must have for some test cases.

[LuciaHMQA]

Please support android and ios native. Playwright is a wonderful tool to test automation and if support this mobile features is a one-tool-all-automation testing! the automation testing world need a tool with all automation phases.
Thank you!.

[dimkin-eu]

500+ 👍
Sounds like something so many people wants

[Xen0byte]

500+ 👍 Sounds like something so many people wants

Not only that, but it's the top most requested feature. I should probably add that for me and other non-TS peeps the problem is even worse, because let's say this gets added to the TS world, it still needs to be ported to other languages. e.g. in the .NET world where I use Playwright we don't even have Android support yet.