On iOS Safari 15.4.1, the local video track is sometimes blank

[sashabugor]

• I have verified that the issue occurs with the latest twilio-video.js release and is not marked as a known issue in the CHANGELOG.md.
• I reviewed the Common Issues and open GitHub issues and verified that this report represents a potentially new issue.
• I verified that the Quickstart application works in my environment.
• I am not sharing any Personally Identifiable Information (PII)
  or sensitive account information (API keys, credentials, etc.) when reporting this issue.

Code to reproduce the issue:

This is the hook used to create local video track:

async function getDeviceInfo() {
  const devices = await navigator.mediaDevices.enumerateDevices();

  return {
    audioInputDevices: devices.filter(device => device.kind === 'audioinput'),
    videoInputDevices: devices.filter(device => device.kind === 'videoinput'),
    audioOutputDevices: devices.filter(device => device.kind === 'audiooutput'),
    hasAudioInputDevices: devices.some(device => device.kind === 'audioinput'),
    hasVideoInputDevices: devices.some(device => device.kind === 'videoinput'),
  };
}

const getLocalVideoTrack = useCallback(async () => {
    const selectedVideoDeviceId = window.localStorage.getItem(SELECTED_VIDEO_INPUT_KEY);

    const { videoInputDevices } = await getDeviceInfo();

    const hasSelectedVideoDevice = videoInputDevices.some(
      device => selectedVideoDeviceId && device.deviceId === selectedVideoDeviceId
    );

    const options: CreateLocalTrackOptions = {
      ...(DEFAULT_VIDEO_CONSTRAINTS as Record<string, unknown>),
      name: `camera-${Date.now()}`,
      ...(hasSelectedVideoDevice && { deviceId: { exact: selectedVideoDeviceId! } }),
    };

    return Video.createLocalVideoTrack(options).then(newTrack => {
      setVideoTrack(newTrack);
      return newTrack;
    });
  }, []);

Expected behavior:

The local video track should have the image for a local participant when launching the room (running Video.connect()).

Actual behavior:
On mobile Safari, the local video track is sometimes blank, however the remote participant can see the user. Re-calling the Video.createLocalVideoTrack by clicking on video-toggle button resolves the issue.

Software versions:

• Browser(s):
  iOS Safari (mobile)
• Operating System:
  iOS 15.4.1
• twilio-video.js:
  2.18.2
• Third-party libraries (e.g., Angular, React, etc.):
  React 16.13.1

[korkushkoihorsnagajob]

Hi @sashabugor

We also using twilio-video in our app, and having the same problem:
mobile box with self-video is black, however, another participant can see the user.

Also, I noticed that the same behavior is on IOS 15.4.1 Chrome
[email protected]
and we are using it in Angular 13

[q-kit]

I have a similar issue that has been open for a while. This issue just happened again to participants PA9e67277a627e7f0cda1267e4df3714c4 and PA019e2a1d81fdade067ba87697b066a5f (a same user joined twice) of room RM2e302c2f4a3efdb2820271c7909fa677. Please let me know if more clue is needed. Thanks!

[timmydoza]

Hey @sashabugor - thanks for the issue!

I've been digging into this, and I've found that I can reproduce this issue in Safari 15.4 using our React Demo App fairly consistently. All I need to do is join a room by myself, and it appears that my local video is completely black, but I can confirm that the video is correctly flowing to the room. I find that this happens about 30% of the time that I join a room.

I think the problem may have to do with a Safari CSS bug. I'm curious - do you (and also @korkushkoihorsnagajob) apply a CSS transformation to the local participant's video track to mirror it?

In the React App, we apply transform: rotateY(180deg) to the local participant's video element to mirror the video, and I think this may be the source of the issue. I'm not sure why this CSS transform causes the video element to disappear, but I thought it may not be ideal to use a 3d transformation for this effect.

As a workaround, I used transform: scaleX(-1) instead (a 2d transform), and I found that with this change I could no longer reproduce the issue. If you use this CSS instead, do you find that the issue is fixed?

Please let me know how this works out for you! I'm curious if this is a viable workaround for everybody.

[taylor1791]

I am interested in this issue and performed the following tests:

• Deployed the demo app and tried to reproduce the issue on an iPhone 6s runing 14.4.1. The local video track was black 7/15 times. The black track occurred 3 times in the lobby and 5 times in the room. The lobby and room local track were both black 1 time.
• Using the same deployment, I tried to reproduce the issue on an iPhone 6s running 15.5. The local video track was black 3/15 times. All 3 times were in the room.
• Using the same deployment, I tried to reproduce the issue on an iPhone 11 running 15.5. After 15 attempts, I was never able to reproduce the black local track.
• I deployed the code using transform: scaleX(-1) instead. Using the same iPhone 6s running 15.5 I was unable to reproduce the black local track after 20 attempts.
• I re-deployed the code using transform: rotateY(180deg) again and was able to reproduce the black local track 4/15 times. The black local track occurred 1 time in lobby and 3 times in the room.

Based on these tests I can conclude:

• This is not fixed on iOS 15.5.
• Using scaleX(-1) does appear to workaround the problem. 🎆

[timmydoza]

Hi @taylor1791 - thanks so much for doing some testing! That's good news that this workaround appears to fix things in Safari 15.5. I've gone ahead and released this workaround in version 0.6.5 of our React Video App, so I'm going to close this issue.

There are still a few reports that users are seeing this issue, but this ticket also appears to be a duplicate of #1724. So I'll close this issue, but I'll leave #1724 open so any further discussion can happen there.

Thanks again!