Session replay with screenshot mode turned on forces SwiftUI redrawing

[NickAtGit]

Version

3.6.1

Steps to Reproduce

When using the provided SwiftUI example of the SwiftFortuneWheel package with PostHog integrated and session replay with screenshot mode turned on then the example project behaves strangely. When turning off screenshot mode it works as expected.

https://github.com/sh-khashimov/SwiftFortuneWheel

        let postHogApiKey = "xxxxxxxxxxxxxx"
        let postHogHost = "https://eu.i.posthog.com"
        let config = PostHogConfig(apiKey: postHogApiKey, host: postHogHost)
        config.sessionReplay = true
        config.sessionReplayConfig.maskAllImages = false
        config.sessionReplayConfig.maskAllTextInputs = false
        config.sessionReplayConfig.screenshotMode = true
        PostHogSDK.shared.setup(config)

Expected Result

The wheel should spin smoothly and come to an end.

Actual Result

The wheel stops abruptly and the console spams some warnings. I guess some constant SwiftUI redrawing is causing the issue.

ScreenRecording_08-01-2024.02-12-06_1.MP4

[marandaneto]

hey @NickAtGit sorry about that and thanks for reporting this issue.
I'm out on sick leave but will come back to it asap.

[NickAtGit]

Thanks! An easy workaround would be to pause the session recording for some views. That would already help a lot...

[marandaneto]

@NickAtGit that would be possible per Activity, but for the Single activity apps approach not really.
Maybe a pause and start option would be more useful, so you know when to start and when to pause better than the SDK.

We are still trying to optimize a few things such as the redimensioning of the screenshot out of the main thread (the screenshot has to be in the main thread though) and converting to base64, also increasing the interval from 500ms to 1s, I believe those would make much better for apps that is constantly redrawing the UI such as transitions and animations.

[NickAtGit]

@marandaneto we are speaking about iOS. There are no activities here ;)

[marandaneto]

@marandaneto we are speaking about iOS. There are no activities here ;)

Ops sorry, indeed, but that applies to iOS as well, on SwiftUI for example, the screen names are auto-generated, so that would not work unless you pass a list of types during the SDK init, will take a look.

[marandaneto]

@NickAtGit I have made some improvements, would you mind trying out https://github.com/PostHog/posthog-ios/releases/tag/3.7.1 ?

Right now the screenshot is taken every ~500ms, you can increase this time interval to minimize even further the performance hit by setting this:

// example 1s
config.sessionReplayConfig.debouncerDelay = 1.0

[NickAtGit]

Hi @marandaneto not sure if that helps in that case. Did you try the provided test project with PostHog? It's not about the performance. It is about PostHog sessionReplay triggering constant redraws on wrapped UIKit views. I would like to pause the sessionReplay on some views to prevent that.

[marandaneto]

Hi @marandaneto not sure if that helps in that case. Did you try the provided test project with PostHog? It's not about the performance. It is about PostHog sessionReplay triggering constant redraws on wrapped UIKit views. I would like to pause the sessionReplay on some views to prevent that.

@NickAtGit I created this issue for the screen-by-screen basis #160
The fact that the image downsizing is out of the main thread may help, as well as the base64 conversion, I'm not sure why the UI is redrawing since we only read the values, nothing else.

[NickAtGit]

Hey, I saw your update - unfortunately it doesn't fix my problem. I am using some animations lately in my app and I found that PostHog session replay seems to mess them up. Animations feel stuttery and this is unfortunately a show stopper to me.

[NickAtGit]

I also played around with the wireframe mode, however I just see blank videos in the PostHog session replay. Is there something I need to do to see them?

[marandaneto]

I also played around with the wireframe mode, however I just see blank videos in the PostHog session replay. Is there something I need to do to see them?

@NickAtGit wireframe isn't supported for SwiftUI

[marandaneto]

Hey, I saw your update - unfortunately it doesn't fix my problem. I am using some animations lately in my app and I found that PostHog session replay seems to mess them up. Animations feel stuttery and this is unfortunately a show stopper to me.

thanks for testing, I believe this should improve considerably more #166

[marandaneto]

I also played around with the wireframe mode, however I just see blank videos in the PostHog session replay. Is there something I need to do to see them?

@NickAtGit I made new changes and tested using your app, the fortune sheel feels smooth, would you like to give it another try? https://github.com/PostHog/posthog-ios/releases/tag/3.7.2
Thanks!

[NickAtGit]

Yes @marandaneto it is much better now. Thanks for fixing it!

One thing you should put in the documentation: Wireframes don't not work in SwiftUI because that was not clear to me.

[marandaneto]

Thanks
Actually its written in the docs already
https://posthog.com/docs/session-replay/ios see the limitations section.

[NickAtGit]

But here it's missing and it would make sense to write it here. (Who reads the docs anyways? 😜)

[marandaneto]

Good point, will make sure yo update this as well, thanks