Documentation Request - Visual seeking with sprite sheets

[mutaben]

Use case description

I would like to have more information on integrating visual seeking through sprite sheets in a Pillarbox player please.

Thank you in advance.

Acceptance criteria

No response

Proposed solution

No response

Alternatives considered

No response

Is there an existing similar feature request?

• I have searched existing features and found no similar request.

• I have browsed the available API and found no way to achieve the use case I described.

[defagos]

Thanks for the suggestion. There is no sprite sheet official support in Pillarbox and I cannot currently say if this is something we will provide support for in the future, so documentation might not be appropriate in this case (an external tutorial would likely be a better fit IMHO, but we can discuss it further).

Let me give you more context, though.

I-frame playlists

Unlike Letterbox, Pillarbox supports I-frame playlists on all platforms, which are special playlists delivered as part of the content itself. Those provide fast seek previews, a feature called trick play or trick mode. Trick play is also the only way to offer a seek preview within the tvOS standard player experience.

Provided the content delivers I-frames in an efficient way (and that your network connection is good enough), previews are then automatic when seeking smoothly with Pillarbox, a special feature of our player enabled by default. Please have a look at the Trickplay example in our Pillarbox demo, under the Examples tab, to have a better understanding of the kind of experience this can provide.

I-frame playlist support is the main reason why Pillarbox should IMHO discourage the use of external sprite sheets. If Pillarbox were to provide an official sprite sheet API, products would be tempted to implement seek previews on iOS only, making the experience poor for tvOS users of the same product. Most packaging solutions I could check recently offer I-frame playlist support, sometimes at the cost of a single checkbox. It would be a shame not to use it.

Note that some SRG SSR content is delivered with I-frame playlists, depending on the packaging that was used. We will try to promote wider adoption of I-frames in the future within our company so that all Apple platforms benefit from a better seeking experience in the future.

Sprite sheets

Sprite sheets are not governed by an associated standard AFAIK. Some backends can namely deliver a single sprite sheet (e.g. SRG SSR), sometimes a sprite sheet tree (e.g. YouTube). We even found products that opted for single-thumbnail delivery per timestamp, something that should be best avoided IMHO as it leads to a poor user experience.

As said above there is currently no official support for sprite sheets in Pillarbox. If you have sprite sheets available and cannot use trick play, you can still integrate them in a custom iOS player user interface (but you will miss them on tvOS), as follows:

• Load the sprite sheet(s) from your backend based on the content currently being played.
• Use a ProgressTracker to drive your slider and read the current time.
• Display a small window above your slider during interactions (isInteracting progress tracker property), cutting an image corresponding to the current progress tracker time into the sprite sheet. The maths to use depend on how sprite sheets are packed in your specific case but this usually simply involves moving the origin of a frame with fixed dimensions in x and y directions. An alternative simpler (and more modern IMHO) approach, visible in YouTube e.g., is to display the thumbnail directly on top of the VideoView during interaction. This spares you from maths involved in ensuring the small window which would be displayed above the slider remains within the player visible bounds and device safe area.