Metadata rendering "hints" from epub we should consider for manifests

[bduga]

As discussed on the March 18th, 2019 WP call, here is a list of rendering related metadata that epub supports and has found some traction in the publishing/reading system community. This is not an exhaustive list, it is intended to contain only those settings that seem to have actual use.

Metadata can be specified at the publication level (applies to the entire publication), the item level (applies to a section of the publication), or both.

page-progression-direction controls the direction (left or right) that pages should turn when implementing next page functionality. Used extensively in Japan, but has traction for other languages. Frequent use and has been implemented multiple times. Critical to support, otherwise some content will be broken. May not be needed for scrolled content, or for all UIs. Publication level.

flow-[auto|paginated|scrolled-continuous|scrolled-doc] indicates whether the content is intended to be paginated or scrolled, and if scrolled whether it is continuous over multiple items. Differs from CSS @page as that describes styling when content is paginated, this specifies whether pagination should occur. Unclear how common this is in practice, though I believe there are some implementations. Both item and publication levels, but unclear if mixed content exists.

layout-[pre-paginated|reflowable] indicates whether an item should be considered a single, high design "page", or whether it is a stream of 1 or more pages. Some overlap with the flow-* properties, above. Widely used and implemented. Both levels, but unclear how common mixed content is.

orientation-[auto|landscape|portrait] hints about the overall aspect ratio of the content. Could be (and is) used to control how a book opens on phones and tablets (auto switches device orientation). Widely used, sometimes correctly. Often coupled with spread-* properties. Both levels.

spread-[auto|both|none|landscape|portrait] indicates when and how synthetic spreads should be generated (that is, when to put pages side-by-side). Widely used and implemented. Both levels.

page-spread-[left|right|center] whether the first (or only) page of an item should appear on the left or right side of (or centered in) the display when showing more than 1 page (that is, in spreads). Widely used and implemented, particularly in pre-paginated content. When missing this can completely break content. Item level.

viewport defines the aspect ratio of pre-paginated content. May also appear in the document content, so may not be needed at the higher level. Both levels.

linear controls whether the item is part of the linear navigation. When true, this is part of the main publication content, when false indicates where it might appear in a printed publication but indicates that it is not part of the main, linear navigation of the publication. Implementations use this as a hint for how and where to display the content. Item level.

[frivoal]

The comment I left in w3c/wpub#207 (comment) is applicable to this issue as well.

Quoting the relevant part here for convenience:

We have two similar issues here:

• Styling on the web should be done in CSS. If we want CSS to solve styling problems that it currently does not solve, we need to work on solving them, and we need to do so by joining and participating in the CSSWG. The CSSWG is not a group of people we need to wait for, it is a place specifically set up for W3C members (that includes us) to discuss that sort of issue. For all the talk about pagination being an important feature for publications, how many of the members of the publishing community have even raised an issue or commented on an existing one in the CSS-WG? Waiting a while for "them web people" to solve our issues, and then rolling our own, separately from the web platform, when “they” don't, is a strong anti-pattern, and we're never going to make anything that is a “first-class entities on the Web” with that approach.

• One of the big difficulty that would get in the way of solving this problem in CSS is that CSS does not have a concept of a publication made of multiple documents stitched together. This is very much not a limitation of CSS that we should work around, it is a limitation of the web platform itself: It's not just CSS that doesn't know how to cope with collections of documents. Neither does the DOM and all other JS APIs; neither does the Single Origin Policy and all other security features; neither does the URL… Fundamentally, this is the same problem as the previous one: waiting a while for "them web people" to solve our issues, and then rolling our own, separately from the web platform, when “they” don't. There's no they; we are W3C members. If we want the web platform to improve, we have to work at it together with the other stakeholders of the web platform, in the various WG chartered to cope with the various parts of the technology stack. I think there are 3 possible ways forward:

  1. We engage with the relevant CGs/WGs that are or could be working on that problem, and try to develop solutions that work for the whole web platform (maybe as part of Web Package, or maybe as a mechanism in HTML). Then we use that as the basis for Web Publications being made of multiple documents.

  2. We give up on the aspiration of making Web Publications a “first-class entities on the Web“. In that case I believe we need to recharter the Working Group, because doing the following sentences form the charter wouldn't be followed (emphasis mine):

     The mission of the Publishing Working Group is to enable all Web Publications [...] to become first-class entities on the Web

     It is the goal [...] to provide, in concert with other W3C Groups as outlined in Section 4.1, the necessary technologies on the Open Web Platform [...]

  3. We give up on the concept of a publication being composed of multiple documents stitched together (aka. “the spine“ or “the reading order“). This probably requires a group recharter too, since that concept is explicitly mentioned as a goal. (edit: Drop the default reading order from WP wpub#302 is about that).

[HadrienGardeur]

IMO some of these are more useful than others:

• page-progression-direction
• flow-*

Both linear and viewport should be dropped:

• we already have "non linear resources" since a resource can be in resources without being in readingOrder
• viewport is deprecated anyway

The rest of them are tied to FXL and I would defer any additional comment on them until we have a proper comics/manga/BD TF.

[iherman]

This issue was discussed in a meeting.

• No actions or resolutions

View the transcript

2.2. Metadata rendering “hints” from epub we should consider for manifests
Ivan Herman: See Manifest issue #21
Wendy Reid: Next issue, moving into publication manifest issues (pulled from WP):
… from Brady, looking at some of the things from EPUB which we might want to consider still using in manifest
… mostly related to page direction – flow, layout, spreads, viewport, linearity, etc.
… there was some suggestion to drop linear and viewport, but the rest
Ivan Herman: To explain the history, when we created the new repo for the manifest, Matt and I transferred a number of old issues across that were relevant
… this is one of those. It may be that it’s not relevant, but that’s how it ended up here.
Dave Cramer: I think as a general principle we should not describe rendering in metadata. The web has a whole language for rendering - I think especially assuming in advance that the EPUB FXL concept of ‘displaying two independent viewports side by side in a particular relationship’ is something that we should not bring to web publications
… I would hope we would say that we’re not going to do rendering in metadata, and leave it at that
Garth Conboy: I disagree with Dave… moving forward we have to figure out how to support FXL - this is clearly irrelevant for audiobooks, so this is a long term defer
Matt Garrish: I agree on deferring generally all of this metadata stuff - the one that interests me here is the page progression direction… there’s no means of establishing that when going from document to document… it might come into play with HTML. That’s maybe the only one we should consider now
… until we have stronger cases for the others, let’s defer
Brady Duga: I’m fine deferring, but wanted to point out these aren’t all FXL.
… page progression direction is the one thing that has no other way of being displayed - CSS is within single documents, nobody has figured out how to implement that across documents
Luc Audrain: +1 to brady
Brady Duga: these are widely used things in EPUBs today which have to somehow get into web publications
Wendy Reid: I’m in strong agreement re: page progression
… because audiobooks have some text in them, we do need this for audiobooks, since we might have a Japanese audiobook with JP-language text attachments
Ivan Herman: There is a direction property in the spec right now which could play this role
Romain Deltour: Agreeing that page progression is not covered by CSS currently, but that’s because CSS is not about a collection of documents - we can’t magically fix by adding this value to manifest - needs more work, including collaborating with CSS working group and others - might need to work with community group on this, rather than arbitrarily picking values
Garth Conboy: I don’t have a strong position, but for audiobooks we do have the ancillary resources. Romain, do you disagree that we need page progression for figuring that out?
Romain Deltour: I’m not familiar enough with the audiobook use case to answer that. Let me think about it.
Garth Conboy: I’m completely willing to think about nothing but audiobooks in this context… an argument could be made that you need to know the order in which to page through ancillary resources
Ivan Herman: I disagree with Romain. What we have is the manifest, it doesn’t say anything about rendering as such… we don’t dictate any way of rendering, it’s just information - we don’t say how a title should be displayed either
… the fact of having the base direction as part of the manifest is perfectly fine. I don’t think that’s on a collision course with CSS
… today we don’t do anything with these documents (??), we know that, but the information is useful for reading systems.
George Kerscher: A reminder that in order to pass accessibility APA, we need a mechanism for providing text transcripts, those need progression information
Dave Cramer: Are we assuming that all ebook ancillary content will be paginated?
Wendy Reid: No
Romain Deltour: To clarify: the primary concern is to avoid any conflict with the web. As long as we stay in the audiobook context, I guess we can do innovative things. But here it’s really about rendering web content, so we should either use existing mechanisms or talk with the groups which define them
… there are questions re: pagination in ancillary content… but do we really need this information? I’m failing to see the real use case here
… rephrasing the issue or making a clearer use case might be appropriate
… if we are to enter the realm of rendering, then we should make a very strong case for anything we do on our own
Brady Duga: Building on what Romain and Dave said - it’s not clear to me that we know how the ancillary content is going to actually be used by reading systems. George made a comment about transcripts, but I don’t know if the reading system needs to know the progression direction - it might just display text on the screen
… I don’t know how this information will be used. We’re trying to cover every possible use of every possible content… we should wait to see how it’s used… I’m not planning to glue the ancillary content together into something like an ebook (not a product announcement, just something I don’t imagine doing)
… I can see there’s a reason we could want this, but it doesn’t seem to me like that’s sufficient to get into the really difficult discussions around CSS before we discover the use cases
Ivan Herman: We have two terms in languages in the document which set the language and the direction. I realise that reading the text can be a problem. The current manifest defines language and base direction for the manifest textual items and the publication as a whole.
… we may want to remove the reference to the publication and refer to the manifest’s textual terms only
Wendy Reid: We don’t want to add the text direction property to our metadata, at least not just yet
… I think we want to keep in the language and direction properties as they are… if reading systems use them, that’s their decision. But during implementation, we should be keeping in contact with implementors to see what they want to do with this ancillary content
… do they need additional information to render it, do they want to infer themselves, do they have a better idea for how we do it
… happy to postpone until we have more opinions on this point.
… we could send this to the publishing community group and ask them to find a solution
Ivan Herman: Do we want to remove references to the direction and language of publication from the manifest? That’s what we need to decide
Wendy Reid: I don’t think we should remove them
… In-language is kind of important for a reading system or user agent to make inferences
… I’m adding the postpone label to this issue.