export to mathml-core

[johanneswilm]

Hey,
I am using the convertLatexToMathMl() function. I have been clicking through https://cortexjs.io/mathlive/ but cannot find it there.

My question is: The mathml that is returned, will it be adhere to mathml-core or mathml 3?

[arnog]

That's a good question. I believe it's MathML-Core, but it's hard to tell.

I couldn't find a validator for MathML-Core, so not sure how to verify that.

I also couldn't find a clear description of what is no longer allowed in MathML-Core that was allowed in MathML-3. The best I could find is this but it focuses on what is allowed in MathML-Core and only alludes that there are 163 elements which are no longer allowed, without specifying what they are.

Note also that the MathML-Core spec is quite open-ended on what it includes or may include in the future:

The details of which math feature will be included in future versions of MathML Core or implemented as polyfills is still open. This question and other potential improvements are tracked on GitHub.
(from [https://w3c.github.io/mathml-core/#mathml-fundamentals](MathML-Core Spec)

So some features that are currently in MathML-3 (or MathML-4) might become part of MathML-Core at some point.

The Notes on MathML document unfortunately does not provide much additional information but perhaps future editions will.

[NSoiffer]

One place to look is the polyfills that I implemented. That gives you a good sense of what is part of MathML 3 that is not in core since I had to write polyfills for them. I tihnk there are a few other minor things that I didn't polyfill because the Core spec changed slightly since I wrote them.

Probably the three most used features not supported in MathML core are:

1. mfenced -- there is an equivalent and more powerful version using mrow
2. menclosed -- there is no equivalent that words everywhere. Most notations can be handled via CSS for chrome, but CSS isn't supported in Firefox & Safari. You can make use of polyfills (e.g, https://w3c.github.io/mathml-polyfills/menclose/menclose.js with these examples/tests
3. mover/munder -- you need to explicitly add accent and accentunder -- they don't inherit based on the base

Also something to watch out for is that (despite many objections), one must use U+2212 rather than the ASCII "-".

There are larger polyfills for linebreaking and elementary math, but browsers have never supported them so they weren't generated by most software.

As Arno indicated, MathML Core (like CSS) will hopefully add functionality over time. First though, we need to get browsers to have better interoperability with MathML Core.

@arnog: are any of the top three generated by MathLive?

[arnog]

@NSoiffer yes, it does generate mover and munder, but nor for accents, but for superscripts and subscripts.

[johanneswilm]

@arnog Do I read correctly here that it also produces <menclose>?

mathlive/src/formats/atom-to-math-ml.ts

Line 1001 in 69e0374

result = '<menclose notation="box"';

That also seems to not be supported. It looks like they have tagged the parts not supported as deprecated on MDN: https://developer.mozilla.org/en-US/docs/Web/MathML/Element/mfenced (see left column)

[arnog]

You are correct. I searched for menclosed, but I think Neil meant to say menclose.

[NSoiffer]

Yes, menclose. I see I made several typos in my reply :-(

\hat{x} (\bar{x}, etc.) should generate

<mover accent='true'><mi>x</mi><mo>^</mo></mover>

It appears that mathlive generates the correct MathML core output.

[johanneswilm]

@arnog @NSoiffer I'm confused. Mathlive outputs <menclose> which is not part of MathML core, right? So the output syntax cannot be rendered by the default browsers for now. Or how is that?

[NSoiffer]

Only Chrome is fully MathML core compliant. The other browsers had implementations before Chrome (and before MathML core). They implemented most of MathML 3 (except line breaking and elementary math), but now they are playing catch up to deal with the CSS integration in MathML Core (among other things).

MathML Core is adopting the "level" approach of CSS. So additional levels of MathML Core will eventually come out and they will add features. I suspect that menclose may be one of those features. menclose can be displayed with CSS, but you lose accessibility. For example., <menclose notation='updiagonalstrike downdiagonalstrike>...` is a cross out, and hence is semantically meaningful.

[johanneswilm]

Ok, but the two Chromium-based browsers Chrome and Edge have a combined marketshare of 70+%, so it's not insignificant. If no development is taking place in creating another level of MathML core, then it can take decades for another level to be published.

[johanneswilm]

The remaining two browsers also plan to switch to mathml core:

Webkit issue: https://bugs.webkit.org/show_bug.cgi?id=195797

Gecko issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1495813
So at least in the browser world it looks like MathML 3 is deprecated.

---

In my project I use mathlive to create output files (among other things) that the end users archive or publish, and so it should preferably be syntax that is following current W3C specs and can be rendered in browsers. I was hoping I could export to MathML Core, but it sounds like that is not the case. @NSoiffer are you saying that <menclose> & other deviations from MathML Core will be displayed correctly, even in Chrome, if one adds certain CSS? Or will further changes to the syntax be necessary before it is being displayed correctly?

[arnog]

I'm not sure what MathLive can do about this issue. Are you suggesting I drop the generation of <menclose> because some browsers may not be able to render it in their current implementation? How would that be better? As Neil pointed out, <menclose> can be semantically significant.

[johanneswilm]

@arnog Looking at w3c/mathml#216, it looks like <mrow> can be used in some cases.

I understand the problem that MathML Core is not delivering as much as MathML 3. Unfortunately it looks like MathML Core will replace MathML 3 in browsers and although there are efforts by @NSoiffer and others to try to add more stuff to MathML Core, as far as I can tell from browsing through the various issues, none of that is being picked up by the browser makers who are happy to just ship MathML Core in its current form - and for Wbekit/Gecko to unship the parts of MathML 3 that are not in MathML Core.

The perspective of people who need to use the output from MathLive's MathML is that at the very least we need to know which version this is for. If the output is MathML 3, then that is not great, but at least it's specific. Then I may need to create some filter that dumbs the MathML 3 output down to MathML Core so that it can be displayed (In Chromium now and in Safari/Firefox once they have implemented MathML Core).

[johanneswilm]

What is a bit unclear to me is whether MathML 3 in general is to be seen as being replaced with MathML Core or whether it is meant that MathML continues to be developed and one is supposed to publish documents with full MathML 3 and then include these polyfills: https://w3c.github.io/mathml-polyfills/ .

It should be possible to rewrite these polyfills slightly so to run them over the MathML 3 output of MathLive if the idea is that MathML is to be replaced entirely.

Based on the source code of the polyfills, it looks like all cases of <menclose> can be replaced by <mrow>.

[arnog]

OK, if the issue is knowing which version of MathML MathLive outputs, let's clarify this by saying it's MathML 3.

[johanneswilm]

@arnog Thanks!

[NSoiffer]

My understanding of what is going on with Webkit and Gecko issues is that they are trying to become more compatible with MathML core and reducing the number of web platform test failures. I suspect progress will continue to be slow as the big players behind the browsers aren't devoting many resources to the effort (most work is done by outsiders such as Igalia). Supporting core doesn't mean that they are planning to remove features they currently support (such as menclose) that are a superset of MathML core -- that wouldn't affect their success rate. Of course, if continuing to support a feature requires work, then it wouldn't be surprising if that feature is dropped. I don't think that is the case for menclose though.

As for "full" MathML, work continues on that. The main focus of MathML 4 is alignment with core (to make it a proper superset) and the addition of author intent to improve accessibility. For example., is $|x|$ absolute value, a determinant, cardinality, etc? It affects how something is spoken and very occasionally how it is brailled. I wish I could tell you when MathML 4 will move to CR, but the wheels of progress are slow and uneven. See https://w3c.github.io/mathml for the current draft if you are interested. We are/have been for months developing a list of "core" and "open" concepts/properties for the intent attribute.

Hopefully at some point in the future MathLive will start to generate intent for ambiguous notations since it is interpreting the notation for the compute engine. I plan to help with that effort. I know that @arnog doesn't like to have open issues with no plans for resolving them, so I haven't opened an issue yet on this. However, once we start the process of moving to CR, I'll open an issue since that will be a time to start work on an implementation (which I hope to help with). I suspect most of what is needed already in the code.

[johanneswilm]

Supporting core doesn't mean that they are planning to remove features they currently support (such as menclose) that are a superset of MathML core

If you look at the bug reports I linked above, they actually state that they have and will remove MathML 3 features from Firefox and Webkit. For example <menclose notation="radical"> that used to be supported in Firefox and that can be found in MathML 3 was unshipped in 2019 [1]. I couldn't find any gecko bug report to unship all of <menclose> even although they start out by saying that things that can be done with Igalia's polyfills should be unshipped.

On MDN <mfenced> is marked as deprecated [2] and it was removed from Firefox five years ago [3] despite it being part of both MathML 3 and MathML 4.

So it does look like they are removing features that re not part of MathML 3, but not all of <menclose> so far.

Conclusion for this: The difference between MathML3 and MathML Core output from MathLive may just be a question of whether to use <menclose> or <mrow> with various style attributes as that looks like is the only non-core thing that is added here. The easiest way to handle that would be to add a switch between Simplified MathML 3 and MathML Core output. Or - if @arnog doesn't like that solution - an added script that modifies the output of MathLive to make it fully MathML Core compliant.

[1] https://groups.google.com/u/1/g/mozilla.dev.platform/c/vwAkuZIEhnY/m/KALRPR3wAQAJ

[2] https://developer.mozilla.org/en-US/docs/Web/MathML/Element/mfenced

[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1587577

[NSoiffer]

You are correct that mfenced and <menclose notation="radical"> were removed. However, both were redundant with existing MathML features, so no functionality was lost; removal was an easy thing to justify. You'll notice that none of the reports mention removing menclose. It is listed as a potential level 2 . Removing it would potentially be a waste of effort given that it might be part of the spec in the future.

FYI: mfenced in MathML has been controversial since its inception given that there always have been a more powerful mrow equivalence, albeit one that is longer.

A switch that allows targeting MathML Core or MathML 3 might be a good idea. I'll leave that judgement to @arnog.

[johanneswilm]

@NSoiffer I came to the same conclusion in relation to those features. It's just too bad there is no spec version anywhere that contains core level 1 + menclose. That would make it easier to ensure compatibility.
I had found your proposal on inclusion into level 2 earlier. I wonder if the browser developers see any need for having it, as the main reason seems to be to keep the semantic information intact. The browser makers may think it is enough to be able to display formulas correctly using mrow. If that is the case, we may be stuckin this situation indefinitely.

[arnog]

MathML confuses me. Between MathML 3, MathML 4 (with a Presentation and Content variants of each), MathML Core and the upcoming MathML Core w/ intents, not counting the actual MathML implemented by various browsers, it's hard to keep it all straight. The lack of conformance suite is not helping.

To give some historical context, the MathML support in MathLive was implemented by Neil so that screen readers could interpret the MathML and present it to users in an appropriate way. I believe NVDA uses MathCAT to read MathML. I'm not sure about other screen readers. I'm also not sure of how valuable this is, since MathLive also has a built-in LaTeX-to-speech engine and also support SRE. Maybe this MathML support is just more aggravation than it's worth. I'm really not sure what the benefit of outputting MathML over LaTeX is.

I don't have the bandwidth or the expertise to make improvements/changes to the MathML support in MathLive. I don't know if it makes sense to even keep what's there. Adding support for MathML Core with Intents in MathLive does not make sense since MathLive is a visual editor and has no semantic understanding of the input. If anywhere, it would make much more sense in Compute Engine. Perhaps the way forward is a separate project that does MathJSON to MathML Core w/ Intents conversion, and perhaps in the other direction too.

Anyway, if someone with the necessary expertise is willing to make improvements to the current support for MathML in MathLive, I'd be happy to consider submitted PR but I'm unlikely to spend some time myself on it.