STAT Format 2 ranges exclusive vs inclusive

[punchcutter]

While checking various implementations I found that the Microsoft STAT implementation for STAT Axis value table Format 2 assumes inclusive ranges (I think? DirectWrite doesn't expose controls to access arbitrary coordinates). So, for example, an Axis value table with nominal/min/max of 400/350/450 includes the max value 450 in the range. Samsa and Adobe apps assume exclusive so the range effectively ends at 449. I believe this needs clarification in the spec first and implementations should be updated to follow.

[anthrotype]

Indeed, this needs clarification in the spec. I suggest you raise the issue at https://github.com/MicrosoftDocs/typography-issues if you haven't already.

[Lorp]

Samsa is in fact already considering >= and <= as you can see in samsa-gui.html line 2039:

(avt.format == 2 && fvs[tag] >= avt.min && fvs[tag] <= avt.max))

I believe the problem is an ambiguity in the spec:

Two format 2 tables for a given axis should not have ranges with overlap greater than zero.

Given that, as you say, STAT ranges are inclusive of their extrema and STAT values are fixed point (16.16), the ranges in the Adobe Source fonts (Medium [450,550], SemiBold [550,650], etc.) can be said to overlap by 1/65536. In practice I assume Adobe is relying on this section of the spec for handling such cases:

• If the range of T1 overlaps the higher end of the range of T2 with a greater max value than T2 (T1.rangeMaxValue > T2.rangeMaxValue and T1.rangeMinValue <= T2.rangeMaxValue), then T1 is used for all values within its range, including the portion that overlaps the range of T2.
• If the range of T2 is contained entirely within the range of T1 (T2.rangeMinValue >= T1.rangeMinValue and T2.rangeMaxValue <= T1.rangeMaValue), then T2 is ignored.

Samsa does not currently implement these rules. The first of these applies in the Adobe fonts such that T1 (the higher range) should take precedence. That is, a value of 550 should invoke SemiBold rather than Medium. And that is what Samsa’s STAT determination actually achieves in the Source fonts, because it uses the last AxisValueTable that matches, and the AxisValueTables in the Source fonts are in ascending order.

This is not the only thing going on, though. The value 550 is round-tripped through the normalization process, becoming 549.9922337669204 (0x11b8 normalized), which is unambiguously within the Medium range, so Samsa ultimately determines Medium for Weight 550.

While writing Samsa as well as Axis-Praxis I’ve been thinking about all the various roundings that take place, and the rounding described above is possibly incorrect. Yet I don’t think the spec is sufficiently clear about how to think about axis locations, rounding and mappings. I sometimes wonder if a font context should ideally keep track of user input value (string), user input value (float), user input value (16.16), normalized value (float), normalized value (2.14)… And is it valid to generate user values from normalized values? One might think so, but what about when avar maps adjacent input values onto a single output value, leading to divide-by-zero for the reverse case?