Normative: add NumericLiteralSeparator

[rwaldron]

There is nothing surprising here, it's the same spec that's been in the proposal for almost two years and is now in many engines. https://github.com/tc39/proposal-numeric-separator

[rwaldron]

I'll fix that failure this weekend, or sometime soon

[ljharb]

@rwaldron i'm pretty confident i've seen this proposal's tests landed; but if not please update the label :-)

[leobalter]

@rwaldron i'm pretty confident i've seen this proposal's tests landed; but if not please update the label :-)

https://test262.report/features/numeric-separator-literal

Landed and shipping in several mainstream implementations.

[leobalter]

on line 11494 or the current file, the CI is flagging the current production not being present. The proposed solution might fix it.

> 11494 |         <emu-grammar>DecimalBigIntegerLiteral :: NonZeroDigit BigIntLiteralSuffix</emu-grammar>
        |                                                  ^
  11495 |         <emu-alg>
  11496 |           1. Return the BigInt value that represents the MV of |NonZeroDigit|.
  11497 |         </emu-alg>

[leobalter]

I forgot to mention this one.

[leobalter]

I applied the changes directly at @rwaldron's branch as he gave me access to do it.

I also rebased the branch and forced pushed the contents here.

[syg]

This has a lot of duplication for MV. Is there a way for the Str-prefixed variants to delegate to the non-Str-prefixed? The only difference is that the Str- variants do not allow separators, right?

[syg]

Should this be an editorial note instead of a normative clause for the MV semantics? Is MV ever "invoked" on NumericLiteralSeparator?

[bakkot]

I would be in favor of reducing the duplication by introducing a new grammar flag to control whether the separator can be used.

[leobalter]

I would be in favor of reducing the duplication by introducing a new grammar flag to control whether the separator can be used.

I wish this could have been solved earlier or if it were simple as it sounds. I might give it a try but I'm open for suggestions how to better do it.

[rwaldron]

I would be in favor of reducing the duplication by introducing a new grammar flag to control whether the separator can be used.

I actually tried to figure out how to do this with some kind of specialized grammar about a year or two ago, and it just got messy. Since this wasn't brought up during the discussion for advancement, I'm going to assume that you're not hard blocking on it?

[bakkot]

I'm not blocking stage 4, but editorially I would prefer to resolve this before landing rather than after. I'm surprised it got messy, though; certainly I wouldn't want to go this route if it's not clean.

I was expecting this to just be a matter of adding a new [Sep] (or whatever) grammar flag which controlled whether _ could be used, and then passing it through as positive from DecimalIntegerLiteral and negative from StrNumericLiteral (or whatever other appropriate entry points). I can write it up more formally when I get time, if that's not clear. Was that the strategy you found to be messy?

[rwaldron]

Was that the strategy you found to be messy?

No, I'd like to see what you come up with.

I'm curious, what duplication are you referring to?

[bakkot]

I'm curious, what duplication are you referring to?

The thing where there is now a StrDecimalDigits as well as a DecimalDigits (etc). These aren't identical, of course, but they're very similar, and I'd prefer to introduce one new grammar flag rather than nine new productions which are so similar to existing ones.

[rwaldron]

@bakkot I haven't had any time to dedicate to making an experiment of changing this design. I'm also starting to feel like this obstacle you've created is unfair, considering how long this proposal has existed and that it's been accepted through several reviews in its current design. I'm also worried that because it's essentially a redesign of the feature, that such an undertaking could put it at risk of losing Stage 4 acceptance.

If we stick to the process, then this should be allowed to merge as-is and if there are refinements to be made, they can be made in follow up PRs.

[rwaldron]

Rebased!

[syg]

@rwaldron We chatted about this on the editor call.

I want to first make sure there's no misunderstanding. We think the refactoring-with-flags proposal is a mechanical refactoring of the current grammar, not a redesign. It won't have normative changes, and definitely won't require going back to an earlier stage. Further, there won't be future hurdles. Editorially we don't like the blowup in the grammar currently.

Out of respect for your time, we're also happy to take over the branch and do the refactoring if you don't have cycles for it.

[rwaldron]

@syg

Out of respect for your time, we're also happy to take over the branch and do the refactoring if you don't have cycles for it.

That's perfectly fine with me!

[bakkot]

Opened #2151 with the alternative implementation I proposed above.

I also added back @leobalter's 2eaaaa3 (not attributed to him due to git mishaps) to this branch, which appears to have been dropped during a rebase, so that the two PRs would be more directly comparable.