Potential conflict in grammar rule order?

[irrationalistic]

I'm not sure if this is the right place to check in on this issue.

I've posted to the language repo in question: devongovett/atom-jade#16 (comment)

I've also posted to the atom discussion board: https://discuss.atom.io/t/conflicts-in-grammar-files-example-jade/15254

The overall problem is that, in the Jade grammar (which does work in Sublime Text), there is a rule towards the bottom of the set (https://github.com/devongovett/atom-jade/blob/master/grammars/jade.cson#L258) that is a sort of catch-all fallback. Except that it catches things that were already matched from what I can tell! By commenting that rule out, jade handles comments properly in the highlighter. With that rule, comments in Jade are read as generic tags instead.

As @lee-dohm mentions in the discussion board, he hasn't come across anything that dictates that rules should apply in any order, so if that's the case, any idea why this would work in Sublime but not in Atom? I'm sure it's possible to adjust that rule to be more specific, just want to see if this is a strange issue in the parser. Thanks!

[kevinsawicki]

any idea why this would work in Sublime but not in Atom?

Probably just an implementation difference, there have been occurrences of these in the past.

[irrationalistic]

So it sounds like the main solution is to adjust the final grammar file to match the implementation of the atom runner?

I just wanted to pass this through your filter to see if there is an intent to adjust the first-mate plugin to be more 1:1 with textmate/sublime text grammars! Since it's an entirely different implementation, it's understandable that trying to match all the rare discrepancies would be ridiculous.

If there's nothing left to do but adjust the grammar file, this can be closed!

[kevinsawicki]

if there is an intent to adjust the first-mate plugin to be more 1:1

Yeah, if we can reduce the problem we could possibly fix this in first-mate, but usually with grammars with dozens/hundreds of patterns it becomes difficult to isolate where the problems are coming from.

[irrationalistic]

it becomes difficult to isolate where the problems are coming from

Is there anything I can do to help with isolation in this case? No worries if you don't have any suggestions there, figured I'd ask :)

[kevinsawicki]

Yeah, a PR with a failing test would be great, where you are expecting it to have one scope (the scope that textmate/sublime is using) but Atom parses it with a different scope.

[irrationalistic]

Right on! Will do, thanks :)

[kevinsawicki]

Will do, thanks :)

Awesome, much appreciated 👍

[irrationalistic]

Let me know if this is helpful or needs more information!

[genoma]

Any news on this issue?

[szimek]

Any updates?

[kevinsawicki]

No updates on my end, I still haven't reduced the grammar in #39 to better isolate the problem.

[Ingramz]

'scopeName': 'text.priority'
'name': 'Priority test grammar'
'patterns': [
  {
    'patterns': [
      {
        'match': 'a'
        'name': 'one.priority'
      }
    ]
  }
  {
    'match': 'b'
    'name': 'two.priority'
  }
  {
    'match': '.'
    'name': 'three.priority'
  }
]

In this example with test data abc we should be matching a with one.priority, b with two.priority and c with three.priority, but for some reason the nested patterns rule loses priority and therefore a gets three.priority. In TextMate, an equivalent grammar would match as expected.

Could this be the issue we're having here?

[izuzak]

@kevinsawicki @nathansobo What do you think of the simplified grammar case from @Ingramz in #38 (comment)? I've been able to reproduce the behavior with first-mate using that grammar, and I think that's not expected.

[nathansobo]

If we can solve it by adjusting the grammar for now, that would be great. I plan another pass on the TextMate parsing for correctness and performance once we finish some planned changes to the DisplayBuffer.

[UziTech]

Any progress on this? It would be really nice to have Heredoc support in PHP