Skip to content

Commit

Permalink
Fix parsing of record & list bodies. (#94)
Browse files Browse the repository at this point in the history
### Brief Description
Change the node structure of the record & list bodies to better
represent the actual syntax of the language. This fixes some bugs as
well as make the syntax more consistent & easier to read.

### Fixes
- TS can't properly detect durantion values in records. It is instead
deteced as integer & a command node that create an error:
```nu
{
    key: 5sec
}
```
- TS can't properly detect when value & key begins & allow invald syntax
with value & key having no space or any separator in between. The
following example is considered valid by TS:
```nu
{
    key: 1536another_key: 8549
}
```

### What is changed
- Records use a new body structure which manages entries & separators in
between. This fixes the separator bugs described above & creates new
nodes that can be used for highlighting & indenting.
- Lists also have their structure changed in the same manner (both
records & list body acts the same in nushell syntax, with exception to
what is actually a record entry ``key: value`` vs a list entry
``value``.
- Record tests are added for the bugs described above. All tests have
been updated with the new node syntax.

### Details
- New function is added to grammar js for general body structure. It
consist of repeated statement all seperated by a mandotory separator,
followed by a final statement with an optional separator.
- This function is applied to the bodies of lists & records with the
corresponding entries.
- List has an added node for the list_entry itself (before it was just
hardcoded into the list rule)
- Separator is added for entries ``[,\s\n\r]``
- Comment is added on top of the code to improve LSP parsing of code
with tsserver (commonly used by other TS projects for grammar.sj)
- Tests are updated with new new node syntax - I've individually checked
the diffs of all tests to make sure they are working as indented
- New test has been added to check if records can accept duration input.
- New test has been added to test if records accept invaild input with
no separator between value & key.

---------

Co-authored-by: CabalCrow <[email protected]>
  • Loading branch information
CabalCrow and CabalCrow committed May 26, 2024
1 parent 8f29630 commit 4fc9b88
Show file tree
Hide file tree
Showing 16 changed files with 274,137 additions and 237,217 deletions.
58 changes: 41 additions & 17 deletions grammar.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
/// <reference types="tree-sitter-cli/dsl" />
// @ts-check
module.exports = grammar({
name: "nu",

Expand Down Expand Up @@ -1080,25 +1082,28 @@ module.exports = grammar({
val_list: ($) =>
seq(
BRACK().open_brack,
repeat(
field(
"item",
seq(
choice(
$._list_item_expression,
alias($._unquoted_in_list, $.val_string),
alias($.short_flag, $.val_string),
alias($.long_flag, $.val_string),
alias($._list_item_starts_with_sign, $.val_string),
),
optional(PUNC().comma),
),
),
),
optional($.list_body),
BRACK().close_brack,
optional($.cell_path),
),

list_body: general_body_rules("entry", "val_entry", "_entry_separator"),

val_entry: ($) =>
prec(
10,
field(
"item",
choice(
$._list_item_expression,
alias($._unquoted_in_list, $.val_string),
alias($.short_flag, $.val_string),
alias($.long_flag, $.val_string),
alias($._list_item_starts_with_sign, $.val_string),
),
),
),

_list_item_expression: ($) =>
choice(
$._value,
Expand All @@ -1116,11 +1121,20 @@ module.exports = grammar({
val_record: ($) =>
seq(
BRACK().open_brace,
repeat(field("entry", $.record_entry)),
optional($.record_body),
BRACK().close_brace,
optional($.cell_path),
),

record_body: general_body_rules(
"entry",
"record_entry",
"_entry_separator",
),

_entry_separator: (_$) =>
prec(20, token(choice(PUNC().comma, /\s/, /[\r\n]/))),

record_entry: ($) =>
seq(
field(
Expand All @@ -1147,7 +1161,6 @@ module.exports = grammar({
alias($.cmd_identifier, $.val_string),
),
),
optional(PUNC().comma),
),

_record_key: ($) =>
Expand Down Expand Up @@ -1278,6 +1291,17 @@ module.exports = grammar({
},
});

function general_body_rules(field_name, entry, separator) {
return ($) =>
prec(
20,
seq(
repeat(seq(field(field_name, $[entry]), $[separator])), // Normal entries MUST have a separator
seq(field(field_name, $[entry]), optional($[separator])), // Final entry may or may not have separator
),
);
}

function parenthesized_body_rules(suffix, terminator) {
const parenthesized = "_parenthesized";
return {
Expand Down
Loading

0 comments on commit 4fc9b88

Please sign in to comment.