Some columns from Postgres DDL are not parsed

[burakuyar]

Describe the bug
Some columns with type character varying are not parsed when NOT NULL constraint does not exist for the column. Please see 3 example inputs and outputs below.
CREATE TABLE statements are generated by pg_dump command in Postgres.

To Reproduce
run result = parse_from_file(path_to_ddl_sql) for the inputs below:

• Input-1:

CREATE TABLE myset.mytable (
    id_a character varying,
    id_b character varying,
    id_c character varying
);

• Output-1: There were 3 columns in the input but only 1 column is parsed.

[{'alter': {},
  'checks': [],
  'columns': [{'check': None,
               'default': None,
               'name': 'id_a',
               'nullable': True,
               'references': None,
               'size': None,
               'type': 'character varying',
               'unique': False}],
  'id_b': 'varying',
  'id_c': 'varying',
  'index': [],
  'partitioned_by': [],
  'primary_key': [],
  'schema': 'myset',
  'table_name': 'mytable',
  'tablespace': None}]

• Input-2: Added NOT NULL constraint to id_b column.

CREATE TABLE myset.mytable (
    id_a character varying,
    id_b character varying NOT NULL,
    id_c character varying
);

• Output-2: There were 3 columns in the input but only 2 column is parsed.

[{'alter': {},
  'checks': [],
  'columns': [{'check': None,
               'default': None,
               'name': 'id_a',
               'nullable': True,
               'references': None,
               'size': None,
               'type': 'character varying',
               'unique': False},
              {'check': None,
               'default': None,
               'name': 'id_b',
               'nullable': False,
               'references': None,
               'size': None,
               'type': 'character varying',
               'unique': False}],
  'id_c': 'varying',
  'index': [],
  'partitioned_by': [],
  'primary_key': [],
  'schema': 'myset',
  'table_name': 'mytable',
  'tablespace': None}]

• Input-3: Added NOT NULL constraint to id_c column, too.

CREATE TABLE myset.mytable (
    id_a character varying,
    id_b character varying NOT NULL,
    id_c character varying NOT NULL
);

• Output-3: All 3 columns in input are parsed.

[{'alter': {},
  'checks': [],
  'columns': [{'check': None,
               'default': None,
               'name': 'id_a',
               'nullable': True,
               'references': None,
               'size': None,
               'type': 'character varying',
               'unique': False},
              {'check': None,
               'default': None,
               'name': 'id_b',
               'nullable': False,
               'references': None,
               'size': None,
               'type': 'character varying',
               'unique': False},
              {'check': None,
               'default': None,
               'name': 'id_c',
               'nullable': False,
               'references': None,
               'size': None,
               'type': 'character varying',
               'unique': False}],
  'index': [],
  'partitioned_by': [],
  'primary_key': [],
  'schema': 'myset',
  'table_name': 'mytable',
  'tablespace': None}]

Expected behavior
All 3 columns from Input-1 should be parsed into Output-1.

Desktop (please complete the following information):

• OS: MacOS Monterey 12.2
• Python 3.8.9 | 3.8.12
• Package Version: 0.25.0

[burakuyar]

Hi @xnuinside , do you think this is an easy-to-fix issue?
I will try to check the root cause but I am not very experienced with lexers.

[xnuinside]

@burakuyar already fixed #124 & released at 0.26.0 version. Sorry for the long answer & long release (have crazy last 2 month). Test was added - https://github.com/xnuinside/simple-ddl-parser/blob/main/tests/test_simple_ddl_parser.py#L2571. If you will find any new issues - feel free to open the issue :) I hope I will fix it faster next time.

[burakuyar]

Thank you very much! For sure, I will let you know if I face unexpected cases :)