Automated tests

[vvoovv]

Due to complexity of the PML grammar it's desirable to introduce automated tests.

A PML grammar and translated output can be defined right in the test file.

input = """
facade[item.front and style.count<=1] {
  class: facade_with_door;
}
"""

output = """
Facade(
  condition = lambda item : item.front and self.count <= 1,
  cl = "facade_with_door"
)
"""

And the related test

def test_output():
  assert input == output

The problem I see is that the output is too sensitive to white spaces. An additional space character introduced by the translator would invalidate all automated tests.

[polarkernel]

Due to complexity of the PML grammar it's desirable to introduce automated tests.

I like this request. It would be a great help to catch side effects due to changes in pml's code.

The problem I see is that the output is too sensitive to white spaces.

Making money with spaces? ;-) This is really a difficult issue. Maybe it could be solved by two combined tests.

The first test could replace all white spaces by a single space in both, the test pattern and the result from the pml translator and then compare these strings. This would check just the code content.

The second test should check at least the correct indents for every line. Maybe a script like reindent.py or similar could be adapted to 'beautify' both, the test pattern and the output of the pml translator and then compare these results. Maybe this could even be combined with the first test for the remaining white spaces beside the indents.

Using a Python parser could be another possibility for the second part. There exist Python grammars for ANTLR4. Such a parser could check whether the translation is correct Python code, while the first test would test the content.

[polarkernel]

The problem I see is that the output is too sensitive to white spaces.

Maybe I have a simple solution for this issue. The Python function tokenize() reads the same tokens as the Python parser does, skipping unrequired whitespaces. Converting tokens back into Python source code by untokenize() , these whitespaces are eliminated. Applying this sequence to the test pattern and the translator output delivers equal strings. The only thing to pay attention is that tabs in the test pattern are either four blanks or a tab character ("\t"). Not yet thoroughly tested, however I think it should work. See the following code:

from tokenize import tokenize, untokenize
from io import BytesIO

def cleanWS(s):
    s = s.expandtabs(4)
    tokens = tokenize(BytesIO(s.encode('utf-8')).readline)
    t_list = [(toknum, tokval) for toknum, tokval, _, _, _ in tokens]
    return untokenize(t_list).decode('utf-8')

tst_out = """
Facade(
    condition = lambda item : item.front and self.count <= 1,
    cl = "facade_with_door"
)
"""

pml_out = """
Facade(
     condition = lambda      item : item.front    and self.count <= 1,
     cl = "facade_with_door"
)
"""

assert cleanWS(tst_out) == cleanWS(pml_out)

[vvoovv]

I'll commit later today my solution. I use ast.parse(python_code).

[vvoovv]

I commited the first automated test that is supposed to be insensitive to white spaces:
https://github.com/prochitecture/pml/blob/master/tests/test_condition.py

ast.parse(python_code) raises an exception if there is a syntax error in python_code.

" ".join( myString.split() ) appears to be the simplest solution to "normalize" white spaces in myString.

[polarkernel]

I commited the first automated test that is supposed to be insensitive to white spaces:

I like your solution! Could it eventually be extended by providing some info about the reason of a syntax error? Something like this (disable capturing of output to stdout and stderr by command-line option ... pytest -s ...):

def test_style_variable():
    output = getPythonCode(input)
    try:
        ast.parse(output)
    except SyntaxError:
        print(sys.exc_info()[1])
        assert False
    assert compare(output, referenceOutput)

[vvoovv]

But an error is reported in the current implementation.
I added "1" to the python output and got from pytest:

  File "<unknown>", line 6
    ]1
     ^
SyntaxError: invalid syntax

[polarkernel]

But an error is reported in the current implementation.

Well, I didn't try it using pytest. I only did some experiments with the tests of bpypolyskel. When I produced an exception there, pytest terminated immediately, so that I thought it would be good to catch the exception here. When this is not the case, then your version is fine.

[vvoovv]

I've just added another test function with the same content as test_style_variable (just to test if it works). Both functions generate a syntax error. Pytest didn't terminate.

[polarkernel]

Fine, then we have a reliable solution, I think!

[vvoovv]

I simplified the structure of the automated tests even further.

Example:

from . import makeTest


def test_indices_positive():
    makeTest(
"""
level[0:3] {
    class: myclass;
}
""",
"""
Level(
    indices = (0,3),
    roofLevels = False,
    allLevels = False,
    cl = "myclass"
)
"""
    )