Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce test_exercise duplication + fix reporter statefulness #20

Open
filipsch opened this issue Apr 24, 2018 · 1 comment
Open

Reduce test_exercise duplication + fix reporter statefulness #20

filipsch opened this issue Apr 24, 2018 · 1 comment

Comments

@filipsch
Copy link
Contributor

  • There is a test_exercise function in sqlwhat and shellwhat, but not in protowhat. Seems like test_exercise should be moved there, with a smart way of including the language-specific SCT context and state

  • The reporter depends on a singleton-like failed_test variable, that it uses to build up a payload. It is not required. Thoughts by @machow about this:

I regret never finishing cleaning up Reporter (and taking out failed_test). It would super clean to have reporter just hold options for reporting outcomes, and then rather than carrying around the backend output with it, test_exercise could pass that at the end. See [here](see https://github.com/datacamp/protowhat/blob/fs/test-not/protowhat/Reporter.py#L57-L58). Something like

def fake_test_exercise(output):
    reporter = Reporter()
    try: test_something('bleh')
    except TestFail as e:
        return reporter.build_payload(e)
    return reporter.build_payload(output)
@machow
Copy link
Contributor

machow commented Apr 24, 2018

For some added clarification, reporter.failed_test is leftover from before SCTs would mark this attribute as true, rather than raising an error. Now, it seems like something like test_exercise should fail from two places: running the SCTs raised TestError, or Reporter judges the backends output to contain an error.

In library's like pythonwhat, I think there are a few places that set failed_test (but if changed to raise an error instead, should be okay; also protowhat isn't used there).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants