Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Several matchers (unordered, containsAll) don't report which element is missing #142

Open
spkersten opened this issue May 8, 2020 · 2 comments

Comments

@spkersten
Copy link

spkersten commented May 8, 2020

With matchers like unordered and containsAll, if the value that it is matched against has a different length (or is shorter in case of containsAll) than the expected value, the output of a failed test is:

Expected: equals [
            many items
            ...
          ] unordered
  Actual: ExpandIterable<List<Foo>, Foo>:[
           many items
            ...
          ]
   Which: has too few elements (39 < 40)

Because the lists are truncated it is not even possible to see what is the mismatch. It would be more useful if the output was the elements that are missing/superfluous, like it does when the lengths are the same.

@natebosch
Copy link
Member

I think they are designed to bail out as soon as possible rather than spend (potentially a lot) of time working on finding multiple reasons for a mismatch. I suspect if we dig on this we'll find a tension between the needs of different users and I'd strongly prefer not to introduce a flag or something. I think we might almost be able to justify increasing the time complexity in more cases if we expect the match case to be more common than the mismatch case, but I do worry about cases like isNot(equals(someOtherList)).

I also worry that there may be two goals here which aren't aligned. One goal is to get a quick and hopefully actionable report of problems. Another goal is to deeply debug some problem. In the former case the very furthest we'd go is probably to report the first index of a mismatch even when the lengths don't match. In the latter case you might want more than that, maybe a full diff algorithm to be able to tell the difference between a single item that was inserted, vs completely disparate lists.

I'll keep this open to consider, but I don't anticipate any short term changes. Thanks for the feedback!

@spkersten
Copy link
Author

I think performance concerns don't apply as strongly to each of them. I can image isNot(equals(someOtherList)) being done commonly. However, I wonder how common isNot(containsAll(elements)) or isNot(unordered(elements)) are.

Another approach (probably requiring a lot more refactoring) could be to only construct the message when the expectation fails. Or, to construct it lazily.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants