Deduplicate compiler messages.

[ehuss]

Describe the problem you are trying to solve

In some cases, Cargo builds the same code in parallel. For example, cargo test will build the library as a test, and concurrently as an rlib to link for doctests or integration tests. This can cause confusion when compiler messages appear twice.

Describe the solution you'd like

Cargo should track which messages have been generated, and remove duplicates.

Notes

There are some challenges with how this will be implemented. For example, rustc emits some summary messages that can be difficult to deduplicate correctly, particularly the warning count warning: 1 warning emitted. The implementation will need to carefully consider these edge cases.

This may also need to consider displaying a summary of deduplicated (hidden) messages, although I am less certain about that.

[rsalmei]

Couldn't the set of errors and warnings be captured internally, from all the concurrent builds, and just output one of them?
I presume they would be strictly subsets of one another, so maybe just output the longest one.

[ehuss]

The intent is that there would be a set of hashes of previous messages, and if the set contains a hash of a message, to skip displaying it.