Create lit test for copyright header validation

[AndrewSB]

cc @gottesmm @shahmishal

I think I have this working when I pass one file into the script, whats the right way to spell "give me all the files so I can xargs them into a script" in lit?

[benrimmington]

This could be useful as a script in swift/utils, for updating the copyright notices of all source files.

It could use the --follow option to get file history beyond renames, and possibly the --find-renames=NN% option to adjust the similarity index.

For example, the commit logs for Bool.swift will end with:

• 29cebdf (June 2013) if using --find-renames=50% (the default)
• 249d66f (June 2012) if using --find-renames=33%

git log               \
  --find-renames=33%  \
  --follow            \
  -- "./stdlib/public/core/Bool.swift"

I've filed SR-7507 to improve the code templates, so that new source files can use Copyright (c) 2018 instead of Copyright (c) 2014 - 2018.

[shahmishal]

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

• How to change the base branch: (Link)
• More detail about the branch update: (Link)

[tbkka]

Please remove the text "for silly legal reasons". Only lawyers are allowed to opine on whether legal issues are silly or not. 😄

[AndrewSB]

quite sage

committed 👍

[tbkka]

@swift-ci Please test

[tbkka]

@swift-ci Please test

[tbkka]

Looks like we have a source file somewhere that has malformed UTF-8? Unfortunately, the script just crashes without reporting the name of the file. We should fix that.

******************** TEST 'Swift-validation(linux-x86_64) :: Python/copyright-conformance.swift' FAILED
: 'RUN: at line 3';   python3 /home/build-user/swift/utils/copyright-conformance.py -rf /home/build-user/build/buildbot_linux/swift-linux-x86_64
--
Exit Code: 1

Command Output (stderr):
--
  File "/home/build-user/swift/utils/copyright-conformance.py", line 84, in has_copyright_notice
    contents = f.read()
  File "/usr/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 8: invalid continuation byte

[tbkka]

This line can fail. We should catch any error that happens here and print out the candidate_filepath

[AndrewSB]

yup, i see. it looks like some non-unicode files have been added to the tree in the last 4 years

i've pushed a change that skips all files that fail to read with a non-unicode error

[AndrewSB]

@tbkka i also went ahead and rebased, then tested the script. it corrected 8785 files, you can see them here AndrewSB@1621206

once this is merged, i'll open that as another PR