[BUG] pyproject.toml allows in unsanitized license text that breaks METADATA format

[ivany4]

setuptools version

setuptools==67.2.0

Python version

Python 3.9.7

OS

Ubuntu Linux

Additional environment information

This happens only with pip>=22.3

Description

When specifying project metadata in pyproject.toml, a license filename can be injected via project.license.file key.

This license file may contain some illegal characters, that get copied into METADATA file when a wheel is built using setuptools. The resulting METADATA file cannot be fully parsed, and pip ignores any metadata that is contained below the suspect character.

I have made a proof of concept repository here: https://github.com/ivany4/metadata_poc, which reveals the problem. In its license.txt, there's a new line with the illegal character. If you pip install this project, its dependencies get ignored completely because in the METADATA they are listed below the license text. In the same project, I've included unused_setup.py, which proves that the same license file gets properly sanitized via the setuptools.setup function. So this problem applies only to pyproject.toml. (I did not try setup.cfg).

As per core metadata spec, field contents of the METADATA must be parseable by the standard library email.parser.

Note that using pip ==21.* or <22.3 correctly removes the character from the METADATA, with the same setuptools version.

Expected behavior

Illegal character is removed from the license text before inclusion into METADATA file.

How to Reproduce

1. Clone the PoC project git clone https://github.com/ivany4/metadata_poc.git && cd metadata_poc
2. Create a new virtual environment
3. Install pip of the specific version python -m pip install 'pip>=22.3'
4. python -m pip install . the project
5. Notice, that it does not install any dependencies, even though requests is listed as a direct dependency in pyproject.toml

Output

/metadata_poc$ python -m pip install .
Looking in indexes: ...
Processing /metadata_poc
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: metadata-poc
  Building wheel for metadata-poc (pyproject.toml) ... done
  Created wheel for metadata-poc: filename=metadata_poc-0.0.1-py3-none-any.whl size=1223 sha256=be7410904f5994525fc3a6400220d6d62672a8cbd0b3467dab19acbc05573fde
  Stored in directory: /tmp/pip-ephem-wheel-cache-045lr9xh/wheels/03/00/b3/b0ad8e69415c7904262e68f122892056c945f4a7575ad04678
Successfully built metadata-poc
Installing collected packages: metadata-poc
Successfully installed metadata-poc-0.0.1