"build/lib" Directory is not cleaned up on bdist_wheel

[mrh1997]

The bdist_wheel command (and probably also other bdist* commands) could result in not-reproducible results:

• create project with file_a.py and file_b.py
• run setup.py bdist_wheel
• rename file_b.py to file_c.py
• run setup.py bdist_wheel (without touching "build" directory in the meantime)
• the resulting wheel will contain file_a.py, file_b.py and file_c.py

This is especially problematic when transforming a module from a single python file to a directory (or vice versa). In this case both module implementations are delivered.

[tobi45]

I just want to emphasize your attention on this issue. Although I don't regularly do such things, I just spend several hours spread over three days because of this issue.

I want to exclude test packages from source and wheel build. Source build worked fine. Wheel build includes test packages every time and I couldn't figure it out. I removed temporary files like egg-info and __pycache__. I even debugged into the find_packages function just to find out that my exclude rules work correctly. I had to clone my repository, run the wheel build again and do a directory diff just to realize that there is a build/lib folder which contents gets regularly updated. But files are only added and never removed from there. I did a lot of research and found some unsolved questions on stackoverflow related to exclude test packages - I can imagine that others might had the same problem.

It would be really great if setuptools would remove all temporary files and directories automatically without manual intervention. I think a tool should care about the files it creates and shouldn't shift that work to the user.

[NateScarlet]

A workaround is to use python ./setup.py bdist_wheel -b build so entire build directory got deleted after build.

This result lib folder got packed

[da1910]

I've also wasted several hours with this issue, we need to ship different libraries for different platforms, and this behaviour was causing issues where the wrong libraries were included in the wrong wheels.

If setuptools is happy to create a build/lib folder, then it absolutely should clear that folder before writing to it. Otherwise disambiguate different builds. The existing behaviour is unexpected and has caused issues for years with building and installing packages.

[abravalheri]

Hi @da1910 , which version of setuptools are you using? Also, is there any chance you using SETUPTOOLS_USE_DISTUTILS=stdlib?

Recently there were some improvements in terms of disambiguation for different platforms that might have improved this behaviour.

[da1910]

Assuming that's a build-time environment variable, then no that's not set in my case.

I'm currently using setuptools 62.1.0 for the process. I'm seeing a build directory created, with an empty "bdist.arch" folder and a lib folder which contains an sdist of the package for that build. It is not cleaned before or after the build, so anything in one build that's not overwritten persists.

[abravalheri]

I believe that the recent changes will not still not automatically perform any clean (the caching situation is analogous to other build methodologies such as make; make install, that might require explicitly cleaning).

Instead, they will use a more refined directory naming using sys.implementation.cache_tag to distinguish between incompatible platforms.

[mhsmith]

This looks like a clear violation of the following rule from PEP 517:

The backend may store intermediate artifacts in cache locations or temporary directories. The presence or absence of any caches should not make a material difference to the final result of the build.

Related: #3236.

[con-f-use]

Not to mention that running rm -rf src/*.egg-info/ ./dist/ ./build/ all the time, because you want a clean source tree, can get tedious.