-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
error: each element of 'ext_modules' option must be an Extension instance or 2-tuple #309
Comments
Original comment by gmturner (Bitbucket: gmturner, GitHub: gmturner): https://bugs.python.org/issue23102. The basic recipe: step 1) in setup.py, do lots of from distutils.foo.bar import baz type stuff. Monsters are hiding under the beds of steps one, three, and four. First we save off a reference to the un-monkey-patched distutils.extension.Extension, then we (indirectly) cause the monkey patch by importing some setuptools modules, and then we create the Extension instance, using the un-monkey class reference stored in the global namespace. Now we're off to the races: hand it to distutils (not setuptools), and it will iterate ext_modules checking that all the things in there are two-ples, or else distutils.extension.Extension instances. But since that is monkey patched, now, they aren't. Distutils therefore branches directly to step five. Resonable people could disagree, but to me, it seems like a setuptools bug, insofar as, i.e., just because some bit of code, somewhere, imports pkg_resources does not at all seem to reliably signify that setup.py is setuptools-base (and that therefore it's a good idea to start monkey-patching distutils). I have no good ideas for how to fix this in setuptools without some novel innovation. Then again I really don't know how setuptools works. But it just "sounds hard". I.e., we could: Make setuptools Extension a true subclass of the un-monkey-patched distutils.extension.Extension; do not monkey patch it. Make setuptools Extension a virtual subclass of the un-monkey-patched distutils.extension.Extension (but is that even possible in 2.7, where it's an old-style class?) Monkey patch distutils.command.check_extensions_list too, to check more liberally. ...? But, I dunno about the first one, presumably there is a good reason it's being monkey patched, the second one seems ... well all-but-guaranteed to cause stupid regressions, and the third kinda feels like throwing good money after bad somehow... I'm not an expert so I'll shut up and hope some of those can chime in. Of course it could be fixed in distutils, btw, and imo should be, but the cat is well out of the bag for many years on this one -- setuptools won't be able to rely on that fix for many years. |
Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):
I don't believe pkg_resources is implicated here. pkg_resources is specifically kept partitioned from the setuptools namespace and never imports or invokes behavior from the setuptools package. Did you simply mean "setuptools" in place of "pkg_resources"? |
Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco): A couple of ideas I have:
I can see distutils being updated to provide a more rigorous API and extension mechanism for its classes, such as Extension which setuptools could leverage rather than monkey patching. I'd be in favor of that in general, but thinking about the practical concerns, I'm not sure there's an elegant solution. To avoid monkey-patching, but still allow a distutils Extension to be used might prove difficult. |
Original comment by floppym (Bitbucket: floppym, GitHub: floppym):
pkg_resources.py currently does this:
This causes setuptools to be imported, and therefore causes the Extension class to be replaced. Also see my stack trace in the Gentoo bug report. https://bugs.gentoo.org/show_bug.cgi?id=532708#c17 In the same bug report, I bisected the issue with pyzmq to dcd552da643c, which is where that setuptools import was introduced in pkg_resources.py. |
Original comment by johnyf (Bitbucket: johnyf, GitHub: johnyf): This error arises also in a different case, without importing First reported in A minimal from setuptools import setup, Extension
print(id(Extension))
name = 'hoho'
version = '0.0.1'
ext_modules = [Extension('foo', ['hoho/foo.c'])]
setup(
name=name,
version=version,
packages=[name],
package_dir={name: name},
install_requires=['psutil'],
ext_modules=ext_modules,
dependency_links=['file:https://localhost/some/path/psutil/dist/psutil-3.3.0.tar.gz#egg=psutil-3.3.0']) where
The above output is obtained by patching from setuptools import setup, Extension
print('in psutil setup.py, id(Extension):')
print(id(Extension)) near the top, where # OS X
elif sys.platform.startswith("darwin"):
print(id(Extension))
ext = Extension(
'psutil._psutil_osx',
sources=[
'psutil/_psutil_osx.c',
'psutil/_psutil_common.c',
'psutil/arch/osx/process_info.c'
],
define_macros=[VERSION_MACRO],
extra_link_args=[
'-framework', 'CoreFoundation', '-framework', 'IOKit'
])
print(id(Extension))
print(id(type(ext)))
print(id(type(posix_extension)))
assert isinstance(ext, Extension)
assert ext.__class__ is Extension
assert type(ext) is not Extension
extensions = [ext, posix_extension] later, where the extensions of Note that The last part of the output is from the following patch to if isinstance(ext, Extension):
continue # OK! (assume type-checking done
# by Extension constructor)
print('type(ext): {t}'.format(t=type(ext)))
print(ext)
print('type(Extension): {t}'.format(t=type(Extension)))
print(Extension)
print('id(type(ext)) = {i} and id(Extension) = {j}'.format(
i=id(type(ext)), j=id(Extension)))
print('id(ext.__class__) = {i} and id(Extension) = {j}'.format(
i=id(ext.__class__), j=id(Extension))) Conclusion: The The instance Therefore, the This behavior is triggered only when Cython is installed. Also, it doesn't happen if Finally, note that this behavior was revealed in So, the question is what should be done to fix this ? |
Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):
I may have an idea what's going on here, and unfortunately, it may be a different bug than the originally reported bug (which is specifically about packages that import from distutils before importing setuptools). I've opened #488 to track it separately. |
Edit: watch out for the fact that # BAD
extension = Extension("example", ["example"+ext])
if USE_CYTHON:
from Cython.Build import cythonize
extensions = cythonize(extensions)
setup(ext_modules = [extension]) This will fail because you're actually passing a Correct version: extensions = [Extension("example", ["example"+ext])]
if USE_CYTHON:
from Cython.Build import cythonize
extensions = cythonize(extensions)
setup(ext_modules = extensions) |
Originally reported by: floppym (Bitbucket: floppym, GitHub: floppym)
On Gentoo Linux, we are running into a problem building pyzmq with recent versions of setuptools.
https://bugs.gentoo.org/show_bug.cgi?id=532708
We get this error:
From what I can tell, it has to do with the order in which
distutils.extenstion.Extension
and setuptools are imported. setuptools replaces the Extension module with its own implementation. If this happens after setup.py has imported distutils.extension.Extension, it causes this problem.Here is a reduced test case:
The obvious workaround is to always import setuptools first. However, it is not always obvious where setuptools is being imported.
For pyzmq, it seems like setuptools is being imported indirectly; setup.py does not import it anywhere. By trial and error, it looks like importing nose brings in setuptools.
The text was updated successfully, but these errors were encountered: