Error on import on MacOS with new release

[stinodego]

Ever since the recent release, one of our CI pipeline fails with this error:

ImportError while importing test module '/Users/runner/work/polars/polars/py-polars/tests/unit/io/test_delta.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/unit/io/test_delta.py:10: in <module>
    from deltalake import DeltaTable
../.venv/lib/python3.12/site-packages/deltalake/__init__.py:1: in <module>
    from ._internal import __version__ as __version__
E   ImportError: dlopen(/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/deltalake/_internal.abi3.so, 0x0002): symbol not found in flat namespace '_PyPyBaseObject_Type'

Link to the job (which runs on macos-13):
https://github.com/pola-rs/polars/actions/runs/9407528634/job/25914397270

This can be reproduced locally on a Mac with M-series processor:

from deltalake import DeltaTable

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/.../tmp/venv/lib/python3.11/site-packages/deltalake/__init__.py", line 1, in <module>
    from ._internal import __version__ as __version__
ImportError: dlopen(/Users/.../tmp/venv/lib/python3.11/site-packages/deltalake/_internal.abi3.so, 0x0002): symbol not found in flat namespace '_PyPyBaseObject_Type'

[rtyler]

By recent release, I assume you're having trouble with deltalake 0.18.0?

[szarnyasg]

@rtyler this fails for me on macOS Sonoma with deltalake version 0.18.0 but works with 0.17.4.

[ion-elgreco]

This might be the same issue: PyO3/pyo3#3376

[stinodego]

By recent release, I assume you're having trouble with deltalake 0.18.0?

Yes - apologies for being unclear. 0.17.4 works correctly.

[giacomorebecchi]

I'm temporarily solving this by installing with pip install --no-binary :all: deltalake==0.18.0 --no-cache

[b45ch1]

I can confirm the issue on a Macbook M1

Steps to reproduce

python3.11 -m venv venv
source venv/bin/activate
pip install deltalake
echo "import deltalake" > run.py
python run.py 
Traceback (most recent call last):
  File "/Users/sewalter/tmp/deltalake1.8/run.py", line 1, in <module>
    from deltalake import DeltaTable
  File "/Users/sewalter/tmp/deltalake1.8/venv/lib/python3.11/site-packages/deltalake/__init__.py", line 1, in <module>
    from ._internal import __version__ as __version__
ImportError: dlopen(/Users/sewalter/tmp/deltalake1.8/venv/lib/python3.11/site-packages/deltalake/_internal.abi3.so, 0x0002): symbol not found in flat namespace '_PyPyBaseObject_Type'

[ion-elgreco]

Can someone try v0.18.1 please?

[stinodego]

Seems to work fine! See the step "Install Python dependencies" in this workflow:
https://github.com/pola-rs/polars/actions/runs/9486777587/job/26141912223?pr=16906

Thanks so much for the fix ❤️

[ion-elgreco]

@stinodego great! Thanks for checking