BadZipFile error while installing via pip #534

karl0ss · 2024-03-05T08:33:00Z

Is there an existing issue for this?

I have searched the existing issues

Description of the bug

Unable to install git-cliff via pip

Steps To Reproduce

Brand new python venv

❯ python -m venv venv
❯ venv/Scripts/activate
❯ pip install setuptools wheel
Requirement already satisfied: setuptools in c:\users\karl.hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (63.2.0)
Requirement already satisfied: wheel in c:\users\karl.hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (0.37.1)

[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
❯ python.exe -m pip install --upgrade pip

zsh: command not found: python.exe
❯ python -m pip install --upgrade pip

Requirement already satisfied: pip in c:\users\karl.hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (23.3.2)
Collecting pip
  Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.0-py3-none-any.whl (2.1 MB)
   ---------------------------------------- 2.1/2.1 MB 33.9 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.3.2
    Uninstalling pip-23.3.2:
      Successfully uninstalled pip-23.3.2
Successfully installed pip-24.0

Expected behavior

Git-Cliff should install

Screenshots / Logs

❯ pip install git-cliff
Collecting git-cliff
  Downloading git_cliff-2.1.2-py3-none-win_amd64.whl.metadata (6.1 kB)
Downloading git_cliff-2.1.2-py3-none-win_amd64.whl (6.4 MB)
   ---------------------------------------- 6.4/6.4 MB 14.1 MB/s eta 0:00:00
Installing collected packages: git-cliff
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\cli\req_command.py", line 245, in wrapper
    return func(self, options, args)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\commands\install.py", line 452, in run
    installed = install_given_reqs(
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\req\__init__.py", line 72, in install_given_reqs
    requirement.install(
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\req\req_install.py", line 856, in install
    install_wheel(
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\operations\install\wheel.py", line 725, in install_wheel
    _install_wheel(
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\operations\install\wheel.py", line 585, in _install_wheel
    file.save()
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\operations\install\wheel.py", line 398, in save
    self._file.save()
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\pip\_internal\operations\install\wheel.py", line 384, in save
    shutil.copyfileobj(f, dest)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\shutil.py", line 195, in copyfileobj
    buf = fsrc_read(length)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\zipfile.py", line 925, in read
    data = self._read1(n)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\zipfile.py", line 1015, in _read1
    self._update_crc(data)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\zipfile.py", line 943, in _update_crc
    raise BadZipFile("Bad CRC-32 for file %r" % self.name)
zipfile.BadZipFile: Bad CRC-32 for file 'git_cliff-2.1.2.data/scripts/git-cliff-completions.exe'

Software information

Operating system: Windows 10
Rust version: NA
Project version: NA
Python version: 3.10.6

Additional context

No response

The text was updated successfully, but these errors were encountered:

welcome · 2024-03-05T08:33:03Z

Thanks for opening your first issue at git-cliff! Be sure to follow the issue template! ⛰️

karl0ss · 2024-03-05T09:21:34Z

Have tried 2.1.2 and 2.1.1 both zipfile.BadZipFile: Bad CRC-32 for file 'git_cliff-2.1.1.data/scripts/git-cliff-completions.exe'

Have also tried 2.0.4 zlib.error: Error -3 while decompressing data: invalid literal/length code

karl0ss · 2024-03-05T09:33:12Z

Also got a collegue who has tried on his windows machine and he gets the same

ERROR: Exception:
Traceback (most recent call last):
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in _main
    status = self.run(options, args)
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\cli\req_command.py", line 204, in wrapper
    return func(self, options, args)
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\commands\install.py", line 393, in run
    installed = install_given_reqs(
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\req\__init__.py", line 77, in install_given_reqs
    requirement.install(
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\req\req_install.py", line 782, in install
    install_wheel(
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 810, in install_wheel
    _install_wheel(
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 653, in _install_wheel
    file.save()
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 436, in save
    self._file.save()
  File "c:\users\alex.reed\git\test-git-cliff\.venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 420, in save
    shutil.copyfileobj(f, dest)
  File "C:\Users\alex.reed\.pyenv\pyenv-win\versions\3.8.10\lib\shutil.py", line 205, in copyfileobj
    buf = fsrc_read(length)
  File "C:\Users\alex.reed\.pyenv\pyenv-win\versions\3.8.10\lib\zipfile.py", line 940, in read
    data = self._read1(n)
  File "C:\Users\alex.reed\.pyenv\pyenv-win\versions\3.8.10\lib\zipfile.py", line 1030, in _read1
    self._update_crc(data)
  File "C:\Users\alex.reed\.pyenv\pyenv-win\versions\3.8.10\lib\zipfile.py", line 958, in _update_crc
    raise BadZipFile("Bad CRC-32 for file %r" % self.name)
zipfile.BadZipFile: Bad CRC-32 for file 'git_cliff-2.1.2.data/scripts/git-cliff-completions.exe'

karl0ss · 2024-03-05T09:39:39Z

Another collegue has tried on this mac (still python 3.10.6) and it installs fine, so this seems to only be a windows issue..

karl0ss · 2024-03-05T09:42:53Z

Also seems to install fine on windows under WSL (Ubuntu) but not natively

karl0ss · 2024-03-05T10:04:03Z

Also, I have manually downloaded the whl from pypi this morning and tried to isntall that, same issue

(venv) C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff>dir
 Volume in drive C is OS
 Volume Serial Number is 686F-B344

 Directory of C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff

05/03/2024  10:02    <DIR>          .
05/03/2024  10:02    <DIR>          ..
05/03/2024  10:01         6,398,708 git_cliff-2.1.2-py3-none-win_amd64.whl
05/03/2024  08:27    <DIR>          venv
               1 File(s)      6,398,708 bytes
               3 Dir(s)  173,182,754,816 bytes free

(venv) C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff>pip install git_cliff-2.1.2-py3-none-win_amd64.whl
Processing c:\users\karl.hudgell\documents\work\he\git-cliff\git_cliff-2.1.2-py3-none-win_amd64.whl
Installing collected packages: git-cliff
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\cli\req_command.py", line 245, in wrapper
    return func(self, options, args)
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\commands\install.py", line 452, in run
    installed = install_given_reqs(
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\req\__init__.py", line 72, in install_given_reqs
    requirement.install(
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\req\req_install.py", line 856, in install
    install_wheel(
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 725, in install_wheel
    _install_wheel(
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 585, in _install_wheel
    file.save()
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 398, in save
    self._file.save()
  File "C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 384, in save
    shutil.copyfileobj(f, dest)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\shutil.py", line 195, in copyfileobj
    buf = fsrc_read(length)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\zipfile.py", line 925, in read
    data = self._read1(n)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\zipfile.py", line 1015, in _read1
    self._update_crc(data)
  File "C:\Users\Karl.Hudgell\.pyenv\pyenv-win\versions\3.10.6\lib\zipfile.py", line 943, in _update_crc
    raise BadZipFile("Bad CRC-32 for file %r" % self.name)
zipfile.BadZipFile: Bad CRC-32 for file 'git_cliff-2.1.2.data/scripts/git-cliff-completions.exe'

(venv) C:\Users\Karl.Hudgell\Documents\Work\HE\git-cliff>

karl0ss · 2024-03-06T09:09:56Z

I have downloaded the zip fro pypi and tried to extract it using peazip, and I still et an error on that file, are we sure the published package for win/amd64 isn't broken?

karl0ss · 2024-03-06T09:40:12Z

I was able to install this globally using winget install --id=orhun.git-cliff -e not sure why it wont install/work by Pypi

orhun · 2024-03-06T20:32:49Z

Hey, thanks for sharing your findings!

I would like to ping @radusuciu who authored the PyPI workflow for installing via pip. Maybe he knows something.

I'm curious about how did you get the zip file for Windows. I tried pip download but it didn't gave me that exact file. I also would like to test that out locally in case it hints what's wrong. But my number 1 suspicion is that something is going on with maturin - I'm not sure though.

radusuciu · 2024-03-06T20:42:05Z

you rang?

I can take a look. I've been using the pypi package in a few projects without issues but it's all been Linux or WSL. I would also suspect that maturin might be involved..

radusuciu · 2024-03-06T20:55:04Z

I can confirm that this happens on Windows 10 with Python 3.11.8 and will add that git-cliff installs fine for every version up until 2.0.2.

pip install git-cliff==2.0.2 and 2.0.4 manifest a slightly different error:

(.venv) PS C:\Users\radu\scratch> pip install git-cliff==2.0.2
Collecting git-cliff==2.0.2
  Using cached git_cliff-2.0.2-py3-none-win_amd64.whl.metadata (6.1 kB)
Using cached git_cliff-2.0.2-py3-none-win_amd64.whl (5.4 MB)
Installing collected packages: git-cliff
  Attempting uninstall: git-cliff
    Found existing installation: git-cliff 1.4.0
    Uninstalling git-cliff-1.4.0:
      Successfully uninstalled git-cliff-1.4.0
  Rolling back uninstall of git-cliff
  Moving to c:\users\radu\scratch\.venv\lib\site-packages\git_cliff-1.4.0.dist-info\
   from C:\Users\radu\scratch\.venv\Lib\site-packages\~it_cliff-1.4.0.dist-info
  Moving to c:\users\radu\scratch\.venv\scripts\git-cliff-completions.exe
   from C:\Users\radu\AppData\Local\Temp\pip-uninstall-wvn1r9yh\git-cliff-completions.exe
  Moving to c:\users\radu\scratch\.venv\scripts\git-cliff-mangen.exe
   from C:\Users\radu\AppData\Local\Temp\pip-uninstall-wvn1r9yh\git-cliff-mangen.exe
  Moving to c:\users\radu\scratch\.venv\scripts\git-cliff.exe
   from C:\Users\radu\AppData\Local\Temp\pip-uninstall-wvn1r9yh\git-cliff.exe
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\cli\req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\commands\install.py", line 452, in run
    installed = install_given_reqs(
                ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\req\__init__.py", line 72, in install_given_reqs
    requirement.install(
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\req\req_install.py", line 856, in install
    install_wheel(
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\operations\install\wheel.py", line 725, in install_wheel
    _install_wheel(
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\operations\install\wheel.py", line 585, in _install_wheel
    file.save()
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\operations\install\wheel.py", line 398, in save
    self._file.save()
  File "C:\Users\radu\scratch\.venv\Lib\site-packages\pip\_internal\operations\install\wheel.py", line 384, in save
    shutil.copyfileobj(f, dest)
  File "C:\Users\radu\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 197, in copyfileobj
    buf = fsrc_read(length)
          ^^^^^^^^^^^^^^^^^
  File "C:\Users\radu\AppData\Local\Programs\Python\Python311\Lib\zipfile.py", line 957, in read
    data = self._read1(n)
           ^^^^^^^^^^^^^^
  File "C:\Users\radu\AppData\Local\Programs\Python\Python311\Lib\zipfile.py", line 1033, in _read1
    data = self._decompressor.decompress(data, n)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
zlib.error: Error -3 while decompressing data: too many length or distance symbols

I see the same CRC-32 error when attempting to install 2.1.2, 2.1.1, and 2.0.3. Together with 2.0.2 and 2.0.4 these represent all of the available releases of git-cliff on PyPI.

orhun · 2024-03-06T21:05:46Z

I can confirm that this happens on Windows 10 with Python 3.11.8 and will add that git-cliff installs fine for every version up until 2.0.2.

That's weird, there were no significant changes between those versions.

But maybe something changed in maturin? 👀

radusuciu · 2024-03-06T21:06:40Z

Guessing this is the culprit:

git-cliff/pypi/pyproject.toml

Line 2 in 16bea51

requires = ["maturin>=0.14,<0.15"]

I was way too specific in pinning this version of maturin. There's been a 1.0 release since then so I'm going to test locally if this creates an installable wheel.

radusuciu · 2024-03-06T23:08:40Z

Alright, so unfortunately it's not so simple. I installed the same version of maturin (0.14.7) on my Windows 10 workstation and was able to build (maturin build --release --sdist --out wheels --target x86_64-pc-windows-msvc) and install the wheel with zero issues.

orhun · 2024-03-07T09:39:50Z

I see, should we expect it to be fixed after #539?

karl0ss · 2024-03-07T10:07:36Z

Glad this has helped, I would much prefer to install via the project deps than have to have people install on their mac/win using wget/brew...

@orhun I just downloaded the wheel file manually via the pypi webpage and tried to install using pip add "pathtowheel" and the whl can be opened in any zip program, i used peazip, but you gett he same crc issue...

glad it looks like this might go soemwehre and get fixed, thanks for your help guys..

radusuciu · 2024-03-07T18:39:18Z

I see, should we expect it to be fixed after #539?

I'm not 100% since I could not reproduce the failure locally, but I did verify that the wheels generated in my hacked (removed a few irrelevant jobs) fork are installable on my Windows computer.

Relevant job: https://github.com/radusuciu/git-cliff/actions/runs/8182476507/job/22373882202
Wheel artifact: https://github.com/radusuciu/git-cliff/actions/runs/8182476507/artifacts/1304538766

To avoid this problem in the future, it might be good to build wheels during CI and test whether they can be installed at least. Alternatively it could be added as another CD step during the pre-release process.

orhun · 2024-03-07T19:39:06Z

I think it is a good idea to build wheels during CI to make sure everything works before creating a release. Are you possibly interested in making a PR? 🐻

radusuciu · 2024-03-07T19:57:17Z

I think it is a good idea to build wheels during CI to make sure everything works before creating a release. Are you possibly interested in making a PR? 🐻

yes I can give it a shot. Two questions:

Are you okay with some duplication between the CI and CD workflows? I'm not that familiar with reusable workflows (or other reuse mechanisms available in GHA) so that might be a higher bar for me.
Do you want the wheel building and testing implemented in ci.yml or a separate workflow?

orhun · 2024-03-07T20:05:27Z

I think duplication is fine - let's have the basic checks and then we can worry about refactoring it 🐻

Yup, we can place it in ci.yml.

karl0ss · 2024-03-08T08:50:09Z

@radusuciu I have downloaded and tested your artifact this morning and all seems to be well...

I used poetry and tried to install using that and did...get and error...

> poetry add .\wheels-x86_64-pc-windows-msvc\git_cliff-2.1.6-py3-none-win_amd64.whl

Unable to create package with no name

But then tried with pip in my poetry env and it worked ok.

> pip install .\wheels-x86_64-pc-windows-msvc\git_cliff-2.1.6-py3-none-win_amd64.whl
Processing \wheels-x86_64-pc-windows-msvc\git_cliff-2.1.6-py3-none-win_amd64.whl
Installing collected packages: git-cliff
Successfully installed git-cliff-2.1.6

And was able to build changelog, so def progress, I will have a play today and see if I can understand why poetry couldn't install, as I quite often install direct from wheels and it works fine..

thanks for your help/work

radusuciu · 2024-03-08T14:08:14Z

@radusuciu I have downloaded and tested your artifact this morning and all seems to be well...

I used poetry and tried to install using that and did...get and error...
> poetry add .\wheels-x86_64-pc-windows-msvc\git_cliff-2.1.6-py3-none-win_amd64.whl



Unable to create package with no name
But then tried with pip in my poetry env and it worked ok.
> pip install .\wheels-x86_64-pc-windows-msvc\git_cliff-2.1.6-py3-none-win_amd64.whl

Processing \wheels-x86_64-pc-windows-msvc\git_cliff-2.1.6-py3-none-win_amd64.whl

Installing collected packages: git-cliff

Successfully installed git-cliff-2.1.6
And was able to build changelog, so def progress, I will have a play today and see if I can understand why poetry couldn't install, as I quite often install direct from wheels and it works fine..

thanks for your help/work

I think I know the error here and can test this afternoon.

Thanks for the feedback!

radusuciu · 2024-03-08T17:39:36Z

@karl0ss can you please share the pyproject.toml file? I just tried with poetry 1.8.2:

First attempting to add without a pyproject.toml:

> poetry add .\git_cliff-2.1.6-py3-none-win_amd64.whl

Poetry could not find a pyproject.toml file in C:\Users\radu\scratch\test or its parents

Then, after poetry init and all defaults:

> poetry add .\git_cliff-2.1.6-py3-none-win_amd64.whl

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 1 install, 0 updates, 0 removals

  - Installing git-cliff (2.1.6 C:/Users/radu/scratch/test/git_cliff-2.1.6-py3-none-win_amd64.whl)

Writing lock file

The resultant pyproject.toml seems well formed:

[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"
git-cliff = {path = "git_cliff-2.1.6-py3-none-win_amd64.whl"}

karl0ss · 2024-03-08T17:39:49Z

@radusuciu I spoke with the poetry team today, and i'm not sure its your issue, they stated "the issue is that the pkginfo version that is installed does not support metadata version 2.3"

They gave me a command to upadte pkginfo in my pipx env to test, but I wont get around to trying that on monday, I also see they pushed the upversioned dep onto main today, so I guess it will be officially supported in coming poetry version. python-poetry/poetry#9130

radusuciu · 2024-03-08T17:54:26Z

@radusuciu I spoke with the poetry team today, and i'm not sure its your issue, they stated "the issue is that the pkginfo version that is installed does not support metadata version 2.3"

They gave me a command to upadte pkginfo in my pipx env to test, but I wont get around to trying that on monday, I also see they pushed the upversioned dep onto main today, so I guess it will be officially supported in coming poetry version. python-poetry/poetry#9130

I guess I must have a different version of pkginfo since I'm testing on 3.11?

If there's something we can tweak in the pyproject.toml for git-cliff then I think that'd be worthwhile to avoid this issue for users that might not be using the most up to date poetry. Or maybe this only happens if you're downloading and installing the wheel?

karl0ss · 2024-03-08T18:15:10Z

I'm not 100% on that, but if the wheel gets published, I can try install it and see if it works, if not then it looks like updating as they said would work, or using the latest with 3.11 would work as you said..thanks for looking into all of this..

orhun · 2024-03-23T14:36:49Z

Yup, I can probably release a minor version pretty soon.

orhun · 2024-03-24T21:42:04Z

I created RC release on Test PyPi - @karl0ss can you test it if you have the chance? I think I will create v2.2.0 tomorrow.

orhun · 2024-03-25T08:51:43Z

Looks like the Maturin workflow failed: https://github.com/orhun/git-cliff/actions/runs/8412795467/job/23034491981

🎛️ Running in non-interactive mode on CI
🚀 Uploading 10 packages
💥 maturin failed
  Caused by: 💥 Failed to upload "git_cliff-2.2.0rc1.tar.gz" (60.4 KB)
  Caused by: Could not read the metadata from the package at /home/runner/work/git-cliff/git-cliff/pypi/wheels/git_cliff-2.2.0rc1.tar.gz
  Caused by: archive header checksum mismatch

@radusuciu can you take a look?

radusuciu · 2024-03-25T13:29:32Z

Odd.. I just got back from a trip but I'll review the log after work and let you know.

orhun · 2024-03-26T09:55:21Z

I looked into this last night but couldn't figure out what happened. FWIW we can download the artifacts from the latest cd run, I inspected them to only find out everything looks good :/

radusuciu · 2024-03-27T15:16:57Z

I took a look as well and nothing stands out. I have also verified that the wheel is installable on my Windows system so that's good at least.

I'll attempt to debug the publishing issue in a fork.

orhun · 2024-03-27T19:27:31Z

Thank you! 🙏🏼 It's the only thing that's blocking the release rn.

radusuciu · 2024-03-28T18:38:26Z

So my release worked. Here it is on TestPyPi. Please ignore the version number, that's just a result of me hacking at it until I could push it through.

Things I chopped off include: NPM, deb, RPM, cargo, and homebrew publishing.. but perhaps most relevant given the checksum mismatch error is the fact that I removed the signing. See this diff, focusing on changes to cd.yaml -- other changes shouldn't matter (I accidentally based this on another branch).

Needs a bit more thought here.

Btw, I might suggest skipping the pypi release for v2.2.0 if you just want to get the release out. We can catch up again on 2.2.1

orhun · 2024-03-28T21:50:08Z

Hmm, weird. Normally signing shouldn't affect the PyPI release but maybe there is some hidden magic going on somewhere.

I will release another RC for testing it again and if it doesn't work then I think I will just disable PyPI releases until we figure things out.

Thanks for debugging it!

radusuciu · 2024-03-28T23:49:48Z

To confirm my guess, I could add back that step (setting the necessary secrets) and verify that the pypi publish step fails, and then we can go from there.

orhun · 2024-03-29T01:29:38Z

Sounds good, let's do that

orhun · 2024-03-30T13:55:13Z

Created v2.2.0 with no issues now, not sure what was wrong before. 🤷🏼‍♂️

@karl0ss can you test the latest version from PyPI and see if the error still happens?

radusuciu · 2024-03-30T16:29:14Z

Oh no, a heisenbug!

karl0ss · 2024-04-04T13:48:33Z

Sorry guys, I have been sick, but I am back now...

Sadly, I seem to still be getting the same issue with 2.2.0 that is on pypi at the moment, not too sure what is going on, I have even update my python to 3.10.6 since (was running on 3.8)

Updating dependencies
Resolving dependencies... (7.6s)

Package operations: 1 install, 1 update, 0 removals

  - Installing git-cliff (2.2.0): Failed

  BadZipFile

  Bad CRC-32 for file 'git_cliff-2.2.0.data/scripts/git-cliff-completions.exe'

  at c:\users\karl.hudgell\.pyenv\pyenv-win\versions\3.8.10\lib\zipfile.py:958 in _update_crc
       954│             return
       955│         self._running_crc = crc32(newdata, self._running_crc)
       956│         # Check the CRC if we're at the end of the file
       957│         if self._eof and self._running_crc != self._expected_crc:
    →  958│             raise BadZipFile("Bad CRC-32 for file %r" % self.name)
       959│
       960│     def read1(self, n):
       961│         """Read up to n bytes with at most one read() system call."""
       962│

Cannot install git-cliff.

PS C:\Users\Karl.Hudgell\Documents\Work\HE\regression-testing-package\data_generator>

karl0ss · 2024-04-04T14:05:41Z

Have just spun up windows sandbox, installed python 3.10.6 (without using pyenv) and installed poetry using pipx as suggested...but still the same issue

radusuciu · 2024-04-04T21:12:14Z

@karl0ss can you confirm that you get the issue just with pip in the same environment?

karl0ss · 2024-04-05T11:28:21Z

Clean windows 10 sandbox, python 3.10.6 installed officially via the installer, git installed via official installer, no poetry installed...

Same issue...

DeadNews · 2024-04-08T12:28:27Z

Corrupted Archive.
Can be reproduced:

$ VERSION=2.2.0

$ wget https://files.pythonhosted.org/packages/py3/g/git_cliff/git_cliff-${VERSION}-py3-none-win_amd64.whl
‘git_cliff-2.2.0-py3-none-win_amd64.whl’ saved [6614980/6614980]

$ python -m zipfile --extract git_cliff-${VERSION}-py3-none-win_amd64.whl tmp
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/lib/python3.11/zipfile.py", line 2581, in <module>
    main()
  File "/usr/lib/python3.11/zipfile.py", line 2548, in main
    zf.extractall(curdir)
  File "/usr/lib/python3.11/zipfile.py", line 1693, in extractall
    self._extract_member(zipinfo, path, pwd)
  File "/usr/lib/python3.11/zipfile.py", line 1748, in _extract_member
    shutil.copyfileobj(source, target)
  File "/usr/lib/python3.11/shutil.py", line 197, in copyfileobj
    buf = fsrc_read(length)
          ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/zipfile.py", line 957, in read
    data = self._read1(n)
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/zipfile.py", line 1047, in _read1
    self._update_crc(data)
  File "/usr/lib/python3.11/zipfile.py", line 975, in _update_crc
    raise BadZipFile("Bad CRC-32 for file %r" % self.name)
BadZipFile: Bad CRC-32 for file 'git_cliff-2.2.0.data/scripts/git-cliff-completions.exe'

$ unzip git_cliff-${VERSION}-py3-none-win_amd64.whl -d tmp2
Archive:  git_cliff-2.2.0-py3-none-win_amd64.whl
  inflating: tmp2/git_cliff-2.2.0.dist-info/METADATA  
  inflating: tmp2/git_cliff-2.2.0.dist-info/WHEEL  
  inflating: tmp2/git_cliff-2.2.0.data/scripts/git-cliff-completions.exe   bad CRC d7dfc6e8  (should be 5bffd430)
  inflating: tmp2/git_cliff-2.2.0.data/scripts/git-cliff-mangen.exe  
  inflating: tmp2/git_cliff-2.2.0.data/scripts/git-cliff.exe  
  inflating: tmp2/git_cliff-2.2.0.dist-info/RECORD 

$ rg git-cliff-completions tmp2/git_cliff-${VERSION}.dist-info/RECORD
3:git_cliff-2.2.0.data/scripts/git-cliff-completions.exe,sha256=KoEd4HiAfgAKFJSRQXjHVlZB9eLsrLWL3z2MCrYL_L4,1998848

$ openssl dgst -sha256 -binary tmp2/git_cliff-${VERSION}.data/scripts/git-cliff-completions.exe | openssl base64
cR/TpY82IhyfYz47UO/fxsECvcnRUk08JXXlBdV7Ero=

All 2.*.* versions do not work.
The last working version is 1.4.0:

$ VERSION=1.4.0

$ wget https://files.pythonhosted.org/packages/py3/g/git_cliff/git_cliff-${VERSION}-py3-none-win_amd64.whl
‘git_cliff-1.4.0-py3-none-win_amd64.whl’ saved [4159693/4159693]

$ python -m zipfile --extract git_cliff-${VERSION}-py3-none-win_amd64.whl tmp

$ unzip git_cliff-${VERSION}-py3-none-win_amd64.whl -d tmp2
Archive:  git_cliff-1.4.0-py3-none-win_amd64.whl
  inflating: tmp2/git_cliff-1.4.0.dist-info/METADATA  
  inflating: tmp2/git_cliff-1.4.0.dist-info/WHEEL  
  inflating: tmp2/git_cliff-1.4.0.data/scripts/git-cliff-completions.exe  
  inflating: tmp2/git_cliff-1.4.0.data/scripts/git-cliff-mangen.exe  
  inflating: tmp2/git_cliff-1.4.0.data/scripts/git-cliff.exe  
  inflating: tmp2/git_cliff-1.4.0.dist-info/RECORD  

$ rg git-cliff-completions tmp2/git_cliff-${VERSION}.dist-info/RECORD
3:git_cliff-1.4.0.data/scripts/git-cliff-completions.exe,sha256=HzohwMyVLQQZVJng8RzDWo1orTJfl0nHAQie0K3_8-E,620544

$ openssl dgst -sha256 -binary tmp2/git_cliff-${VERSION}.data/scripts/git-cliff-completions.exe | openssl base64
HzohwMyVLQQZVJng8RzDWo1orTJfl0nHAQie0K3/8+E=

DeadNews · 2024-04-08T13:52:48Z

https://github.com/orhun/git-cliff/actions/runs/8491060435

The git_cliff-2.2.0-py3-none-win_amd64.whl in these archives are valid.

I don't know where wheels-x86_64-pc-windows-gnu is used.
The version on pypi is wheels-x86_64-pc-windows-msvc with corrupted git-cliff-completions.exe.

$ find . -type f -exec sha256sum {} 

201db433a945a7502cf4f6e487044c1dae911336e55babd0d02147483851fd6a  ./pypi/git_cliff-2.2.0.dist-info/METADATA
e0d67ecd3747a0f9fadeeff14da771270862326289eeb949f6772e96aea7efe6  ./pypi/git_cliff-2.2.0.dist-info/WHEEL
8f1c58ea8c9a3f5cccd171beac989de62da7e11c2137107a9ab34d2e2ac02c6c  ./pypi/git_cliff-2.2.0.dist-info/RECORD
711fd3a58f36221c9f633e3b50efdfc6c102bdc9d1524d3c2575e505d57b12ba  ./pypi/git_cliff-2.2.0.data/scripts/git-cliff-completions.exe
dca0f554e1010708963d3a4e0c71f196639d3376c68cda62cf8286fe3195ab45  ./pypi/git_cliff-2.2.0.data/scripts/git-cliff-mangen.exe
5fec7ed499486e433c889efd55e8d3b748301cc4a966e1501acdb5e1d05ea8ac  ./pypi/git_cliff-2.2.0.data/scripts/git-cliff.exe

201db433a945a7502cf4f6e487044c1dae911336e55babd0d02147483851fd6a  ./wheels-x86_64-pc-windows-msvc/git_cliff-2.2.0.dist-info/METADATA
e0d67ecd3747a0f9fadeeff14da771270862326289eeb949f6772e96aea7efe6  ./wheels-x86_64-pc-windows-msvc/git_cliff-2.2.0.dist-info/WHEEL
8f1c58ea8c9a3f5cccd171beac989de62da7e11c2137107a9ab34d2e2ac02c6c  ./wheels-x86_64-pc-windows-msvc/git_cliff-2.2.0.dist-info/RECORD
2a811de078807e000a1494914178c7565641f5e2ecacb58bdf3d8c0ab60bfcbe  ./wheels-x86_64-pc-windows-msvc/git_cliff-2.2.0.data/scripts/git-cliff-completions.exe
dca0f554e1010708963d3a4e0c71f196639d3376c68cda62cf8286fe3195ab45  ./wheels-x86_64-pc-windows-msvc/git_cliff-2.2.0.data/scripts/git-cliff-mangen.exe
5fec7ed499486e433c889efd55e8d3b748301cc4a966e1501acdb5e1d05ea8ac  ./wheels-x86_64-pc-windows-msvc/git_cliff-2.2.0.data/scripts/git-cliff.exe

201db433a945a7502cf4f6e487044c1dae911336e55babd0d02147483851fd6a  ./wheels-x86_64-pc-windows-gnu/git_cliff-2.2.0.dist-info/METADATA
e0d67ecd3747a0f9fadeeff14da771270862326289eeb949f6772e96aea7efe6  ./wheels-x86_64-pc-windows-gnu/git_cliff-2.2.0.dist-info/WHEEL
b363d8db7f68b5f65f0a63fdbe781fa6cb4a3a5cc5cbfb026df3ba6bf3f857a0  ./wheels-x86_64-pc-windows-gnu/git_cliff-2.2.0.dist-info/RECORD
e95816287d3eed917e14f1ad480edbf87a8612b5a0f5459e7e03bcd231d60d6f  ./wheels-x86_64-pc-windows-gnu/git_cliff-2.2.0.data/scripts/git-cliff-completions.exe
712c5c05b72fa3b5fc996cc31b1bcf8714292cb8b50afb7a24c9da16b136485b  ./wheels-x86_64-pc-windows-gnu/git_cliff-2.2.0.data/scripts/git-cliff-mangen.exe
5bdf95d64be282af3d1935fadd6687e62f6a52ed73545eb2c290484d40338895  ./wheels-x86_64-pc-windows-gnu/git_cliff-2.2.0.data/scripts/git-cliff.exe

--- a/pypi
+++ b/wheels-x86_64-pc-windows-msvc
@@ -1,6 +1,6 @@
 201db433a945a7502cf4f6e487044c1dae911336e55babd0d02147483851fd6a  ./git_cliff-2.2.0.dist-info/METADATA
 e0d67ecd3747a0f9fadeeff14da771270862326289eeb949f6772e96aea7efe6  ./git_cliff-2.2.0.dist-info/WHEEL
 8f1c58ea8c9a3f5cccd171beac989de62da7e11c2137107a9ab34d2e2ac02c6c  ./git_cliff-2.2.0.dist-info/RECORD
-711fd3a58f36221c9f633e3b50efdfc6c102bdc9d1524d3c2575e505d57b12ba  ./git_cliff-2.2.0.data/scripts/git-cliff-completions.exe
+2a811de078807e000a1494914178c7565641f5e2ecacb58bdf3d8c0ab60bfcbe  ./git_cliff-2.2.0.data/scripts/git-cliff-completions.exe
 dca0f554e1010708963d3a4e0c71f196639d3376c68cda62cf8286fe3195ab45  ./git_cliff-2.2.0.data/scripts/git-cliff-mangen.exe
 5fec7ed499486e433c889efd55e8d3b748301cc4a966e1501acdb5e1d05ea8ac  ./git_cliff-2.2.0.data/scripts/git-cliff.exe

Apparently the error is somewhere at the Publish PyPI package stage.

git-cliff/.github/workflows/cd.yml

Lines 321 to 338 in 9864780

    
           publish-pypi: 
        
             name: Publish PyPI package 
        
             runs-on: ubuntu-22.04 
        
             needs: publish-binaries 
        
             steps: 
        
               - uses: actions/download-artifact@v4 
        
                 with: 
        
                   path: pypi/wheels 
        
                   pattern: wheels-* 
        
                   merge-multiple: true 
        
               - name: Publish to PyPI 
        
                 uses: PyO3/maturin-action@v1 
        
                 env: 
        
                   MATURIN_PYPI_TOKEN: ${{ vars.USE_TESTPYPI == 'true' && secrets.TESTPYPI_API_TOKEN || secrets.PYPI_API_TOKEN }} 
        
                   MATURIN_REPOSITORY: ${{ vars.USE_TESTPYPI == 'true' && 'testpypi' || 'pypi' }} 
        
                 with: 
        
                   command: upload 
        
                   args: --skip-existing pypi/wheels/*

I can only note that git_cliff-2.2.0-py3-none-win_amd64.whl is extracted from both wheels-x86_64-pc-windows-gnu and wheels-x86_64-pc-windows-msvc. Apparently, the latter (msvc) overwrites the former (gnu). Maybe the archive gets corrupted somehow when it is overwritten?

Changes to Publish PyPI package stage between working and non-working versions:

https://github.com/orhun/git-cliff/compare/v1.4.0..v2.0.0#diff-ea3ea8c9932adc7ba8161ceda844fedd43b006848ef1140c050cbd7ea0788a18R318-R325

radusuciu · 2024-04-08T16:50:58Z

Great analysis @DeadNews. Is this the only such collision that you saw?

Here are two proposed solutions:

Set PYPI_PUBLISH for the x86_64-pc-windows-gnu platform to false
Ensure that the msvc and gnu wheels get a different name.

@orhun, does having mingw and msvc compiled versions on PyPI serve an important purpose?

orhun · 2024-04-08T19:46:13Z

Nope, not really. Those are there for the sake of "supporting more architectures".

karl0ss · 2024-04-30T08:01:41Z

.....so....is this currently dead in the water re pypi distribution? as if this isn't going to get fixed "soon" I will need to rethink how I am planning on distrubuting this to my wider audience..

Thanks

orhun · 2024-04-30T10:34:06Z

If I understood correctly, we can move forward with @radusuciu's suggestions above, right?

orhun · 2024-04-30T10:39:55Z

I pushed f675b0b and planning to create a release soon, let's see

orhun · 2024-05-11T20:16:34Z

Created v2.2.2 - any chance you can test this again? @karl0ss

DeadNews · 2024-05-11T23:04:46Z

v2.2.2

It works.

karl0ss · 2024-05-13T07:51:43Z

Will have a check today and get back to you, sorry for the delay, and thanks for all the help on this one...

karl0ss · 2024-05-13T15:14:59Z

🥳 Can confirm that 2.2.2 seems to be working for me 😃

orhun · 2024-05-13T16:24:38Z

Awesome! Thanks for your patience and help with this @karl0ss @DeadNews @radusuciu 🚀

karl0ss added the bug Something isn't working label Mar 5, 2024

karl0ss assigned orhun Mar 5, 2024

orhun changed the title ~~BadZipFile?~~ BadZipFile error while installing via pip Mar 6, 2024

radusuciu mentioned this issue Mar 6, 2024

chore(pypi): update maturin version #539

Merged

12 tasks

orhun closed this as completed May 13, 2024

BadZipFile error while installing via pip #534

BadZipFile error while installing via pip #534

Comments

karl0ss commented Mar 5, 2024

Is there an existing issue for this?

Description of the bug

Steps To Reproduce

Expected behavior

Screenshots / Logs

Software information

Additional context

welcome bot commented Mar 5, 2024

karl0ss commented Mar 5, 2024

karl0ss commented Mar 5, 2024

karl0ss commented Mar 5, 2024

karl0ss commented Mar 5, 2024

karl0ss commented Mar 5, 2024

karl0ss commented Mar 6, 2024

karl0ss commented Mar 6, 2024

orhun commented Mar 6, 2024

radusuciu commented Mar 6, 2024

radusuciu commented Mar 6, 2024 • edited Loading

orhun commented Mar 6, 2024

radusuciu commented Mar 6, 2024

radusuciu commented Mar 6, 2024

orhun commented Mar 7, 2024

karl0ss commented Mar 7, 2024

radusuciu commented Mar 7, 2024

orhun commented Mar 7, 2024

radusuciu commented Mar 7, 2024

orhun commented Mar 7, 2024

karl0ss commented Mar 8, 2024

radusuciu commented Mar 8, 2024

radusuciu commented Mar 8, 2024

karl0ss commented Mar 8, 2024 • edited Loading

radusuciu commented Mar 8, 2024 • edited Loading

karl0ss commented Mar 8, 2024

orhun commented Mar 23, 2024

orhun commented Mar 24, 2024

orhun commented Mar 25, 2024

radusuciu commented Mar 25, 2024

orhun commented Mar 26, 2024

radusuciu commented Mar 27, 2024

orhun commented Mar 27, 2024

radusuciu commented Mar 28, 2024

orhun commented Mar 28, 2024

radusuciu commented Mar 28, 2024

orhun commented Mar 29, 2024

orhun commented Mar 30, 2024

radusuciu commented Mar 30, 2024

karl0ss commented Apr 4, 2024

karl0ss commented Apr 4, 2024

radusuciu commented Apr 4, 2024

karl0ss commented Apr 5, 2024

DeadNews commented Apr 8, 2024 • edited Loading

DeadNews commented Apr 8, 2024 • edited Loading

radusuciu commented Apr 8, 2024

orhun commented Apr 8, 2024

karl0ss commented Apr 30, 2024

orhun commented Apr 30, 2024

orhun commented Apr 30, 2024

orhun commented May 11, 2024

DeadNews commented May 11, 2024

karl0ss commented May 13, 2024

karl0ss commented May 13, 2024

orhun commented May 13, 2024

radusuciu commented Mar 6, 2024 •

edited

Loading

karl0ss commented Mar 8, 2024 •

edited

Loading

radusuciu commented Mar 8, 2024 •

edited

Loading

DeadNews commented Apr 8, 2024 •

edited

Loading

DeadNews commented Apr 8, 2024 •

edited

Loading