Skip to content

Latest commit

 

History

History
122 lines (101 loc) · 4.73 KB

cljit7u5u000108jo8a7979nc.md

File metadata and controls

122 lines (101 loc) · 4.73 KB
Raw
title seoDescription datePublished cuid slug cover tags
The one with a large project in a GitHub repository
Troubleshooting Git: Resolving 'fatal: pack exceeds maximum allowed size' error when pushing large repositories to GitHub
Fri Jun 30 2023 16:49:36 GMT+0000 (Coordinated Universal Time)
cljit7u5u000108jo8a7979nc
the-one-with-a-large-project-in-a-github-repository
linux, github, version-control, git, gitlab

Today I was migrating an old project repository from GitLab to a private repository on GitHub when I found an unexpected error.

First I changed the remote repository:

$ git remote remove origin
$ git remote add origin [email protected]:backpackerhh/my-repo.git

After that, I pushed changes to the new remote repository:

$ git push -u origin main
Enumerating objects: 438822, done.
Counting objects: 100% (438822/438822), done.
Delta compression using up to 8 threads
Compressing objects: 100% (125694/125694), done.
remote: fatal: pack exceeds maximum allowed size (2.00 GiB)
error: remote unpack failed: index-pack failedB | 3.79 MiB/s
To github.com:backpackerhh/my-repo.git
 ! [remote rejected] main -> main (failed)
error: failed to push some refs to 'github.com:backpackerhh/my-repo.git'

Notice the error message:

remote: fatal: pack exceeds maximum allowed size (2.00 GiB)

After some research, I found the solution on StackOverflow, as usual.

What it basically does is take the range of the commits it needs to push and then goes and pushes them one by one.

Open your Git configuration file:

$ vim ~/.gitconfig

At the end of that file paste following lines:

[alias]
    partial-push = "!sh -c
        'REMOTE=$0;BRANCH=$1;BATCH_SIZE=100;
        if git show-ref --quiet --verify refs/remotes/$REMOTE/$BRANCH; then
            range=$REMOTE/$BRANCH..HEAD;
        else
            range=HEAD;
        fi;
        n=$(git log --first-parent --format=format:x $range | wc -l);
        echo "Have to push $n packages in range of $range";
        for i in $(seq $n -$BATCH_SIZE 1); do
            h=$(git log --first-parent --reverse --format=format:%H --skip $i -n1);
            echo "Pushing $h...";
            git push $REMOTE ${h}:refs/heads/$BRANCH;
        done;
        git push $REMOTE HEAD:refs/heads/$BRANCH'
    "

Run following command after saving those changes:

$ git partial-push origin main
Have to push 16063 packages in range of HEAD
Pushing 101ed695358d87f98d921414958d56aa0c9cc436...
Enumerating objects: 315, done.
Counting objects: 100% (315/315), done.
Delta compression using up to 8 threads
Compressing objects: 100% (237/237), done.
Writing objects: 100% (315/315), 74.97 KiB | 12.49 MiB/s, done.
Total 315 (delta 67), reused 288 (delta 67), pack-reused 0
remote: Resolving deltas: 100% (67/67), done.
To github.com:backpackerhh/my-repo.git
 * [new branch]            101ed695358d87f98d921414958d56aa0c9cc436 -> main
Pushing be2391ce545d0244bdbb7c83a0deae09cb2094b3...
Enumerating objects: 2065, done.
Counting objects: 100% (2063/2063), done.
Delta compression using up to 8 threads
Compressing objects: 100% (1007/1007), done.
Writing objects: 100% (2027/2027), 5.82 MiB | 3.39 MiB/s, done.
Total 2027 (delta 1102), reused 1806 (delta 960), pack-reused 0
remote: Resolving deltas: 100% (1102/1102), completed with 16 local objects.
To github.com:backpackerhh/my-repo.git
   101ed69535..be2391ce54  be2391ce545d0244bdbb7c83a0deae09cb2094b3 -> main
... (lots of similar messages omitted)

In case you have any really large file in your version control, at some point you will see a message like the one that follows:

Pushing 740373e7440c366e01ec3f033322980e00103c5b...
Enumerating objects: 31615, done.
Counting objects: 100% (21682/21682), done.
Delta compression using up to 8 threads
Compressing objects: 100% (5930/5930), done.
Writing objects: 100% (20241/20241), 58.63 MiB | 3.91 MiB/s, done.
Total 20241 (delta 15095), reused 18406 (delta 14097), pack-reused 0
remote: Resolving deltas: 100% (15095/15095), completed with 971 local objects.
remote: warning: See https://gh.io/lfs for more information.
remote: warning: File <large-file> is 58.28 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
To github.com:backpackerhh/my-repo.git
   554aaa16af..740373e744  740373e7440c366e01ec3f033322980e00103c5b -> main

But the process will continue, so nothing to worry about.

It took some time, but on the end it did the job as expected.

Thank you for reading and see you in the next one!