Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: Add linux/arm64 Platform To CI #35

Closed
wants to merge 1 commit into from
Closed

CI: Add linux/arm64 Platform To CI #35

wants to merge 1 commit into from

Conversation

vkhitrin
Copy link

In addition to linux/amd64, CI will also build and ship linux/arm64 images.

The CI will be relatively slow since it has to run twice the amount of jobs on the same runners and emulating arm64.
It is possible to speed up this by archiving digests and uploading them separately.

Resolves #33

@vkhitrin
CI: Add linux/arm64 Platform To CI
a80eebb 
In addition to `linux/amd64`, CI will also build and
ship `linux/arm64` images.

The CI will be relatively slow since it has to run twice the amount
of jobs on the same runners and emulating arm64.
It is possible to speed up this by archiving digests and
uploading them separately.
@vkhitrin
Copy link
Author

vkhitrin commented May 29, 2023 •
edited
Loading

@razonyang,
(in my testing, I did not log into any container registry)
An example of an image was being able to be built on emulated arm64: https://github.com/vkhitrin/docker/actions/runs/5114528946/jobs/9194861270

This will increase the CI time by a lot. Any objections?
There is an option to use digests to speed it up, but it requires a rewrite.

@razonyang
Copy link
Member

razonyang commented May 30, 2023 •
edited
Loading

Thanks, will take a close look on it.

I'm concerned about the matrix way will override/ignore the image, which may cause the image only support one of the platforms. Since the jobs were executed in parallel. For example, will the arm64 override the existing amd64? Since they use the same image name and tag.

sequenceDiagram
    amd64->>Docker: Build
    arm64->>Docker: Build
    Docker-->>amd64: Done
    amd64->>Registry: Push
    Registry-->>amd64: Done
    Docker-->>arm64: Done
    arm64->>Registry: Push
    Registry-->>arm64: Done
Loading

@vkhitrin
Copy link
Author

vkhitrin commented Jun 1, 2023

@razonyang, they shouldn't override the existing image, registries support the same name:tag of images, and they can differentiate architecture images:

image

I will try to test this in my local repo.

@razonyang
Copy link
Member

razonyang commented Jun 1, 2023 •
edited
Loading

Thanks, I've rewritten the workflow, https://github.com/razonyang/hugomods-docker/blob/main/.github/workflows/build.yml. But I'm not sure if I'm doing in the right way.

The testing results of the razonyang/hugo-dev images, please feel free to test if the images work well on ARM64 host, I didn't have device to confirm, I hope I'm emulating it correctly with the following script.

Btw, the ARM64 doesn't include the exts images, since EmbeddedDartSass not available on that Alpine arm64.

There is only thing stop me merging it is that, it took a very long time to build (about an hr), even I enable caching (perhap I missing something).

for prefix in latest base go node exts reg reg-base reg-go reg-node reg-exts; do
  for platform in "linux/amd64" "linux/arm64"; do
    echo "image: razonyang/hugo-dev:$prefix platform: $platform"
    echo "========================================================="
    docker run --rm --platform=$platform razonyang/hugo-dev:$prefix /bin/sh -c "uname -m && hugo version"
    echo "=========================================================\n"
  done
done
$ ./test.sh
image: razonyang/hugo-dev:latest platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:latest' locally
latest: Pulling from razonyang/hugo-dev
Digest: sha256:fe0282fd9dcc140cd71fc8c5ecda49e851731fd42cebe5130df22bc0a5fbd138
Status: Downloaded newer image for razonyang/hugo-dev:latest
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:latest platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:latest' locally
latest: Pulling from razonyang/hugo-dev
Digest: sha256:fe0282fd9dcc140cd71fc8c5ecda49e851731fd42cebe5130df22bc0a5fbd138
Status: Downloaded newer image for razonyang/hugo-dev:latest
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:base platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:base' locally
base: Pulling from razonyang/hugo-dev
Digest: sha256:c0d7d413262c7a30fa6ed61911ad6467647bc0ea248334fb2c92ea40ea41d5bf
Status: Downloaded newer image for razonyang/hugo-dev:base
WARNING: image with reference razonyang/hugo-dev was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:base platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:base' locally
base: Pulling from razonyang/hugo-dev
Digest: sha256:c0d7d413262c7a30fa6ed61911ad6467647bc0ea248334fb2c92ea40ea41d5bf
Status: Downloaded newer image for razonyang/hugo-dev:base
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:go platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:go' locally
go: Pulling from razonyang/hugo-dev
Digest: sha256:e1f3f53a8d771c5a624c2bf958cb64d1c624922c5b76206c13762c19cdfd6f55
Status: Downloaded newer image for razonyang/hugo-dev:go
WARNING: image with reference razonyang/hugo-dev was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:go platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:go' locally
go: Pulling from razonyang/hugo-dev
Digest: sha256:e1f3f53a8d771c5a624c2bf958cb64d1c624922c5b76206c13762c19cdfd6f55
Status: Downloaded newer image for razonyang/hugo-dev:go
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:node platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:node' locally
node: Pulling from razonyang/hugo-dev
Digest: sha256:e9269d93ada50765171996399a34981580d6e53c9b28662bbdb205f592560854
Status: Downloaded newer image for razonyang/hugo-dev:node
WARNING: image with reference razonyang/hugo-dev was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:node platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:node' locally
node: Pulling from razonyang/hugo-dev
Digest: sha256:e9269d93ada50765171996399a34981580d6e53c9b28662bbdb205f592560854
Status: Downloaded newer image for razonyang/hugo-dev:node
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:exts platform: linux/amd64
=========================================================
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a+extended linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:exts platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:exts' locally
exts: Pulling from razonyang/hugo-dev
docker: no matching manifest for linux/arm64 in the manifest list entries.
See 'docker run --help'.
=========================================================

image: razonyang/hugo-dev:reg platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg' locally
reg: Pulling from razonyang/hugo-dev
Digest: sha256:a248ee296c9b5a4f52462a79142938f0025a36c94abce9e0afeb99b8547f03ed
Status: Downloaded newer image for razonyang/hugo-dev:reg
WARNING: image with reference razonyang/hugo-dev was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg' locally
reg: Pulling from razonyang/hugo-dev
Digest: sha256:a248ee296c9b5a4f52462a79142938f0025a36c94abce9e0afeb99b8547f03ed
Status: Downloaded newer image for razonyang/hugo-dev:reg
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-base platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg-base' locally
reg-base: Pulling from razonyang/hugo-dev
Digest: sha256:091815184b7a7e55aa89f6eaa8a9bff1cd552642a9d71c9f6651bf330a59a025
Status: Downloaded newer image for razonyang/hugo-dev:reg-base
WARNING: image with reference razonyang/hugo-dev was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-base platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg-base' locally
reg-base: Pulling from razonyang/hugo-dev
Digest: sha256:091815184b7a7e55aa89f6eaa8a9bff1cd552642a9d71c9f6651bf330a59a025
Status: Downloaded newer image for razonyang/hugo-dev:reg-base
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-go platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg-go' locally
reg-go: Pulling from razonyang/hugo-dev
Digest: sha256:b6ccd3382513c02ad2237dbd59b711447e7d90d295afc23dab118b65ce629812
Status: Downloaded newer image for razonyang/hugo-dev:reg-go
WARNING: image with reference razonyang/hugo-dev was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-go platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg-go' locally
reg-go: Pulling from razonyang/hugo-dev
Digest: sha256:b6ccd3382513c02ad2237dbd59b711447e7d90d295afc23dab118b65ce629812
Status: Downloaded newer image for razonyang/hugo-dev:reg-go
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-node platform: linux/amd64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg-node' locally
reg-node: Pulling from razonyang/hugo-dev
Digest: sha256:4991782def8d0d414adc4f8ea6d80c978f9de1da7fffc906a4963381f7aafedf
Status: Downloaded newer image for razonyang/hugo-dev:reg-node
WARNING: image with reference razonyang/hugo-dev was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-node platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg-node' locally
reg-node: Pulling from razonyang/hugo-dev
Digest: sha256:4991782def8d0d414adc4f8ea6d80c978f9de1da7fffc906a4963381f7aafedf
Status: Downloaded newer image for razonyang/hugo-dev:reg-node
aarch64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/arm64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-exts platform: linux/amd64
=========================================================
x86_64
hugo v0.112.6-2ca0fcc4485c74859382ffab4abf8610cdd3771a linux/amd64 BuildDate=2023-06-01T09:03:29Z VendorInfo=hugomods
=========================================================

image: razonyang/hugo-dev:reg-exts platform: linux/arm64
=========================================================
Unable to find image 'razonyang/hugo-dev:reg-exts' locally
reg-exts: Pulling from razonyang/hugo-dev
docker: no matching manifest for linux/arm64 in the manifest list entries.
See 'docker run --help'.
=========================================================

@razonyang razonyang mentioned this pull request Jun 5, 2023
Merged
@razonyang razonyang closed this in #38 Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Any Plans To Ship ARM Images?
2 participants
@vkhitrin @razonyang