Super confused by the glob

[cecton]

Hello 👋

Thanks for this GitHub Actions!

I'm testing it today but I got super confused by the glob:

It's a private repository so I can't really show more than that. But you see that I use archive/*.zip and the glob failed to found it. But on the other hand I did ls -l archive/*.zip just before and it worked. This seems very strange.

[cecton]

It is worth mentioning that the file I blurred is a .zip file and not a directory

[cecton]

Something must be wrong in my workflow. It felt like the file did not exist. So I merged the 2 jobs (build and release) to avoid using actions/upload-artifact@v2 and actions/download-artifact@v2, and now it works properly.

I will need to use the assets at some point though. So I will update this issue when I have more information that could help understand what happened.

[ga-dmorey]

I'm having a similar issue using upload/download artifact. I have a matrix build that uploads 2 artifacts and I have separate job for the release so I can upload both artifacts to the release. I did an ls of the the directory where the artifacts are downloaded and the files are present but the glob pattern does not find them.

[cecton]

I completely forgot about this issue but it is currently working for me even with the upload/download artifact. Here is the workflow (I redacted a bit because it's a private project):

name: Release Workflow

on:
  push:
    tags:
      - 'cli-v*'

jobs:
  cli-build-linux:
    runs-on: ubuntu-latest
    steps:
      - name: Install libusb
        run: |
          sudo DEBIAN_FRONTEND=noninteractive apt-get update
          sudo DEBIAN_FRONTEND=noninteractive apt-get install libusb-1.0-0-dev
      - name: Checkout source
        uses: actions/checkout@v2
        with:
          lfs: true
      - name: SSH Agent
        uses: webfactory/[email protected]
        with:
          ssh-private-key: XXXXXXXXX
      - name: Install latest stable
        uses: actions-rs/toolchain@v1
        with:
            toolchain: stable
            override: true
            components: rustfmt, clippy
      - name: Build release (Linux)
        uses: actions-rs/cargo@v1
        with:
          command: build
          args: --release -p cli
      - run: strip target/release/XX
      - run: strip target/release/XXXX
      - uses: actions/upload-artifact@v2
        with:
          name: cli-binary-linux
          path: |
            target/release/XX
            target/release/XXXX

  cli-build-windows:
    runs-on: windows-latest
    steps:
      - name: Checkout source
        uses: actions/checkout@v2
        with:
          lfs: true
      - name: SSH Agent
        uses: webfactory/[email protected]
        with:
          ssh-private-key: XXXXXXXXXX
      - name: Update cargo config to use Git CLI
        run: Set-Content -Path $env:USERPROFILE\.cargo\config.toml "[net]`ngit-fetch-with-cli = true"
      - name: Install latest stable
        uses: actions-rs/toolchain@v1
        with:
            toolchain: stable
            override: true
            components: rustfmt, clippy
      - name: Build release (Windows)
        uses: actions-rs/cargo@v1
        with:
          command: build
          args: --release -p cli --target=x86_64-pc-windows-msvc
      - uses: actions/upload-artifact@v2
        with:
          name: cli-binary-windows
          path: |
            target/x86_64-pc-windows-msvc/release/XX.exe
            target/x86_64-pc-windows-msvc/release/XXXX.exe

  release:
    needs: [cli-build-linux, cli-build-windows]
    runs-on: ubuntu-latest
    steps:
      - name: Get the version
        id: get_version
        run: echo ::set-output name=VERSION::${GITHUB_REF#refs/tags/cli-}
      - uses: actions/download-artifact@v2
        with:
          name: cli-binary-linux
          path: cli-binary-linux
      - uses: actions/download-artifact@v2
        with:
          name: cli-binary-windows
          path: cli-binary-windows
      - run: |
          mv cli-binary-linux/XX cli-binary-linux/XX-${{ steps.get_version.outputs.VERSION }}-linux-x86_64
          mv cli-binary-linux/XXXX cli-binary-linux/XXXX-${{ steps.get_version.outputs.VERSION }}-linux-x86_64
          mv cli-binary-windows/XX.exe cli-binary-windows/XX-${{ steps.get_version.outputs.VERSION }}-windows-x86_64.exe
          mv cli-binary-windows/XXXX.exe cli-binary-windows/XXXX-${{ steps.get_version.outputs.VERSION }}-windows-x86_64.exe
      - name: Release
        uses: softprops/action-gh-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          files: |
            cli-binary-linux/*
            cli-binary-windows/*

[ga-dmorey]

Thanks for your example @cecton . Here's what I have for the release job that runs after my build:

  release:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download Artifacts
        uses: actions/download-artifact@v2
        with:
          path: release-files

      - name: List Files
        run: |
          ls release-files

      - name: Release
        uses: softprops/action-gh-release@v1
        with:
          files: |
            release-files/*
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

The ls step confirms that the files are present but the release step doesn't find the files. I was on v0.1.5 and upgraded to v1 as well with same result.

[cecton]

On another repo where I had the issue I just merged the 2 jobs to be one single job without download/upload artifacts and it worked. Maybe you should consider this too

[ga-dmorey]

That won't work for me. I'm building on windows and linux, uploading both artifacts. Then downloading both and releasing them at the same time.

[XiaowenHu96]

I'm having the same issue. I'm trying to download all the artifacts from my previous jobs and release them at once and the release step can't find the file (either with or without wildcard).

Could it be an issue with download/upload-artifact actions?

[XiaowenHu96]

I've solved the issue. After reading the documentation from download-artifact:

If the name input parameter is not provided, all artifacts will be downloaded. To differentiate between downloaded artifacts, a directory denoted by the artifacts name will be created for each individual artifact.

So really, the glob we are looking for is path-to-dowloads/*/*

[timraasveld]

I've solved the issue. After reading the documentation from download-artifact:

If the name input parameter is not provided, all artifacts will be downloaded. To differentiate between downloaded artifacts, a directory denoted by the artifacts name will be created for each individual artifact.

So really, the glob we are looking for is path-to-dowloads/*/*

Thank you, this helped me! Assuming you are working in the default directory, files: */*.zip will do.

The reason ls -l *.zip gives the impression that all zip files are in the main directory, is that download-action actually creates a directory named ./your-artifact.zip. ls gives a list of directories rather than files. So the correct path would be your-artifact.zip/your-artifact.zip.

[SimonHeidrich]

Hey Guys, I think I've depleted myself of all combinations and ideas for fixing this issue for my build. Does anybody have an idea for me? That'd be really great. Here's my current workfile (I deleted windows and macOS build for brevity):

name: Release Binaries

on:
  push:
    branches: ["main"]
    tags:
      - "v[0-9]+.[0-9]+.[0-9]+"
  pull_request:
    branches: ["main"]

jobs:
  buildLinux:
    name: Build on Linux
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Install dependencies
        run: ./install_build_dependencies_linux.sh

      - run: cargo build --release

      - name: Add artifacts to folder
        run: |
          mkdir artifacts
          cp target/release/loregui artifacts
          cp target/release/liblorecore.so artifacts
          cp lorecore_api.h artifacts

      - name: Upload Artifacts
        uses: actions/upload-artifact@v3
        with:
          name: LoreCoreLinuxLibrary
          path: artifacts

  publishBinaries:
    name: Publish Binaries
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs: [buildLinux]
    permissions: write-all

    steps:
      - uses: actions/download-artifact@v3
        with:
          name: LoreCoreLinuxLibrary
          path: artifactsLinux

      - name: Checkout Repo to have access to CHANGELOG.md
        uses: actions/checkout@v3

      - name: Create Release
        uses: actions/create-release@v1
        id: create_release
        with:
          release_name: ${{ steps.version.outputs.version }}
          tag_name: ${{ github.ref }}
          body_path: CHANGELOG.md
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Upload Artifact
        uses: softprops/action-gh-release@v1
        with:
          files: |
            /home/runner/work/LoreCore/LoreCore/artifactsLinux
            /home/runner/work/LoreCore/LoreCore/artifactsLinux/*

[cecton]

Hello!

1. Try to use relative paths.
2. Add ls -l ..... to see if the files you are targeting are there or not

run: ls -l artifactsLinux/*

[cecton]

It's really unfortunate that I forgot about this ticket because I'm pretty sure I fixed my workflow at some point but can't remember what was wrong. I kept the ticket opened because many people seems to be struggling with a similar issue where the files can't be found and it's not clear why

[TheComamba]

(Sorry, switched computers and accounts.)
Yes, I started out with relative paths, this was just my last attempt at fixing the behaviour.
And I just managed to increase my confusion: For faster feedback I created a workflow that just creates and uploads an empty file. The upload found the files (and failed because they're empty, granted):

https://github.com/TheComamba/LoreCore/actions/runs/4970938920/jobs/8895120972
Using essentially the same workflow with my actual artifacts failed, on the other hand:

https://github.com/TheComamba/LoreCore/actions/runs/4970954444/jobs/8895214714

Excellent use of emojis, by the way. :D

[TheComamba]

I found the culprit: Apparently actions/checkout@v3 overrides any changes I made to the file system. Or at least modifies it in some way such that I do not find my files any more.
Thanks so much for the support!

[cecton]

Excellent use of emojis, by the way. :D

Well, I'm not a maintainer on this project. I'm just trying to help others.

Thank you for the feedback. Others might get the same issue and this will help

[James-E-A]

I'm having the same issue. I'm trying to download all the artifacts from my previous jobs and release them at once and the release step can't find the file (either with or without wildcard).

@XiaowenHu96 Two things to remember:

1. The "artifact zip" you download from the web interface is not what's downloaded by download-artifact; it actually downloads each file in that zip into the target folder, not zipped up.
2. Remember that if you run on multiple operating systems, you can't include globs in filenames, since Windows doesn't support those.

For example, here's what my own workflow looks like:

jobs:
  sdist:
    name: Source build for pip
    steps:
      - ...
      - name: upload sdist
        uses: actions/upload-artifact@v4
        with:
          name: sdist
          # We're uploading some files:
          path: ./dist/*.tar.gz

  bdist:
    name: Binary build for ${{ matrix.py-impl }} on ${{ matrix.os }}
    needs: sdist
    ...
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: sdist
          # Those files get downloaded:
          path: ./inputs/
      - uses: tj-actions/glob@v20
        # (Windows doesn't support Globs, annoyingly)
        id: sdist_glob
        with:
          # !! Note that we DON'T need to unzip them: !!
          files: ./inputs/*.tar.gz
      - name: Build wheels
        run: python -m cibuildwheel ${{ steps.sdist_glob.outputs.paths }} --output-dir ./dist
      - ...

[jcucuzza]

I found the culprit: Apparently actions/checkout@v3 overrides any changes I made to the file system. Or at least modifies it in some way such that I do not find my files any more. Thanks so much for the support!

This was the issue!

Using checkout wiped my artifact that I downloaded, the fix is to position the download artifact step after the checkout to avoid your artifact from getting erased.