Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix malt tests that are using a 739.0 MiB zip as input #5983

Open
maxulysse opened this issue Jul 16, 2024 · 7 comments
Open

Fix malt tests that are using a 739.0 MiB zip as input #5983

maxulysse opened this issue Jul 16, 2024 · 7 comments

Comments

@maxulysse
Copy link
Member

          so the input for unzip on line 19 is a `739.0 MiB` zip...

I'm guessing that's why the tests are failing...

Originally posted by @maxulysse in #5897 (comment)

@jfy133
Copy link
Member

jfy133 commented Jul 16, 2024

Currently this is required for the module to test.

The actual error reported in that CI was:

    > ERROR ~ Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: XD54JYBDPDWDSX0E; S3 Extended Request ID: JmDS4i8VpQQTDw/5SrGhY7knZ3AJxsxKCfWID2bGjrMZrmvZn/3cZ/FjOIKuvwnBCJPOGxTfEao=; Proxy: null)

I'm wondering now if I copied the required taxdmp files to our bucket somewhere...

@jfy133
Copy link
Member

jfy133 commented Jul 16, 2024

Ok indeed I did:


   input[0] = [[], file("s3:https://ngi-igenomes/test-data/createtaxdb/taxonomy/megan-nucl-Feb2022.db.zip", checkIfExists: true)]

@nf-core/infrastructure can you think why reason why GHA can't access the igenomes bucket?

@bebosudo
Copy link

bebosudo commented Jul 16, 2024

That file seems to be openly available, both with aws cli

[alberto@fedora]$ podman run --rm -it docker.io/amazon/aws-cli s3 --no-sign-request cp s3:https://ngi-igenomes/test-data/createtaxdb/taxonomy/megan-nucl-Feb2022.db.zip /tmp

and as http https://ngi-igenomes.s3.amazonaws.com/test-data/createtaxdb/taxonomy/megan-nucl-Feb2022.db.zip

Note the --no-sign-request, otherwise you get a unable to locate credentials error, see this thread: aws/aws-cli#904

I don't know how you're retrieving that file though, if with something different than the aws cli, you may want to check the current role/user you're assuming, as that may have some restrictions that prevent it from reaching that bucket and object.

@jfy133
Copy link
Member

jfy133 commented Jul 16, 2024

Hrm, must be something nextflow side then...

@ewels
Copy link
Member

ewels commented Jul 17, 2024

This sounds like the same "just rm -r ~/.aws" answer that we give on slack on a regular basis.

Are the CI tests configuring AWS access for anything else?

@maxulysse
Copy link
Member Author

Not that I'm aware

@jfy133
Copy link
Member

jfy133 commented Jul 17, 2024

This sounds like the same "just rm -r ~/.aws" answer that we give on slack on a regular basis.

Are the CI tests configuring AWS access for anything else?

Ooff, ok. So we should add that as an upstream step of the actual test in the gha?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants