Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Do not merge!] Pseudo PR for first release #133

Open
wants to merge 825 commits into
base: TEMPLATE
Choose a base branch
from
Open

[Do not merge!] Pseudo PR for first release #133

wants to merge 825 commits into from

Conversation

mashehu
Copy link

@mashehu mashehu commented Aug 8, 2024

Do not merge! This is a PR of dev compared to first release for whole-pipeline reviewing purposes. Changes should be made to dev and this PR should not be merged into first-commit-for-pseudo-pr!

tillenglert and others added 30 commits June 27, 2023 14:47
Move some operations to subworkflow
Typo and documentation suggestions

Co-authored-by: Sabrina Krakau <[email protected]>
Co-authored-by: Sabrina Krakau <[email protected]>
Copy link

github-actions bot commented Aug 8, 2024

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit a09de95

+| ✅ 195 tests passed       |+
#| ❔   2 tests were ignored |#
!| ❗   3 tests had warnings |!

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 1.0.0
  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • nfcore_yml - nf-core version not set in .nf-core.yml

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 2.14.1
  • Run at 2024-08-16 07:25:06

@tillenglert
Copy link
Collaborator

Thank you @mashehu !! Thats really helpful 🙂

Copy link
Author

@mashehu mashehu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a few small things I saw. not a complete review

.github/workflows/ci.yml Show resolved Hide resolved
with:
report_paths: "*.xml"

test_profile_download:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you do this as separate job?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the pipeline optionally can download assemblies from Entrez using taxids, for this a registered account is needed (implemented as nextflow secrets). The GitHub secrets defined on the repository (used then in the process) are only available to PRs from nf-core/metapep, but not any forks! We decided before the download "path" is not tested at all, the tests run after merging the PR on dev, so if there is something failing it is not unnoticed.

@@ -3,14 +3,38 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## v1.0dev - [date]
## v1.0.0 - [2022-01-20]
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any plan for fun release names?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still have to come up with a cool schema, any suggestions?

CHANGELOG.md Show resolved Hide resolved
CITATIONS.md Show resolved Hide resolved
@tillenglert
Copy link
Collaborator

Thank you @mashehu ! Still very helpful!

Copy link

@nictru nictru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice code quality overall!

Some notes:

  1. For each module, you only have an .nf file. This is generally sufficient and fine for local modules, but you could better bundle the module resources by using a module directory with a main.nf and adding the script as a template in the module directory, instead of having it in bin/. This way you could get rid of all the argument parsing builerplate.
  2. When running the test configuration I saw that most modules don't provide a tag. Providing them would help users understanding what's going on.
  3. Related: You don't use a meta map at all. While this is generally acceptable as long as the modules are local, having them would make the pipeline more easily understandable for other nf-core developers. However this would require a lot of refactoring in the channel handling, so I think the current state is fine for now.
  4. I think your pipeline delivers quite some output that could be presented as a MultiQC report. Maybe you could implement this?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use the tag directive more frequently in your processes. It helps users identify which data already has already gone through a certain process and which has not.
Commenting at this process because it is actually executed multiple times in the test config. As you are not using the meta map, you could still derive a tag from the file name.

modules/nf-core/multiqc/multiqc.diff Show resolved Hide resolved
nextflow_schema.json Show resolved Hide resolved
nextflow_schema.json Show resolved Hide resolved
nextflow_schema.json Show resolved Hide resolved
Comment on lines +17 to +18
path "stats.txt", emit: ch_stats
path "versions.yml", emit: versions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
path "stats.txt", emit: ch_stats
path "versions.yml", emit: versions
path "stats.txt", emit: ch_stats
path "versions.yml", emit: versions

'https://depot.galaxyproject.org/singularity/mulled-v2-0be74e7b0c2e289bc8098b1491baf4f181012b1c:a1635746bc2c13635cbea8c29bd5a2837bdd7cd5-0' :
'biocontainers/mulled-v2-0be74e7b0c2e289bc8098b1491baf4f181012b1c:a1635746bc2c13635cbea8c29bd5a2837bdd7cd5-0' }"

publishDir "${params.outdir}/figures", mode: params.publish_dir_mode
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be better to have this in modules.config, for consistency with other modules

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely true, I have no Idea how that ended up in the process 😆

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check if you could use this module

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I will check 👍

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow is quite long, splitting it up into multiple subworkflows could increase readability

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be deleted?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't get nf-test to run without it, do you know how I could solve this?

@nictru nictru mentioned this pull request Aug 14, 2024
@tillenglert
Copy link
Collaborator

tillenglert commented Aug 16, 2024

Whoops I messed up and my mouse jumped 😆

Very nice code quality overall!

First of all thank you for reviewing! I will look into your comments and see how I can resolve them! 🙂

Some notes:

  1. For each module, you only have an .nf file. This is generally sufficient and fine for local modules, but you could better bundle the module resources by using a module directory with a main.nf and adding the script as a template in the module directory, instead of having it in bin/. This way you could get rid of all the argument parsing builerplate.

This is a good point and could maybe help with readability. As most of those scripts were developed as standalones, they were implemented as such. I will think about taking it into the refactoring for the metamap (see point 3), though maybe not for this release but a follow-up 1.1.!!

  1. When running the test configuration I saw that most modules don't provide a tag. Providing them would help users understanding what's going on.

A good point, probably very helpful, for users not so proficient, I will implement some tags!

  1. Related: You don't use a meta map at all. While this is generally acceptable as long as the modules are local, having them would make the pipeline more easily understandable for other nf-core developers. However this would require a lot of refactoring in the channel handling, so I think the current state is fine for now.

There was some development for the epitopeprediction modules within nf-core/epitopeprediction, and we wanted to align our prediction step, in this effort I would also implement the metamap, as well as refactor all module inputs, possibly as templates as mentioned above.

  1. I think your pipeline delivers quite some output that could be presented as a MultiQC report. Maybe you could implement this?
    We decided actually against it, using multiqc as QC report and not output report. I will discuss again internally if we want to change that and the accordingly attach the figures (more downstream analysis, will probably come soon, so maybe together with the new plots?)

@tillenglert tillenglert reopened this Aug 16, 2024
@nictru
Copy link

nictru commented Aug 16, 2024

Did you want to add a comment on MultiQC? :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants