MultiQC report: Issues with FastQC #1308
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
MatthiasZepper
force-pushed
the
MultiQC_FastQC_bug
branch
from
5a8912a
to
e75b875
Compare
…in the General Stats table of MultiQC.
|
Some progress:
|
…d 'umi_log' and not 'log'. The meta.yml is wrong.
|
Thanks @MatthiasZepper !!
I read through your write-up but was a little unclear as to what is still missing here? |
|
To copy in @MatthiasZepper's note on this from Slack: I am somewhat stuck with #1308, both because of a lack of time recently and also a lack of ideas. I believed that I fixed 3 of the 4 issues with the 4th, the inconsistent naming of the TrimGalore! output, being somewhat neglectable. However, it turns out that I did not fix the main issue yet. The reports generated by MultiQC when run inside the pipeline and manually on the outdir of the pipeline differ. The manual runs look exactly how I want them, so I thought it should be good, but the pipeline version does not work alike. In the pipeline version, the path_filters in the MultiQC config (workflows/rnaseq/assets/multiqc/multiqc_config.yml) are not applied: I think that is because the file paths in the ch_multiqc_files are still those to the work dir and to not correspond yet to the final folder structure specified by the publishDir directives when I mix the output into the channel… … but since I can’t do a proper introspection into the channel (a .view() or .collectFile() completely crashes the pipeline), I don’t know for sure. |
|
OK, I know the fix @MatthiasZepper, I sorted this in riboseq. The issue is that the file structure is flat by the time it gets to MultiQC. We need to do like: ... and then: So we're using the |
Thank you so much! That would be fantastic! You should be able to push to the branch since you are a maintainer, but just in case, I have also invited to as a collaborator to my fork! |
|
@MatthiasZepper OK, committed! Had a quick check and I think this works, though I note that the trimgalore subworkflow doesn't do a post-trim FASTQ, which we might want to address at some point.... Anyway, I'll let you take it home from here :-) |
|
Thank you so much! I will try my best to finish this quickly now!
Oh, it does. It is just confusing, because TrimGalore! in itself is a wrapper script around cutadapt and FastQC. So FastQC is not run as a Nextflow process but by the TrimGalore Perl script. |
|
Ahh right, thought I was forgetting something ;-). So there is probably a missing bit to get those outputs prefixed correctly, but you know what to do. |
|
@MatthiasZepper in case it's impacting on your work, we've noticed that the lastest MultiQC has generated some issues in the workflow. We're looking into it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This draft PR comprises my current progress towards fixing issue #1303.
It does modify the
publishDirdirectives in the FastQC module config such that the reports are consistently published in${params.outdir}/fastqc/rawand${params.outdir}/fastqc/trimregardless of the chosen trimmer (TrimGalore!, Fastp), and adapts the custom MultiQC config of the pipeline accordingly.This is, however, not sufficient to fix the issue, because recent versions of MultiQC have a bug that prevents running the same module twice. There are still separate entries and columns in the General Statistics table, but the modules are not shown in the report and navigation bar:
For both screenshots, I ran MultiQC on the output directory of a
testprofile run of this pipeline using the custom profile inworkflows/rnaseq/assets/multiqc/multiqc_config.yml.It should be stressed that the FastQC module itself works in modern versions, because if the custom config is omitted, it is also shown. But forcing the module to run twice via a custom config seemingly breaks it. Only in the
General Statisticstable, it still works like a charm. Thus, the reports are parsed, but the module output is not displayed in the report.Further issues
In the course of troubleshooting this issue, I discovered more issues that need to be tackled. Help would be greatly appreciated with those:
Inconsistent naming of FastQC output:
For FastP, the file names are retained before and after trimming:
For TrimGalore!, the RAP1_UNINDUCED samples are renamed with a trimmed suffix and the others receive
_val1_and_val2_suffixes.Unfortunately, I have no idea why. I have quadruplechecked the
publishDirdirectives and can't explain. Help and inspiration needed!Duplicate column is actually shown in the General Statistics table (FIXED!)
According to the config, the duplicate column from FastQC should be hidden in the General Statistics table. However, it is shown. Might be another MultiQC bug or that I just stared myself blind.
umi-tools dedup stats not shown
According to our current
master/devbranch config, the umi_tools module is not run. Seeing this, I believed that would be an easy fix for #1277 and added the module in the config. However, no reports are shown. Either the module is broken or the deduplication stats are not channelled to MultiQC. In either way, also no quick solution in sight here.PR checklist
nf-core lint).nextflow run . -profile test,docker --outdir <OUTDIR>).nextflow run . -profile debug,test,docker --outdir <OUTDIR>).docs/usage.mdis updated.docs/output.mdis updated.CHANGELOG.mdis updated.README.mdis updated (including new tool citations and authors/contributors).