Error with CONCAT_H5AD

[ashwinikumarkulkarni]

Description of the bug

nf-core/scrnaseq test using conda fails with the following error message. Fetched from '.nextflow.log'. Thanks!

Caused by:
Process NFCORE_SCRNASEQ:SCRNASEQ:MTX_CONVERSION:CONCAT_H5AD (1) terminated with an error exit status (1)

Command used and terminal output

nextflow run nf-core/scrnaseq -r 2.6.0 -profile test,conda --outdir test_scrnaseq

Relevant files

No response

System information

NEXTFLOW version 23.10.1 build 5891
nf-core/scrnaseq v2.6.0-gc5a6445
Linux HPC
BioConda

[grst]

Thanks for reporting! Could you please paste the entire .command.log from the work directory? The work directory of the failed process is usually shown as part of the error message.

[ashwinikumarkulkarni]

Here is the '.command.log' output:

Traceback (most recent call last):
  File "/home2/akulk1/.nextflow/assets/nf-core/scrnaseq/bin/concat_h5ad.py", line 40, in <module>
    df_samplesheet = read_samplesheet(args["input"])
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home2/akulk1/.nextflow/assets/nf-core/scrnaseq/bin/concat_h5ad.py", line 20, in read_samplesheet
    df = df.groupby(["sample"]).agg(lambda column: ",".join(set(column)))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 1482, in aggregate
    result = gba.agg()
             ^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/apply.py", line 193, in agg
    return self.agg_list_like()
           ^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/apply.py", line 326, in agg_list_like
    return self.agg_or_apply_list_like(op_name="agg")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/apply.py", line 1571, in agg_or_apply_list_like
    keys, results = self.compute_list_like(op_name, selected_obj, kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/apply.py", line 385, in compute_list_like
    new_res = getattr(colg, op_name)(func, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 257, in aggregate
    ret = self._aggregate_multiple_funcs(func, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 362, in _aggregate_multiple_funcs
    results[key] = self.aggregate(func, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 294, in aggregate
    return self._python_agg_general(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 327, in _python_agg_general
    result = self._grouper.agg_series(obj, f)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/ops.py", line 864, in agg_series
    result = self._aggregate_series_pure_python(obj, func)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/ops.py", line 885, in _aggregate_series_pure_python
    res = func(group)
          ^^^^^^^^^^^
  File "/project/Neuroinformatics_Core/Konopka_lab/akulk1/RESOURCES/BIOCONDA_NEXTFLOW/work/conda/env-8ceee76ce5640a76fa8925a8ce8f49a3/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 324, in <lambda>
    f = lambda x: func(x, *args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home2/akulk1/.nextflow/assets/nf-core/scrnaseq/bin/concat_h5ad.py", line 20, in <lambda>
    df = df.groupby(["sample"]).agg(lambda column: ",".join(set(column)))
                                                   ^^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, int found```

[nicoaira]

It seems like when the function read_samplesheet reads the samplesheet, it takes the values in the column expected_cells as int, and the join method expects a str.
It can be solved by adding str(x) within the set comprehension:

df = df.groupby(["sample"]).agg(lambda column: ",".join(set(str(x) for x in column)))

[grst]

We are currently refactoring the matrix conversion code in #369 which should hopefully also fix this error.