0.29.0 (#142)

* refactor(utils.gene): redesign `gene_name_conversion` functions for both python and R

* tests(utils.gene): update tests for `gene_name_conversion`

* choir(misc.Str2File): add default for `in.name`

* refactor(gene.GeneNameConversion): use R for implementation

* choir(gene.GeneNameConversion): allow `envs.notfound` to be `ignore` or `skip` when `envs.output` is `append` or `replace`

* tests(gene.GeneNameConversion): use right environment for tests

* docs(MatrixEQTL): fix choice items of `envs.model`

* feat(snp): add PlinkFromVcf

* feat(snp): add Plink2GTMat for converting PLINK files to genotype matrix

* choir(utils.misc.py): flush output for command printing in run_command function

* feat(utils.misc.R): add run_command function

* fix(utils.reference): avoid index file to be created again for the same infile for `tabix_index` function

* feat(snp): add PlinkIBD analysis for identity by descent

* feat(snp): add PlinkHWE for Hardy-Weinberg Equilibrium test

* feat(snp): add PlinkHet for calculating sample heterozygosity

* feat(snp): add PlinkCallRate for calculating call rate of samples and variants

* tests(snp): add tests for plink related processes

* tests(snp): disable report generation for plink related tests

* tests: add python package mygene to conda environment biopipen-r

* feat(gene): add GenePromoters for retrieving gene promoter regions and tests for it

* deps: update pipen-filters to version 0.13

* feat(bcftools): add BcftoolsView process for viewing, subsetting, and filtering VCF files

* feat(bed): add BedtoolsIntersect process for finding the intersection of two BED files

* feat(utils.reference): allow `tabix_index` to index infile directly

* feat(bcftools.BcftoolsSort): allow sort contigs based on a chrom size file

* feat(BcftoolsFilter): allow indexing output file

* feat(bcftools.BcftoolsAnnotate): allow providing annotation file as input file and allow indexing output file

* feat(bcftools): add run_bcftools function for running bcftools with given arguments

* feat(gene.GenePromoters): add envs.sort to sort the output file

* choir(utils.misc.py): print command with a new line for run_command function

* tests: add tests for bcftools processes

* choir: remove __init__.py in scripts

which are unnecessary but breaks the documentation building, because mkapi will treat the templates in scripts as modules.

* BREAKING: merge namespace bcftools to vcf

* chore(snp.Plink2GTMat): remove unused import in Plink2GTMat.py

* feat(utils.misc.R): add support for foreground execution in run_command function

* feat(snp): add PlinkFilter process for filtering PLINK files

* feat(snp): add PlinkFreq process for calculating and filtering by allele frequencies

* feat(regulator): add the namespace and MotifScan to use fimo for motif scanning

* deps: add meme to env_bio.yml for tests

* feat(regulation): add MotifAffinityTest to test the affinity of motifs to the sequences and the affinity change due the mutations.

* tests(regulation): specify envs.genome for MotifAffinityTest

* tests: do not download reference data for hg38

* feat(tests): add lazy loading for reference data download and --local flag for downloading more references locally

* feat(plot): add to for support for plotting Manhattan plots

* tests(plot.Manhattan): add tests

* feat(plot.Manhattan): fix `envs.rescale`  not working

* tests: update bioconductor-ggmanh dependency to version 1.9.6

* feat(vcf.BcftoolsSort): allow `~` to be used as user home directory in `envs.chrsize`

* feat(bed.BedtoolsIntersect): allow `~` to be used as user home directory in `envs.chrsize`

* feat(plot.Manhattan): update chromosome plotting logic to use 'auto' for `envs.chroms`

* deps: bump pipen-board to 0.15.2

* feat(stats.MetaPvalue): add `envs.keep_single` flag to keep the single p-values

* feat(stats): add MetaPvalue1 to combine pvalues from the same file

* feat(tcr.TCRClusterStats): fix `envs.shared_clusters.heatmap_meta` being broken by `envs.shared_clusters.sample_order`

* ci: update GitHub Actions versions and dependencies

* feat(snp.MatrixEQTL): add envs.match_samples flag to subset snp, expr and cov data with common samples

* feat(snp.MatrixEQTL): fix cov data being wrongly transposed

* enh(snp.MatrixEQTL): use rtracklayer to read the position files

* refactor(misc.Shell): save envs.cmd to a file and run it to fix the escaping issues of the command

* feat(snp): add PlinkUpdateName proc to update variant names in PLINK files

* feat(bed.BedtoolsIntersect): add envs.postcmd option to transform the output file

* feat(gene.GenePromoters): remove `envs.withbody` option and use `GenomicRanges::promoters` to generate promoter regions

* fix(bed.BedtoolsIntersect): sort the input files based on chrom size file (envs.chrsize)

* choir(snp): try to use plink2 for Plink related processes

* fix(utils.reference): pass `-f` to bgzip or gunzip to overwrite the output if exists

* chore(snp.PlinkFromVcf): indicate sex is to be handled

* feat(plot): add QQPlot proc for generating QQ-plot or PP-plot

* tests: add bcftools to conda environment dependencies for tests

* feat(plot.QQPlot): add envs.ggs option to allow adding extra components to the plot

* docs(cellranger_pipeline): fix types of some items in docs, which should be 'list', instead of 'type=list'

* chore(cellranger_pipeline): remove unused import

* fix(vcf): fix passing vcffile as a string in fix_vcffile in VcfFix_utils.py

* fix(cnvkit_pipeline): fix sex in process channels

* chore(bam.CNVpytor): implement cnvnator2vcf directly instead of using cnvnator2vcf.pl

* feat(cnv.AneuploidyScore): allow BED and VCF files as in.segfile

* feat(cnv.TMADScore): allow BED and VCF files as in.segfile

* fix(cnv.AneuploidyScoreSummary): fix when Sample column is already in metafile

* fix(cnv.TMADScoreSummary): fix when Sample column is already in metafile

* feat(cnvkit): allow user home directory (~) to be used in `envs.ref` in mulitple processes

* feat(snp/MatrixEQTL): allow extra columns when snp and gene position file is BED

* feat(MotifAffinityTest): allow input file to only contain regulators

* choir(scrna.SeuratMap2Ref): present better error message when envs.use or values of MapQuery.refdata not in reference

* fix(scrna.MarkersFinder): run PrepSCTFindMarkers when needed

* docs(scrna.SeuratMap2Ref): add comments

* choir(scrna.SeuratClustering): use FindClusters to run for multiple resolutions

* choir(scrna.SeuratSubClustering): use FindClusters to run for multiple resolutions

* feat(scrna.SeuratClustering): add clustree plot

* feat(scrna.SeuratSubClustering): add clustree plot

* tests(scrna): update tests for scrna/SeuratClusterStats

* tests(scrna.SeuratClusterStats): revert tests with pbmc_small data

* tests(scrna.SeuratClusterStats): add assertion for clustree plot generation

* 0.29.0

0.28.1 (#141)

* fix(scrna.CellsDistribution): fix `devpars` and `hm_devpars` not working

* 0.28.1

* tests: add r-sccatch to conda environment biopipen-r

0.28.0 (#140)

* tests: add tests for  CellTypeAnnotation using scCATCH

* feat: add docker image building for cellranger pipeline

* choir(cellranger_pipeine): install npm in docker

* chore: Update Dockerfile to use Ubuntu base image and install necessary dependencies

* chore: fix docker image building

* chore(cellranger.CellRangerCount): add `envs.create_bam` to control whether create bams

* docker(cellranger_pipeline): fix docker image building

* fix(cellranger.CellRangerCount): fix for cellranger v7

* docs(cellranger_pipeline): add docs

* docs(cellranger_pipeline): add docs

* 0.28.0

* fix(cellranger): change `in.sample` to `in.id` as it is supposed to be

* chore(cnvkit_pipeline): update entry.sh for running via pipen-board in container

* fix(cellranger_pipeline): fix running the pipeline with pipen-board in container

* docs(cellranger_pipeline): update sample ids in input

* chore(cellranger.CellRangerSummary): set the default value of `report_paging` to 8

* choir(cellranger_pipeline): add test for vdj pipeline for the docker image

* chore(cellranger_pipeline): update CellrangerVdjPipeline.config.toml with correct reference

* docs: update CHANGELOG

* chore(cellranger.CellRangerVdj): update regex to fix web_summary.html script tag

0.27.9 (#139)

* feat(tcr.TCRClusterStats): add `sample_order` to set sample order on heatmap and `cluster_rows` to switch row clustering on/off

* 0.27.9

0.27.8 (#138)

* fix(scrna.SeuratClusterStats): fix selected columns not unique for stats

* feat(scrna.SeuratMap2Ref): allow non-SCTransform'ed reference

* feat(scrna.SeuratMap2Ref): allow splitting query object for mapping

* deps: update pipen-board to version 0.15.1

* 0.27.8

0.27.7 (#137)

* fix(utils.gsea): fix gsea table not being printed for `runFGSEA`

* fix(core.filters): fix slugified pathway plot file name in report

* chore: fix typo in class name ExprImpution to ExprImputation

* fix(scrna_metabolic_landscape): fix mutaters not working

* fix(scrna_metabolic_landscape.MetabolicFeatures/MetabolicFeaturesIntraSubset): skip groups with less than 5 cells in `do_one_group` and save a warning file under the case

* fix(utils.gsea): do not switch 1st and 2nd columns when 2nd column is numeric for `localizeGmtfile`

* choir(tests): remove KEGG_metabolism.gmt for `prep_reference.py`

* tests(scrna_metabolic_landscape): fix tests

* 0.27.7

0.27.6 (#136)

* fix(scrna_metabolic_landscape.MetabolicFeatures): fix return value of groups with less than 5 cells in `do_one_group`

* choir(utils.gsea): avoid printing NULL for `runFGSEA`

* tests: use the return value of pipen.run() to test the success

* 0.27.6

0.27.5 (#135)

* fix(scrna.Subset10X/SeuratTo10X): correct the paths to the scripts

* feat(testing): allow to enable report for testing pipelines

* feat(scrna.SeuratPreparing): add `envs.cell_qc_per_sample` to filter cells before merging instead after

* test: add tests to scrna.SeuratTo10X and scrna.SeuratPreparing

* fix(scrna.SeuratClusterStats): fix color palette for ridge plots

* 0.27.5

0.27.4 (#134)

* feat: add plot.ROC

* choir(delim.SampleInfo): add alpha to the colors of the plots using biopipen color pallete

* feat: add snp.MatrixEQTL

* docs(tcr.TCRClusterStats): update urls of images in docs

* docs(tcr/scrna/scrna_metabolic_landscape): update links of images in docs

* 0.27.4

0.27.3 (#133)

* deps: temporary fix copier breaks with pyyaml-include v2 (copier-org/copier#1568)

* deps: bump pipen-poplog to 0.1.1

* deps: bump pipen-poplog to 0.1.2

* 0.27.3

* choir(scrna.ScFGSEA): Skip cases when no cells found (pwwang/immunopipe#50)

* choir(scrna.MarkersFinder): Skip cases when no cells found (pwwang/immunopipe#50)

* choir(scrna.MetaMarkers): Skip cases when no cells found (pwwang/immunopipe#50)

* docs: update CHANGELOG

* feat(scrna.SeuratPreparing): support DoubletFinder

* docs: update CHANGELOG