HiC-Pro WDL is a way to specify data processing workflows with a human-readable and writeable syntax. WDL makes it straightforward to define complex analysis tasks, chain them together in workflows, and parallelize the execution.
- Easy to understand and change specify task
- Make parallelize on cloud easier
- Fastq Partition: Split the fastq data into smaller files (Cloud Only)
- Global Mapping: Use Bowtie2 align the reads to reference genome
- Trimming: Remaining unmapped reads spanning the ligation junction are trimmmed from their 3’ end.
- Local Mapping: Align trimmed reads to reference genome again.
- Mapping Combine: Combine global mapping and local mapping results
- Mapping Stat: Stat the mapping quality.
- Merge Pairs: Merge mapped R1 and R2 bam files.
- Mapped HicFragments: Filter out invalid pairs and output valid reads
- Merge Valid Interaction: Merge all the valid reads from different cpu outcome (sample.allValidPairs).
- Build Matrix: Build contact matrix base on sample.allValidPairs.
- Making Plot: Merge stat files and visualize mapping quality.
- Ice Normaliztion: Use package iced to normalize contact matrix.
- Create json file - Womtool
java -jar womtool-52.jar inputs xxx.wdl > xxx.wdl.json
- Run wdl file - Cromwell
java -jar cromwell-XY.jar run myWorkflow.wdl -i myWorkflow.json
Azure D13_v2 VM has specification 8 cores CPU + 56GB RAM.
Performance |
Non-Parallel Azure D13_v2 x 1 |
Parallel Azure D13_v2 x 3 |
Parallel Azure D13_v2 x 10 |
Parallel Azure D13_v2 x 20 |
Bowtie_Global_Mapping |
8h:01m:51s |
1h:24m:14s |
39m:52s |
37m:11s |
Bowtie_Local_Trimming |
1m:58s |
1m:16s |
1m:25s |
1m:34s |
Bowtie_Local_Mapping |
1h:27m:51s |
6m:50s |
3m:08s |
3m:19s |
Mapping_Combine |
43m:42s |
3m:35s |
1m:55s |
1m:41s |
Mapping_Stat |
10m:24s |
2m:21s |
1m:24s |
1m:16s |
Merge_Pairs |
33m:48s |
4m:29s |
2m:37s |
1m:42s |
Mapped_Hic_Fragments |
1h:51m:29s |
12m:0s |
4m:40s |
4m:37s |
Merge_Valid_Interaction |
3m:28s |
4m:11s |
3m:47s |
3m:57s |
Making_Plot |
1m:10s |
1m:27s |
1m:44s |
1m:21s |
Build_Matrix |
1m:37s |
2m:0s |
2m:31s |
1m:41s |
Ice_Normalization |
2m:16s |
3m:22s |
2m:54s |
3m:04s |
Total |
13h:13m:6s |
2h:4m:22s |
1h:4m:06s |
1h:1m:08s |
- You can't run this wdl file locally, it may cause error with uncorrect directory setting. Please run it though Atgenomix.