WARNING: While bind mounting '/mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40:/mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40': destination is already in the mount point list [COMMAND] /opt/bin/run_clairs --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --threads 40 --platform ont_r10_dorado_sup_5khz_ssrs --output_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40 --region chr9:55624206-150617247 --sample_name CHAHA_diag --include_all_ctgs --print_germline_calls --enable_indel_calling --enable_clair3_germline_output --use_longphase_for_intermediate_haplotagging True [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/split_beds [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/candidates [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/pileup_tensor_can [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/fa_tensor_can [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/vcf_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/tmp_vcf_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/vcf [INFO] --include_all_ctgs enabled [INFO] Call variants in contigs: chr9 [INFO] Number of chunks for each contig: 31 [INFO] CALLER VERSION: 0.4.4 [INFO] NORMAL BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam [INFO] TUMOR BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam [INFO] REFERENCE FILE PATH: /home/t_steimle/ref/hs1/chm13v2.0.fa [INFO] PLATFORM: ont_r10_dorado_sup_5khz_ssrs [INFO] THREADS: 40 [INFO] OUTPUT FOLDER: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40 [INFO] OUTPUT VCF PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/output.vcf.gz [INFO] PILEUP MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/pileup.pkl [INFO] FULL-ALIGNMENT MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/full_alignment.pkl [INFO] BED FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/region.bed [INFO] GENOTYPING MODE VCF FILE PATH: None [INFO] HYBRID MODE VCF FILE PATH: None [INFO] REGION FOR CALLING: chr9:55624206-150617247 [INFO] CONTIGS FOR CALLING: None [INFO] CONDA BINARY PREFIX: /opt/conda/envs/clairs [INFO] SAMTOOLS BINARY PATH: samtools [INFO] PYTHON BINARY PATH: python3 [INFO] PYPY BINARY PATH: pypy3 [INFO] PARALLEL BINARY PATH: parallel [INFO] LONGPHASE BINARY PATH: /opt/conda/envs/clairs/bin/longphase [INFO] CHUNK SIZE: 5000000 [INFO] SNV MINIMUM AF: 0.05 [INFO] SNV MINIMUM QUAL: 8 [INFO] INDEL MINIMUM AF: 0.1 [INFO] INDEL PILEUP MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/indel/pileup.pkl [INFO] INDEL FULL-ALIGNMENT MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/indel/full_alignment.pkl [INFO] INDEL MINIMUM QUAL: 8 [INFO] NORMAL VCF FILE PATH: None [INFO] DISABLE PHASING: False [INFO] ENABLE DRY RUN: False [INFO] ENABLE INDEL CALLING: True [INFO] ENABLE PRINTING REFERENCE CALLS: False [INFO] ENABLE PRINTING GERMLINE CALLS: True [INFO] ENABLE INCLUDING ALL CTGS FOR CALLING: True [INFO] ENABLE REMOVING INTERMEDIATE FILES: False [INFO] Call Germline Variants in Normal BAM using Clair3 [INFO] RUN THE FOLLOWING COMMAND: ( /opt/conda/envs/clairs/bin/run_clair3.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --threads 40 --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output --ctg_name=chr9 --samtools=samtools --pypy=pypy3 --python=python3 --min_coverage=4 --snp_min_af=0.08 --indel_min_af=0.15 --longphase_for_phasing ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/1_CLAIR3_NORMAL.log [INFO] CLAIR3 VERSION: v1.0.8 [INFO] BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam [INFO] REFERENCE FILE PATH: /home/t_steimle/ref/hs1/chm13v2.0.fa [INFO] MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 [INFO] OUTPUT FOLDER: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output [INFO] PLATFORM: ont [INFO] THREADS: 40 [INFO] BED FILE PATH: EMPTY [INFO] VCF FILE PATH: EMPTY [INFO] CONTIGS: chr9 [INFO] CONDA PREFIX: [INFO] SAMTOOLS PATH: samtools [INFO] PYTHON PATH: python3 [INFO] PYPY PATH: pypy3 [INFO] PARALLEL PATH: parallel [INFO] WHATSHAP PATH: whatshap [INFO] LONGPHASE PATH: /opt/conda/envs/clairs/bin/longphase [INFO] CHUNK SIZE: 5000000 [INFO] FULL ALIGN PROPORTION: 0.7 [INFO] FULL ALIGN REFERENCE PROPORTION: 0.1 [INFO] PHASING PROPORTION: 0.7 [INFO] MINIMUM MQ: 5 [INFO] MINIMUM COVERAGE: 4 [INFO] SNP AF THRESHOLD: 0.08 [INFO] INDEL AF THRESHOLD: 0.15 [INFO] BASE ERROR IN GVCF: 0.001 [INFO] GQ BIN SIZE IN GVCF: 5 [INFO] ENABLE FILEUP ONLY CALLING: False [INFO] ENABLE FAST MODE CALLING: False [INFO] ENABLE CALLING SNP CANDIDATES ONLY: False [INFO] ENABLE PRINTING REFERENCE CALLS: False [INFO] ENABLE OUTPUT GVCF: False [INFO] ENABLE HAPLOID PRECISE MODE: False [INFO] ENABLE HAPLOID SENSITIVE MODE: False [INFO] ENABLE INCLUDE ALL CTGS CALLING: False [INFO] ENABLE NO PHASING FOR FULL ALIGNMENT: False [INFO] ENABLE REMOVING INTERMEDIATE FILES: False [INFO] ENABLE LONGPHASE FOR INTERMEDIATE VCF PHASING: True [INFO] ENABLE PHASING FINAL VCF OUTPUT USING WHATSHAP: False [INFO] ENABLE PHASING FINAL VCF OUTPUT USING LONGPHASE: False [INFO] ENABLE HAPLOTAGGING FINAL BAM: False [INFO] ENABLE LONG INDEL CALLING: False [INFO] ENABLE C_IMPLEMENT: True + /opt/conda/envs/clairs/bin/scripts/clair3_c_impl.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --threads 40 --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output --bed_fn=EMPTY --vcf_fn=EMPTY --ctg_name=chr9 --sample_name=SAMPLE --chunk_num=0 --chunk_size=5000000 --samtools=samtools --python=python3 --pypy=pypy3 --parallel=parallel --whatshap=whatshap --qual=2 --var_pct_full=0.7 --ref_pct_full=0.1 --var_pct_phasing=0.7 --snp_min_af=0.08 --indel_min_af=0.15 --min_mq=5 --min_coverage=4 --min_contig_size=0 --pileup_only=False --gvcf=False --base_err=0.001 --gq_bin_size=5 --fast_mode=False --call_snp_only=False --print_ref_calls=False --haploid_precise=False --haploid_sensitive=False --include_all_ctgs=False --no_phasing_for_fa=False --pileup_model_prefix=pileup --fa_model_prefix=full_alignment --remove_intermediate_dir=False --enable_phasing=False --enable_long_indel=False --keep_iupac_bases=False --use_gpu=False --longphase_for_phasing=True --longphase=/opt/conda/envs/clairs/bin/longphase --use_whatshap_for_intermediate_phasing=True --use_longphase_for_intermediate_phasing=True --use_whatshap_for_final_output_phasing=False --use_longphase_for_final_output_phasing=False --use_whatshap_for_final_output_haplotagging=False [INFO] Check environment variables [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/log [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/pileup_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/merge_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/phase_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/gvcf_tmp_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/full_alignment_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_vcf [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_bam [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/full_alignment_output/candidate_bed [INFO] Call variant in contigs: chr9 [INFO] Chunk number for each contig: 31 [INFO] 1/7 Call variants using pileup model Calling variants ... Total processed positions in chr9 (chunk 12/31) : 1257 Total time elapsed: 17.96 s Calling variants ... Total processed positions in chr9 (chunk 13/31) : 2641 Total time elapsed: 20.38 s Calling variants ... Total processed positions in chr9 (chunk 15/31) : 17476 Total time elapsed: 48.85 s Calling variants ... Total processed positions in chr9 (chunk 14/31) : 21538 Total time elapsed: 53.60 s Calling variants ... Total processed positions in chr9 (chunk 10/31) : 33522 Total time elapsed: 78.54 s Calling variants ... Total processed positions in chr9 (chunk 11/31) : 37053 Total time elapsed: 86.05 s Calling variants ... Total processed positions in chr9 (chunk 16/31) : 56455 Total time elapsed: 119.76 s Calling variants ... Total processed positions in chr9 (chunk 17/31) : 62283 Total time elapsed: 127.62 s Calling variants ... Total processed positions in chr9 (chunk 9/31) : 63514 Total time elapsed: 131.78 s Calling variants ... Total processed positions in chr9 (chunk 3/31) : 71362 Total time elapsed: 135.99 s Calling variants ... Total processed positions in chr9 (chunk 5/31) : 74395 Total time elapsed: 139.66 s Calling variants ... Total processed positions in chr9 (chunk 24/31) : 79394 Total time elapsed: 141.54 s Calling variants ... Total processed positions in chr9 (chunk 29/31) : 78305 Total time elapsed: 142.68 s Calling variants ... Total processed positions in chr9 (chunk 7/31) : 76859 Total time elapsed: 143.38 s Calling variants ... Total processed positions in chr9 (chunk 8/31) : 79476 Total time elapsed: 144.97 s Calling variants ... Total processed positions in chr9 (chunk 25/31) : 76301 Total time elapsed: 145.07 s Calling variants ... Total processed positions in chr9 (chunk 6/31) : 72961 Total time elapsed: 146.47 s Calling variants ... Total processed positions in chr9 (chunk 26/31) : 82443 Total time elapsed: 148.27 s Calling variants ... Total processed positions in chr9 (chunk 20/31) : 75744 Total time elapsed: 148.83 s Calling variants ... Total processed positions in chr9 (chunk 23/31) : 81576 Total time elapsed: 150.46 s Calling variants ... Total processed positions in chr9 (chunk 2/31) : 77468 Total time elapsed: 151.04 s Calling variants ... Total processed positions in chr9 (chunk 4/31) : 76648 Total time elapsed: 151.73 s Calling variants ... Total processed positions in chr9 (chunk 22/31) : 81615 Total time elapsed: 154.06 s Calling variants ... Total processed positions in chr9 (chunk 19/31) : 78752 Total time elapsed: 154.05 s Calling variants ... Total processed positions in chr9 (chunk 28/31) : 79687 Total time elapsed: 155.51 s Calling variants ... Total processed positions in chr9 (chunk 18/31) : 77860 Total time elapsed: 155.64 s Calling variants ... Total processed positions in chr9 (chunk 27/31) : 81764 Total time elapsed: 157.11 s Calling variants ... Total processed positions in chr9 (chunk 1/31) : 85324 Total time elapsed: 159.15 s Calling variants ... Total processed positions in chr9 (chunk 30/31) : 88072 Total time elapsed: 162.12 s Calling variants ... Total processed positions in chr9 (chunk 21/31) : 89341 Total time elapsed: 152.60 s Calling variants ... Total processed positions in chr9 (chunk 31/31) : 95505 Total time elapsed: 155.05 s real 3m4.784s user 62m4.077s sys 3m33.310s [INFO] 2/7 Select heterozygous SNP variants for Whatshap phasing and haplotagging [INFO] Select heterozygous pileup variants exceeding phasing quality cutoff 14 [INFO] Total heterozygous SNP positions selected: chr9: 77784 real 0m4.754s user 0m4.489s sys 0m0.192s [INFO] 3/7 Phase VCF file using LongPhase LongPhase Ver 1.7 --- File Parameter --- SNP File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_vcf/chr9.vcf SV File : MOD File : REF File : /home/t_steimle/ref/hs1/chm13v2.0.fa Output Prefix : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_vcf/phased_chr9 Generate Dot : False BAM File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --- Phasing Parameter --- Seq Platform : ONT Phase Indel : False Distance Threshold : 300000 Connect Adjacent : 20 Edge Threshold : 0.7 Mapping Quality : 1 Variant Confidence : 0.75 ReadTag Confidence : 0.65 parsing VCF ... 0s parsing SV VCF ... 0s parsing Meth VCF ... 0s reading reference ... 2s (chr9,10s) parsing total: 10s merge results ... 0s writeResult SNP ... 0s total process: 12s real 0m12.323s user 0m23.247s sys 0m1.686s [INFO] 5/7 Select candidates for full-alignment calling [INFO] Set variants quality cutoff 19.0 [INFO] Set reference calls quality cutoff 13.0 [INFO] Low quality reference calls to be processed in chr9: 181543 [INFO] Low quality variants to be processed in chr9: 165238 real 0m5.203s user 0m4.807s sys 0m0.313s [INFO] 6/7 Call low-quality variants using full-alignment model Calling variants ... Total processed positions in chr9 (chunk 19/35) : 10000 Total time elapsed: 92.62 s Calling variants ... Total processed positions in chr9 (chunk 17/35) : 10000 Total time elapsed: 96.32 s Calling variants ... Total processed positions in chr9 (chunk 29/35) : 10000 Total time elapsed: 96.61 s Calling variants ... Total processed positions in chr9 (chunk 2/35) : 10000 Total time elapsed: 96.49 s Calling variants ... Total processed positions in chr9 (chunk 26/35) : 10000 Total time elapsed: 98.14 s Calling variants ... Total processed positions in chr9 (chunk 10/35) : 10000 Total time elapsed: 98.23 s Calling variants ... Total processed positions in chr9 (chunk 7/35) : 10000 Total time elapsed: 98.31 s Calling variants ... Total processed positions in chr9 (chunk 1/35) : 10000 Total time elapsed: 102.22 s Calling variants ... Total processed positions in chr9 (chunk 30/35) : 10000 Total time elapsed: 102.47 s Calling variants ... Total processed positions in chr9 (chunk 25/35) : 10000 Total time elapsed: 102.89 s Calling variants ... Total processed positions in chr9 (chunk 4/35) : 10000 Total time elapsed: 103.42 s Calling variants ... Total processed positions in chr9 (chunk 23/35) : 10000 Total time elapsed: 103.94 s Calling variants ... Total processed positions in chr9 (chunk 28/35) : 10000 Total time elapsed: 104.38 s Calling variants ... Total processed positions in chr9 (chunk 6/35) : 10000 Total time elapsed: 104.32 s Calling variants ... Total processed positions in chr9 (chunk 3/35) : 10000 Total time elapsed: 104.69 s Calling variants ... Total processed positions in chr9 (chunk 20/35) : 10000 Total time elapsed: 104.80 s Calling variants ... Total processed positions in chr9 (chunk 15/35) : 10000 Total time elapsed: 105.81 s Calling variants ... Total processed positions in chr9 (chunk 24/35) : 10000 Total time elapsed: 106.00 s Calling variants ... Total processed positions in chr9 (chunk 12/35) : 10000 Total time elapsed: 106.02 s Calling variants ... Total processed positions in chr9 (chunk 18/35) : 10000 Total time elapsed: 106.35 s Calling variants ... Total processed positions in chr9 (chunk 16/35) : 10000 Total time elapsed: 106.68 s Calling variants ... Total processed positions in chr9 (chunk 8/35) : 10000 Total time elapsed: 106.87 s Calling variants ... Total processed positions in chr9 (chunk 9/35) : 10000 Total time elapsed: 106.89 s Calling variants ... Total processed positions in chr9 (chunk 27/35) : 10000 Total time elapsed: 107.09 s Calling variants ... Total processed positions in chr9 (chunk 21/35) : 10000 Total time elapsed: 107.27 s Calling variants ... Total processed positions in chr9 (chunk 14/35) : 10000 Total time elapsed: 107.48 s Calling variants ... Total processed positions in chr9 (chunk 13/35) : 10000 Total time elapsed: 107.50 s Calling variants ... Total processed positions in chr9 (chunk 11/35) : 10000 Total time elapsed: 107.78 s Calling variants ... Total processed positions in chr9 (chunk 5/35) : 10000 Total time elapsed: 107.99 s Calling variants ... Total processed positions in chr9 (chunk 22/35) : 10000 Total time elapsed: 108.92 s Calling variants ... Total processed positions in chr9 (chunk 35/35) : 6781 Total time elapsed: 55.49 s Calling variants ... Total processed positions in chr9 (chunk 32/35) : 10000 Total time elapsed: 69.14 s Calling variants ... Total processed positions in chr9 (chunk 31/35) : 10000 Total time elapsed: 74.70 s Calling variants ... Total processed positions in chr9 (chunk 34/35) : 10000 Total time elapsed: 78.02 s Calling variants ... Total processed positions in chr9 (chunk 33/35) : 10000 Total time elapsed: 81.45 s real 3m4.889s user 55m57.941s sys 3m16.335s [INFO] 7/7 Merge pileup VCF and full-alignment VCF [INFO] Pileup variants processed in chr9: 70909 [INFO] Full-alignment variants processed in chr9: 180962 real 0m6.255s user 0m6.847s sys 0m0.266s [INFO] Finish calling, output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/merge_output.vcf.gz real 7m3.909s user 119m5.300s sys 6m54.136s [INFO] Call Germline Variant in Tumor BAM using Clair3 [INFO] RUN THE FOLLOWING COMMAND: ( /opt/conda/envs/clairs/bin/run_clair3.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --threads 40 --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output --ctg_name=chr9 --samtools=samtools --pypy=pypy3 --python=python3 --min_coverage=4 --snp_min_af=0.08 --indel_min_af=0.15 --longphase_for_phasing ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/2_CLAIR3_TUMOR.log [INFO] CLAIR3 VERSION: v1.0.8 [INFO] BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam [INFO] REFERENCE FILE PATH: /home/t_steimle/ref/hs1/chm13v2.0.fa [INFO] MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 [INFO] OUTPUT FOLDER: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output [INFO] PLATFORM: ont [INFO] THREADS: 40 [INFO] BED FILE PATH: EMPTY [INFO] VCF FILE PATH: EMPTY [INFO] CONTIGS: chr9 [INFO] CONDA PREFIX: [INFO] SAMTOOLS PATH: samtools [INFO] PYTHON PATH: python3 [INFO] PYPY PATH: pypy3 [INFO] PARALLEL PATH: parallel [INFO] WHATSHAP PATH: whatshap [INFO] LONGPHASE PATH: /opt/conda/envs/clairs/bin/longphase [INFO] CHUNK SIZE: 5000000 [INFO] FULL ALIGN PROPORTION: 0.7 [INFO] FULL ALIGN REFERENCE PROPORTION: 0.1 [INFO] PHASING PROPORTION: 0.7 [INFO] MINIMUM MQ: 5 [INFO] MINIMUM COVERAGE: 4 [INFO] SNP AF THRESHOLD: 0.08 [INFO] INDEL AF THRESHOLD: 0.15 [INFO] BASE ERROR IN GVCF: 0.001 [INFO] GQ BIN SIZE IN GVCF: 5 [INFO] ENABLE FILEUP ONLY CALLING: False [INFO] ENABLE FAST MODE CALLING: False [INFO] ENABLE CALLING SNP CANDIDATES ONLY: False [INFO] ENABLE PRINTING REFERENCE CALLS: False [INFO] ENABLE OUTPUT GVCF: False [INFO] ENABLE HAPLOID PRECISE MODE: False [INFO] ENABLE HAPLOID SENSITIVE MODE: False [INFO] ENABLE INCLUDE ALL CTGS CALLING: False [INFO] ENABLE NO PHASING FOR FULL ALIGNMENT: False [INFO] ENABLE REMOVING INTERMEDIATE FILES: False [INFO] ENABLE LONGPHASE FOR INTERMEDIATE VCF PHASING: True [INFO] ENABLE PHASING FINAL VCF OUTPUT USING WHATSHAP: False [INFO] ENABLE PHASING FINAL VCF OUTPUT USING LONGPHASE: False [INFO] ENABLE HAPLOTAGGING FINAL BAM: False [INFO] ENABLE LONG INDEL CALLING: False [INFO] ENABLE C_IMPLEMENT: True + /opt/conda/envs/clairs/bin/scripts/clair3_c_impl.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --threads 40 --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output --bed_fn=EMPTY --vcf_fn=EMPTY --ctg_name=chr9 --sample_name=SAMPLE --chunk_num=0 --chunk_size=5000000 --samtools=samtools --python=python3 --pypy=pypy3 --parallel=parallel --whatshap=whatshap --qual=2 --var_pct_full=0.7 --ref_pct_full=0.1 --var_pct_phasing=0.7 --snp_min_af=0.08 --indel_min_af=0.15 --min_mq=5 --min_coverage=4 --min_contig_size=0 --pileup_only=False --gvcf=False --base_err=0.001 --gq_bin_size=5 --fast_mode=False --call_snp_only=False --print_ref_calls=False --haploid_precise=False --haploid_sensitive=False --include_all_ctgs=False --no_phasing_for_fa=False --pileup_model_prefix=pileup --fa_model_prefix=full_alignment --remove_intermediate_dir=False --enable_phasing=False --enable_long_indel=False --keep_iupac_bases=False --use_gpu=False --longphase_for_phasing=True --longphase=/opt/conda/envs/clairs/bin/longphase --use_whatshap_for_intermediate_phasing=True --use_longphase_for_intermediate_phasing=True --use_whatshap_for_final_output_phasing=False --use_longphase_for_final_output_phasing=False --use_whatshap_for_final_output_haplotagging=False [INFO] Check environment variables [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/log [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/pileup_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/merge_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/phase_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/gvcf_tmp_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/full_alignment_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_vcf [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_bam [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/full_alignment_output/candidate_bed [INFO] Call variant in contigs: chr9 [INFO] Chunk number for each contig: 31 [INFO] 1/7 Call variants using pileup model Calling variants ... Total processed positions in chr9 (chunk 12/31) : 3313 Total time elapsed: 63.75 s Calling variants ... Total processed positions in chr9 (chunk 13/31) : 9677 Total time elapsed: 75.54 s Calling variants ... Total processed positions in chr9 (chunk 8/31) : 8048 Total time elapsed: 76.99 s Calling variants ... Total processed positions in chr9 (chunk 5/31) : 8157 Total time elapsed: 78.17 s Calling variants ... Total processed positions in chr9 (chunk 18/31) : 8449 Total time elapsed: 78.96 s Calling variants ... Total processed positions in chr9 (chunk 23/31) : 9152 Total time elapsed: 80.13 s Calling variants ... Total processed positions in chr9 (chunk 27/31) : 10222 Total time elapsed: 80.37 s Calling variants ... Total processed positions in chr9 (chunk 9/31) : 7557 Total time elapsed: 81.04 s Calling variants ... Total processed positions in chr9 (chunk 19/31) : 11050 Total time elapsed: 81.73 s Calling variants ... Total processed positions in chr9 (chunk 4/31) : 11173 Total time elapsed: 81.98 s Calling variants ... Total processed positions in chr9 (chunk 6/31) : 10501 Total time elapsed: 82.23 s Calling variants ... Total processed positions in chr9 (chunk 20/31) : 11420 Total time elapsed: 83.61 s Calling variants ... Total processed positions in chr9 (chunk 24/31) : 11713 Total time elapsed: 84.00 s Calling variants ... Total processed positions in chr9 (chunk 26/31) : 12517 Total time elapsed: 84.11 s Calling variants ... Total processed positions in chr9 (chunk 2/31) : 13146 Total time elapsed: 86.25 s Calling variants ... Total processed positions in chr9 (chunk 17/31) : 11841 Total time elapsed: 87.22 s Calling variants ... Total processed positions in chr9 (chunk 1/31) : 15967 Total time elapsed: 89.50 s Calling variants ... Total processed positions in chr9 (chunk 15/31) : 22912 Total time elapsed: 93.68 s Calling variants ... Total processed positions in chr9 (chunk 28/31) : 6980 Total time elapsed: 47.43 s Calling variants ... Total processed positions in chr9 (chunk 7/31) : 10077 Total time elapsed: 48.37 s Calling variants ... Total processed positions in chr9 (chunk 29/31) : 8642 Total time elapsed: 49.57 s Calling variants ... Total processed positions in chr9 (chunk 3/31) : 9920 Total time elapsed: 49.92 s Calling variants ... Total processed positions in chr9 (chunk 25/31) : 9578 Total time elapsed: 50.63 s Calling variants ... Total processed positions in chr9 (chunk 22/31) : 12435 Total time elapsed: 51.57 s Calling variants ... Total processed positions in chr9 (chunk 11/31) : 40307 Total time elapsed: 115.89 s Calling variants ... Total processed positions in chr9 (chunk 30/31) : 12175 Total time elapsed: 53.41 s Calling variants ... Total processed positions in chr9 (chunk 21/31) : 11701 Total time elapsed: 54.40 s Calling variants ... Total processed positions in chr9 (chunk 10/31) : 17506 Total time elapsed: 58.44 s Calling variants ... Total processed positions in chr9 (chunk 31/31) : 16000 Total time elapsed: 55.65 s Calling variants ... Total processed positions in chr9 (chunk 14/31) : 27421 Total time elapsed: 65.13 s Calling variants ... Total processed positions in chr9 (chunk 16/31) : 56368 Total time elapsed: 141.70 s real 2m25.563s user 26m45.350s sys 3m12.482s [INFO] 2/7 Select heterozygous SNP variants for Whatshap phasing and haplotagging [INFO] Select heterozygous pileup variants exceeding phasing quality cutoff 19 [INFO] Total heterozygous SNP positions selected: chr9: 77791 real 0m1.682s user 0m1.459s sys 0m0.151s [INFO] 3/7 Phase VCF file using LongPhase LongPhase Ver 1.7 --- File Parameter --- SNP File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_vcf/chr9.vcf SV File : MOD File : REF File : /home/t_steimle/ref/hs1/chm13v2.0.fa Output Prefix : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_vcf/phased_chr9 Generate Dot : False BAM File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --- Phasing Parameter --- Seq Platform : ONT Phase Indel : False Distance Threshold : 300000 Connect Adjacent : 20 Edge Threshold : 0.7 Mapping Quality : 1 Variant Confidence : 0.75 ReadTag Confidence : 0.65 parsing VCF ... 1s parsing SV VCF ... 0s parsing Meth VCF ... 0s reading reference ... 1s (chr9,45s) parsing total: 45s merge results ... 0s writeResult SNP ... 1s total process: 48s real 0m47.622s user 1m39.107s sys 0m6.712s [INFO] 5/7 Select candidates for full-alignment calling [INFO] Set variants quality cutoff 22.0 [INFO] Set reference calls quality cutoff 8.0 [INFO] Low quality reference calls to be processed in chr9: 19049 [INFO] Low quality variants to be processed in chr9: 171149 real 0m1.655s user 0m1.429s sys 0m0.187s [INFO] 6/7 Call low-quality variants using full-alignment model Calling variants ... Total processed positions in chr9 (chunk 20/20) : 198 Total time elapsed: 5.14 s Calling variants ... Total processed positions in chr9 (chunk 9/20) : 10000 Total time elapsed: 103.97 s Calling variants ... Total processed positions in chr9 (chunk 6/20) : 10000 Total time elapsed: 106.05 s Calling variants ... Total processed positions in chr9 (chunk 1/20) : 10000 Total time elapsed: 108.09 s Calling variants ... Total processed positions in chr9 (chunk 8/20) : 10000 Total time elapsed: 109.62 s Calling variants ... Total processed positions in chr9 (chunk 2/20) : 10000 Total time elapsed: 112.64 s Calling variants ... Total processed positions in chr9 (chunk 3/20) : 10000 Total time elapsed: 115.34 s Calling variants ... Total processed positions in chr9 (chunk 19/20) : 10000 Total time elapsed: 118.04 s Calling variants ... Total processed positions in chr9 (chunk 13/20) : 10000 Total time elapsed: 118.74 s Calling variants ... Total processed positions in chr9 (chunk 17/20) : 10000 Total time elapsed: 119.76 s Calling variants ... Total processed positions in chr9 (chunk 10/20) : 10000 Total time elapsed: 119.63 s Calling variants ... Total processed positions in chr9 (chunk 14/20) : 10000 Total time elapsed: 120.51 s Calling variants ... Total processed positions in chr9 (chunk 7/20) : 10000 Total time elapsed: 121.34 s Calling variants ... Total processed positions in chr9 (chunk 15/20) : 10000 Total time elapsed: 123.04 s Calling variants ... Total processed positions in chr9 (chunk 18/20) : 10000 Total time elapsed: 123.50 s Calling variants ... Total processed positions in chr9 (chunk 4/20) : 10000 Total time elapsed: 125.47 s Calling variants ... Total processed positions in chr9 (chunk 12/20) : 10000 Total time elapsed: 131.99 s Calling variants ... Total processed positions in chr9 (chunk 16/20) : 10000 Total time elapsed: 134.56 s Calling variants ... Total processed positions in chr9 (chunk 11/20) : 10000 Total time elapsed: 139.32 s Calling variants ... Total processed positions in chr9 (chunk 5/20) : 10000 Total time elapsed: 155.07 s real 2m38.507s user 35m49.457s sys 1m53.587s [INFO] 7/7 Merge pileup VCF and full-alignment VCF [INFO] Pileup variants processed in chr9: 73865 [INFO] Full-alignment variants processed in chr9: 160201 real 0m2.867s user 0m2.939s sys 0m0.181s [INFO] Finish calling, output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/merge_output.vcf.gz real 6m8.256s user 64m28.253s sys 5m14.468s [INFO] Select Heterozygous SNP for Phasing [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/parallel_1_select_hetero_snp_for_phasing.log -j 40 pypy3 /opt/bin/clairs.py select_hetero_snp_for_phasing --tumor_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_tumor_output/merge_output.vcf.gz --normal_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/clair3_normal_output/merge_output.vcf.gz --output_folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/vcf --ctg_name {1} --use_heterozygous_snp_in_normal_sample_for_intermediate_phasing True :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/CONTIGS ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/1_select_hetero_snp_for_phasing.log [INFO] Total HET SNP calls selected: chr9: 107191, not found:2348, not match:108, low_qual_count:0. Total normal:119728 Total tumor:109647, pro: 0.9776 [INFO] Phase the Tumor BAM [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/parallel_4_phase_tumor.log -j 40 /opt/conda/envs/clairs/bin/longphase phase -s /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/vcf/{1}.vcf -b /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam -r /home/t_steimle/ref/hs1/chm13v2.0.fa -t 40 -o /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_phased_{1} --ont :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/CONTIGS && parallel -j 40 bgzip -f /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_phased_{1}.vcf :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/CONTIGS ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/4_phase_tumor.log && parallel -j 40 tabix -f -p vcf /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_phased_{1}.vcf.gz :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/CONTIGS LongPhase Ver 1.7 --- File Parameter --- SNP File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/vcf/chr9.vcf SV File : MOD File : REF File : /home/t_steimle/ref/hs1/chm13v2.0.fa Output Prefix : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_phased_chr9 Generate Dot : False BAM File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --- Phasing Parameter --- Seq Platform : ONT Phase Indel : False Distance Threshold : 300000 Connect Adjacent : 20 Edge Threshold : 0.7 Mapping Quality : 1 Variant Confidence : 0.75 ReadTag Confidence : 0.65 parsing VCF ... 1s parsing SV VCF ... 0s parsing Meth VCF ... 0s reading reference ... 1s (chr9,49s) parsing total: 49s merge results ... 0s writeResult SNP ... 1s total process: 52s [INFO] Haplotag the Tumor BAM [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/parallel_5_haplotag_tumor.log -j 40 /opt/conda/envs/clairs/bin/longphase haplotag -o /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_{1} --reference /home/t_steimle/ref/hs1/chm13v2.0.fa --region {1} -s /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_phased_{1}.vcf.gz -b /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/CONTIGS ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/clair3_log/5_tumor_haplotag.log && parallel -j 40 samtools index -@40 /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_{1}.bam :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/CONTIGS phased SNP file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_phased_chr9.vcf.gz phased SV file: phased MOD file: input bam file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam input ref file: /home/t_steimle/ref/hs1/chm13v2.0.fa output bam file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/clair3_output/phased_output/tumor_chr9.bam number of threads: 1 write log file: false log file: ------------------------------------------- tag region: chr9 filter mapping quality below: 1 percentage threshold: 0.6 tag supplementary: false ------------------------------------------- parsing SNP VCF ... 1s tag read start ... chr: chr9 ... 1322s tag read 1325s ------------------------------------------- total process time: 1326s total alignment: 1870680 total supplementary: 62722 total secondary: 0 total unmapped: 0 total tag alignment: 709540 total untagged: 1161140 [INFO] STEP 1: Extract Variant Candidates from Tumor and Normal BAMs [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/parallel_1_extract_tumor_candidates.log -C " " -j 40 pypy3 /opt/bin/clairs.py extract_pair_candidates --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --samtools samtools --snv_min_af 0.05 --indel_min_af 0.1 --chunk_id {2} --chunk_num {3} --ctg_name {1} --platform ont --min_coverage 4 --bed_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/split_beds/{1} --candidates_folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/candidates --output_depth True --select_indel_candidates True --hybrid_mode_vcf_fn None --genotyping_mode_vcf_fn None :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/CHUNK_LIST ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/logs/1_EC.log && pypy3 /opt/bin/clairs.py concat_files --input_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part40/tmp/candidates --input_prefix CANDIDATES_FILE_ --output_fn CANDIDATES_FILES [INFO] chr9 chunk 0/31: Total snv candidates found: 74, total indel candidates found: 46 [INFO] chr9 chunk 1/31: Total snv candidates found: 1194, total indel candidates found: 371 [INFO] chr9 chunk 4/31: Total snv candidates found: 656, total indel candidates found: 215