WARNING: While bind mounting '/mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19:/mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19': destination is already in the mount point list [COMMAND] /opt/bin/run_clairs --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --threads 40 --platform ont_r10_dorado_sup_5khz_ssrs --output_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19 --region chr19:19687872-61707364 --sample_name CHAHA_diag --include_all_ctgs --print_germline_calls --enable_indel_calling --enable_clair3_germline_output --use_longphase_for_intermediate_haplotagging True [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/split_beds [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/pileup_tensor_can [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/fa_tensor_can [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/tmp_vcf_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/vcf [INFO] --include_all_ctgs enabled [INFO] Call variants in contigs: chr19 [INFO] Number of chunks for each contig: 13 [INFO] CALLER VERSION: 0.4.4 [INFO] NORMAL BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam [INFO] TUMOR BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam [INFO] REFERENCE FILE PATH: /home/t_steimle/ref/hs1/chm13v2.0.fa [INFO] PLATFORM: ont_r10_dorado_sup_5khz_ssrs [INFO] THREADS: 40 [INFO] OUTPUT FOLDER: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19 [INFO] OUTPUT VCF PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/output.vcf.gz [INFO] PILEUP MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/pileup.pkl [INFO] FULL-ALIGNMENT MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/full_alignment.pkl [INFO] BED FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/region.bed [INFO] GENOTYPING MODE VCF FILE PATH: None [INFO] HYBRID MODE VCF FILE PATH: None [INFO] REGION FOR CALLING: chr19:19687872-61707364 [INFO] CONTIGS FOR CALLING: None [INFO] CONDA BINARY PREFIX: /opt/conda/envs/clairs [INFO] SAMTOOLS BINARY PATH: samtools [INFO] PYTHON BINARY PATH: python3 [INFO] PYPY BINARY PATH: pypy3 [INFO] PARALLEL BINARY PATH: parallel [INFO] LONGPHASE BINARY PATH: /opt/conda/envs/clairs/bin/longphase [INFO] CHUNK SIZE: 5000000 [INFO] SNV MINIMUM AF: 0.05 [INFO] SNV MINIMUM QUAL: 8 [INFO] INDEL MINIMUM AF: 0.1 [INFO] INDEL PILEUP MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/indel/pileup.pkl [INFO] INDEL FULL-ALIGNMENT MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/indel/full_alignment.pkl [INFO] INDEL MINIMUM QUAL: 8 [INFO] NORMAL VCF FILE PATH: None [INFO] DISABLE PHASING: False [INFO] ENABLE DRY RUN: False [INFO] ENABLE INDEL CALLING: True [INFO] ENABLE PRINTING REFERENCE CALLS: False [INFO] ENABLE PRINTING GERMLINE CALLS: True [INFO] ENABLE INCLUDING ALL CTGS FOR CALLING: True [INFO] ENABLE REMOVING INTERMEDIATE FILES: False [INFO] Call Germline Variants in Normal BAM using Clair3 [INFO] RUN THE FOLLOWING COMMAND: ( /opt/conda/envs/clairs/bin/run_clair3.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --threads 40 --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output --ctg_name=chr19 --samtools=samtools --pypy=pypy3 --python=python3 --min_coverage=4 --snp_min_af=0.08 --indel_min_af=0.15 --longphase_for_phasing ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/1_CLAIR3_NORMAL.log [INFO] CLAIR3 VERSION: v1.0.8 [INFO] BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam [INFO] REFERENCE FILE PATH: /home/t_steimle/ref/hs1/chm13v2.0.fa [INFO] MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 [INFO] OUTPUT FOLDER: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output [INFO] PLATFORM: ont [INFO] THREADS: 40 [INFO] BED FILE PATH: EMPTY [INFO] VCF FILE PATH: EMPTY [INFO] CONTIGS: chr19 [INFO] CONDA PREFIX: [INFO] SAMTOOLS PATH: samtools [INFO] PYTHON PATH: python3 [INFO] PYPY PATH: pypy3 [INFO] PARALLEL PATH: parallel [INFO] WHATSHAP PATH: whatshap [INFO] LONGPHASE PATH: /opt/conda/envs/clairs/bin/longphase [INFO] CHUNK SIZE: 5000000 [INFO] FULL ALIGN PROPORTION: 0.7 [INFO] FULL ALIGN REFERENCE PROPORTION: 0.1 [INFO] PHASING PROPORTION: 0.7 [INFO] MINIMUM MQ: 5 [INFO] MINIMUM COVERAGE: 4 [INFO] SNP AF THRESHOLD: 0.08 [INFO] INDEL AF THRESHOLD: 0.15 [INFO] BASE ERROR IN GVCF: 0.001 [INFO] GQ BIN SIZE IN GVCF: 5 [INFO] ENABLE FILEUP ONLY CALLING: False [INFO] ENABLE FAST MODE CALLING: False [INFO] ENABLE CALLING SNP CANDIDATES ONLY: False [INFO] ENABLE PRINTING REFERENCE CALLS: False [INFO] ENABLE OUTPUT GVCF: False [INFO] ENABLE HAPLOID PRECISE MODE: False [INFO] ENABLE HAPLOID SENSITIVE MODE: False [INFO] ENABLE INCLUDE ALL CTGS CALLING: False [INFO] ENABLE NO PHASING FOR FULL ALIGNMENT: False [INFO] ENABLE REMOVING INTERMEDIATE FILES: False [INFO] ENABLE LONGPHASE FOR INTERMEDIATE VCF PHASING: True [INFO] ENABLE PHASING FINAL VCF OUTPUT USING WHATSHAP: False [INFO] ENABLE PHASING FINAL VCF OUTPUT USING LONGPHASE: False [INFO] ENABLE HAPLOTAGGING FINAL BAM: False [INFO] ENABLE LONG INDEL CALLING: False [INFO] ENABLE C_IMPLEMENT: True + /opt/conda/envs/clairs/bin/scripts/clair3_c_impl.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --threads 40 --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output --bed_fn=EMPTY --vcf_fn=EMPTY --ctg_name=chr19 --sample_name=SAMPLE --chunk_num=0 --chunk_size=5000000 --samtools=samtools --python=python3 --pypy=pypy3 --parallel=parallel --whatshap=whatshap --qual=2 --var_pct_full=0.7 --ref_pct_full=0.1 --var_pct_phasing=0.7 --snp_min_af=0.08 --indel_min_af=0.15 --min_mq=5 --min_coverage=4 --min_contig_size=0 --pileup_only=False --gvcf=False --base_err=0.001 --gq_bin_size=5 --fast_mode=False --call_snp_only=False --print_ref_calls=False --haploid_precise=False --haploid_sensitive=False --include_all_ctgs=False --no_phasing_for_fa=False --pileup_model_prefix=pileup --fa_model_prefix=full_alignment --remove_intermediate_dir=False --enable_phasing=False --enable_long_indel=False --keep_iupac_bases=False --use_gpu=False --longphase_for_phasing=True --longphase=/opt/conda/envs/clairs/bin/longphase --use_whatshap_for_intermediate_phasing=True --use_longphase_for_intermediate_phasing=True --use_whatshap_for_final_output_phasing=False --use_longphase_for_final_output_phasing=False --use_whatshap_for_final_output_haplotagging=False [INFO] Check environment variables [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/log [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/pileup_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/merge_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/phase_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/gvcf_tmp_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/full_alignment_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_vcf [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_bam [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/full_alignment_output/candidate_bed [INFO] Call variant in contigs: chr19 [INFO] Chunk number for each contig: 13 [INFO] 1/7 Call variants using pileup model Calling variants ... Total processed positions in chr19 (chunk 6/13) : 37439 Total time elapsed: 76.50 s Calling variants ... Total processed positions in chr19 (chunk 5/13) : 71150 Total time elapsed: 111.60 s Calling variants ... Total processed positions in chr19 (chunk 7/13) : 77159 Total time elapsed: 123.33 s Calling variants ... Total processed positions in chr19 (chunk 11/13) : 80011 Total time elapsed: 124.76 s Calling variants ... Total processed positions in chr19 (chunk 2/13) : 80468 Total time elapsed: 126.74 s Calling variants ... Total processed positions in chr19 (chunk 3/13) : 77903 Total time elapsed: 128.57 s Calling variants ... Total processed positions in chr19 (chunk 9/13) : 76878 Total time elapsed: 134.04 s Calling variants ... Total processed positions in chr19 (chunk 8/13) : 81026 Total time elapsed: 135.88 s Calling variants ... Total processed positions in chr19 (chunk 12/13) : 83832 Total time elapsed: 136.52 s Calling variants ... Total processed positions in chr19 (chunk 4/13) : 82363 Total time elapsed: 137.86 s Calling variants ... Total processed positions in chr19 (chunk 10/13) : 82264 Total time elapsed: 137.94 s Calling variants ... Total processed positions in chr19 (chunk 1/13) : 86660 Total time elapsed: 138.89 s Calling variants ... Total processed positions in chr19 (chunk 13/13) : 90012 Total time elapsed: 143.08 s real 2m29.543s user 26m56.703s sys 0m54.584s [INFO] 2/7 Select heterozygous SNP variants for Whatshap phasing and haplotagging [INFO] Select heterozygous pileup variants exceeding phasing quality cutoff 14 [INFO] Total heterozygous SNP positions selected: chr19: 52254 real 0m1.699s user 0m1.541s sys 0m0.121s [INFO] 3/7 Phase VCF file using LongPhase LongPhase Ver 1.7 --- File Parameter --- SNP File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_vcf/chr19.vcf SV File : MOD File : REF File : /home/t_steimle/ref/hs1/chm13v2.0.fa Output Prefix : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/tmp/phase_output/phase_vcf/phased_chr19 Generate Dot : False BAM File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --- Phasing Parameter --- Seq Platform : ONT Phase Indel : False Distance Threshold : 300000 Connect Adjacent : 20 Edge Threshold : 0.7 Mapping Quality : 1 Variant Confidence : 0.75 ReadTag Confidence : 0.65 parsing VCF ... 0s parsing SV VCF ... 0s parsing Meth VCF ... 0s reading reference ... 0s (chr19,6s) parsing total: 6s merge results ... 0s writeResult SNP ... 0s total process: 6s real 0m6.096s user 0m11.240s sys 0m0.898s [INFO] 5/7 Select candidates for full-alignment calling [INFO] Set variants quality cutoff 19.0 [INFO] Set reference calls quality cutoff 12.0 [INFO] Low quality reference calls to be processed in chr19: 86848 [INFO] Low quality variants to be processed in chr19: 96058 real 0m1.960s user 0m1.724s sys 0m0.183s [INFO] 6/7 Call low-quality variants using full-alignment model Calling variants ... Total processed positions in chr19 (chunk 19/19) : 2906 Total time elapsed: 36.42 s Calling variants ... Total processed positions in chr19 (chunk 13/19) : 10000 Total time elapsed: 93.49 s Calling variants ... Total processed positions in chr19 (chunk 3/19) : 10000 Total time elapsed: 94.05 s Calling variants ... Total processed positions in chr19 (chunk 11/19) : 10000 Total time elapsed: 93.35 s Calling variants ... Total processed positions in chr19 (chunk 6/19) : 10000 Total time elapsed: 94.34 s Calling variants ... Total processed positions in chr19 (chunk 9/19) : 10000 Total time elapsed: 95.86 s Calling variants ... Total processed positions in chr19 (chunk 5/19) : 10000 Total time elapsed: 117.23 s Calling variants ... Total processed positions in chr19 (chunk 12/19) : 10000 Total time elapsed: 120.15 s Calling variants ... Total processed positions in chr19 (chunk 17/19) : 10000 Total time elapsed: 123.44 s Calling variants ... Total processed positions in chr19 (chunk 4/19) : 10000 Total time elapsed: 123.86 s Calling variants ... Total processed positions in chr19 (chunk 15/19) : 10000 Total time elapsed: 127.14 s Calling variants ... Total processed positions in chr19 (chunk 16/19) : 10000 Total time elapsed: 126.93 s Calling variants ... Total processed positions in chr19 (chunk 18/19) : 10000 Total time elapsed: 128.06 s Calling variants ... Total processed positions in chr19 (chunk 14/19) : 10000 Total time elapsed: 129.98 s Calling variants ... Total processed positions in chr19 (chunk 7/19) : 10000 Total time elapsed: 130.95 s Calling variants ... Total processed positions in chr19 (chunk 1/19) : 10000 Total time elapsed: 130.95 s Calling variants ... Total processed positions in chr19 (chunk 2/19) : 10000 Total time elapsed: 133.96 s Calling variants ... Total processed positions in chr19 (chunk 10/19) : 10000 Total time elapsed: 137.57 s Calling variants ... Total processed positions in chr19 (chunk 8/19) : 10000 Total time elapsed: 138.21 s real 2m21.463s user 35m16.409s sys 1m42.241s [INFO] 7/7 Merge pileup VCF and full-alignment VCF [INFO] Pileup variants processed in chr19: 41215 [INFO] Full-alignment variants processed in chr19: 104681 real 0m2.349s user 0m2.608s sys 0m0.178s [INFO] Finish calling, output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/merge_output.vcf.gz real 5m13.675s user 62m39.427s sys 2m39.420s [INFO] Call Germline Variant in Tumor BAM using Clair3 [INFO] RUN THE FOLLOWING COMMAND: ( /opt/conda/envs/clairs/bin/run_clair3.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --threads 40 --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output --ctg_name=chr19 --samtools=samtools --pypy=pypy3 --python=python3 --min_coverage=4 --snp_min_af=0.08 --indel_min_af=0.15 --longphase_for_phasing ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/2_CLAIR3_TUMOR.log [INFO] CLAIR3 VERSION: v1.0.8 [INFO] BAM FILE PATH: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam [INFO] REFERENCE FILE PATH: /home/t_steimle/ref/hs1/chm13v2.0.fa [INFO] MODEL PATH: /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 [INFO] OUTPUT FOLDER: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output [INFO] PLATFORM: ont [INFO] THREADS: 40 [INFO] BED FILE PATH: EMPTY [INFO] VCF FILE PATH: EMPTY [INFO] CONTIGS: chr19 [INFO] CONDA PREFIX: [INFO] SAMTOOLS PATH: samtools [INFO] PYTHON PATH: python3 [INFO] PYPY PATH: pypy3 [INFO] PARALLEL PATH: parallel [INFO] WHATSHAP PATH: whatshap [INFO] LONGPHASE PATH: /opt/conda/envs/clairs/bin/longphase [INFO] CHUNK SIZE: 5000000 [INFO] FULL ALIGN PROPORTION: 0.7 [INFO] FULL ALIGN REFERENCE PROPORTION: 0.1 [INFO] PHASING PROPORTION: 0.7 [INFO] MINIMUM MQ: 5 [INFO] MINIMUM COVERAGE: 4 [INFO] SNP AF THRESHOLD: 0.08 [INFO] INDEL AF THRESHOLD: 0.15 [INFO] BASE ERROR IN GVCF: 0.001 [INFO] GQ BIN SIZE IN GVCF: 5 [INFO] ENABLE FILEUP ONLY CALLING: False [INFO] ENABLE FAST MODE CALLING: False [INFO] ENABLE CALLING SNP CANDIDATES ONLY: False [INFO] ENABLE PRINTING REFERENCE CALLS: False [INFO] ENABLE OUTPUT GVCF: False [INFO] ENABLE HAPLOID PRECISE MODE: False [INFO] ENABLE HAPLOID SENSITIVE MODE: False [INFO] ENABLE INCLUDE ALL CTGS CALLING: False [INFO] ENABLE NO PHASING FOR FULL ALIGNMENT: False [INFO] ENABLE REMOVING INTERMEDIATE FILES: False [INFO] ENABLE LONGPHASE FOR INTERMEDIATE VCF PHASING: True [INFO] ENABLE PHASING FINAL VCF OUTPUT USING WHATSHAP: False [INFO] ENABLE PHASING FINAL VCF OUTPUT USING LONGPHASE: False [INFO] ENABLE HAPLOTAGGING FINAL BAM: False [INFO] ENABLE LONG INDEL CALLING: False [INFO] ENABLE C_IMPLEMENT: True + /opt/conda/envs/clairs/bin/scripts/clair3_c_impl.sh --bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --threads 40 --model_path /opt/conda/envs/clairs/bin/clairs_models/clair3_models/r1041_e82_400bps_sup_v420 --platform ont --output /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output --bed_fn=EMPTY --vcf_fn=EMPTY --ctg_name=chr19 --sample_name=SAMPLE --chunk_num=0 --chunk_size=5000000 --samtools=samtools --python=python3 --pypy=pypy3 --parallel=parallel --whatshap=whatshap --qual=2 --var_pct_full=0.7 --ref_pct_full=0.1 --var_pct_phasing=0.7 --snp_min_af=0.08 --indel_min_af=0.15 --min_mq=5 --min_coverage=4 --min_contig_size=0 --pileup_only=False --gvcf=False --base_err=0.001 --gq_bin_size=5 --fast_mode=False --call_snp_only=False --print_ref_calls=False --haploid_precise=False --haploid_sensitive=False --include_all_ctgs=False --no_phasing_for_fa=False --pileup_model_prefix=pileup --fa_model_prefix=full_alignment --remove_intermediate_dir=False --enable_phasing=False --enable_long_indel=False --keep_iupac_bases=False --use_gpu=False --longphase_for_phasing=True --longphase=/opt/conda/envs/clairs/bin/longphase --use_whatshap_for_intermediate_phasing=True --use_longphase_for_intermediate_phasing=True --use_whatshap_for_final_output_phasing=False --use_longphase_for_final_output_phasing=False --use_whatshap_for_final_output_haplotagging=False [INFO] Check environment variables [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/log [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/pileup_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/merge_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/phase_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/gvcf_tmp_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/full_alignment_output [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_vcf [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_bam [INFO] Create folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/full_alignment_output/candidate_bed [INFO] Call variant in contigs: chr19 [INFO] Chunk number for each contig: 13 [INFO] 1/7 Call variants using pileup model Calling variants ... Total processed positions in chr19 (chunk 10/13) : 10530 Total time elapsed: 50.88 s Calling variants ... Total processed positions in chr19 (chunk 8/13) : 10757 Total time elapsed: 52.23 s Calling variants ... Total processed positions in chr19 (chunk 3/13) : 11930 Total time elapsed: 52.66 s Calling variants ... Total processed positions in chr19 (chunk 5/13) : 13827 Total time elapsed: 55.87 s Calling variants ... Total processed positions in chr19 (chunk 11/13) : 13999 Total time elapsed: 56.00 s Calling variants ... Total processed positions in chr19 (chunk 4/13) : 15863 Total time elapsed: 56.26 s Calling variants ... Total processed positions in chr19 (chunk 9/13) : 15113 Total time elapsed: 57.61 s Calling variants ... Total processed positions in chr19 (chunk 12/13) : 15431 Total time elapsed: 59.04 s Calling variants ... Total processed positions in chr19 (chunk 13/13) : 19185 Total time elapsed: 62.90 s Calling variants ... Total processed positions in chr19 (chunk 2/13) : 18004 Total time elapsed: 62.98 s Calling variants ... Total processed positions in chr19 (chunk 1/13) : 18455 Total time elapsed: 65.21 s Calling variants ... Total processed positions in chr19 (chunk 7/13) : 43919 Total time elapsed: 97.41 s Calling variants ... Total processed positions in chr19 (chunk 6/13) : 53083 Total time elapsed: 107.16 s real 1m50.498s user 10m58.237s sys 0m47.782s [INFO] 2/7 Select heterozygous SNP variants for Whatshap phasing and haplotagging [INFO] Select heterozygous pileup variants exceeding phasing quality cutoff 18 [INFO] Total heterozygous SNP positions selected: chr19: 62515 real 0m0.831s user 0m0.695s sys 0m0.107s [INFO] 3/7 Phase VCF file using LongPhase LongPhase Ver 1.7 --- File Parameter --- SNP File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_vcf/chr19.vcf SV File : MOD File : REF File : /home/t_steimle/ref/hs1/chm13v2.0.fa Output Prefix : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/tmp/phase_output/phase_vcf/phased_chr19 Generate Dot : False BAM File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --- Phasing Parameter --- Seq Platform : ONT Phase Indel : False Distance Threshold : 300000 Connect Adjacent : 20 Edge Threshold : 0.7 Mapping Quality : 1 Variant Confidence : 0.75 ReadTag Confidence : 0.65 parsing VCF ... 0s parsing SV VCF ... 0s parsing Meth VCF ... 0s reading reference ... 0s (chr19,20s) parsing total: 20s merge results ... 0s writeResult SNP ... 1s total process: 21s real 0m21.317s user 0m54.224s sys 0m2.968s [INFO] 5/7 Select candidates for full-alignment calling [INFO] Set variants quality cutoff 21.0 [INFO] Set reference calls quality cutoff 7.0 [INFO] Low quality reference calls to be processed in chr19: 10511 [INFO] Low quality variants to be processed in chr19: 108203 real 0m0.832s user 0m0.670s sys 0m0.126s [INFO] 6/7 Call low-quality variants using full-alignment model Calling variants ... Total processed positions in chr19 (chunk 12/12) : 8714 Total time elapsed: 93.54 s Calling variants ... Total processed positions in chr19 (chunk 6/12) : 10000 Total time elapsed: 95.64 s Calling variants ... Total processed positions in chr19 (chunk 7/12) : 10000 Total time elapsed: 97.47 s Calling variants ... Total processed positions in chr19 (chunk 8/12) : 10000 Total time elapsed: 100.08 s Calling variants ... Total processed positions in chr19 (chunk 5/12) : 10000 Total time elapsed: 101.49 s Calling variants ... Total processed positions in chr19 (chunk 4/12) : 10000 Total time elapsed: 106.04 s Calling variants ... Total processed positions in chr19 (chunk 2/12) : 10000 Total time elapsed: 112.68 s Calling variants ... Total processed positions in chr19 (chunk 11/12) : 10000 Total time elapsed: 113.52 s Calling variants ... Total processed positions in chr19 (chunk 3/12) : 10000 Total time elapsed: 114.85 s Calling variants ... Total processed positions in chr19 (chunk 1/12) : 10000 Total time elapsed: 116.61 s Calling variants ... Total processed positions in chr19 (chunk 10/12) : 10000 Total time elapsed: 118.10 s Calling variants ... Total processed positions in chr19 (chunk 9/12) : 10000 Total time elapsed: 123.39 s real 2m5.940s user 20m24.906s sys 1m5.423s [INFO] 7/7 Merge pileup VCF and full-alignment VCF [INFO] Pileup variants processed in chr19: 46569 [INFO] Full-alignment variants processed in chr19: 101152 real 0m1.492s user 0m1.512s sys 0m0.131s [INFO] Finish calling, output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/merge_output.vcf.gz real 4m26.312s user 32m24.642s sys 1m57.158s [INFO] Select Heterozygous SNP for Phasing [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/parallel_1_select_hetero_snp_for_phasing.log -j 40 pypy3 /opt/bin/clairs.py select_hetero_snp_for_phasing --tumor_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/merge_output.vcf.gz --normal_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_normal_output/merge_output.vcf.gz --output_folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/vcf --ctg_name {1} --use_heterozygous_snp_in_normal_sample_for_intermediate_phasing True :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/1_select_hetero_snp_for_phasing.log [INFO] Total HET SNP calls selected: chr19: 84452, not found:765, not match:81, low_qual_count:0. Total normal:76459 Total tumor:85298, pro: 0.9901 [INFO] Phase the Tumor BAM [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/parallel_4_phase_tumor.log -j 40 /opt/conda/envs/clairs/bin/longphase phase -s /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/vcf/{1}.vcf -b /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam -r /home/t_steimle/ref/hs1/chm13v2.0.fa -t 40 -o /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_phased_{1} --ont :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS && parallel -j 40 bgzip -f /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_phased_{1}.vcf :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/4_phase_tumor.log && parallel -j 40 tabix -f -p vcf /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_phased_{1}.vcf.gz :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS LongPhase Ver 1.7 --- File Parameter --- SNP File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/vcf/chr19.vcf SV File : MOD File : REF File : /home/t_steimle/ref/hs1/chm13v2.0.fa Output Prefix : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_phased_chr19 Generate Dot : False BAM File : /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --- Phasing Parameter --- Seq Platform : ONT Phase Indel : False Distance Threshold : 300000 Connect Adjacent : 20 Edge Threshold : 0.7 Mapping Quality : 1 Variant Confidence : 0.75 ReadTag Confidence : 0.65 parsing VCF ... 0s parsing SV VCF ... 0s parsing Meth VCF ... 0s reading reference ... 1s (chr19,35s) parsing total: 35s merge results ... 0s writeResult SNP ... 0s total process: 36s [INFO] Haplotag the Tumor BAM [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/parallel_5_haplotag_tumor.log -j 40 /opt/conda/envs/clairs/bin/longphase haplotag -o /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_{1} --reference /home/t_steimle/ref/hs1/chm13v2.0.fa --region {1} -s /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_phased_{1}.vcf.gz -b /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/clair3_log/5_tumor_haplotag.log && parallel -j 40 samtools index -@40 /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_{1}.bam :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS phased SNP file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_phased_chr19.vcf.gz phased SV file: phased MOD file: input bam file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam input ref file: /home/t_steimle/ref/hs1/chm13v2.0.fa output bam file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_chr19.bam number of threads: 1 write log file: false log file: ------------------------------------------- tag region: chr19 filter mapping quality below: 1 percentage threshold: 0.6 tag supplementary: false ------------------------------------------- parsing SNP VCF ... 0s tag read start ... chr: chr19 ... 624s tag read 625s ------------------------------------------- total process time: 625s total alignment: 734582 total supplementary: 28447 total secondary: 0 total unmapped: 0 total tag alignment: 390188 total untagged: 344394 [INFO] STEP 1: Extract Variant Candidates from Tumor and Normal BAMs [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_1_extract_tumor_candidates.log -C " " -j 40 pypy3 /opt/bin/clairs.py extract_pair_candidates --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --samtools samtools --snv_min_af 0.05 --indel_min_af 0.1 --chunk_id {2} --chunk_num {3} --ctg_name {1} --platform ont --min_coverage 4 --bed_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/split_beds/{1} --candidates_folder /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates --output_depth True --select_indel_candidates True --hybrid_mode_vcf_fn None --genotyping_mode_vcf_fn None :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CHUNK_LIST ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/1_EC.log && pypy3 /opt/bin/clairs.py concat_files --input_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates --input_prefix CANDIDATES_FILE_ --output_fn CANDIDATES_FILES [INFO] chr19 chunk 2/13: Total snv candidates found: 8283, total indel candidates found: 192 [INFO] chr19 chunk 1/13: Total snv candidates found: 8305, total indel candidates found: 982 [INFO] chr19 chunk 9/13: Total snv candidates found: 1134, total indel candidates found: 1711 [INFO] chr19 chunk 8/13: Total snv candidates found: 724, total indel candidates found: 1286 [INFO] chr19 chunk 6/13: Total snv candidates found: 1237, total indel candidates found: 1303 [INFO] chr19 chunk 3/13: Total snv candidates found: 8985, total indel candidates found: 1029 [INFO] chr19 chunk 11/13: Total snv candidates found: 1775, total indel candidates found: 1483 [INFO] chr19 chunk 5/13: Total snv candidates found: 930, total indel candidates found: 1172 [INFO] chr19 chunk 4/13: Total snv candidates found: 622, total indel candidates found: 1052 [INFO] chr19 chunk 10/13: Total snv candidates found: 1091, total indel candidates found: 1615 [INFO] chr19 chunk 12/13: Total snv candidates found: 1165, total indel candidates found: 1426 [faidx] Truncated sequence: chr19:58474127-61708401 [INFO] chr19 chunk 0/13: Total snv candidates found: 1028, total indel candidates found: 1065 [INFO] chr19 chunk 7/13: Total snv candidates found: 774, total indel candidates found: 1459 [INFO] STEP 2: Pileup Model Calling [INFO] Create Paired Tensors [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_2-1_create_pair_tensor.log -j 40 pypy3 /opt/bin/clairs.py create_pair_tensor_pileup --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --ctg_name {1/.} --samtools samtools --candidates_bed_regions {1} --tensor_can_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/pileup_tensor_can/{1/} --platform ont :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/2-1_CPT.log [INFO] chr19 chunk 1-1/4 Tensors generated: 612 [INFO] chr19 chunk 1-1/8 Tensors generated: 721 [INFO] chr19 chunk 1-1/5 Tensors generated: 908 [INFO] chr19 chunk 1-1/7 Tensors generated: 772 [INFO] chr19 chunk 1-1/0 Tensors generated: 1025 [INFO] chr19 chunk 1-1/9 Tensors generated: 1129 [INFO] chr19 chunk 1-1/12 Tensors generated: 1143 [INFO] chr19 chunk 1-1/6 Tensors generated: 1221 [INFO] chr19 chunk 1-1/10 Tensors generated: 1090 [INFO] chr19 chunk 1-1/11 Tensors generated: 1774 [INFO] chr19 chunk 1-1/3 Tensors generated: 8980 [INFO] chr19 chunk 1-1/1 Tensors generated: 8202 [INFO] chr19 chunk 1-1/2 Tensors generated: 8263 [INFO] Pileup Model Prediction [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_2-2_predict.log -j 40 python3 /opt/bin/clairs.py predict --tensor_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/pileup_tensor_can/{1/} --call_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/p_{1/}.vcf --chkpnt_fn /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/pileup.pkl --use_gpu False --platform ont --ctg_name {1/.} --pileup --show_germline :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/2-2_PREDICT.log [INFO] chr19 total processed positions: 612, time elapsed: 1.4s [INFO] chr19 total processed positions: 721, time elapsed: 1.6s [INFO] chr19 total processed positions: 772, time elapsed: 1.7s [INFO] chr19 total processed positions: 908, time elapsed: 1.9s [INFO] chr19 total processed positions: 1143, time elapsed: 2.3s [INFO] chr19 total processed positions: 1090, time elapsed: 2.3s [INFO] chr19 total processed positions: 1129, time elapsed: 2.3s [INFO] chr19 total processed positions: 1221, time elapsed: 2.4s [INFO] chr19 total processed positions: 1025, time elapsed: 2.7s [INFO] chr19 total processed positions: 1774, time elapsed: 4.0s [INFO] chr19 total processed positions: 8263, time elapsed: 13.1s [INFO] chr19 total processed positions: 8202, time elapsed: 13.1s [INFO] chr19 total processed positions: 8980, time elapsed: 14.4s [INFO] Merge Pileup VCFs [INFO] RUN THE FOLLOWING COMMAND: pypy3 /opt/bin/clairs.py sort_vcf --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --contigs_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS --input_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output --vcf_fn_prefix p_ --output_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/pileup.vcf [INFO] Sorting VCFs... [INFO] Finished VCF sorting! [INFO] STEP 3: Full-alignment Model Calling [INFO] Create Full-alignment Paired Tensors [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_3-1_create_pair_tensor_fa.log -j 40 pypy3 /opt/bin/clairs.py create_pair_tensor --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_{1/.}.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --ctg_name {1/.} --samtools samtools --candidates_bed_regions {1} --tensor_can_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/fa_tensor_can/{1/} --platform ont :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/3-1_CPT.log [INFO] chr19 chunk 1-1/4 Tensors generated: 612 [INFO] chr19 chunk 1-1/8 Tensors generated: 721 [INFO] chr19 chunk 1-1/7 Tensors generated: 772 [INFO] chr19 chunk 1-1/5 Tensors generated: 908 [INFO] chr19 chunk 1-1/0 Tensors generated: 1025 [INFO] chr19 chunk 1-1/6 Tensors generated: 1091 [INFO] chr19 chunk 1-1/9 Tensors generated: 1121 [INFO] chr19 chunk 1-1/10 Tensors generated: 1089 [INFO] chr19 chunk 1-1/12 Tensors generated: 1139 [INFO] chr19 chunk 1-1/11 Tensors generated: 1774 [INFO] chr19 chunk 1-1/2 Tensors generated: 6524 [INFO] chr19 chunk 1-1/1 Tensors generated: 7891 [INFO] chr19 chunk 1-1/3 Tensors generated: 8612 [INFO] Full-alignment Model Prediction [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_3-2_predict.log -j 40 python3 /opt/bin/clairs.py predict --tensor_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/fa_tensor_can/{1/} --call_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/fa_{1/}.vcf --chkpnt_fn /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/full_alignment.pkl --use_gpu False --platform ont --ctg_name {1/.} --show_germline :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/3-2_PREDICT.log [INFO] chr19 total processed positions: 612, time elapsed: 108.1s [INFO] chr19 total processed positions: 721, time elapsed: 127.1s [INFO] chr19 total processed positions: 772, time elapsed: 136.3s [INFO] chr19 total processed positions: 908, time elapsed: 160.0s [INFO] chr19 total processed positions: 1025, time elapsed: 180.3s [INFO] chr19 total processed positions: 1089, time elapsed: 191.9s [INFO] chr19 total processed positions: 1091, time elapsed: 192.8s [INFO] chr19 total processed positions: 1121, time elapsed: 198.1s [INFO] chr19 total processed positions: 1139, time elapsed: 201.0s [INFO] chr19 total processed positions: 1774, time elapsed: 312.7s [INFO] chr19 total processed positions: 6524, time elapsed: 1146.6s [INFO] chr19 total processed positions: 7891, time elapsed: 1393.6s [INFO] chr19 total processed positions: 8612, time elapsed: 1517.7s [INFO] Merge Full-alignment VCFs [INFO] RUN THE FOLLOWING COMMAND: pypy3 /opt/bin/clairs.py sort_vcf --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --contigs_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS --input_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output --vcf_fn_prefix fa_ --output_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/full_alignment.vcf [INFO] Sorting VCFs... [INFO] Finished VCF sorting! [INFO] STEP 4: Haplotype filtering [INFO] RUN THE FOLLOWING COMMAND: ( pypy3 /opt/bin/clairs.py haplotype_filtering --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_ --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --germline_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/merge_output.vcf.gz --pileup_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/pileup.vcf --full_alignment_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/full_alignment.vcf --output_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output --samtools samtools --pypy3 pypy3 --parallel parallel --threads 40 ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/4_HAP_FILTER.log [INFO] Processing in chr19, total processed positions: 1000 Total input calls: 4369, filtered by haplotype match 24 [INFO] STEP 5: Merge and sort VCF [INFO] RUN THE FOLLOWING COMMAND: ( pypy3 /opt/bin/clairs.py merge_vcf --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --pileup_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/pileup_filter.vcf --full_alignment_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/full_alignment_filter.vcf --output_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/output.vcf --platform ont --qual 8 --sample_name CHAHA_diag --enable_indel_calling True --prefer_recall False --cmdline /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CMD ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/5_MV.log [INFO] Full-alignment variants filtered by pileup: 400 [INFO] STEP 6: Indel Pileup Model Calling [INFO] Create Paired Tensors [INFO] RUN THE FOLLOWING COMMAND: pypy3 /opt/bin/clairs.py concat_files --input_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates --input_prefix INDEL_CANDIDATES_FILE_ --output_fn INDEL_CANDIDATES_FILES && ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_6-1_create_pair_tensor_indel.log -j 40 pypy3 /opt/bin/clairs.py create_pair_tensor_pileup --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/CHAHA_diag_hs1.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --ctg_name {1/.} --samtools samtools --candidates_bed_regions {1} --tensor_can_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/pileup_tensor_can/indel_{1/} --platform ont :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/INDEL_CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/6-1_CPTI.log [INFO] chr19 Tensors generated: 192 [INFO] chr19 Tensors generated: 1047 [INFO] chr19 Tensors generated: 1063 [INFO] chr19 Tensors generated: 976 [INFO] chr19 Tensors generated: 1158 [INFO] chr19 Tensors generated: 1281 [INFO] chr19 Tensors generated: 1029 [INFO] chr19 Tensors generated: 1299 [INFO] chr19 Tensors generated: 1483 [INFO] chr19 Tensors generated: 1417 [INFO] chr19 Tensors generated: 1456 [INFO] chr19 Tensors generated: 1707 [INFO] chr19 Tensors generated: 1615 [INFO] Indel Pileup Model Prediction [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_6-2_predict_indel.log -j 40 python3 /opt/bin/clairs.py predict --tensor_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/pileup_tensor_can/indel_{1/} --call_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_p_{1/}.vcf --chkpnt_fn /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/indel/pileup.pkl --use_gpu False --platform ont --ctg_name {1/.} --pileup --enable_indel_calling True --show_germline :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/INDEL_CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/6-2_PREDICT_INDEL.log [INFO] chr19 total processed positions: 192, time elapsed: 0.5s [INFO] chr19 total processed positions: 976, time elapsed: 1.8s [INFO] chr19 total processed positions: 1047, time elapsed: 1.9s [INFO] chr19 total processed positions: 1063, time elapsed: 2.0s [INFO] chr19 total processed positions: 1158, time elapsed: 2.1s [INFO] chr19 total processed positions: 1029, time elapsed: 2.1s [INFO] chr19 total processed positions: 1299, time elapsed: 2.5s [INFO] chr19 total processed positions: 1281, time elapsed: 2.4s [INFO] chr19 total processed positions: 1456, time elapsed: 2.5s [INFO] chr19 total processed positions: 1615, time elapsed: 2.8s [INFO] chr19 total processed positions: 1417, time elapsed: 2.5s [INFO] chr19 total processed positions: 1483, time elapsed: 2.6s [INFO] chr19 total processed positions: 1707, time elapsed: 3.4s [INFO] Merge Pileup VCFs [INFO] RUN THE FOLLOWING COMMAND: pypy3 /opt/bin/clairs.py sort_vcf --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --contigs_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS --input_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output --vcf_fn_prefix indel_p_ --output_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_pileup.vcf [INFO] Sorting VCFs... [INFO] Finished VCF sorting! [INFO] STEP 7: Indel Full-alignment Model Calling [INFO] Create Full-alignment Paired Tensors [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_7-1_create_pair_tensor_fa_indel.log -j 40 pypy3 /opt/bin/clairs.py create_pair_tensor --normal_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/norm/CHAHA_norm_hs1.bam --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_{1/.}.bam --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --ctg_name {1/.} --samtools samtools --candidates_bed_regions {1} --tensor_can_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/fa_tensor_can/indel_{1/} --platform ont :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/INDEL_CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/7-1_CPTI.log [INFO] chr19 Tensors generated: 158 [INFO] chr19 Tensors generated: 960 [INFO] chr19 Tensors generated: 1014 [INFO] chr19 Tensors generated: 1047 [INFO] chr19 Tensors generated: 1063 [INFO] chr19 Tensors generated: 1158 [INFO] chr19 Tensors generated: 1281 [INFO] chr19 Tensors generated: 1293 [INFO] chr19 Tensors generated: 1456 [INFO] chr19 Tensors generated: 1483 [INFO] chr19 Tensors generated: 1417 [INFO] chr19 Tensors generated: 1702 [INFO] chr19 Tensors generated: 1613 [INFO] Indel Full-alignment Model Prediction [INFO] RUN THE FOLLOWING COMMAND: ( parallel --joblog /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/parallel_7-2_predict.log -j 40 python3 /opt/bin/clairs.py predict --tensor_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/fa_tensor_can/indel_{1/} --call_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_fa_{1/}.vcf --chkpnt_fn /opt/conda/envs/clairs/bin/clairs_models/ont_r10_dorado_sup_5khz_ssrs/indel/full_alignment.pkl --use_gpu False --platform ont --ctg_name {1/.} --enable_indel_calling True --show_germline :::: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/candidates/INDEL_CANDIDATES_FILES ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/7-2_PREDICT_INDEL.log [INFO] chr19 total processed positions: 158, time elapsed: 24.8s [INFO] chr19 total processed positions: 960, time elapsed: 158.9s [INFO] chr19 total processed positions: 1014, time elapsed: 170.0s [INFO] chr19 total processed positions: 1047, time elapsed: 177.1s [INFO] chr19 total processed positions: 1063, time elapsed: 178.3s [INFO] chr19 total processed positions: 1158, time elapsed: 194.8s [INFO] chr19 total processed positions: 1281, time elapsed: 214.9s [INFO] chr19 total processed positions: 1293, time elapsed: 216.6s [INFO] chr19 total processed positions: 1417, time elapsed: 238.2s [INFO] chr19 total processed positions: 1456, time elapsed: 244.8s [INFO] chr19 total processed positions: 1483, time elapsed: 249.6s [INFO] chr19 total processed positions: 1613, time elapsed: 271.6s [INFO] chr19 total processed positions: 1702, time elapsed: 284.8s [INFO] Merge Full-alignment VCFs [INFO] RUN THE FOLLOWING COMMAND: pypy3 /opt/bin/clairs.py sort_vcf --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --contigs_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CONTIGS --input_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output --vcf_fn_prefix indel_fa_ --output_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_full_alignment.vcf [INFO] Sorting VCFs... [INFO] Finished VCF sorting! [INFO] Indel Haplotype filtering [INFO] RUN THE FOLLOWING COMMAND: ( pypy3 /opt/bin/clairs.py haplotype_filtering --tumor_bam_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/phased_output/tumor_ --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --germline_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/merge_output.vcf.gz --pileup_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_pileup.vcf --full_alignment_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_full_alignment.vcf --output_dir /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output --samtools samtools --pypy3 pypy3 --parallel parallel --threads 40 --is_indel ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/8_INDEL_HAP_FILTER.log Total input calls: 1438, filtered by haplotype match 12 [INFO] STEP 8: Merge and sort Indel VCF [INFO] RUN THE FOLLOWING COMMAND: ( pypy3 /opt/bin/clairs.py merge_vcf --ref_fn /home/t_steimle/ref/hs1/chm13v2.0.fa --pileup_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_pileup_filter.vcf --full_alignment_vcf_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/vcf_output/indel_full_alignment_filter.vcf --output_fn /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/indel.vcf --platform ont --qual 8 --sample_name CHAHA_diag --enable_indel_calling True --indel_calling --prefer_recall False --cmdline /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/CMD ) 2>&1 | tee /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/logs/8_MVI.log [INFO] Full-alignment variants filtered by pileup: 125 [INFO] Total time elapsed: 54m50.00s [INFO] Finish calling, output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/output.vcf.gz [INFO] Finish calling, snv output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/snv.vcf.gz [INFO] Finish calling, indel output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/indel.vcf.gz [INFO] Finish calling, Clair3 tumor BAM germline output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/clair3_tumor_germline_output.vcf.gz [INFO] Finish calling, Clair3 normal BAM germline output file: /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/clair3_normal_germline_output.vcf.gz cp /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/tmp/clair3_output/clair3_tumor_output/merge_output.vcf.gz /mnt/beegfs02/scratch/t_steimle/data/wgs/CHAHA/diag/ClairS/part19/clair3_tumor_germline_output.vcf.gz ---------------------------------------------- | SLURM EPILOG | ---------------------------------------------- Job ID: 2556286 Cluster: flamingo User/Group: t_steimle/gs_hpc_biopath Nodelist: n18 Nodes: 1 Cores per node: 40 Job started at: 2025-12-08 15:50:03 Job ended at: 2025-12-08 16:44:55 Job Wall-clock time: 00:54:52 CPU Utilized: 05:06:15 CPU Efficiency: 13.95% of 1-12:34:40 core-walltime Memory Utilized: 20.57 GB Memory Efficiency: 51.43% of 40.00 GB