This repo contains scripts and utilities for analyzing tandem repeats (TRs).
The latest docker images:
str-analysis python scripts:
weisburd/str-analysis@sha256:fac61a91d6273496bc1083e85c43a0a07963b7207e47483a03091e76ed15e5be
scripts + ExpansionHunter:
weisburd/str-analysis-with-expansion-hunter@sha256:c1d6d82076eded612da038a3597fd89a67426f6e8af94b21d55cfbaefc7fa44a
scripts + TRGT:
weisburd/str-analysis-with-trgt@sha256:e6d901a44705b7aab57984df617bdc0398cdf35b1dae29917736d009beaa449c
scripts + ATaRVa:
weisburd/str-analysis-with-atarva@sha256:87ac0ac07c9244ebfa0d0724f2ed7c402a4da07e4b35f0fdb1b161b38646bdd9
scripts + inquiSTR:
weisburd/str-analysis-with-inquistr@sha256:444928661ab5e064fc9bf4b796a09d53b198d785a488a7c6b247f04d3c09992e
scripts + HipSTR:
weisburd/str-analysis-with-hipstr@sha256:ca5653d981fe9635e62254195b65842b33930cdd945952c0822344895ed446ee
scripts + GangSTR:
weisburd/str-analysis-with-gangstr@sha256:226e2528762509d3d26e95abfa659c2646ebdd5380a9ebf6fcca18cfa9a55edc
scripts + EnsembleTR:
weisburd/str-analysis-with-ensembletr@sha256:b73799a79033ea90e20beb61c32dabaa0a01431091614870bf4fb64f2bd222b4
scripts + LongTR:
weisburd/str-analysis-with-longtr@sha256:efb9ca09e7ab75745773e39c7aab35af70e36024a32b2ad71ab28da6c930f3e5
To install the latest version using pip, run:
python3 -m pip install --upgrade str-analysis@git+https://github.com/broadinstitute/str-analysis
or use the docker image:
docker run -it weisburd/str-analysis:latest
- Variant Filtering & Detection
- Catalog Management & Annotation
- ExpansionHunter Results Processing
- TRGT and LPS Processing
- ExpansionHunterDenovo Post-Processing
- Read Extraction
- Simulation
- gnomAD STR Data Generation
- Catalog Format Conversion
- call_non_ref_motifs - Takes a BAM/CRAM file and, optionally, an ExpansionHunter variant catalog. For each locus, determines which STR motifs are supported by reads overlapping that locus before running ExpansionHunter on the detected motif(s). Useful for detecting non-reference pathogenic motifs (e.g., RFC1). (docs)
- filter_vcf_to_tandem_repeats - Replaces & supersedes filter_vcf_to_STR_variants. This script takes a VCF (from short-read, long-read, or assembly-based callers - it just has to have full REF and ALT sequences) and filters it to insertions and deletions that represent tandem repeat expansions or contractions. It uses both brute-force k-mer search for perfect repeats and Tandem Repeats Finder (TRF) for imperfect repeats, thereby improving detection of VNTRs. It includes 3 subcommands: catalog, merge, genotype (intended to be run in this order). The
catalogsubcommand takes a VCF and a reference FASTA (no TR catalog needed) and detects which insertions and deletions are actually tandem repeat expansions or contractions. It then outputs a BED file of the reference repeat intervals and motifs (a TR catalog) based on what it detected in the VCF.merge(optionally) combines two or more catalogs into a single catalog using some simple heuristics to partially deduplicate redundant entries. Thegenotypesubcommand takes a VCF (not necessarily the same one), a TR catalog BED file and a reference FASTA, and then outputs the diploid TR genotypes at each locus in the catalog derived from the insertion and deletion genotypes in the VCF. - filter_vcf_to_STR_variants - Original version of the above tool. Takes a single-sample VCF file and filters it to the INS/DEL variants that represent tandem repeat expansions or contractions by performing brute-force k-mer search on each variant's inserted or deleted bases. This tool was a core part of Weisburd, B., Tiao, G. & Rehm, H. L. Insights from a genome-wide truth set of tandem repeat variation. (2023)
- parse_motif_composition - Simple algorithm that takes a table of motifs known to occur at a particular VNTR or STR locus with motif variability (such as CACNA1C or RFC1), and also takes a BAM/CRAM data file, and then computes the observed frequency of these motifs in the reads aligned to this locus.
- merge_loci - Takes one or more STR catalogs and combines them into a single catalog while removing duplicates based on overlap and repeat motif.
- annotate_and_filter_str_catalog - Takes an STR catalog and annotates loci based on their overlap with genes and known disease-associated STRs. Allows filtering by motif size, gene region, and various other criteria.
- compute_catalog_stats - Takes an annotated catalog output by annotate_and_filter_str_catalog and computes various summary statistics.
- add_offtarget_regions - Takes an ExpansionHunter variant catalog and adds a list of off-target regions to each locus definition by querying a database of off-target regions precomputed for each TR motif. This database was generated by using wgsim to simulate fully repetitive reads for each motif, and then recording where these reads mapped on hg19 and hg38 after aligning them using bwa.
- add_adjacent_loci_to_expansion_hunter_catalog - Takes an ExpansionHunter variant catalog and a BED file containing all simple repeats in the reference genome. Outputs a new catalog with updated LocusStructures and ReferenceRegions that include any adjacent repeats found near each locus.
- split_adjacent_loci_in_expansion_hunter_catalog - Splits loci with adjacent repeats back into separate loci.
- filter_out_loci_with_Ns_in_flanks - Removes loci from an ExpansionHunter catalog if their flanks contain enough Ns to trigger an ExpansionHunter error.
- maximize_motif_purity - Adjusts motifs to maximize purity for each catalog interval.
- trim_repeat_loci - Trims BED intervals so locus size is a multiple of motif size.
- combine_str_json_to_tsv - Takes a set of ExpansionHunter JSON output files and combines them into a single TSV table.
- combine_json_to_tsv - Takes a set of arbitrary JSON files that share the same schema and combines their top-level fields into a single TSV file.
- postprocess_expansion_hunter_results - Cleans and post-processes ExpansionHunter output (filter hom-ref calls, convert formats).
- filter_to_largest_expansions - Takes a combined table of TR genotypes (from combine_str_json_to_tsv) and filters it to the N most expanded genotypes per locus, based on either the long allele or the short allele (for recessive searches). Optionally filters by quality score, purity, and can discard non-polymorphic loci.
- copy_EH_vcf_fields_to_json - Takes the ExpansionHunter output VCF and JSON file for a given sample and copies fields that are only present in the VCF to the JSON file.
- check_trios_for_mendelian_violations - Takes a table of combined ExpansionHunter calls generated by combine_str_json_to_tsv as well as a FAM or PED file with parent/child relationships, and outputs a table of Mendelian violations in the callset.
- compute_average_genotype_quality_per_sample - Computes average Q score per sample from a genotype table.
- check_combined_results_tsv_for_pathogenic_repeats - Filters combined results for potential pathogenic expansions at known disease loci.
- check_combined_results_tsv_for_genome_wide_repeats - Filters combined results for potential pathogenic expansions genome-wide.
- run_reviewer - Takes ExpansionHunter output files for a single sample and runs REViewer on the subset of loci where the genotypes exceed locus-specific thresholds specified in the variant catalog.
- combine_single_sample_LPS_to_allele_histograms - Takes one or more tables of LPS (longest pure segment) scores and combines them into per-locus allele histograms. Useful for aggregating genotype data across large cohorts.
- convert_multisample_LPS_table_to_allele_frequency_histograms - Takes a wide-format TSV of multisample LPS genotypes (one column per sample, allele sizes as comma-separated pairs) and outputs a per-locus summary table with allele frequency histograms and statistics (min, max, mean, median, mode, stdev, percentiles).
- annotate_EHdn_locus_outliers - Takes an ExpansionHunterDenovo outlier result table (locus outliers or case-control) as well as a BED file containing all simple repeats in the reference genome and, optionally, a gene models GTF file, a variant catalog of known disease-associated loci, and/or other BED files with genomic regions of interest. Outputs a new table where each EHdn outlier is annotated with multiple columns related to the provided reference data.
- convert_annotated_EHdn_locus_outliers_to_expansion_hunter_catalog - Takes the output table from annotate_EHdn_locus_outliers and lets the user apply a range of filters before writing out the passing loci to an ExpansionHunter variant catalog.
- convert_expansion_hunter_denovo_locus_tsv_to_bed - Converts ExpansionHunterDenovo locus TSV results to BED format.
- make_bamlet - Optimized version of ExpansionHunterDenovo's make-bamlet.py. For a given STR region, extracts all relevant reads from a BAM or CRAM file into a much smaller BAMlet which can be used as input to ExpansionHunter instead of the full BAM/CRAM but still yields the same genotype. Reduces I/O operations for better performance.
- make_minicram_for_expansion_hunter - Extracts the minimal CRAM subset needed for ExpansionHunter genotyping.
- print_reads - Extracts reads from CRAM/BAM files overlapping genomic intervals (lightweight alternative to GATK PrintReads).
- simulate_str_expansions - Uses wgsim to generate BAM files with simulated read data containing STR expansions at a given locus, with a specified number of repeats, motif, zygosity, etc.
- generate_gnomad_v2_json - Generates gnomAD-formatted tables and JSON files with readviz metadata. Used to combine the gnomAD STR calls into the files available for download on the gnomAD website. This is the current version of generate_gnomad_json.
- generate_gnomad_json - Original version of generate_gnomad_v2_json (legacy, no longer updated).
- convert_bed_to_expansion_hunter_catalog - Converts BED files to ExpansionHunter variant catalog JSON format.
- convert_expansion_hunter_catalog_to_bed - Converts ExpansionHunter variant catalogs to BED format.
- convert_dat_to_bed - Converts Tandem Repeats Finder (TRF)
.datoutput files to BED format. - convert_expansion_hunter_catalog_to_gangstr_spec - Converts ExpansionHunter catalogs to GangSTR spec format.
- convert_gangstr_spec_to_expansion_hunter_catalog - Converts GangSTR spec to ExpansionHunter variant catalog.
- convert_gangstr_vcf_to_expansion_hunter_json - Converts GangSTR VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_hipstr_format - Converts ExpansionHunter catalogs to HipSTR format.
- convert_hipstr_vcf_to_expansion_hunter_json - Converts HipSTR VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_trgt_catalog - Converts ExpansionHunter catalogs to TRGT catalog format.
- convert_trgt_catalog_to_expansion_hunter_catalog - Converts TRGT repeat catalog BED files to ExpansionHunter variant catalog JSON format.
- convert_trgt_vcf_to_expansion_hunter_json - Converts TRGT VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_longtr_format - Converts ExpansionHunter catalogs to LongTR format.
- convert_expansion_hunter_catalog_to_vamos_catalog - Converts ExpansionHunter catalogs to vamos catalog format.
- convert_straglr_bed_to_expansion_hunter_json - Converts Straglr BED output to ExpansionHunter JSON format.
- convert_strling_calls_to_expansion_hunter_json - Converts STRling calls to ExpansionHunter JSON format.