We often think of Next-generation sequencing (NGS) as a powerful technology that can analyse millions of DNA variants simultaneously, making it widely used for genetic variant detection. However, whilst NGS offers significant advantages, it also faces challenges related to the rate at which it can positively detect genetic mutations and rule out their presence, which we quantify using sensitivity and specificity.
Whole exome sequencing (WES), which focuses on sequencing all protein-coding exons, is becoming a standard diagnostic tool for various medical conditions, particularly in cancer mutation detection. However, there is a growing demand for established standards and guidelines, as studies suggest that whole genome sequencing (WGS) may be more effective and cost-efficient for diagnosing several disorders.
In this month’s suggested read[1], we will explore how a method was established to evaluate the quality of NGS results, revealing significant variations in sensitivity and specificity amongst NGS-certified service providers. The main course of action was to prepare DNA samples (named DNA1 and DNA2) which contained only homozygous alleles, that were then sequenced using three providers, AA, BB and CC. The three tests’ respective sensitivities and specificities were computed using each provider’s in-house proprietary bioinformatics technology, usually based on sequence alignment and variant calling, as well as using the DRAGEN system, which consists of a bioinformatic pipeline used for processing of NGS data.
Using the in-house analysis of WES results, the study found significant differences in sensitivity amongst the three companies. Company AA demonstrated a sensitivity of 4.82% for detecting diluted variants at the lowest variant allelic fraction (VAF), whilst companies BB and CC had sensitivities of 19.78% and 19.75%, respectively.
The DRAGEN analysis brought different results, with a sensitivity of around 10.64% for company A, compared to 17.48% and 14.93% for companies B and C, respectively. The DRAGEN system identified significantly more variants than the in-house methods, suggesting that DRAGEN enhances both the detection rate and the total number of identified variants. Further analyses, which involved increasing the cutoff thresholds for classification of true positive rates, highlighted that the low sensitivity in the in-house results was primarily due to high VAF cutoff values used in their bioinformatics analysis, rather than the capture procedure itself. Additionally, since the difference in sensitivity between AA and BB/CC is still present using DRAGEN analysis, this suggests that other factors may contribute to differences in sensitivity.
The analysis compared the sensitivity of genetic variants across DRAGEN and in-house methods, categorizing variants into common (C variants), DRAGEN-specific (Dra-Spe), and in-house-specific variants (in-house-Spe). The results showed that C variants constituted the largest group, with DRAGEN identifying the majority of variants detected by in-house methods (up to 98.8%). However, the number of in-house-specific variants was limited, with totals of 155, 250, and 66 for companies AA, BB, and CC, respectively.
All in all, the study successfully highlighted significant variability in sensitivity (approximately four-fold) and specificity amongst the WES results from three certified service providers. The use of the DRAGEN system for bioinformatics analysis improved sensitivity and reduced false positive errors, identifying 1.3 to 1.7 times more variants than in-house methods. Although the sensitivity of company AA decreased with DRAGEN analysis, it still performed comparably to companies BB and CC, which saw increased sensitivity.
The differences in sensitivity were largely due to varying bioinformatics conditions rather than the target capture methods. For instance, company AA’s use of the Twist target enrichment system was associated with higher sensitivity and a greater number of detected variants. However, efforts to reduce FP errors could negatively impact sensitivity, particularly when strict cutoff values for variant allele frequency were applied. Despite the progress made in bioinformatics and target enrichment technologies, such as DRAGEN and Twist, there are still inconsistencies that may affect clinical applications. This highlights the critical need for strong quality control protocols and standardised reference materials to guarantee precise and reliable WES results.
We at 4bases present the WholEX pro kit which allows WES with a coverage of over 99% of protein-coding genes, in order to identify various pathologies.
[1] Moon, Y., Hong, C. H., Kim, Y., Kim, J., Ye, S., Kang, E., Choi, H. W., Cho, H., Choi, H., Lee, D., Choi, Y., Kim, T., Heo, S. G., Han, N., & Hong, K. (2024). Enhancing clinical applications by evaluation of sensitivity and specificity in whole exome sequencing. International Journal of Molecular Sciences, 25(24), 13250. https://doi.org/10.3390/ijms252413250
