Chapter 22 – Evaluation of predictive genetic tests for common diseases: bridging epidemiological, clinical, and public health measures

A. Cecile J. W. Janssens, Marta Gwinn, and Muin J. Khoury

Introduction

Common diseases such as type 2 diabetes, osteoporosis, and cardiovascular disease are caused by a complex interplay of many genetic and nongenetic factors, each of which conveys a minor increase in the risk of disease. Although the genetic contributions to these multifactorial diseases are still poorly understood, enormous progress in the identification of susceptibility genes is expected from the large-scale genome-wide association studies (1,2) and biobank initiatives that have been launched worldwide (3). The vast amount of information issuing from these studies is fueling the search for useful applications of genetic testing to guide prevention and early detection of common diseases with substantial public health impact. One of the greatest expectations is that unraveling the genetic origins of common diseases will lead to individualized medicine, in which prevention and treatment strategies are personalized on the basis of the results of predictive genetic tests. Examples of multifactorial diseases showing promise for predictive genetic testing include type 2 diabetes and age-related macular degeneration (4,5).

Predictive genetic tests can be used to identify persons who have a disease at the time of testing (diagnosis) or who will develop the disease in the future (prediction). Genetic testing is useful when the value it adds to existing efforts to reduce morbidity or mortality (or to efficiency or effectiveness of health care programs) outweighs the additional costs. This evaluation includes not only measures of test performance, but also of health benefits, side effects, financial costs, and psychosocial, ethical, legal, and social implications (6). A brief assessment of the clinical validity and utility of a potential genetic test can help decide whether it merits further, in-depth evaluation.

In this chapter, we first explain how genetic contributions to monogenetic and complex diseases differ, and how these differences affect the predictive value of genetic tests. Then we review some measures for the clinical validity and utility of a single genetic test; we demonstrate that although they are based on the same epidemiological parameters, they provide different information about the usefulness of a genetic test.

Genetic Predisposition to Monogenic and Complex Diseases

Monogenic Diseases

Monogenic diseases such as Huntington disease, familial hypercholesterolemia, cystic fibrosis, and several hereditary forms of cancer are completely or predominantly caused by mutations in a single gene. Predictive testing for these mutations is very informative because disease risks differ substantially between carriers and noncarriers, as shown in Figure 22.1 for Huntington disease, hereditary breast cancer, and nonpolyposis colorectal cancer (7–9). These mutations are typically rare; thus, the risk of disease in carriers is substantially increased, whereas the risk of disease in noncarriers approximates the population average.

Because of the large difference in disease risk between carriers and noncarriers, genetic testing can be useful for targeting preventive or therapeutic interventions to the relatively small group of individuals at increased risk. Examples include intensive surveillance and prophylactic surgery for breast and ovarian cancer and prescription of statins for familial hypercholesterolemia. Genetic testing is also considered valuable in the absence of effective interventions to relieve uncertainty— even when test results are positive—and to prepare for the future.

Multifactorial Diseases

Because multifactorial diseases are caused by complex interactions of many genetic and nongenetic factors, the predictive value of testing for a single genetic variant is limited. The disease risk in carriers of the risk variant is only slightly higher than that in noncarriers. For example, several examples of genetic test results for predicting type 2 diabetes are shown in Figure 22.2. Because risk variants are generally common (>1%), carriers and noncarriers have disease risks that are only slightly higher or lower, respectively, than the population average. The differences in disease risk are small and noncarriers remain at risk.

Because multiple genetic and nongenetic factors each have only a minor role in the etiology of multifactorial diseases, researchers and test developers have turned their attention to genetic prediction of disease based on simultaneous testing for multiple genetic variants. This approach is called genetic profiling. A genetic profile describes the genotypes for all tested variants and predicts disease risk as a function of their combined effects. For example, when single genetic variants are equally associated with disease, predicted risk is simply proportional to the number of risk genotypes in the genetic profile, as illustrated in Figure 22.3. Figure 22.3a shows the expected distribution of the number of risk genotypes when 40 genes are tested simultaneously: all individuals have at least some risk genotypes but none have risk genotypes for all variants tested. Figure 22.3b shows the associated risk when each single risk genotype increases the risk of disease by 50% (odds ratio = 1.5): the greater the number of risk genotypes present, the higher the risk of disease. Genetic profiles associated with very high disease risks are rare. Most people have disease risks that are only slightly higher or lower than the average disease risk in the population.

In Figure 22.3c, we consider a more realistic scenario in which some genetic factors are stronger predictors of disease than others. The odds ratios of the individual genetic variants vary from 1.05 for genotypes that are more common to 2.0 for those that are less common. In this case, a person’s disease risk depends on both the number of risk genotypes carried and on each genotype-specific risk. Genotypes more strongly associated with disease contribute more to a person’s disease risk than do those with weaker associations. The result is a scattering of disease risks rather than clearly distinguishable risk categories. Considering the role of environmental factors, as well as gene–environment interactions, would contribute to further variation in disease risks for individuals with the same genetic profile.

Evaluation of Single Genetic Tests

Clinical validity and clinical utility are key measures for evaluating genetic tests (6). Clinical validity defines the ability of a test to detect or predict disease; clinical utility focuses on health outcomes, both positive and negative, associated with testing. Several measures of clinical validity and clinical utility (Table 22.1) can be calculated from a basic 2×2 table summarizing the numbers of carriers and noncarriers of risk genotypes who will and will not develop the disease (Table 22.2). A 2×2 table simply assumes that the genetic marker has a risk genotype and a referent genotype (assuming a dominant or recessive effect), but three or more genotypes can be considered as well. The table is defined by basic epidemiological parameters: the population disease risk, the genotype frequencies, and the association between genotypes and the risk of disease (Table 22.1).

To illustrate the formulas of Table 22.1, we present examples of genetic testing for monogenic diseases in offspring of patients (Huntington disease) or of mutation carriers (hereditary breast and colorectal cancer) and for multifactorial diseases in the general population (type 2 diabetes) in Table 22.3. Note that examples refer to predictive testing for future disease, but all measures can also be calculated for diagnostic tests that aim to identify persons with or without the disease.

Epidemiological Parameters

The population disease risk is the probability that a member of a defined population will develop disease within a specified period of time (Table 22.1). Genotype frequencies are the population proportions of carriers and noncarriers of the risk genotype. Measures of association, such as the relative risk, risk difference, and odds ratio compare the risk or odds of disease in carriers and noncarriers of the risk genotype. Relative risk is the ratio of the disease risk in carriers divided by the disease risk in the noncarriers; risk difference is the absolute difference between the disease risk of carriers and noncarriers; and odds ratio is the ratio of the odds of disease in carriers divided by the odds of disease in noncarriers. Odds ratio is also the ratio of the odds of the risk genotype in individuals who will develop the disease and those who will not (referred to as affected and unaffected individuals in Table 22.1). In contrast to the relative risk and risk difference, the odds ratio is the same whether one looks from the genotype or from the disease perspective, that is, horizontally or vertically in Table 22.2 (16). Odds ratios may be used to approximate relative risks in rare diseases; however, odds ratios overestimate relative risks in common diseases.

In monogenic diseases such as Huntington disease, where disease develops in all carriers but in no noncarriers, the genotype frequency is equal to the disease risk in the population (17). In this situation, the disease risk is 100% for carriers and 0% for noncarriers (Figure 22.1 ; Table 22.3). In contrast, carriers of genetic variants associated with risk for complex diseases have risks that are only slightly higher than the risks in noncarriers (Figure 22.2).

Clinical Validity

Clinical validity measures the ability of genetic markers to detect or predict disease. Clinical validity comprises both the discriminative accuracy of the test and the predictive value of the test results. The discriminative accuracy of a genetic marker is the extent to which the marker can discriminate between individuals who will develop the disease and those who will not. Key indicators of discriminative accuracy are sensitivity and specificity. Sensitivity is the proportion of carriers among persons who will develop the disease. Specificity is the proportion of noncarriers among persons who will not develop the disease. Sensitivity and specificity are measures of the genetic marker’s ability to correctly classify persons according to their future disease status. Sensitivity, and specificity are also known as the true positive rate and the true negative rate. Conversely, the false positive rate is equal to one minus the specificity, and the false negative rate is equal to one minus the sensitivity.

The predictive value of a genetic marker is its ability to predict disease. Positive predictive value is the absolute risk of disease in carriers, and negative predictive value is the probability that noncarriers will not develop the disease. Positive predictive value is related to the genetic epidemiological concept of penetrance.

Clinical Utility

Clinical utility is defined in terms of the extent to which genetic testing improves disease prediction beyond conventional risk factors, improves population health outcomes, and improves health care services by increasing the efficiency of interventions. A comprehensive assessment of clinical utility further requires data on social, economic, and behavioral factors as well as knowledge of test performance and disease risks. Genetic testing is useful when it sufficiently changes the distribution of risks predicted before testing. When a genetic marker is associated with risk of disease, carriers of the risk genotype have a higher risk of disease and noncarriers a lower risk of disease compared to the average or pretest disease risk. The likelihood ratio is the magnitude of change from the pretest to the posttest disease risk. The likelihood ratio of a certain genotype differs from its odds ratio in that the odds ratio compares the odds of disease to a referent genotype, whereas the likelihood ratio compares to the pretest odds of disease. A likelihood ratio higher than 1.0 indicates that the genotype is associated with increased risk of disease, and a likelihood ratio lower than 1.0 with a decreased disease risk compared to the risk of disease before testing. When the likelihood ratio is approximately 1.0 (see Table 22.3 for the risk genotypes of PPARG and CAPN10 and the referent genotype of TCF7L2), the penetrance approaches the pretest risk of disease. When the likelihood ratios of all genotypes approximate 1.0, their odds ratios approximate 1.0 and the test is uninformative.

Population-attributable fraction is an epidemiologic parameter that aims to assess the potential of a genetic test to improve population health outcomes. The population-attributable fraction is the proportion of cases that can be prevented when a particular risk factor is eliminated. Population-attributable fraction increases with higher frequency of the risk genotype and with stronger association of the risk genotype with disease risk. Common interpretations of the population-attributable fraction (Table 22.1) are based on assumptions that the risk factor can be eliminated and that there is no confounding by other risk factors. For risk factors that cannot be eliminated, such as genetic risk factors, population-attributable fraction is the proportion of cases that could be prevented by a preventive intervention that is 100% effective and is adopted by all carriers. Thus, population-attributable fraction can be interpreted as the maximum number of cases that could be prevented by eliminating the adverse effects of the genetic risk factor.

While population-attributable fraction indicates the proportion of cases that can be prevented, the efficiency of interventions to achieve this reduction is indicated by the number needed to treat and the number needed to screen. The number needed to treat is the number of at-risk persons who would need to adopt the preventive intervention to prevent one case. The number needed to screen is the number of persons who would have to be tested to find a sufficient number of persons needed to treat to prevent one case.

Evaluation of Genetic Profiling

Genetic profiles based on multiple variants can be evaluated for clinical validity and utility by the same measures used to evaluate single genetic tests; however, their calculation is sometimes more complex because testing at multiple loci yields a large number of different profiles (Figure 22.3). Regression modeling can be used to estimate some measures—such as the odds ratio, risk difference, predictive value, likelihood ratio, and population-attributable fraction—for specific profiles. Other measures, such as sensitivity and specificity, can only be calculated for genetic markers with two genotypes but have analogous measures for tests with continuous results. The area under the receiver operating characteristic curve (AUC) is a summary measure of discriminative accuracy for continuous tests that is related to the sensitivity and specificity.

Discussion

Several measures of clinical validity and clinical utility can be calculated when information is available for estimating genotype frequencies, disease risk in the population, and the association of genotypes with disease risk. We demonstrate that these different measures, though calculated from the same three epidemiological parameters, provide different and complementary information about the clinical validity and utility of a genetic test.

The measures of clinical validity and clinical utility that we have discussed are related; all of them can be calculated from the same 2×2 table, and hence each can be calculated from the others. For example, the likelihood ratio of the risk genotype is the odds of disease in genotype carriers divided by the odds of disease in the total population; it is also equivalent to the true positive rate divided by the false positive rate, or the sensitivity divided by (1-specificity). Likewise, the odds ratio is the likelihood ratio of the risk genotype divided by the likelihood ratio of the referent genotype. Although these indicators can be calculated from one another, they have different interpretations. For example, a genetic test with appreciable clinical validity may have a low population-attributable fraction when the frequency of the risk genotype is low. Furthermore, a test with a substantial population-attributable fraction may have poor clinical validity when the disease is very common and the odds ratio is low (e.g., as for PPARG in type 2 diabetes risk; see Table 22.3).

In this chapter, we present several key points relevant to the evaluation of genetic testing. Most important, we demonstrate that clinical validity and utility vary with differences in the same three epidemiological parameters, whether testing single or multiple genetic variants. Thus, a test that is useful for predicting disease in one population may not be useful in another population, for example, where the risk of disease is lower, the frequency of the risk genotype is lower, or the gene–disease association is weaker. Because disease risks, genotype frequencies, and risk ratios may vary among populations, the clinical validity and utility of a genetic test should be evaluated for each disease in every setting in which the test will be applied (6). For example, the frequency of BRCA1/2 mutations and the risk of breast cancer in the general population are very different from the parameters for diseases included in Table 22.3; thus, their clinical validity and utility vary accordingly.

Sensitivity and specificity, as well as positive and negative predictive values, should be evaluated simultaneously in the context of one another. Table 22.3 demonstrates that a genetic marker with a frequent risk genotype by definition has good sensitivity. A test based on a genetic marker that is not associated with a disease will have sensitivity and 1-specificity equal to the frequency of the risk genotype. For example, the minimum sensitivity of the PPARG P12A polymorphism is 73%, irrespective of which disease is tested for and irrespective of whether the marker is associated with the disease. The same holds for the positive predictive value and 1-negative predictive value, which are at least equal to the population risk of disease. This means, for example, that the positive predictive value of any diabetes risk genotype is at least 33%, regardless of the strength of the association (see Table 22.3) (14).

By definition, a “risk genotype” is associated with a higher risk of disease compared with the referent genotypes; however, this does not mean that the posttest disease risk in carriers will be markedly higher than the average population disease risk (i.e., the probability of disease prior to testing). When the risk genotype is very common (>50%, such as the PPARG and CAPN10 genotypes in Table 22.3), the absolute increase in risk among carriers is smaller than the decrease among non-carriers. Advocates for the potential clinical or public health impact of a genetic test often emphasize the proportion of the population that carries the risk genotype; however, a genetic test for a very common risk genotype might be more useful for identifying individuals at low risk of disease.

Deciding which measure of clinical validity or clinical utility is of primary interest is not an arbitrary decision, but instead is determined by the intended use of the test and the perspective of the user. For example, a test that is used for screening to select persons for further diagnostic testing should have high sensitivity and reasonably good specificity, so that it identifies most affected individuals without yielding an excessive number of false positives. Persons who undergo genetic testing want to know their own risks of disease based on genotype (indicated by the penetrance or predictive value) and the extent to which genetic test results change their estimated risk (indicated by the likelihood ratio). Policy makers or health care payers are likely more interested in the number of individuals that need to be screened or treated to achieve a certain reduction in disease risk and morbidity. Because different perspectives rely on different primary indicators, genetic testing can easily be useful from one perspective but not from another (21).

 Top of Page

Figures

• Figure 22-1
• Figure 22-2
• Figure 22-3

Tables

• Table 22-1
• Table 22-2
• Table 22-3

 Top of Page

References

1. Herbert A, et al. A common genetic variant is associated with adult and childhood obesity. Science. 2006;312:279–283.
2. Klein RJ, et al. Complement factor H polymorphism in age-related macular degeneration. Science. 2005;308:385–389.
3. Davey Smith G, et al. Genetic epidemiology and public health: hope, hype, and future prospects. Lancet. 2005;366:1484–1498.
4. Weedon MN, et al. Combining information from common type 2 diabetes risk polymorphisms improves disease prediction. PLoS Med. 2006;3:e374.
5. Maller J, et al. Common variation in three genes, including a noncoding variant in CFH, strongly influences risk of age-related macular degeneration. Nat Genet. 2006;38:1055–1059.
6. Haddow JE, Palomaki GE. ACCE: a model process for evaluating data on emerging genetic tests. In: Khoury MJ, Little J, Burke W, eds. Human Genome Epidemiology: A Scientific Foundation for Using Genetic Information to Improve Health and Prevent Disease. New York: Oxford University Press; 2003:217–233.
7. Kremer B, et al. A worldwide study of the Huntington’s disease mutation. The sensitivity and specificity of measuring CAG repeats. N Engl J Med. 1994;330:1401–1406.
8. Ford D, et al. Genetic heterogeneity and penetrance analysis of the BRCA1 and BRCA2 genes in breast cancer families. The Breast Cancer Linkage Consortium. Am J Hum Genet. 1998;62:676–689.
9. Vasen HF, et al. MSH2 mutation carriers are at higher risk of cancer than MLH1 mutation carriers: a study of hereditary nonpolyposis colorectal cancer families. J Clin Oncol. 2001;19:4074–4080.
10. Janssens ACJW, Khoury MJ. Predictive value of testing for multiple genetic variants in multifactorial diseases: implications for the discourse on ethical, legal and social issues. Ital J Public Health. 2006;3:35–41.
11. Zeggini E, et al. Examining the relationships between the Pro12Ala variant in PPARG and Type 2 diabetes-related traits in UK samples. Diabet Med. 2005;22:1696–1700.
12. McCarthy MI. Progress in defining the molecular basis of type 2 diabetes mellitus through susceptibility-gene identification. Hum Mol Genet. 2004;13 Spec No 1:R33–R41.
13. Grant SF, et al. Variant of transcription factor 7-like 2 (TCF7L2) gene confers risk of type 2 diabetes. Nat Genet. 2006;38:320–323.
14. Narayan KM, Boyle JP, Thompson TJ, Sorensen SW, Williamson DF. Lifetime risk for diabetes mellitus in the United States. JAMA. 2003;290:1884–1890.
15. Janssens ACJW, et al. Predictive testing for complex diseases using multiple genes: fact or fiction? Genet Med. 2006;8:395–400.
16. Bland JM, Altman DG. Statistics notes. The odds ratio. BMJ. 2000;320:1468.
17. Khoury MJ, Newill CA, Chase GA. Epidemiologic evaluation of screening for risk factors: application to genetic screening. Am J Public Health. 1985;75:1204–1208.
18. Lyssenko V, et al. Genetic prediction of future type 2 Diabetes. PLOS Med. 2005;2:e345.
19. Antoniou A, et al. Average risks of breast and ovarian cancer associated with BRCA1 or BRCA2 mutations detected in case Series unselected for family history: a combined analysis of 22 studies. Am J Hum Genet. 2003;72:1117–1130.
20. Ries LAG, Harkins D, Krapcho M, et al. SEER Cancer Statistics Review, 1975–2003. Bethesda, MD: National Cancer Institute; 2006.
21. Grosse SD, Khoury MJ. What is the clinical utility of genetic testing? Genet Med. 2006;8:448–450.

 Top of Page