Is 58% sensitivity for detection of cervical intraepithelial neoplasia 3 and invasive cervical cancer optimal for cervical screening?
R. Marshall Austin, Chengquan Zhao
Department of Pathology, Magee Womens Hospital of University of Pittsburgh Medical Center, PA, USA
|Date of Submission||10-Apr-2014|
|Date of Acceptance||15-Apr-2014|
|Date of Web Publication||22-May-2014|
R. Marshall Austin
Department of Pathology, Magee Womens Hospital of University of Pittsburgh Medical Center, PA
Source of Support: None, Conflict of Interest: None
Recent Food and Drug Administration (FDA) approval of a Roche cobas human papillomavirus (HPV) test application as a first line primary cervical screening tool in women 25 and older introduces a new era of complex cervical screening choices. Perhaps the most surprising findings in Roche's supporting ATHENA trial data were the unexpectedly low verification bias-adjusted CIN3+ sensitivities documented by the FDA for both the proposed cobas HPV testing algorithm (58.26%) and Pap testing algorithm (42.63%). These unexpectedly low sensitivity estimates suggest intuitively that there is still considerable room for improvement in cervical screening, and available data from large systems point to routine cytology and HPV co-testing as offering the greatest protection against development of cervical cancer. Observational studies of large populations screened over time remain essential to document actual protection from development of cervical cancer with any new cervical screening options, as natural history studies and available data from large systems indicate that most CIN2/3 cases detected in short term clinical trials would not progress to invasive cervical cancer. Interpretation of ATHENA trial data and its application to routine clinical practice is further limited by published studies which document that a significant proportion of CIN2/3 biopsy diagnoses in the ATHENA trial could not be confirmed as accurate when evaluated with p16 immunohistochemistry and that cytology laboratory performance in the trial was notably suboptimal.
Keywords: Cervical cancer, human papillomavirus test, Pap test, screening
|How to cite this article:|
Austin RM, Zhao C. Is 58% sensitivity for detection of cervical intraepithelial neoplasia 3 and invasive cervical cancer optimal for cervical screening?. CytoJournal 2014;11:14
|How to cite this URL:|
Austin RM, Zhao C. Is 58% sensitivity for detection of cervical intraepithelial neoplasia 3 and invasive cervical cancer optimal for cervical screening?. CytoJournal [serial online] 2014 [cited 2017 Apr 26];11:14. Available from: http://www.cytojournal.com/text.asp?2014/11/1/14/132997
The recent recommendation by a Food and Drug Administration (FDA) panel of the Medical Devices Advisory Committee for approval of a Roche Cobas HPV test application as a first line primary cervical screening tool in women 25 and older has been the subject of considerable medical news coverage. ,,, Following approval by the FDA on April 24 of the committee recommendation, clinicians in the US face a new and more complex set of options for FDA-approved routine periodic cervical screening. Available choices will include: (1) cytology alone for women 21 and older; (2) cytology with adjunctive reflex HPV testing after atypical squamous cells of undetermined significance (ASC-US) cytology results; (3) cytology with routine adjunctive HPV co-testing of women 30 and older; and (4) Roche's new algorithm for primary HPV screening of women 25 and older. According to the New York Times, Johns Hopkins University's Dorothy Rosenthal testified to the Committee that "there would be a tremendous gain by moving to the new test".  More cautiously, Dr. Alan G. Waxman, a panel member and professor of Obstetrics and Gynecology at the University of New Mexico commented-"I think women are going to be well served by having more choices, but it's going to be very interesting to watch over the next several years as this rolls out".  What exactly did the data presented to the FDA document regarding this new option for cervical screening?
The FDA panel recommendation was based on review of data from the addressing the need for advanced HPV diagnostics (ATHENA) trial, Roche's US - based cohort prospective trial which enrolled and screened more than 47,000 women for cervical disease and analyzed the impact of proposed screening algorithms based on these data.  A few days before the March 12, 2014 committee panel meeting, an FDA summary and analysis of the trial data was posted on the FDA website.  Highlights of the summary included: (1) The sensitivity for ≥ cervical intraepithelial neoplasia three (CIN3) was 58.26% for the cobas HPV test algorithm compared to 42.63% for the Pap test algorithm; (2) the risk of ≥ CIN3 (positive predictive value) in women referred to colposcopy by the cobas HPV test algorithm was 12.25% compared to 6.47% for the women referred by the Pap test algorithm; (3) the risk of ≥ CIN3 in women who were not referred to colposcopy by the cobas HPV test algorithm (negative predict value) was 0.42% and 0.59% among the women not referred by the Pap test algorithm; (4) the false-positive rate (specificity) for ≥ CIN3 was 4.09% for the cobas HPV test algorithm compared with 6.04% for the Pap test algorithm.
Perhaps, the most surprising findings of the study were the unexpectedly low verification bias-adjusted CIN3+ sensitivities for both the cobas HPV test (58.26%) and Pap testing (42.63%) algorithms in the trial. Recent modeling studies seeking to estimate the impact of primary HPV testing on incidence and morbidity of cervical cancer in have, for example, assumed that HPV testing can achieve a much higher sensitivity of 95% when cytology is negative and 97% when cytology is positive.  Other studies have reported verification bias-adjusted sensitivity for detection of CIN2/3+ with optimized liquid-based cytology (LBC) exceeding 90%. ,, Why then were the sensitivities of both cobas HPV test and cytology so low for detection of CIN3+ in the ATHENA trial? Furthermore, what are the implications of these findings for application of the new proposed primary HPV testing algorithm in clinical practice?
The ATHENA trial study design acknowledged that crude estimates of cervical disease prevalence would result in verification bias, as all women with positive Pap tests or HPV results in the ATHENA trial were selected to undergo colposcopy, whereas only a much smaller subset of women with both negative Pap and HPV test results were randomly selected to undergo colposcopy.  Verification bias adjustment was applied to account for the difference in rates of selection to colposcopy by calculating the likely number of CIN2/3+ cases that would have been found if all women had undergone colposcopy for disease verification. Overall there were 274 cases of CIN3+ and 157 cases of CIN2 identified in the trial, of which nine cases of CIN3 and 20 cases of CIN2 were documented in women referred to colposcopy after "double negative results" (DNR) for both HPV and Pap testing.  Extrapolating from the nine cases of CIN3 found in colposcoped and biopsied doubly negative women, there were another estimated 160 (roughly) CIN3 cases in the trial that were not detected; these additional estimated CIN3 cases caused the CIN3+ sensitivities of both the HPV and Pap testing algorithms to decrease from crude unadjusted estimates. 
The FDA summary specifically cautions that the results of the trial cannot be readily compared to the results of most randomized clinical trial studies conducted with nonadjustable verification bias. Nonetheless, the verification-bias adjusted sensitivities calculated in the ATHENA trial turned out to be remarkably similar to estimates of "near 50%" for verification bias-adjusted sensitivity with the conventional Pap smear made by the Agency for Healthcare Policy and research around the time new cervical screening technologies began to be introduced in the late 1990s. 
Very low verification bias-adjusted CIN3+ cytology sensitivity of 42% was also calculated for laboratories utilizing LBC in the ATHENA trial.  The 7% average abnormal rate documented overall for the four US laboratories  place them as a group in the lowest 25 th percentile of US laboratories, based on College of American Pathologists (CAP) benchmarking data.  Furthermore, data on the ASC/squamous intraepithelial lesion ratios of these laboratories place them in a laboratory performance category linked to suboptimal screening sensitivity.  Performance of these four US cytology laboratories  appears particularly suboptimal when compared to the performance of United Kingdom (UK) laboratories competing against hybrid capture two HPV testing in the first two rounds of the ARTISTIC trial.  The performance of the UK LBC laboratories was so strong that the UK investigators commented: "It is difficult to escape the conclusion that LBC was more sensitive in ARTISTIC than earlier conventional cytology".  In contrast, the low sensitivity of LBC in the ATHENA trial can best be attributed to suboptimal cytology screening performance within the selected four large laboratories.No data on cytotechnologist daily workload, a well-known factor potentially impairing cytology sensitivity, is available in the ATHENA trial publications or FDA summary.  Suboptimal cytology screening would impact not only the cytology arm of the ATHENA trial but also the cytology triage component of the HPV arm.
Another factor possibly impacting the ATHENA trial data is misclassification of CIN2/3 biopsy interpretations. , Cervical biopsy interpretations in the ATHENA trial were performed on hematoxylin and eosin, stained histopathology slides by three pathologists working with the manufacturer. p16 INK4A (p16) is a cyclin-dependent kinase inhibitor involved in regulation of cell proliferation,  is overexpressed in high-grade cervical neoplasia, and is readily detectible with immunohistochemical staining, potentially improving the reliability of CIN interpretations. , When p16 stains were performed on a subset of eight biopsies of ATHENA trial patients with CIN2/3 diagnoses and negative cobas HPV test results, the pathologists concluded that five of the eight CIN2/3 diagnoses (62.5%) were misclassifications.  According to one of the pathologist-investigators answering queries from the FDA panel, p16 staining was only performed on cases that were cobas negative; however, after further panel questioning the same investigator acknowledged for the first time "other studies" with "wider p16 immunostaining on cases with adjudicated diagnoses of CIN2 and CIN3"  No p16 data is disclosed in extensive trial-related materials posted on-line by the FDA before the panel meeting. Misclassification in a subset of ATHENA trial CIN2/3 biopsies raises troubling questions about the extent of misclassification of all the CIN2/3 biopsies, both in women with positive and negative cobas HPV test results.  P16 evaluation of all CIN2/3 biopsies following discordant Pap and HPV results or following DNR would be of particular interest.
Since verification bias-adjustment is especially sensitive to disease verification and extrapolation from the very limited number of CIN2/3 cases detected after DNR, it is plausible that additional CIN2/3 biopsy misclassifications could substantially alter the verification bias-adjusted sensitivities for each of the arms of the trial. Furthermore, since roughly 12% of ATHENA trial patients tested cobas HPV-positive, while only 7% had abnormal (ASC-US+) cytology results,  any trend toward overdiagnoses of CIN2/3 may have disproportionately contributed to increased CIN2/3 diagnoses in the HPV arm of the trial. Only p16 imunohistochemical staining of additional CIN2/3+ biopsies in the ATHENA trial and independent adjudication of the results can clarify these uncertainties.
Although, FDA panel approval of the cobas HPV test application focused mainly on comparing trial results projected from using the cobas primary HPV testing algorithm with results projected from using cytology alone, the manufacturer also compared its proposed primary HPV testing algorithm with other trial results projected with the use of routine ASC-US triage for women ≥25 and co-testing for women 30 and older.  According to a brief summary of the microbiology devices panel meeting, the manufacturer asserts that their proposed cobas HPV primary screening algorithm "provides similar protection against CIN3 and invasive cervical cancer as cytology and HPV co-testing".  This is highly questionable, however, as noted by the United States preventive services task force - "the degree of benefit in preventing invasive cervical cancer cannot be determined from test performance studies alone. Such studies suffer from determining sensitivity, specificity, and related values for a surrogate outcome (CIN2/3+) and not invasive cervical cancer".  In available published US studies utilizing conventional Pap smear and hybrid Capture two HPV co-testing at a 3 year screening interval after DNR, projected cervical cancer rates over a 5 year period were 10-15% lower with co-testing than estimated using HPV testing alone.  At the FDA hearing, data were also presented to the panel from Quest Diagnostics' internal clinical database with results from 3,727,894 women ages 30-65 who had co-testing in 2005-2007. Based on Quest data and American Cancer Society estimates of 12,360 women expected to be diagnosed with cervical cancer this year, quest estimated approximately 1,670 women (13.5%) would be undiagnosed with primary HPV screening alone.  Co-testing is the current preferred cervical screening strategy for women 30 and older in consensus guidelines from the American Cancer Society, the American Society for Colposcopy and Cervical Pathology, and the American Society for Clinical Pathology. 
Food and Drug Administration approval of the cobas HPV test application as a first line primary cervical screening tool in women 25 and older will introduce a fourth FDA-approved screening option and an unprecedented level of complexity in cervical screening choices. Available data indicates that the greatest protection against cervical cancer is achievable with cytology and HPV co-testing at 3 year intervals. , The verification bias-adjusted CIN3+ sensitivity of 58.26% documented by the FDA with the new cobas HPV test screening algorithm is not so different from the "near 50%" sensitivity estimated by the Agency for Health Care Policy and Research for conventional Pap smear screening in 1999. 
Due to this low documented CIN3+ sensitivity for Roche's new primary cobas HPV testing algorithm, we believe most clinicians and patients will continue to choose cytology and HPV co-testing as their preferred option. Choice of co-testing is also supported by available studies documenting HPV false-negative rates around 10% for women tested with FDA-approved HPV tests with established cervical cancer diagnoses. ,,,
Furthermore, reported false-negative HPV rates have increased to 31% for baseline HPV testing of women developing cervical cancer over the next 5 years  and to 42% for baseline HPV testing of women developing cervical cancer over the next 8 years.  Optimized LBC, as utilized in the UK ARTISTIC trial, appears to increase protection against cervical cancer when compared with conventional smear trial data.  In contrast, suboptimal LBC, as documented in the ATHENA trial, clearly led to less favorable results. Choosing quality cytology laboratory services carefully remains as important as ever, even in this new era in which HPV testing is playing an increasing role in cervical screening.
Newer technologies such as HPV testing and p16 are significant advances in cervical screening and diagnosis. Nevertheless, disturbing the time tested the role of periodic cervical cytology, which has dramatically reduced cervical cancer should be taken very seriously. In addition to an immediate risk of confusion at the clinical practice level, any approach relying on newer methodology without data on the long-term impact on cervical cancer in screened populations could lead to results that are less favorable than anticipated. 
| » Competing interests statement by all authors|| |
Both authors declare that they have no competing interests.
| » Authorship statement by all authors|| |
Each author has participated sufficiently in the work and takes public responsibility for appropriate portions of the content of this article.
| » References|| |
|1.||Pollack A. FDA Panel Recommends Replacement for the Pap Test. New York Times. Available from: http://www.nytimes.com/2014/03/13/health/an-fda-panel-recommends-a-possible-replacement-for-the-pap-test.html?_r=1. [Last accessed on 2014 Mar 12]. |
|2.||Brown T. FDA Panel: HPV DNA Test before Pap for Cervical Cancer Screening. Medcape Pathology and Lab Medicine; March 13, 2014. |
|3.||Willingham V. FDA: Use HPV test over pap tests to find cervical cancer. CNN health. Available from: http://www.cnn.com/2014/03/13/health/pap-test-fda/index.html?hpt=hp_bn13. [Last accessed on 2014 Mar 13]. |
|4.||FDA panel approves Roche cobas HPV test as first line cervical cancer screen. The PATHOLOGY BLAWG. Available from: http://www.pathologyblawg.com/pathology-news/fda-panel-approves- roche-cobas-hpv-test-first-line-cervical-cancer-screen/. [Last accessed on 2014 Mar 12]. |
|5.||Wright TC Jr, Stoler MH, Behrens CM, Apple R, Derion T, Wright TL. The ATHENA human papillomavirus study: Design, methods, and baseline results. Am J Obstet Gynecol 2012;206:46.e1-4611. |
|6.||Available from: http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/MedicalDevices/MedicalDevicesAdvisoryCommittee/MicrobiologyDevicesPanel/UCM388564.pdf. [Last accessed on 2014 May 12]. |
|7.||Castanon A, Landy R, Sasieni P. How much could primary human papillomavirus testing reduce cervical cancer incidence and morbidity? J Med Screen 2013;20:99-103. |
|8.||Belinson J, Qiao YL, Pretorius R, Zhang WH, Elson P, Li L, et al. Shanxi Province Cervical Cancer Screening Study: A cross-sectional comparative trial of multiple techniques to detect cervical neoplasia. Gynecol Oncol 2001;83:439-44. |
|9.||Zhao C, Florea A, Onisko A, Austin RM. Histologic follow-up results in 662 patients with Pap test findings of atypical glandular cells: Results from a large academic womens hospital laboratory employing sensitive screening methods. Gynecol Oncol 2009;114:383-9. |
|10.||Dudding N, Crossley J. Sensitivity and specificity of HPV testing: What are the facts? Cytopathology 2013;24:283-8. |
|11.||Castle PE, Stoler MH, Wright TC Jr, Sharma A, Wright TL, Behrens CM. Performance of carcinogenic human papillomavirus (HPV) testing and HPV16 or HPV18 genotyping for cervical cancer screening of women aged 25 years and older: A subanalysis of the ATHENA study. Lancet Oncol 2011;12:880-90. |
|12.||McCrory DB, Matchar DB, Bastian L, Datta S, Hasselblad V, Hickey J. et al. Evaluation of Cervical Cytology. Evidence Report Technology Assessment No. 5. (Prepared by Duke University under Contract No. 290-97-0014.) AHCPR Publication N0. 99-E010. Rockville, MD: Agency for Health Care Policy and Research; 1999. |
|13.||Eversole GM, Moriarty AT, Schwartz MR, Clayton AC, Souers R, Fatheree LA, et al. Practices of participants in the College of American Pathologists interlaboratory comparison program in cervicovaginal cytology, 2006. Arch Pathol Lab Med 2010;134:331-5. |
|14.||Renshaw AA, Deschênes M, Auger M. ASC/SIL Ratio for Cytotechnologists: A surrogate marker of screening sensitivity. Am J Clin Pathol 2009;131:776-81. |
|15.||Wright TC Jr, Stoler MH, Behrens CM, Sharma A, Sharma K, Apple R. Interlaboratory variation in the performance of liquid-based cytology: Insights from the ATHENA trial. Int J Cancer 2014;134:1835-43. |
|16.||Kitchener HC, Almonte M, Thomson C, Wheeler P, Sargent A, Stoykova B, et al. HPV testing in combination with liquid-based cytology in primary cervical screening (ARTISTIC): A randomised controlled trial. Lancet Oncol 2009;10:672-82. |
|17.||Kitchener HC, Almonte M, Gilham C, Dowie R, Stoykova B, Sargent A, et al. ARTISTIC: A randomised trial of human papillomavirus (HPV) testing in primary cervical screening. Health Technol Assess 2009;13:1-150, iii-iv. |
|18.||Renshaw AA, Elsheikh TM. Predicting screening sensitivity from workload in gynecologic cytology: A review. Diagn Cytopathol 2011;39:832-6. |
|19.||Stoler MH, Wright TC Jr, Sharma A, Zhang G, Apple R, Wright TL, et al. The interplay of age stratification and HPV testing on the predictive value of ASC-US cytology. Results from the ATHENA HPV study. Am J Clin Pathol 2012;137:295-303. |
|20.||Austin RM. ATHENA trial CIN 2+cervical biopsy misclassifications raise questions. Am J Clin Pathol 2012;137:1012. |
|21.||von Knebel Doeberitz M. New markers for cervical dysplasia to visualise the genomic chaos created by aberrant oncogenic papillomavirus infections. Eur J Cancer 2002;38:2229-42. |
|22.||Galgano MT, Castle PE, Atkins KA, Brix WK, Nassau SR, Stoler MH. Using biomarkers as objective standards in the diagnosis of cervical biopsies. Am J Surg Pathol 2010;34:1077-87. |
|23.||Bergeron C, Ordi J, Schmidt D, Trunk MJ, Keller T, Ridder R, et al. Conjunctive p16INK4a testing significantly increases accuracy in diagnosing high-grade cervical intraepithelial neoplasia. Am J Clin Pathol 2010;133:395-406. |
|24.||Transcript of March 12, 2014 Microbiology Devices Panel Meeting; http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/MedicalDevices/MedicalDevicesAdvisoryCommittee/MicrobiologyDevicesPanel/UCM393482.pdf. |
|25.||Department of Health and Human Services. Public Health Service. Brief Summary of the Microbiology Devices Panel Meeting-March 12, 2014. Available from: http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/MedicalDevices/MedicalDevicesAdvisoryCommittee/MicrobiologyDevicesPanel/UCM389192.pdf. [Last accessed on 2014 May 12]. |
|26.||Kulasingam SL, Havrilesky L, Ghebre R, Myers ER. Screening for Cervical Cancer: A Decision Analysis for the U.S. Preventive Services Task Force. AHRQ Publication No. 11-05157-EF-1. Rockville, MD: Agency for Healthcare Research and Quality; 2011. |
|27.||Katki HA, Kinney WK, Fetterman B, Lorey T, Poitras NE, Cheung L, et al. Cervical cancer risk for women undergoing concurrent testing for human papillomavirus and cervical cytology: A population-based study in routine clinical practice. Lancet Oncol 2011;12:663-72. |
|28.||Quest diagnostics weighs in on HPV test as first-line cervical cancer screen. The PATHOLOGY BLAWG. Available from: http://www.pathologyblawg.com/pathology-news/pathology-vendors/quest-diagnostics/quest-diagnostics-weighs-in-hpv-test-first-line-cervical-cancer-screen/. [Last accessed on 2014 Mar 12]. |
|29.||Saslow D, Solomon D, Lawson HW, Killackey M, Kulasingam SL, Cain J, et al. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. Am J Clin Pathol 2012;137:516-42. |
|30.||Kinney W, Poitras N, Fetterman B, Lorey T. Magnitude of increased lifetime risk of cervical cancer and death from cervical cancer with new screening recommendations (Poster 93) Society of Gynecologic Oncology 45 th Annual Meeting on Women′s Cancer, Tampa Florida, March 22-25, 2014. Available from: https://www.sgo.org/wp-content/uploads/2014/03/ABSUPP_3.14.14.pdf. [Last accessed on 2014 May 12]. |
|31.||Wu Y, Chen Y, Li L, Yu G, Zhang Y, He Y. Associations of high-risk HPV types and viral load with cervical cancer in China. J Clin Virol 2006;35:264-9. |
|32.||Kang WD, Kim CH, Cho MK, Kim JW, Kim YH, Choi HS, et al. Comparison of the hybrid capture II assay with the human papillomavirus DNA chip test for the detection of high-grade cervical lesions. Int J Gynecol Cancer 2009;19:924-8. |
|33.||Moreira MA, Longato-Filho A, Taromaru E, Queiroz G, Jubé LF, Pinto SA, et al. Investigation of human papillomavirus by hybrid capture II in cervical carcinomas including 113 adenocarcinomas and related lesions. Int J Gynecol Cancer 2006;16:586-90. |
|34.||Poljak M, Kovanda A, Kocjan BJ, Seme K, Jancar N, Vrtacnik-Bokal E. The Abbott RealTime High Risk HPV test: Comparative evaluation of analytical specificity and clinical sensitivity for cervical carcinoma and CIN 3 lesions with the Hybrid Capture 2 HPV DNA test. Acta Dermatovenerol Alp Panonica Adriat 2009;18:94-103. |
|35.||Ronco G, Dillner J, Elfström KM, Tunesi S, Snijders PJ, Arbyn M, et al. Efficacy of HPV-based screening for prevention of invasive cervical cancer: Follow-up of four European randomised controlled trials. Lancet 2014;383:524-32. |
|36.||Austin RM. Can HPV primary screening reduce cervical cancer incidence and mortality? Scan 2014;25:7-8. |
|37.||Kinney W, Fetterman B, Poitras N, Lorey T. Cervical intraepithelial neoplasia 3+ (CIN3+) is not the right endpoint for evaluating screening algorithms, as it does not reflect cancer risk accurately. Abstract Number 474 Presented at the 45 th Meeting of the Society of Gynecologic Oncology, Tampa, Florida, March 2014. Available from: https://www.sgo.org/wp-content/uploads/2014/03/ABSUPP_3.14.14.pdf. [Last accessed on 2014 May 12]. |
|This article has been cited by|
||Comparison of cervical cancer screening results among 256,648 women in multiple clinical practices
| ||Amy J. Blatt,Ronald Kennedy,Ronald D. Luff,R. Marshall Austin,Douglas S. Rabin |
| ||Cancer Cytopathology. 2015; : n/a |
|[Pubmed] | [DOI]|