Verification bias is common in cytopathology studies on diagnostic accuracy
Robert L. Schmidt
Department of Pathology, University of Utah School of Medicine and ARUP Laboratories, Salt Lake City, Utah, USA
|Date of Submission||11-Feb-2014|
|Date of Acceptance||05-Apr-2014|
|Date of Web Publication||22-May-2014|
Robert L. Schmidt
Department of Pathology, University of Utah School of Medicine and ARUP Laboratories, Salt Lake City, Utah
Source of Support: None, Conflict of Interest: None
|How to cite this article:|
Schmidt RL. Verification bias is common in cytopathology studies on diagnostic accuracy. CytoJournal 2014;11:13
Editorial/Peer Review Statement
Editorial disclosure: The authors of the initial article (Ref #1) could not be reached even after multiple attempts by the editorial team and did not reply to this letter.
To the Editor,
I wish to comment on the study by Jain et al., entitled "Fine-needle aspiration cytology in diagnosis of salivary gland lesions: A study with histologic comparison."  Studies based on comparisons with histologic reference standards are common in cytopathology; however, this type of study design, generally produces biased estimates of sensitivity and specificity known as verification bias. ,
The impact of verification bias can be shown as follows. Consider a diagnostic test with a sensitivity, Sn, a specificity, Sp, and disease prevalence, θ. For such a test, the true positive rate, TP, false positive rate, FP, false negative rate, FN and true negative rate, TN, are:
The verification rate is the proportion of samples that are verified by the gold standard. The positive and negative verification rates are designated as α and β, respectively. Verification bias occurs when the negative and positive verification rates differ (i.e. α ≠ β). We use the superscript "v" to designate accuracy statistics observed when verification bias is present. For example, TP v is the true positive rate observed under verification bias. When verification bias is present, the observed accuracy statistics are [Figure 1]:
sensitivity and specificity will be biased unless the verification rates for positive and negative fine-needle aspiration (FNA) samples are equivalent (i.e. α = β). In general, cases with a positive FNA diagnosis will be referred to surgery at a higher rate than cases with a negative diagnosis (α > β). As can be seen from Eqs. 2.9 and 2.10, this means that observed sensitivity will be greater than the true sensitivity and the observed specificity will be lower than the true sensitivity (negative bias). For example, suppose that Sn = 0.8, Sp = 0.9, and α/β =2. In this case, the observed sensitivity and specificity would be 0.89 (positive bias of 0.09) and 0.81 (negative bias of 0.09), respectively.
|Figure 1: Flow diagram illustrating verification bias. θ = disease prevalence, Sn = true sensitivity, Sp = true specificity, α = verification rate of positive test results, β = verification rate of negative test results. TPv, FNv, FPv, and TNv are the observed (i.e. verified) true positive, false negative, false positive, and true negative results, respectively. uTPv, uFNv, uFPv, and uTNv represent the unobserved (unverified) values|
Click here to view
It is important to note that the predictive values are unaffected by the verification rates (Eqs. 2.11 and 2.12). Thus, the predictive values can be safely reported in studies based on surgical samples. However, predictive values depend on the prevalence and for that reason, can only be applied in a similar context.
Zhou et al., have published correction formulas that can be used to estimate the true sensitivity and specificity in studies with verification bias.  To apply these formulas, one must know the FNA diagnoses for all patients who received FNA. Jain et al., only reported FNA results for patients who were verified by surgery. Thus, it is impossible to estimate the extent of bias.
The STARD initiative provides guidelines for reporting of diagnostic accuracy studies. ,, These reporting guidelines are designed to enable readers to assess the potential for bias in diagnostic accuracy studies. Unfortunately, the results reported by Jain et al., are likely to be biased and because of inadequate reporting, the degree of bias cannot be determined. Diagnostic accuracy studies for FNA cytology commonly suffer from verification bias.  Although CytoJournal adopts the STARD guidelines,  authors and peer reviewers should be aware of the problem and adhere to the STARD guidelines for reporting of diagnostic accuracy studies.
| » Competing interests statement by all authors|| |
None to declare.
| » Authorship statement by all authors|| |
Robert Schmidt is the sole contributor to this manuscript.
| » References|| |
|1.||Jain R, Gupta R, Kudesia M, Singh S. Fine needle aspiration cytology in diagnosis of salivary gland lesions: A study with histologic comparison. Cytojournal 2013;10:5. |
|2.||Schmidt RL, Jedrzkiewicz JD, Allred RJ, Matsuoka S, Witt BL. Verification bias in diagnostic accuracy studies for fine- and core needle biopsy of salivary gland lesions in otolaryngology journals: A systematic review and analysis. Head Neck 2013;[Epub ahead of print]. |
|3.||Zhou XH, Obuchowski N, McLish D. Statistical Methods in Diagnostic Medicine, 2 nd ed. Hoboken, New Jersey: John Wiley and Sons; 2011. |
|4.||Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Ann Clin Biochem 2003;40:357-63. |
|5.||Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. The STARD statement for reporting studies of diagnostic accuracy: Explanation and elaboration. Ann Intern Med 2003;138:W1-12. |
|6.||The STARD Statement. [Last accessed date on 2014 Feb 11]. |
|7.||Instructions for CytoJournal Authors. Available from: http://www.cytojournal.com/contributors.asp. [Last accessed date on 2014 Feb 11]. |