Cytological analysis of small branch-duct intraductal papillary mucinous neoplasms provides a more accurate risk assessment of malignancy than symptoms
Jill Ono MD 1, Kurt A Yaeger BS 1, Muriel Genevay MD 2, Mari Mino-Kenudson MD 1, William R Brugge MD 3, Martha B Pitman MD 1
1 Department of Pathology, Division of Gastroenterology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
2 Service of Clinical Pathology, University Hospitals of Geneva, Geneva, Swaziland
3 Department of Medicine, Division of Gastroenterology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
|Date of Submission||20-Sep-2011|
|Date of Acceptance||19-Oct-2011|
|Date of Web Publication||21-Nov-2011|
Martha B Pitman
Department of Pathology, Division of Gastroenterology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
Source of Support: None, Conflict of Interest: None
Objectives: The Sendai guidelines for management of patients with clinically suspected intraductal papillary mucinous neoplasms (IPMN) recommend resection of cysts > 30 mm, a dilated main pancreatic duct (MPD) > 6 mm, a mural nodule (MN), symptoms or positive cytology. Although sensitive, asymptomatic cysts, nonspecific symptoms, and a high threshold for positive cytology limit the specificity of the guidelines. We have assessed the value of cytology relative to symptom for predicting malignancy in IPMNs without high-risk imaging features. Materials and Methods: We retrospectively reviewed the clinical, radiological, and cytological data of 31 small branch-duct IPMNs without a MN. The cytological presence of high-grade atypical epithelial cells (HGA) was considered true positive, with a corresponding histology of high-grade dysplasia or invasive carcinoma. The performance of cytology versus symptoms was evaluated by calculating the sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and accuracy. Results: The sensitivity (0.80), specificity (0.85), and accuracy (0.84) of HGA were higher than the corresponding performance characteristics of symptoms (0.60, 0.45, and 0.48, respectively). The NPV of no HGA on cytology was > 95%. Conclusions: Cytology is a better predictor of malignancy than symptoms, for the conservative management of small branch-duct IPMNs. Cytology contributes to a highly accurate triple negative test for malignancy in small IPMN: No dilated MPD, MN or HGA.
Keywords: Branch-duct, cytology, fine needle aspiration biopsy, intraductal papillary mucinous neoplasm, pancreas, Sendai guidelines, symptoms
|How to cite this article:|
Ono J, Yaeger KA, Genevay M, Mino-Kenudson M, Brugge WR, Pitman MB. Cytological analysis of small branch-duct intraductal papillary mucinous neoplasms provides a more accurate risk assessment of malignancy than symptoms. CytoJournal 2011;8:21
|How to cite this URL:|
Ono J, Yaeger KA, Genevay M, Mino-Kenudson M, Brugge WR, Pitman MB. Cytological analysis of small branch-duct intraductal papillary mucinous neoplasms provides a more accurate risk assessment of malignancy than symptoms. CytoJournal [serial online] 2011 [cited 2017 Feb 20];8:21. Available from: http://www.cytojournal.com/text.asp?2011/8/1/21/90084
| » Introduction|| |
Surgical management of patients with intraductal papillary mucinous neoplasm (IPMN) has changed since the publication of the international consensus algorithm for the management of mucinous cysts, in 2006 (Sendai guidelines).  Growing evidence suggests that most branch-duct IPMNs behave in a benign fashion and are identified in the elderly with comorbid conditions that increases their surgical risk. This has resulted in a treatment algorithm that has limited surgical resection to those cysts with high-risk features. ,,,,,,,, High-risk features include symptoms, cyst size > 30 mm, association with a dilated main pancreatic duct (MPD) > 6 mm, a mural nodule (MN), or positive cytology.  Validation studies of the Sendai guidelines have shown very high sensitivity (97.3 to 100%), but a low specificity (21.7 - 29.8%) in predicting malignancy. ,, The use of symptoms as a high-risk feature has contributed to this low specificity. The utility of resection is undeniable for symptomatic relief in patients with pancreatic cysts, however, the presence of symptoms may be completely unrelated to the cyst or due to mechanical issues in benign cysts.  The presence of symptoms are now recognized to be of limited value in determining the risk of malignancy in these patients. 
The value of cytology in the risk assessment of malignancy in patients with a clinical diagnosis of mucinous cysts is controversial. We have previously shown that cytological features can be identified, which correlate with histological grade,  and that 'positive' cytology, although highly specific, lacks sensitivity in identifying malignant mucinous cysts. ,, In addition, we and others have shown that atypical cytology is a strong predictor of malignancy, and we have suggested that a threshold of high grade atypical (dysplastic) epithelial cells (AEC) rather than 'positive cytology' is a marker for malignancy in IPMN, including small (≤ 30 mm), asymptomatic branch-duct IPMNs that are clinically and radiologically low grade. ,,
We sought to analyze the predictive value of symptoms versus cytology in small branch-duct IPMNs without high risk imaging features. We hypothesize that cytology is an important preoperative test in this setting and that cytological analysis of cyst fluid for HGA is a more accurate test than symptoms for malignant risk.
| » Materials and Methods|| |
The Massachusetts General Hospital internal review board approval was obtained to conduct this study. Histologically confirmed cysts from imaging studies, less than 3 cm in greatest dimension, were selected retrospectively between the years 1993 and 2008 from the pathology laboratory information system. Inclusion criteria included the absence of a dilated main pancreatic duct (< 6 mm), absence of a mural nodule by endoscopic ultrasound (EUS), adequate endoscopic ultrasound-guided-fine needle aspiration (EUS-FNA) fluid for evaluation, and available pathological material for review. Clinical, radiological, and cytological data was retrieved from the electronic medical record. The presence or absence and type of symptoms were documented. All cytological material was obtained by EUS-FNA of the pancreatic cyst. The cytological material included direct smears, or liquid-based preparations (cytospins, ThinPrep® or Surepath™ preparations) stained with routine cytological stains. The cytology slides were reviewed and blinded to the histological grade. The epithelial cells were classified as either low grade (gastrointestinal contamination or low-grade dysplasia) or HGA (high grade AEC or malignant cells) [Figure 1]. Cytology with high-grade atypical epithelial cells or malignancy (HGA) was considered true positive, and their absence as true negative for predicting the histology of high-grade dysplasia or invasive carcinoma in resection specimens. The histological slides were reviewed and classified according to the Armed Forces Institute of Pathology (AFIP) classification scheme, using low-grade dysplasia (adenoma), moderate dysplasia, high grade-dysplasia / carcinoma in-situ and invasive carcinoma.  Low-grade and moderate-dysplasia were considered negative for carcinoma, and high-grade dysplasia and invasive carcinoma were considered positive for carcinoma.
|Figure 1: a) Low-grade epithelial cells corresponding to either gastrointestinal contamination or low-grade dysplasia (Papanicolaou, x600), b) High-grade atypical epithelial cells corresponding to either high grade dysplasia / carcinoma in-situ or invasive carcinoma (Papanicolaou, x600)|
Click here to view
Performance characteristics including sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and accuracy were calculated for both clinical symptoms and cytology with HGA.
| » Results|| |
The study cohort consisted of 31 small branch duct IPMNs from patients in the age range of 43 - 83 years (mean = 67 years), with a female predominance: 22 (71%) females and nine males (29%). The cysts were primarily located in the pancreatic head (8), uncinate (11), and body (9), with two cysts located in the pancreatic neck and one in the tail. The average cyst size was 18.7 mm (range 10 - 30 mm). On histology there were 26 non-malignant cysts: 14 IPMNs with low-grade dysplasia and 12 with moderate dysplasia. Five cysts were malignant, three with high-grade dysplasia, and two with invasive adenocarcinoma. [Table 1] outlines the distribution of symptoms and HGA among these cysts.
|Table 1: Distribution of symptoms and cytology with HGA by histological outcome|
Click here to view
Over half (55%) of the patients were symptomatic. Most of the symptomatic patients did not have malignancy (14 / 17, 82%), and almost an equal number of patients with malignancy did not have symptoms (2 / 5, versus 3 / 5 with symptoms). A majority of the symptomatic patients experienced abdominal pain (8/17; 47%) or abdominal pain with evidence of pancreatitis (7 / 17; 41%). Additionally, one patient had diarrhea and one presented with weight loss. No patient presented with either diabetes or jaundice.
Eight (25.8%) cyst fluids demonstrated HGA on cytology and four of these cysts were malignant (two CIS and two invasive carcinoma) on histology. The four remaining cyst fluids with HGA demonstrated moderate dysplasia on histology. There was one cyst with CIS that did not have HGA on cytology.
The performance characteristics of symptoms versus cytology with HGA are outlined in [Table 2]. The sensitivity (0.80), specificity (0.85), and accuracy (0.84) of cytology with HGA were much higher than the corresponding performance characteristics of symptoms (0.60, 0.46, and 0.48, respectively). The NPV of the cyst fluid with no HGA on cytology was 96%.
|Table 2: Performance characteristics of symptoms versus cytology with HGA|
Click here to view
| » Discussion|| |
We report on the value of cytological analysis in predicting malignancy in the preoperative evaluation of patients with small (< 3 cm) branch-duct IPMNs. Cytology was almost twice as sensitive, specific, and accurate as symptoms in predicting malignancy in small cysts, without imaging risk factors. The absence of HGA on cytology yielded a NPV of 96%, contributing to a powerful triple negative test for malignancy - no dilated MPD (< 6 mm), no MN, and no HGA on cytology - providing a much more accurate assessment of no malignancy for conservative clinical management.
The Sendai guidelines for the management of mucinous cysts are mainly based on the radiological features. Resection is recommended for symptomatic cysts, cysts greater than 30 mm, or 10 - 30 mm cysts with high-risk stigmata, which include dilated MPD (> 6 mm), a MN on radiological studies or positive cytology.  Although, overall highly sensitive in the appropriate surgical triage (sensitivity 97.3 to 100%), in specificity (21.7 to 29.8%) it suffers. ,, This high sensitivity arises from a surgical triage for any of the features in the algorithm.
The use of symptoms as a predictor of malignancy contributed to the low specificity of the guidelines, especially in small branch-duct IPMNs. ,,,, Even as patients who were symptomatic were significantly more likely to have a malignant cyst, in the recent study by Mimura,  an asymptomatic patient was equally likely to have a malignant cyst. Our data shows similar findings in small branch-duct IPMNs. Three patients with malignant cysts were symptomatic, but two were not. In another study, Weisenaur et al., found that new onset diabetes and jaundice were significant predictors of malignancy, but a variety of symptoms were present in patients with non-malignant cysts, especially abdominal pain. Patients with small cysts were not likely to present with such dramatic symptoms, as was true of the 31 patients in our study. Additionally, a recent multivariate analysis by Shin et al., revealed that a history of pancreatitis was predictive of invasive IPMN.  In our cohort, the two patients with invasive carcinoma did present with pancreatitis; however, the remaining five patients with pancreatitis had only low-grade dysplasia on histology. Of the patients with HGD / carcinoma in-situ, two were asymptomatic and one presented with abdominal pain.
Large tertiary referral centers like the MGH, with a symptomatic patient population, may bias the data to some degree as was the case in a large series by Schmidt,  where over 90% of the 150 patients in the study were symptomatic. Symptoms are clearly important in patient evaluation, and the presence of symptoms is a valid indication for resection, if for no other reason than to alleviate the symptom. The significance and association of the symptoms with the cyst may be quite difficult to assess in the clinical setting of an elderly patient with comorbid conditions, however.
Even as the risk of malignancy was low in a small cyst, it was not zero. The rate of malignancy in this study was 16%, falling within the previously reported range of 0 - 20% in cysts < 3 cm. ,, Furthermore, Sawhney showed that although cyst size was a predictor of malignancy in pancreatic cysts, nearly half (9 / 21) of the malignant cysts were < 3 cm. ,
Tissue confirmation of malignancy is a strong motivator for surgical intervention. To achieve a cytological diagnosis of 'positive,' indicating > 90% PPV of malignancy, is a high standard in cyst specimens, where cellularity and cellular preservation often limit optimal interpretation. Aspirating cyst contents may underestimate the highest histological grade of the cyst. ,,, Experience with these relatively rare specimen types and specialty training in cytology are also often lacking in many settings, leading to a conservative interpretation of the cells that may be present in the fluid.
We have recommended that cells that meet the criteria for high-grade atypia  [Figure 1] trigger resection, as these cells have an ~70% sensitivity and 85% specificity for predicting malignancy, very likely preinvasive disease. Given that invasion greatly decreases the prognosis of patients with IPMN,  resection prior to invasion is optimal, especially in young patients who have many years for potential progression of the dysplasia to invasive carcinoma. Knowing that some patients may indeed have only moderate or low-grade dysplasia on histology warrants careful consideration of morbidity related to surgical intervention. High-risk surgical candidates and very elderly patients require assessment of the greatest risk to survival: surgery or the possibility of progression to invasive carcinoma. Importantly, however, the absence of HGA on cytology yielded a NPV of 96%, contributing to a powerful triple negative test for malignancy - no dilated MPD (< 6 mm), no MN, and no HGA on cytology - providing a much more accurate assessment of no malignancy for conservative clinical management.
| » Conclusions|| |
In summary, we show the value of cytology as a screening test for malignancy in small branch-duct IPMN of the pancreas. Cytological assessment of the pancreatic cyst fluid for HGA was almost twice as accurate as symptoms for the detection of malignancy. We recommend that cytology be combined with imaging studies for the preoperative assessment of small pancreatic cysts clinically consistent with a branch-duct IPMN, as an accurate triple negative screening test for malignancy.
| » Competing Interest Statement By All Authors|| |
The authors declare that they have no competing interests.
| » Authorship Statement By All Authors|| |
All authors of this article declare that we qualify for authorship as defined by ICMJE. All authors are responsible for the conception of this study, participated in its design and coordination, and helped to draft the manuscript. All authors read and approved the final manuscript.
| » Ethics Statement By All Authors|| |
This study was conducted with approval from Institutional Review Board (IRB) (or its equivalent) of all the institutions associated with this study. Authors take responsibility to maintain relevant documentation in this respect.
| » References|| |
|1.||Tanaka M, Chari S, Adsay V, Fernandez-del Castillo C, Falconi M, Shimizu M, et al. International consensus guidelines for management of intraductal papillary mucinous neoplasms and mucinous cystic neoplasms of the pancreas. Pancreatology 2006;6:17-32. |
|2.||Ferrone CR, Correa-Gallego C, Warshaw AL, Brugge WR, Forcione DG, Thayer SP, et al. Current trends in pancreatic cystic neoplasms. Arch Surg 2009;144:448-54. |
|3.||Sahani DV, Lin DJ, Venkatesan AM, Sainani N, Mino-Kenudson M, Brugge WR, et al. Multidisciplinary approach to diagnosis and management of intraductal papillary mucinous neoplasms of the pancreas. Clin Gastroenterol Hepatol 2009;7:259-69. |
|4.||Allen PJ, Brennan MF. The management of cystic lesions of the pancreas. Adv Surg 2007;41:211-28. |
|5.||Farnell MB. Surgical management of intraductal papillary mucinous neoplasm (IPMN) of the pancreas. J Gastrointest Surg 2008;12:414-6. |
|6.||Garcea G, Ong SL, Rajesh A, Neal CP, Pollard CA, Berry DP, et al. Cystic lesions of the pancreas. A diagnostic and management dilemma. Pancreatology 2008;8:236-51. |
|7.||Lahav M, Maor Y, Avidan B, Novis B, Bar-Meir S. Nonsurgical management of asymptomatic incidental pancreatic cysts. Clin Gastroenterol Hepatol 2007;5:813-7. |
|8.||Matthes K, Mino-Kenudson M, Sahani DV, Holalkere N, Brugge WR. Concentration-dependent ablation of pancreatic tissue by EUS-guided ethanol injection. Gastrointest Endosc 2007;65:272-7. |
|9.||Tseng JF, Warshaw AL, Sahani DV, Lauwers GY, Rattner DW, Fernandez-del Castillo C. Serous cystadenoma of the pancreas: tumor growth rates and recommendations for treatment. Ann Surg 2005;242:413-9; discussion 419-21. |
|10.||Fernández-del Castillo C, Targarona J, Thayer SP, Rattner DW, Brugge WR, Warshaw AL. Incidental pancreatic cysts: clinicopathologic characteristics and comparison with symptomatic patients. Arch Surg 2003;138:427-3; discussion 433-4. |
|11.||Nagai K, Doi R, Ito T, Kida A, Koizumi M, Masui T, et al. Single-institution validation of the international consensus guidelines for treatment of branch duct intraductal papillary mucinous neoplasms of the pancreas. J Hepatobiliary Pancreat Surg 2009;16:353-8. |
|12.||Pelaez-Luna M, Chari ST, Smyrk TC, Takahashi N, Clain JE, Levy MJ, et al. Do consensus indications for resection in branch duct intraductal papillary mucinous neoplasm predict malignancy? A study of 147 patients. Am J Gastroenterol 2007;102:1759-64. |
|13.||Tang RS, Weinberg B, Dawson DW, Reber H, Hines OJ, Tomlinson JS, et al. Evaluation of the guidelines for management of pancreatic branch-duct intraductal papillary mucinous neoplasm. Clin Gastroenterol Hepatol 2008;6:815-9; quiz 719. |
|14.||Dua KS, Vijayapal AS, Kengis J, Shidham VB. Ciliated foregut cyst of the pancreas: Preoperative diagnosis using endoscopic ultrasound guided fine needle aspiration cytology - A case report with a review of the literature. CytoJournal 2009;6:22. |
|15.||Waters JA, Schmidt CM. Intraductal papillary mucinous neoplasm-when to resect? Adv Surg 2008;42:87-108. |
|16.||Michaels PJ, Brachtel EF, Bounds BC, Brugge WR, Pitman MB. Intraductal papillary mucinous neoplasm of the pancreas: cytologic features predict histologic grade. Cancer 2006;108:163-73. |
|17.||Pitman MB, Michaels PJ, Deshpande V, Brugge WR, Bounds BC. Cytological and cyst fluid analysis of small (≤3 cm) branch duct intraductal papillary mucinous neoplasms adds value to patient management decisions. Pancreatology 2008;8:277-84. |
|18.||Shen J, Brugge WR, Dimaio CJ, Pitman MB. Molecular analysis of pancreatic cyst fluid: a comparative analysis with current practice of diagnosis. Cancer 2009;117:217-27. |
|19.||Pitman MB. Cytology of the pancreas, in diagnostic cytopathology. In: Gray W, Kocjan G, editors. London: Churchill Livingstone; 2010. |
|20.||Pitman MB, Genevay M, Yaeger K, Chebib I, Turner BG, Mino-Kenudson M, et al. High-grade atypical epithelial cells in pancreatic mucinous cysts are a more accurate predictor of malignancy than "positive" cytology. Cancer Cytopathol 2010;118:434-40. |
|21.||Hruban RH, Pitman MB, Klimstra DS. Tumors of the Pancreas. Atlas of Tumor Pathology, 4th series, fascicle 6. Washington, D.C.: American Registry of Pathology; Armed Forces Institutes of Pathology; 2007. |
|22.||Mimura T, Masuda A, Matsumoto I, Shiomi H, Yoshida S, Sugimoto M, et al. Predictors of malignant intraductal papillary mucinous neoplasm of the pancreas. J Clin Gastroenterol 2010;44:e224-9. |
|23.||Schmidt CM, White PB, Waters JA, Yiannoutsos CT, Cummings OW, Baker M, et al. Intraductal papillary mucinous neoplasms: predictors of malignant and invasive pathology. Ann Surg 2007;246:644-51; discussion 651-4. |
|24.||Salvia R, Fernández-del Castillo C, Bassi C, Thayer SP, Falconi M, Mantovani W, et al. Main-duct intraductal papillary mucinous neoplasms of the pancreas: clinical predictors of malignancy and long-term survival following resection. Ann Surg 2004;239:678-85; discussion 685-7. |
|25.||Wiesenauer CA, Schmidt CM, Cummings OW, Yiannoutsos CT, Howard TJ, Wiebke EA, et al. Preoperative predictors of malignancy in pancreatic intraductal papillary mucinous neoplasms. Arch Surg 2003;138:610-7; discussion 617-8. |
|26.||Shin SH, Han DJ, Park KT, Kim YH, Park JB, Kim SC. Validating a simple scoring system to predict malignancy and invasiveness of intraductal papillary mucinous neoplasms of the pancreas. World J Surg 2010;34:776-83. |
|27.||Allen PJ, D'Angelica M, Gonen M, Jaques DP, Coit DG, Jarnagin WR, et al. A selective approach to the resection of cystic lesions of the pancreas: results from 539 consecutive patients. Ann Surg 2006;244:572-82. |
|28.||Sawhney MS, Al-Bashir S, Cury MS, Brown A, Chuttani R, Pleskow DK, et al. International consensus guidelines for surgical resection of mucinous neoplasms cannot be applied to all cystic lesions of the pancreas. Clin Gastroenterol Hepatol 2009;7:1373-6. |
|29.||Genevay M, Mino-Kenudson M, Yaeger K, Konstantinidis IT, Ferrone CR, Thayer S, et al. Cytology adds value to imaging studies for risk assessment of malignancy in pancreatic mucinous cysts. Ann Surg. 2011 Oct 28. [Epub ahead of print]. |
|30.||Sohn TA, Yeo CJ, Cameron JL, Hruban RH, Fukushima N, Campbell KA, et al. Intraductal papillary mucinous neoplasms of the pancreas: An updated experience. Ann Surg 2004;239:788-97; discussion 797-9. |
[Table 1], [Table 2]
|This article has been cited by|
||Current roles of endoscopy in the management of intraductal papillary mucinous neoplasm of the pancreas
| ||Masao Tanaka |
| ||Digestive Endoscopy. 2015; : n/a |
|[Pubmed] | [DOI]|
||Managing Pancreatic Cysts: Less Is More?
| ||Marcia Irene Canto,Ralph H. Hruban |
| ||Gastroenterology. 2015; |
|[Pubmed] | [DOI]|
||Locus/Chromosome Aberrations in Intraductal Papillary Mucinous Neoplasms Analyzed by Fluorescence In Situ Hybridization
| ||Katsuyuki Miyabe,Yasuki Hori,Takahiro Nakazawa,Kazuki Hayashi,Itaru Naitoh,Shuya Shimizu,Hiromu Kondo,Yuji Nishi,Michihiro Yoshida,Shuichiro Umemura,Akihisa Kato,Hirotaka Ohara,Takashi Joh,Hiroshi Inagaki |
| ||The American Journal of Surgical Pathology. 2015; 39(4): 512 |
|[Pubmed] | [DOI]|
||mAb Das-1 is specific for high-risk and malignant intraductal papillary mucinous neoplasm (IPMN)
| ||K. K. Das,H. Xiao,X. Geng,C. Fernandez-del-Castillo,V. Morales-Oyarvide,E. Daglilar,D. G. Forcione,B. C. Bounds,W. R. Brugge,M. B. Pitman,M. Mino-Kenudson,K. M. Das |
| ||Gut. 2014; 63(10): 1626 |
|[Pubmed] | [DOI]|
||Intraductal Papillary Mucinous Neoplasms of the Pancreas
| ||Masao Tanaka |
| ||Pancreas. 2014; 43(8): 1136 |
|[Pubmed] | [DOI]|
||High-Grade Dysplasia and Adenocarcinoma Are Frequent in Side-Branch Intraductal Papillary Mucinous Neoplasm Measuring Less than 3 cm on Endoscopic Ultrasound
| ||Joyce Wong,Jill Weber,Barbara A. Centeno,Shivakumar Vignesh,Cynthia L. Harris,Jason B. Klapman,Pamela Hodul |
| ||Journal of Gastrointestinal Surgery. 2013; 17(1): 78 |
|[Pubmed] | [DOI]|
||Endoscopic ultrasound-guided fine needle aspiration (EUS-FNA) contributes to a triple-negative test in preoperative screening of pancreatic cysts
| ||Roseann I. Wu,Won Jae Yoon,William R. Brugge,Mari Mino-Kenudson,Martha B. Pitman |
| ||Cancer Cytopathology. 2013; : n/a |
|[Pubmed] | [DOI]|
||Intraductal Papillary Mucinous Neoplasm of the Pancreas: An Update
| ||Shu-Yuan Xiao |
| ||Scientifica. 2012; 2012: 1 |
|[Pubmed] | [DOI]|
||Revised international consensus guidelines for the management of patients with mucinous cysts
| ||Martha Bishop Pitman |
| ||Cancer Cytopathology. 2012; 120(6): 361 |
|[Pubmed] | [DOI]|