2018 Track 2: Adverse Drug Events and Medication Extraction in EHRs
This task aims to answer the question: “Can NLP systems automatically discover drug to adverse event (ADE) relations in clinical narratives?” The task builds on past medication extraction tasks, but examines a broader set of patients, diseases, and relations as compared with previous challenges. The task consists of three subtasks:
The World Health Organization (WHO) gives the following definitions:
These well-established definitions for ADE and ADR, which have been used for almost 50 years[3], continue to be used in a variety of recent research studies[4]-[6]. In their publication, Bates et al. state that the ADE definition is preferred because it is both “more comprehensive and clinically significant than the ADR”[7]. For this reason, our aim is to automatically detect ADEs. In addition, we are interested in accurately detecting drug names, dosages, durations, routes and other drug relations.
The data consists of discharge summaries of nearly ~500 discharge summaries drawn from the MIMIC-III (Medical Information Mart for Intensive Care III) clinical care database[1]. These data were annotated by 7 domain experts, including four physician assistant students and three nurses. Both entity tags and attributes were used to indicate the presence of drug and ADE information.
Data for the challenge will be released under a Rules of Conduct and Data Use Agreement. The data use agreement needs to be completed directly with MIMIC.
Sample gold standard files and the annotation guidelines will soon be available for download.
The evaluation will be conducted using withheld test data. Participating teams are asked to stop development as soon as they download the test data. Each team is allowed to upload (through this website) up to three system runs for each of the subtasks. System output is to be submitted in the exact format of the ground truth annotations, which will be provided by the organizers.
Participants are asked to submit a 500-word long abstract describing their methodologies. Abstracts may also have a graphical summary of the proposed architecture. The abstract should not exceed 2 pages (1.5pt line spacing, 12pt-font size). The authors of either top performing systems or particularly novel approaches will be invited to present or demonstrate their systems at the workshop. A special issue of a journal is planned following the workshop.
[1] Johnson AE, Pollard TJ, Shen L, Lehman LW, Feng M, Ghassemi M, Moody B, Szolovits P, Celi LA, Mark RG. "MIMIC-III, a freely accessible critical care database," Sci Data. 2016 May 24;3:160035. doi: 10.1038/sdata.2016.35.
[2] WHO. International drug monitoring: the role of national centres. Tech Rep Ser WHO 1972, no 498.
[3] Edwards IR, Aronson JK. "Adverse drug reactions: definitions, diagnosis, and management,” Lancet. 2000 Oct 7;356(9237):1255-9.
[4] Reddy VL, Pasha SJ, Rathinavelu M, and Reddy YP, “Assessment of knowledge, attitude and perception of pharmacovigilance and adverse drug reaction (ADR) reporting among the pharmacy students in south India,” IOSR J Pharm Biol Sci. 2014 February, 9(2):34-43.
[5] Petrovic M, Tangiisuran B, Rajkumar C, van der Cammen T, Onder G, “Predicting the Risk of Adverse Drug Reactions in Older Inpatients: External Validation of the GerontoNet ADR Risk Score Using the CRIME Cohort,” Drugs Aging. 2017 Feb;34(2):135-142. doi: 10.1007/s40266-016-0428-4.
[6] Toklu HZ, Soyalan M, Gültekin O, Özpolat M, Ayd?n MD, Günay AC, Yavuz DO, Akici A, Demirdamar R. "The Knowledge and Attitude of the Healthcare Professionals towards Pharmacovigilance and Adverse Drug Reaction Reporting in Northern Cyprus,” Pharmacovigilance 4:193. doi:10.4172/2329-6887.1000193.
[7] Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D, Laffel G, Sweitzer BJ, Shea BF, Hallisey R, et al. "Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group,” JAMA. 1995 Jul 5;274(1):29-34.