TRANSLATING BIOLOGY: TEXT MINING TOOLS THAT WORK

This year is the culmination of two series of sessions on natural language processing and text mining at the Pacific Symposium on Biocomputing. The first series of sessions, held in 2001, 2002, and 2003, explored information extraction and retrieval applications for a range of possible biomedical applications. The second series of sessions began in 2006. In the first two years of this series, the sessions focused on tasks that required mapping to or between grounded entities in databases (2006) and on cutting-edge problems in the field (2007). The goal of this final session of the second series has been to assess where the past several years’ worth of work have gotten us, what sorts of deployed systems have resulted, how well the systems have integrated genomic databases and the biomedical literature, and how usable these systems are. To this end, we solicited papers that addressed the following questions: What is the actual utility of text mining in the workflows of the various communities of potential users—model organism database curators, bedside clinicians, biologists utilizing high-throughput experimental assays, hospital billing departments, etc.? How usable are biomedical text mining applications? How does the application fit into the workflow of a complex bioinformatics pipeline? What kind of training does a bioscientist require to be able to use an application? Is it possible to build portable text mining systems? Can systems be adapted to specific domains and specific tasks without the assistance of an experienced language processing specialist? How robust and reliable are biomedical text mining applications? What are the best ways to assess robustness and reliability? Are the standard evaluation paradigms of the natural language processing world—intrinsic evaluation against a gold standard, post-hoc judging of outputs by trained judges, extrinsic evaluation in the context of some other task—the best evaluation paradigms for biomedical text mining, or even sufficient evaluation paradigms? 2. The session We received 29 submissions and accepted nine papers. Each paper received at least three reviews by members of a program committee composed of biomedical language processing specialists and computational biologists from North America, Europe, and Asia. All four of the broad questions were addressed by at least one paper. We review all nine papers briefly here. Utility A number of papers addressed the issue of utility. Alex et al.1 experimented with a variety of forms of automated curator assistance, measuring curation time and assessing curator attitudes by questionnaire, and found that text mining techniques can reduce curation times by as much as one third. Caporaso et al.3 examined potential roles for text-based and alignment-based methods of annotating mutations in a database curation workflow. They found that text mining techniques can provide a quality assurance mechanism for genomic databases. Roberts and Hayes9 analyzed a large collection of information requests from an understudied population—commercial drug developers—and found that various families of text mining solutions can play a role in meeting the information needs of this group. Wang et al. 11 evaluated a variety of algorithms for gene normalization, and found that there are complex interactions between performance on a gold standard, improvement in curator efficiency, portability, and the demands of different kinds of curation tasks.