Objective: An accurate, comprehensive and up-to-date problem list can help clinicians focus on providing patient-centered care. In this study, we report on physicians’ assessment of IBM Watson generated problem lists and comparison with an existing manually curated problem list in an institution’s EHR system.
Materials and Methods: Fifteen randomly selected, de-identified patient records from a large healthcare system were analyzed using Watson. Ten internal medicine physicians each reviewed five randomly selected patient records and created their own problem lists (P) for each patient record. Then, they evaluated the Watson generated problem lists (W), and rated the overall usefulness of P and W, as well as the existing EHR problem lists (E). The primary outcome was the physicians’ usefulness ratings of the problem lists on a 10-point scale and their pairwise comparisons.
Results: Six out of the 10 invited physicians completed 27 assessments of P, W, and E, consisting of 732 Watson generated problems and 444 problems in the EHR system. As expected, physicians rated their own lists, P, best. However, they rated W higher than E. In 89% of the assessments, Watson identified at least one important problem that the physicians missed. The higher ratings of W relative to E were influenced by the number of problems missing from E.
Conclusion: Cognitive computing systems hold the potential for accurate, problem-list-centered summarization of patient records, leading to increased efficiency, better clinical decision support, and improved quality of patient care.
By: Murthy V. Devarakonda, Neil Mehta, Ching-Huei Tsou, Jennifer L. Liang, Amy S. Nowacki, John Eric Jelovsek
Published in: RC25615 in 2016
LIMITED DISTRIBUTION NOTICE:
This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.
Questions about this service can be mailed to reports@us.ibm.com .