Extracting Health Evidence Information from Biomedical Literature using Large Language Models

The current biomedical literature is huge, unstructured, and complex, posing a significant challenge to efficient information processing and consequently creating a substantial gap between medical research and clinical practice. This underscores the need for innovative approaches to convert unstructured information into a computable format readily available for clinical decision-making. The study aims to leverage large language models (LLMs) to accurately extract health evidence from unstructured biomedical text by utilizing the PICO (Patient, Intervention, Comparison, Outcome) framework. We implemented different variations of Generative Pretrained Transformers (GPT), where GPT 4 and GPT 4 Turbo Preview consistently improve recall, precision, and accuracy compared to competitors like GPT 4o and GPT 3.5. We employed the document retrieval strategy, which involved the use of the ”stuff method” instead of the ”map-reduce” and ”maprerank” methods, aligning to achieve higher accuracy within a limited document set. Challenges include accurately synthesizing fragmented PICO elements and ensuring answer correctness due to limitations in context retrieval length. The performance of the proposed system is evaluated using the Retrieval Augmented Generation assessment (RAGAs) framework.
The findings highlight LLMs’ potential in medical research while emphasizing the need for enhancements in context retrieval and model correctness. The study’s outcomes suggest that leveraging LLMs for biomedical information extraction can potentially improve the healthcare decision-making landscape.