MedPAIR:
Measuring Physicians and AI Relevance Alignment in Medical Question Answering

Yuexing Hao1,2, Kumail Alhamoud1, Hyewon Jeong1, Haoran Zhang1, Grace Yan1, Isha Puri1, Philip Torr3, Mike Schaekermann6, Saleh Kalantari2, Ariel D. Stern4,5, Marzyeh Ghassemi1†
1 MIT, 2 Cornell University, 3 Oxford University, 4 University of Potsdam, 5 Hasso Plattner Institute, 6 Independent Research
†Corresponding author: mghassem@mit.edu
Paper Code arXiv
Study Design Pipeline

We began with four QA data sources. All questions were consolidated and structured into two components: patient profile and query. In the first step, physician trainee labelers and LLMs independently selected the best answer. In the second step, physicians experts then annotated the relevance of each sentence in the patient profile. We excluded annotations from physician trainees who selected incorrect answers and applied majority voting to derive binary sentence-level relevance labels. In parallel, we also asked LLMs to label each sentence in Patient Profile through prompting and ContextCite.


Abstract

Large Language Models (LLMs) have demonstrated remarkable performance on various medical question-answering (QA) benchmarks, including standardized medical exams. However, correct answers alone do not ensure correct logic, and models may reach accurate conclusions through flawed processes. In this study, we introduce the comparative reasoning benchmark MedPAIR (Medical Physicians and AI Relevance Alignment in Medical Question Answering) to evaluate how physicians and LLMs prioritize relevant information when answering QA questions. We obtain annotations on 2,000 QA pairs from 36 physicians, labeling each sentence within the question components for relevancy. We compare these relevancy estimates to those for LLMs, and further evaluate the impact of these ``relevant'' subsets on downstream task performance for both humans and LLMs. We further implemented a structured intervention where physicians guided LLMs to focus on relevant information. We find that LLMs are frequently not aligned with physicians' estimates of content relevancy. Further, we highlight the limitations of current LLM behavior and the importance of aligning AI reasoning with clinical standards.





Third Image

The dataset overview represents the annotation process for 2000 clinical question-answer pairs. We consolidated four QA data sources into two main components: the patient profile and the query. In the first step, 36 physician trainees and 4 LLMs independently selected the most appropriate answer. In the second step, physician trainees annotated the relevance of each sentence within the patient profile, excluding annotations linked to incorrect answers. Majority voting was used to produce binary relevance labels for physician trainees. Concurrently, we employed ContextCite with open-source LLMs (Qwen-14B, Qwen-72B, Llama-70B) to generate relevance scores, while GPT-4o was prompted to replicate the physician annotation process for each sentence following the same instructions.





Fourth Image

Sentence Position Analysis. Plot (a) Distribution of physician trainees’ majority‐vote relevance labels by sentence position. Plot (b) Distribution of GPT-4o self-reported relevance labels by sentence position. Plot (c) ContextCite scores across the context for three open-source models (Qwen-14B, Llama-70B, Qwen-72B).





Fifth Image

Sentence Position Analysis. (a) Distribution of LLM self-reported relevance labels by sentence position; (b) Distribution of physician trainee majority-voted relevance labels by sentence position; (c) Qwen-14B ContextCite scores across the clinical context.





Evaluation Pipeline Overview

Effect of Filtering Context on Final Performance. Accuracy on four medical benchmarks (MMLU, JAMA, MedXpert, MedBullets) when predictions use only the sentences a given labeler marked highly relevant. Solid lines give the original accuracy and dotted lines the best across labeler configurations; coloured markers show each configuration (CC: ContextCite, SR: self-reported). Annotations give best minus original in percentage points. Stars are McNemar's paired test with Bonferroni correction, * p<0.01/7 and ** p<0.01/28, shown only for improvements.



Pooled results across all four datasets

Pooled Results Across All Four Datasets. The same comparison with MMLU, JAMA, MedXpert and MedBullets combined into a single 933-question total. Accuracy is the number of correct answers over the full cohort, so a question a model does not answer counts as incorrect and every condition shares one denominator.




Video Presentation

BibTeX

@misc{hao2025medpairmeasuringphysiciansai,
      title={MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering}, 
      author={Yuexing Hao and Kumail Alhamoud and Hyewon Jeong and Haoran Zhang and Isha Puri and Philip Torr and Mike Schaekermann and Ariel D. Stern and Marzyeh Ghassemi},
      year={2025},
      eprint={2505.24040},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.24040}, 
}