Vision Foundation Models
01Generalizable visual representations adaptable across imaging modalities and organs for lesion detection, anatomical segmentation, risk stratification, and quantitative biomarkers.
Department of Interventional & Diagnostic Radiology · Institute of Medical Imaging, Shanghai Jiao Tong University
We develop artificial intelligence methods for disease detection, diagnostic reasoning, quantitative analysis, and image reconstruction in clinical radiology — evaluated through multicenter cohorts and radiologist reader studies, while investigating physiological and pathological mechanisms reflected in medical imaging.
Our work examines how computational models represent medical images, integrate clinical context, restore fine anatomical detail, and connect image-derived structures with physiological function.
Generalizable visual representations adaptable across imaging modalities and organs for lesion detection, anatomical segmentation, risk stratification, and quantitative biomarkers.
Combining medical images, radiology reports, clinical history, and diagnostic criteria for structured reporting, multi-agent clinical reasoning, and reader-validated decision support.
Super-resolution, denoising, artifact restoration, and cross-modal synthesis designed to preserve diagnostically critical micro-architecture and quantitative accuracy.
3D anatomical modeling, registration, vascular hemodynamics, and musculoskeletal biomechanics linking structural imaging phenotypes to physiological function.
Large-scale representation learning, imaging-grounded clinical reasoning, and micro-architectural super-resolution validated across multicenter cohorts and structured radiologist reader studies.
Built on BoneFM, a DINOv2-style self-supervised whole-body skeletal CT backbone, BoneCoT encodes clinician-derived task dependencies to guide step-by-step reasoning from anatomical localization and bone density changes to interpretable bone metastasis assessment across multicenter cohorts.
Pretrained via self-supervision on 103,989 non-contrast abdominal CT scans and fine-tuned on 5,816 annotated cases across 11 acute abdominal conditions and 3 risk-stratification tasks. Across 2,528 external patients from 3 cohorts, macro-AUROC reached 0.919 on emergent conditions; in a crossover reader study, AI assistance raised mean AUROC from 0.812 to 0.924 and cut median reading time by 52.5 s per case.
Emulates a multidisciplinary stroke team via specialized agents coordinating emergency triage, time-window estimation, multimodal neuroimaging interpretation, contraindication screening, and retrieval-augmented guideline reasoning. Evaluated on 1,658 suspected acute ischemic stroke cases across two centers, improving treatment recommendation accuracy from 0.636 to 0.785 (Macro-F1 0.562 → 0.768).
Establishes a reproducible multicenter benchmark and blinded radiologist reader study evaluating 10 mainstream large language models on generating diagnostic impressions from free-text brain MRI report findings, systematically quantifying factual accuracy, clinical completeness, and reporting utility.
Releases the first spatially registered paired dataset of routine clinical CT and high-resolution photon-counting micro-CT volumes of human lumbar vertebrae, bridging in-vivo clinical imaging and trabecular bone micro-architecture through an open super-resolution benchmark and community challenge.
Addresses rapid multi-source clinical synthesis within narrow stroke reperfusion windows by orchestrating specialized agents for structured imaging readout, neurological feature extraction, and guideline-aligned treatment recommendation, validated across multicenter cohorts with an interactive web demo.
Impactful radiology research does not always require 100,000-scan pretraining — starting from anatomical variants, cardiopulmonary physiology, and concrete clinical questions, students lead peer-reviewed studies via cohort analysis, 3D quantitative measurement, surgical trajectory simulation, and lightweight LLM prompting:
Evaluated 10 LLMs and a 4-radiologist reader study across 433 multicenter patients with free-text carotid MRI reports, demonstrating that explicitly encoding clinical modified-AHA staging criteria enables a locally deployable lightweight 8B model to improve inter-reader classification agreement.
Grounded in cardiopulmonary hemodynamics — elevated pulmonary vascular resistance and reduced left-atrial venous return — this study extracts 3D pulmonary artery-to-vein volume difference on chest CTA as an interpretable quantitative biomarker for acute pulmonary embolism risk stratification.
Addresses distorted cervical bony landmarks and vertebral artery injury risk in ponticulus posticus variants by performing 3D screw trajectory simulation across 141 cervical CTA examinations to establish a quantitative preoperative safety workflow.
Analyzes a consecutive 16-year hospital trauma imaging cohort to characterize age-dependent risks of acute intracranial injury, highlighting occult vulnerability in older adults presenting with isolated facial fractures to guide emergency head CT screening.
Loading the latest news…
Led by Prof. Yuehua Li together with PIs Ruipeng Zhang, Jianyong Wei, and Zheng Sun, our group brings together radiologists, engineering researchers, postdoctoral fellows, and graduate students inside a high-volume clinical imaging department.
We welcome multicenter collaborations and recruit PhD and master’s students, postdoctoral fellows, and visiting researchers in medical imaging AI, computer vision, and multimodal learning. Send your CV and research interests to [email protected].
We warmly encourage undergraduates from SJTU School of Medicine (Clinical Medicine 5/8-year programs, Medical Imaging Technology, Biomedical Sciences, Stomatology, and Public Health) to join our lab! No prior deep-learning coding background is required — students receive dual mentorship from frontline radiologists and AI mentors, starting from clinical cohort curation, 3D anatomical quantification, surgical simulation, or LLM reader studies. Both individual students and pre-formed RBL / competition teams are welcome.
Email a brief introduction (year, major & interests) to Dr. Ruipeng Zhang: [email protected]