The vision language model, named Pillar, analyzes CT and MRI images with an average AUC of .87 across 350+ findings, 10% - 17% more accurate than the leading publicly available AI models

University of California, Berkeley and University of California, San Francisco researchers today released Pillar (Pillar-0), an open-source AI model that analyzes medical images and recognizes conditions with an unprecedented degree of diagnostic accuracy. Unlike existing tools that are limited to a handful of conditions or models primarily designed for 2D images, Pillar-0 interprets 3D volumes directly and can recognize hundreds of conditions from a single CT or MRI exam. With over 500 million CT and MRI scans performed annually, imaging volumes are creating unsustainable capacity gaps for radiologists. Pillar-0 represents a new frontier in medical imaging models and aims to serve as the backbone for AI-powered advances that augment human expertise in radiology.

"Pillar-0 marks a major milestone in our mission to push the frontier of AI for radiology,” said Adam Yala, Assistant Professor of Computational Precision Health at UC Berkeley and UCSF and senior author of the research. “Pillar-0 outperforms leading models from Google, Microsoft and Alibaba by over 10% across 366 tasks and four diverse modalities; Pillar-0 also runs an order of magnitude faster, finetunes with minimal effort, and drives large downstream performance gains.”

The Pillar-0 research team validated Pillar-0 on chest CT, abdomen CT, brain CT, and breast MRI scans from UCSF. The model achieved a .87 AUC across 350+ findings on this data, outperforming all publicly available AI models for radiology, including Google’s MedGemma (.76 AUC), Microsoft's MI2 (.75 AUC), and Alibaba’s Lingshu (.70 AUC).

As a general-purpose backbone, Pillar-0 can easily be extended to tackle new clinical challenges. By finetuning Pillar-0, the team improved over the state-of-the-art lung cancer prediction tool, Sybil-1, by 7% in an external validation study at Massachusetts General Hospital. When fine-tuning for brain CT hemorrhage detection, Pillar-0 outperformed all baselines while using only a quarter of the training data.

“Leading foundation models for radiology have relied on processing 2D slices independently, because they are too inefficient to scale to the full imaging volumes,” said Kumar Krishna Agrawal, a PhD student at UC Berkeley and first author of the research on Pillar-0, which can be found at the link below. “To enable Pillar-0 to effectively process 3D volumes, we implemented innovations across data, pretraining and neural network architectures. Our novel Atlas neural network architecture is over 150x faster than traditional vision transformers at processing an abdomen CT, allowing us to train models at fraction of the cost.”

“We’re excited to release a rich, clinically-grounded evaluation framework, RaTE, alongside Pillar-0,” said Dr. Maggie Chung, Assistant Professor in Radiology and Biomedical Imaging at UCSF and senior author of the research on Pillar-0. “Existing benchmarks, like VQA-Rad, have relied on artificial questions posed on 2D slices that are poor measures of model utility. To address this gap, we assembled a large collection of diagnostic questions and findings that radiologists routinely evaluate in clinical practice. We also developed tools that enable any hospital to independently test or fine-tune Pillar-0 on their own data.”

The complete Pillar-0 codebase, trained models, evaluation and data pipelines are being released publicly to accelerate research and clinical adoption. The team plans to expand capabilities across additional imaging modalities, expanding to full grounded report generation. Pillar-0 research, code, and documentation can be found HERE.

"Transparency is essential to advancing the science of AI in health," said Yala. "Open-sourcing enables the global research community to independently validate our tools and build on our work. We’re excited to support folks building on the Pillar series."

About the Research Team

Kumar Krishna Agrawal is a PhD Candidate in Computer Science at UC Berkeley, jointly advised by Prof Yala and Prof Trevor Darrell. Krishna led the development of Atlas, the neural network architecture underlying Pillar-0.

Dr. Adam Yala is Assistant Professor of Computational Precision Health at UC Berkeley and UCSF, with additional affiliations in statistics and computer science at Berkeley. His breast cancer risk prediction tool has been validated on more than 2 million mammograms across 72+ hospitals in 22 countries.

Dr. Maggie Chung is Assistant Professor in Radiology and Biomedical Imaging at UCSF and a practicing radiologist specializing in translation of AI to improve diagnostic accuracy, optimize radiology workflows, and reduce delays in care.

View source version on businesswire.com: https://www.businesswire.com/news/home/20251120255214/en/