FleXray: Flexible Full-Body
X-ray Segmentation

Victor Ion Butoi · Vivek Gopalakrishnan · John V. Guttag · Adrian V. Dalca · Neel Dey

Massachusetts Institute of Technology · Massachusetts General Hospital · Harvard Medical School

Code Model soon Demo soon Paper soon

python -m pip install flexray

FleXray predictions on held-out radiographs across body regions and projections. Top: input X-rays. Bottom: predicted anatomy.

Abstract

We present FleXray, the first foundation model for segmenting anatomy from standard X-rays that enables quantitative radiographic analysis. Segmenting X-rays is challenging: X-rays are 2D projections of 3D anatomy, so overlapping structures—common in regions like the abdomen, pelvis, and neck—appear visually ambiguous, and differences in acquisition angle, exposure, and equipment make X-rays of the same region look markedly different. These obstacles have limited the collection of expert-labeled X-ray segmentation data, and consequently the development of general-purpose X-ray segmentation tools. FleXray is designed to segment X-rays from diverse anatomical regions, clinical environments, and acquisition settings. We train it on a label-standardized aggregation of CT and X-ray segmentation datasets, covering 62 foreground anatomical labels. From the CT data, we synthesize radiographic views spanning realistic acquisition conditions with physics-based projections. Across a wide range of held-out datasets, FleXray produces accurate segmentations and substantially outperforms existing baselines, and it supports clinical workflows such as scoliosis monitoring. We release FleXray as a one-step tool, together with its data engine and training code.

Motivation

More than 2.3 billion conventional X-ray examinations are acquired each year, yet public pixel-level annotations cover only a narrow set of views and structures. Every radiograph collapses 3D anatomy into a 2D projection: overlapping organs, variable angles, scanner settings, field of view, and patient morphology can make even the same body region look radically different.

Labeled CT volumes offer a way out. Segmented CT scans turn scarce 3D annotations into many anatomically correct 2D training views.

Bar chart showing 2.28 billion annual X-ray examinations versus 403 million CT studies, next to skeletons showing that publicly available segmentation data covers CT anatomy far more completely than X-ray. — **Figure 1.** X-rays are acquired far more often than CT (2.28B vs. 403M studies per year), yet publicly available segmentation data covers X-ray anatomy only sparsely while CT anatomy is densely labeled.

Method: online CT-to-DRR rendering

FleXray ray-casts labeled CT volumes into digitally reconstructed radiographs (DRRs). Geometry, framing, and per-structure attenuation vary online, so a single subject becomes a stream of distinct paired images and masks—a new view at every training step. Randomly sampled cameras expose the model to anatomy from standard and non-standard acquisition angles.

Load the CT volume, dense native labels, and physical-space affine.
Sample detector geometry, view angle, field of view, and anatomy-centered crops.
Ray-cast image intensities and masks into the common model label space.

Animated diagram of a camera at a shared position ray-casting a CT volume into a synthetic radiograph, and the CT's 3D labels into matching 2D segmentation labels. — **Figure 2.** A camera at a shared position ray-casts the CT volume into a synthetic radiograph and its 3D labels into perfectly aligned 2D segmentation labels.

Results

Across nine real-radiograph datasets held out from training, FleXray achieves the highest macro Dice among generalist methods in every comparison (9 of 9)—covering lungs, ribs, peripheral bones, spine, and pelvis. Qualitative predictions remain coherent across variation in anatomy, field of view, image polarity, projection, and acquisition modality.

Benchmark figure showing FleXray scores and segmentation examples across nine held-out real X-ray datasets against FluoroSAM, TotalSegmentator2D, PAXray, and task-specific networks. — **Figure 3.** Held-out benchmark performance against FluoroSAM, TotalSegmentator2D, PAXray, and task-specific networks.

Application: scoliosis measurement

Anatomical masks can support quantitative workflows beyond segmentation itself. On the AASCE scoliosis challenge, FleXray derives vertebral endplate orientation from predicted masks and yields a measurable angle for 261 of 262 images, reaching a mean absolute error of 7.1° for the major Cobb angle. Under the same protocol, PAXray reaches 22.8° MAE and TotalSeg2D 16.4°.

Completed scoliosis panels showing major Cobb angle agreement plots and normal, minor, moderate, and severe spine examples with reference and predicted endplate orientations. — **Figure 4.** Major Cobb angle estimation on AASCE. Reference orientations in green; mask-derived FleXray orientations in ultraviolet.

Resources

The release includes inference, the training data engine, online DRR rendering, label protocols, experiment configuration, and launch tools.

Limitations

FleXray currently targets conventional, non-dental and non-mammographic radiographs. Its public output space covers anatomy rather than pathology; laterality, uncertainty estimation, and broader clinical validation remain future work. The release is intended for research use.

Acknowledgements

This work was supported by the National Science Foundation Graduate Research Fellowship Program, Quanta Computer Inc., and the NIH, with grants R01 EB033773 and S10 OD038222. Compute provided by the Massachusetts Life Sciences Center (MLSC).

Citation

If you use FleXray or build on its methods in your work, please cite the associated paper:

@software{butoi2026flexray,
  title = {FleXray: Flexible Full-Body X-ray Segmentation},
  author = {Butoi, Victor Ion and Gopalakrishnan, Vivek and
            Guttag, John V. and Dalca, Adrian V. and Dey, Neel},
  year = {2026},
  license = {MIT},
  url = {https://github.com/VictorButoi/FleXray}
}