FleXray: Flexible Full-Body
X-ray Segmentation

Victor Ion Butoi · Vivek Gopalakrishnan · John V. Guttag · Adrian V. Dalca · Neel Dey

Massachusetts Institute of Technology · Massachusetts General Hospital · Harvard Medical School

Code Models + Data Tutorial MCP Paper soon
python -m pip install flexray

FleXray is a foundation model that segments anatomy from standard X-rays across diverse body regions, views, and acquisition settings. Trained on real radiographs, physics-based CT-to-DRR projections, and generated FluXray images unified under a shared 62-label protocol, it predicts 60 anatomical structures and outperforms existing generalist baselines on diverse real X-ray datasets.

Try FleXray in your browser
Collage of radiographs from head, chest, abdomen, and wrist; the top row shows input X-rays and the bottom row shows colored FleXray anatomy predictions.
FleXray predictions on held-out radiographs across body regions and projections. Top: input X-rays. Bottom: predicted anatomy.

Interactive demo

Try FleXray in your browser

Upload a radiograph and FleXray segments every visible structure — entirely on your device. The model runs in your browser through WebGPU (or WebAssembly as a fallback), so your image never leaves your computer.

Research demo — not a medical device. Outputs are for illustration only and must not be used for clinical or diagnostic purposes.

The demo needs to be served over HTTP — opening index.html directly from disk cannot load the model. From the repository root run .venv/bin/python -m http.server 8000 and open http://localhost:8000/docs/website/index.html#demo.

Motivation

More than 2.3 billion conventional X-ray examinations are acquired each year, yet public pixel-level annotations cover only a narrow set of views and structures. Every radiograph collapses 3D anatomy into a 2D projection: overlapping organs, variable angles, scanner settings, field of view, and patient morphology can make even the same body region look radically different.

Labeled CT volumes offer a way out. Segmented CT scans turn scarce 3D annotations into many anatomically correct 2D training views.

Stacked bar chart showing 2.28 billion annual X-ray examinations versus 403 million CT studies, next to a full-body anatomy map colored by the relative availability of labeled X-rays.
Figure 1. X-rays are acquired far more often than CT (2.28B vs. 403M studies per year), yet publicly available segmentation labels cover some X-ray anatomy substantially better than other regions.

Method: online CT-to-DRR rendering

FleXray ray-casts labeled CT volumes into digitally reconstructed radiographs (DRRs). Geometry, framing, and per-structure attenuation vary online, so a single subject becomes a stream of distinct paired images and masks—a new view at every training step. Randomly sampled cameras expose the model to anatomy from standard and non-standard acquisition angles.

  1. Load the CT volume, dense native labels, and physical-space affine.
  2. Sample detector geometry, view angle, field of view, and anatomy-centered crops.
  3. Ray-cast image intensities and masks into the common model label space.
Animation of X-ray rays traveling from a source through an abdominal and pelvic CT volume to a detector. Animation of the same X-ray geometry ray-casting labeled abdominal and pelvic anatomy to a detector.
Figure 2. A camera at a shared position ray-casts the CT volume into a synthetic radiograph and its 3D labels into perfectly aligned 2D segmentation labels.

Method: fluxifying DRRs

Raw DRRs are geometrically faithful but visually pristine—no film grain, no detector artifacts, no burned-in annotations. FleXray closes this gap by fluxifying its DRRs: the FluXray generative model translates each clean rendering into a realistic radiograph with sensor noise, exposure variation, collimation, markers, and annotations, while preserving the underlying anatomy—so training images look like the real X-rays the model will see at test time.

Figure 3. Raw DRRs (left of the divider) and the same views fluxified (right): FluXray adds realistic film texture, exposure variation, and burned-in annotations while keeping anatomy intact. Drag the divider to compare.

Results

We evaluate FleXray on eight real X-ray datasets left out of training, spanning hand and wrist, elbow, lower limb, hip and pelvis, spine, ribs, adult and pediatric chest, and both radiography and fluoroscopy. Against released generalist baselines (PAXray, TotalSegmentator2D, FluoroSAM), FleXray reaches an equal-dataset mean Dice of 0.875 and ranks first or ties for first on every dataset, significantly better than every zero-shot baseline on six of the eight. Despite never training on these datasets, it approaches dataset-specific nnU-Nets trained on each dataset’s own labels (0.875 vs. 0.914) and exceeds them on DeepFluoro (0.915 vs. 0.866). Predictions stay anatomically coherent under variation in field of view, projection angle, image polarity, acquisition modality, and patient population.

Benchmark figure with one row per held-out dataset: bar charts of per-image mean Dice for FleXray, FluoroSAM, TotalSegmentator2D, and PAXray with a dashed dataset-specific nnU-Net reference, beside an example X-ray, its ground-truth segmentation, and each method's prediction overlay.
Figure 4. Across eight held-out X-ray datasets FleXray is competitive with or exceeds existing baselines. Left: per-image mean Dice with 95% confidence intervals; dashed line and band: a dataset-specific nnU-Net trained in-domain. Right: an example X-ray, ground truth, and thresholded prediction overlays per dataset.

Application: scoliosis measurement

Scoliosis severity is monitored with the Cobb angle, measured manually on radiographs with roughly 3–5° variability within an observer and 6–7° between observers. Without any task-specific training, FleXray derives each vertebra’s end-plate orientation from the centerline of its predicted mask and computes the major Cobb angle. On the AASCE MICCAI 2019 challenge test set, FleXray yields a measurable curve on every image (0 of 218 failures) and the lowest error of all methods: a mean absolute error of 5.66°, below inter-observer variability, versus 7.86° for TotalSegmentator2D, 25.5° for PAXray, and 28.0° for FluoroSAM. 56% of estimates fall within 5° of the reference and 85% within 10°.

Left: a spinal X-ray with predicted vertebra masks, the spinal centerline, and end-plate orientations used to compute the major Cobb angle. Right: box plots of absolute major Cobb-angle error for FleXray, TotalSegmentator2D, PAXray, and FluoroSAM split by low, moderate, and severe reference curves, with failure counts above each box.
Figure 5. Left: predicted vertebra masks and mask-derived end-plate orientations yield the major Cobb angle. Right: absolute error by reference severity; labels give unmeasurable images per method; dashed line: 5° intra-observer variability.

Application: 2D/3D registration

Interventional specialties plan on 3D CT but navigate with live 2D fluoroscopy. Aligning the two, known as 2D/3D registration, is a brittle optimization with a small capture radius. We add a bidirectional Chamfer loss between FleXray’s segmentation of the X-ray and the projected CT structures to xvr, an image-based iterative registration pipeline; the term supplies informative gradients even when the rendered DRR and the X-ray do not overlap. On 213 held-out DeepFluoro X-rays, image-based registration from a manual frontal initialization fails on 86% of frames (median error 335 mm); adding the FleXray term lowers the median to 1.2 mm and the failure rate to 13%. From a foundation-model initialization it recovers every one of the eight catastrophic failures (7.0% → 3.3%, worst case 21 mm) without changing accuracy on frames that already registered.

Left: a grid of DeepFluoro X-rays for a standard pose initialization and an XVR initialization, showing the X-ray, the initial pose, the XVR result, and the XVR with FleXray Chamfer result with projected landmarks and their mean target registration error. Right: two log-scale strip plots of mean target registration error per X-ray, one per initialization, comparing the initial pose, XVR, and XVR with the FleXray Chamfer term; a dashed line marks the 10 mm success threshold and gray lines connect the same X-ray across methods.
Figure 6. Left: example registrations from a standard pose initialization (top) and an XVR initialization (bottom), with projected landmarks and the mean target registration error of each estimated pose. Right: mean target registration error per X-ray for each initialization and method; dashed line: 10 mm success threshold; gray trajectories link the same X-ray across methods.

Application: finetuning on new targets

As a foundation model, FleXray also transfers to targets outside its training vocabulary. We replace its output head and finetune on 1–100% of two new datasets, fine-grained pediatric pelvic anatomy in hip dysplasia (MTDDH) and bone tumors (BTXRD), against nnU-Net trained from scratch. Initialization from FleXray matters most where labels are scarcest: with 1% of MTDDH (6 patients) it reaches 0.896 mean Dice versus 0.867, and with 10% of BTXRD it finds 54% of tumors at one false positive per image versus 21%; nnU-Net needs five times as many labeled images to match. The gap closes as supervision grows, and a specialist trained on abundant labels remains competitive.

Learning curves of finetuned FleXray versus nnU-Net against the fraction of labeled training data: MTDDH mean Dice and per-label Dice, BTXRD lesion sensitivity overall and by tumor-size quartile, and a row of cropped radiographs showing detected tumors at each training budget.
Figure 7. Finetuned FleXray versus nnU-Net from random initialization against the fraction of labeled training data: MTDDH mean and per-label Dice (a, b), BTXRD lesion sensitivity overall and by tumor size (c, d), and detections at every budget for one test tumor (e).

Resources

The release includes inference, the training data engine, online DRR rendering, label protocols, experiment configuration, launch tools, and an MCP server for AI clients. Models and data are hosted on Hugging Face: the VictorButoi/flexray repository holds the flagship weights together with the FluXray-proportion ensemble members that the demo’s High and X-High modes average, and the flexray-data release repackages every redistributable real X-ray source in the FleXray label protocol, includes the generated FluXray database, and documents how to obtain the sources we cannot redistribute.

Limitations

FleXray currently targets conventional, non-dental and non-mammographic radiographs. Its public output space covers anatomy rather than pathology; laterality, uncertainty estimation, and broader clinical validation remain future work. The release is intended for research use.

Acknowledgements

This work was supported by the National Science Foundation Graduate Research Fellowship Program, Quanta Computer Inc., and the NIH, with grants R01 EB033773 and S10 OD038222. Compute provided by the Massachusetts Life Sciences Center (MLSC).

Citation

If you use FleXray or build on its methods in your work, please cite the associated paper: 

@software{butoi2026flexray,
  title = {FleXray: Flexible Full-Body X-ray Segmentation},
  author = {Butoi, Victor Ion and Gopalakrishnan, Vivek and
            Guttag, John V. and Dalca, Adrian V. and Dey, Neel},
  year = {2026},
  license = {MIT},
  url = {https://github.com/VictorButoi/FleXray}
}