FleXray: Universal Clinical
X-ray Segmentation

Victor Ion Butoi · Vivek Gopalakrishnan · John V. Guttag · Adrian V. Dalca · Neel Dey

Massachusetts Institute of Technology · Massachusetts General Hospital · Harvard Medical School

Paper Code Models Data Tutorial MCP
python -m pip install flexray

FleXray is a foundation model that segments anatomy from standard X-rays across diverse body regions, views, and acquisition settings. Trained on physics-based CT-to-DRR projections, real radiographs, and generative-model-enhanced DRRs, it predicts 60 anatomical structures, outperforms existing generalist baselines on diverse real X-ray datasets, and unlocks unique downstream workflows.

Try FleXray in your browser
Collage of neck, lumbar spine, chest, and hip X-ray sequences, from left to right; the top row shows input X-rays and the bottom row shows FleXray ensemble predictions with test-time augmentation. The chest overlays show lungs, heart, and liver.
FleXray predictions using a five-model ensemble and 16-sample test-time augmentation. Left to right: neck, lumbar spine, chest, and hip. Top: input X-rays. Bottom: predicted anatomy. Chest overlays show lungs, heart, and liver.

Interactive demo

Try FleXray in your browser

Upload a radiograph and FleXray segments every visible structure — entirely on your device. The model runs in your browser through WebGPU (or WebAssembly as a fallback), so your image never leaves your computer.

Research demo — not a medical device. Outputs are for illustration only and must not be used for clinical or diagnostic purposes.

The demo needs to be served over HTTP — opening index.html directly from disk cannot load the model. From the repository root run .venv/bin/python -m http.server --directory site 8000 and open http://localhost:8000/index.html#demo.

Motivation

More than 2.3 billion conventional X-ray examinations are acquired each year, yet public pixel-level annotations cover only a narrow set of views and structures. Every radiograph collapses 3D anatomy into a 2D projection: overlapping organs, variable angles, scanner settings, field of view, and patient morphology can make even the same body region look radically different.

Labeled CT volumes offer a way out. Segmented CT scans turn scarce 3D annotations into many anatomically correct 2D training views.

Stacked bar chart showing 2.28 billion annual X-ray examinations versus 403 million CT studies, next to a full-body anatomy map colored by the relative availability of labeled X-rays.
Figure 1. X-rays are acquired far more often than CT (2.28B vs. 403M studies per year), yet publicly available segmentation labels cover some X-ray anatomy substantially better than other regions.

Method: online CT-to-DRR rendering

FleXray ray-casts labeled CT volumes into digitally reconstructed radiographs (DRRs). Geometry, framing, and per-structure attenuation vary online, so a single subject becomes a stream of distinct paired images and masks—a new view at every training step. Randomly sampled cameras expose the model to anatomy from standard and non-standard acquisition angles.

  1. Load the CT volume, dense native labels, and physical-space affine.
  2. Sample detector geometry, view angle, field of view, and anatomy-centered crops.
  3. Ray-cast image intensities and masks into the common model label space.
Top: an X-ray source, an abdominal and pelvic CT volume, and a detector. Bottom: the same geometry with labeled anatomy. Both panels animate together to show ray-casting into paired radiographs and segmentation labels.
Figure 2. A camera at a shared position ray-casts the CT volume into a synthetic radiograph and its 3D labels into perfectly aligned 2D segmentation labels.

Method: fluxifying DRRs

Raw DRRs are geometrically faithful but visually pristine—no film grain, no detector artifacts, no burned-in annotations. FleXray closes this gap by fluxifying its DRRs with Black Forest Labs’ off-the-shelf FLUX.2 image-editing model. It translates each clean rendering into a realistic radiograph with sensor noise, exposure variation, collimation, markers, and annotations, while preserving the underlying anatomy—so training images look like the real X-rays the model will see at test time.

Figure 3. Raw DRRs (left of the divider) and the same views fluxified (right): FLUX.2 adds realistic film texture, exposure variation, and burned-in annotations while keeping anatomy intact. Drag the divider to compare.

Results

We evaluate FleXray on eight real X-ray datasets left out of training, spanning hand and wrist, elbow, lower limb, hip and pelvis, spine, ribs, adult and pediatric chest, and both radiography and fluoroscopy. Against released generalist baselines (PAXray, TotalSegmentator2D, FluoroSAM), FleXray reaches an equal-dataset mean Dice of 0.875 and the highest mean Dice among the generalist baselines on seven of eight datasets. Despite never training on these datasets, it approaches dataset-specific nnU-Nets trained on each dataset’s own labels (0.875 vs. 0.914) and exceeds them on DeepFluoro (0.916 vs. 0.866). Predictions stay anatomically coherent under variation in field of view, projection angle, image polarity, acquisition modality, and patient population.

Benchmark figure with one row per held-out dataset: bar charts of per-image mean Dice for FleXray, FluoroSAM, TotalSegmentator2D, and PAXray with a dashed dataset-specific nnU-Net reference, beside an example X-ray, its ground-truth segmentation, and each method's prediction overlay.
Figure 4. Across eight held-out X-ray datasets FleXray is competitive with or exceeds existing baselines. Left: per-image mean Dice with 95% confidence intervals; dashed line and band: a dataset-specific nnU-Net trained in-domain. Right: an example X-ray, ground truth, and thresholded prediction overlays per dataset.

Application: scoliosis measurement

Scoliosis severity is monitored with the Cobb angle, measured manually on radiographs with roughly 3–5° variability within an observer and 6–7° between observers. Without any task-specific training, FleXray derives each vertebra’s end-plate orientation from the centerline of its predicted mask and computes the major Cobb angle. On the AASCE MICCAI 2019 challenge test set, FleXray yields a measurable curve on every image (0 of 218 failures) and the lowest error of all methods: a mean absolute error of 5.13°, below inter-observer variability, versus 6.38° for TotalSegmentator2D, 24.9° for PAXray, and 31.8° for FluoroSAM. 61% of estimates fall within 5° of the reference and 88% within 10°.

Left: four stages from the spinal X-ray through predicted vertebra masks and centerline-derived orientations to the major Cobb angle. Right: box plots of absolute major Cobb-angle error for FleXray, TotalSegmentator2D, PAXray, and FluoroSAM split by low, moderate, and severe reference curves, with failure counts and mean errors above each box.
Figure 5. Left: predicted vertebra masks and mask-derived end-plate orientations yield the major Cobb angle. Right: absolute error by reference severity; labels give unmeasurable images per method and mean errors over measurable images; dashed line: 5° intra-observer variability.

Application: 2D/3D registration

Interventional specialties plan on 3D CT but navigate with live 2D fluoroscopy. Aligning the two, known as 2D/3D registration, is a brittle optimization with a small capture radius. We add a bidirectional Chamfer loss between FleXray’s segmentation of the X-ray and the projected CT structures to xvr, an image-based iterative registration pipeline; the term supplies informative gradients even when the rendered DRR and the X-ray do not overlap. On 213 held-out DeepFluoro X-rays, image-based registration from a manual frontal initialization fails on 86% of frames (median error 335 mm); adding the FleXray term lowers the median to 1.2 mm and the failure rate to 13%. From a foundation-model initialization, it lowers the failure rate from 7.0% to 3.3%, increasing successful registrations from 198 to 206 of 213 frames while keeping the median error near 1 mm. Success is defined as a mean target registration error below 10 mm.

Left: DeepFluoro X-rays for standard pose and foundation model initializations, showing the target X-ray, initial pose, XVR baseline, and XVR with the FleXray Chamfer term, with projected landmarks and mean target registration errors. Right: log-scale box plots and individual errors comparing the three stages for each initialization, with median errors, success counts, a dashed 10 mm success threshold, and gray trajectories connecting selected outlier frames across methods.
Figure 6. Left: example registrations from a standard pose initialization (top) and a foundation model initialization (bottom), with projected landmarks and the mean target registration error of each estimated pose. Right: mean target registration error per X-ray for each initialization and method, with median errors and success counts; dashed line: 10 mm success threshold; gray trajectories link selected outlier frames across methods.

Application: finetuning on new targets

As a foundation model, FleXray also transfers to targets outside its training vocabulary. We replace its output head and finetune on 1–100% of two new datasets, fine-grained pediatric pelvic anatomy in hip dysplasia (MTDDH) and bone tumors (BTXRD), against nnU-Net trained from scratch. Initialization from FleXray matters most where labels are scarcest: with 1% of MTDDH (6 patients) it reaches 0.898 mean Dice versus 0.869, and with 10% of BTXRD it reaches 23.2% detection AP50 versus 7.4%. AP50 measures average precision at a bounding-box intersection-over-union threshold of 0.5; this evaluation contains 354 annotated tumors in 281 positive radiographs. At full supervision, MTDDH Dice converges to 0.923 versus 0.921, while BTXRD AP50 reaches 53.1% versus 47.7%.

Six panels comparing finetuned FleXray with nnU-Net: MTDDH overall and regional Dice learning curves, an X-ray with regional segmentation contours at 1% training data, BTXRD detection AP50 overall and by lesion-size quartile, and tumor detections on the same radiograph at each training budget.
Figure 7. Finetuned FleXray versus nnU-Net from random initialization against the fraction of labeled training data: MTDDH mean and regional Dice (a, b), regional segmentation examples at 1% supervision (c), BTXRD detection AP50 overall and by tumor size (d, e), and detections at every budget for one test tumor (f). Shaded bands show 95% confidence intervals; red overlays mark ground-truth segmentations in the examples.

Resources

The release includes inference, the training data engine, online DRR rendering, label protocols, experiment configuration, launch tools, and an MCP server for AI clients. Models and data are hosted on Hugging Face: the VictorButoi/flexray repository holds the flagship weights together with the FluXray-proportion ensemble members that the demo’s High and X-High modes average, and the flexray-data release repackages every redistributable real X-ray source in the FleXray label protocol, includes the generated FluXray database, and documents how to obtain the sources we cannot redistribute.

Limitations

FleXray currently targets conventional, non-dental and non-mammographic radiographs. Its public output space covers anatomy rather than pathology; laterality, uncertainty estimation, and broader clinical validation remain future work. The release is intended for research use.

Acknowledgements

This work was supported by the National Science Foundation Graduate Research Fellowship Program, Quanta Computer Inc., and the NIH, with grants R01 EB033773 and S10 OD038222. Compute provided by the Massachusetts Life Sciences Center (MLSC).

Citation

If you use FleXray or build on its methods in your work, please cite the associated paper: 

@misc{butoi2026flexray,
      title={FleXray: Universal Clinical X-ray Segmentation},
      author={Victor Ion Butoi and Vivek Gopalakrishnan and John V. Guttag and Adrian V. Dalca and Neel Dey},
      year={2026},
      eprint={2609.26756},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.26756},
}