Scalable Bayesian data curation for next-generation radio experiments

Next-generation radio telescopes produce data volumes that preclude manual quality assessment, yet data curation remains essential for science. The new analysis still awaits peer review, but it already lays out the central claim clearly.

That matters because cosmology operates at the edge of what current instruments can measure, where systematic errors and model assumptions are never trivial. Small discrepancies between independent measurements have historically pointed toward missing physics rather than simple calibration errors, and the ongoing tension in the Hubble constant is a live example of how a persistent disagreement between methods can reshape the theoretical landscape. Each new dataset that approaches this territory with independent systematics adds real information to a problem that has resisted easy resolution for more than a decade. We present a general, fully automatic Bayesian anomaly-detection method for radio science experiments in which data curation is performed inside the inference: a latent anomaly. Implemented in JAX with GPU-accelerated inference, the pipeline assigns probabilistic data-curation scores without prior knowledge and requires no thresholds, manual inspection.

We demonstrate the method on the Radio Experiment for the Analysis of Cosmic Hydrogen (REACH), applying it to 4655 observations (one year of REACH data). The pipeline assigns scores across time and frequency, enabling identification of the optimal observations to carry forward into scientific inference while reducing the risk that.

In doing so, it simultaneously recovers weather-driven systematics, instrument-component drifts, and narrow-band radio-frequency interference, while revealing complex dependencies. This turns data curation from an external manual bottleneck into autonomous, inference-level infrastructure for the Square Kilometre Array era.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. ArXiv is committed to these values and only works with partners that adhere to them.

The relevance goes beyond one dataset because even small shifts in measured parameters can matter when the field is testing the limits of the standard cosmological model. The Lambda-CDM framework describes the observable universe with remarkable economy, but its success rests on two components, dark matter and dark energy, whose physical nature remains entirely unknown. Any credible measurement that tightens or loosens the constraints on those components moves the entire theoretical enterprise forward, regardless of whether the immediate result looks dramatic on its own terms.

Have an idea for a project that will add value for arXiv's community.

Because this is still a preprint, the result should be read with genuine interest and proportionate caution. Peer review is not a guarantee of correctness, but it is a process that forces authors to respond to technical criticism from specialists who have no stake in a particular outcome. Preprints that survive that process, often with substantive revisions, emerge with a stronger evidential base than the version that first appeared. Until that stage is complete, the responsible reading keeps uncertainty explicitly visible rather than treating the claims as established findings.

The next step is to see whether the effect survives when independent surveys, different calibration strategies and tighter control of systematic uncertainties enter the picture. Programmes such as Euclid, DESI and the Rubin Observatory will deliver datasets over the next several years that cover the same parameter space with largely independent methods. If the current signal persists through those tests, its theoretical implications will become impossible to set aside. Until peer review and independent follow-up address those open questions, skepticism is not a failure of appreciation for the work; it is part of how science decides what to keep.

Source