The proof page

Reproducible proof, not marketing.

The numbers below are measured, sourced, and scoped. Where a figure can be rerun, you can rerun it: the live demo is one click away, and every Rosa job leaves a hash-verifiable manifest behind it.

The headline results

Real-world bias removal

~75-90% reduction in a recidivism model's measured racial disparity, training on Rosa-debiased COMPAS instead of the raw data: 0.54 before, 0.09 after - a 6× reduction Pre-conditioned COMPAS, downstream model · methodology

COMPAS is the canonical real-world example of racial bias encoded in data, made famous by ProPublica's 2016 investigation. This is a demonstration on the public dataset, not a deployment at any named organisation. The figure is the race-disparity in a downstream recidivism model's predictions - measured the way the Demonstrating Rosa white paper measures it - when the model is trained on Rosa-debiased data instead of the raw data. You can reproduce it in the Customer Portal.

No fairness/accuracy trade-off (controlled experiment)

0.747 downstream R² with Rosa debiasing the data, then a model trained on it Controlled synthetic credit-risk experiment · methodology
0.626 R² for the same model alone, on the original data Controlled synthetic credit-risk experiment · methodology
0.569 R² for the model with a parity constraint, the standard alternative Controlled synthetic credit-risk experiment · methodology

R² (R-squared) measures how well a model's predictions fit reality; higher is better. Fairness was added and accuracy improved, while the parity-constraint approach paid the usual accuracy price. This is the proof behind "no fairness/accuracy trade-off". It does not mean every dataset improves on every metric.

At production scale

95.2% bias reduction at production scale: dataset bias 0.8745 before, 0.0421 after, on a synthetic capacity run at full production width At-scale capacity run · methodology

On the same run, a model trained on the Rosa-debiased data beat the biased-data model on both fairness and generalisation when both were tested against the same unbiased ground truth. The downstream comparison is directional evidence, not a guarantee for your dataset.

Data integrity

bit-for-bit distribution preservation in training output, to float64 CSV precision; inference output preserved to approximately 1e-6 Validation suite, distribution-preservation checks · methodology
0.30 to 0.04 standardised gender bias in heart-disease risk estimates for healthy patients (0.20 to 0.05 for patients with disease); after Rosa the model assigned men and women equal estimates Demonstrating Rosa white paper, Cleveland heart-disease dataset · methodology

How we measure it

The only honest way to prove that debiasing improves downstream fairness is to test against a known fair ground truth. Rosa's synthetic-population method works like this:

  1. Build a synthetic population with a known, fair ground-truth outcome.
  2. Inject bias into the features, the way real-world data acquires it (directly and through proxies).
  3. Train two identical models: one on the biased data, one on the Rosa-debiased data.
  4. Evaluate both against the same unbiased holdout, the ground truth neither model saw.

The model trained on Rosa-debiased data comes out fairer, and generalises at least as well, because it learned the real structure instead of the injected prejudice. That is the method behind the controlled figures above. Naively comparing model accuracy on biased data versus debiased data is not valid (the rank-mapping changes individual values), which is why every comparison here goes through the shared unbiased ground truth.

The COMPAS figure is different in kind: it is a real-world public dataset, not synthetic. The measured quantity is the race-disparity in a downstream recidivism model's predictions (the white paper's method): a model trained on Rosa-debiased COMPAS is ~75-90% less racially disparate than the same model trained on the raw data. Raw COMPAS has heavy-tailed count columns that are pre-conditioned first (the standard "cap or transform extreme outliers" data prep) so the run completes reliably; the cleaned dataset is available in the Customer Portal, so you can reproduce the figure there.

The earliest published evidence is the Demonstrating Rosa white paper (PDF), which ran the same exercise on five real-world datasets (recidivism, absenteeism, heart disease, school funding, community crime) and measured the bias in a simple model's estimates before and after Rosa. The paper itself raises the caveat that motivates the method above: accuracy measured on biased data cannot fairly judge a debiased model, which is exactly why Rosa's modern validation tests both models against a shared unbiased ground truth. The paper describes the original web release of Rosa; today's service is univariate (one protected attribute per run) and runs at the Customer Portal.

Provenance: the Fair Adversarial Network technology and this white paper were originally developed and published by illumr Ltd, London. The Rosa intellectual property has since been acquired and is now operated and developed independently. The paper is reproduced here unaltered, as published, so the audit trail is complete.

Scope, stated plainly

  • Univariate. One protected attribute per run, in this phase.
  • Residual bias is dataset-specific. The reduction you see depends on your data's proxy strength and structure. Rosa reports the residual for your data in your manifest; we do not promise a single universal number.
  • Rosa removes what it can detect. If it cannot statistically measure a bias signal, it declines to "debias" it, and says so.
  • "No fairness/accuracy trade-off" means Rosa does not force the degradation that parity-constraint methods do. It does not mean every dataset improves on every metric.
  • Well-conditioned data. Rosa works best on well-conditioned, sensibly-scaled numeric features. Columns with extreme outliers or heavy skew can cause a training run to fail - and it fails safely: the run is aborted and Rosa never returns altered or garbage data. If a run fails that way, Rosa will direct you to pre-scale or drop the extreme-outlier columns and resubmit.

This box is the point. A proof page you can trust is one that tells you where the edges are.

Run it yourself

Test 1 in the Customer Portal debiases the Apple Card gender-bias scenario in your browser in one click, and hands you the debiased CSV, the PDF report, and the Run Manifest. Every figure on this page comes from a method you can inspect or a run you can reproduce.

The COMPAS recidivism figure is reproducible too in the Customer Portal with race as the protected attribute.

Open the Customer Portal How the engine works

Bring your own dataset. Keep the evidence.

Start free Talk to us

Free to try. Reproducible. See the methodology above.