Reproducible proof, not marketing.

The only honest way to prove that debiasing improves downstream fairness is to test against a known fair ground truth. Rosa's synthetic-population method works like this:

1. Build a synthetic population with a known, fair ground-truth outcome.
2. Inject bias into the features, the way real-world data acquires it (directly and through proxies).
3. Train two identical models: one on the biased data, one on the Rosa-debiased data.
4. Evaluate both against the same unbiased holdout, the ground truth neither model saw.

The model trained on Rosa-debiased data comes out fairer, and generalises at least as well, because it learned the real structure instead of the injected prejudice. That is the method behind the controlled figures above. Naively comparing model accuracy on biased data versus debiased data is not valid (the rank-mapping changes individual values), which is why every comparison here goes through the shared unbiased ground truth.

The COMPAS figure is different in kind: it is a real-world public dataset, not synthetic. The measured quantity is the false-positive-rate gap by race in a downstream recidivism model's predictions (ProPublica's own disparity metric): a model trained on Rosa-debiased COMPAS narrows that gap by about 81% (mean, range 54-100% across 10 independent Rosa runs), with a demographic-parity-gap reduction of about 90%, versus the same model trained on the raw data, on a held-out test split. This is ProPublica's two-year cohort with no pre-conditioning - Rosa debiases its heavy-tailed count columns directly.

COMPAS is deliberately the hard, weak-signal case, and we show the honest cost rather than bury it: the debiased model's overall accuracy falls by about 14 points, toward the majority-class baseline, because the raw model was leaning heavily on race - so removing race's influence takes much of its ranking power with it. That is the trade-off Rosa is built to make visible and measurable, not the exception it hides. And it is specific to weak-signal data: on the credit experiment above, where the predictive signal is genuine, debiasing costs nothing - the Rosa-trained model actually generalises better. Two honest results: where a model earns its accuracy, fairness is free; where it was riding on a protected attribute, Rosa tells you the price. The dataset and configs are available in the Customer Portal, so you can reproduce the figure there.

The earliest published evidence is the Demonstrating Rosa white paper (PDF), which ran the same exercise on five real-world datasets (recidivism, absenteeism, heart disease, school funding, community crime) and measured the bias in a simple model's estimates before and after Rosa. The paper itself raises the caveat that motivates the method above: accuracy measured on biased data cannot fairly judge a debiased model, which is exactly why Rosa's modern validation tests both models against a shared unbiased ground truth. The paper describes the original web release of Rosa; today's service is univariate (one protected attribute per run) and runs at the Customer Portal.

Provenance: the Fair Adversarial Network technology and this white paper were originally developed and published by illumr Ltd, London. The Rosa intellectual property has since been acquired and is now operated and developed independently. The paper is reproduced here unaltered, as published, so the audit trail is complete.