Empirical test failures reflect current mathematical limitations between federated and centralised computations

[JHogenboom]

Describe the bug

Some empirical tests in tests/empirical/ may occasionally fail with the current repository versions. These failures are not due to implementation bugs but reflect inherent mathematical differences between federated and centralised computations.

• test_single_organisation_equivalence: May fail due to division-by-zero issues in quantile calculations for single-organisation scenarios. These failures do not affect multi-organisation federated scenarios.
• test_mixed_distribution_quantiles: May fail when organisations have very different data distributions. Federated quantiles mathematically differ from centralised ones in these cases.

This behaviour is observed with the current versions of this repository and has real-world implications for practitioners: federated quantile computations can yield results that diverge from centralised calculations, especially when data distributions are mixed across organisations.

To Reproduce

Steps to reproduce the behaviour:

1. Run the empirical tests in tests/empirical/ on the latest version of the repository.
2. Observe intermittent failures in the tests mentioned above, especially in edge cases of data distribution.

Expected behaviour

• Failures due to quantile calculations in the above scenarios are mathematical expectations, not code bugs.
• Basic statistics (mean, count, min, max) maintain equivalence between federated and centralised approaches.
• Standard deviation is considered correct within ±15% relative or ±0.5 absolute.

Additional context

• Federated quantiles with mixed distributions across organisations will naturally differ from centralised calculations, even in real-world deployments.
• Single-organisation scenarios generally work, but quantile computation may have edge cases.
• These issues reflect the mathematical properties of federated quantile computation and should be taken into account when interpreting results.
• Consider updating test documentation and tolerances to reflect these expectations.