BBSure validation run (seed 8000-8099, signal without E modes generated exactly): 200 simulated sets
10 November 2026 · N. Krachmalnicoff, L. Vacher, G. Galloni, C. Ranucci, A. Carones
Decision tree + profile scheme on 200 simulated SO SAT data sets.
100 CMB + noise realizations (seeds 8000–8099); r log-uniform in [10−3, 0.03], 10 sets with r = 0.
Each realization analysed twice: with PySM d9s4 (uniform spectral indices, so the true model has no moments) and with one PySM sky drawn among d9/d10/d12 × s4/s5/s7.
nside 128, no E modes in the signal, hit-weighted noise (pessimistic goal); six SO SAT bands (LF1–2, MF1–2, UHF1–2). The E modes are never generated: the CMB has EE = 0 in the input spectra, and the PySM foregrounds lose their E modes at nside 512, in the harmonic downgrade to 128 (no E → B leakage from removing E at nside 128).
Likelihood: Hamimeche–Lewis on the BB cross-spectra of all band pairs, ℓ = 30–250. Fiducial and covariance: the BBPower model with r = 0, Alens = 1 and power-law foregrounds fitted on PySM d1s1; block-diagonal covariance from 500 Gaussian simulations of that same model (seeds 5002–5501), with the Hartlap correction of its inverse (blocks of 21 spectra: × 0.956).
Code: BBSure branch noee-exact (commit 05447ad) with BBPower Bsure-hartlap (commit b4abbaf): vectorized BBPower likelihood; evidences from nautilus (1000 live points, 5000 effective samples), on the prior restricted by the moment constraints; undecided flag at |ln B − threshold| < 1.
Computational cost: mean 19 minutes per set including maps and spectra (median 16, max 54) on 4 cores, 32 trees per Perlmutter node.
r in units of 10−3; pull = (r − rtrue)/σ, with σ the standard deviation of the posterior.
Parameters and priors: base model
component
parameter
prior
note
CMB
r
flat [−0.02, 0.06]
negative values allowed, so that the posterior is not cut at 0
Alens
fixed 1
Dust
βd
Gaussian 1.54 ± 0.11
Td
fixed 20 K; flat [10, 40] K with the dT family
AdBB
flat [0, 100]
Dℓ at 353 GHz, ℓ = 80
αdBB
flat [−2, 0.5]
power law in ℓ, pivot ℓ = 80
Synchrotron
βs
Gaussian −3.0 ± 0.3
AsBB
flat [0, 8]
Dℓ at 23 GHz, ℓ = 80
αsBB
flat [−2, 0.5]
Dust × synch
εds
flat [−1, 1]
correlation of the dust and synchrotron amplitudes
Parameters and priors: moment spectra
family
spectrum amplitude A
prior
d
Ad×βd
flat [−5, 5]
βd×βd
flat [0, 5]
s
As×βs
flat [−2, 2]
βs×βs
flat [0, 1]
dT
Ad×Td
flat [−0.5, 0.5]
Td×Td
flat [0, 0.025]
βd×Td (with d)
flat [−0.25, 0.25]
d2
Ad×βd2
flat [−5, 5]
βd×βd2
flat [−2.5, 2.5]
βd2×βd2
flat [0, 2.5]
s2
As×βs2
flat [−1, 1]
βs×βs2
flat [−0.5, 0.5]
βs2×βs2
flat [0, 0.5]
ds1
Ad×βs
flat [−2, 2]
βd×As
flat [−2, 2]
ds2
βd×βs
flat [−1, 1]
Each spectrum is A (ℓ/80)α in Dℓ at the pivot frequency of its component (units of √(AdAs) for the dust × synch terms; Td moments in K−1); every slope α flat in [−2, 0.5]. Constraints: the covariance of the moment maps is positive semi-definite at every ℓ (auto-spectra ≥ 0, cross terms bounded by Cauchy–Schwarz), and each second-order auto-spectrum is at most 10 × (first order)2 / (component).
How the pipeline works
Start simple. Fit the model without moments (dust and synchrotron with a fixed SED, r free). If its χ² is good and r is compatible with 0 at 1σ, stop with an upper limit.
Look for moments in the residuals, first with r at its best fit (Reswr: first order in βd and βs), then with r = 0 (Resr0: also Td, second order in β, dust × synchrotron), so that a moment can show up even when r has absorbed it. A family is a candidate if it lowers the AIC of the residual fit.
Keep a moment only if the evidence wants it. Each candidate is fitted with all the parameters free, its spectra compatible with 0 are pruned, and it is adopted if its Bayesian evidence beats the current model (ln B > 0). Clearly worse candidates are rejected from the ML fits without the evidence run; borderline comparisons are repeated with other sampler seeds. After an adoption the search starts again from the new model.
Moments instead of r. If a model with r = 0 and more moments is preferred over the current one, the search stops: the Bayesian result stays the 3σ test on the current model, and the case is flagged.
Conclusion of the tree. 0 outside the central 3σ of the posterior of r → r detection; otherwise upper limit (95%, r ≥ 0).
Profile scheme. The final model of the tree and the moment families found with r = 0 are compared again, each at its own r (minimum of its profile likelihood); the posterior of r from the profile of the winner gives the final conclusion with the same 3σ rule.
Flags mark what to look at, without changing the result: low PTE, a borderline decision (≈), a model with r = 0 preferred or nearly preferred (r=0).
Decision tree and profile scheme
The flowchart run on every set, step by step: left the search with r free (Reswr, steps 1–11), centre the search with r = 0 (Resr0, steps 13–18), bottom the 3σ test of the tree (19), right in violet the profile scheme (20–24). The badges mark where each flag of the table can be raised. Click to zoom.
Main results (profile scheme)
Bias on r: mean pull +0.06 ± 0.11 for d9s4 (width 1.05) and +0.67 ± 0.11 for the random skies (width 1.10); with the profile scheme +0.03 ± 0.11 and +0.54 ± 0.11. Fitted with the model without moments (M0,r, the first fit of the tree), d9s4 gives a mean pull +0.19 ± 0.10 (width 0.99). With the same CMB + noise, the random skies give r higher by +0.53 ± 0.09 σ than this d9s4 fit (profile +0.41 ± 0.08), mostly from d12s5 (+1.06), d12s7 (+0.98) and d10s7 (+0.68); d9s7 is unbiased (−0.12).
Coverage: the true r is inside the 68% interval in 63% (d9s4) and 58% (random) of the sets (profile scheme: 64% and 61%); |pull| > 2 in 6 d9s4 and 15 random sets (profile: 6 and 13; about 5 expected per 100).
Conclusions: 46 detections / 54 upper limits for d9s4, 45 / 55 for random (profile scheme: 46 / 54 and 41 / 59). Wrong conclusions: 3 d9s4 upper limits below the true r in the tree (sets 018, 033, 066), 2 in the profile scheme (033, 066); in the random skies sets 016 (d9s5) and 033 (d9s7), upper limits below the true r, and, in the tree only, set 000 (d12s5: detection with r = 0). No false detection at r = 0 in d9s4 (0/10); 1/10 in random for the tree, 0/10 for the profile scheme.
Sensitivity: every set with r ≥ 0.01 is a detection in d9s4 (29/29); 25/29 in the random skies.
Foreground model: 85 of the 100 random sets end with at least one moment spectrum. 13 d9s4 sets adopt a moment that is not in the sky (12 βs×βs); 11 of them carry an undecided flag. M1,0 is preferred over M0,r in 15 d9s4 and 10 random sets. The profile scheme keeps the model of the tree in every set.
Goodness of fit: χ²/dof 1.003 for d9s4 and 1.021 for random; PTE < 0.01 in 2 sets of each sky. The PTE is uniform for d9s4 (KS p = 0.90), not for the random skies (p = 0.003): the moment models do not fully describe some of them.
Flags:≈ (a decision within ln B = 1 of its threshold): 33 d9s4 and 39 random.
Summary: tree → profile scheme
Bias = mean pull ± its error over the sets; width = standard deviation of the pulls (1 if σ is right). Coverage with the 68% central interval of each set (expected 68%). Detections and upper limits as concluded by each scheme.
Goodness of fit of the final model
Number of sets per PTE bin; tree: χ² of the tree model, profile scheme: χ² at r = r*. With ~450 degrees of freedom the mean χ²/dof over 100 sets should be 1.000 ± 0.007.
Mean pull by sky: Bayesian tree
Mean pull by sky: profile scheme
Pull = (r − rtrue)/σ, mean ± its error over the sets of each sky, number of sets below; the d9 × s4 cell is the d9s4 sets.
Plots
Estimated r (median ± σ) against the true r; left: Bayesian tree, right: profile scheme; top: d9s4, bottom: random skies. Blue: detection, orange: upper limit. Dashed: weighted linear fit rest = a + b rtrue.
Pull histograms, d9s4: all sets, upper limits, detections. Filled: profile scheme; black outline: tree; solid curve: N(0,1).Pull histograms, random skies, same layout.
r set by set, d9s4, profile scheme: median and 68% interval (detections), 95% upper limit (orange bar); the star is the true r. Red circle and band: wrong conclusion (true r above the 95% upper limit, or detection with true r = 0); yellow: detection with true r < 3×10⁻³, as in the table.r set by set, random skies, profile scheme. Black star: true r; green diamond: r of d9s4 of the same set (same CMB + noise) from the first fit of the tree, without moments (M0,r, done in every set), so the distance from the diamond shows the effect of the foregrounds alone. Red circle and band: wrong conclusion; yellow: detection with true r < 3×10⁻³. Sky under each set.same plots for the Bayesian tree
d9s4, tree.Random skies, tree; star: true r, diamond: d9s4 of the same set, fit without moments.
The PTE flag here is set at 0.01. At 0.05 it fires in about a fifth of the sets, also in d9s4 where Mnone is the true model (PTE < 0.05 in about 20%, expected 5%): the χ² is systematically high, a normalisation (covariance) issue rather than a sign of bad foreground modelling. At 0.01 it fires in 4% of the d9s4 and 7% of the rand sets.
All sets
PTEfinal model PTE < 0.01≈a decision taken on inconclusive evidence: |ln B − threshold| < 1 (Jeffreys scale, Trotta 2008), or < 2σ of the sampler after the rerunsmodel with r = 0 against M₀,r: r=0 ✓✓preferred by a lot r=0 ✓preferred by a little r=0 ✗not preferred, by a little (a little: |ln B| < 1, or < 2σ of the sampler)detyellow cells: detection with true r < 3×10⁻³UL ✗ / det ✗ red cells: wrong conclusion: true r above the 95% upper limit, or detection with true r = 0