Decision tree + profile scheme on 200 simulated SO SAT data sets: 100 CMB + noise realizations (seeds 8000–8099; r log-uniform in [10−3, 0.03], 10 with r = 0), each analysed with PySM d9s4 (constant spectral indices: the true model has no moments) and with one PySM sky drawn among d9/d10/d12 × s4/s5/s7. nside 128, no E modes in the signal, hit-weighted noise (pessimistic goal), covariance from Gaussian simulations with foregrounds fitted on d1s1. Code: BBSure branch r0_pruning (commit 0bd6188) with BBPower Bsure. r in units of 10−3; pull = (r − rtrue)/σ, σ the standard deviation of the posterior.
Decision tree and profile scheme
The flowchart run on every set. Resr0: each round adds the most significant family with r = 0 and prunes only its new spectra with r = 0 (M1,0); then M1,r (pruned with r free) and M1,0 are compared with M0,r; families discarded in a round are not retested. Borderline choices are repeated with up to 3 PolyChord runs and flagged as undecided. Click to zoom.
Main results (profile scheme)
Bias on r: mean pull +0.19 ± 0.11 for d9s4 (width 1.11) and +0.55 ± 0.11 for the random skies (width 1.12). With the same CMB + noise, the random skies give r higher by +0.44 ± 0.08 σ than d9s4: the residual bias comes from the foreground complexity, mostly d10s7 (+0.99) and d12s5 (+0.94).
Coverage: the true r is inside the 68% interval in 54% (d9s4) and 57% (random) of the sets; |pull| > 2 in 7 d9s4 and 12 random sets (about 5 expected per 100).
Conclusions: 50 detections / 50 upper limits for d9s4, 43 / 57 for random. Wrong conclusions: 3 d9s4 upper limits below the true r (sets 025, 033, 066) and 2 random (set 016: upper limit below the true r; set 000, d12s5: detection with r = 0). No false detection at r = 0 in d9s4 (0/10), 1/10 in random.
Sensitivity: every set with r ≥ 0.01 is a detection in d9s4 (29/29); 25/29 in the random skies.
Foreground model: 86 of the 100 random sets end with at least one moment spectrum. 12 d9s4 sets adopt a moment that is not in the sky (mostly βs×βs, with ln B between 0.1 and 3.8); 7 of them carry an undecided flag. M1,0 is preferred over M0,r in 18 d9s4 and 16 random sets.
Goodness of fit: PTE < 0.01 in 4 d9s4 and 7 random sets; the mean χ²/dof (1.05–1.07) is above 1 also for d9s4, a covariance normalisation issue.
Summary: tree → profile scheme
Bias = mean pull ± its error over the sets; width = standard deviation of the pulls (1 if σ is right). Coverage with the 68% central interval of each set (expected 68%). Detections and upper limits as concluded by each scheme.
Goodness of fit of the final model
Number of sets per PTE bin; tree: χ² of the tree model, profile scheme: χ² at r = r*. With ~450 degrees of freedom the mean χ²/dof over 100 sets should be 1.000 ± 0.007.
Paired difference (rrand − rd9s4)/σrand
Same CMB + noise in each pair, only the foregrounds change; profile scheme, with the tree and the number of pairs below.
Plots
Estimated r (median ± σ) against the true r; left: Bayesian tree, right: profile scheme; top: d9s4, bottom: random skies. Blue: detection, orange: upper limit. Dashed: weighted linear fit rest = a + b rtrue.
Pull histograms, d9s4: all sets, upper limits, detections. Filled: profile scheme; black outline: tree; solid curve: N(0,1).Pull histograms, random skies, same layout.
r set by set, d9s4, profile scheme: median and 68% interval (detections), 95% upper limit (orange bar); the star is the true r. Red circle and band: wrong conclusion (true r above the 95% upper limit, or detection with true r = 0).r set by set, random skies, profile scheme. Black star: true r; green diamond: d9s4 result of the same set (same CMB + noise), so the distance from the diamond shows the effect of the foregrounds alone. Red circle and band: wrong conclusion. Sky under each set.same plots for the Bayesian tree
d9s4, tree.Random skies, tree; star: true r, diamond: d9s4 of the same set.
Mean pull by sky
The PTE flag here is set at 0.01. At 0.05 it fired in about a quarter of the sets, also in d9s4 where Mnone is the true model (PTE < 0.05 in 23%, expected 5%): the χ² is systematically high, a normalisation (covariance) issue rather than a sign of bad foreground modelling. At 0.01 it fires in 4% of the d9s4 and 7% of the rand sets.
All sets
PTEfinal model PTE < 0.01≈mommoment adopted or rejected with |ln B| < 2σ≈cutpruning undecided≈besttwo best models of the profile scheme within 2σM₁,₀ vs M₀,r: M₁,₀ ✓preferred M₁,₀ ≈✓preferred, |ln B| < 2σ M₁,₀ ≈✗rejected, |ln B| < 2σdetyellow cells: detection with true r < 3×10⁻³UL ✗ / det ✗ red cells: wrong conclusion: true r above the 95% upper limit, or detection with true r = 0