BBSure validation (seeds 8000 and 9000, signal without E modes generated exactly): 400 simulated sets

Parameters and priors: base model

componentparameterpriornote
CMBrflat [−0.02, 0.06]negative values allowed, so that the posterior is not cut at 0
Alensfixed 1
DustβdGaussian 1.54 ± 0.11
Tdfixed 20 K; flat [10, 40] K with the dT family
AdBBflat [0, 100]Dℓ at 353 GHz, ℓ = 80
αdBBflat [−2, 0.5]power law in ℓ, pivot ℓ = 80
SynchrotronβsGaussian −3.0 ± 0.3
AsBBflat [0, 8]Dℓ at 23 GHz, ℓ = 80
αsBBflat [−2, 0.5]
Dust × synchεdsflat [−1, 1]correlation of the dust and synchrotron amplitudes

Parameters and priors: moment spectra

familyspectrum amplitude Aprior
dAd×βdflat [−5, 5]
βd×βdflat [0, 5]
sAs×βsflat [−2, 2]
βs×βsflat [0, 1]
dTAd×Tdflat [−0.5, 0.5]
Td×Tdflat [0, 0.025]
βd×Td (with d)flat [−0.25, 0.25]
d2Ad×βd2flat [−5, 5]
βd×βd2flat [−2.5, 2.5]
βd2×βd2flat [0, 2.5]
s2As×βs2flat [−1, 1]
βs×βs2flat [−0.5, 0.5]
βs2×βs2flat [0, 0.5]
ds1Ad×βsflat [−2, 2]
βd×Asflat [−2, 2]
ds2βd×βsflat [−1, 1]

Each spectrum is A (ℓ/80)α in Dℓ at the pivot frequency of its component (units of √(AdAs) for the dust × synch terms; Td moments in K−1); every slope α flat in [−2, 0.5]. Constraints: the covariance of the moment maps is positive semi-definite at every ℓ (auto-spectra ≥ 0, cross terms bounded by Cauchy–Schwarz), and each second-order auto-spectrum is at most 10 × (first order)2 / (component).

How the pipeline works

Decision tree and profile scheme

flowchart of the decision tree, of Res_r0 and of the profile scheme, with the flags
The flowchart run on every set, step by step: left the search with r free (Reswr, steps 1–11), centre the search with r = 0 (Resr0, steps 13–18), bottom the 3σ test of the tree (19), right in violet the profile scheme (20–24). The badges mark where each flag of the table can be raised. Click to zoom.

Main results (profile scheme)

Summary: tree → profile scheme

Bias = mean pull ± its error over the sets; width = standard deviation of the pulls (1 if σ is right). Coverage with the 68% central interval of each set (expected 68%). Detections and upper limits as concluded by each scheme.

Goodness of fit of the final model

Number of sets per PTE bin; tree: χ² of the tree model, profile scheme: χ² at r = r*. With ~450 degrees of freedom the mean χ²/dof over 100 sets should be 1.000 ± 0.007.

Mean pull by sky: Bayesian tree

Mean pull by sky: profile scheme

Pull = (r − rtrue)/σ, mean ± its error over the sets of each sky, number of sets below; the d9 × s4 cell is the d9s4 sets.

Plots

estimated r against true r
Estimated r (median ± σ) against the true r; left: Bayesian tree, right: profile scheme; top: d9s4, bottom: random skies. Blue: detection, orange: upper limit. Dashed: weighted linear fit rest = a + b rtrue.
pull histograms, d9s4
Pull histograms, d9s4: all sets, upper limits, detections. Filled: profile scheme; black outline: tree; solid curve: N(0,1).
pull histograms, random skies
Pull histograms, random skies, same layout.
r set by set, d9s4
r set by set, d9s4, profile scheme: median and 68% interval (detections), 95% upper limit (orange bar); the star is the true r. Red circle and band: wrong conclusion (true r above the 95% upper limit, or detection with true r = 0); yellow: detection with true r < 3×10⁻³, as in the table.
r set by set, random skies, with d9s4 of the same set
r set by set, random skies, profile scheme. Black star: true r; green diamond: r of d9s4 of the same set (same CMB + noise) from the first fit of the tree, without moments (M0,r, done in every set), so the distance from the diamond shows the effect of the foregrounds alone. Red circle and band: wrong conclusion; yellow: detection with true r < 3×10⁻³. Sky under each set.
same plots for the Bayesian tree
r set by set, d9s4, tree
d9s4, tree.
r set by set, random skies, tree
Random skies, tree; star: true r, diamond: d9s4 of the same set, fit without moments.

The PTE flag here is set at 0.01. At 0.05 it fires in about a fifth of the sets, also in d9s4 where Mnone is the true model (PTE < 0.05 in about 20%, expected 5%): the χ² is systematically high, a normalisation (covariance) issue rather than a sign of bad foreground modelling. At 0.01 it fires in 4% of the d9s4 and 7% of the rand sets.

All sets

PTEfinal model PTE < 0.01 ≈a decision taken on inconclusive evidence: |ln B − threshold| < 1 (Jeffreys scale, Trotta 2008), or < 2σ of the sampler after the reruns model with r = 0 against M₀,r: r=0 ✓✓preferred by a lot r=0 ✓preferred by a little r=0 ✗not preferred, by a little (a little: |ln B| < 1, or < 2σ of the sampler) detyellow cells: detection with true r < 3×10⁻³ UL ✗ / det ✗ red cells: wrong conclusion: true r above the 95% upper limit, or detection with true r = 0