Binning a ground schedule¶
In this notebook, we take an observing schedule from toast_ground_sim.py and translate it into a depth map.
In [1]:
# Capture C++ output in the jupyter cells
%reload_ext wurlitzer
First, we need a focalplane. If one does not already exist, TOAST pipelines includes a tool for generating mock hexagonal focalplanes:
In [2]:
! toast_fake_focalplane.py --help
usage: toast_fake_focalplane.py [-h] [--minpix MINPIX] [--out OUT]
[--fwhm FWHM] [--fwhm_sigma FWHM_SIGMA]
[--fov FOV] [--psd_fknee PSD_FKNEE]
[--psd_NET PSD_NET] [--psd_alpha PSD_ALPHA]
[--psd_fmin PSD_FMIN]
[--bandcenter_ghz BANDCENTER_GHZ]
[--bandcenter_sigma BANDCENTER_SIGMA]
[--bandwidth_ghz BANDWIDTH_GHZ]
[--bandwidth_sigma BANDWIDTH_SIGMA]
[--random_seed RANDOM_SEED]
Simulate fake hexagonal focalplane.
optional arguments:
-h, --help show this help message and exit
--minpix MINPIX minimum number of pixels to use
--out OUT Root name of output pickle file
--fwhm FWHM beam FWHM in arcmin
--fwhm_sigma FWHM_SIGMA
Relative beam FWHM distribution width
--fov FOV Field of View in degrees
--psd_fknee PSD_FKNEE
Detector noise model f_knee in Hz
--psd_NET PSD_NET Detector noise model NET in K*sqrt(sec)
--psd_alpha PSD_ALPHA
Detector noise model slope
--psd_fmin PSD_FMIN Detector noise model f_min in Hz
--bandcenter_ghz BANDCENTER_GHZ
Band center frequency [GHz]
--bandcenter_sigma BANDCENTER_SIGMA
Relative band center distribution width
--bandwidth_ghz BANDWIDTH_GHZ
Bandwidth [GHz]
--bandwidth_sigma BANDWIDTH_SIGMA
Relative bandwidth distribution width
--random_seed RANDOM_SEED
Random number generator seed for randomized detector
parameters
Here we create a focalplane with 10-degree FOV and a mininimum of 20 pixels:
In [3]:
! toast_fake_focalplane.py \
--minpix 20 \
--out focalplane \
--fwhm 30 \
--fov 10 \
--psd_fknee 5e-2 \
--psd_NET 1e-3 \
--psd_alpha 1 \
--psd_fmin 1e-5
TOAST INFO: using 37 pixels (74 detectors)
The actual focalplane ends up having 37 pixels, instead of the minimum of 20. This is because regular packing of the hexagon is quantized. Notice that the final name of the focalplane is focalplane_37.pkl. We'll need the name to run the simulation script. We also need a schedule file. This may already exist if you previously ran the simscan_ground notebook, but we'll re-create it here just in case:
In [4]:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-01 12:00:00" \
--patch-coord C \
--patch small_patch,1,40,-40,44,-44 \
--out schedule.txt
TOAST INFO: Adding patch "small_patch" TOAST INFO: Rectangular format TOAST INFO: Global timer: toast_ground_schedule: 0.13 seconds (1 calls)
We will use the versatile ground simulation pipeline, toast_ground_sim.py, to bin the map. It will be covered in detail in lesson 7 so here we simply write out a parameter file:
In [5]:
%%writefile bin_schedule.par
--sample-rate
10.0
--scan-rate
0.3
--scan-accel
10.0
--nside
64
--focalplane
focalplane_37.pkl
--schedule
schedule.txt
--out
out
--simulate-noise
--freq
100
--no-destripe
--no-binmap
--hits
--wcov
Overwriting bin_schedule.par
Now we run the pipeline with multiple MPI processes that divide into processing groups:
In [6]:
! toast_ground_sim.py @bin_schedule.par
<toast.Environment
Source code version = 2.3.5.dev1474
Logging level = INFO
Handling enabled for 0 signals:
Max threads = 12
MPI build enabled
MPI runtime enabled
>
TOAST INFO: Running with 1 processes at 2020-01-02 12:05:52.196730
TOAST INFO: All parameters:
TOAST INFO: group_size = None
TOAST INFO: do_daymaps = False
TOAST INFO: do_seasonmaps = False
TOAST INFO: debug = False
TOAST INFO: scan_rate = 0.3
TOAST INFO: scan_accel = 10.0
TOAST INFO: sun_angle_min = 30.0
TOAST INFO: schedule = schedule.txt
TOAST INFO: weather = None
TOAST INFO: timezone = 0
TOAST INFO: sample_rate = 10.0
TOAST INFO: coord = C
TOAST INFO: split_schedule = None
TOAST INFO: sort_schedule = True
TOAST INFO: hwp_rpm = None
TOAST INFO: hwp_step_deg = None
TOAST INFO: hwp_step_time_s = None
TOAST INFO: elevation_noise_a = 0
TOAST INFO: elevation_noise_b = 0
TOAST INFO: nside = 64
TOAST INFO: nside_submap = 16
TOAST INFO: single_precision_pointing = False
TOAST INFO: common_flag_mask = 1
TOAST INFO: flush = False
TOAST INFO: apply_polyfilter = False
TOAST INFO: poly_order = 0
TOAST INFO: apply_groundfilter = False
TOAST INFO: ground_order = 0
TOAST INFO: simulate_atmosphere = False
TOAST INFO: focalplane_radius_deg = None
TOAST INFO: atm_verbosity = 0
TOAST INFO: atm_lmin_center = 0.01
TOAST INFO: atm_lmin_sigma = 0.001
TOAST INFO: atm_lmax_center = 10.0
TOAST INFO: atm_lmax_sigma = 10.0
TOAST INFO: atm_gain = 3e-05
TOAST INFO: atm_zatm = 40000.0
TOAST INFO: atm_zmax = 200.0
TOAST INFO: atm_xstep = 10.0
TOAST INFO: atm_ystep = 10.0
TOAST INFO: atm_zstep = 10.0
TOAST INFO: atm_nelem_sim_max = 1000
TOAST INFO: atm_wind_dist = 500.0
TOAST INFO: atm_z0_center = 2000.0
TOAST INFO: atm_z0_sigma = 0.0
TOAST INFO: atm_T0_center = 280.0
TOAST INFO: atm_T0_sigma = 10.0
TOAST INFO: atm_cache = atm_cache
TOAST INFO: simulate_noise = True
TOAST INFO: apply_gainscrambler = False
TOAST INFO: gain_sigma = 0.01
TOAST INFO: madam_prefix = toast
TOAST INFO: madam_iter_max = 1000
TOAST INFO: madam_precond_width = 100
TOAST INFO: madam_precond_width_min = None
TOAST INFO: madam_precond_width_max = None
TOAST INFO: madam_baseline_length = 10000.0
TOAST INFO: madam_baseline_order = 0
TOAST INFO: madam_noisefilter = False
TOAST INFO: madam_parfile = None
TOAST INFO: madam_allreduce = False
TOAST INFO: madam_concatenate_messages = True
TOAST INFO: destripe = False
TOAST INFO: write_binmap = False
TOAST INFO: write_hits = True
TOAST INFO: write_wcov = True
TOAST INFO: write_wcov_inv = True
TOAST INFO: conserve_memory = True
TOAST INFO: input_map = None
TOAST INFO: simulate_sky = True
TOAST INFO: pysm_model = None
TOAST INFO: pysm_apply_beam = True
TOAST INFO: pysm_precomputed_cmb_K_CMB = None
TOAST INFO: pysm_mpi_comm = group
TOAST INFO: ground_map = None
TOAST INFO: ground_nside = 128
TOAST INFO: ground_fwhm_deg = 10
TOAST INFO: ground_lmax = 256
TOAST INFO: ground_scale = 0.001
TOAST INFO: ground_power = -1
TOAST INFO: simulate_ground = False
TOAST INFO: tidas = None
TOAST INFO: spt3g = None
TOAST INFO: MC_start = 0
TOAST INFO: MC_count = 1
TOAST INFO: outdir = out
TOAST INFO: focalplane = focalplane_37.pkl
TOAST INFO: freq = 100
TOAST INFO: Parsed parameters: 0.01 seconds (1 calls)
TOAST INFO: Load 23 (sub)scans in schedule.txt: 0.01 seconds (1 calls)
TOAST INFO: Loading schedule(s): 0.01 seconds (1 calls)
TOAST INFO: Loading focalplanes: 0.00 seconds (1 calls)
TOAST INFO: Group # 0 has 23 observations.
TOAST INFO: Simulated scans: 0.18 seconds (1 calls)
TOAST INFO: Expanding pointing
TOAST INFO: Pointing generation: 1.88 seconds (1 calls)
TOAST INFO: Processing frequency 100.0GHz 1 / 1, MC = 0
TOAST INFO: Simulating noise
TOAST INFO: Simulate noise: 3.15 seconds (1 calls)
TOAST INFO: Making maps
TOAST INFO: Mapping toast_100_telescope_all_time_all
TOAST INFO: OpMadam: 100.00 % of samples are included in valid intervals.
TOAST INFO: Collect period ranges: 0.05 seconds (1 calls)
TOAST INFO: Collect dataset dimensions: 0.05 seconds (1 calls)
TOAST INFO: Stage time: 0.04 seconds (1 calls)
Memory usage after staging time
total : 62.251 GB
available : 56.208 GB
percent : 9.700 %
used : 4.825 GB
free : 50.283 GB
active : 8.694 GB
inactive : 2.230 GB
buffers : 321.047 MB
cached : 6.830 GB
shared : 741.910 MB
slab : 474.160 MB
Node has 0.428 GB allocated in TOAST TOD caches and 0.001 GB in Madam caches (0.430 GB total) after staging time
TOAST INFO: Stage signal: 0.13 seconds (1 calls)
Memory usage after staging signal
total : 62.251 GB
available : 56.123 GB
percent : 9.800 %
used : 4.909 GB
free : 50.198 GB
active : 8.779 GB
inactive : 2.230 GB
buffers : 321.047 MB
cached : 6.830 GB
shared : 742.090 MB
slab : 474.191 MB
Node has 0.428 GB allocated in TOAST TOD caches and 0.085 GB in Madam caches (0.514 GB total) after staging signal
TOAST INFO: Stage pixels: 0.23 seconds (1 calls)
Memory usage after staging pixels
total : 62.251 GB
available : 56.081 GB
percent : 9.900 %
used : 4.951 GB
free : 50.157 GB
active : 8.822 GB
inactive : 2.230 GB
buffers : 321.047 MB
cached : 6.830 GB
shared : 742.090 MB
slab : 474.160 MB
Node has 0.355 GB allocated in TOAST TOD caches and 0.128 GB in Madam caches (0.482 GB total) after staging pixels
TOAST INFO: Stage pixel weights: 0.22 seconds (1 calls)
Memory usage after staging pixel weights
total : 62.251 GB
available : 55.954 GB
percent : 10.100 %
used : 5.078 GB
free : 50.030 GB
active : 8.948 GB
inactive : 2.230 GB
buffers : 321.047 MB
cached : 6.830 GB
shared : 742.090 MB
slab : 474.160 MB
Node has 0.102 GB allocated in TOAST TOD caches and 0.254 GB in Madam caches (0.356 GB total) after staging pixel weights
TOAST INFO: Stage all data: 0.72 seconds (1 calls)
TOAST INFO: Collect PSD info: 0.00 seconds (1 calls)
Memory usage just before calling libmadam.destripe
total : 62.251 GB
available : 55.954 GB
percent : 10.100 %
used : 5.078 GB
free : 50.030 GB
active : 8.948 GB
inactive : 2.230 GB
buffers : 321.047 MB
cached : 6.830 GB
shared : 742.090 MB
slab : 474.160 MB
OMP: 1 tasks with 12 procs per node, 12 threads per task.
Program MADAM
Destriping of CMB data with a noise filter
Version 3.7
Examining periods
Flagged 0 samples on 0 periods that had less than 0.100% of unflagged samples
Total number of samples (single detector): 152900
Zero-weight fraction (all detectors): 0.000 %
Flagged fraction (all detectors): 0.829 %
Total number of intervals: 23
Max number of samples per task: 152900
Initializing parameters
Adjusting noise weights using noise spectra
Polarized detectors present: Will produce polarization maps
noise_weights_from_psd = T
radiometers = T
mode_detweight = 0
istart_mission = 0
nosamples_tot = 152900
Initializing parallelization
ntasks = 1 Number or processes
nosamples_tot = 152900 Total samples
nosamples_proc_max = 152900 Samples/process
MCMode = F
write_cut = F
basis_func = Legendre Destriping function basis
basis_order = 0 Destriping function order
bin_subsets = F
ntasks = 1 Number of processes
nthreads = 12 Number of threads per process
info = 3 Screen output level
fsample = 10.0000 Sampling frequency (Hz)
nmap = 3 Polarization included
ncc = 6 Independent wcov elements
Input files:
nside_map = 64 Healpix resolution (output map)
nside_cross = 32 Healpix resolution (destriping)
nside_submap = 16 Submap resolution
concatenate_messages = T use mpi_alltoallv to communicate
reassign_submaps = T minimize communication by reassigning submaps
pixmode_map = 2 Pixel rejection criterion (output map)
pixmode_cross = 2 Pixel rejection criterion (destriping)
pixlim_map = 1.00000E-02 Pixel rejection limit (output map)
pixlim_cross = 1.00000E-03 Pixel rejection limit (destriping)
Standard mode
psdlen = 1000000 Length of requested noise PSD
psd_downsample = 10 PSD downsampling factor
kfirst = F First destriping OFF
cglimit = 1.00000E-12 Iteration convergence limit
iter_min = 3 Minimum number of iterations
iter_max = 1000 Maximum number of iterations
precond_width_min = 0 Min width of the preconditioner band matrix
precond_width_max = 0 Max width of the preconditioner band matrix
No preconditioning
flag_by_horn = F Flags are independent
mode_detweight = 0 Detector weighting mode: sigma from simulation file
time_unit = pp Time unit = pointing period
mission_time = 23 Mission length in time units
nosamples_tot = 152900 Total samples
= 4.2472 hours
Detectors available on the FIRST process and noise according to the FIRST period
detector sigma weight 1/sqrt(weight)
fake_00A 0.31780E-02 99010. 0.31780E-02
fake_00B 0.31780E-02 99010. 0.31780E-02
fake_01A 0.31780E-02 99010. 0.31780E-02
fake_01B 0.31780E-02 99010. 0.31780E-02
fake_02A 0.31780E-02 99010. 0.31780E-02
fake_02B 0.31780E-02 99010. 0.31780E-02
fake_03A 0.31780E-02 99010. 0.31780E-02
fake_03B 0.31780E-02 99010. 0.31780E-02
fake_04A 0.31780E-02 99010. 0.31780E-02
fake_04B 0.31780E-02 99010. 0.31780E-02
fake_05A 0.31780E-02 99010. 0.31780E-02
fake_05B 0.31780E-02 99010. 0.31780E-02
fake_06A 0.31780E-02 99010. 0.31780E-02
fake_06B 0.31780E-02 99010. 0.31780E-02
fake_07A 0.31780E-02 99010. 0.31780E-02
fake_07B 0.31780E-02 99010. 0.31780E-02
fake_08A 0.31780E-02 99010. 0.31780E-02
fake_08B 0.31780E-02 99010. 0.31780E-02
fake_09A 0.31780E-02 99010. 0.31780E-02
fake_09B 0.31780E-02 99010. 0.31780E-02
fake_10A 0.31780E-02 99010. 0.31780E-02
fake_10B 0.31780E-02 99010. 0.31780E-02
fake_11A 0.31780E-02 99010. 0.31780E-02
fake_11B 0.31780E-02 99010. 0.31780E-02
fake_12A 0.31780E-02 99010. 0.31780E-02
fake_12B 0.31780E-02 99010. 0.31780E-02
fake_13A 0.31780E-02 99010. 0.31780E-02
fake_13B 0.31780E-02 99010. 0.31780E-02
fake_14A 0.31780E-02 99010. 0.31780E-02
fake_14B 0.31780E-02 99010. 0.31780E-02
fake_15A 0.31780E-02 99010. 0.31780E-02
fake_15B 0.31780E-02 99010. 0.31780E-02
fake_16A 0.31780E-02 99010. 0.31780E-02
fake_16B 0.31780E-02 99010. 0.31780E-02
fake_17A 0.31780E-02 99010. 0.31780E-02
fake_17B 0.31780E-02 99010. 0.31780E-02
fake_18A 0.31780E-02 99010. 0.31780E-02
fake_18B 0.31780E-02 99010. 0.31780E-02
fake_19A 0.31780E-02 99010. 0.31780E-02
fake_19B 0.31780E-02 99010. 0.31780E-02
fake_20A 0.31780E-02 99010. 0.31780E-02
fake_20B 0.31780E-02 99010. 0.31780E-02
fake_21A 0.31780E-02 99010. 0.31780E-02
fake_21B 0.31780E-02 99010. 0.31780E-02
fake_22A 0.31780E-02 99010. 0.31780E-02
fake_22B 0.31780E-02 99010. 0.31780E-02
fake_23A 0.31780E-02 99010. 0.31780E-02
fake_23B 0.31780E-02 99010. 0.31780E-02
fake_24A 0.31780E-02 99010. 0.31780E-02
fake_24B 0.31780E-02 99010. 0.31780E-02
fake_25A 0.31780E-02 99010. 0.31780E-02
fake_25B 0.31780E-02 99010. 0.31780E-02
fake_26A 0.31780E-02 99010. 0.31780E-02
fake_26B 0.31780E-02 99010. 0.31780E-02
fake_27A 0.31780E-02 99010. 0.31780E-02
fake_27B 0.31780E-02 99010. 0.31780E-02
fake_28A 0.31780E-02 99010. 0.31780E-02
fake_28B 0.31780E-02 99010. 0.31780E-02
fake_29A 0.31780E-02 99010. 0.31780E-02
fake_29B 0.31780E-02 99010. 0.31780E-02
fake_30A 0.31780E-02 99010. 0.31780E-02
fake_30B 0.31780E-02 99010. 0.31780E-02
fake_31A 0.31780E-02 99010. 0.31780E-02
fake_31B 0.31780E-02 99010. 0.31780E-02
fake_32A 0.31780E-02 99010. 0.31780E-02
fake_32B 0.31780E-02 99010. 0.31780E-02
fake_33A 0.31780E-02 99010. 0.31780E-02
fake_33B 0.31780E-02 99010. 0.31780E-02
fake_34A 0.31780E-02 99010. 0.31780E-02
fake_34B 0.31780E-02 99010. 0.31780E-02
fake_35A 0.31780E-02 99010. 0.31780E-02
fake_35B 0.31780E-02 99010. 0.31780E-02
fake_36A 0.31780E-02 99010. 0.31780E-02
fake_36B 0.31780E-02 99010. 0.31780E-02
Output files
file_root = toast_100_telescope_all_time_all
file_hit = out/00000000/100/toast_100_telescope_all_time_all_hmap.fits
file_matrix = out/00000000/100/toast_100_telescope_all_time_all_wcov_inv.fits
file_wcov = out/00000000/100/toast_100_telescope_all_time_all_wcov.fits
binary_output = F
concatenate_binary = F
TOD memory min = 87.64 MB max = 87.64 MB total = 87.64 MB
Baseline memory min = 0.00 B max = 0.00 B total = 0.00 B
Pointing memory min = 215.81 MB max = 215.81 MB total = 215.81 MB
Basis function memory min = 0.00 B max = 0.00 B total = 0.00 B
Clock = 0.055 s
Total submaps = 3072 submap size = 16
Local submaps: min = 21 max = 21 mean = 21.00
Submap table memory min = 12.00 kB max = 12.00 kB total = 12.00 kB
local maps memory min = 32.91 kB max = 32.91 kB total = 32.91 kB
All2allv memory min = 36.77 kB max = 36.77 kB total = 36.77 kB
Map memory min = 4.88 MB max = 4.88 MB total = 4.88 MB
Building pixel matrices...
Counting hits...
Binning TOD...
Clock = 0.297 s
Finalization begins
Clock = 0.297 s
Writing pixel matrix...
Pixel matrix written in out/00000000/100/toast_100_telescope_all_time_all_wcov_inv.fits
Inverting pixel matrices...
214 pixels solved
48938 pixels unsolved
0 pixels had decoupled intensity
Constructing output map...
Destriped map:
Map 1 2 3
Std 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Mean 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Min 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Max 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Writing pixel matrix...
Pixel matrix written in out/00000000/100/toast_100_telescope_all_time_all_wcov.fits
Writing hits...
Hit count written in out/00000000/100/toast_100_telescope_all_time_all_hmap.fits
Clock = 0.375 s
MEMORY (MB):
Detector pointing min = 215.81 MB max = 215.81 MB total = 215.81 MB
TOD buffer min = 87.64 MB max = 87.64 MB total = 87.64 MB
Maps min = 4.88 MB max = 4.88 MB total = 4.88 MB
Baselines min = 0.92 kB max = 0.92 kB total = 0.92 kB
Basis functions min = 0.00 B max = 0.00 B total = 0.00 B
Noise filter min = 0.00 B max = 0.00 B total = 0.00 B
Preconditioner min = 0.00 B max = 0.00 B total = 0.00 B
Submap table min = 12.00 kB max = 12.00 kB total = 12.00 kB
Temporary maps min = 32.91 kB max = 32.91 kB total = 32.91 kB
All2All buffers min = 36.77 kB max = 36.77 kB total = 36.77 kB
CG work space min = 0.00 B max = 0.00 B total = 0.00 B
NCM min = 0.00 B max = 0.00 B total = 0.00 B
Total min = 308.40 MB max = 308.40 MB total = 308.40 MB
WALL-CLOCK TIME (s):
Initialization mean = 0.1 min = 0.1 max = 0.1
Building pixel matrices mean = 0.1 min = 0.1 max = 0.1
Binning TOD mean = 0.1 min = 0.1 max = 0.1
Finalization and output mean = 0.1 min = 0.1 max = 0.1
Total 0.4 s ( 0.00 CPU hours)
TOAST INFO: Unstage signal 1 / 1: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 1 / 1: 0.09 seconds (1 calls)
TOAST INFO: Unstage pixel weights 1 / 1: 0.11 seconds (1 calls)
TOAST INFO: Unstage all data: 0.20 seconds (1 calls)
TOAST INFO: Mapping toast_100_telescope_all_time_all: 1.42 seconds (1 calls)
TOAST INFO: Madam total: 1.42 seconds (1 calls)
TOAST INFO: Gather and dump timing info: 0.00 seconds (1 calls)
TOAST INFO: toast_ground_sim.py: 6.66 seconds (1 calls)
Let's examine the resulting hits and depth map
In [7]:
import matplotlib.pyplot as plt
%matplotlib inline
import healpy
hits = healpy.read_map("out/00000000/100/toast_100_telescope_all_time_all_hmap.fits")
hits[hits == 0] = healpy.UNSEEN
healpy.mollview(hits, unit="hits", title="Total hits")
healpy.graticule(22.5, verbose=False)
NSIDE = 64 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING
In [8]:
wcov = healpy.read_map("out/00000000/100/toast_100_telescope_all_time_all_wcov.fits")
wcov *= 1e12 # from K^2 to uK^2
wcov[wcov == 0] = healpy.UNSEEN
healpy.mollview(wcov, unit="$\mu$K$^2$", title="White noise variance", min=1e0, max=1e3)
healpy.graticule(22.5, verbose=False)
NSIDE = 64 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING
In [ ]: