Pipelines¶
TOAST pipelines are standalone Python scripts that apply one or more TOAST operators to existing or simulated data. They typically divide into two major parts:
- Creating the TOAST
dataobject - Applying TOAST operators to the
data
Experiments and simulations typically require specialized methods for creating the data object. This part is hard to generalize. For the subsequent data processing TOAST provides a number of convenience functions are kept in toast.pipeline_tools.
In [1]:
# Load common tools for all lessons
import sys
sys.path.insert(0, "..")
from lesson_tools import (
fake_focalplane
)
# Capture C++ output in the jupyter cells
%reload_ext wurlitzer
Brief example¶
We demonstrate these concepts by writing a very simple pipeline that creates its own observations, fills them with noise and applies a polynomial filter.
In [2]:
%%writefile my_first_pipeline.py
import sys
sys.path.insert(0, "..")
import argparse
import numpy as np
import toast
from toast.mpi import MPI
import toast.pipeline_tools
from lesson_tools import fake_focalplane
def parse_arguments(comm):
""" Declare and parse command line arguments
"""
parser = argparse.ArgumentParser(fromfile_prefix_chars="@")
# pipeline_tools provides arguments to supported operators
toast.pipeline_tools.add_noise_args(parser)
toast.pipeline_tools.add_polyfilter_args(parser)
# The pipeline may add its own arguments
parser.add_argument(
"--nsample",
required=False,
default=10000,
type=np.int,
help="Length of the simulation",
)
args = parser.parse_args()
return args
def create_observations(comm, args):
""" Create the TOAST data object using `args`
This method will look very different for different pipelines.
"""
focalplane = fake_focalplane(samplerate=args.sample_rate, fknee=1.0)
detnames = list(sorted(focalplane.keys()))
detquats = {x: focalplane[x]["quat"] for x in detnames}
tod = toast.tod.TODCache(None, detnames, args.nsample, detquats=detquats)
# Write some auxiliary data
tod.write_times(stamps=np.arange(args.nsample) / args.sample_rate)
tod.write_common_flags(flags=np.zeros(args.nsample, dtype=np.uint8))
for d in detnames:
tod.write_flags(detector=d, flags=np.zeros(args.nsample, dtype=np.uint8))
noise = toast.pipeline_tools.get_analytic_noise(args, comm, focalplane)
observation = {"tod" : tod, "noise" : noise, "name" : "obs-000", "id" : 0}
data = toast.Data(comm)
data.obs.append(observation)
return data
def main():
""" The `main` will instantiate and process the data.
"""
mpiworld, procs, rank, comm = toast.pipeline_tools.get_comm()
args = parse_arguments(comm)
data = create_observations(comm, args)
mc = 0
name = "signal"
tod = data.obs[0]["tod"]
det = tod.local_dets[0]
# Simulate noise, write out one detector
toast.pipeline_tools.simulate_noise(args, comm, data, mc, cache_prefix=name)
tod.local_signal(det, name).tofile("noise.npy")
# Apply polyfilter, write the filtered data
toast.pipeline_tools.apply_polyfilter(args, comm, data, cache_name=name)
tod.local_signal(det, name).tofile("filtered.npy")
if __name__ == "__main__":
main()
Overwriting my_first_pipeline.py
In [3]:
! python3 my_first_pipeline.py --help
<toast.Environment
Source code version = 2.3.5.dev1474
Logging level = INFO
Handling enabled for 0 signals:
Max threads = 12
MPI build enabled
MPI runtime enabled
>
TOAST INFO: Running with 1 processes at 2020-01-02 14:41:32.631583
usage: my_first_pipeline.py [-h] [--noise] [--simulate-noise] [--no-noise]
[--no-simulate-noise] [--sample-rate SAMPLE_RATE]
[--polyfilter] [--no-polyfilter]
[--poly-order POLY_ORDER]
[--common-flag-mask COMMON_FLAG_MASK]
[--nsample NSAMPLE]
optional arguments:
-h, --help show this help message and exit
--noise Add simulated noise
--simulate-noise Add simulated noise
--no-noise Do not add simulated noise
--no-simulate-noise Do not add simulated noise
--sample-rate SAMPLE_RATE
Detector sample rate (Hz)
--polyfilter Apply polynomial filter
--no-polyfilter Do not apply polynomial filter
--poly-order POLY_ORDER
Polynomial order for the polyfilter
--common-flag-mask COMMON_FLAG_MASK
Common flag mask
--nsample NSAMPLE Length of the simulation
In [4]:
! python3 my_first_pipeline.py --simulate-noise --polyfilter --poly-order 6 --nsample 10000
<toast.Environment Source code version = 2.3.5.dev1474 Logging level = INFO Handling enabled for 0 signals: Max threads = 12 MPI build enabled MPI runtime enabled > TOAST INFO: Running with 1 processes at 2020-01-02 14:41:35.075289 TOAST INFO: Creating noise model: 0.00 seconds (1 calls) TOAST INFO: Simulating noise TOAST INFO: Simulate noise: 0.04 seconds (1 calls) TOAST INFO: Polyfiltering signal TOAST INFO: Polynomial filtering: 0.63 seconds (1 calls)
In [5]:
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
noise = np.fromfile("noise.npy")
filtered = np.fromfile("filtered.npy")
plt.plot(noise, '.', label="noise")
plt.plot(noise - filtered, '-', label="polynomial")
plt.legend()
Out[5]:
<matplotlib.legend.Legend at 0x7f58dab3a390>
Satellite simulation pipeline¶
In [6]:
! toast_satellite_sim.py --help
<toast.Environment
Source code version = 2.3.5.dev1474
Logging level = INFO
Handling enabled for 0 signals:
Max threads = 12
MPI build enabled
MPI runtime enabled
>
TOAST INFO: Running with 1 processes at 2020-01-02 14:41:41.107556
usage: toast_satellite_sim.py [-h] [--group-size GROUP_SIZE] [--day-maps]
[--no-day-maps] [--season-maps]
[--no-season-maps] [--nside NSIDE]
[--nside-submap NSIDE_SUBMAP] [--coord COORD]
[--single-precision-pointing]
[--no-single-precision-pointing]
[--common-flag-mask COMMON_FLAG_MASK] [--flush]
[--tidas TIDAS] [--spt3g SPT3G] [--dipole]
[--no-dipole] [--dipole-mode DIPOLE_MODE]
[--dipole-solar-speed-kms DIPOLE_SOLAR_SPEED_KMS]
[--dipole-solar-gal-lat-deg DIPOLE_SOLAR_GAL_LAT_DEG]
[--dipole-solar-gal-lon-deg DIPOLE_SOLAR_GAL_LON_DEG]
[--pysm-model PYSM_MODEL] [--pysm-apply-beam]
[--no-pysm-apply-beam]
[--pysm-precomputed-cmb-K_CMB PYSM_PRECOMPUTED_CMB_K_CMB]
[--pysm-mpi-comm PYSM_MPI_COMM] [--sky]
[--simulate-sky] [--no-sky] [--no-simulate-sky]
[--MC-start MC_START] [--MC-count MC_COUNT]
[--noise] [--simulate-noise] [--no-noise]
[--no-simulate-noise]
[--sample-rate SAMPLE_RATE]
[--start-time START_TIME]
[--spin-period-min SPIN_PERIOD_MIN]
[--spin-angle-deg SPIN_ANGLE_DEG]
[--prec-period-min PREC_PERIOD_MIN]
[--prec-angle-deg PREC_ANGLE_DEG]
[--obs-time-h OBS_TIME_H] [--gap-h GAP_H]
[--obs-num OBS_NUM] [--hwp-rpm HWP_RPM]
[--hwp-step-deg HWP_STEP_DEG]
[--hwp-step-time-s HWP_STEP_TIME_S]
[--conviqt-sky-file CONVIQT_SKY_FILE]
[--conviqt-lmax CONVIQT_LMAX]
[--conviqt-fwhm CONVIQT_FWHM]
[--conviqt-beam-file CONVIQT_BEAM_FILE]
[--conviqt-beam-mmax CONVIQT_BEAM_MMAX]
[--conviqt-pxx] [--conviqt-dxx]
[--conviqt-order CONVIQT_ORDER]
[--conviqt-normalize-beam]
[--no-conviqt-normalize-beam]
[--conviqt-remove-monopole]
[--no-conviqt-remove-monopole]
[--conviqt-remove-dipole]
[--no-conviqt-remove-dipole]
[--conviqt-mpi-comm CONVIQT_MPI_COMM]
[--outdir OUTDIR] [--debug]
[--madam-prefix MADAM_PREFIX]
[--madam-iter-max MADAM_ITER_MAX]
[--madam-precond-width MADAM_PRECOND_WIDTH]
[--madam-precond-width-min MADAM_PRECOND_WIDTH_MIN]
[--madam-precond-width-max MADAM_PRECOND_WIDTH_MAX]
[--madam-baseline-length MADAM_BASELINE_LENGTH]
[--madam-baseline-order MADAM_BASELINE_ORDER]
[--madam-noisefilter]
[--madam-parfile MADAM_PARFILE]
[--madam-allreduce] [--no-madam-allreduce]
[--madam-concatenate-messages]
[--no-madam-concatenate-messages] [--destripe]
[--no-destripe] [--binmap] [--no-binmap]
[--hits] [--no-hits] [--wcov] [--no-wcov]
[--wcov-inv] [--no-wcov-inv] [--conserve-memory]
[--no-conserve-memory] [--zip] [--no-zip]
[--madam] [--no-madam] [--focalplane FOCALPLANE]
[--gain GAIN]
Simulate satellite boresight pointing and make a map.
optional arguments:
-h, --help show this help message and exit
--group-size GROUP_SIZE
Size of a process group assigned to an observation
--day-maps Enable daily maps
--no-day-maps Disable daily maps
--season-maps Enable season maps
--no-season-maps Disable season maps
--nside NSIDE Healpix NSIDE
--nside-submap NSIDE_SUBMAP
Number of sub pixels is 12 * nside-submap ** 2
--coord COORD Sky coordinate system [C,E,G]
--single-precision-pointing
Use single precision for pointing in memory.
--no-single-precision-pointing
Use double precision for pointing in memory.
--common-flag-mask COMMON_FLAG_MASK
Common flag mask
--flush Flush every print statement.
--tidas TIDAS Output TIDAS export path
--spt3g SPT3G Output SPT3G export path
--dipole Add simulated dipole
--no-dipole Do not add simulated dipole
--dipole-mode DIPOLE_MODE
Dipole mode is 'total', 'orbital' or 'solar'
--dipole-solar-speed-kms DIPOLE_SOLAR_SPEED_KMS
Solar system speed [km/s]
--dipole-solar-gal-lat-deg DIPOLE_SOLAR_GAL_LAT_DEG
Solar system speed galactic latitude [degrees]
--dipole-solar-gal-lon-deg DIPOLE_SOLAR_GAL_LON_DEG
Solar system speed galactic longitude[degrees]
--pysm-model PYSM_MODEL
Comma separated models for on-the-fly PySM simulation,
e.g. "s1,d6,f1,a2"
--pysm-apply-beam Convolve sky with detector beam
--no-pysm-apply-beam Do not convolve sky with detector beam.
--pysm-precomputed-cmb-K_CMB PYSM_PRECOMPUTED_CMB_K_CMB
Precomputed CMB map for PySM in K_CMBit overrides any
model defined in pysm_model"
--pysm-mpi-comm PYSM_MPI_COMM
MPI communicator used by the PySM operator, either
'rank' or 'group'
--sky Add simulated sky signal
--simulate-sky Add simulated sky
--no-sky Do not add simulated sky
--no-simulate-sky Do not add simulated sky
--MC-start MC_START First Monte Carlo noise realization
--MC-count MC_COUNT Number of Monte Carlo noise realizations
--noise Add simulated noise
--simulate-noise Add simulated noise
--no-noise Do not add simulated noise
--no-simulate-noise Do not add simulated noise
--sample-rate SAMPLE_RATE
Detector sample rate (Hz)
--start-time START_TIME
The overall start time of the simulation
--spin-period-min SPIN_PERIOD_MIN
The period (in minutes) of the rotation about the spin
axis
--spin-angle-deg SPIN_ANGLE_DEG
The opening angle (in degrees) of the boresight from
the spin axis
--prec-period-min PREC_PERIOD_MIN
The period (in minutes) of the rotation about the
precession axis
--prec-angle-deg PREC_ANGLE_DEG
The opening angle (in degrees) of the spin axis from
the precession axis
--obs-time-h OBS_TIME_H
Number of hours in one science observation
--gap-h GAP_H Cooler cycle time in hours between science obs
--obs-num OBS_NUM Number of complete observations
--hwp-rpm HWP_RPM The rate (in RPM) of the HWP rotation
--hwp-step-deg HWP_STEP_DEG
For stepped HWP, the angle in degrees of each step
--hwp-step-time-s HWP_STEP_TIME_S
For stepped HWP, the time in seconds between steps
--conviqt-sky-file CONVIQT_SKY_FILE
Path to sky alm files. Tag DETECTOR will be replaced
with detector name.
--conviqt-lmax CONVIQT_LMAX
Simulation lmax. May not exceed the expansion order in
conviqt-sky-file
--conviqt-fwhm CONVIQT_FWHM
Sky fwhm [arcmin] to deconvolve
--conviqt-beam-file CONVIQT_BEAM_FILE
Path to beam alm files. Tag DETECTOR will be replaced
with detector name.
--conviqt-beam-mmax CONVIQT_BEAM_MMAX
Beam mmax. May not exceed the expansion order in
conviqt-beam-file
--conviqt-pxx Beams are in Pxx frame, not Dxx
--conviqt-dxx Beams are in Dxx frame, not Pxx
--conviqt-order CONVIQT_ORDER
Iteration order
--conviqt-normalize-beam
Normalize the beams
--no-conviqt-normalize-beam
Do not normalize the beams
--conviqt-remove-monopole
Remove the sky monopole before convolution
--no-conviqt-remove-monopole
Do not remove the sky monopole before convolution
--conviqt-remove-dipole
Remove the sky dipole before convolution
--no-conviqt-remove-dipole
Do not remove the sky dipole before convolution
--conviqt-mpi-comm CONVIQT_MPI_COMM
MPI communicator used by the OpSimConviqt operator,
either 'rank' or 'group'
--outdir OUTDIR Output directory
--debug Write diagnostics
--madam-prefix MADAM_PREFIX
Output map prefix
--madam-iter-max MADAM_ITER_MAX
Maximum number of CG iterations in Madam
--madam-precond-width MADAM_PRECOND_WIDTH
Width of the Madam band preconditioner
--madam-precond-width-min MADAM_PRECOND_WIDTH_MIN
Minimum width of the Madam band preconditioner
--madam-precond-width-max MADAM_PRECOND_WIDTH_MAX
Maximum width of the Madam band preconditioner
--madam-baseline-length MADAM_BASELINE_LENGTH
Destriping baseline length (seconds)
--madam-baseline-order MADAM_BASELINE_ORDER
Destriping baseline polynomial order
--madam-noisefilter Destripe with the noise filter enabled
--madam-parfile MADAM_PARFILE
Madam parameter file
--madam-allreduce Use the allreduce communication pattern in Madam
--no-madam-allreduce Do not use the allreduce communication pattern in
Madam
--madam-concatenate-messages
Use the alltoallv commucation pattern in Madam
--no-madam-concatenate-messages
Use the point-to-point communication pattern in Madam
--destripe Write destriped maps [default]
--no-destripe Do not write destriped maps
--binmap Write binned maps [default]
--no-binmap Do not write binned maps
--hits Write hit maps [default]
--no-hits Do not write hit maps
--wcov Write white noise covariance [default]
--no-wcov Do not write white noise covariance
--wcov-inv Write inverse white noise covariance [default]
--no-wcov-inv Do not write inverse white noise covariance
--conserve-memory Conserve memory when staging libMadam buffers
[default]
--no-conserve-memory Do not conserve memory when staging libMadam buffers
--zip Compress the map outputs
--no-zip Do not compress the map outputs
--madam Use libmadam for map-making
--no-madam Do not use libmadam for map-making [default]
--focalplane FOCALPLANE
Pickle file containing a dictionary of detector
properties. The keys of this dict are the detector
names, and each value is also a dictionary with keys
"quat" (4 element ndarray), "fwhm" (float, arcmin),
"fknee" (float, Hz), "alpha" (float), and "NET"
(float). For optional plotting, the key "color" can
specify a valid matplotlib color string.
--gain GAIN Calibrate the input timelines with a set of gains from
aFITS file containing 3 extensions:HDU named DETECTORS
: table with list of detector names in a column named
DETECTORSHDU named TIME: table with common timestamps
column named TIMEHDU named GAINS: 2D image of floats
with one row per detector and one column per value.
main without the profiling commands:
def main():
env = Environment.get()
mpiworld, procs, rank, comm = get_comm()
args, comm, groupsize = parse_arguments(comm, procs)
# Parse options
if comm.world_rank == 0:
os.makedirs(args.outdir, exist_ok=True)
focalplane, gain, detweights = load_focalplane(args, comm)
data = create_observations(args, comm, focalplane, groupsize)
expand_pointing(args, comm, data)
signalname = None
skyname = simulate_sky_signal(
args, comm, data, [focalplane], "signal"
)
if skyname is not None:
signalname = skyname
diponame = simulate_dipole(args, comm, data, "signal")
if diponame is not None:
signalname = diponame
# Mapmaking
if not args.use_madam:
# THIS BRANCH WILL SOON BE REPLACED WITH THE NATIVE MAPMAKER
else:
# Initialize madam parameters
madampars = setup_madam(args)
# Loop over Monte Carlos
firstmc = args.MC_start
nmc = args.MC_count
for mc in range(firstmc, firstmc + nmc):
# create output directory for this realization
outpath = os.path.join(args.outdir, "mc_{:03d}".format(mc))
simulate_noise(args, comm, data, mc, "tot_signal", overwrite=True)
# add sky signal
add_signal(args, comm, data, "tot_signal", signalname)
if gain is not None:
op_apply_gain = OpApplyGain(gain, name="tot_signal")
op_apply_gain.exec(data)
apply_madam(args, comm, data, madampars, outpath, detweights, "tot_signal")Check TOAST examples for all available pipelines at https://github.com/hpc4cmb/toast/tree/master/examples
In [7]:
%mkdir out
mkdir: cannot create directory ‘out’: File exists
In [8]:
! toast_fake_focalplane.py --bandcenter_ghz 25 --bandwidth_ghz 6.3 --minpix 1 --out "out/fp"
TOAST INFO: using 1 pixels (2 detectors)
In [11]:
%%writefile toast_satellite_sim.par
--dipole
--noise
--sample-rate
40.0
--spin-period-min
10.0
--spin-angle-deg
30.0
--prec-period-min
50.0
--prec-angle-deg
65.0
--hwp-rpm
20.0
--obs-time-h
24.0
--gap
4.0
--nside
512
--pysm-model
s1,d1,f1,a1
--pysm-apply-beam
--flush
--dipole-mode
total
--dipole-solar-speed-kms
369
--dipole-solar-gal-lat-deg
48.26
--dipole-solar-gal-lon-deg
263.99
--group-size
1
--focalplane
out/fp_1.pkl
--obs-num
8
--outdir
out
Overwriting toast_satellite_sim.par
In [13]:
import subprocess as sp
runcom = "toast_satellite_sim.py @toast_satellite_sim.par"
print(runcom, flush=True)
sp.check_call(runcom, stderr=sp.STDOUT, shell=True)
toast_satellite_sim.py @toast_satellite_sim.par TOAST INFO: Running with 1 processes at 2020-01-02 14:50:55.751684 TOAST INFO: TOAST INFO: All parameters: TOAST INFO: group_size = 1 TOAST INFO: do_daymaps = False TOAST INFO: do_seasonmaps = False TOAST INFO: nside = 512 TOAST INFO: nside_submap = 16 TOAST INFO: coord = C TOAST INFO: single_precision_pointing = False TOAST INFO: common_flag_mask = 1 TOAST INFO: flush = True TOAST INFO: tidas = None TOAST INFO: spt3g = None TOAST INFO: simulate_dipole = True TOAST INFO: dipole_mode = total TOAST INFO: dipole_solar_speed_kms = 369.0 TOAST INFO: dipole_solar_gal_lat_deg = 48.26 TOAST INFO: dipole_solar_gal_lon_deg = 263.99 TOAST INFO: pysm_model = s1,d1,f1,a1 TOAST INFO: pysm_apply_beam = True TOAST INFO: pysm_precomputed_cmb_K_CMB = None TOAST INFO: pysm_mpi_comm = group TOAST INFO: simulate_sky = True TOAST INFO: MC_start = 0 TOAST INFO: MC_count = 1 TOAST INFO: simulate_noise = True TOAST INFO: sample_rate = 40.0 TOAST INFO: start_time = 0.0 TOAST INFO: spin_period_min = 10.0 TOAST INFO: spin_angle_deg = 30.0 TOAST INFO: prec_period_min = 50.0 TOAST INFO: prec_angle_deg = 65.0 TOAST INFO: obs_time_h = 24.0 TOAST INFO: gap_h = 4.0 TOAST INFO: obs_num = 8 TOAST INFO: hwp_rpm = 20.0 TOAST INFO: hwp_step_deg = None TOAST INFO: hwp_step_time_s = None TOAST INFO: conviqt_sky_file = None TOAST INFO: conviqt_lmax = None TOAST INFO: conviqt_fwhm = None TOAST INFO: conviqt_beam_file = None TOAST INFO: conviqt_beam_mmax = None TOAST INFO: conviqt_dxx = True TOAST INFO: conviqt_order = 11 TOAST INFO: conviqt_normalize_beam = False TOAST INFO: conviqt_remove_monopole = False TOAST INFO: conviqt_remove_dipole = False TOAST INFO: no_conviqt_remove_dipole = True TOAST INFO: conviqt_mpi_comm = rank TOAST INFO: outdir = out TOAST INFO: debug = False TOAST INFO: madam_prefix = toast TOAST INFO: madam_iter_max = 1000 TOAST INFO: madam_precond_width = 100 TOAST INFO: madam_precond_width_min = None TOAST INFO: madam_precond_width_max = None TOAST INFO: madam_baseline_length = 10000.0 TOAST INFO: madam_baseline_order = 0 TOAST INFO: madam_noisefilter = False TOAST INFO: madam_parfile = None TOAST INFO: madam_allreduce = False TOAST INFO: madam_concatenate_messages = True TOAST INFO: destripe = True TOAST INFO: write_binmap = True TOAST INFO: write_hits = True TOAST INFO: write_wcov = True TOAST INFO: write_wcov_inv = True TOAST INFO: conserve_memory = True TOAST INFO: zip_maps = True TOAST INFO: use_madam = False TOAST INFO: focalplane = out/fp_1.pkl TOAST INFO: gain = None TOAST INFO: TOAST INFO: Create focalplane (2 dets): 0.00 seconds (0 calls) TOAST INFO: Creating noise model: 0.00 seconds (1 calls) TOAST INFO: Read parameters, compute data distribution: 0.69 seconds (1 calls) TOAST INFO: Construct boresight pointing: 8.94 seconds (1 calls) TOAST INFO: Expanding pointing TOAST INFO: Pointing generation: 11.50 seconds (1 calls) TOAST INFO: PySM Operator: 41.46 seconds (1 calls) TOAST INFO: Simulating dipole TOAST INFO: Simulate dipole: 8.93 seconds (1 calls) TOAST INFO: Not using Madam, will only make a binned map TOAST INFO: Accumulate N_pp'^1: 0.56 seconds (1 calls) TOAST INFO: All reduce N_pp'^1: 0.05 seconds (1 calls) TOAST INFO: Wrote hits to out/hits.fits.gz TOAST INFO: Wrote inverse white noise covariance to out/invnpp.fits.gz TOAST INFO: Writing hits and N_pp'^1: 11.19 seconds (1 calls) TOAST INFO: Invert N_pp'^1: 0.90 seconds (1 calls) TOAST INFO: Wrote white noise covariance to out/npp.fits.gz TOAST INFO: Write N_pp': 2.27 seconds (1 calls) TOAST INFO: Simulating noise TOAST INFO: Simulate noise: 11.93 seconds (1 calls) TOAST INFO: Adding signal from signal to tot_signal. TOAST INFO: Add signal: 0.05 seconds (1 calls) TOAST INFO: Building noise-weighted map: 0.00 seconds (0 calls) TOAST INFO: Computing binned map: 0.00 seconds (0 calls) TOAST INFO: Wrote binned map to out/mc_000/binned.fits.gz TOAST INFO: Writing binned map: 0.00 seconds (0 calls) TOAST INFO: Map-making 0000: 13.58 seconds (1 calls) TOAST INFO: Gather and dump timing info: 0.00 seconds (1 calls) TOAST INFO: toast_satellite_sim.py: 101.89 seconds (1 calls) <toast.Environment Source code version = 2.3.5.dev1474 Logging level = INFO Handling enabled for 0 signals: Max threads = 12 MPI build enabled MPI runtime enabled > Sigma is 0.000000 arcmin (0.000000 rad) -> fwhm is 0.000000 arcmin Sigma is 0.000000 arcmin (0.000000 rad) -> fwhm is 0.000000 arcmin
Out[13]:
0
In [14]:
import healpy as hp
import numpy as np
binned = hp.read_map("out/mc_000/binned.fits.gz")
hits = hp.read_map("out/hits.fits.gz")
unseen = hits==0
hits[unseen] = hp.UNSEEN
binned[unseen] = hp.UNSEEN
cmap = "coolwarm"
amp = 0.02
unit = "K"
plt.figure(figsize=[12, 8])
hp.mollview(np.log10(hits), sub=[1, 2, 1], title="hitcount", unit="log10(hits)")
hp.mollview(binned, cmap=cmap, min=-amp, max=amp, sub=[1, 2, 2], title="binned", unit=unit)
NSIDE = 512 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING NSIDE = 512 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING
/home/kisner/software/cmbenv/cmbenv_python/lib/python3.7/site-packages/ipykernel_launcher.py:13: RuntimeWarning: invalid value encountered in log10 del sys.path[0] /home/kisner/software/cmbenv/cmbenv_aux/lib/python3.7/site-packages/healpy-1.12.10-py3.7-linux-x86_64.egg/healpy/pixelfunc.py:345: RuntimeWarning: invalid value encountered in less_equal return np.absolute(m - badval) <= atol + rtol * np.absolute(badval) /home/kisner/software/cmbenv/cmbenv_aux/lib/python3.7/site-packages/healpy-1.12.10-py3.7-linux-x86_64.egg/healpy/projaxes.py:1184: RuntimeWarning: invalid value encountered in less result.data[result.data < 0] = 0.0 /home/kisner/software/cmbenv/cmbenv_aux/lib/python3.7/site-packages/healpy-1.12.10-py3.7-linux-x86_64.egg/healpy/projaxes.py:1185: RuntimeWarning: invalid value encountered in greater result.data[result.data > 1] = 1.0
Ground simulation pipeline¶
In [15]:
! toast_ground_sim.py --help
<toast.Environment
Source code version = 2.3.5.dev1474
Logging level = INFO
Handling enabled for 0 signals:
Max threads = 12
MPI build enabled
MPI runtime enabled
>
TOAST INFO: Running with 1 processes at 2020-01-02 14:53:31.061711
usage: toast_ground_sim.py [-h] [--group-size GROUP_SIZE] [--day-maps]
[--no-day-maps] [--season-maps] [--no-season-maps]
[--debug] [--no-debug] [--scan-rate SCAN_RATE]
[--scan-accel SCAN_ACCEL]
[--sun-angle-min SUN_ANGLE_MIN] --schedule SCHEDULE
[--weather WEATHER] [--timezone TIMEZONE]
[--sample-rate SAMPLE_RATE] [--coord COORD]
[--split-schedule SPLIT_SCHEDULE] [--sort-schedule]
[--no-sort-schedule] [--hwp-rpm HWP_RPM]
[--hwp-step-deg HWP_STEP_DEG]
[--hwp-step-time-s HWP_STEP_TIME_S]
[--elevation-noise-a ELEVATION_NOISE_A]
[--elevation-noise-b ELEVATION_NOISE_B]
[--nside NSIDE] [--nside-submap NSIDE_SUBMAP]
[--single-precision-pointing]
[--no-single-precision-pointing]
[--common-flag-mask COMMON_FLAG_MASK] [--flush]
[--polyfilter] [--no-polyfilter]
[--poly-order POLY_ORDER] [--groundfilter]
[--no-groundfilter] [--ground-order GROUND_ORDER]
[--atmosphere] [--simulate-atmosphere]
[--no-atmosphere] [--no-simulate-atmosphere]
[--focalplane-radius-deg FOCALPLANE_RADIUS_DEG]
[--atm-verbosity ATM_VERBOSITY]
[--atm-lmin-center ATM_LMIN_CENTER]
[--atm-lmin-sigma ATM_LMIN_SIGMA]
[--atm-lmax-center ATM_LMAX_CENTER]
[--atm-lmax-sigma ATM_LMAX_SIGMA]
[--atm-gain ATM_GAIN] [--atm-zatm ATM_ZATM]
[--atm-zmax ATM_ZMAX] [--atm-xstep ATM_XSTEP]
[--atm-ystep ATM_YSTEP] [--atm-zstep ATM_ZSTEP]
[--atm-nelem-sim-max ATM_NELEM_SIM_MAX]
[--atm-wind-dist ATM_WIND_DIST]
[--atm-z0-center ATM_Z0_CENTER]
[--atm-z0-sigma ATM_Z0_SIGMA]
[--atm-T0-center ATM_T0_CENTER]
[--atm-T0-sigma ATM_T0_SIGMA]
[--atm-cache ATM_CACHE] [--noise]
[--simulate-noise] [--no-noise]
[--no-simulate-noise] [--gainscrambler]
[--no-gainscrambler] [--gain-sigma GAIN_SIGMA]
[--madam-prefix MADAM_PREFIX]
[--madam-iter-max MADAM_ITER_MAX]
[--madam-precond-width MADAM_PRECOND_WIDTH]
[--madam-precond-width-min MADAM_PRECOND_WIDTH_MIN]
[--madam-precond-width-max MADAM_PRECOND_WIDTH_MAX]
[--madam-baseline-length MADAM_BASELINE_LENGTH]
[--madam-baseline-order MADAM_BASELINE_ORDER]
[--madam-noisefilter]
[--madam-parfile MADAM_PARFILE] [--madam-allreduce]
[--no-madam-allreduce]
[--madam-concatenate-messages]
[--no-madam-concatenate-messages] [--destripe]
[--no-destripe] [--binmap] [--no-binmap] [--hits]
[--no-hits] [--wcov] [--no-wcov] [--wcov-inv]
[--no-wcov-inv] [--conserve-memory]
[--no-conserve-memory] [--input-map INPUT_MAP]
[--sky] [--simulate-sky] [--no-sky]
[--no-simulate-sky] [--pysm-model PYSM_MODEL]
[--pysm-apply-beam] [--no-pysm-apply-beam]
[--pysm-precomputed-cmb-K_CMB PYSM_PRECOMPUTED_CMB_K_CMB]
[--pysm-mpi-comm PYSM_MPI_COMM]
[--ground-map GROUND_MAP]
[--ground-nside GROUND_NSIDE]
[--ground-fwhm-deg GROUND_FWHM_DEG]
[--ground-lmax GROUND_LMAX]
[--ground-scale GROUND_SCALE]
[--ground-power GROUND_POWER] [--simulate-ground]
[--no-simulate-ground] [--tidas TIDAS]
[--spt3g SPT3G] [--MC-start MC_START]
[--MC-count MC_COUNT] [--outdir OUTDIR]
[--focalplane FOCALPLANE] --freq FREQ
Simulate ground-based boresight pointing. Simulate atmosphere and make maps
for some number of noise Monte Carlos.
optional arguments:
-h, --help show this help message and exit
--group-size GROUP_SIZE
Size of a process group assigned to an observation
--day-maps Enable daily maps
--no-day-maps Disable daily maps
--season-maps Enable season maps
--no-season-maps Disable season maps
--debug Enable extra debugging outputs
--no-debug Disable extra debugging outputs
--scan-rate SCAN_RATE
Scanning rate [deg / s]
--scan-accel SCAN_ACCEL
Scanning rate change [deg / s^2]
--sun-angle-min SUN_ANGLE_MIN
Minimum azimuthal distance between the Sun and the
bore sight [deg]
--schedule SCHEDULE Comma-separated list CES schedule files (from
toast_ground_schedule.py)
--weather WEATHER Comma-separated list of TOAST weather files for every
schedule. Repeat the same file if the schedules share
observing site.
--timezone TIMEZONE Offset to apply to MJD to separate days [hours]
--sample-rate SAMPLE_RATE
Detector sample rate (Hz)
--coord COORD Sky coordinate system [C,E,G]
--split-schedule SPLIT_SCHEDULE
Only use a subset of the schedule. The argument is a
string of the form "[isplit],[nsplit]" and only
observations that satisfy scan modulo nsplit == isplit
are included
--sort-schedule Reorder the observing schedule so that observations of
thesame patch are consecutive. This will reduce the
sky area observed by individual process groups.
--no-sort-schedule Do not reorder the observing schedule so that
observations of thesame patch are consecutive.
--hwp-rpm HWP_RPM The rate (in RPM) of the HWP rotation
--hwp-step-deg HWP_STEP_DEG
For stepped HWP, the angle in degrees of each step
--hwp-step-time-s HWP_STEP_TIME_S
For stepped HWP, the time in seconds between steps
--elevation-noise-a ELEVATION_NOISE_A
Evaluate noise PSD as (a / sin(el) + b) ** 2 * fsample
* 1e-12
--elevation-noise-b ELEVATION_NOISE_B
Evaluate noise PSD as (a / sin(el) + b) ** 2 * fsample
* 1e-12
--nside NSIDE Healpix NSIDE
--nside-submap NSIDE_SUBMAP
Number of sub pixels is 12 * nside-submap ** 2
--single-precision-pointing
Use single precision for pointing in memory.
--no-single-precision-pointing
Use double precision for pointing in memory.
--common-flag-mask COMMON_FLAG_MASK
Common flag mask
--flush Flush every print statement.
--polyfilter Apply polynomial filter
--no-polyfilter Do not apply polynomial filter
--poly-order POLY_ORDER
Polynomial order for the polyfilter
--groundfilter Apply ground filter
--no-groundfilter Do not apply ground filter
--ground-order GROUND_ORDER
Ground template order
--atmosphere Add simulated atmoshere
--simulate-atmosphere
Add simulated atmoshere
--no-atmosphere Do not add simulated atmosphere
--no-simulate-atmosphere
Do not add simulated atmosphere
--focalplane-radius-deg FOCALPLANE_RADIUS_DEG
Override focal plane radius [deg]
--atm-verbosity ATM_VERBOSITY
Atmospheric sim verbosity level
--atm-lmin-center ATM_LMIN_CENTER
Kolmogorov turbulence dissipation scale center
--atm-lmin-sigma ATM_LMIN_SIGMA
Kolmogorov turbulence dissipation scale sigma
--atm-lmax-center ATM_LMAX_CENTER
Kolmogorov turbulence injection scale center
--atm-lmax-sigma ATM_LMAX_SIGMA
Kolmogorov turbulence injection scale sigma
--atm-gain ATM_GAIN Atmospheric gain factor.
--atm-zatm ATM_ZATM atmosphere extent for temperature profile
--atm-zmax ATM_ZMAX atmosphere extent for water vapor integration
--atm-xstep ATM_XSTEP
size of volume elements in X direction
--atm-ystep ATM_YSTEP
size of volume elements in Y direction
--atm-zstep ATM_ZSTEP
size of volume elements in Z direction
--atm-nelem-sim-max ATM_NELEM_SIM_MAX
controls the size of the simulation slices
--atm-wind-dist ATM_WIND_DIST
Maximum wind drift to simulate without discontinuity
--atm-z0-center ATM_Z0_CENTER
central value of the water vapor distribution
--atm-z0-sigma ATM_Z0_SIGMA
sigma of the water vapor distribution
--atm-T0-center ATM_T0_CENTER
central value of the temperature distribution
--atm-T0-sigma ATM_T0_SIGMA
sigma of the temperature distribution
--atm-cache ATM_CACHE
Atmosphere cache directory
--noise Add simulated noise
--simulate-noise Add simulated noise
--no-noise Do not add simulated noise
--no-simulate-noise Do not add simulated noise
--gainscrambler Add simulated noise
--no-gainscrambler Do not add simulated noise
--gain-sigma GAIN_SIGMA
Simulated gain fluctuation amplitude
--madam-prefix MADAM_PREFIX
Output map prefix
--madam-iter-max MADAM_ITER_MAX
Maximum number of CG iterations in Madam
--madam-precond-width MADAM_PRECOND_WIDTH
Width of the Madam band preconditioner
--madam-precond-width-min MADAM_PRECOND_WIDTH_MIN
Minimum width of the Madam band preconditioner
--madam-precond-width-max MADAM_PRECOND_WIDTH_MAX
Maximum width of the Madam band preconditioner
--madam-baseline-length MADAM_BASELINE_LENGTH
Destriping baseline length (seconds)
--madam-baseline-order MADAM_BASELINE_ORDER
Destriping baseline polynomial order
--madam-noisefilter Destripe with the noise filter enabled
--madam-parfile MADAM_PARFILE
Madam parameter file
--madam-allreduce Use the allreduce communication pattern in Madam
--no-madam-allreduce Do not use the allreduce communication pattern in
Madam
--madam-concatenate-messages
Use the alltoallv commucation pattern in Madam
--no-madam-concatenate-messages
Use the point-to-point communication pattern in Madam
--destripe Write destriped maps [default]
--no-destripe Do not write destriped maps
--binmap Write binned maps [default]
--no-binmap Do not write binned maps
--hits Write hit maps [default]
--no-hits Do not write hit maps
--wcov Write white noise covariance [default]
--no-wcov Do not write white noise covariance
--wcov-inv Write inverse white noise covariance [default]
--no-wcov-inv Do not write inverse white noise covariance
--conserve-memory Conserve memory when staging libMadam buffers
[default]
--no-conserve-memory Do not conserve memory when staging libMadam buffers
--input-map INPUT_MAP
Input map for signal
--sky Add simulated sky signal
--simulate-sky Add simulated sky
--no-sky Do not add simulated sky
--no-simulate-sky Do not add simulated sky
--pysm-model PYSM_MODEL
Comma separated models for on-the-fly PySM simulation,
e.g. "s1,d6,f1,a2"
--pysm-apply-beam Convolve sky with detector beam
--no-pysm-apply-beam Do not convolve sky with detector beam.
--pysm-precomputed-cmb-K_CMB PYSM_PRECOMPUTED_CMB_K_CMB
Precomputed CMB map for PySM in K_CMBit overrides any
model defined in pysm_model"
--pysm-mpi-comm PYSM_MPI_COMM
MPI communicator used by the PySM operator, either
'rank' or 'group'
--ground-map GROUND_MAP
Fixed ground template map
--ground-nside GROUND_NSIDE
Ground template resolution
--ground-fwhm-deg GROUND_FWHM_DEG
Ground template smoothing in degrees
--ground-lmax GROUND_LMAX
Ground template expansion order
--ground-scale GROUND_SCALE
Ground template RMS at el=45 deg
--ground-power GROUND_POWER
Exponential for suppressing ground pick-up at higher
observing elevation
--simulate-ground Enable simulating ground pickup.
--no-simulate-ground Disable simulating ground pickup.
--tidas TIDAS Output TIDAS export path
--spt3g SPT3G Output SPT3G export path
--MC-start MC_START First Monte Carlo noise realization
--MC-count MC_COUNT Number of Monte Carlo noise realizations
--outdir OUTDIR Output directory
--focalplane FOCALPLANE
Pickle file containing a dictionary of detector
properties. The keys of this dict are the detector
names, and each value is also a dictionary with keys
"quat" (4 element ndarray), "fwhm" (float, arcmin),
"fknee" (float, Hz), "alpha" (float), and "NET"
(float).
--freq FREQ Comma-separated list of frequencies with identical
focal planes. They override the bandpasses in the
focalplane for the purpose of scaling the atmospheric
signal but not for simulating the sky signal.
main without profiling
def main():
mpiworld, procs, rank, comm = get_comm()
args, comm = parse_arguments(comm)
# Initialize madam parameters
madampars = setup_madam(args)
# Load and broadcast the schedule file
schedules = load_schedule(args, comm)
# Load the weather and append to schedules
load_weather(args, comm, schedules)
# load or simulate the focalplane
detweights = load_focalplanes(args, comm, schedules)
# Create the TOAST data object to match the schedule. This will
# include simulating the boresight pointing
data, telescope_data = create_observations(args, comm, schedules)
# Split the communicator for day and season mapmaking
time_comms = get_time_communicators(args, comm, data)
# Expand boresight quaternions into detector pointing weights and
# pixel numbers
expand_pointing(args, comm, data)
# Purge the pointing if we are NOT going to export the
# data to a TIDAS volume
if (args.tidas is None) and (args.spt3g is None):
for ob in data.obs:
tod = ob["tod"]
tod.free_radec_quats()
if args.pysm_model:
focalplanes = [s.telescope.focalplane.detector_data for s in schedules]
signalname = simulate_sky_signal(
args, comm, data, focalplanes, "signal"
)
else:
signalname = scan_sky_signal(args, comm, data, "signal")
# Set up objects to take copies of the TOD at appropriate times
totalname, totalname_freq = setup_sigcopy(args)
# Loop over Monte Carlos
firstmc = args.MC_start
nsimu = args.MC_count
freqs = [float(freq) for freq in args.freq.split(",")]
nfreq = len(freqs)
for mc in range(firstmc, firstmc + nsimu):
simulate_atmosphere(args, comm, data, mc, totalname)
# Loop over frequencies with identical focal planes and identical
# atmospheric noise
for ifreq, freq in enumerate(freqs):
# Make a copy of the atmosphere so we can scramble the gains and apply
# frequency-dependent scaling
copy_signal(args, comm, data, totalname, totalname_freq)
scale_atmosphere_by_frequency(
args, comm, data, freq=freq, mc=mc, cache_name=totalname_freq
)
update_atmospheric_noise_weights(args, comm, data, freq, mc)
# Add previously simulated sky signal to the atmospheric noise
add_signal(args, comm, data, totalname_freq, signalname, purge=(nsimu == 1))
mcoffset = ifreq * 1000000
simulate_noise(args, comm, data, mc + mcoffset, totalname_freq)
simulate_sss(args, comm, data, mc + mcoffset, totalname_freq)
scramble_gains(args, comm, data, mc + mcoffset, totalname_freq)
if (mc == firstmc) and (ifreq == 0):
# For the first realization and frequency, optionally
# export the timestream data
output_tidas(args, comm, data, totalname)
output_spt3g(args, comm, data, totalname)
outpath = setup_output(args, comm, mc + mcoffset, freq)
# Bin and destripe maps
apply_madam(
args,
comm,
data,
madampars,
outpath,
detweights,
totalname_freq,
freq=freq,
time_comms=time_comms,
telescope_data=telescope_data,
first_call=(mc == firstmc),
)
if args.apply_polyfilter or args.apply_groundfilter:
# Filter signal
apply_polyfilter(args, comm, data, totalname_freq)
apply_groundfilter(args, comm, data, totalname_freq)
# Bin filtered maps
apply_madam(
args,
comm,
data,
madampars,
outpath,
detweights,
totalname_freq,
freq=freq,
time_comms=time_comms,
telescope_data=telescope_data,
first_call=False,
extra_prefix="filtered",
bin_only=True,
)Here is a full working example of the ground simulation pipeline
First we need an observing schedule. This one is for one patch and covers 24 hours:
In [16]:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-02 00:00:00" \
--patch-coord C \
--patch small_patch,1,40,-40,10 \
--el-min 45 \
--el-max 60 \
--out schedule.txt
! cat schedule.txt
TOAST INFO: Adding patch "small_patch" TOAST INFO: Center-and-width format TOAST INFO: Global timer: toast_ground_schedule: 0.13 seconds (1 calls) #Site Telescope Latitude [deg] Longitude [deg] Elevation [m] Atacama LAT -22.958 -67.786 5200.0 #Start time UTC Stop time UTC Start MJD Stop MJD Patch name Az min Az max El R/S Sun el1 Sun az1 Sun el2 Sun az2 Moon el1 Moon az1 Moon el2 Moon az2 Phase Pass Sub 2020-01-01 02:00:00 2020-01-01 02:14:30 58849.083333 58849.093403 small_patch 217.44 234.41 59.64 S -30.65 221.82 -32.81 218.83 19.11 267.74 15.85 266.47 0.31 0 0 2020-01-01 02:14:40 2020-01-01 02:29:10 58849.093519 58849.103588 small_patch 217.53 238.45 59.64 S -32.83 218.80 -34.85 215.60 15.81 266.45 12.55 265.18 0.31 0 1 2020-01-01 02:29:20 2020-01-01 02:43:50 58849.103704 58849.113773 small_patch 219.01 239.50 59.64 S -34.87 215.56 -36.73 212.15 12.52 265.17 9.28 263.90 0.31 0 2 2020-01-01 02:44:00 2020-01-01 02:58:00 58849.113889 58849.123611 small_patch 222.66 238.93 59.64 S -36.75 212.11 -38.38 208.61 9.24 263.89 6.13 262.65 0.31 0 3 2020-01-01 02:59:40 2020-01-01 03:12:55 58849.124769 58849.133970 small_patch 227.28 240.30 49.46 S -38.56 208.18 -39.92 204.65 5.76 262.51 2.86 261.32 0.31 1 0 2020-01-01 03:13:05 2020-01-01 03:26:20 58849.134086 58849.143287 small_patch 226.82 242.20 49.46 S -39.94 204.61 -41.12 200.91 2.83 261.31 0.05 260.11 0.31 1 1 2020-01-01 03:26:30 2020-01-01 03:39:45 58849.143403 58849.152604 small_patch 227.33 242.53 49.46 S -41.13 200.86 -42.12 197.01 0.02 260.09 -2.42 258.86 0.31 1 2 2020-01-01 03:39:55 2020-01-01 03:52:40 58849.152720 58849.161574 small_patch 229.43 241.58 49.46 S -42.13 196.96 -42.90 193.12 -2.45 258.85 -5.97 257.64 0.31 1 3 2020-01-01 20:44:20 2020-01-01 20:57:35 58849.864120 58849.873322 small_patch 117.99 129.78 45.23 R 33.02 256.67 30.06 255.82 67.82 42.00 69.66 34.59 0.38 2 0 2020-01-01 20:57:45 2020-01-01 21:11:00 58849.873438 58849.882639 small_patch 117.12 131.14 45.23 R 30.02 255.81 27.07 254.94 69.68 34.49 71.16 25.84 0.38 2 1 2020-01-01 21:11:10 2020-01-01 21:24:25 58849.882755 58849.891956 small_patch 117.33 131.42 45.23 R 27.04 254.93 24.10 254.02 71.17 25.73 72.21 15.90 0.38 2 2 2020-01-01 21:24:35 2020-01-01 21:37:20 58849.892072 58849.900926 small_patch 119.23 130.71 45.23 R 24.06 254.01 21.25 253.12 72.22 15.77 72.71 5.50 0.38 2 3 2020-01-01 21:39:00 2020-01-01 21:52:45 58849.902083 58849.911632 small_patch 119.49 133.93 55.47 R 20.89 253.00 17.87 251.99 72.74 4.13 72.61 352.79 0.38 3 0 2020-01-01 21:52:55 2020-01-01 22:06:40 58849.911748 58849.921296 small_patch 118.79 136.62 55.47 R 17.84 251.98 14.84 250.93 72.61 352.65 71.87 341.84 0.38 3 1 2020-01-01 22:06:50 2020-01-01 22:20:35 58849.921412 58849.930961 small_patch 119.55 137.47 55.47 R 14.80 250.92 11.83 249.83 71.86 341.71 70.59 331.99 0.38 3 2 2020-01-01 22:20:45 2020-01-01 22:34:00 58849.931076 58849.940278 small_patch 122.69 137.09 55.47 R 11.80 249.82 8.96 248.72 70.57 331.88 68.92 323.76 0.38 3 3
Then we need a focalplane
In [17]:
! toast_fake_focalplane.py \
--minpix 100 \
--out focalplane \
--fwhm 30 \
--fwhm_sigma 0.05 \
--fov 10 \
--psd_fknee 5e-2 \
--psd_NET 1e-3 \
--psd_alpha 1 \
--psd_fmin 1e-5 \
--bandcenter_ghz 100 \
--bandcenter_sigma 0.01 \
--bandwidth_ghz 10 \
--bandwidth_sigma 0.1
TOAST INFO: using 127 pixels (254 detectors)
And of course we need the weather file
In [19]:
! if [ ! -e weather_Atacama.fits ]; then wget http://portal.nersc.gov/project/cmb/toast_data/example_data/weather_Atacama.fits; fi
--2020-01-02 14:54:20-- http://portal.nersc.gov/project/cmb/toast_data/example_data/weather_Atacama.fits Resolving portal.nersc.gov (portal.nersc.gov)... 128.55.201.128 Connecting to portal.nersc.gov (portal.nersc.gov)|128.55.201.128|:80... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://portal.nersc.gov/project/cmb/toast_data/example_data/weather_Atacama.fits [following] --2020-01-02 14:54:20-- https://portal.nersc.gov/project/cmb/toast_data/example_data/weather_Atacama.fits Connecting to portal.nersc.gov (portal.nersc.gov)|128.55.201.128|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 2180160 (2.1M) [image/fits] Saving to: ‘weather_Atacama.fits’ weather_Atacama.fit 100%[===================>] 2.08M 7.90MB/s in 0.3s 2020-01-02 14:54:21 (7.90 MB/s) - ‘weather_Atacama.fits’ saved [2180160/2180160]
Now write a parameter file that uses the above inputs and simulates sky signal (using PySM), atmosphere and instrument noise
In [20]:
%%writefile toast_ground_sim.par
--weather
weather_Atacama.fits
--scan-rate
1
--scan-accel
3
--schedule
schedule.txt
--sample-rate
30
--coord
C
--hwp-rpm
2
--nside
512
--common-flag-mask
1
--polyfilter
--poly-order
5
--groundfilter
--ground-order
3
--atmosphere
--atm-lmin-center
0.1
--atm-lmax-center
30
--atm-gain
3e-5
--atm-zmax
1000
--atm-xstep
30
--atm-ystep
30
--atm-zstep
30
--atm-nelem-sim-max
10000
--atm-wind-dist
5000
--atm-cache
atm_cache
--noise
--gainscrambler
--gain-sigma
0.05
--madam-prefix
groundsim000
--madam-iter-max
200
--madam-precond-width
100
--madam-baseline-length
1
--madam-noisefilter
--madam-allreduce
--destripe
--binmap
--hits
--wcov
--ground-nside
512
--ground-fwhm-deg
3
--ground-lmax
512
--ground-scale
1e-3
--simulate-ground
--focalplane
focalplane_127.pkl
--freq
100
Overwriting toast_ground_sim.par
In [21]:
# --pysm-model
# s1,d1,f1,a1
# --pysm-apply-beam
With the PySM part (last two lines) the simulation will take about 20 minutes on a single Cori Haswell node at NERSC. If you remove them, it will run in about 4 minutes. Here we just run this job locally with MPI.
In [22]:
runcom = "toast_ground_sim.py @toast_ground_sim.par"
print(runcom, flush=True)
sp.check_call(runcom, stderr=sp.STDOUT, shell=True)
toast_ground_sim.py @toast_ground_sim.par
TOAST INFO: Running with 1 processes at 2020-01-02 14:58:32.704212
TOAST INFO: All parameters:
TOAST INFO: group_size = None
TOAST INFO: do_daymaps = False
TOAST INFO: do_seasonmaps = False
TOAST INFO: debug = False
TOAST INFO: scan_rate = 1.0
TOAST INFO: scan_accel = 3.0
TOAST INFO: sun_angle_min = 30.0
TOAST INFO: schedule = schedule.txt
TOAST INFO: weather = weather_Atacama.fits
TOAST INFO: timezone = 0
TOAST INFO: sample_rate = 30.0
TOAST INFO: coord = C
TOAST INFO: split_schedule = None
TOAST INFO: sort_schedule = True
TOAST INFO: hwp_rpm = 2.0
TOAST INFO: hwp_step_deg = None
TOAST INFO: hwp_step_time_s = None
TOAST INFO: elevation_noise_a = 0
TOAST INFO: elevation_noise_b = 0
TOAST INFO: nside = 512
TOAST INFO: nside_submap = 16
TOAST INFO: single_precision_pointing = False
TOAST INFO: common_flag_mask = 1
TOAST INFO: flush = False
TOAST INFO: apply_polyfilter = True
TOAST INFO: poly_order = 5
TOAST INFO: apply_groundfilter = True
TOAST INFO: ground_order = 3
TOAST INFO: simulate_atmosphere = True
TOAST INFO: focalplane_radius_deg = None
TOAST INFO: atm_verbosity = 0
TOAST INFO: atm_lmin_center = 0.1
TOAST INFO: atm_lmin_sigma = 0.001
TOAST INFO: atm_lmax_center = 30.0
TOAST INFO: atm_lmax_sigma = 10.0
TOAST INFO: atm_gain = 3e-05
TOAST INFO: atm_zatm = 40000.0
TOAST INFO: atm_zmax = 1000.0
TOAST INFO: atm_xstep = 30.0
TOAST INFO: atm_ystep = 30.0
TOAST INFO: atm_zstep = 30.0
TOAST INFO: atm_nelem_sim_max = 10000
TOAST INFO: atm_wind_dist = 5000.0
TOAST INFO: atm_z0_center = 2000.0
TOAST INFO: atm_z0_sigma = 0.0
TOAST INFO: atm_T0_center = 280.0
TOAST INFO: atm_T0_sigma = 10.0
TOAST INFO: atm_cache = atm_cache
TOAST INFO: simulate_noise = True
TOAST INFO: apply_gainscrambler = True
TOAST INFO: gain_sigma = 0.05
TOAST INFO: madam_prefix = groundsim000
TOAST INFO: madam_iter_max = 200
TOAST INFO: madam_precond_width = 100
TOAST INFO: madam_precond_width_min = None
TOAST INFO: madam_precond_width_max = None
TOAST INFO: madam_baseline_length = 1.0
TOAST INFO: madam_baseline_order = 0
TOAST INFO: madam_noisefilter = True
TOAST INFO: madam_parfile = None
TOAST INFO: madam_allreduce = True
TOAST INFO: madam_concatenate_messages = True
TOAST INFO: destripe = True
TOAST INFO: write_binmap = True
TOAST INFO: write_hits = True
TOAST INFO: write_wcov = True
TOAST INFO: write_wcov_inv = True
TOAST INFO: conserve_memory = True
TOAST INFO: input_map = None
TOAST INFO: simulate_sky = True
TOAST INFO: pysm_model = None
TOAST INFO: pysm_apply_beam = True
TOAST INFO: pysm_precomputed_cmb_K_CMB = None
TOAST INFO: pysm_mpi_comm = group
TOAST INFO: ground_map = None
TOAST INFO: ground_nside = 512
TOAST INFO: ground_fwhm_deg = 3.0
TOAST INFO: ground_lmax = 512
TOAST INFO: ground_scale = 0.001
TOAST INFO: ground_power = -1
TOAST INFO: simulate_ground = True
TOAST INFO: tidas = None
TOAST INFO: spt3g = None
TOAST INFO: MC_start = 0
TOAST INFO: MC_count = 1
TOAST INFO: outdir = out
TOAST INFO: focalplane = focalplane_127.pkl
TOAST INFO: freq = 100
TOAST INFO: Parsed parameters: 0.00 seconds (1 calls)
TOAST INFO: Load 16 (sub)scans in schedule.txt: 0.01 seconds (1 calls)
TOAST INFO: Loading schedule(s): 0.01 seconds (1 calls)
TOAST INFO: Load weather_Atacama.fits: 0.15 seconds (1 calls)
TOAST INFO: Loading focalplanes: 0.01 seconds (1 calls)
TOAST INFO: Group # 0 has 16 observations.
TOAST INFO: Simulated scans: 0.17 seconds (1 calls)
TOAST INFO: Expanding pointing
TOAST INFO: Pointing generation: 15.81 seconds (1 calls)
TOAST INFO: Simulating atmosphere
<toast.Environment
Source code version = 2.3.5.dev1474
Logging level = INFO
Handling enabled for 0 signals:
Max threads = 12
MPI build enabled
MPI runtime enabled
>
Creating atm_cache/8/3/3
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0793319 s.
5728 / 18941(30.2413 %) volume elements are needed for the simulation
nx = 31 ny = 13 nz = 47
wx = 0.885478 wy = -0.32874 wz = 1.51168
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Simulated atmosphere: 3.31 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Observe atmosphere: 1.21 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0387473 s.
3274 / 3780(86.6138 %) volume elements are needed for the simulation
nx = 21 ny = 10 nz = 18
wx = 0.885478 wy = -0.32874 wz = 1.51168
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Simulated atmosphere: 2.85 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Observe atmosphere: 1.50 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00255707 s.
320 / 320(100 %) volume elements are needed for the simulation
nx = 8 ny = 5 nz = 8
wx = 0.885478 wy = -0.32874 wz = 1.51168
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Simulated atmosphere: 2.31 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : OpSimAtmosphere: Observe atmosphere: 0.97 seconds (1 calls)
Creating atm_cache/9/3/5
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.068814 s.
5703 / 17484(32.6184 %) volume elements are needed for the simulation
nx = 31 ny = 12 nz = 47
wx = 0.890899 wy = -0.265291 wz = 1.52093
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Simulated atmosphere: 2.96 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Observe atmosphere: 1.06 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0311333 s.
2955 / 3402(86.8607 %) volume elements are needed for the simulation
nx = 21 ny = 9 nz = 18
wx = 0.890899 wy = -0.265291 wz = 1.52093
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Simulated atmosphere: 2.36 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Observe atmosphere: 1.18 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00664602 s.
320 / 320(100 %) volume elements are needed for the simulation
nx = 8 ny = 5 nz = 8
wx = 0.890899 wy = -0.265291 wz = 1.52093
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Simulated atmosphere: 2.29 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : OpSimAtmosphere: Observe atmosphere: 0.91 seconds (1 calls)
Creating atm_cache/0/3/7
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0726719 s.
5554 / 16027(34.654 %) volume elements are needed for the simulation
nx = 31 ny = 11 nz = 47
wx = 0.89361 wy = -0.226802 wz = 1.52556
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Simulated atmosphere: 4.14 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Observe atmosphere: 1.03 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0286125 s.
3148 / 3591(87.6636 %) volume elements are needed for the simulation
nx = 21 ny = 9 nz = 19
wx = 0.89361 wy = -0.226802 wz = 1.52556
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Simulated atmosphere: 3.17 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Observe atmosphere: 1.21 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00572082 s.
320 / 320(100 %) volume elements are needed for the simulation
nx = 8 ny = 5 nz = 8
wx = 0.89361 wy = -0.226802 wz = 1.52556
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Simulated atmosphere: 2.30 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : OpSimAtmosphere: Observe atmosphere: 0.96 seconds (1 calls)
Creating atm_cache/1/3/8
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0636629 s.
4934 / 12420(39.7262 %) volume elements are needed for the simulation
nx = 30 ny = 9 nz = 46
wx = 0.896367 wy = -0.179217 wz = 1.53027
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Simulated atmosphere: 2.44 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Observe atmosphere: 1.45 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0212307 s.
2627 / 2880(91.2153 %) volume elements are needed for the simulation
nx = 20 ny = 8 nz = 18
wx = 0.896367 wy = -0.179217 wz = 1.53027
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Simulated atmosphere: 2.35 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Observe atmosphere: 1.41 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00159407 s.
224 / 224(100 %) volume elements are needed for the simulation
nx = 8 ny = 4 nz = 7
wx = 0.896367 wy = -0.179217 wz = 1.53027
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Simulated atmosphere: 2.40 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : OpSimAtmosphere: Observe atmosphere: 0.87 seconds (1 calls)
Creating atm_cache/2/4/7
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.058518 s.
3678 / 8424(43.661 %) volume elements are needed for the simulation
nx = 36 ny = 6 nz = 39
wx = 1.15723 wy = -0.0864089 wz = 1.35302
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Simulated atmosphere: 2.47 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Observe atmosphere: 1.07 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0166473 s.
2281 / 2464(92.5731 %) volume elements are needed for the simulation
nx = 22 ny = 7 nz = 16
wx = 1.15723 wy = -0.0864089 wz = 1.35302
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Simulated atmosphere: 2.37 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Observe atmosphere: 1.63 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.000501066 s.
252 / 252(100 %) volume elements are needed for the simulation
nx = 9 ny = 4 nz = 7
wx = 1.15723 wy = -0.0864089 wz = 1.35302
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Simulated atmosphere: 2.41 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : OpSimAtmosphere: Observe atmosphere: 0.83 seconds (1 calls)
Creating atm_cache/3/4/0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0562103 s.
7147 / 32994(21.6615 %) volume elements are needed for the simulation
nx = 26 ny = 47 nz = 27
wx = -0.778339 wy = 1.64021 wz = -0.91003
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Simulated atmosphere: 5.04 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Observe atmosphere: 1.30 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0265778 s.
3949 / 4788(82.477 %) volume elements are needed for the simulation
nx = 19 ny = 21 nz = 12
wx = -0.778339 wy = 1.64021 wz = -0.91003
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Simulated atmosphere: 2.75 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Observe atmosphere: 1.19 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00716825 s.
432 / 432(100 %) volume elements are needed for the simulation
nx = 8 ny = 9 nz = 6
wx = -0.778339 wy = 1.64021 wz = -0.91003
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Simulated atmosphere: 2.37 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : OpSimAtmosphere: Observe atmosphere: 0.87 seconds (1 calls)
Creating atm_cache/4/3/4
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0434568 s.
7132 / 32994(21.6161 %) volume elements are needed for the simulation
nx = 26 ny = 47 nz = 27
wx = -0.786375 wy = 1.63111 wz = -0.919426
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Simulated atmosphere: 5.19 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Observe atmosphere: 0.92 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0192945 s.
3788 / 4560(83.0702 %) volume elements are needed for the simulation
nx = 19 ny = 20 nz = 12
wx = -0.786375 wy = 1.63111 wz = -0.919426
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Simulated atmosphere: 2.81 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Observe atmosphere: 1.13 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00130598 s.
384 / 384(100 %) volume elements are needed for the simulation
nx = 8 ny = 8 nz = 6
wx = -0.786375 wy = 1.63111 wz = -0.919426
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Simulated atmosphere: 2.27 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : OpSimAtmosphere: Observe atmosphere: 0.81 seconds (1 calls)
Creating atm_cache/5/2/7
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0429755 s.
6925 / 30375(22.7984 %) volume elements are needed for the simulation
nx = 25 ny = 45 nz = 27
wx = -0.796726 wy = 1.61918 wz = -0.931529
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Simulated atmosphere: 4.43 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Observe atmosphere: 0.84 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0239603 s.
3678 / 4332(84.903 %) volume elements are needed for the simulation
nx = 19 ny = 19 nz = 12
wx = -0.796726 wy = 1.61918 wz = -0.931529
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Simulated atmosphere: 3.34 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Observe atmosphere: 1.26 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0010181 s.
384 / 384(100 %) volume elements are needed for the simulation
nx = 8 ny = 8 nz = 6
wx = -0.796726 wy = 1.61918 wz = -0.931529
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Simulated atmosphere: 2.37 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : OpSimAtmosphere: Observe atmosphere: 0.91 seconds (1 calls)
Creating atm_cache/6/4/1
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.120595 s.
13189 / 62304(21.1688 %) volume elements are needed for the simulation
nx = 24 ny = 118 nz = 22
wx = 0.69894 wy = -4.32774 wz = 0.704574
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Simulated atmosphere: 6.96 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Observe atmosphere: 1.17 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0187091 s.
6714 / 8170(82.1787 %) volume elements are needed for the simulation
nx = 19 ny = 43 nz = 10
wx = 0.69894 wy = -4.32774 wz = 0.704574
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Simulated atmosphere: 2.67 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Observe atmosphere: 1.20 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00197843 s.
640 / 640(100 %) volume elements are needed for the simulation
nx = 8 ny = 16 nz = 5
wx = 0.69894 wy = -4.32774 wz = 0.704574
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Simulated atmosphere: 2.47 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : OpSimAtmosphere: Observe atmosphere: 0.87 seconds (1 calls)
Creating atm_cache/7/3/4
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0424719 s.
13099 / 62304(21.0243 %) volume elements are needed for the simulation
nx = 24 ny = 118 nz = 22
wx = 0.711822 wy = -4.3235 wz = 0.71756
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Simulated atmosphere: 7.85 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Observe atmosphere: 0.97 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0164715 s.
7256 / 8987(80.7388 %) volume elements are needed for the simulation
nx = 19 ny = 43 nz = 11
wx = 0.711822 wy = -4.3235 wz = 0.71756
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Simulated atmosphere: 2.66 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Observe atmosphere: 1.11 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00194374 s.
640 / 640(100 %) volume elements are needed for the simulation
nx = 8 ny = 16 nz = 5
wx = 0.711822 wy = -4.3235 wz = 0.71756
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Simulated atmosphere: 2.31 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : OpSimAtmosphere: Observe atmosphere: 0.81 seconds (1 calls)
Creating atm_cache/8/2/7
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.54781 s.
24348 / 834240(2.91858 %) volume elements are needed for the simulation
nx = 80 ny = 132 nz = 79
wx = -4.46752 wy = -7.67197 wz = -4.50353
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Simulated atmosphere: 4.90 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Observe atmosphere: 0.82 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0479133 s.
13129 / 48692(26.9634 %) volume elements are needed for the simulation
nx = 37 ny = 47 nz = 28
wx = -4.46752 wy = -7.67197 wz = -4.50353
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Simulated atmosphere: 4.07 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Observe atmosphere: 0.80 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0051247 s.
2115 / 2618(80.7869 %) volume elements are needed for the simulation
nx = 14 ny = 17 nz = 11
wx = -4.46752 wy = -7.67197 wz = -4.50353
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Simulated atmosphere: 2.34 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Observe atmosphere: 0.53 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Instantiating atmosphere for t = 502.2666666507721
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0496896 s.
13788 / 181937(7.57845 %) volume elements are needed for the simulation
nx = 49 ny = 79 nz = 47
wx = -4.46752 wy = -7.67197 wz = -4.50353
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Simulated atmosphere: 4.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Observe atmosphere: 0.45 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0180386 s.
7413 / 14580(50.8436 %) volume elements are needed for the simulation
nx = 27 ny = 30 nz = 18
wx = -4.46752 wy = -7.67197 wz = -4.50353
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Simulated atmosphere: 3.11 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Observe atmosphere: 0.53 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.00763406 s.
1021 / 1056(96.6856 %) volume elements are needed for the simulation
nx = 11 ny = 12 nz = 8
wx = -4.46752 wy = -7.67197 wz = -4.50353
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Simulated atmosphere: 2.87 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : OpSimAtmosphere: Observe atmosphere: 0.37 seconds (1 calls)
Creating atm_cache/9/2/0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.42242 s.
24200 / 819546(2.95285 %) volume elements are needed for the simulation
nx = 79 ny = 133 nz = 78
wx = -4.41085 wy = -7.7378 wz = -4.4464
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Simulated atmosphere: 14.29 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Observe atmosphere: 0.72 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0837811 s.
13059 / 47376(27.5646 %) volume elements are needed for the simulation
nx = 36 ny = 47 nz = 28
wx = -4.41085 wy = -7.7378 wz = -4.4464
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Simulated atmosphere: 4.65 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Observe atmosphere: 0.77 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00575278 s.
2109 / 2618(80.5577 %) volume elements are needed for the simulation
nx = 14 ny = 17 nz = 11
wx = -4.41085 wy = -7.7378 wz = -4.4464
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Simulated atmosphere: 2.48 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Observe atmosphere: 0.57 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Instantiating atmosphere for t = 502.2666666507721
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0520545 s.
12226 / 131208(9.31803 %) volume elements are needed for the simulation
nx = 44 ny = 71 nz = 42
wx = -4.41085 wy = -7.7378 wz = -4.4464
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Simulated atmosphere: 6.46 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Observe atmosphere: 0.41 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0226531 s.
6720 / 11900(56.4706 %) volume elements are needed for the simulation
nx = 25 ny = 28 nz = 17
wx = -4.41085 wy = -7.7378 wz = -4.4464
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Simulated atmosphere: 3.03 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Observe atmosphere: 0.46 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.00144109 s.
762 / 770(98.961 %) volume elements are needed for the simulation
nx = 10 ny = 11 nz = 7
wx = -4.41085 wy = -7.7378 wz = -4.4464
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Simulated atmosphere: 2.50 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : OpSimAtmosphere: Observe atmosphere: 0.33 seconds (1 calls)
Creating atm_cache/0/2/0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.372803 s.
23343 / 725152(3.21905 %) volume elements are needed for the simulation
nx = 62 ny = 136 nz = 86
wx = -3.41536 wy = -7.92436 wz = -4.96382
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Simulated atmosphere: 15.40 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Observe atmosphere: 0.72 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0390765 s.
13116 / 46128(28.4339 %) volume elements are needed for the simulation
nx = 31 ny = 48 nz = 31
wx = -3.41536 wy = -7.92436 wz = -4.96382
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Simulated atmosphere: 3.31 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Observe atmosphere: 0.80 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00421609 s.
1878 / 2244(83.6898 %) volume elements are needed for the simulation
nx = 11 ny = 17 nz = 12
wx = -3.41536 wy = -7.92436 wz = -4.96382
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Simulated atmosphere: 2.50 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Observe atmosphere: 0.55 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Instantiating atmosphere for t = 502.2666666507721
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0696728 s.
14880 / 213066(6.98375 %) volume elements are needed for the simulation
nx = 42 ny = 89 nz = 57
wx = -3.41536 wy = -7.92436 wz = -4.96382
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Simulated atmosphere: 4.32 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Observe atmosphere: 0.48 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0216808 s.
8108 / 16632(48.7494 %) volume elements are needed for the simulation
nx = 24 ny = 33 nz = 21
wx = -3.41536 wy = -7.92436 wz = -4.96382
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Simulated atmosphere: 2.73 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Observe atmosphere: 0.58 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.00813484 s.
953 / 972(98.0453 %) volume elements are needed for the simulation
nx = 9 ny = 12 nz = 9
wx = -3.41536 wy = -7.92436 wz = -4.96382
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Simulated atmosphere: 2.47 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : OpSimAtmosphere: Observe atmosphere: 0.73 seconds (1 calls)
Creating atm_cache/1/1/7
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.353218 s.
23233 / 707112(3.28562 %) volume elements are needed for the simulation
nx = 61 ny = 138 nz = 84
wx = -3.33602 wy = -8.02886 wz = -4.84851
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Simulated atmosphere: 4.82 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Observe atmosphere: 0.68 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0422098 s.
12988 / 44100(29.4512 %) volume elements are needed for the simulation
nx = 30 ny = 49 nz = 30
wx = -3.33602 wy = -8.02886 wz = -4.84851
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Simulated atmosphere: 3.53 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Observe atmosphere: 0.82 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00363514 s.
1834 / 2178(84.2057 %) volume elements are needed for the simulation
nx = 11 ny = 18 nz = 11
wx = -3.33602 wy = -8.02886 wz = -4.84851
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Simulated atmosphere: 2.40 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Observe atmosphere: 0.67 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Instantiating atmosphere for t = 502.2666666507721
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0591059 s.
14713 / 202950(7.24957 %) volume elements are needed for the simulation
nx = 41 ny = 90 nz = 55
wx = -3.33602 wy = -8.02886 wz = -4.84851
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Simulated atmosphere: 3.67 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Observe atmosphere: 0.47 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.0203768 s.
8239 / 17136(48.0801 %) volume elements are needed for the simulation
nx = 24 ny = 34 nz = 21
wx = -3.33602 wy = -8.02886 wz = -4.84851
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Simulated atmosphere: 2.46 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Observe atmosphere: 0.52 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Simulating the atmosphere for t = 502.2666666507721
Volume compressed in 0.00140537 s.
920 / 936(98.2906 %) volume elements are needed for the simulation
nx = 9 ny = 13 nz = 8
wx = -3.33602 wy = -8.02886 wz = -4.84851
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Simulated atmosphere: 2.41 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : OpSimAtmosphere: Observe atmosphere: 0.41 seconds (1 calls)
Creating atm_cache/2/1/4
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.319919 s.
22879 / 620928(3.68465 %) volume elements are needed for the simulation
nx = 56 ny = 144 nz = 77
wx = -2.85238 wy = -7.81801 wz = -4.14558
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Simulated atmosphere: 11.55 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Observe atmosphere: 0.81 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0535055 s.
12922 / 41412(31.2035 %) volume elements are needed for the simulation
nx = 29 ny = 51 nz = 28
wx = -2.85238 wy = -7.81801 wz = -4.14558
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Simulated atmosphere: 4.82 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Observe atmosphere: 1.12 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00656105 s.
1849 / 2178(84.8944 %) volume elements are needed for the simulation
nx = 11 ny = 18 nz = 11
wx = -2.85238 wy = -7.81801 wz = -4.14558
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Simulated atmosphere: 2.60 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Observe atmosphere: 0.60 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Instantiating atmosphere for t = 537.7999999523163
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Simulating the atmosphere for t = 537.7999999523163
Volume compressed in 0.0338742 s.
11931 / 108704(10.9757 %) volume elements are needed for the simulation
nx = 32 ny = 79 nz = 43
wx = -2.85238 wy = -7.81801 wz = -4.14558
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Simulated atmosphere: 7.52 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Observe atmosphere: 0.46 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Simulating the atmosphere for t = 537.7999999523163
Volume compressed in 0.0120021 s.
6717 / 11067(60.694 %) volume elements are needed for the simulation
nx = 21 ny = 31 nz = 17
wx = -2.85238 wy = -7.81801 wz = -4.14558
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Simulated atmosphere: 2.85 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Observe atmosphere: 0.81 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Simulating the atmosphere for t = 537.7999999523163
Volume compressed in 0.0270543 s.
672 / 672(100 %) volume elements are needed for the simulation
nx = 8 ny = 12 nz = 7
wx = -2.85238 wy = -7.81801 wz = -4.14558
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Simulated atmosphere: 2.62 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : OpSimAtmosphere: Observe atmosphere: 0.37 seconds (1 calls)
Creating atm_cache/3/1/0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Setting up atmosphere simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Instantiating atmosphere for t = 0.0
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.298858 s.
22788 / 591300(3.85388 %) volume elements are needed for the simulation
nx = 54 ny = 146 nz = 75
wx = -2.74515 wy = -7.93658 wz = -3.98974
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Simulated atmosphere: 13.62 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Observe atmosphere: 0.84 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.0458637 s.
12599 / 38556(32.6771 %) volume elements are needed for the simulation
nx = 28 ny = 51 nz = 27
wx = -2.74515 wy = -7.93658 wz = -3.98974
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Simulated atmosphere: 5.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Observe atmosphere: 0.93 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Simulating the atmosphere for t = 0.0
Volume compressed in 0.00339516 s.
1636 / 1800(90.8889 %) volume elements are needed for the simulation
nx = 10 ny = 18 nz = 10
wx = -2.74515 wy = -7.93658 wz = -3.98974
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Simulated atmosphere: 2.54 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Observe atmosphere: 0.62 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Instantiating atmosphere for t = 537.7999999523163
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Simulating the atmosphere for t = 537.7999999523163
Volume compressed in 0.022136 s.
10376 / 77256(13.4307 %) volume elements are needed for the simulation
nx = 29 ny = 72 nz = 37
wx = -2.74515 wy = -7.93658 wz = -3.98974
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Simulated atmosphere: 5.80 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Observe atmosphere: 0.70 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Simulating the atmosphere for t = 537.7999999523163
Volume compressed in 0.0138107 s.
5774 / 8400(68.7381 %) volume elements are needed for the simulation
nx = 20 ny = 28 nz = 15
wx = -2.74515 wy = -7.93658 wz = -3.98974
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Simulated atmosphere: 2.70 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Observe atmosphere: 0.48 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Initialize atmosphere: 0.00 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Simulating the atmosphere for t = 537.7999999523163
Volume compressed in 0.0136404 s.
616 / 616(100 %) volume elements are needed for the simulation
nx = 8 ny = 11 nz = 7
wx = -2.74515 wy = -7.93658 wz = -3.98974
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Simulated atmosphere: 2.59 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Observing the atmosphere
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : OpSimAtmosphere: Observe atmosphere: 0.56 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Simulated and observed atmosphere: 36.38 seconds (1 calls)
TOAST INFO: Atmosphere simulation: 319.81 seconds (1 calls)
TOAST INFO: Processing frequency 100.0GHz 1 / 1, MC = 0
TOAST INFO: Scaling atmosphere by frequency
TOAST INFO: Atmosphere scaling: 29.14 seconds (1 calls)
TOAST INFO: Simulating noise
TOAST INFO: Simulate noise: 18.04 seconds (1 calls)
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-0 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-1 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-2 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-0-3 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-0 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-1 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-2 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-1-3 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-0 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-1 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-2 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-2-3 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-0 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-1 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-2 : Observing the scan-synchronous signal
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Setting up SSS simulation
TOAST INFO: 0 : CES-Atacama-LAT-small_patch-3-3 : Observing the scan-synchronous signal
TOAST INFO: Making maps
TOAST INFO: Mapping groundsim000_100_telescope_all_time_all
TOAST INFO: OpMadam: 100.00 % of samples are included in valid intervals.
TOAST INFO: Collect period ranges: 0.10 seconds (1 calls)
TOAST INFO: Collect dataset dimensions: 0.10 seconds (1 calls)
TOAST INFO: Stage time: 0.11 seconds (1 calls)
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Sigma is 76.438962 arcmin (0.022235 rad)
-> fwhm is 180.000000 arcmin
Sigma is 0.000000 arcmin (0.000000 rad)
-> fwhm is 0.000000 arcmin
Memory usage after staging time
total : 62.251 GB
available : 51.458 GB
percent : 17.300 %
used : 9.559 GB
free : 44.296 GB
active : 14.215 GB
inactive : 2.604 GB
buffers : 406.410 MB
cached : 7.999 GB
shared : 749.758 MB
slab : 555.844 MB
TOAST INFO: Stage signal: 0.79 seconds (1 calls)
Node has 3.809 GB allocated in TOAST TOD caches and 0.003 GB in Madam caches (3.812 GB total) after staging time
Memory usage after staging signal
total : 62.251 GB
available : 50.719 GB
percent : 18.500 %
used : 10.298 GB
free : 43.557 GB
active : 14.954 GB
inactive : 2.604 GB
buffers : 406.410 MB
cached : 8.000 GB
shared : 749.758 MB
slab : 555.906 MB
TOAST INFO: Stage pixels: 1.33 seconds (1 calls)
Node has 3.809 GB allocated in TOAST TOD caches and 0.742 GB in Madam caches (4.551 GB total) after staging signal
Memory usage after staging pixels
total : 62.251 GB
available : 50.346 GB
percent : 19.100 %
used : 10.671 GB
free : 43.184 GB
active : 15.325 GB
inactive : 2.604 GB
buffers : 406.426 MB
cached : 7.999 GB
shared : 749.758 MB
slab : 555.848 MB
TOAST INFO: Stage pixel weights: 1.53 seconds (1 calls)
Node has 3.070 GB allocated in TOAST TOD caches and 1.112 GB in Madam caches (4.182 GB total) after staging pixels
Memory usage after staging pixel weights
total : 62.251 GB
available : 49.238 GB
percent : 20.900 %
used : 11.779 GB
free : 42.076 GB
active : 16.430 GB
inactive : 2.604 GB
buffers : 406.426 MB
cached : 7.999 GB
shared : 749.758 MB
slab : 555.848 MB
TOAST INFO: Stage all data: 4.12 seconds (1 calls)
TOAST INFO: Collect PSD info: 0.00 seconds (1 calls)
Node has 0.852 GB allocated in TOAST TOD caches and 2.220 GB in Madam caches (3.073 GB total) after staging pixel weights
Memory usage just before calling libmadam.destripe
total : 62.251 GB
available : 49.238 GB
percent : 20.900 %
used : 11.779 GB
free : 42.076 GB
active : 16.429 GB
inactive : 2.604 GB
buffers : 406.426 MB
cached : 7.999 GB
shared : 749.758 MB
slab : 555.848 MB
OMP: 1 tasks with 12 procs per node, 12 threads per task.
Program MADAM
Destriping of CMB data with a noise filter
Version 3.7
Examining periods
Flagged 0 samples on 0 periods that had less than 0.100% of unflagged samples
Total number of samples (single detector): 390600
Zero-weight fraction (all detectors): 0.000 %
Flagged fraction (all detectors): 0.364 %
Total number of intervals: 16
Max number of samples per task: 390600
Initializing parameters
Adjusting noise weights using noise spectra
Polarized detectors present: Will produce polarization maps
noise_weights_from_psd = T
radiometers = T
mode_detweight = 0
istart_mission = 0
nosamples_tot = 390600
Initializing parallelization
ntasks = 1 Number or processes
nosamples_tot = 390600 Total samples
nosamples_proc_max = 390600 Samples/process
noba_short_tot = 13020 Total baselines
noba_short_pp_max = 870 Longest interval in baselines
noba_short_max = 13020 Baselines/process
MCMode = F
write_cut = F
basis_func = Legendre Destriping function basis
basis_order = 0 Destriping function order
bin_subsets = F
ntasks = 1 Number of processes
nthreads = 12 Number of threads per process
info = 3 Screen output level
fsample = 30.0000 Sampling frequency (Hz)
nmap = 3 Polarization included
ncc = 6 Independent wcov elements
Input files:
nside_map = 512 Healpix resolution (output map)
nside_cross = 256 Healpix resolution (destriping)
nside_submap = 16 Submap resolution
allreduce = T use allreduce to communicate
reassign_submaps = T minimize communication by reassigning submaps
pixmode_map = 2 Pixel rejection criterion (output map)
pixmode_cross = 2 Pixel rejection criterion (destriping)
pixlim_map = 1.00000E-02 Pixel rejection limit (output map)
pixlim_cross = 1.00000E-03 Pixel rejection limit (destriping)
Standard mode
psdlen = 1000000 Length of requested noise PSD
psd_downsample = 10 PSD downsampling factor
kfirst = T First destriping ON
dnshort = 30.0000 Baseline length (samples)
= 1.0000 seconds
kfilter = T Noise filter ON
unaligned_fft = F
cglimit = 1.00000E-12 Iteration convergence limit
iter_min = 3 Minimum number of iterations
iter_max = 200 Maximum number of iterations
precond_width_min = 100 Min width of the preconditioner band matrix
precond_width_max = 100 Max width of the preconditioner band matrix
use_fprecond = F use C_a preconditioner
use_cgprecond = F use CG preconditioner
flag_by_horn = F Flags are independent
mode_detweight = 0 Detector weighting mode: sigma from simulation file
time_unit = pp Time unit = pointing period
mission_time = 16 Mission length in time units
nosamples_tot = 390600 Total samples
= 3.6167 hours
Detectors available on the FIRST process and noise according to the FIRST period
detector sigma weight 1/sqrt(weight)
fake_000A 0.52049E-01 369.13 0.52049E-01
fake_000B 0.52049E-01 369.13 0.52049E-01
fake_001A 0.52049E-01 369.13 0.52049E-01
fake_001B 0.52049E-01 369.13 0.52049E-01
fake_002A 0.52049E-01 369.13 0.52049E-01
fake_002B 0.52049E-01 369.13 0.52049E-01
fake_003A 0.52049E-01 369.13 0.52049E-01
fake_003B 0.52049E-01 369.13 0.52049E-01
fake_004A 0.52049E-01 369.13 0.52049E-01
fake_004B 0.52049E-01 369.13 0.52049E-01
fake_005A 0.52049E-01 369.13 0.52049E-01
fake_005B 0.52049E-01 369.13 0.52049E-01
fake_006A 0.52049E-01 369.13 0.52049E-01
fake_006B 0.52049E-01 369.13 0.52049E-01
fake_007A 0.52049E-01 369.13 0.52049E-01
fake_007B 0.52049E-01 369.13 0.52049E-01
fake_008A 0.52049E-01 369.13 0.52049E-01
fake_008B 0.52049E-01 369.13 0.52049E-01
fake_009A 0.52049E-01 369.13 0.52049E-01
fake_009B 0.52049E-01 369.13 0.52049E-01
fake_010A 0.52049E-01 369.13 0.52049E-01
fake_010B 0.52049E-01 369.13 0.52049E-01
fake_011A 0.52049E-01 369.13 0.52049E-01
fake_011B 0.52049E-01 369.13 0.52049E-01
fake_012A 0.52049E-01 369.13 0.52049E-01
fake_012B 0.52049E-01 369.13 0.52049E-01
fake_013A 0.52049E-01 369.13 0.52049E-01
fake_013B 0.52049E-01 369.13 0.52049E-01
fake_014A 0.52049E-01 369.13 0.52049E-01
fake_014B 0.52049E-01 369.13 0.52049E-01
fake_015A 0.52049E-01 369.13 0.52049E-01
fake_015B 0.52049E-01 369.13 0.52049E-01
fake_016A 0.52049E-01 369.13 0.52049E-01
fake_016B 0.52049E-01 369.13 0.52049E-01
fake_017A 0.52049E-01 369.13 0.52049E-01
fake_017B 0.52049E-01 369.13 0.52049E-01
fake_018A 0.52049E-01 369.13 0.52049E-01
fake_018B 0.52049E-01 369.13 0.52049E-01
fake_019A 0.52049E-01 369.13 0.52049E-01
fake_019B 0.52049E-01 369.13 0.52049E-01
fake_020A 0.52049E-01 369.13 0.52049E-01
fake_020B 0.52049E-01 369.13 0.52049E-01
fake_021A 0.52049E-01 369.13 0.52049E-01
fake_021B 0.52049E-01 369.13 0.52049E-01
fake_022A 0.52049E-01 369.13 0.52049E-01
fake_022B 0.52049E-01 369.13 0.52049E-01
fake_023A 0.52049E-01 369.13 0.52049E-01
fake_023B 0.52049E-01 369.13 0.52049E-01
fake_024A 0.52049E-01 369.13 0.52049E-01
fake_024B 0.52049E-01 369.13 0.52049E-01
fake_025A 0.52049E-01 369.13 0.52049E-01
fake_025B 0.52049E-01 369.13 0.52049E-01
fake_026A 0.52049E-01 369.13 0.52049E-01
fake_026B 0.52049E-01 369.13 0.52049E-01
fake_027A 0.52049E-01 369.13 0.52049E-01
fake_027B 0.52049E-01 369.13 0.52049E-01
fake_028A 0.52049E-01 369.13 0.52049E-01
fake_028B 0.52049E-01 369.13 0.52049E-01
fake_029A 0.52049E-01 369.13 0.52049E-01
fake_029B 0.52049E-01 369.13 0.52049E-01
fake_030A 0.52049E-01 369.13 0.52049E-01
fake_030B 0.52049E-01 369.13 0.52049E-01
fake_031A 0.52049E-01 369.13 0.52049E-01
fake_031B 0.52049E-01 369.13 0.52049E-01
fake_032A 0.52049E-01 369.13 0.52049E-01
fake_032B 0.52049E-01 369.13 0.52049E-01
fake_033A 0.52049E-01 369.13 0.52049E-01
fake_033B 0.52049E-01 369.13 0.52049E-01
fake_034A 0.52049E-01 369.13 0.52049E-01
fake_034B 0.52049E-01 369.13 0.52049E-01
fake_035A 0.52049E-01 369.13 0.52049E-01
fake_035B 0.52049E-01 369.13 0.52049E-01
fake_036A 0.52049E-01 369.13 0.52049E-01
fake_036B 0.52049E-01 369.13 0.52049E-01
fake_037A 0.52049E-01 369.13 0.52049E-01
fake_037B 0.52049E-01 369.13 0.52049E-01
fake_038A 0.52049E-01 369.13 0.52049E-01
fake_038B 0.52049E-01 369.13 0.52049E-01
fake_039A 0.52049E-01 369.13 0.52049E-01
fake_039B 0.52049E-01 369.13 0.52049E-01
fake_040A 0.52049E-01 369.13 0.52049E-01
fake_040B 0.52049E-01 369.13 0.52049E-01
fake_041A 0.52049E-01 369.13 0.52049E-01
fake_041B 0.52049E-01 369.13 0.52049E-01
fake_042A 0.52049E-01 369.13 0.52049E-01
fake_042B 0.52049E-01 369.13 0.52049E-01
fake_043A 0.52049E-01 369.13 0.52049E-01
fake_043B 0.52049E-01 369.13 0.52049E-01
fake_044A 0.52049E-01 369.13 0.52049E-01
fake_044B 0.52049E-01 369.13 0.52049E-01
fake_045A 0.52049E-01 369.13 0.52049E-01
fake_045B 0.52049E-01 369.13 0.52049E-01
fake_046A 0.52049E-01 369.13 0.52049E-01
fake_046B 0.52049E-01 369.13 0.52049E-01
fake_047A 0.52049E-01 369.13 0.52049E-01
fake_047B 0.52049E-01 369.13 0.52049E-01
fake_048A 0.52049E-01 369.13 0.52049E-01
fake_048B 0.52049E-01 369.13 0.52049E-01
fake_049A 0.52049E-01 369.13 0.52049E-01
fake_049B 0.52049E-01 369.13 0.52049E-01
fake_050A 0.52049E-01 369.13 0.52049E-01
fake_050B 0.52049E-01 369.13 0.52049E-01
fake_051A 0.52049E-01 369.13 0.52049E-01
fake_051B 0.52049E-01 369.13 0.52049E-01
fake_052A 0.52049E-01 369.13 0.52049E-01
fake_052B 0.52049E-01 369.13 0.52049E-01
fake_053A 0.52049E-01 369.13 0.52049E-01
fake_053B 0.52049E-01 369.13 0.52049E-01
fake_054A 0.52049E-01 369.13 0.52049E-01
fake_054B 0.52049E-01 369.13 0.52049E-01
fake_055A 0.52049E-01 369.13 0.52049E-01
fake_055B 0.52049E-01 369.13 0.52049E-01
fake_056A 0.52049E-01 369.13 0.52049E-01
fake_056B 0.52049E-01 369.13 0.52049E-01
fake_057A 0.52049E-01 369.13 0.52049E-01
fake_057B 0.52049E-01 369.13 0.52049E-01
fake_058A 0.52049E-01 369.13 0.52049E-01
fake_058B 0.52049E-01 369.13 0.52049E-01
fake_059A 0.52049E-01 369.13 0.52049E-01
fake_059B 0.52049E-01 369.13 0.52049E-01
fake_060A 0.52049E-01 369.13 0.52049E-01
fake_060B 0.52049E-01 369.13 0.52049E-01
fake_061A 0.52049E-01 369.13 0.52049E-01
fake_061B 0.52049E-01 369.13 0.52049E-01
fake_062A 0.52049E-01 369.13 0.52049E-01
fake_062B 0.52049E-01 369.13 0.52049E-01
fake_063A 0.52049E-01 369.13 0.52049E-01
fake_063B 0.52049E-01 369.13 0.52049E-01
fake_064A 0.52049E-01 369.13 0.52049E-01
fake_064B 0.52049E-01 369.13 0.52049E-01
fake_065A 0.52049E-01 369.13 0.52049E-01
fake_065B 0.52049E-01 369.13 0.52049E-01
fake_066A 0.52049E-01 369.13 0.52049E-01
fake_066B 0.52049E-01 369.13 0.52049E-01
fake_067A 0.52049E-01 369.13 0.52049E-01
fake_067B 0.52049E-01 369.13 0.52049E-01
fake_068A 0.52049E-01 369.13 0.52049E-01
fake_068B 0.52049E-01 369.13 0.52049E-01
fake_069A 0.52049E-01 369.13 0.52049E-01
fake_069B 0.52049E-01 369.13 0.52049E-01
fake_070A 0.52049E-01 369.13 0.52049E-01
fake_070B 0.52049E-01 369.13 0.52049E-01
fake_071A 0.52049E-01 369.13 0.52049E-01
fake_071B 0.52049E-01 369.13 0.52049E-01
fake_072A 0.52049E-01 369.13 0.52049E-01
fake_072B 0.52049E-01 369.13 0.52049E-01
fake_073A 0.52049E-01 369.13 0.52049E-01
fake_073B 0.52049E-01 369.13 0.52049E-01
fake_074A 0.52049E-01 369.13 0.52049E-01
fake_074B 0.52049E-01 369.13 0.52049E-01
fake_075A 0.52049E-01 369.13 0.52049E-01
fake_075B 0.52049E-01 369.13 0.52049E-01
fake_076A 0.52049E-01 369.13 0.52049E-01
fake_076B 0.52049E-01 369.13 0.52049E-01
fake_077A 0.52049E-01 369.13 0.52049E-01
fake_077B 0.52049E-01 369.13 0.52049E-01
fake_078A 0.52049E-01 369.13 0.52049E-01
fake_078B 0.52049E-01 369.13 0.52049E-01
fake_079A 0.52049E-01 369.13 0.52049E-01
fake_079B 0.52049E-01 369.13 0.52049E-01
fake_080A 0.52049E-01 369.13 0.52049E-01
fake_080B 0.52049E-01 369.13 0.52049E-01
fake_081A 0.52049E-01 369.13 0.52049E-01
fake_081B 0.52049E-01 369.13 0.52049E-01
fake_082A 0.52049E-01 369.13 0.52049E-01
fake_082B 0.52049E-01 369.13 0.52049E-01
fake_083A 0.52049E-01 369.13 0.52049E-01
fake_083B 0.52049E-01 369.13 0.52049E-01
fake_084A 0.52049E-01 369.13 0.52049E-01
fake_084B 0.52049E-01 369.13 0.52049E-01
fake_085A 0.52049E-01 369.13 0.52049E-01
fake_085B 0.52049E-01 369.13 0.52049E-01
fake_086A 0.52049E-01 369.13 0.52049E-01
fake_086B 0.52049E-01 369.13 0.52049E-01
fake_087A 0.52049E-01 369.13 0.52049E-01
fake_087B 0.52049E-01 369.13 0.52049E-01
fake_088A 0.52049E-01 369.13 0.52049E-01
fake_088B 0.52049E-01 369.13 0.52049E-01
fake_089A 0.52049E-01 369.13 0.52049E-01
fake_089B 0.52049E-01 369.13 0.52049E-01
fake_090A 0.52049E-01 369.13 0.52049E-01
fake_090B 0.52049E-01 369.13 0.52049E-01
fake_091A 0.52049E-01 369.13 0.52049E-01
fake_091B 0.52049E-01 369.13 0.52049E-01
fake_092A 0.52049E-01 369.13 0.52049E-01
fake_092B 0.52049E-01 369.13 0.52049E-01
fake_093A 0.52049E-01 369.13 0.52049E-01
fake_093B 0.52049E-01 369.13 0.52049E-01
fake_094A 0.52049E-01 369.13 0.52049E-01
fake_094B 0.52049E-01 369.13 0.52049E-01
fake_095A 0.52049E-01 369.13 0.52049E-01
fake_095B 0.52049E-01 369.13 0.52049E-01
fake_096A 0.52049E-01 369.13 0.52049E-01
fake_096B 0.52049E-01 369.13 0.52049E-01
fake_097A 0.52049E-01 369.13 0.52049E-01
fake_097B 0.52049E-01 369.13 0.52049E-01
fake_098A 0.52049E-01 369.13 0.52049E-01
fake_098B 0.52049E-01 369.13 0.52049E-01
fake_099A 0.52049E-01 369.13 0.52049E-01
fake_099B 0.52049E-01 369.13 0.52049E-01
fake_100A 0.52049E-01 369.13 0.52049E-01
fake_100B 0.52049E-01 369.13 0.52049E-01
fake_101A 0.52049E-01 369.13 0.52049E-01
fake_101B 0.52049E-01 369.13 0.52049E-01
fake_102A 0.52049E-01 369.13 0.52049E-01
fake_102B 0.52049E-01 369.13 0.52049E-01
fake_103A 0.52049E-01 369.13 0.52049E-01
fake_103B 0.52049E-01 369.13 0.52049E-01
fake_104A 0.52049E-01 369.13 0.52049E-01
fake_104B 0.52049E-01 369.13 0.52049E-01
fake_105A 0.52049E-01 369.13 0.52049E-01
fake_105B 0.52049E-01 369.13 0.52049E-01
fake_106A 0.52049E-01 369.13 0.52049E-01
fake_106B 0.52049E-01 369.13 0.52049E-01
fake_107A 0.52049E-01 369.13 0.52049E-01
fake_107B 0.52049E-01 369.13 0.52049E-01
fake_108A 0.52049E-01 369.13 0.52049E-01
fake_108B 0.52049E-01 369.13 0.52049E-01
fake_109A 0.52049E-01 369.13 0.52049E-01
fake_109B 0.52049E-01 369.13 0.52049E-01
fake_110A 0.52049E-01 369.13 0.52049E-01
fake_110B 0.52049E-01 369.13 0.52049E-01
fake_111A 0.52049E-01 369.13 0.52049E-01
fake_111B 0.52049E-01 369.13 0.52049E-01
fake_112A 0.52049E-01 369.13 0.52049E-01
fake_112B 0.52049E-01 369.13 0.52049E-01
fake_113A 0.52049E-01 369.13 0.52049E-01
fake_113B 0.52049E-01 369.13 0.52049E-01
fake_114A 0.52049E-01 369.13 0.52049E-01
fake_114B 0.52049E-01 369.13 0.52049E-01
fake_115A 0.52049E-01 369.13 0.52049E-01
fake_115B 0.52049E-01 369.13 0.52049E-01
fake_116A 0.52049E-01 369.13 0.52049E-01
fake_116B 0.52049E-01 369.13 0.52049E-01
fake_117A 0.52049E-01 369.13 0.52049E-01
fake_117B 0.52049E-01 369.13 0.52049E-01
fake_118A 0.52049E-01 369.13 0.52049E-01
fake_118B 0.52049E-01 369.13 0.52049E-01
fake_119A 0.52049E-01 369.13 0.52049E-01
fake_119B 0.52049E-01 369.13 0.52049E-01
fake_120A 0.52049E-01 369.13 0.52049E-01
fake_120B 0.52049E-01 369.13 0.52049E-01
fake_121A 0.52049E-01 369.13 0.52049E-01
fake_121B 0.52049E-01 369.13 0.52049E-01
fake_122A 0.52049E-01 369.13 0.52049E-01
fake_122B 0.52049E-01 369.13 0.52049E-01
fake_123A 0.52049E-01 369.13 0.52049E-01
fake_123B 0.52049E-01 369.13 0.52049E-01
fake_124A 0.52049E-01 369.13 0.52049E-01
fake_124B 0.52049E-01 369.13 0.52049E-01
fake_125A 0.52049E-01 369.13 0.52049E-01
fake_125B 0.52049E-01 369.13 0.52049E-01
fake_126A 0.52049E-01 369.13 0.52049E-01
fake_126B 0.52049E-01 369.13 0.52049E-01
Output files
file_root = groundsim000_100_telescope_all_time_all
file_map = out/00000000/100/groundsim000_100_telescope_all_time_all_map.fits
file_binmap = out/00000000/100/groundsim000_100_telescope_all_time_all_bmap.fits
file_hit = out/00000000/100/groundsim000_100_telescope_all_time_all_hmap.fits
file_matrix = out/00000000/100/groundsim000_100_telescope_all_time_all_wcov_inv.fits
file_wcov = out/00000000/100/groundsim000_100_telescope_all_time_all_wcov.fits
binary_output = F
concatenate_binary = F
TOD memory min = 760.28 MB max = 760.28 MB total = 760.28 MB
Baseline memory min = 100.92 MB max = 100.92 MB total = 100.92 MB
Pointing memory min = 1.85 GB max = 1.85 GB total = 1.85 GB
Initializing filter...
FFT length = 2048
0 : init_filter completed in 0.809 s
Building filter...
Filter memory min = 20.27 MB max = 20.27 MB total = 20.27 MB
Basis function memory min = 0.23 kB max = 0.23 kB total = 0.23 kB
Clock = 1.271 s
0 : reduce_pixels completed in 0.465 s
Total submaps = 3072 submap size = 1024
Local submaps: min = 41 max = 41 mean = 41.00
Submap table memory min = 12.00 kB max = 12.00 kB total = 12.00 kB
local maps memory min = 4.00 MB max = 4.00 MB total = 4.00 MB
Allreduce memory min = 0.01 kB max = 0.01 kB total = 0.01 kB
Map memory min = 456.00 MB max = 456.00 MB total = 456.00 MB
Building pixel matrices...
0 : pixel_matrix completed in 0.696 s
Counting hits...
0 : count_hits completed in 0.398 s
Binning TOD...
0 : bin_tod completed in 0.620 s
Clock = 3.507 s
Destriping TOD
Clock = 3.507 s
Inverting pixel matrices...
6721 pixels solved
779711 pixels unsolved
0 pixels had decoupled intensity
Building RHS...
Succesfully inverted 100.00000000000000 % of the baseline matrices
Inverse rejected on 0.0000000000000000 % of the baseline matrices
0.0000000000000000 % were empty
0 : initialize_a completed in 0.682 s
Constructing preconditioner
Constructed 4064 band preconditioners.
4064 at width = 100
Precond memory min = 2.46 GB max = 2.46 GB total = 2.46 GB
0 : construct_preconditioner completed in 3.855 s
CG iteration begins
rrinit = 2.557489856473344E+12
<rr> = 4.118114726073621E+10
ngood = 3307080
iter rr/rrinit rz2/rz alpha beta time
1 5.468400E-03 6.829579E-11 1.000913E+00 1.820883E-03 ( 0.892s)
2 1.784374E-04 -3.668318E-13 1.154022E+00 9.277998E-02 ( 0.960s)
3 5.851218E-05 6.187400E-14 1.581355E+00 3.191214E-01 ( 0.892s)
4 9.246707E-05 -1.200948E-13 2.922265E+00 6.345769E-01 ( 0.905s)
5 3.739861E-05 3.615546E-14 2.324947E+00 5.652501E-01 ( 0.921s)
6 2.427269E-05 5.103265E-15 3.714984E+00 9.421918E-01 ( 0.934s)
7 1.817642E-05 -5.153771E-14 2.211780E+00 5.460088E-01 ( 0.869s)
8 1.010358E-05 -1.947514E-14 2.505374E+00 4.890754E-01 ( 0.865s)
9 4.625253E-06 5.162390E-14 2.266873E+00 6.170558E-01 ( 0.886s)
10 1.568383E-06 -1.815854E-14 2.421177E+00 3.825338E-01 ( 0.929s)
11 1.979359E-06 2.997472E-15 3.844254E+00 9.205246E-01 ( 0.963s)
12 1.603622E-06 4.612215E-15 3.391072E+00 8.324489E-01 ( 0.862s)
13 1.363277E-06 -2.399863E-14 3.116764E+00 8.559067E-01 ( 0.875s)
14 1.319232E-06 -5.613041E-14 3.551325E+00 9.231652E-01 ( 0.856s)
15 7.026361E-07 4.657277E-14 2.227748E+00 4.874769E-01 ( 0.858s)
16 2.568863E-07 1.331444E-13 2.013022E+00 3.725218E-01 ( 0.867s)
17 5.520192E-08 -2.345712E-14 1.903462E+00 2.483322E-01 ( 0.855s)
18 1.314410E-08 4.236739E-13 2.205333E+00 2.468694E-01 ( 0.880s)
19 5.267298E-09 4.644148E-15 2.166003E+00 4.230376E-01 ( 0.845s)
20 3.059090E-09 -1.925771E-14 2.910539E+00 5.118429E-01 ( 0.878s)
21 1.788248E-09 -1.859522E-13 2.573855E+00 5.231634E-01 ( 0.856s)
22 4.043053E-10 -1.438731E-14 1.764937E+00 2.550270E-01 ( 0.877s)
23 1.298158E-10 4.819783E-13 1.765723E+00 2.984595E-01 ( 0.856s)
24 3.106378E-11 -1.481934E-13 2.060495E+00 2.308294E-01 ( 0.876s)
25 9.035331E-12 -1.988241E-13 2.004777E+00 2.927745E-01 ( 0.866s)
26 2.133207E-12 3.556829E-14 1.893758E+00 2.698460E-01 ( 0.861s)
27 7.420829E-13 2.065133E-13 1.836765E+00 3.161949E-01 ( 0.863s)
Iteration done
27 iteration steps
0 : iterate_a completed in 24.214 s
Subtracting baselines...
0 : subtract_baselines_a completed in 0.768 s
Finalization begins
Clock = 33.044 s
Writing pixel matrix...
Pixel matrix written in out/00000000/100/groundsim000_100_telescope_all_time_all_wcov_inv.fits
0 : Write matrix completed in 0.215 s
Inverting pixel matrices...
26267 pixels solved
3119461 pixels unsolved
0 pixels had decoupled intensity
Constructing output map...
Constructing output map...
Destriped map:
Map 1 2 3
Std 9597.564 uK 762.036906 uK 752.752572 uK
Mean -4165.293 uK -1.697655 uK 14.855244 uK
Min -133824.744 uK -22552.878 uK -29397.548 uK
Max 128525.552 uK 32978.212 uK 23231.619 uK
Writing pixel matrix...
Pixel matrix written in out/00000000/100/groundsim000_100_telescope_all_time_all_wcov.fits
0 : Write matrix completed in 0.228 s
Writing destriped map...
Map written in out/00000000/100/groundsim000_100_telescope_all_time_all_map.fits
Writing binned map...
Binned map written in out/00000000/100/groundsim000_100_telescope_all_time_all_bmap.fits
Writing hits...
Hit count written in out/00000000/100/groundsim000_100_telescope_all_time_all_hmap.fits
Clock = 33.827 s
MEMORY (MB):
Detector pointing min = 1.85 GB max = 1.85 GB total = 1.85 GB
TOD buffer min = 760.28 MB max = 760.28 MB total = 760.28 MB
Maps min = 456.00 MB max = 456.00 MB total = 456.00 MB
Baselines min = 101.43 MB max = 101.43 MB total = 101.43 MB
Basis functions min = 0.00 B max = 0.00 B total = 0.00 B
Noise filter min = 20.27 MB max = 20.27 MB total = 20.27 MB
Preconditioner min = 2.46 GB max = 2.46 GB total = 2.46 GB
Submap table min = 12.00 kB max = 12.00 kB total = 12.00 kB
Temporary maps min = 4.00 MB max = 4.00 MB total = 4.00 MB
All2All buffers min = 0.01 kB max = 0.01 kB total = 0.01 kB
CG work space min = 100.92 MB max = 100.92 MB total = 100.92 MB
NCM min = 0.00 B max = 0.00 B total = 0.00 B
Total min = 5.72 GB max = 5.72 GB total = 5.72 GB
WALL-CLOCK TIME (s):
Initialization mean = 0.4 min = 0.4 max = 0.4
I/O mean = 0.2 min = 0.2 max = 0.2
- Pointing Period boundaries mean = 0.2 min = 0.2 max = 0.2
Building pixel matrices mean = 0.7 min = 0.7 max = 0.7
Inverting pixel matrices mean = 0.1 min = 0.1 max = 0.1
Binning TOD mean = 0.6 min = 0.6 max = 0.6
Counting hits mean = 0.4 min = 0.4 max = 0.4
Filter - initialize mean = 0.8 min = 0.8 max = 0.8
Filter - build mean = 0.1 min = 0.1 max = 0.1
Building preconditioner mean = 3.9 min = 3.9 max = 3.9
Initialization (1. phase) mean = 0.7 min = 0.7 max = 0.7
CG iteration mean = 24.2 min = 24.2 max = 24.2
- TOD - map mean = 10.0 min = 10.0 max = 10.0
- MPI Reduce mean = 0.1 min = 0.1 max = 0.1
- CG ccmultiply mean = 0.1 min = 0.1 max = 0.1
- Map - TOD mean = 5.6 min = 5.6 max = 5.6
- Filtering mean = 2.0 min = 2.0 max = 2.0
- Preconditioning mean = 3.9 min = 3.9 max = 3.9
- Other mean = 2.4 min = 2.4 max = 2.4
Finalization and output mean = 0.8 min = 0.8 max = 0.8
Other mean = 1.0 min = 1.0 max = 1.0
Total 33.8 s ( 0.11 CPU hours)
TOAST INFO: Unstage signal 1 / 1: 0.02 seconds (1 calls)
TOAST INFO: Unstage pixels 1 / 1: 0.48 seconds (1 calls)
TOAST INFO: Unstage pixel weights 1 / 1: 0.67 seconds (1 calls)
TOAST INFO: Unstage all data: 1.17 seconds (1 calls)
TOAST INFO: Mapping groundsim000_100_telescope_all_time_all: 39.56 seconds (1 calls)
TOAST INFO: Madam total: 39.56 seconds (1 calls)
TOAST INFO: Polyfiltering signal
TOAST INFO: Polynomial filtering: 2.76 seconds (1 calls)
TOAST INFO: Ground-filtering signal
TOAST INFO: Ground filtering: 15.40 seconds (1 calls)
TOAST INFO: Making maps
TOAST INFO: Mapping groundsim000_filtered_100_telescope_all_time_all
TOAST INFO: OpMadam: 100.00 % of samples are included in valid intervals.
TOAST INFO: Collect period ranges: 0.12 seconds (1 calls)
TOAST INFO: Collect dataset dimensions: 0.12 seconds (1 calls)
TOAST INFO: Stage time: 0.11 seconds (1 calls)
Memory usage after staging time
total : 62.251 GB
available : 51.125 GB
percent : 17.900 %
used : 9.895 GB
free : 43.589 GB
active : 14.549 GB
inactive : 2.970 GB
buffers : 406.656 MB
cached : 8.370 GB
shared : 746.688 MB
slab : 566.629 MB
TOAST INFO: Stage signal: 0.76 seconds (1 calls)
Node has 3.809 GB allocated in TOAST TOD caches and 0.003 GB in Madam caches (3.812 GB total) after staging time
Memory usage after staging signal
total : 62.251 GB
available : 50.383 GB
percent : 19.100 %
used : 10.636 GB
free : 42.847 GB
active : 15.289 GB
inactive : 2.970 GB
buffers : 406.664 MB
cached : 8.370 GB
shared : 746.750 MB
slab : 566.629 MB
TOAST INFO: Stage pixels: 1.25 seconds (1 calls)
Node has 3.809 GB allocated in TOAST TOD caches and 0.742 GB in Madam caches (4.551 GB total) after staging signal
Memory usage after staging pixels
total : 62.251 GB
available : 50.013 GB
percent : 19.700 %
used : 11.007 GB
free : 42.477 GB
active : 15.659 GB
inactive : 2.970 GB
buffers : 406.664 MB
cached : 8.371 GB
shared : 746.750 MB
slab : 566.879 MB
TOAST INFO: Stage pixel weights: 1.59 seconds (1 calls)
Node has 3.070 GB allocated in TOAST TOD caches and 1.112 GB in Madam caches (4.182 GB total) after staging pixels
Memory usage after staging pixel weights
total : 62.251 GB
available : 48.902 GB
percent : 21.400 %
used : 12.117 GB
free : 41.366 GB
active : 16.768 GB
inactive : 2.971 GB
buffers : 406.672 MB
cached : 8.371 GB
shared : 747.250 MB
slab : 566.672 MB
TOAST INFO: Stage all data: 4.06 seconds (1 calls)
TOAST INFO: Collect PSD info: 0.00 seconds (1 calls)
Node has 0.852 GB allocated in TOAST TOD caches and 2.220 GB in Madam caches (3.073 GB total) after staging pixel weights
Memory usage just before calling libmadam.destripe
total : 62.251 GB
available : 48.902 GB
percent : 21.400 %
used : 12.117 GB
free : 41.366 GB
active : 16.768 GB
inactive : 2.970 GB
buffers : 406.672 MB
cached : 8.371 GB
shared : 746.934 MB
slab : 566.703 MB
OMP: 1 tasks with 12 procs per node, 12 threads per task.
Program MADAM
Destriping of CMB data with a noise filter
Version 3.7
Examining periods
Flagged 0 samples on 0 periods that had less than 0.100% of unflagged samples
Total number of samples (single detector): 390600
Zero-weight fraction (all detectors): 0.000 %
Flagged fraction (all detectors): 0.364 %
Total number of intervals: 16
Max number of samples per task: 390600
Initializing parameters
Adjusting noise weights using noise spectra
Polarized detectors present: Will produce polarization maps
noise_weights_from_psd = T
radiometers = T
mode_detweight = 0
istart_mission = 0
nosamples_tot = 390600
Initializing parallelization
ntasks = 1 Number or processes
nosamples_tot = 390600 Total samples
nosamples_proc_max = 390600 Samples/process
MCMode = F
write_cut = F
basis_func = Legendre Destriping function basis
basis_order = 0 Destriping function order
bin_subsets = F
ntasks = 1 Number of processes
nthreads = 12 Number of threads per process
info = 3 Screen output level
fsample = 30.0000 Sampling frequency (Hz)
nmap = 3 Polarization included
ncc = 6 Independent wcov elements
Input files:
nside_map = 512 Healpix resolution (output map)
nside_cross = 256 Healpix resolution (destriping)
nside_submap = 16 Submap resolution
allreduce = T use allreduce to communicate
reassign_submaps = T minimize communication by reassigning submaps
pixmode_map = 2 Pixel rejection criterion (output map)
pixmode_cross = 2 Pixel rejection criterion (destriping)
pixlim_map = 1.00000E-02 Pixel rejection limit (output map)
pixlim_cross = 1.00000E-03 Pixel rejection limit (destriping)
Standard mode
psdlen = 1000000 Length of requested noise PSD
psd_downsample = 10 PSD downsampling factor
kfirst = F First destriping OFF
cglimit = 1.00000E-12 Iteration convergence limit
iter_min = 3 Minimum number of iterations
iter_max = 200 Maximum number of iterations
precond_width_min = 0 Min width of the preconditioner band matrix
precond_width_max = 0 Max width of the preconditioner band matrix
No preconditioning
flag_by_horn = F Flags are independent
mode_detweight = 0 Detector weighting mode: sigma from simulation file
time_unit = pp Time unit = pointing period
mission_time = 16 Mission length in time units
nosamples_tot = 390600 Total samples
= 3.6167 hours
Detectors available on the FIRST process and noise according to the FIRST period
detector sigma weight 1/sqrt(weight)
fake_000A 0.52049E-01 369.13 0.52049E-01
fake_000B 0.52049E-01 369.13 0.52049E-01
fake_001A 0.52049E-01 369.13 0.52049E-01
fake_001B 0.52049E-01 369.13 0.52049E-01
fake_002A 0.52049E-01 369.13 0.52049E-01
fake_002B 0.52049E-01 369.13 0.52049E-01
fake_003A 0.52049E-01 369.13 0.52049E-01
fake_003B 0.52049E-01 369.13 0.52049E-01
fake_004A 0.52049E-01 369.13 0.52049E-01
fake_004B 0.52049E-01 369.13 0.52049E-01
fake_005A 0.52049E-01 369.13 0.52049E-01
fake_005B 0.52049E-01 369.13 0.52049E-01
fake_006A 0.52049E-01 369.13 0.52049E-01
fake_006B 0.52049E-01 369.13 0.52049E-01
fake_007A 0.52049E-01 369.13 0.52049E-01
fake_007B 0.52049E-01 369.13 0.52049E-01
fake_008A 0.52049E-01 369.13 0.52049E-01
fake_008B 0.52049E-01 369.13 0.52049E-01
fake_009A 0.52049E-01 369.13 0.52049E-01
fake_009B 0.52049E-01 369.13 0.52049E-01
fake_010A 0.52049E-01 369.13 0.52049E-01
fake_010B 0.52049E-01 369.13 0.52049E-01
fake_011A 0.52049E-01 369.13 0.52049E-01
fake_011B 0.52049E-01 369.13 0.52049E-01
fake_012A 0.52049E-01 369.13 0.52049E-01
fake_012B 0.52049E-01 369.13 0.52049E-01
fake_013A 0.52049E-01 369.13 0.52049E-01
fake_013B 0.52049E-01 369.13 0.52049E-01
fake_014A 0.52049E-01 369.13 0.52049E-01
fake_014B 0.52049E-01 369.13 0.52049E-01
fake_015A 0.52049E-01 369.13 0.52049E-01
fake_015B 0.52049E-01 369.13 0.52049E-01
fake_016A 0.52049E-01 369.13 0.52049E-01
fake_016B 0.52049E-01 369.13 0.52049E-01
fake_017A 0.52049E-01 369.13 0.52049E-01
fake_017B 0.52049E-01 369.13 0.52049E-01
fake_018A 0.52049E-01 369.13 0.52049E-01
fake_018B 0.52049E-01 369.13 0.52049E-01
fake_019A 0.52049E-01 369.13 0.52049E-01
fake_019B 0.52049E-01 369.13 0.52049E-01
fake_020A 0.52049E-01 369.13 0.52049E-01
fake_020B 0.52049E-01 369.13 0.52049E-01
fake_021A 0.52049E-01 369.13 0.52049E-01
fake_021B 0.52049E-01 369.13 0.52049E-01
fake_022A 0.52049E-01 369.13 0.52049E-01
fake_022B 0.52049E-01 369.13 0.52049E-01
fake_023A 0.52049E-01 369.13 0.52049E-01
fake_023B 0.52049E-01 369.13 0.52049E-01
fake_024A 0.52049E-01 369.13 0.52049E-01
fake_024B 0.52049E-01 369.13 0.52049E-01
fake_025A 0.52049E-01 369.13 0.52049E-01
fake_025B 0.52049E-01 369.13 0.52049E-01
fake_026A 0.52049E-01 369.13 0.52049E-01
fake_026B 0.52049E-01 369.13 0.52049E-01
fake_027A 0.52049E-01 369.13 0.52049E-01
fake_027B 0.52049E-01 369.13 0.52049E-01
fake_028A 0.52049E-01 369.13 0.52049E-01
fake_028B 0.52049E-01 369.13 0.52049E-01
fake_029A 0.52049E-01 369.13 0.52049E-01
fake_029B 0.52049E-01 369.13 0.52049E-01
fake_030A 0.52049E-01 369.13 0.52049E-01
fake_030B 0.52049E-01 369.13 0.52049E-01
fake_031A 0.52049E-01 369.13 0.52049E-01
fake_031B 0.52049E-01 369.13 0.52049E-01
fake_032A 0.52049E-01 369.13 0.52049E-01
fake_032B 0.52049E-01 369.13 0.52049E-01
fake_033A 0.52049E-01 369.13 0.52049E-01
fake_033B 0.52049E-01 369.13 0.52049E-01
fake_034A 0.52049E-01 369.13 0.52049E-01
fake_034B 0.52049E-01 369.13 0.52049E-01
fake_035A 0.52049E-01 369.13 0.52049E-01
fake_035B 0.52049E-01 369.13 0.52049E-01
fake_036A 0.52049E-01 369.13 0.52049E-01
fake_036B 0.52049E-01 369.13 0.52049E-01
fake_037A 0.52049E-01 369.13 0.52049E-01
fake_037B 0.52049E-01 369.13 0.52049E-01
fake_038A 0.52049E-01 369.13 0.52049E-01
fake_038B 0.52049E-01 369.13 0.52049E-01
fake_039A 0.52049E-01 369.13 0.52049E-01
fake_039B 0.52049E-01 369.13 0.52049E-01
fake_040A 0.52049E-01 369.13 0.52049E-01
fake_040B 0.52049E-01 369.13 0.52049E-01
fake_041A 0.52049E-01 369.13 0.52049E-01
fake_041B 0.52049E-01 369.13 0.52049E-01
fake_042A 0.52049E-01 369.13 0.52049E-01
fake_042B 0.52049E-01 369.13 0.52049E-01
fake_043A 0.52049E-01 369.13 0.52049E-01
fake_043B 0.52049E-01 369.13 0.52049E-01
fake_044A 0.52049E-01 369.13 0.52049E-01
fake_044B 0.52049E-01 369.13 0.52049E-01
fake_045A 0.52049E-01 369.13 0.52049E-01
fake_045B 0.52049E-01 369.13 0.52049E-01
fake_046A 0.52049E-01 369.13 0.52049E-01
fake_046B 0.52049E-01 369.13 0.52049E-01
fake_047A 0.52049E-01 369.13 0.52049E-01
fake_047B 0.52049E-01 369.13 0.52049E-01
fake_048A 0.52049E-01 369.13 0.52049E-01
fake_048B 0.52049E-01 369.13 0.52049E-01
fake_049A 0.52049E-01 369.13 0.52049E-01
fake_049B 0.52049E-01 369.13 0.52049E-01
fake_050A 0.52049E-01 369.13 0.52049E-01
fake_050B 0.52049E-01 369.13 0.52049E-01
fake_051A 0.52049E-01 369.13 0.52049E-01
fake_051B 0.52049E-01 369.13 0.52049E-01
fake_052A 0.52049E-01 369.13 0.52049E-01
fake_052B 0.52049E-01 369.13 0.52049E-01
fake_053A 0.52049E-01 369.13 0.52049E-01
fake_053B 0.52049E-01 369.13 0.52049E-01
fake_054A 0.52049E-01 369.13 0.52049E-01
fake_054B 0.52049E-01 369.13 0.52049E-01
fake_055A 0.52049E-01 369.13 0.52049E-01
fake_055B 0.52049E-01 369.13 0.52049E-01
fake_056A 0.52049E-01 369.13 0.52049E-01
fake_056B 0.52049E-01 369.13 0.52049E-01
fake_057A 0.52049E-01 369.13 0.52049E-01
fake_057B 0.52049E-01 369.13 0.52049E-01
fake_058A 0.52049E-01 369.13 0.52049E-01
fake_058B 0.52049E-01 369.13 0.52049E-01
fake_059A 0.52049E-01 369.13 0.52049E-01
fake_059B 0.52049E-01 369.13 0.52049E-01
fake_060A 0.52049E-01 369.13 0.52049E-01
fake_060B 0.52049E-01 369.13 0.52049E-01
fake_061A 0.52049E-01 369.13 0.52049E-01
fake_061B 0.52049E-01 369.13 0.52049E-01
fake_062A 0.52049E-01 369.13 0.52049E-01
fake_062B 0.52049E-01 369.13 0.52049E-01
fake_063A 0.52049E-01 369.13 0.52049E-01
fake_063B 0.52049E-01 369.13 0.52049E-01
fake_064A 0.52049E-01 369.13 0.52049E-01
fake_064B 0.52049E-01 369.13 0.52049E-01
fake_065A 0.52049E-01 369.13 0.52049E-01
fake_065B 0.52049E-01 369.13 0.52049E-01
fake_066A 0.52049E-01 369.13 0.52049E-01
fake_066B 0.52049E-01 369.13 0.52049E-01
fake_067A 0.52049E-01 369.13 0.52049E-01
fake_067B 0.52049E-01 369.13 0.52049E-01
fake_068A 0.52049E-01 369.13 0.52049E-01
fake_068B 0.52049E-01 369.13 0.52049E-01
fake_069A 0.52049E-01 369.13 0.52049E-01
fake_069B 0.52049E-01 369.13 0.52049E-01
fake_070A 0.52049E-01 369.13 0.52049E-01
fake_070B 0.52049E-01 369.13 0.52049E-01
fake_071A 0.52049E-01 369.13 0.52049E-01
fake_071B 0.52049E-01 369.13 0.52049E-01
fake_072A 0.52049E-01 369.13 0.52049E-01
fake_072B 0.52049E-01 369.13 0.52049E-01
fake_073A 0.52049E-01 369.13 0.52049E-01
fake_073B 0.52049E-01 369.13 0.52049E-01
fake_074A 0.52049E-01 369.13 0.52049E-01
fake_074B 0.52049E-01 369.13 0.52049E-01
fake_075A 0.52049E-01 369.13 0.52049E-01
fake_075B 0.52049E-01 369.13 0.52049E-01
fake_076A 0.52049E-01 369.13 0.52049E-01
fake_076B 0.52049E-01 369.13 0.52049E-01
fake_077A 0.52049E-01 369.13 0.52049E-01
fake_077B 0.52049E-01 369.13 0.52049E-01
fake_078A 0.52049E-01 369.13 0.52049E-01
fake_078B 0.52049E-01 369.13 0.52049E-01
fake_079A 0.52049E-01 369.13 0.52049E-01
fake_079B 0.52049E-01 369.13 0.52049E-01
fake_080A 0.52049E-01 369.13 0.52049E-01
fake_080B 0.52049E-01 369.13 0.52049E-01
fake_081A 0.52049E-01 369.13 0.52049E-01
fake_081B 0.52049E-01 369.13 0.52049E-01
fake_082A 0.52049E-01 369.13 0.52049E-01
fake_082B 0.52049E-01 369.13 0.52049E-01
fake_083A 0.52049E-01 369.13 0.52049E-01
fake_083B 0.52049E-01 369.13 0.52049E-01
fake_084A 0.52049E-01 369.13 0.52049E-01
fake_084B 0.52049E-01 369.13 0.52049E-01
fake_085A 0.52049E-01 369.13 0.52049E-01
fake_085B 0.52049E-01 369.13 0.52049E-01
fake_086A 0.52049E-01 369.13 0.52049E-01
fake_086B 0.52049E-01 369.13 0.52049E-01
fake_087A 0.52049E-01 369.13 0.52049E-01
fake_087B 0.52049E-01 369.13 0.52049E-01
fake_088A 0.52049E-01 369.13 0.52049E-01
fake_088B 0.52049E-01 369.13 0.52049E-01
fake_089A 0.52049E-01 369.13 0.52049E-01
fake_089B 0.52049E-01 369.13 0.52049E-01
fake_090A 0.52049E-01 369.13 0.52049E-01
fake_090B 0.52049E-01 369.13 0.52049E-01
fake_091A 0.52049E-01 369.13 0.52049E-01
fake_091B 0.52049E-01 369.13 0.52049E-01
fake_092A 0.52049E-01 369.13 0.52049E-01
fake_092B 0.52049E-01 369.13 0.52049E-01
fake_093A 0.52049E-01 369.13 0.52049E-01
fake_093B 0.52049E-01 369.13 0.52049E-01
fake_094A 0.52049E-01 369.13 0.52049E-01
fake_094B 0.52049E-01 369.13 0.52049E-01
fake_095A 0.52049E-01 369.13 0.52049E-01
fake_095B 0.52049E-01 369.13 0.52049E-01
fake_096A 0.52049E-01 369.13 0.52049E-01
fake_096B 0.52049E-01 369.13 0.52049E-01
fake_097A 0.52049E-01 369.13 0.52049E-01
fake_097B 0.52049E-01 369.13 0.52049E-01
fake_098A 0.52049E-01 369.13 0.52049E-01
fake_098B 0.52049E-01 369.13 0.52049E-01
fake_099A 0.52049E-01 369.13 0.52049E-01
fake_099B 0.52049E-01 369.13 0.52049E-01
fake_100A 0.52049E-01 369.13 0.52049E-01
fake_100B 0.52049E-01 369.13 0.52049E-01
fake_101A 0.52049E-01 369.13 0.52049E-01
fake_101B 0.52049E-01 369.13 0.52049E-01
fake_102A 0.52049E-01 369.13 0.52049E-01
fake_102B 0.52049E-01 369.13 0.52049E-01
fake_103A 0.52049E-01 369.13 0.52049E-01
fake_103B 0.52049E-01 369.13 0.52049E-01
fake_104A 0.52049E-01 369.13 0.52049E-01
fake_104B 0.52049E-01 369.13 0.52049E-01
fake_105A 0.52049E-01 369.13 0.52049E-01
fake_105B 0.52049E-01 369.13 0.52049E-01
fake_106A 0.52049E-01 369.13 0.52049E-01
fake_106B 0.52049E-01 369.13 0.52049E-01
fake_107A 0.52049E-01 369.13 0.52049E-01
fake_107B 0.52049E-01 369.13 0.52049E-01
fake_108A 0.52049E-01 369.13 0.52049E-01
fake_108B 0.52049E-01 369.13 0.52049E-01
fake_109A 0.52049E-01 369.13 0.52049E-01
fake_109B 0.52049E-01 369.13 0.52049E-01
fake_110A 0.52049E-01 369.13 0.52049E-01
fake_110B 0.52049E-01 369.13 0.52049E-01
fake_111A 0.52049E-01 369.13 0.52049E-01
fake_111B 0.52049E-01 369.13 0.52049E-01
fake_112A 0.52049E-01 369.13 0.52049E-01
fake_112B 0.52049E-01 369.13 0.52049E-01
fake_113A 0.52049E-01 369.13 0.52049E-01
fake_113B 0.52049E-01 369.13 0.52049E-01
fake_114A 0.52049E-01 369.13 0.52049E-01
fake_114B 0.52049E-01 369.13 0.52049E-01
fake_115A 0.52049E-01 369.13 0.52049E-01
fake_115B 0.52049E-01 369.13 0.52049E-01
fake_116A 0.52049E-01 369.13 0.52049E-01
fake_116B 0.52049E-01 369.13 0.52049E-01
fake_117A 0.52049E-01 369.13 0.52049E-01
fake_117B 0.52049E-01 369.13 0.52049E-01
fake_118A 0.52049E-01 369.13 0.52049E-01
fake_118B 0.52049E-01 369.13 0.52049E-01
fake_119A 0.52049E-01 369.13 0.52049E-01
fake_119B 0.52049E-01 369.13 0.52049E-01
fake_120A 0.52049E-01 369.13 0.52049E-01
fake_120B 0.52049E-01 369.13 0.52049E-01
fake_121A 0.52049E-01 369.13 0.52049E-01
fake_121B 0.52049E-01 369.13 0.52049E-01
fake_122A 0.52049E-01 369.13 0.52049E-01
fake_122B 0.52049E-01 369.13 0.52049E-01
fake_123A 0.52049E-01 369.13 0.52049E-01
fake_123B 0.52049E-01 369.13 0.52049E-01
fake_124A 0.52049E-01 369.13 0.52049E-01
fake_124B 0.52049E-01 369.13 0.52049E-01
fake_125A 0.52049E-01 369.13 0.52049E-01
fake_125B 0.52049E-01 369.13 0.52049E-01
fake_126A 0.52049E-01 369.13 0.52049E-01
fake_126B 0.52049E-01 369.13 0.52049E-01
Output files
file_root = groundsim000_filtered_100_telescope_all_time_all
file_binmap = out/00000000/100/groundsim000_filtered_100_telescope_all_time_all_bmap.fits
binary_output = F
concatenate_binary = F
TOD memory min = 760.28 MB max = 760.28 MB total = 760.28 MB
Baseline memory min = 0.00 B max = 0.00 B total = 0.00 B
Pointing memory min = 1.85 GB max = 1.85 GB total = 1.85 GB
0 : init_pointing completed in 0.104 s
Basis function memory min = 0.00 B max = 0.00 B total = 0.00 B
Clock = 0.397 s
0 : reduce_pixels completed in 0.497 s
Total submaps = 3072 submap size = 1024
Local submaps: min = 41 max = 41 mean = 41.00
Submap table memory min = 12.00 kB max = 12.00 kB total = 12.00 kB
local maps memory min = 4.00 MB max = 4.00 MB total = 4.00 MB
Allreduce memory min = 0.01 kB max = 0.01 kB total = 0.01 kB
Map memory min = 372.00 MB max = 372.00 MB total = 372.00 MB
Building pixel matrices...
0 : pixel_matrix completed in 0.734 s
Binning TOD...
0 : bin_tod completed in 0.575 s
Clock = 2.250 s
Finalization begins
Clock = 2.251 s
Inverting pixel matrices...
26267 pixels solved
3119461 pixels unsolved
0 pixels had decoupled intensity
Constructing output map...
Constructing output map...
Destriped map:
Map 1 2 3
Std 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Mean 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Min 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Max 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Writing binned map...
Binned map written in out/00000000/100/groundsim000_filtered_100_telescope_all_time_all_bmap.fits
Clock = 2.458 s
MEMORY (MB):
Detector pointing min = 1.85 GB max = 1.85 GB total = 1.85 GB
TOD buffer min = 760.28 MB max = 760.28 MB total = 760.28 MB
Maps min = 372.00 MB max = 372.00 MB total = 372.00 MB
Baselines min = 0.64 kB max = 0.64 kB total = 0.64 kB
Basis functions min = 0.00 B max = 0.00 B total = 0.00 B
Noise filter min = 20.27 MB max = 20.27 MB total = 20.27 MB
Preconditioner min = 2.46 GB max = 2.46 GB total = 2.46 GB
Submap table min = 12.00 kB max = 12.00 kB total = 12.00 kB
Temporary maps min = 4.00 MB max = 4.00 MB total = 4.00 MB
All2All buffers min = 0.01 kB max = 0.01 kB total = 0.01 kB
CG work space min = 100.92 MB max = 100.92 MB total = 100.92 MB
NCM min = 0.00 B max = 0.00 B total = 0.00 B
Total min = 5.54 GB max = 5.54 GB total = 5.54 GB
WALL-CLOCK TIME (s):
Initialization mean = 0.4 min = 0.4 max = 0.4
I/O mean = 0.2 min = 0.2 max = 0.2
- Pointing Period boundaries mean = 0.2 min = 0.2 max = 0.2
Building pixel matrices mean = 0.7 min = 0.7 max = 0.7
Inverting pixel matrices mean = 0.1 min = 0.1 max = 0.1
Binning TOD mean = 0.6 min = 0.6 max = 0.6
Finalization and output mean = 0.2 min = 0.2 max = 0.2
Other mean = 0.2 min = 0.2 max = 0.2
Total 2.5 s ( 0.01 CPU hours)
TOAST INFO: Unstage signal 1 / 1: 0.02 seconds (1 calls)
TOAST INFO: Unstage pixels 1 / 1: 0.51 seconds (1 calls)
TOAST INFO: Unstage pixel weights 1 / 1: 0.71 seconds (1 calls)
TOAST INFO: Unstage all data: 1.24 seconds (1 calls)
TOAST INFO: Mapping groundsim000_filtered_100_telescope_all_time_all: 8.19 seconds (1 calls)
TOAST INFO: Madam total: 8.19 seconds (1 calls)
TOAST INFO: Gather and dump timing info: 0.00 seconds (1 calls)
TOAST INFO: toast_ground_sim.py: 478.07 seconds (1 calls)
Out[22]:
0
Here we examine the intensity part of the simulated maps.
In [23]:
import healpy as hp
try:
binned = hp.read_map("out/00000000/100/groundsim000_100_telescope_all_time_all_bmap.fits")
destriped = hp.read_map("out/00000000/100/groundsim000_100_telescope_all_time_all_map.fits")
filtered = hp.read_map("out/00000000/100/groundsim000_filtered_100_telescope_all_time_all_bmap.fits")
rot = [40, -40]
reso = 10
cmap = "coolwarm"
amp = 0.02
unit = "K"
plt.figure(figsize=[12, 8])
hp.gnomview(binned, rot=rot, reso=reso, cmap=cmap, min=-amp, max=amp, sub=[1, 3, 1], title="binned", unit=unit)
hp.gnomview(destriped, rot=rot, reso=reso, cmap=cmap, min=-amp, max=amp, sub=[1, 3, 2], title="destriped", unit=unit)
hp.gnomview(filtered, rot=rot, reso=reso, cmap=cmap, min=-amp, max=amp, sub=[1, 3, 3], title="filtered", unit=unit)
except:
print("Maps are not available. `Either toast_ground_sim.slrm` failed or you did not submit it yet.")
NSIDE = 512 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING NSIDE = 512 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING NSIDE = 512 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING
In [ ]: