Ground observing schedules¶
In this notebook we learn about creating ground observing schedules.
In [1]:
# Are you using a special reservation for a workshop?
# If so, set it here:
nersc_reservation = "toast3"
# Load common tools for all lessons
import sys
sys.path.insert(0, "..")
from lesson_tools import (
check_nersc,
fake_focalplane
)
nersc_host, nersc_repo, nersc_resv = check_nersc(reservation=nersc_reservation)
# Capture C++ output in the jupyter cells
%reload_ext wurlitzer
Running on NERSC machine 'cori' with access to repos: mp107, desi, m1759, cosmo Using default repo mp107 Reservation 'toast3' valid from 2019-10-16T09:00:00 to 2019-10-16T17:00:00 Current time is 2019-10-16T05:33:17.549904 Reservation 'toast3' not currently valid
TOAST pipelines include a tool called toast_ground_schedule.py, also known as the opportunistic scheduler. It builds observing schedules heuristically by building a list of available targets and scheduling and always choosing the highest priority target. toast_ground_schedule.py can be used to create site-specific observing schedules subject to a number of constraints. At the minimum, the tool needs the location of the observatory, observing window and at least one target. Here is a minimal example:
In [2]:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-01 12:00:00" \
--patch-coord C \
--patch small_patch,1,40,-40,44,-44 \
--out schedule.txt
TOAST INFO: Adding patch "small_patch" TOAST INFO: Rectangular format TOAST INFO: Global timer: toast_ground_schedule: 0.15 seconds (1 calls)
Let's look at the contents of the schedule file.
In [3]:
! cat schedule.txt
#Site Telescope Latitude [deg] Longitude [deg] Elevation [m] Atacama LAT -22.958 -67.786 5200.0 #Start time UTC Stop time UTC Start MJD Stop MJD Patch name Az min Az max El R/S Sun el1 Sun az1 Sun el2 Sun az2 Moon el1 Moon az1 Moon el2 Moon az2 Phase Pass Sub 2020-01-01 00:50:00 2020-01-01 01:04:45 58849.034722 58849.044965 small_patch 189.14 196.61 68.69 S -18.68 233.69 -21.38 231.50 34.93 274.35 31.60 272.86 0.30 0 0 2020-01-01 01:04:55 2020-01-01 01:19:40 58849.045081 58849.055324 small_patch 189.19 205.13 68.69 S -21.41 231.47 -24.02 229.13 31.56 272.84 28.22 271.42 0.30 0 1 2020-01-01 01:19:50 2020-01-01 01:34:35 58849.055440 58849.065683 small_patch 196.64 213.71 68.69 S -24.05 229.10 -26.57 226.59 28.18 271.40 24.85 270.03 0.30 0 2 2020-01-01 01:34:45 2020-01-01 01:49:00 58849.065799 58849.075694 small_patch 205.16 213.71 68.69 S -26.59 226.56 -28.92 223.95 24.81 270.02 21.59 268.72 0.31 0 3 2020-01-01 01:50:40 2020-01-01 02:02:00 58849.076852 58849.084722 small_patch 213.17 220.01 63.42 S -29.19 223.64 -30.95 221.42 21.22 268.57 18.66 267.57 0.31 1 0 2020-01-01 02:02:10 2020-01-01 02:13:30 58849.084838 58849.092708 small_patch 213.16 224.69 63.42 S -30.98 221.39 -32.66 219.04 18.62 267.55 16.07 266.55 0.31 1 1 2020-01-01 02:13:40 2020-01-01 02:24:40 58849.092824 58849.100463 small_patch 217.42 224.69 63.42 S -32.69 219.01 -34.24 216.62 16.03 266.54 13.56 265.58 0.31 1 2 2020-01-01 02:26:20 2020-01-01 02:40:20 58849.101620 58849.111343 small_patch 221.48 230.10 58.39 S -34.47 216.24 -36.30 212.99 13.19 265.43 10.06 264.21 0.31 2 0 2020-01-01 02:40:30 2020-01-01 02:54:20 58849.111458 58849.121065 small_patch 221.48 230.10 58.39 S -36.32 212.95 -37.97 209.54 10.02 264.19 6.94 262.98 0.31 2 1 2020-01-01 02:56:00 2020-01-01 03:08:00 58849.122222 58849.130556 small_patch 225.94 233.12 53.65 S -38.16 209.12 -39.44 205.98 6.57 262.83 3.93 261.76 0.31 3 0 2020-01-01 03:08:10 2020-01-01 03:20:00 58849.130671 58849.138889 small_patch 225.93 233.12 53.65 S -39.45 205.94 -40.58 202.70 3.89 261.75 1.35 260.68 0.31 3 1 2020-01-01 03:21:40 2020-01-01 03:32:40 58849.140046 58849.147685 small_patch 228.49 234.81 49.30 S -40.72 202.23 -41.62 199.09 1.00 260.53 -1.15 259.52 0.31 4 0 2020-01-01 03:32:50 2020-01-01 03:43:40 58849.147801 58849.155324 small_patch 228.48 234.81 49.30 S -41.63 199.04 -42.38 195.84 -1.18 259.51 -3.38 258.50 0.31 4 1 2020-01-01 03:45:20 2020-01-01 03:55:20 58849.156481 58849.163426 small_patch 230.03 235.78 45.15 S -42.48 195.34 -43.03 192.30 -3.95 258.34 -6.59 257.38 0.31 5 0 2020-01-01 03:55:30 2020-01-01 04:05:20 58849.163542 58849.170370 small_patch 230.03 235.78 45.15 S -43.04 192.25 -43.46 189.20 -6.63 257.36 -8.82 256.40 0.31 5 1 2020-01-01 04:07:00 2020-01-01 04:16:00 58849.171528 58849.177778 small_patch 230.90 236.25 41.30 S -43.52 188.68 -43.78 185.84 -9.19 256.23 -11.15 255.32 0.31 6 0 2020-01-01 04:16:10 2020-01-01 04:25:00 58849.177894 58849.184028 small_patch 230.90 236.25 41.30 S -43.79 185.79 -43.94 182.98 -11.18 255.31 -13.09 254.39 0.32 6 1 2020-01-01 04:26:40 2020-01-01 04:35:10 58849.185185 58849.191088 small_patch 231.33 236.40 37.76 S -43.96 182.45 -44.00 179.73 -13.45 254.22 -15.28 253.31 0.32 7 0 2020-01-01 04:35:20 2020-01-01 04:43:40 58849.191204 58849.196991 small_patch 231.33 236.39 37.76 S -44.00 179.68 -43.94 177.01 -15.32 253.29 -17.10 252.37 0.32 7 1 2020-01-01 04:45:20 2020-01-01 04:53:50 58849.198148 58849.204051 small_patch 231.46 236.31 34.32 S -43.92 176.48 -43.76 173.78 -17.46 252.18 -19.27 251.21 0.32 8 0 2020-01-01 04:54:00 2020-01-01 05:02:20 58849.204167 58849.209954 small_patch 231.46 236.31 34.32 S -43.75 173.73 -43.50 171.11 -19.30 251.20 -21.06 250.21 0.32 8 1 2020-01-01 05:04:00 2020-01-01 05:13:00 58849.211111 58849.217361 small_patch 231.34 236.02 30.74 S -43.44 170.58 -43.05 167.79 -21.42 250.01 -23.30 248.91 0.32 9 0 2020-01-01 05:13:10 2020-01-01 05:22:00 58849.217477 58849.223611 small_patch 231.34 236.01 30.74 S -43.04 167.74 -42.57 165.05 -23.34 248.89 -25.17 247.76 0.32 9 1
The rectangular patch definition takes the form --patch <name>,<priority>,<RA left>,<DEC top>,<RA right>,<DEC bottom>. No spaces are allowed in the definition. Other patch definition formats will be discussed below.
The start and stop times are given in UTC.
The resulting schedule is a plain ASCII file. The header defines the telescope and each line after that defines a constant elevation scan (CES) with a fixed azimuth range. When a full pass of the target takes longer than allowed observation time, --ces-max-time, the CES is broken up into sub passes that use the same observing elevation but adjust the azimuth range. The above schedule includes 10 passes of the target "small_patch" that fit in the given 12-hour observing window. Some passes are split into as many as 4 sub passes, each no longer than 20 minutes (default).
Let's add another patch, this time using the circular patch definition format, set the observing elevation limits and enable Sun avoidance. We'll also increase ces-max-time so we get fewer entries in the schedule. The circular patch format is
--patch <name>,<priority>,<RA>,<DEC>,<radius>
In [4]:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-04 00:00:00" \
--patch-coord C \
--patch small_patch,1,80,-13,10 \
--patch large_patch,1,80,-33,20 \
--el-min 30 \
--el-max 60 \
--ces-max-time 86400 \
--sun-avoidance-angle 20 \
--out schedule.txt \
--debug
! cat schedule.txt
TOAST INFO: Adding patch "small_patch" TOAST INFO: Center-and-width format TOAST INFO: Adding patch "large_patch" TOAST INFO: Center-and-width format TOAST INFO: small_patch corners: lon = [80.0, 82.52889698392441, 84.40387023305097, 85.13152053896695, 84.49355559956005, 82.61888616230056, 80.0, 77.38111383769943, 75.50644440043996, 74.86847946103305, 75.596129766949, 77.47110301607557, 80.0] lat= [-8.000000000000002, -8.669872981077807, -10.5, -13.0, -15.5, -17.33012701892219, -18.0, -17.330127018922195, -15.500000000000002, -13.000000000000002, -10.500000000000004, -8.66987298107781, -8.000000000000002] TOAST INFO: large_patch corners: lon = [80.0, 85.48776585034872, 89.80834435462137, 91.92363292835948, 90.99002012118036, 86.69254923971052, 80.0, 73.3074507602895, 69.00997987881962, 68.07636707164052, 70.19165564537862, 74.51223414965126, 80.0] lat= [-23.0, -24.339745962155618, -28.0, -33.0, -38.0, -41.66025403784439, -43.0, -41.66025403784439, -38.00000000000001, -33.0, -28.000000000000007, -24.339745962155618, -23.0] TOAST INFO: small_patch corners: lon = [80.0, 82.52889698392441, 84.40387023305097, 85.13152053896695, 84.49355559956005, 82.61888616230056, 80.0, 77.38111383769943, 75.50644440043996, 74.86847946103305, 75.596129766949, 77.47110301607557, 80.0] lat= [-8.000000000000002, -8.669872981077807, -10.5, -13.0, -15.5, -17.33012701892219, -18.0, -17.330127018922195, -15.500000000000002, -13.000000000000002, -10.500000000000004, -8.66987298107781, -8.000000000000002] TOAST INFO: large_patch corners: lon = [80.0, 85.48776585034872, 89.80834435462137, 91.92363292835948, 90.99002012118036, 86.69254923971052, 80.0, 73.3074507602895, 69.00997987881962, 68.07636707164052, 70.19165564537862, 74.51223414965126, 80.0] lat= [-23.0, -24.339745962155618, -28.0, -33.0, -38.0, -41.66025403784439, -43.0, -41.66025403784439, -38.00000000000001, -33.0, -28.000000000000007, -24.339745962155618, -23.0] TOAST INFO: small_patch corners: lon = [80.0, 82.52889698392441, 84.40387023305097, 85.13152053896695, 84.49355559956005, 82.61888616230056, 80.0, 77.38111383769943, 75.50644440043996, 74.86847946103305, 75.596129766949, 77.47110301607557, 80.0] lat= [-8.000000000000002, -8.669872981077807, -10.5, -13.0, -15.5, -17.33012701892219, -18.0, -17.330127018922195, -15.500000000000002, -13.000000000000002, -10.500000000000004, -8.66987298107781, -8.000000000000002] TOAST INFO: large_patch corners: lon = [80.0, 85.48776585034872, 89.80834435462137, 91.92363292835948, 90.99002012118036, 86.69254923971052, 80.0, 73.3074507602895, 69.00997987881962, 68.07636707164052, 70.19165564537862, 74.51223414965126, 80.0] lat= [-23.0, -24.339745962155618, -28.0, -33.0, -38.0, -41.66025403784439, -43.0, -41.66025403784439, -38.00000000000001, -33.0, -28.000000000000007, -24.339745962155618, -23.0] TOAST INFO: Global timer: toast_ground_schedule: 10.86 seconds (1 calls) #Site Telescope Latitude [deg] Longitude [deg] Elevation [m] Atacama LAT -22.958 -67.786 5200.0 #Start time UTC Stop time UTC Start MJD Stop MJD Patch name Az min Az max El R/S Sun el1 Sun az1 Sun el2 Sun az2 Moon el1 Moon az1 Moon el2 Moon az2 Phase Pass Sub 2020-01-01 00:00:00 2020-01-01 00:44:00 58849.000000 58849.030556 small_patch 75.46 91.56 48.49 R -9.03 240.13 -17.56 234.54 46.19 280.16 36.29 274.97 0.30 0 0 2020-01-01 00:45:40 2020-01-01 01:30:40 58849.031713 58849.062963 small_patch 65.75 86.88 58.99 R -17.87 234.30 -25.91 227.27 35.91 274.80 25.73 270.39 0.30 1 0 2020-01-01 04:32:20 2020-01-01 06:11:20 58849.189120 58849.257870 large_patch 222.58 263.25 59.27 S -44.00 180.64 -38.32 151.21 -14.67 253.61 -35.11 240.49 0.32 0 0 2020-01-01 06:13:00 2020-01-01 06:56:00 58849.259028 58849.288889 small_patch 265.58 279.51 40.77 S -38.13 150.78 -32.55 140.76 -35.44 240.21 -43.43 231.94 0.32 2 0 2020-01-01 06:57:40 2020-01-01 07:40:40 58849.290046 58849.319907 small_patch 261.98 274.23 30.49 S -32.31 140.41 -25.46 132.25 -43.72 231.58 -50.69 220.54 0.33 3 0 2020-01-01 21:52:20 2020-01-01 23:28:20 58849.911343 58849.978009 large_patch 103.93 127.54 30.99 R 17.96 252.02 -1.87 243.69 72.62 353.13 59.60 301.38 0.38 1 0 2020-01-01 23:30:00 2020-01-02 01:06:00 58849.979167 58850.045833 large_patch 98.61 131.01 51.46 R -2.15 243.52 -21.57 231.45 59.27 300.89 38.84 282.61 0.39 2 0 2020-01-02 04:27:40 2020-01-02 06:06:40 58850.185880 58850.254630 large_patch 222.43 263.26 59.40 S -44.05 182.28 -38.95 152.49 -6.45 262.46 -28.13 251.78 0.41 3 0 2020-01-02 06:08:20 2020-01-02 06:51:20 58850.255787 58850.285648 small_patch 265.64 279.61 40.94 S -38.77 152.06 -33.35 141.80 -28.49 251.56 -37.44 245.39 0.41 4 0 2020-01-02 06:53:00 2020-01-02 07:36:00 58850.286806 58850.316667 small_patch 262.03 274.31 30.66 S -33.12 141.44 -26.39 133.07 -37.78 245.12 -46.20 237.08 0.42 5 0 2020-01-02 21:47:40 2020-01-02 23:23:40 58850.908102 58850.974769 large_patch 103.97 127.54 30.84 R 19.06 252.49 -1.04 244.29 66.34 23.38 63.67 325.60 0.48 4 0 2020-01-02 23:25:20 2020-01-03 00:08:20 58850.975926 58851.005787 small_patch 79.57 93.88 42.31 R -1.33 244.12 -10.59 239.41 63.45 324.82 56.60 308.50 0.48 6 0 2020-01-03 00:10:00 2020-01-03 00:54:00 58851.006944 58851.037500 small_patch 72.23 89.87 52.61 R -10.92 239.21 -19.35 233.39 56.30 308.00 47.78 297.05 0.48 7 0 2020-01-03 04:25:40 2020-01-03 06:03:40 58851.184491 58851.252546 large_patch 222.87 263.20 59.04 S -44.11 183.08 -39.39 153.36 1.41 270.81 -20.72 261.47 0.50 5 0 2020-01-03 06:05:20 2020-01-03 07:41:20 58851.253704 58851.320370 large_patch 232.47 258.22 38.77 S -39.22 152.92 -25.63 132.15 -21.09 261.30 -41.84 249.12 0.51 6 0 2020-01-03 21:43:00 2020-01-03 23:19:00 58851.904861 58851.971528 large_patch 104.01 127.55 30.68 R 20.16 252.96 -0.17 244.88 56.74 39.36 62.85 352.84 0.57 7 0
Note that we added the --debug option to the command line. This produces a helpful diagnostic plot, patches.png, that shows the locations of your patches, the Sun, the Moon and their avoidance areas. The plot is shown below. The motion of the Moon is already apparent in this 3-day schedule. The Sun (on the right) is effectively stationary. --debug can be expensive, especially if you have lots of patches or request a long observing schedule.
In [5]:
from IPython.display import Image
Image("patches.png")
Out[5]:
We deliberately chose the locations of the patches so that they compete over the observing time. This allows us to point out some advanced features of the scheduler. If you examine the very end of the observing schedule, you can note that both small_patch and large_patch were observed 7 times. Given that large_patch is twice as wide and only takes twice as long to observe, equal number of observations actually implies that large_patch will end up with half as many hits per sky pixel.
The scheduler offers two ways to remedy this issue. First, one can simply increase the priority of the large patch to dedicate more observing time to it. All things being equal, the number of visits to a given patch is inversely proportional to the priority in the patch definition:
In [6]:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-04 00:00:00" \
--patch-coord C \
--patch small_patch,1,80,-13,10 \
--patch large_patch,0.5,80,-33,20 \
--el-min 30 \
--el-max 60 \
--ces-max-time 86400 \
--sun-avoidance-angle 20 \
--out schedule.txt
! cat schedule.txt
TOAST INFO: Adding patch "small_patch" TOAST INFO: Center-and-width format TOAST INFO: Adding patch "large_patch" TOAST INFO: Center-and-width format TOAST INFO: Global timer: toast_ground_schedule: 0.64 seconds (1 calls) #Site Telescope Latitude [deg] Longitude [deg] Elevation [m] Atacama LAT -22.958 -67.786 5200.0 #Start time UTC Stop time UTC Start MJD Stop MJD Patch name Az min Az max El R/S Sun el1 Sun az1 Sun el2 Sun az2 Moon el1 Moon az1 Moon el2 Moon az2 Phase Pass Sub 2020-01-01 00:00:00 2020-01-01 01:37:00 58849.000000 58849.067361 large_patch 97.31 134.88 56.91 R -9.03 240.13 -26.97 226.16 46.19 280.16 24.30 269.81 0.30 0 0 2020-01-01 04:28:40 2020-01-01 06:07:40 58849.186574 58849.255324 large_patch 221.76 263.40 59.94 S -43.98 181.81 -38.72 152.16 -13.89 254.00 -34.40 241.10 0.32 1 0 2020-01-01 06:09:20 2020-01-01 06:52:20 58849.256481 58849.286343 small_patch 265.89 280.01 41.61 S -38.53 151.73 -33.08 141.54 -34.72 240.83 -42.78 232.74 0.32 0 0 2020-01-01 06:54:00 2020-01-01 08:30:00 58849.287500 58849.354167 large_patch 232.42 255.85 30.25 S -32.84 141.19 -16.57 124.70 -43.08 232.38 -56.63 203.44 0.33 2 0 2020-01-01 21:51:40 2020-01-01 23:27:40 58849.910880 58849.977546 large_patch 103.97 127.54 30.85 R 18.11 252.07 -1.76 243.76 72.64 353.67 59.73 301.57 0.38 3 0 2020-01-01 23:29:20 2020-01-02 00:12:20 58849.978704 58850.008565 small_patch 79.56 93.88 42.33 R -2.03 243.59 -11.42 238.80 59.40 301.09 50.55 291.03 0.39 1 0 2020-01-02 00:14:00 2020-01-02 00:58:00 58850.009722 58850.040278 small_patch 72.21 89.86 52.63 R -11.75 238.59 -20.12 232.66 50.20 290.72 40.62 283.69 0.39 2 0 2020-01-02 04:29:40 2020-01-02 06:07:40 58850.187269 58850.255324 large_patch 222.88 263.20 59.03 S -44.06 181.64 -38.84 152.23 -6.92 262.27 -28.35 251.65 0.41 4 0 2020-01-02 06:09:20 2020-01-02 06:52:20 58850.256481 58850.286343 small_patch 265.56 279.48 40.71 S -38.66 151.80 -33.21 141.58 -28.70 251.43 -37.64 245.22 0.41 3 0 2020-01-02 06:54:00 2020-01-02 07:37:00 58850.287500 58850.317361 small_patch 261.95 274.20 30.43 S -32.97 141.23 -26.22 132.90 -37.98 244.95 -46.39 236.86 0.42 4 0 2020-01-02 21:48:40 2020-01-02 23:24:40 58850.908796 58850.975463 large_patch 103.92 127.53 31.04 R 18.84 252.42 -1.21 244.19 66.43 22.81 63.54 325.13 0.48 5 0 2020-01-02 23:26:20 2020-01-03 01:02:20 58850.976620 58851.043287 large_patch 98.59 131.04 51.51 R -1.49 244.02 -20.87 232.15 63.32 324.36 46.07 295.39 0.48 6 0 2020-01-03 04:24:00 2020-01-03 06:03:00 58851.183333 58851.252083 large_patch 222.48 263.26 59.35 S -44.09 183.61 -39.46 153.54 1.77 270.96 -20.57 261.54 0.50 7 0 2020-01-03 06:04:40 2020-01-03 07:40:40 58851.253241 58851.319907 large_patch 232.46 258.25 38.90 S -39.28 153.09 -25.74 132.27 -20.94 261.36 -41.70 249.22 0.51 8 0 2020-01-03 21:42:20 2020-01-03 23:18:20 58851.904398 58851.971065 large_patch 104.05 127.56 30.54 R 20.30 253.01 -0.04 244.95 56.64 39.61 62.87 353.20 0.57 9 0
Now the large patch is observed 9 times and the small patch is observed 4 times. Typically though, we don't use the priority field to normalize the depths. Instead, the user can balance the integration depths with two command line arguments: --equalize-area and --equalize time.
With --equalize-area the scheduler will automatically modulate the user-given priorities with the area of each patch.
With --equalize-time the scheduler will balance the actual time spent in each patch rather than the number of visits. There is a difference, because the observing time per pass can vary greatly depending on the patch shape and orientation
In [7]:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-04 00:00:00" \
--patch-coord C \
--patch small_patch,1,80,-13,10 \
--patch large_patch,1,80,-33,20 \
--el-min 30 \
--el-max 60 \
--ces-max-time 86400 \
--sun-avoidance-angle 20 \
--equalize-area \
--equalize-time \
--out schedule.txt
! cat schedule.txt
TOAST INFO: Adding patch "small_patch" TOAST INFO: Center-and-width format TOAST INFO: Adding patch "large_patch" TOAST INFO: Center-and-width format TOAST INFO: Global timer: toast_ground_schedule: 6.02 seconds (1 calls) #Site Telescope Latitude [deg] Longitude [deg] Elevation [m] Atacama LAT -22.958 -67.786 5200.0 #Start time UTC Stop time UTC Start MJD Stop MJD Patch name Az min Az max El R/S Sun el1 Sun az1 Sun el2 Sun az2 Moon el1 Moon az1 Moon el2 Moon az2 Phase Pass Sub 2020-01-01 00:00:00 2020-01-01 01:37:00 58849.000000 58849.067361 large_patch 97.31 134.88 56.91 R -9.03 240.13 -26.97 226.16 46.19 280.16 24.30 269.81 0.30 0 0 2020-01-01 04:28:40 2020-01-01 06:07:40 58849.186574 58849.255324 large_patch 221.76 263.40 59.94 S -43.98 181.81 -38.72 152.16 -13.89 254.00 -34.40 241.10 0.32 1 0 2020-01-01 06:09:20 2020-01-01 06:52:20 58849.256481 58849.286343 small_patch 265.89 280.01 41.61 S -38.53 151.73 -33.08 141.54 -34.72 240.83 -42.78 232.74 0.32 0 0 2020-01-01 06:54:00 2020-01-01 08:30:00 58849.287500 58849.354167 large_patch 232.42 255.85 30.25 S -32.84 141.19 -16.57 124.70 -43.08 232.38 -56.63 203.44 0.33 2 0 2020-01-01 21:51:40 2020-01-01 23:27:40 58849.910880 58849.977546 large_patch 103.97 127.54 30.85 R 18.11 252.07 -1.76 243.76 72.64 353.67 59.73 301.57 0.38 3 0 2020-01-01 23:29:20 2020-01-02 00:12:20 58849.978704 58850.008565 small_patch 79.56 93.88 42.33 R -2.03 243.59 -11.42 238.80 59.40 301.09 50.55 291.03 0.39 1 0 2020-01-02 00:14:00 2020-01-02 00:58:00 58850.009722 58850.040278 small_patch 72.21 89.86 52.63 R -11.75 238.59 -20.12 232.66 50.20 290.72 40.62 283.69 0.39 2 0 2020-01-02 04:29:40 2020-01-02 06:07:40 58850.187269 58850.255324 large_patch 222.88 263.20 59.03 S -44.06 181.64 -38.84 152.23 -6.92 262.27 -28.35 251.65 0.41 4 0 2020-01-02 06:09:20 2020-01-02 06:52:20 58850.256481 58850.286343 small_patch 265.56 279.48 40.71 S -38.66 151.80 -33.21 141.58 -28.70 251.43 -37.64 245.22 0.41 3 0 2020-01-02 06:54:00 2020-01-02 07:37:00 58850.287500 58850.317361 small_patch 261.95 274.20 30.43 S -32.97 141.23 -26.22 132.90 -37.98 244.95 -46.39 236.86 0.42 4 0 2020-01-02 21:48:40 2020-01-02 23:24:40 58850.908796 58850.975463 large_patch 103.92 127.53 31.04 R 18.84 252.42 -1.21 244.19 66.43 22.81 63.54 325.13 0.48 5 0 2020-01-02 23:26:20 2020-01-03 01:02:20 58850.976620 58851.043287 large_patch 98.59 131.04 51.51 R -1.49 244.02 -20.87 232.15 63.32 324.36 46.07 295.39 0.48 6 0 2020-01-03 04:24:00 2020-01-03 06:03:00 58851.183333 58851.252083 large_patch 222.48 263.26 59.35 S -44.09 183.61 -39.46 153.54 1.77 270.96 -20.57 261.54 0.50 7 0 2020-01-03 06:04:40 2020-01-03 07:40:40 58851.253241 58851.319907 large_patch 232.46 258.25 38.90 S -39.28 153.09 -25.74 132.27 -20.94 261.36 -41.70 249.22 0.51 8 0 2020-01-03 21:42:20 2020-01-03 23:18:20 58851.904398 58851.971065 large_patch 104.05 127.56 30.54 R 20.30 253.01 -0.04 244.95 56.64 39.61 62.87 353.20 0.57 9 0
As with the by-hand-modulated priorities, large_patch ends up with twice as many visits.
Binning the schedule¶
We take an observing schedule from toast_ground_sim.py and translate it into a depth map.
First, we need a focalplane. If one does not already exist, TOAST pipelines includes a tool for generating mock hexagonal focalplanes:
In [8]:
! toast_fake_focalplane.py --help
usage: toast_fake_focalplane.py [-h] [--minpix MINPIX] [--out OUT]
[--fwhm FWHM] [--fwhm_sigma FWHM_SIGMA]
[--fov FOV] [--psd_fknee PSD_FKNEE]
[--psd_NET PSD_NET] [--psd_alpha PSD_ALPHA]
[--psd_fmin PSD_FMIN]
[--bandcenter_ghz BANDCENTER_GHZ]
[--bandcenter_sigma BANDCENTER_SIGMA]
[--bandwidth_ghz BANDWIDTH_GHZ]
[--bandwidth_sigma BANDWIDTH_SIGMA]
[--random_seed RANDOM_SEED]
Simulate fake hexagonal focalplane.
optional arguments:
-h, --help show this help message and exit
--minpix MINPIX minimum number of pixels to use
--out OUT Root name of output pickle file
--fwhm FWHM beam FWHM in arcmin
--fwhm_sigma FWHM_SIGMA
Relative beam FWHM distribution width
--fov FOV Field of View in degrees
--psd_fknee PSD_FKNEE
Detector noise model f_knee in Hz
--psd_NET PSD_NET Detector noise model NET in K*sqrt(sec)
--psd_alpha PSD_ALPHA
Detector noise model slope
--psd_fmin PSD_FMIN Detector noise model f_min in Hz
--bandcenter_ghz BANDCENTER_GHZ
Band center frequency [GHz]
--bandcenter_sigma BANDCENTER_SIGMA
Relative band center distribution width
--bandwidth_ghz BANDWIDTH_GHZ
Bandwidth [GHz]
--bandwidth_sigma BANDWIDTH_SIGMA
Relative bandwidth distribution width
--random_seed RANDOM_SEED
Random number generator seed for randomized detector
parameters
Now we create a focalplane with 10-degree FOV and a mininimum of 20 pixels:
In [9]:
! toast_fake_focalplane.py \
--minpix 20 \
--out focalplane \
--fwhm 30 \
--fov 10 \
--psd_fknee 5e-2 \
--psd_NET 1e-3 \
--psd_alpha 1 \
--psd_fmin 1e-5
TOAST INFO: using 37 pixels (74 detectors)
The actual focalplane ends up having 37 pixels, instead of the minimum of 20. This is because regular packing of the hexagon is quantized. Notice that the final name of the focalplane is focalplane_37.pkl. We'll need the name to run the simulation script.
We will use the versatile ground simulation pipeline, toast_ground_sim.py, to bin the map. It will be covered in detail in lesson 7 so here we simply write out a parameter file:
In [10]:
%%writefile bin_schedule.par
--sample-rate
1
--scan-rate
0.3
--scan-accel
10.0
--nside
64
--focalplane
focalplane_37.pkl
--schedule
schedule.txt
--out
out
--simulate-noise
--freq
100
--no-destripe
--no-binmap
--hits
--wcov
Writing bin_schedule.par
Then run the pipeline. Because the pipeline uses libMadam, an MPI code, we must submit the job to a compute node.
In [11]:
import subprocess as sp
command = "toast_ground_sim.py @bin_schedule.par"
runstr = None
if nersc_host is not None:
runstr = "srun -N 1 -C haswell -n 32 -c 2 --cpu_bind=cores -t 00:05:00"
if nersc_resv is not None:
runstr = "{} --reservation {}".format(runstr, nersc_resv)
else:
# Just use mpirun
runstr = "mpirun -np 4"
runcom = "{} {}".format(runstr, command)
print(runcom, flush=True)
sp.check_call(runcom, stderr=sp.STDOUT, shell=True)
srun -N 1 -C haswell -n 32 -c 2 --cpu_bind=cores -t 00:05:00 toast_ground_sim.py @bin_schedule.par
Launched in background. Redirecting stdin to /dev/null
srun: job 25182216 queued and waiting for resources
srun: job 25182216 has been allocated resources
TOAST INFO: Running with 32 processes at 2019-10-16 05:34:10.165789
TOAST INFO: All parameters:
TOAST INFO: group_size = None
TOAST INFO: do_daymaps = False
TOAST INFO: do_seasonmaps = False
TOAST INFO: debug = False
TOAST INFO: scan_rate = 0.3
TOAST INFO: scan_accel = 10.0
TOAST INFO: sun_angle_min = 30.0
TOAST INFO: schedule = schedule.txt
TOAST INFO: weather = None
TOAST INFO: timezone = 0
TOAST INFO: sample_rate = 1.0
TOAST INFO: coord = C
TOAST INFO: split_schedule = None
TOAST INFO: sort_schedule = True
TOAST INFO: hwp_rpm = None
TOAST INFO: hwp_step_deg = None
TOAST INFO: hwp_step_time_s = None
TOAST INFO: nside = 64
TOAST INFO: single_precision_pointing = False
TOAST INFO: common_flag_mask = 1
TOAST INFO: flush = False
TOAST INFO: apply_polyfilter = False
TOAST INFO: poly_order = 0
TOAST INFO: apply_groundfilter = False
TOAST INFO: ground_order = 0
TOAST INFO: simulate_atmosphere = False
TOAST INFO: focalplane_radius_deg = None
TOAST INFO: atm_verbosity = 0
TOAST INFO: atm_lmin_center = 0.01
TOAST INFO: atm_lmin_sigma = 0.001
TOAST INFO: atm_lmax_center = 10.0
TOAST INFO: atm_lmax_sigma = 10.0
TOAST INFO: atm_gain = 3e-05
TOAST INFO: atm_zatm = 40000.0
TOAST INFO: atm_zmax = 200.0
TOAST INFO: atm_xstep = 10.0
TOAST INFO: atm_ystep = 10.0
TOAST INFO: atm_zstep = 10.0
TOAST INFO: atm_nelem_sim_max = 1000
TOAST INFO: atm_wind_dist = 500.0
TOAST INFO: atm_z0_center = 2000.0
TOAST INFO: atm_z0_sigma = 0.0
TOAST INFO: atm_T0_center = 280.0
TOAST INFO: atm_T0_sigma = 10.0
TOAST INFO: atm_cache = atm_cache
TOAST INFO: simulate_noise = True
TOAST INFO: apply_gainscrambler = False
TOAST INFO: gain_sigma = 0.01
TOAST INFO: madam_prefix = toast
TOAST INFO: madam_iter_max = 1000
TOAST INFO: madam_precond_width = 100
TOAST INFO: madam_precond_width_min = None
TOAST INFO: madam_precond_width_max = None
TOAST INFO: madam_baseline_length = 10000.0
TOAST INFO: madam_baseline_order = 0
TOAST INFO: madam_noisefilter = False
TOAST INFO: madam_parfile = None
TOAST INFO: madam_allreduce = False
TOAST INFO: madam_concatenate_messages = True
TOAST INFO: destripe = False
TOAST INFO: write_binmap = False
TOAST INFO: write_hits = True
TOAST INFO: write_wcov = True
TOAST INFO: write_wcov_inv = True
TOAST INFO: conserve_memory = True
TOAST INFO: input_map = None
TOAST INFO: pysm_model = None
TOAST INFO: pysm_apply_beam = True
TOAST INFO: pysm_precomputed_cmb_K_CMB = None
TOAST INFO: ground_map = None
TOAST INFO: ground_nside = 128
TOAST INFO: ground_fwhm_deg = 10
TOAST INFO: ground_lmax = 256
TOAST INFO: ground_scale = 0.001
TOAST INFO: ground_power = -1
TOAST INFO: simulate_ground = False
TOAST INFO: tidas = None
TOAST INFO: spt3g = None
TOAST INFO: MC_start = 0
TOAST INFO: MC_count = 1
TOAST INFO: outdir = out
TOAST INFO: focalplane = focalplane_37.pkl
TOAST INFO: freq = 100
TOAST INFO: Parsed parameters: 0.03 seconds (1 calls)
TOAST INFO: Load 15 (sub)scans in schedule.txt: 0.01 seconds (1 calls)
TOAST INFO: Loading schedule(s): 0.02 seconds (1 calls)
TOAST INFO: Loading focalplanes: 0.01 seconds (1 calls)
TOAST INFO: Group # 0 has 15 observations.
TOAST INFO: Simulated scans: 0.61 seconds (1 calls)
TOAST INFO: Expanding pointing
TOAST INFO: Pointing generation: 0.12 seconds (1 calls)
TOAST INFO: Scanning local submaps
TOAST INFO: Identify local submaps: 0.02 seconds (1 calls)
TOAST INFO: Processing frequency 100.0GHz 1 / 1, MC = 0
TOAST INFO: Simulating noise
TOAST INFO: Simulate noise: 0.11 seconds (1 calls)
TOAST INFO: Making maps
TOAST INFO: Mapping toast_100_telescope_all_time_all
TOAST INFO: OpMadam: 100.00 % of samples are included in valid intervals.
TOAST INFO: Collect period ranges: 0.00 seconds (1 calls)
TOAST INFO: Collect dataset dimensions: 0.00 seconds (1 calls)
TOAST INFO: Stage time: 0.00 seconds (1 calls)
<toast.Environment
Source code version = 2.3.1.dev1428
Logging level = INFO
Handling enabled for 0 signals:
Max threads = 4
MPI build enabled
MPI runtime enabled
>
Memory usage after staging time
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.152 GB < 119.152 +- 0.000 GB < 119.152 GB
percent : 5.300 % < 5.300 +- 0.000 % < 5.300 %
used : 5.494 GB < 5.494 +- 0.000 GB < 5.494 GB
free : 119.497 GB < 119.497 +- 0.000 GB < 119.497 GB
active : 3.504 GB < 3.504 +- 0.000 GB < 3.504 GB
inactive : 660.160 MB < 660.160 +- 0.000 MB < 660.160 MB
buffers : 15.516 MB < 15.516 +- 0.000 MB < 15.516 MB
cached : 796.797 MB < 796.797 +- 0.000 MB < 796.797 MB
shared : 394.254 MB < 394.254 +- 0.000 MB < 394.254 MB
slab : 496.387 MB < 496.387 +- 0.000 MB < 496.387 MB
TOAST INFO: Stage signal: 0.07 seconds (1 calls)
Node has 0.300 GB allocated in TOAST TOD caches and 0.017 GB in Madam caches (0.317 GB total) after staging time
Memory usage after staging signal
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.112 GB < 119.112 +- 0.000 GB < 119.112 GB
percent : 5.300 % < 5.300 +- 0.000 % < 5.300 %
used : 5.534 GB < 5.534 +- 0.000 GB < 5.534 GB
free : 119.457 GB < 119.457 +- 0.000 GB < 119.457 GB
active : 3.549 GB < 3.549 +- 0.000 GB < 3.549 GB
inactive : 660.629 MB < 660.629 +- 0.000 MB < 660.629 MB
buffers : 15.516 MB < 15.516 +- 0.000 MB < 15.516 MB
cached : 797.332 MB < 797.332 +- 0.000 MB < 797.332 MB
shared : 394.297 MB < 394.297 +- 0.000 MB < 394.297 MB
slab : 496.438 MB < 496.438 +- 0.000 MB < 496.438 MB
TOAST INFO: Stage pixels 1 / 32: 0.07 seconds (1 calls)
TOAST INFO: Stage pixels 2 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 3 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 4 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 5 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 6 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 7 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 8 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 9 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 10 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixels: 0.13 seconds (1 calls)
Node has 0.300 GB allocated in TOAST TOD caches and 0.056 GB in Madam caches (0.356 GB total) after staging signal
Memory usage after staging pixels
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.092 GB < 119.092 +- 0.000 GB < 119.092 GB
percent : 5.300 % < 5.300 +- 0.000 % < 5.300 %
used : 5.554 GB < 5.554 +- 0.000 GB < 5.554 GB
free : 119.436 GB < 119.436 +- 0.000 GB < 119.436 GB
active : 3.570 GB < 3.570 +- 0.000 GB < 3.570 GB
inactive : 661.781 MB < 661.781 +- 0.000 MB < 661.781 MB
buffers : 15.516 MB < 15.516 +- 0.000 MB < 15.516 MB
cached : 798.172 MB < 798.172 +- 0.000 MB < 798.172 MB
shared : 394.422 MB < 394.422 +- 0.000 MB < 394.422 MB
slab : 496.555 MB < 496.555 +- 0.000 MB < 496.555 MB
TOAST INFO: Stage pixel weights 1 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 2 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 3 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 4 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 5 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 6 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 7 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 8 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 9 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 10 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights: 0.17 seconds (1 calls)
Node has 0.266 GB allocated in TOAST TOD caches and 0.076 GB in Madam caches (0.341 GB total) after staging pixels
Memory usage after staging pixel weights
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.034 GB < 119.034 +- 0.000 GB < 119.034 GB
percent : 5.400 % < 5.400 +- 0.000 % < 5.400 %
used : 5.613 GB < 5.613 +- 0.000 GB < 5.613 GB
free : 119.377 GB < 119.377 +- 0.000 GB < 119.377 GB
active : 3.631 GB < 3.631 +- 0.000 GB < 3.631 GB
inactive : 662.211 MB < 662.211 +- 0.000 MB < 662.211 MB
buffers : 15.516 MB < 15.516 +- 0.000 MB < 15.516 MB
cached : 798.809 MB < 798.809 +- 0.000 MB < 798.809 MB
shared : 394.500 MB < 394.500 +- 0.000 MB < 394.500 MB
slab : 496.637 MB < 496.637 +- 0.000 MB < 496.637 MB
TOAST INFO: Stage all data: 0.52 seconds (1 calls)
TOAST INFO: Collect PSD info: 0.00 seconds (1 calls)
Node has 0.148 GB allocated in TOAST TOD caches and 0.135 GB in Madam caches (0.282 GB total) after staging pixel weights
Memory usage just before calling libmadam.destripe
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.033 GB < 119.033 +- 0.000 GB < 119.033 GB
percent : 5.400 % < 5.400 +- 0.000 % < 5.400 %
used : 5.613 GB < 5.613 +- 0.000 GB < 5.613 GB
free : 119.377 GB < 119.377 +- 0.000 GB < 119.377 GB
active : 3.631 GB < 3.631 +- 0.000 GB < 3.631 GB
inactive : 662.219 MB < 662.219 +- 0.000 MB < 662.219 MB
buffers : 15.516 MB < 15.516 +- 0.000 MB < 15.516 MB
cached : 798.816 MB < 798.816 +- 0.000 MB < 798.816 MB
shared : 394.508 MB < 394.508 +- 0.000 MB < 394.508 MB
slab : 496.637 MB < 496.637 +- 0.000 MB < 496.637 MB
OMP: 32 tasks with 2 procs per node, 4 threads per task.
Program MADAM
Destriping of CMB data with a noise filter
Version 3.7
Examining periods
Flagged 0 samples on 0 periods that had less than 0.100% of unflagged samples
Total number of samples (single detector): 2275200
Zero-weight fraction (all detectors): 0.000 %
Flagged fraction (all detectors): 2.316 %
Total number of intervals: 480
Max number of samples per task: 71100
Initializing parameters
Adjusting noise weights using noise spectra
Polarized detectors present: Will produce polarization maps
noise_weights_from_psd = T
radiometers = T
mode_detweight = 0
istart_mission = 0
nosamples_tot = 2275200
Initializing parallelization
ntasks = 32 Number or processes
nosamples_tot = 2275200 Total samples
nosamples_proc_max = 71100 Samples/process
MCMode = F
write_cut = F
basis_func = Legendre Destriping function basis
basis_order = 0 Destriping function order
bin_subsets = F
ntasks = 32 Number of processes
nthreads = 4 Number of threads per process
info = 3 Screen output level
fsample = 1.0000 Sampling frequency (Hz)
nmap = 3 Polarization included
ncc = 6 Independent wcov elements
Input files:
nside_map = 64 Healpix resolution (output map)
nside_cross = 32 Healpix resolution (destriping)
nside_submap = 16 Submap resolution
concatenate_messages = T use mpi_alltoallv to communicate
reassign_submaps = T minimize communication by reassigning submaps
pixmode_map = 2 Pixel rejection criterion (output map)
pixmode_cross = 2 Pixel rejection criterion (destriping)
pixlim_map = 1.00000E-02 Pixel rejection limit (output map)
pixlim_cross = 1.00000E-03 Pixel rejection limit (destriping)
Standard mode
psdlen = 1000000 Length of requested noise PSD
psd_downsample = 10 PSD downsampling factor
kfirst = F First destriping OFF
cglimit = 1.00000E-12 Iteration convergence limit
iter_min = 3 Minimum number of iterations
iter_max = 1000 Maximum number of iterations
precond_width_min = 0 Min width of the preconditioner band matrix
precond_width_max = 0 Max width of the preconditioner band matrix
No preconditioning
flag_by_horn = F Flags are independent
mode_detweight = 0 Detector weighting mode: sigma from simulation file
time_unit = pp Time unit = pointing period
mission_time = 480 Mission length in time units
nosamples_tot = 2275200 Total samples
= 632.0000 hours
Detectors available on the FIRST process and noise according to the FIRST period
detector sigma weight 1/sqrt(weight)
fake_00A 0.10488E-02 0.90911E+06 0.10488E-02
fake_00B 0.10488E-02 0.90911E+06 0.10488E-02
fake_01A 0.10488E-02 0.90911E+06 0.10488E-02
Output files
file_root = toast_100_telescope_all_time_all
file_hit = out/00000000/100/toast_100_telescope_all_time_all_hmap.fits
file_matrix = out/00000000/100/toast_100_telescope_all_time_all_wcov_inv.fits
file_wcov = out/00000000/100/toast_100_telescope_all_time_all_wcov.fits
binary_output = F
concatenate_binary = F
TOD memory min = 1.70 MB max = 2.24 MB total = 59.67 MB
Baseline memory min = 0.00 B max = 0.00 B total = 0.00 B
Pointing memory min = 2.71 MB max = 4.07 MB total = 100.35 MB
Basis function memory min = 0.00 B max = 0.00 B total = 0.00 B
Clock = 0.014 s
Total submaps = 3072 submap size = 16
Local submaps: min = 74 max = 94 mean = 85.28
Submap table memory min = 384.00 kB max = 384.00 kB total = 12.00 MB
local maps memory min = 115.72 kB max = 146.97 kB total = 4.17 MB
All2allv memory min = 97.62 kB max = 186.00 kB total = 4.68 MB
Map memory min = 156.00 kB max = 156.00 kB total = 4.88 MB
Building pixel matrices...
Counting hits...
Binning TOD...
Clock = 0.062 s
Finalization begins
Clock = 0.064 s
Writing pixel matrix...
0 : Write matrix completed in 0.282 s
Inverting pixel matrices...
Pixel matrix written in out/00000000/100/toast_100_telescope_all_time_all_wcov_inv.fits
1417 pixels solved
47735 pixels unsolved
0 pixels had decoupled intensity
0 : Invert pixelmatrix_map completed in 0.345 s
Constructing output map...
Destriped map:
Map 1 2 3
Writing pixel matrix...
Std 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Mean 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Min 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Max 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
0 : Write matrix completed in 0.210 s
Writing hits...
Pixel matrix written in out/00000000/100/toast_100_telescope_all_time_all_wcov.fits
Hit count written in out/00000000/100/toast_100_telescope_all_time_all_hmap.fits
Clock = 0.976 s
MEMORY (MB):
Detector pointing min = 2.71 MB max = 4.07 MB total = 100.35 MB
TOD buffer min = 1.70 MB max = 2.24 MB total = 59.67 MB
Maps min = 156.00 kB max = 156.00 kB total = 4.88 MB
Baselines min = 0.60 kB max = 0.60 kB total = 19.22 kB
Basis functions min = 0.00 B max = 0.00 B total = 0.00 B
Noise filter min = 0.00 B max = 0.00 B total = 0.00 B
Preconditioner min = 0.00 B max = 0.00 B total = 0.00 B
Submap table min = 384.00 kB max = 384.00 kB total = 12.00 MB
Temporary maps min = 115.72 kB max = 146.97 kB total = 4.17 MB
All2All buffers min = 97.62 kB max = 186.00 kB total = 4.68 MB
CG work space min = 0.00 B max = 0.00 B total = 0.00 B
NCM min = 0.00 B max = 0.00 B total = 0.00 B
Total min = 5.16 MB max = 7.15 MB total = 185.76 MB
WALL-CLOCK TIME (s):
Inverting pixel matrices mean = 0.3 min = 0.3 max = 0.3
Finalization and output mean = 0.9 min = 0.9 max = 0.9
Total 1.0 s ( 0.03 CPU hours)
TOAST INFO: Unstage signal 1 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 1 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 1 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 1 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 2 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 2 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 2 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 2 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 3 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 3 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 3 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 3 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 4 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 4 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 4 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 4 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 5 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 5 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 5 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 5 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 6 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 6 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 6 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 6 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 7 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 7 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 7 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 7 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 8 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 8 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 8 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 8 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 9 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 9 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 9 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 9 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 10 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 10 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 10 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 10 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage signal 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage all data: 0.17 seconds (1 calls)
TOAST INFO: Mapping toast_100_telescope_all_time_all: 1.74 seconds (1 calls)
TOAST INFO: Madam total: 1.75 seconds (1 calls)
TOAST INFO: Gather and dump timing info: 0.01 seconds (1 calls)
TOAST INFO: toast_ground_sim.py: 2.71 seconds (1 calls)
Out[11]:
0
Let's examine the resulting hits and depth map. The file naming convention may seem a little awkward but follows from the fact that a single run of toast_ground_sim.py may map multiple telescopes, frequencies and time splits.
In [12]:
import matplotlib.pyplot as plt
%matplotlib inline
import healpy
hits = healpy.read_map("out/00000000/100/toast_100_telescope_all_time_all_hmap.fits")
hits[hits == 0] = healpy.UNSEEN
healpy.mollview(hits, unit="hits", title="Total hits")
healpy.graticule(22.5, verbose=False)
NSIDE = 64 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING
In [13]:
wcov = healpy.read_map("out/00000000/100/toast_100_telescope_all_time_all_wcov.fits")
wcov *= 1e12 # from K^2 to uK^2
wcov[wcov == 0] = healpy.UNSEEN
healpy.mollview(wcov, unit="$\mu$K$^2$", title="White noise variance", min=1e0, max=1e3)
healpy.graticule(22.5, verbose=False)
NSIDE = 64 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING
Advanced topics in ground scheduling¶
Cooler cycle format¶
it is possible to instruct the scheduler to add regular breaks in the schedule to cycle the cooler or to perform other maintenance activities. The cooler cycle is a pseudo patch that the scheduler considers like other targets when deciding what to observe next. The full syntax is:
--patch <name>,COOLER,<weight>,<power>,<hold_time_min>,<hold_time_max>,<cycle_time>,<az>,<el>
All of the time arguments are given in hours. The priority of the patch depends on the time since the last cycle occurred. It is infinity until hold_time_min has elapsed and then begins to decrease according to a power law set by power. Priority at hold_time_max is zero.
Planet scans¶
The scheduler can target planets just like stationary patches. The SSO (solar system object) format is
--patch <name>,SSO,<priority>,<radius [deg]>
All orbiting bodies recognized by pyEphem are supported.
Oscillating patches¶
The scheduler designs the scans so that the azimuth range is kept fixed and the boresight sweeps the entire patch. This usually implies a certain amount of spillover integration time outside the patch. This can produce an excess of hits at the boundary of two patches. The scheduler offers a way to smear the spillover by systematically shifting the position of the patches in RA and DEC. The arguments to accomplish this are
--ra-period <period [visits]>
--ra-amplitude <amplitude [deg]>
--dec-period <period [visits]>
--dec-amplitude <amplitude [deg]>
Patches will systematically shift after each visit, returning to their fiducial positions after each period.
Horizontal (high cadence) patch definition¶
Horizontal patch definition specifies the observing elevation and the azimuth range. The scheduler parks the telescope at the given elevation and scans until the constraints (Sun, Moon, cooler hold time) prevent continuing. If possible, scanning is continued by switching between rising and setting scan.
--patch <name>,HORIZONTAL,<priority>,<az min [deg]>,<az max [deg]>,<el [deg]>,<scan time [min]>
Polygon patch definition¶
Patches do not need to be rectangular or circular. An arbitrary polygon shape can be specified by giving the corner coordinates.
--patch <name>,<priority>,<RA_0 [deg]>,<DEC_0 [deg]>,...,,<RA_N-1 [deg]>,<DEC_N-1 [deg]>
Elevation penalty¶
Lower observing elevations are subject to higher levels of photon noise from the atmosphere. It is possible to instruct the scheduler to modulate the relative priorities of the available patches based on their elevation.
--elevation-penalty-limit <elevation [deg]>
--elevation-penalty-power <power>
If the available patch is below elevation-penalty-limit, the priority is modulated by $\left(\frac{limit}{elevation}\right)^{power}$. This way low elevation scans are reserved for targets that cannot be observed at higher elevation or when no targets are available higher.
Block-out¶
January and February weather in the Atacama is known to be problematic for observing. It is possible to instruct the scheduler to skip certain periods of the calendar year with
--block-out <start month>/<start day>-<end month>/<end day>
or with
--block-out <start year>/<start month>/<start day>-<end year>/<end month>/<end day>
All fields are integers. The dates are in UTC.
Gaps¶
The are two short gap lengths in the scheduler
--gap-small <gap [s]>
--gap <gap [s]>
The gap-small is applied when a single CES is broken up into sub scans. The regular gap is applied between separate observations.
Pole scheduling¶
Observing from the Poles is unlike anywhere else on Earth. Patches will not drift across a constant elevation line. Instead, the telescope must be stepped in elevation. The Pole scheduling mode is enabled with
--pole-mode
And the step time and size are controlled with
--pole-el-step <step [deg]>
--pole-ces-time <time [s]>
In [14]:
# --site-lat "-89:59.464" \
# --site-lon "-44:39" \
! toast_ground_schedule.py \
--site-lat "-89.991" \
--site-lon "-44.65" \
--site-alt 2843 \
--site-name South_Pole \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-01 12:00:00" \
--patch-coord C \
--patch small_patch,1,40,-40,44,-44 \
--pole-mode \
--pole-el-step 0.25 \
--pole-ces-time 600 \
--out pole_schedule.txt
TOAST INFO: Adding patch "small_patch" TOAST INFO: Rectangular format TOAST INFO: Global timer: toast_ground_schedule: 0.44 seconds (1 calls)
The resulting schedule has each pass of the target split into (0.25$^\circ$, 10min) steps. It takes 16 steps (2:40h) to cover the 4$^\circ\times$4$^\circ$ degree field.
In [15]:
! cat pole_schedule.txt
#Site Telescope Latitude [deg] Longitude [deg] Elevation [m] South_Pole LAT -89.991 -44.650 2843.0 #Start time UTC Stop time UTC Start MJD Stop MJD Patch name Az min Az max El R/S Sun el1 Sun az1 Sun el2 Sun az2 Moon el1 Moon az1 Moon el2 Moon az2 Phase Pass Sub 2020-01-01 00:00:00 2020-01-01 00:10:00 58849.000000 58849.006944 small_patch 342.47 348.72 39.94 S 23.08 225.42 23.08 222.92 9.16 293.70 9.13 291.27 0.30 0 0 2020-01-01 00:11:40 2020-01-01 00:21:40 58849.008102 58849.015046 small_patch 339.54 345.80 40.19 S 23.08 222.50 23.08 220.00 9.12 290.87 9.09 288.44 0.30 0 1 2020-01-01 00:23:20 2020-01-01 00:33:20 58849.016204 58849.023148 small_patch 336.62 342.87 40.44 S 23.08 219.59 23.08 217.09 9.09 288.03 9.06 285.60 0.30 0 2 2020-01-01 00:35:00 2020-01-01 00:45:00 58849.024306 58849.031250 small_patch 333.69 339.95 40.69 S 23.08 216.67 23.07 214.17 9.05 285.20 9.02 282.77 0.30 0 3 2020-01-01 00:46:40 2020-01-01 00:56:40 58849.032407 58849.039352 small_patch 330.77 337.02 40.94 S 23.07 213.76 23.07 211.26 9.02 282.37 8.99 279.94 0.30 0 4 2020-01-01 00:58:20 2020-01-01 01:08:20 58849.040509 58849.047454 small_patch 327.84 334.10 41.19 S 23.07 210.84 23.07 208.34 8.98 279.53 8.95 277.10 0.30 0 5 2020-01-01 01:10:00 2020-01-01 01:20:00 58849.048611 58849.055556 small_patch 324.92 331.17 41.44 S 23.07 207.93 23.07 205.43 8.95 276.70 8.92 274.27 0.30 0 6 2020-01-01 01:21:40 2020-01-01 01:31:40 58849.056713 58849.063657 small_patch 321.99 328.25 41.69 S 23.07 205.01 23.07 202.51 8.91 273.86 8.88 271.44 0.30 0 7 2020-01-01 01:33:20 2020-01-01 01:43:20 58849.064815 58849.071759 small_patch 319.06 325.32 41.94 S 23.07 202.09 23.07 199.60 8.88 271.03 8.85 268.60 0.31 0 8 2020-01-01 01:45:00 2020-01-01 01:55:00 58849.072917 58849.079861 small_patch 316.14 322.39 42.19 S 23.07 199.18 23.07 196.68 8.84 268.20 8.81 265.77 0.31 0 9 2020-01-01 01:56:40 2020-01-01 02:06:40 58849.081019 58849.087963 small_patch 313.21 319.47 42.44 S 23.07 196.26 23.07 193.76 8.81 265.36 8.78 262.93 0.31 0 10 2020-01-01 02:08:20 2020-01-01 02:18:20 58849.089120 58849.096065 small_patch 310.29 316.54 42.69 S 23.07 193.35 23.07 190.85 8.77 262.53 8.74 260.10 0.31 0 11 2020-01-01 02:20:00 2020-01-01 02:30:00 58849.097222 58849.104167 small_patch 307.36 313.62 42.94 S 23.07 190.43 23.07 187.93 8.74 259.70 8.71 257.27 0.31 0 12 2020-01-01 02:31:40 2020-01-01 02:41:40 58849.105324 58849.112269 small_patch 304.44 310.69 43.19 S 23.07 187.52 23.07 185.02 8.70 256.86 8.67 254.43 0.31 0 13 2020-01-01 02:43:20 2020-01-01 02:53:20 58849.113426 58849.120370 small_patch 301.51 307.77 43.44 S 23.07 184.60 23.07 182.10 8.67 254.03 8.64 251.60 0.31 0 14 2020-01-01 02:55:00 2020-01-01 03:05:00 58849.121528 58849.128472 small_patch 298.59 304.84 43.69 S 23.07 181.69 23.07 179.19 8.63 251.19 8.60 248.77 0.31 0 15 2020-01-01 03:06:40 2020-01-01 03:16:40 58849.129630 58849.136574 small_patch 295.67 301.92 39.94 S 23.07 178.77 23.06 176.27 8.60 248.36 8.57 245.93 0.31 1 0 2020-01-01 03:18:20 2020-01-01 03:28:20 58849.137731 58849.144676 small_patch 292.74 299.00 40.19 S 23.06 175.85 23.06 173.36 8.56 245.53 8.53 243.10 0.31 1 1 2020-01-01 03:30:00 2020-01-01 03:40:00 58849.145833 58849.152778 small_patch 289.82 296.07 40.44 S 23.06 172.94 23.06 170.44 8.53 242.69 8.50 240.26 0.31 1 2 2020-01-01 03:41:40 2020-01-01 03:51:40 58849.153935 58849.160880 small_patch 286.89 293.15 40.69 S 23.06 170.02 23.06 167.52 8.49 239.86 8.46 237.43 0.31 1 3 2020-01-01 03:53:20 2020-01-01 04:03:20 58849.162037 58849.168981 small_patch 283.97 290.22 40.94 S 23.06 167.11 23.06 164.61 8.46 237.02 8.43 234.60 0.31 1 4 2020-01-01 04:05:00 2020-01-01 04:15:00 58849.170139 58849.177083 small_patch 281.04 287.30 41.19 S 23.06 164.19 23.06 161.69 8.42 234.19 8.39 231.76 0.31 1 5 2020-01-01 04:16:40 2020-01-01 04:26:40 58849.178241 58849.185185 small_patch 278.12 284.37 41.44 S 23.06 161.28 23.06 158.78 8.39 231.36 8.36 228.93 0.32 1 6 2020-01-01 04:28:20 2020-01-01 04:38:20 58849.186343 58849.193287 small_patch 275.19 281.45 41.69 S 23.06 158.36 23.06 155.86 8.35 228.52 8.32 226.09 0.32 1 7 2020-01-01 04:40:00 2020-01-01 04:50:00 58849.194444 58849.201389 small_patch 272.27 278.52 41.94 S 23.06 155.45 23.06 152.95 8.32 225.69 8.29 223.26 0.32 1 8 2020-01-01 04:51:40 2020-01-01 05:01:40 58849.202546 58849.209491 small_patch 269.34 275.60 42.19 S 23.06 152.53 23.06 150.03 8.28 222.85 8.25 220.43 0.32 1 9 2020-01-01 05:03:20 2020-01-01 05:13:20 58849.210648 58849.217593 small_patch 266.42 272.67 42.44 S 23.06 149.62 23.06 147.12 8.25 220.02 8.22 217.59 0.32 1 10 2020-01-01 05:15:00 2020-01-01 05:25:00 58849.218750 58849.225694 small_patch 263.49 269.75 42.69 S 23.06 146.70 23.06 144.20 8.21 217.19 8.18 214.76 0.32 1 11 2020-01-01 05:26:40 2020-01-01 05:36:40 58849.226852 58849.233796 small_patch 260.57 266.82 42.94 S 23.06 143.78 23.06 141.28 8.18 214.35 8.14 211.92 0.32 1 12 2020-01-01 05:38:20 2020-01-01 05:48:20 58849.234954 58849.241898 small_patch 257.64 263.89 43.19 S 23.06 140.87 23.06 138.37 8.14 211.52 8.11 209.09 0.32 1 13 2020-01-01 05:50:00 2020-01-01 06:00:00 58849.243056 58849.250000 small_patch 254.72 260.97 43.44 S 23.06 137.95 23.06 135.45 8.10 208.68 8.07 206.25 0.32 1 14 2020-01-01 06:01:40 2020-01-01 06:11:40 58849.251157 58849.258102 small_patch 251.79 258.05 43.69 S 23.06 135.04 23.06 132.54 8.07 205.85 8.04 203.42 0.32 1 15 2020-01-01 06:13:20 2020-01-01 06:23:20 58849.259259 58849.266204 small_patch 248.87 255.13 39.93 S 23.06 132.12 23.06 129.62 8.03 203.02 8.00 200.59 0.32 2 0 2020-01-01 06:25:00 2020-01-01 06:35:00 58849.267361 58849.274306 small_patch 245.95 252.20 40.18 S 23.06 129.21 23.06 126.71 8.00 200.18 7.97 197.75 0.32 2 1 2020-01-01 06:36:40 2020-01-01 06:46:40 58849.275463 58849.282407 small_patch 243.03 249.28 40.43 S 23.06 126.29 23.06 123.79 7.96 197.35 7.93 194.92 0.32 2 2 2020-01-01 06:48:20 2020-01-01 06:58:20 58849.283565 58849.290509 small_patch 240.10 246.35 40.68 S 23.06 123.37 23.06 120.88 7.93 194.51 7.90 192.08 0.32 2 3 2020-01-01 07:00:00 2020-01-01 07:10:00 58849.291667 58849.298611 small_patch 237.17 243.43 40.93 S 23.06 120.46 23.06 117.96 7.89 191.68 7.86 189.25 0.33 2 4 2020-01-01 07:11:40 2020-01-01 07:21:40 58849.299769 58849.306713 small_patch 234.25 240.50 41.18 S 23.06 117.54 23.06 115.04 7.86 188.84 7.83 186.42 0.33 2 5 2020-01-01 07:23:20 2020-01-01 07:33:20 58849.307870 58849.314815 small_patch 231.33 237.58 41.43 S 23.06 114.63 23.06 112.13 7.82 186.01 7.79 183.58 0.33 2 6 2020-01-01 07:35:00 2020-01-01 07:45:00 58849.315972 58849.322917 small_patch 228.40 234.65 41.68 S 23.06 111.71 23.06 109.21 7.79 183.18 7.76 180.75 0.33 2 7 2020-01-01 07:46:40 2020-01-01 07:56:40 58849.324074 58849.331019 small_patch 225.47 231.73 41.93 S 23.06 108.80 23.06 106.30 7.75 180.34 7.72 177.91 0.33 2 8 2020-01-01 07:58:20 2020-01-01 08:08:20 58849.332176 58849.339120 small_patch 222.55 228.80 42.18 S 23.06 105.88 23.06 103.38 7.72 177.51 7.69 175.08 0.33 2 9 2020-01-01 08:10:00 2020-01-01 08:20:00 58849.340278 58849.347222 small_patch 219.63 225.88 42.43 S 23.06 102.97 23.06 100.47 7.68 174.67 7.65 172.24 0.33 2 10 2020-01-01 08:21:40 2020-01-01 08:31:40 58849.348380 58849.355324 small_patch 216.70 222.95 42.68 S 23.06 100.05 23.06 97.55 7.65 171.84 7.62 169.41 0.33 2 11 2020-01-01 08:33:20 2020-01-01 08:43:20 58849.356481 58849.363426 small_patch 213.77 220.03 42.93 S 23.06 97.13 23.06 94.64 7.61 169.00 7.58 166.57 0.33 2 12 2020-01-01 08:45:00 2020-01-01 08:55:00 58849.364583 58849.371528 small_patch 210.85 217.10 43.18 S 23.06 94.22 23.06 91.72 7.58 166.17 7.55 163.74 0.33 2 13 2020-01-01 08:56:40 2020-01-01 09:06:40 58849.372685 58849.379630 small_patch 207.93 214.18 43.43 S 23.06 91.30 23.05 88.80 7.54 163.33 7.51 160.91 0.33 2 14 2020-01-01 09:08:20 2020-01-01 09:18:20 58849.380787 58849.387731 small_patch 205.00 211.25 43.68 S 23.05 88.39 23.05 85.89 7.51 160.50 7.48 158.07 0.33 2 15 2020-01-01 09:20:00 2020-01-01 09:30:00 58849.388889 58849.395833 small_patch 202.08 208.34 39.93 S 23.05 85.47 23.05 82.97 7.47 157.67 7.44 155.24 0.33 3 0 2020-01-01 09:31:40 2020-01-01 09:41:40 58849.396991 58849.403935 small_patch 199.16 205.41 40.18 S 23.05 82.56 23.05 80.06 7.44 154.83 7.41 152.40 0.33 3 1 2020-01-01 09:43:20 2020-01-01 09:53:20 58849.405093 58849.412037 small_patch 196.24 202.49 40.43 S 23.05 79.64 23.05 77.14 7.40 152.00 7.37 149.57 0.34 3 2 2020-01-01 09:55:00 2020-01-01 10:05:00 58849.413194 58849.420139 small_patch 193.31 199.56 40.68 S 23.05 76.72 23.05 74.23 7.37 149.16 7.34 146.73 0.34 3 3 2020-01-01 10:06:40 2020-01-01 10:16:40 58849.421296 58849.428241 small_patch 190.39 196.64 40.93 S 23.05 73.81 23.05 71.31 7.33 146.33 7.30 143.90 0.34 3 4 2020-01-01 10:18:20 2020-01-01 10:28:20 58849.429398 58849.436343 small_patch 187.46 193.71 41.18 S 23.05 70.89 23.05 68.39 7.30 143.49 7.27 141.06 0.34 3 5 2020-01-01 10:30:00 2020-01-01 10:40:00 58849.437500 58849.444444 small_patch 184.54 190.79 41.43 S 23.05 67.98 23.05 65.48 7.26 140.66 7.23 138.23 0.34 3 6 2020-01-01 10:41:40 2020-01-01 10:51:40 58849.445602 58849.452546 small_patch 181.61 187.87 41.68 S 23.05 65.06 23.05 62.56 7.23 137.82 7.20 135.39 0.34 3 7 2020-01-01 10:53:20 2020-01-01 11:03:20 58849.453704 58849.460648 small_patch 178.69 184.94 41.93 S 23.05 62.15 23.05 59.65 7.19 134.99 7.16 132.56 0.34 3 8 2020-01-01 11:05:00 2020-01-01 11:15:00 58849.461806 58849.468750 small_patch 175.76 182.01 42.18 S 23.05 59.23 23.05 56.73 7.16 132.15 7.13 129.72 0.34 3 9 2020-01-01 11:16:40 2020-01-01 11:26:40 58849.469907 58849.476852 small_patch 172.84 179.09 42.43 S 23.05 56.31 23.05 53.81 7.12 129.32 7.09 126.89 0.34 3 10 2020-01-01 11:28:20 2020-01-01 11:38:20 58849.478009 58849.484954 small_patch 169.91 176.17 42.68 S 23.05 53.40 23.05 50.90 7.09 126.48 7.06 124.05 0.34 3 11 2020-01-01 11:40:00 2020-01-01 11:50:00 58849.486111 58849.493056 small_patch 166.99 173.24 42.93 S 23.05 50.48 23.05 47.98 7.05 123.65 7.02 121.22 0.34 3 12
Let's bin this schedule as well. We also demonstrate how parameters in the parameter file may be overridden
In [16]:
command = "toast_ground_sim.py @bin_schedule.par --schedule pole_schedule.txt --out out_pole"
runstr = None
if nersc_host is not None:
runstr = "srun -N 1 -C haswell -n 32 -c 2 --cpu_bind=cores -t 00:05:00"
if nersc_resv is not None:
runstr = "{} --reservation {}".format(runstr, nersc_resv)
else:
# Just use mpirun
runstr = "mpirun -np 4"
runcom = "{} {}".format(runstr, command)
print(runcom, flush=True)
sp.check_call(runcom, stderr=sp.STDOUT, shell=True)
srun -N 1 -C haswell -n 32 -c 2 --cpu_bind=cores -t 00:05:00 toast_ground_sim.py @bin_schedule.par --schedule pole_schedule.txt --out out_pole
Launched in background. Redirecting stdin to /dev/null
srun: job 25182218 queued and waiting for resources
srun: job 25182218 has been allocated resources
TOAST INFO: Running with 32 processes at 2019-10-16 05:34:47.531949
TOAST INFO: All parameters:
TOAST INFO: group_size = None
TOAST INFO: do_daymaps = False
TOAST INFO: do_seasonmaps = False
TOAST INFO: debug = False
TOAST INFO: scan_rate = 0.3
TOAST INFO: scan_accel = 10.0
TOAST INFO: sun_angle_min = 30.0
TOAST INFO: schedule = pole_schedule.txt
TOAST INFO: weather = None
TOAST INFO: timezone = 0
TOAST INFO: sample_rate = 1.0
TOAST INFO: coord = C
TOAST INFO: split_schedule = None
TOAST INFO: sort_schedule = True
TOAST INFO: hwp_rpm = None
TOAST INFO: hwp_step_deg = None
TOAST INFO: hwp_step_time_s = None
TOAST INFO: nside = 64
TOAST INFO: single_precision_pointing = False
TOAST INFO: common_flag_mask = 1
TOAST INFO: flush = False
TOAST INFO: apply_polyfilter = False
TOAST INFO: poly_order = 0
TOAST INFO: apply_groundfilter = False
TOAST INFO: ground_order = 0
TOAST INFO: simulate_atmosphere = False
TOAST INFO: focalplane_radius_deg = None
TOAST INFO: atm_verbosity = 0
TOAST INFO: atm_lmin_center = 0.01
TOAST INFO: atm_lmin_sigma = 0.001
TOAST INFO: atm_lmax_center = 10.0
TOAST INFO: atm_lmax_sigma = 10.0
TOAST INFO: atm_gain = 3e-05
TOAST INFO: atm_zatm = 40000.0
TOAST INFO: atm_zmax = 200.0
TOAST INFO: atm_xstep = 10.0
TOAST INFO: atm_ystep = 10.0
TOAST INFO: atm_zstep = 10.0
TOAST INFO: atm_nelem_sim_max = 1000
TOAST INFO: atm_wind_dist = 500.0
TOAST INFO: atm_z0_center = 2000.0
TOAST INFO: atm_z0_sigma = 0.0
TOAST INFO: atm_T0_center = 280.0
TOAST INFO: atm_T0_sigma = 10.0
TOAST INFO: atm_cache = atm_cache
TOAST INFO: simulate_noise = True
TOAST INFO: apply_gainscrambler = False
TOAST INFO: gain_sigma = 0.01
TOAST INFO: madam_prefix = toast
TOAST INFO: madam_iter_max = 1000
TOAST INFO: madam_precond_width = 100
TOAST INFO: madam_precond_width_min = None
TOAST INFO: madam_precond_width_max = None
TOAST INFO: madam_baseline_length = 10000.0
TOAST INFO: madam_baseline_order = 0
TOAST INFO: madam_noisefilter = False
TOAST INFO: madam_parfile = None
TOAST INFO: madam_allreduce = False
TOAST INFO: madam_concatenate_messages = True
TOAST INFO: destripe = False
TOAST INFO: write_binmap = False
TOAST INFO: write_hits = True
TOAST INFO: write_wcov = True
TOAST INFO: write_wcov_inv = True
TOAST INFO: conserve_memory = True
TOAST INFO: input_map = None
TOAST INFO: pysm_model = None
TOAST INFO: pysm_apply_beam = True
TOAST INFO: pysm_precomputed_cmb_K_CMB = None
TOAST INFO: ground_map = None
TOAST INFO: ground_nside = 128
TOAST INFO: ground_fwhm_deg = 10
TOAST INFO: ground_lmax = 256
TOAST INFO: ground_scale = 0.001
TOAST INFO: ground_power = -1
TOAST INFO: simulate_ground = False
TOAST INFO: tidas = None
TOAST INFO: spt3g = None
TOAST INFO: MC_start = 0
TOAST INFO: MC_count = 1
TOAST INFO: outdir = out_pole
TOAST INFO: focalplane = focalplane_37.pkl
TOAST INFO: freq = 100
TOAST INFO: Parsed parameters: 0.02 seconds (1 calls)
TOAST INFO: Load 61 (sub)scans in pole_schedule.txt: 0.05 seconds (1 calls)
TOAST INFO: Loading schedule(s): 0.05 seconds (1 calls)
TOAST INFO: Loading focalplanes: 0.00 seconds (1 calls)
TOAST INFO: Group # 0 has 61 observations.
TOAST INFO: Simulated scans: 3.10 seconds (1 calls)
TOAST INFO: Expanding pointing
TOAST INFO: Pointing generation: 0.13 seconds (1 calls)
TOAST INFO: Scanning local submaps
TOAST INFO: Identify local submaps: 0.02 seconds (1 calls)
TOAST INFO: Processing frequency 100.0GHz 1 / 1, MC = 0
TOAST INFO: Simulating noise
TOAST INFO: Simulate noise: 0.14 seconds (1 calls)
TOAST INFO: Making maps
TOAST INFO: Mapping toast_100_telescope_all_time_all
TOAST INFO: OpMadam: 100.00 % of samples are included in valid intervals.
TOAST INFO: Collect period ranges: 0.01 seconds (1 calls)
TOAST INFO: Collect dataset dimensions: 0.01 seconds (1 calls)
TOAST INFO: Stage time: 0.01 seconds (1 calls)
<toast.Environment
Source code version = 2.3.1.dev1428
Logging level = INFO
Handling enabled for 0 signals:
Max threads = 4
MPI build enabled
MPI runtime enabled
>
Memory usage after staging time
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.273 GB < 119.273 +- 0.000 GB < 119.273 GB
percent : 5.200 % < 5.200 +- 0.000 % < 5.200 %
used : 5.374 GB < 5.374 +- 0.000 GB < 5.374 GB
free : 119.614 GB < 119.614 +- 0.000 GB < 119.614 GB
active : 3.410 GB < 3.410 +- 0.000 GB < 3.410 GB
inactive : 665.215 MB < 665.215 +- 0.000 MB < 665.215 MB
buffers : 15.723 MB < 15.723 +- 0.000 MB < 15.723 MB
cached : 799.660 MB < 799.660 +- 0.000 MB < 799.660 MB
shared : 394.648 MB < 394.648 +- 0.000 MB < 394.648 MB
slab : 496.895 MB < 496.895 +- 0.000 MB < 496.895 MB
TOAST INFO: Stage signal: 0.08 seconds (1 calls)
Node has 0.154 GB allocated in TOAST TOD caches and 0.009 GB in Madam caches (0.163 GB total) after staging time
Memory usage after staging signal
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.251 GB < 119.251 +- 0.000 GB < 119.251 GB
percent : 5.200 % < 5.200 +- 0.000 % < 5.200 %
used : 5.397 GB < 5.397 +- 0.000 GB < 5.397 GB
free : 119.592 GB < 119.592 +- 0.000 GB < 119.592 GB
active : 3.437 GB < 3.437 +- 0.000 GB < 3.437 GB
inactive : 665.215 MB < 665.215 +- 0.000 MB < 665.215 MB
buffers : 15.723 MB < 15.723 +- 0.000 MB < 15.723 MB
cached : 799.660 MB < 799.660 +- 0.000 MB < 799.660 MB
shared : 394.648 MB < 394.648 +- 0.000 MB < 394.648 MB
slab : 496.895 MB < 496.895 +- 0.000 MB < 496.895 MB
TOAST INFO: Stage pixels 1 / 32: 0.09 seconds (1 calls)
TOAST INFO: Stage pixels 2 / 32: 0.02 seconds (1 calls)
TOAST INFO: Stage pixels 3 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 4 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 5 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 6 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 7 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 8 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 9 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 10 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 11 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 12 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 13 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 14 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 15 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 16 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 17 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 18 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 19 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 20 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 21 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 22 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 23 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 24 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 25 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 26 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 27 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 28 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 29 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 30 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 31 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels 32 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixels: 0.35 seconds (1 calls)
Node has 0.154 GB allocated in TOAST TOD caches and 0.029 GB in Madam caches (0.183 GB total) after staging signal
Memory usage after staging pixels
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.242 GB < 119.242 +- 0.000 GB < 119.242 GB
percent : 5.200 % < 5.200 +- 0.000 % < 5.200 %
used : 5.406 GB < 5.406 +- 0.000 GB < 5.406 GB
free : 119.581 GB < 119.581 +- 0.000 GB < 119.581 GB
active : 3.453 GB < 3.453 +- 0.000 GB < 3.453 GB
inactive : 667.000 MB < 667.000 +- 0.000 MB < 667.000 MB
buffers : 15.723 MB < 15.723 +- 0.000 MB < 15.723 MB
cached : 801.543 MB < 801.543 +- 0.000 MB < 801.543 MB
shared : 394.828 MB < 394.828 +- 0.000 MB < 394.828 MB
slab : 497.270 MB < 497.270 +- 0.000 MB < 497.270 MB
TOAST INFO: Stage pixel weights 1 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 2 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 3 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 4 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 5 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 6 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 7 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 8 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 9 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 10 / 32: 0.01 seconds (1 calls)
TOAST INFO: Stage pixel weights 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Stage pixel weights: 0.22 seconds (1 calls)
Node has 0.137 GB allocated in TOAST TOD caches and 0.039 GB in Madam caches (0.176 GB total) after staging pixels
Memory usage after staging pixel weights
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.211 GB < 119.211 +- 0.000 GB < 119.211 GB
percent : 5.200 % < 5.200 +- 0.000 % < 5.200 %
used : 5.436 GB < 5.436 +- 0.000 GB < 5.436 GB
free : 119.550 GB < 119.550 +- 0.000 GB < 119.550 GB
active : 3.483 GB < 3.483 +- 0.000 GB < 3.483 GB
inactive : 667.355 MB < 667.355 +- 0.000 MB < 667.355 MB
buffers : 15.723 MB < 15.723 +- 0.000 MB < 15.723 MB
cached : 802.004 MB < 802.004 +- 0.000 MB < 802.004 MB
shared : 394.914 MB < 394.914 +- 0.000 MB < 394.914 MB
slab : 497.359 MB < 497.359 +- 0.000 MB < 497.359 MB
TOAST INFO: Stage all data: 0.81 seconds (1 calls)
TOAST INFO: Collect PSD info: 0.00 seconds (1 calls)
Node has 0.076 GB allocated in TOAST TOD caches and 0.069 GB in Madam caches (0.145 GB total) after staging pixel weights
Memory usage just before calling libmadam.destripe
total : 125.785 GB < 125.785 +- 0.000 GB < 125.785 GB
available : 119.211 GB < 119.211 +- 0.000 GB < 119.211 GB
percent : 5.200 % < 5.200 +- 0.000 % < 5.200 %
used : 5.437 GB < 5.437 +- 0.000 GB < 5.437 GB
free : 119.549 GB < 119.549 +- 0.000 GB < 119.549 GB
active : 3.483 GB < 3.483 +- 0.000 GB < 3.483 GB
inactive : 667.422 MB < 667.422 +- 0.000 MB < 667.422 MB
buffers : 15.723 MB < 15.723 +- 0.000 MB < 15.723 MB
cached : 802.070 MB < 802.070 +- 0.000 MB < 802.070 MB
shared : 394.926 MB < 394.926 +- 0.000 MB < 394.926 MB
slab : 497.371 MB < 497.371 +- 0.000 MB < 497.371 MB
OMP: 32 tasks with 2 procs per node, 4 threads per task.
Program MADAM
Destriping of CMB data with a noise filter
Version 3.7
Examining periods
Flagged 0 samples on 0 periods that had less than 0.100% of unflagged samples
Total number of samples (single detector): 1171200
Zero-weight fraction (all detectors): 0.000 %
Flagged fraction (all detectors): 97.328 %
Total number of intervals: 1952
Max number of samples per task: 36600
Initializing parameters
Adjusting noise weights using noise spectra
Polarized detectors present: Will produce polarization maps
noise_weights_from_psd = T
radiometers = T
mode_detweight = 0
istart_mission = 0
nosamples_tot = 1171200
Initializing parallelization
ntasks = 32 Number or processes
nosamples_tot = 1171200 Total samples
nosamples_proc_max = 36600 Samples/process
MCMode = F
write_cut = F
basis_func = Legendre Destriping function basis
basis_order = 0 Destriping function order
bin_subsets = F
ntasks = 32 Number of processes
nthreads = 4 Number of threads per process
info = 3 Screen output level
fsample = 1.0000 Sampling frequency (Hz)
nmap = 3 Polarization included
ncc = 6 Independent wcov elements
Input files:
nside_map = 64 Healpix resolution (output map)
nside_cross = 32 Healpix resolution (destriping)
nside_submap = 16 Submap resolution
concatenate_messages = T use mpi_alltoallv to communicate
reassign_submaps = T minimize communication by reassigning submaps
pixmode_map = 2 Pixel rejection criterion (output map)
pixmode_cross = 2 Pixel rejection criterion (destriping)
pixlim_map = 1.00000E-02 Pixel rejection limit (output map)
pixlim_cross = 1.00000E-03 Pixel rejection limit (destriping)
Standard mode
psdlen = 1000000 Length of requested noise PSD
psd_downsample = 10 PSD downsampling factor
kfirst = F First destriping OFF
cglimit = 1.00000E-12 Iteration convergence limit
iter_min = 3 Minimum number of iterations
iter_max = 1000 Maximum number of iterations
precond_width_min = 0 Min width of the preconditioner band matrix
precond_width_max = 0 Max width of the preconditioner band matrix
No preconditioning
flag_by_horn = F Flags are independent
mode_detweight = 0 Detector weighting mode: sigma from simulation file
time_unit = pp Time unit = pointing period
mission_time = 1952 Mission length in time units
nosamples_tot = 1171200 Total samples
= 325.3333 hours
Detectors available on the FIRST process and noise according to the FIRST period
detector sigma weight 1/sqrt(weight)
fake_00A 0.10488E-02 0.90911E+06 0.10488E-02
fake_00B 0.10488E-02 0.90911E+06 0.10488E-02
fake_01A 0.10488E-02 0.90911E+06 0.10488E-02
Output files
file_root = toast_100_telescope_all_time_all
file_hit = out_pole/00000000/100/toast_100_telescope_all_time_all_hmap.fits
file_matrix =
out_pole/00000000/100/toast_100_telescope_all_time_all_wcov_inv.fits
file_wcov = out_pole/00000000/100/toast_100_telescope_all_time_all_wcov.fits
binary_output = F
concatenate_binary = F
TOD memory min = 893.55 kB max = 1.15 MB total = 30.72 MB
Baseline memory min = 0.00 B max = 0.00 B total = 0.00 B
Pointing memory min = 1.40 MB max = 2.09 MB total = 51.66 MB
Basis function memory min = 0.00 B max = 0.00 B total = 0.00 B
Clock = 0.013 s
Total submaps = 3072 submap size = 16
Local submaps: min = 4 max = 8 mean = 5.91
Submap table memory min = 384.00 kB max = 384.00 kB total = 12.00 MB
local maps memory min = 6.35 kB max = 12.60 kB total = 298.44 kB
All2allv memory min = 4.00 kB max = 30.25 kB total = 346.75 kB
Map memory min = 156.00 kB max = 156.00 kB total = 4.88 MB
Building pixel matrices...
Counting hits...
Binning TOD...
Clock = 0.050 s
Finalization begins
Clock = 0.050 s
Writing pixel matrix...
0 : Write matrix completed in 0.254 s
Inverting pixel matrices...
Pixel matrix written in out_pole/00000000/100/toast_100_telescope_all_time_all_wcov_inv.fits
147 pixels solved
49005 pixels unsolved
0 pixels had decoupled intensity
0 : Invert pixelmatrix_map completed in 0.291 s
Constructing output map...
Destriped map:
Map 1 2 3
Std 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Mean 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Min 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Writing pixel matrix...
Max 0.00000E+00 K 0.00000E+00 K 0.00000E+00 K
Writing hits...
Pixel matrix written in out_pole/00000000/100/toast_100_telescope_all_time_all_wcov.fits
Hit count written in out_pole/00000000/100/toast_100_telescope_all_time_all_hmap.fits
Clock = 0.900 s
MEMORY (MB):
Detector pointing min = 1.40 MB max = 2.09 MB total = 51.66 MB
TOD buffer min = 893.55 kB max = 1.15 MB total = 30.72 MB
Maps min = 156.00 kB max = 156.00 kB total = 4.88 MB
Baselines min = 2.44 kB max = 2.44 kB total = 78.16 kB
Basis functions min = 0.00 B max = 0.00 B total = 0.00 B
Noise filter min = 0.00 B max = 0.00 B total = 0.00 B
Preconditioner min = 0.00 B max = 0.00 B total = 0.00 B
Submap table min = 384.00 kB max = 384.00 kB total = 12.00 MB
Temporary maps min = 6.35 kB max = 12.60 kB total = 298.44 kB
All2All buffers min = 4.00 kB max = 30.25 kB total = 346.75 kB
CG work space min = 0.00 B max = 0.00 B total = 0.00 B
NCM min = 0.00 B max = 0.00 B total = 0.00 B
Total min = 2.81 MB max = 3.81 MB total = 99.96 MB
WALL-CLOCK TIME (s):
Inverting pixel matrices mean = 0.3 min = 0.3 max = 0.3
Finalization and output mean = 0.9 min = 0.9 max = 0.9
Total 0.9 s ( 0.03 CPU hours)
TOAST INFO: Unstage signal 1 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 1 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 1 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage data 1 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 2 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 2 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 2 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage data 2 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 3 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 3 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 3 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 3 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 4 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 4 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 4 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage data 4 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 5 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 5 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 5 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 5 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 6 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 6 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 6 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage data 6 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 7 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 7 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 7 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 7 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 8 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 8 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 8 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage data 8 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 9 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 9 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 9 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 9 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 10 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 10 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 10 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 10 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 11 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 11 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 12 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 12 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 13 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 13 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 14 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 14 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 15 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 15 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 16 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 16 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 17 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 17 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 18 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 18 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 19 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 19 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 20 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 20 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 21 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 21 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 22 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 22 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 23 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 23 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 24 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 24 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 25 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 25 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 26 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 26 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 26 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 27 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 27 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 28 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 28 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 29 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 29 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixel weights 30 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 30 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 31 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 31 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 31 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage signal 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage pixels 32 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage pixel weights 32 / 32: 0.00 seconds (1 calls)
TOAST INFO: Unstage data 32 / 32: 0.01 seconds (1 calls)
TOAST INFO: Unstage all data: 0.30 seconds (1 calls)
TOAST INFO: Mapping toast_100_telescope_all_time_all: 2.09 seconds (1 calls)
TOAST INFO: Madam total: 2.10 seconds (1 calls)
TOAST INFO: Gather and dump timing info: 0.02 seconds (1 calls)
TOAST INFO: toast_ground_sim.py: 5.62 seconds (1 calls)
Out[16]:
0
In [17]:
hits = healpy.read_map("out_pole/00000000/100/toast_100_telescope_all_time_all_hmap.fits")
hits[hits == 0] = healpy.UNSEEN
healpy.mollview(hits, unit="hits", title="Total hits, Pole")
healpy.graticule(22.5, verbose=False)
NSIDE = 64 ORDERING = NESTED in fits file INDXSCHM = IMPLICIT Ordering converted to RING
