Ground observing schedules¶
In this notebook we learn about creating ground observing schedules.
# Load common tools for all lessons
import sys
sys.path.insert(0, "..")
from lesson_tools import (
fake_focalplane
)
# Capture C++ output in the jupyter cells
%reload_ext wurlitzer
First schedule¶
TOAST pipelines include a tool called toast_ground_schedule.py, also known as the opportunistic scheduler. It builds observing schedules heuristically by building a list of available targets and scheduling and always choosing the highest priority target. toast_ground_schedule.py can be used to create site-specific observing schedules subject to a number of constraints. At the minimum, the tool needs the location of the observatory, observing window and at least one target. Here is a minimal example:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-01 12:00:00" \
--patch-coord C \
--patch small_patch,1,40,-40,44,-44 \
--out schedule.txt
Let's look at the contents of the schedule file.
! cat schedule.txt
The rectangular patch definition takes the form --patch <name>,<priority>,<RA left>,<DEC top>,<RA right>,<DEC bottom>. No spaces are allowed in the definition. Other patch definition formats will be discussed below.
The start and stop times are given in UTC.
The resulting schedule is a plain ASCII file. The header defines the telescope and each line after that defines a constant elevation scan (CES) with a fixed azimuth range. When a full pass of the target takes longer than allowed observation time, --ces-max-time, the CES is broken up into sub passes that use the same observing elevation but adjust the azimuth range. The above schedule includes 10 passes of the target "small_patch" that fit in the given 12-hour observing window. Some passes are split into as many as 4 sub passes, each no longer than 20 minutes (default).
Second patch¶
Let's add another patch, this time using the circular patch definition format, set the observing elevation limits and enable Sun avoidance. We'll also increase ces-max-time so we get fewer entries in the schedule. The circular patch format is
--patch <name>,<priority>,<RA>,<DEC>,<radius>
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-04 00:00:00" \
--patch-coord C \
--patch small_patch,1,80,-13,10 \
--patch large_patch,1,80,-33,20 \
--el-min 30 \
--el-max 60 \
--ces-max-time 86400 \
--sun-avoidance-angle 20 \
--out schedule.txt \
--debug
! cat schedule.txt
Note that we added the --debug option to the command line. This produces a helpful diagnostic plot, patches.png, that shows the locations of your patches, the Sun, the Moon and their avoidance areas. The plot is shown below. The motion of the Moon is already apparent in this 3-day schedule. The Sun (on the right) is effectively stationary. --debug can be expensive, especially if you have lots of patches or request a long observing schedule.
from IPython.display import Image
Image("patches.png")
We deliberately chose the locations of the patches so that they compete over the observing time. This allows us to point out some advanced features of the scheduler. If you examine the very end of the observing schedule, you can note that both small_patch and large_patch were observed 7 times. Given that large_patch is twice as wide and only takes twice as long to observe, equal number of observations actually implies that large_patch will end up with half as many hits per sky pixel.
Patch priority¶
The scheduler offers two ways to remedy this issue. First, one can simply increase the priority of the large patch to dedicate more observing time to it. All things being equal, the number of visits to a given patch is inversely proportional to the priority in the patch definition:
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-04 00:00:00" \
--patch-coord C \
--patch small_patch,1,80,-13,10 \
--patch large_patch,0.5,80,-33,20 \
--el-min 30 \
--el-max 60 \
--ces-max-time 86400 \
--sun-avoidance-angle 20 \
--out schedule.txt
! cat schedule.txt
Now the large patch is observed 9 times and the small patch is observed 4 times.
Equalize area and time¶
Typically we do not use the priority field to normalize the depths. Instead, the user can balance the integration depths with two command line arguments: --equalize-area and --equalize time.
With --equalize-area the scheduler will automatically modulate the user-given priorities with the area of each patch.
With --equalize-time the scheduler will balance the actual time spent in each patch rather than the number of visits. There is a difference, because the observing time per pass can vary greatly depending on the patch shape and orientation
! toast_ground_schedule.py \
--site-lat "-22.958064" \
--site-lon "-67.786222" \
--site-alt 5200 \
--site-name Atacama \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-04 00:00:00" \
--patch-coord C \
--patch small_patch,1,80,-13,10 \
--patch large_patch,1,80,-33,20 \
--el-min 30 \
--el-max 60 \
--ces-max-time 86400 \
--sun-avoidance-angle 20 \
--equalize-area \
--equalize-time \
--out schedule.txt
! cat schedule.txt
As with the by-hand-modulated priorities, large_patch ends up with twice as many visits.
Binning the schedule¶
We take an observing schedule from toast_ground_sim.py and translate it into a depth map.
First, we need a focalplane. If one does not already exist, TOAST pipelines includes a tool for generating mock hexagonal focalplanes:
! toast_fake_focalplane.py --help
Now we create a focalplane with 10-degree FOV and a mininimum of 20 pixels:
! toast_fake_focalplane.py \
--minpix 20 \
--out focalplane \
--fwhm 30 \
--fov 10 \
--psd_fknee 5e-2 \
--psd_NET 1e-3 \
--psd_alpha 1 \
--psd_fmin 1e-5
The actual focalplane ends up having 37 pixels, instead of the minimum of 20. This is because regular packing of the hexagon is quantized. Notice that the final name of the focalplane is focalplane_37.pkl. We'll need the name to run the simulation script.
We will use the versatile ground simulation pipeline, toast_ground_sim.py, to bin the map. It will be covered in detail in lesson 7 so here we simply write out a parameter file:
%%writefile bin_schedule.par
--sample-rate
10.0
--scan-rate
0.3
--scan-accel
10.0
--nside
64
--focalplane
focalplane_37.pkl
--schedule
schedule.txt
--out
out
--simulate-noise
--freq
100
--no-destripe
--no-binmap
--hits
--wcov
Then run the pipeline. Because the pipeline uses libMadam, an MPI code, we must submit the job to a compute node.
import subprocess as sp
runcom = "toast_ground_sim.py @bin_schedule.par"
print(runcom, flush=True)
sp.check_call(runcom, stderr=sp.STDOUT, shell=True)
Let's examine the resulting hits and depth map. The file naming convention may seem a little awkward but follows from the fact that a single run of toast_ground_sim.py may map multiple telescopes, frequencies and time splits.
import matplotlib.pyplot as plt
%matplotlib inline
import healpy
hits = healpy.read_map("out/00000000/100/toast_100_telescope_all_time_all_hmap.fits")
hits[hits == 0] = healpy.UNSEEN
healpy.mollview(hits, unit="hits", title="Total hits")
healpy.graticule(22.5, verbose=False)
wcov = healpy.read_map("out/00000000/100/toast_100_telescope_all_time_all_wcov.fits")
wcov *= 1e12 # from K^2 to uK^2
wcov[wcov == 0] = healpy.UNSEEN
healpy.mollview(wcov, unit="$\mu$K$^2$", title="White noise variance", min=1e0, max=1e3)
healpy.graticule(22.5, verbose=False)
Advanced topics in ground scheduling¶
Cooler cycle format¶
it is possible to instruct the scheduler to add regular breaks in the schedule to cycle the cooler or to perform other maintenance activities. The cooler cycle is a pseudo patch that the scheduler considers like other targets when deciding what to observe next. The full syntax is:
--patch <name>,COOLER,<weight>,<power>,<hold_time_min>,<hold_time_max>,<cycle_time>,<az>,<el>
All of the time arguments are given in hours. The priority of the patch depends on the time since the last cycle occurred. It is infinity until hold_time_min has elapsed and then begins to decrease according to a power law set by power. Priority at hold_time_max is zero.
Planet scans¶
The scheduler can target planets just like stationary patches. The SSO (solar system object) format is
--patch <name>,SSO,<priority>,<radius [deg]>
All orbiting bodies recognized by pyEphem are supported.
Oscillating patches¶
The scheduler designs the scans so that the azimuth range is kept fixed and the boresight sweeps the entire patch. This usually implies a certain amount of spillover integration time outside the patch. This can produce an excess of hits at the boundary of two patches. The scheduler offers a way to smear the spillover by systematically shifting the position of the patches in RA and DEC. The arguments to accomplish this are
--ra-period <period [visits]>
--ra-amplitude <amplitude [deg]>
--dec-period <period [visits]>
--dec-amplitude <amplitude [deg]>
Patches will systematically shift after each visit, returning to their fiducial positions after each period.
Horizontal (high cadence) patch definition¶
Horizontal patch definition specifies the observing elevation and the azimuth range. The scheduler parks the telescope at the given elevation and scans until the constraints (Sun, Moon, cooler hold time) prevent continuing. If possible, scanning is continued by switching between rising and setting scan.
--patch <name>,HORIZONTAL,<priority>,<az min [deg]>,<az max [deg]>,<el [deg]>,<scan time [min]>
Polygon patch definition¶
Patches do not need to be rectangular or circular. An arbitrary polygon shape can be specified by giving the corner coordinates.
--patch <name>,<priority>,<RA_0 [deg]>,<DEC_0 [deg]>,...,,<RA_N-1 [deg]>,<DEC_N-1 [deg]>
Elevation penalty¶
Lower observing elevations are subject to higher levels of photon noise from the atmosphere. It is possible to instruct the scheduler to modulate the relative priorities of the available patches based on their elevation.
--elevation-penalty-limit <elevation [deg]>
--elevation-penalty-power <power>
If the available patch is below elevation-penalty-limit, the priority is modulated by $\left(\frac{limit}{elevation}\right)^{power}$. This way low elevation scans are reserved for targets that cannot be observed at higher elevation or when no targets are available higher.
Block-out¶
January and February weather in the Atacama is known to be problematic for observing. It is possible to instruct the scheduler to skip certain periods of the calendar year with
--block-out <start month>/<start day>-<end month>/<end day>
or with
--block-out <start year>/<start month>/<start day>-<end year>/<end month>/<end day>
All fields are integers. The dates are in UTC.
Gaps¶
The are two short gap lengths in the scheduler
--gap-small <gap [s]>
--gap <gap [s]>
The gap-small is applied when a single CES is broken up into sub scans. The regular gap is applied between separate observations.
Pole scheduling¶
Observing from the Poles is unlike anywhere else on Earth. Patches will not drift across a constant elevation line. Instead, the telescope must be stepped in elevation. The Pole scheduling mode is enabled with
--pole-mode
And the step time and size are controlled with
--pole-el-step <step [deg]>
--pole-ces-time <time [s]>
# --site-lat "-89:59.464" \
# --site-lon "-44:39" \
! toast_ground_schedule.py \
--site-lat "-89.991" \
--site-lon "-44.65" \
--site-alt 2843 \
--site-name South_Pole \
--telescope LAT \
--start "2020-01-01 00:00:00" \
--stop "2020-01-01 12:00:00" \
--patch-coord C \
--patch small_patch,1,40,-40,44,-44 \
--pole-mode \
--pole-el-step 0.25 \
--pole-ces-time 600 \
--out pole_schedule.txt
The resulting schedule has each pass of the target split into (0.25$^\circ$, 10min) steps. It takes 16 steps (2:40h) to cover the 4$^\circ\times$4$^\circ$ degree field.
! cat pole_schedule.txt
Let's bin this schedule as well. We also demonstrate how parameters in the parameter file may be overridden
runcom = "toast_ground_sim.py @bin_schedule.par --schedule pole_schedule.txt --out out_pole"
print(runcom, flush=True)
sp.check_call(runcom, stderr=sp.STDOUT, shell=True)
hits = healpy.read_map("out_pole/00000000/100/toast_100_telescope_all_time_all_hmap.fits")
hits[hits == 0] = healpy.UNSEEN
healpy.mollview(hits, unit="hits", title="Total hits, Pole")
healpy.graticule(22.5, verbose=False)
Exercises¶
- Go back to the "Second patch" section of the notebook and add a third patch of your choosing. Executing that cell will visualize the patch on the sky.
- In the same section, increase the observation length to one month, this will cause the Moon avoidance region to draw a band around the Equator.
- In "Binning the schedule", try increasing the FOV from 10 degrees to 35, representantive of modern SATs