so3g · sotodlib · design note
This note describes how instrument-centred coordinates were added to the so3g projection engine and to sotodlib's P.for_tod, and how sidelobe cuts for the Moon and Sun now use them. It goes through the quaternion chain, the one change needed in the C++ kernel, how roll and polarization angle are handled, the conventions of the resulting maps, and what has been checked on real LAT data.
The previous sidelobe code (coords/sidelobes.py) used object-centred coordinates. A fixed rotation, evaluated at mid-observation, moved the Moon to (0,0) with the zenith "up". Each detector's pointing was then expressed in that frame. That answers the question "where was the telescope looking when it saw the object?"
Instrument-centred coordinates answer the question we actually care about: "where was the object, relative to the detector, when it was seen?" A sidelobe is a feature of the instrument. In a frame attached to the instrument it stays in one place, while in the object-centred frame it is smeared by the changing orientation of the instrument relative to the object. The extreme case is the Moon at the zenith with a thin vertical sidelobe. Every azimuth of a constant-elevation scan sees the Moon through that sidelobe, so the object-centred map shows a ring (and a disk after several elevations) instead of a line.
so3g quaternions are in (1, i, j, k) order. A rotation qB<A takes coordinates in frame A to frame B, and products chain as qC<A = qC<B qB<A. A pointing is where the local ẑ axis ends up. For the usual celestial projection the kernel evaluates, per detector d and sample t:
Here qbore is CelestialSightLine.Q, which for real data is qpoint.azel2bore(az, el, …) · Rz(π + roll), and qdet = rotation_xieta(ξ, η, γ). The object-centred version adds a constant qmoon⁻¹ on the left, which keeps the same form.
For instrument-centred coordinates the roles of detector and object swap. We want the object seen from each detector:
Read right to left:
qsrc[t] = rotation_lonlat(ra, dec) puts ẑ on the source.qbore⁻¹ · qsrc is the source in focal-plane coordinates, with the boresight at the pole.qdet⁻¹ · (…) re-centres on the detector, so every detector sits at the pole and the map position is the source's offset from it.qto_eq = Ry(π/2) = rotation_lonlat(0, 0) moves the pole to (lon, lat) = (0, 0), so a CAR map can be used without the pole singularity.Only the product qbore⁻¹ · qsrc enters. If both are expressed in celestial coordinates through a common horizon-to-celestial rotation H, then (H qb)⁻¹ (H qs) = qb⁻¹ qs, so H cancels exactly. The implementation therefore reuses the standard, full-precision CelestialSightLine.az_el boresight together with the source's RA/Dec, and never forms horizon quaternions itself.
Every so3g projection routine is built around one product, computed inside the C++ loop for each (detector, sample):
quatd qdet = (*qbore) * (*qofs); // q_bore[t] · q_ofs[d]
The per-detector factor is always on the right. In the instrument-centred chain the per-detector factor qdet⁻¹ is left of the per-sample one, so the chain doesn't fit f(t) · qdet. Nothing is more expensive; the ordering just doesn't match what the engine does.
After grouping, the chain always has the form A[d] · X[t]. Two ways to fit that into so3g were considered:
A[d] as the "boresight" and X[t] as the "focal plane". This needs no C++ change, but every output comes back as [nsamp, ndet]. Signal, cuts, thread ranges and responsivities would all have to be transposed to match. Rejected.qofs[d] · qbore[t]. The data layout stays [ndet, nsamp], and every existing feature (all projections, tiling, bilinear interpolation, HEALPix, threads, cuts, weights) works unchanged. This is what was implemented.Orange: per-detector quaternions (the FocalPlane). Teal: per-sample quaternions (the CelestialSightLine). With det_left the engine receives the same two buffers and only reverses the multiplication.
src/Projection.cxx, include/Projection.h)Pointer<CoordSys> has a new constructor Pointer(bool det_left=false) and a private member _det_left.Pointer::GetQdet(i_time, dofs), reads the boresight quaternion and returns q_ofs * q_bore when _det_left is set, q_bore * q_ofs otherwise.GetCoords specializations that combine quaternions (ProjQuat, ARC, SIN, TAN, ZEA, CEA, CAR) each had an identical copy of the read-and-multiply block. All of them now call GetQdet. The code that decodes the quaternion into projection coordinates and cos/sin ψ is untouched. ProjFlat adds offsets and doesn't need an order.ProjectionEngine has a private _det_left = false with get_det_left/set_det_left. All nine places that construct a Pointer (coords, pixels, pointing_matrix, pixel_ranges, tile_hits, tile_ranges, from_map, to_map, to_weight_map) now pass _det_left.EXPORT_ENGINE exposes it as a read/write Python property, det_left. The HEALPix engines use the same macro, so they have it too. The default is False, so existing behaviour and results are unchanged.python/proj/)Assembly carries a det_left attribute: Assembly.attach(sight, fplane, det_left=False)._ProjectionistBase.get_ProjEng(..., det_left=False) sets the property on the engine it creates._ProjectionistBase._get_pointing(assembly), is used by every projection method (get_pixels, get_pointing_matrix, get_coords, get_planar, to_map, to_weights, from_map, assign_threads, assign_threads_from_map, get_active_tiles, and the HEALPix get_coords). It returns (q_bore, q_det, det_left). The one subtle point is the map's own rotation qNS<CS (the WCS reference point). It must always multiply the leftmost factor, so:
q_native);det_left it is applied to the detector quaternions instead: q_celestial_to_native * fplane.quats.assign_threads(method='domdir') falls back to 'simple' for a det_left Assembly. The dominant-direction algorithm builds its own representative focal plane assuming the normal ordering.P.for_tod(..., instrument_centered=…)P gained a det_left attribute, passed to Assembly.attach in _get_asm. P.for_tod has a new keyword instrument_centered. When it is set to a source name ('moon', 'sun', any name planets.get_source_pos accepts), for_tod:
comps == 'T', that geom is given, that rot is None, and that boresight_equ isn't used, raising ValueError otherwise. Roll is needed, so the boresight must be az/el/roll.CelestialSightLine.az_el (same site, weather and qpoint options as usual), or uses a sight= passed in;helpers.get_fplane (for ξ, η and the responsivities);get_instrument_centered(sight, fp, roll, source, timestamps, site), and returns a P with det_left=True.get_instrument_centered builds the two factors:
roll₀ is the circular mean of the roll. A warning is logged if the roll strays more than 0.1° from it. The responsivities are copied from the original focal plane.
The source position comes from the new planets.get_source_quat(name, timestamps, site, dt=600). It evaluates get_source_pos (skyfield, topocentric for the given site) on a grid with at most 600 s spacing, unwraps RA, interpolates linearly to every sample, and returns rotation_lonlat(ra, dec). For the Moon the linear-interpolation error at 600 s is well below an arcsecond. A one-hour observation needs only about seven ephemeris evaluations.
The boresight quaternion factors as qbore = qtel · Rz(roll). Here qtel is the orientation of the telescope structure (qpoint's az/el part, together with so3g's fixed Rz(π)) and Rz(roll) turns the focal plane relative to it. The chain without any roll handling gives a frame that rotates with the focal plane. The sidelobes don't rotate with roll, so the roll is removed as proposed: qto_eq is replaced by qto_eq · Rz(roll). Substituting qbore shows what that does:
So the roll correction amounts to conjugating each detector quaternion by the roll. Rz(roll)·qdet moves the detector's (ξ, η) into telescope-structure coordinates. The trailing Rz(−roll) aligns the detector frame's local axes with the telescope axes instead of the rotated focal-plane axes. The result is the source relative to each detector, on axes fixed to the telescope structure.
For this to be a per-detector constant, the conjugation uses the mean roll roll₀, while X[t] keeps the exact per-sample Rz(roll[t]). With constant roll (a SAT at fixed boresight rotation, or a LAT constant-elevation scan with the corotator fixed) the result is exact. If roll varies, only the rotation of the small detector offsets is approximated; the telescope-frame position of the source stays exact.
rotation_xieta(ξ, η, γ) = Rz(φ)·Ry(θ)·Rz(γ−φ) has its local axes rotated by the polarization angle γ. In the normal chain that is the point: the resulting ψ is the polarization angle on the sky. With qdet⁻¹ on the left, though, the final Rz(φ−γ) would rotate each detector's whole map about the detector by −γ. Detectors with different γ would then disagree on where the sidelobe is. The detector quaternions are therefore rebuilt with γ = 0 from the decomposed ξ, η, and the mode only allows comps='T'. The HWP sign flip on γ doesn't matter here for the same reason.
cdelt[0]) the usual image orientation therefore has right = +az and up = +el.qNS<CS) is applied on the left of the detector factor. A full-sky CAR mask with crval = (0,0) is the natural choice, and for it that rotation is the identity.coords/sidelobes.py)The masks are no longer fixed files per object. They come with the observation metadata, so each tube or wafer gets its own. The existing Enmap metadata loader wraps the map in an AxisManager as aman.sidelobes.map, a pixell enmap of shape [nfields, ny, nx], with a fields LabelAxis naming each plane, e.g. ['moon', 'sun']. Each plane is a 0/1 mask in the instrument-centred coordinates of that object.
get_cuts(aman, sidelobes="sidelobes", object_list=None, rise_tol=1°):
aman.sidelobes is missing or its map has no WCS;object_list selects a subset;aman.site/aman.weather (defaults 'so'/'typical');RangesMatrix [ndet, nsamp] per object, in the order requested.For each object, make_cuts:
planets.get_source_azel at the first, middle and last sample, and returns empty cuts if it is below rise_tol at all three;P.for_tod(aman, comps='T', geom=mask geometry, instrument_centered=name, threads=False);astype(int) could round a 1 down to 0 for T < 1);from_map (nearest neighbour) and cuts every sample where it is non-zero: RangesMatrix.from_mask(tod != 0).In site_pipeline/make_ml_map.py the call is now sidelobes.get_cuts(obs), on the fully loaded observation before downsampling, and the result is added to glitch_flags as before. The --sun-mask and --moon-mask options are no longer used.
test_40_det_left_engine: the raw engine output, with det_left off and on, is compared with numpy's q_bore·q_ofs and q_ofs·q_bore.test_41_det_left_projectionist: a det_left Assembly must give exactly the same pixels, coordinates, to_map and from_map results as an equivalent normal Assembly built one detector at a time. This is checked on tiled and untiled TAN maps, where qNS<CS isn't the identity, and with both thread-assignment methods.test_pmat_instrument_centered:
from_map/to_map run end to end;comps='TQU' is rejected.A single-wafer LAT observation, 35.4 min long, with commanded az 110.7°–167.8°, commanded el 120° (past the zenith) and roll 60.00° (constant to 0.001°). The Moon moved from az 0.9°, el 48.8° to az 348.5°, el 48.0°.
Spread across detectors at a fixed time should match the wafer size:
| Sample | Spread in lon | Spread in lat | Median (lon, lat) |
|---|---|---|---|
| 0 | 0.61° | 0.42° | (+36.37°, −10.05°) |
| 106 264 | 0.60° | 0.51° | (+15.33°, +6.61°) |
| 212 528 | 0.60° | 0.52° | (+7.04°, +9.28°) |
| 318 792 | 0.60° | 0.53° | (+1.60°, +10.30°) |
| 425 057 | 0.60° | 0.52° | (+6.27°, +10.09°) |
The spread is constant and about the size of a wafer. The large motion of the median follows the scan and the Moon, including the axis flip past the zenith: near the Moon lat ≈ +10°, where spherical trigonometry predicts about +11° for the boresight.
Exact orientation check for the boresight. A single detector with A = Ry(π/2) sits exactly at the boresight. For it, sin(lat) must equal the Moon's direction projected on the telescope's elevation axis, m·ûel, and cos(lat)·sin(lon) must equal −m·ûaz, using the commanded boresight az/el and skyfield's apparent Moon az/el:
| Sample | sin(lat) | m·ûel | cos(lat)·sin(lon) | −m·ûaz |
|---|---|---|---|---|
| 0 | −0.1575 | −0.1574 | +0.6077 | +0.6077 |
| 60 722 | −0.0015 | −0.0014 | +0.4962 | +0.4962 |
| 121 444 | −0.0771 | −0.0771 | +0.5617 | +0.5616 |
| 182 167 | −0.1247 | −0.1246 | +0.5936 | +0.5935 |
| 242 889 | −0.0142 | −0.0141 | +0.5148 | +0.5147 |
| 303 612 | +0.0798 | +0.0799 | +0.4084 | +0.4084 |
| 364 334 | +0.1511 | +0.1512 | +0.2805 | +0.2804 |
| 425 057 | +0.1963 | +0.1964 | +0.1372 | +0.1371 |
They agree to about 1e-4 across the whole scan, at el = 120° and roll = 60°. The residual matches refraction, which is included in the az_el boresight but not in skyfield's az/el. This confirms the full chain through the reversed kernel product, the axis convention and the roll handling: a wrong roll sign would have shown up as a disagreement of about 60°.
Pipeline: make_ml_map.py with the metadata masks has been run on observations known to have Moon and Sun sidelobe pickup, and appears to work.
Rz(roll₀)·qdet⁻¹·Rz(−roll₀) factor, because it is the identity at the boresight. Its size is confirmed (wafer spread, and the distance invariant in the unit test). Whether the wafer is turned by +roll or −roll hasn't been checked independently. For a ~1° LAT wafer the effect is at most about 1°. A check using samples close to the Moon (detector minus boresight position should equal the roll-rotated −(ξ, η)) would settle it.comps='T' is allowed. Polarized instrument-centred maps would need a separate definition of the polarization angle.domdir falls back to simple in this mode. It doesn't matter for cuts, since from_map needs no thread assignment.sidelobes metadata is missing, get_cuts raises KeyError, which make_ml_map doesn't catch, so the run stops. Use --no-sidelobe for data without masks.get_coords returns float64 [ndet, nsamp, 4], which is tens of GB for a full LAT wafer and observation. The cuts themselves only use a float32 [ndet, nsamp] array.include/Projection.h, src/Projection.cxx, python/proj/coords.py, python/proj/wcs.py, test/test_proj_eng.pysotodlib/coords/pmat.py, sotodlib/coords/planets.py, sotodlib/coords/sidelobes.py, sotodlib/site_pipeline/make_ml_map.py, tests/test_pmat.pyMinimal usage:
from sotodlib import coords
# Pointing matrix in Moon instrument-centred coordinates
P = coords.pmat.P.for_tod(obs, comps='T', geom=(shape, wcs), instrument_centered='moon')
tod = P.from_map(mask) # mask: enmap [1, ny, nx]
# Cuts from the per-wafer masks in obs.sidelobes
cuts_moon, cuts_sun = coords.sidelobes.get_cuts(obs) # order of obs.sidelobes.fields