APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Friday May 10, 2002 (MJD 52405) ---=== OBSERVING TEAM ===--- Swing/night: Mike Harvanek, Pete Newman ---=== OBSERVING PLAN ===--- Science with standard priorities. ---=== OBSERVING SUMMARY ===--- Variable cloud all night, so spectroscopy. Two plates finished (820, 954). Good data on three more. Continued variable smears. Minor software problems probably self-inflicted with recovery by the book. ---=== OBSERVING LOG ===--- Pre-flight checks: SoS reported that test spectrograph flat and arc exposures with cartridge 5 mounted had Xsigma = 1.11 for r2 (flagged red) and Xsigma = 0.95 for r2 (flagged yellow), at enclosure temperature 14.4C (external air temp 17.1C). All wavelength ranges were in spec. See the last section of this log for details of adjustments made. Night log: Thin high clouds from afternoon continuing through sunset, so spectroscopy when we could, logged to clouds when too thick to get photons. SoS died around 04:45Z (perhaps earlier, this is when we noticed). See problem section #1 for the recovery. No science time lost. We saw both good and bad smears tonight but still do not understand why. Both plate 954 (good smear) and plate 919 (bad smear) were taken under very similar cloud conditions. We think the fiber positions on plate 820 look very strange, forming diagonal strips across the plate judging fromthe SoS S/N plots. Steve Kent emailed for advice. From now on, this observer is not going to write "guide stars on gotoField" any more as this is the default state. Watch for exciting "NO guide stars" messages instead ;-) Observing sequence: 02:26Z Plate 946, cartridge 5. Not done in 6 exposures even with existing (S/N)^2 >= 2.5 from MJD 52400 before running off the design time. Two exposures useful, 4 useless because of cloud. Two showed bad exposure times (PR3402). postCals taken because SoS was still catching up. Plugging for this plate technically expires today, but we recommend holding it at least one more day to see if it can be finished. No smear yet. 05:37Z Plate 954, cartridge 2. Done in 3 exposures. Clouds thinner but variable so we over-estimated the required exposure time a little. postCalibs done though not required for science exposures but SoS had not caught up. Smear values all in black from SoS. 07:11Z Plate 919, cartridge 4. Not done in 3 exposures, but useful data before we ran out of time on this plate. Smear values all in red or yellow. postCalibs called for on the last science exposure but we missed the warning from SoS before we changed cartridge. 08:46Z Plate 924, cartridge 3. Not done in 2 exposures before we ran out of time on this plate. Smear good. 09:43Z Plate 820, cartridge 1. Done in 3 exposures, plus (S/N)^2 >= 5.3 from MJD 52404. Tonight's smear taken under better conditions than last night, although not photometric. Even so, SoS reported it as bad S/N. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 3190 23:15Z 23:42Z 100 O -73.28 -66.53 58 ignore ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- ---=== LTMATCH RESULTS ===--- Run Field nFields nGood rowMean rowSig colMean colSig rot --------------------------------------------------------------- ---=== SPECTROSCOPY DATA SUMMARY ===--- UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 14969, plate -9999 ------- 23:17 14969 0.0 bias ----- sequence 14970, plate 9999 ------- 00:35 14970 10.0 flat r2 Xsigma out of spec 00:38 14971 2.0 arc ----- sequence 14970, plate 820 ------- 00:57 14972 3.0 arc Hartman exposures 00:59 14973 3.0 arc 01:18 14974 3.0 arc Hartman exposures post-changes 01:21 14975 3.0 arc ----- sequence 14970, plate 9999 ------- 01:34 14976 10.0 flat All X/Wsigma values in spec 01:37 14977 2.0 arc ----- sequence 14978, plate -9999 ------- 02:05 14978 0.0 bias ----- sequence 14979, plate 946 ------- 7.6 3.4 7.0 3.6 NOT DONE 03:04 14979 10.0 flat 03:06 14980 2.0 arc 03:31 14981 1201.0 target 03:54 14982 22065.0 target USELESS - marked bad 04:16 14983 1200.0 target USELESS - marked bad 04:39 14984 1200.0 target USELESS - marked bad 05:01 14985 23536.0 target USELESS - marked bad 05:25 14986 1200.0 target 05:28 14987 10.0 flat 05:30 14988 2.0 arc ----- sequence 14989, plate 954 ------- 23.9 22.2 22.9 18.3 DONE 05:52 14989 10.0 flat 05:54 14990 2.0 arc 06:18 14991 1200.0 target 06:36 14992 900.0 target 06:55 14993 902.0 target 07:02 14994 240.0 smear GOOD SMEAR 07:05 14995 10.0 flat 07:07 14996 2.0 arc ----- sequence 14997, plate 919 ------- 13.5 10.8 14.7 13.2 NOT DONE 07:20 14997 10.0 flat 07:22 14998 2.0 arc 07:47 14999 1202.0 target 08:10 15000 1200.0 target 08:33 15001 1201.0 target 08:41 15002 240.0 smear POOR SMEAR? ----- sequence 15003, plate 924 ------- 7.8 7.8 7.3 9.3 NOT DONE 08:53 15003 10.0 flat 08:55 15004 2.0 arc 09:14 15005 900.0 target 09:31 15006 901.0 target 09:38 15007 240.0 smear ----- sequence 15008, plate 820 ------- 14.5 17.5 15.3 15.2 DONE (plus 52404) 09:52 15008 10.0 flat 09:54 15009 2.0 arc 10:14 15010 900.0 target 10:32 15011 900.0 target 10:39 15012 240.0 smear 10:57 15013 900.0 target NB This last exposure is not required to complete (S/N)^2 > 15 for the plate, and may have required postCalibs. If it does not reduce well, reject it. ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 02:46Z 5 946 194.22 -0.0008 65.47 0.0023 21.81 -0.0218 1.000200 06:52Z 2 954 213.23 0.0016 50.08 0.0010 67.38 -0.0130 1.000120 07:30Z 4 919 -27.69 0.0014 51.72 0.0015 156.93 -0.0197 1.000040 08:58Z 3 924 -41.84 0.0015 47.09 0.0007 145.81 -0.0190 1.000070 10:06Z 1 820 252.43 0.0020 72.86 0.0019 84.05 -0.0016 1.000190 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL5334 Stays: JL5335 ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 02:45Z 5 946 1.00020 -1974 -775 -280 194 65.5 15.0 24 215 BG38 1.7 06:52Z 2 954 1.00012 -1184 0 -200 213 50.1 13.0 16 267 BG38 1.7 07:30Z 4 919 1.00004 -394 649 -200 -28 51.7 13.1 0 192 BG38 1.7 08:58Z 3 924 1.00007 -690 438 -180 -42 47.1 11.4 15 264 BG38 1.8 10:06Z 1 820 1.00019 -1875 -606 -180 252 72.9 10.7 18 267 BG38 1.8 ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 23:00Z 65 1 23 215 (SW) 1449 - Variable cloud 23:32Z 66 0 26 216 (SW) 1418 - " 00:05Z 64 0 26 224 (SW) 1423 - " 00:37Z 63 1 20 214 (SW) 1898 - " 01:11Z 60 0 21 235 (SW) 2192 - " 01:42Z 59 0 18 212 (SSW) 2305 - " 02:13Z 59 0 21 206 (SSW) 2923 - " 02:43Z 58 0 17 223 (SW) 2394 - " 03:14Z 58 0 19 226 (SW) 1945 - " 03:44Z 58 0 14 244 (WSW) 1897 - " 04:15Z 57 1 14 256 (WSW) 1533 - " 04:45Z 57 1 17 235 (SW) 1322 - " 05:17Z 57 2 13 262 (W) 1258 - " 05:47Z 56 3 23 251 (WSW) 1322 - " 06:20Z 55 5 12 269 (W) 1337 - " 06:52Z 55 4 15 267 (W) 1364 - " 07:23Z 55 5 15 258 (WSW) 1227 - " 07:56Z 54 5 18 262 (W) 1296 - " 08:28Z 52 6 15 251 (WSW) 1382 - " 09:00Z 52 8 20 265 (W) 1453 - " 09:30Z 52 9 16 261 (W) 1499 - " 10:02Z 51 9 18 257 (WSW) 1515 - " 10:33Z 50 8 14 258 (WSW) 1636 - " 11:04Z 50 8 11 277 (W) 1829 - " ---=== TELESCOPE STATUS ===--- 01:50Z Doors open, fans on 02:00Z Enclosure off 11:13Z Enclosure on, fans off Status at 11:14Z: Telescope stowed at: 30 deg Instrument mounted: Cartridge 9 Counterweights at: 280 Autofill systems: On 180L LN2 dewar weights: Spectro 251 lb Imager 126 lb (filled recently) Interlocks bypassed: Slip detection (velocity), building. ---=== SOFTWARE USED ===--- IOP/SOP: v3_78_1 Watcher: v2_17_0 MCP: v5_11_3 TPM: v2_8_0 AstroDa: v14_47 TCC: 2.6.2 sdssProcedures: v1_44 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.0760 0.0760 Axial B 0.7990 0.7990 Axial C 0.0800 0.0810 Trans D -1.5180 -1.5180 Lateral E 1.8330 1.8240 Lateral F 1.4010 1.3920 GALILS Commanded: 5400. -3700. 900. -200. 31550. 30650. Actual: 5400. -3700. 900. -200. 31550. 30650. SETMIR VALUES PriDesOrient: 0.00 -11.80 23.00 1256.90 642.10 PriOrient: 0.00 -12.16 22.81 1257.41 642.19 SECONDARY: ---------- Focus: 0.00 Air Temp.: 15.0 Alt.: 29.986106 MIGS TONIGHT NOMINAL Axial A 1.5480 1.5500 Axial B 1.2200 1.2220 Axial C 1.2130 1.2140 Trans D -0.5800 -0.5770 GALILS Commanded: 1599561. 1565287. 1586370. -10400. -9750. Actual: 1599600. 1565250. 1586350. -10400. -9750. SETMIR VALUES SecDesOrient: 1257.00 -3.40 -10.00 0.00 133.82 SecOrient: 1257.00 -3.41 -9.97 -0.22 133.70 ---=== PROBLEMS IN DETAIL ===--- Problem #1 - SoS died --------------------- SoS apparently died, almost certainly while we were messing with exposure 14982's bad exposure time, as this was the last exposure reduced before it died. Summary: we think SoS's log file got clobbered. Here's how we got it going: [observer@sos ~]$ sos_stop Son-of-Spectro stopped. [observer@sos ~]$ sos_status The Son-of-Spectro cron job is NOT running for USER=observer MACHINE=sos.apo.nms u.edu. The following aporsync jobs are currently running: F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 000 S observer 19292 1 0 60 0 - 412 wait4 22:18 ? 00:00:00 sh /local/ups/Linux+2.2/idlspec2d/v4_9_9/bin/aporsync_blue.sh 000 S observer 19302 1 0 60 0 - 412 wait4 22:18 ? 00:00:00 sh /local/ups/Linux+2.2/idlspec2d/v4_9_9/bin/aporsync_red.sh You may need to kill old jobs if there is a problem. The following startapo jobs are currently running: F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 000 S observer 19299 19292 0 60 0 - 414 wait4 22:18 ? 00:00:00 sh /local/ups/Linux+2.2/idlspec2d/v4_9_9/bin/startapo.sh 000 S observer 19306 19302 0 60 0 - 414 wait4 22:18 ? 00:00:00 sh /local/ups/Linux+2.2/idlspec2d/v4_9_9/bin/startapo.sh You may need to kill old jobs if there is a problem. No lock files exist. [observer@sos ~]$ kill 19292 19302 19299 19306 [observer@sos ~]$ sos_status The Son-of-Spectro cron job is NOT running for USER=observer MACHINE=sos.apo.nms u.edu. No aporsync jobs are currently running. No startapo jobs are currently running. No lock files exist. [observer@sos ~]$ sos_start Setting $RAWDATA_DIR=/data/spectro Setting $ASTROLOG_DIR=/data/spectro/astrolog Setting $SPECLOG_DIR=/data/spectro/astrolog Setting $SPECTROLOG_DIR=/data/spectro/spectrologs Now running Son-of-Spectro version v4_9_9. ...However, that just failed again without reducing any more frames. Next, we tried to remove the bad exposure, 14982, using sos_redo, after again stopping sos: [observer@sos ~]$ sos_redo expnum=14982 Executing: echo "remove2redo,expnum=14982" | idl IDL Version 5.4 (linux x86). (c) 2000, Research Systems, Inc. Installation number: 96636-4. Licensed for use by: Apache Point Observatory, NMSU % Compiled module: REMOVE2REDO. REMOVE2REDO: Selecting MJD=52405 REMOVE2REDO: Trying to lock the logfile /data/spectro/spectrologs/52405/logfile-52405.fits at which point it hung indefinitely. We then followed the trouble shooting procedure at http://sdsshost.apo.nmsu.edu/sdssProcedures/spectroSOS.html#trouble step 6 for a corrupted log file, and restarted all the reductions for the night by removing all the data from sos.apo.nmsu.edu, with: rm -rf /data/spectro/spectrologs/52405 /data/spectro/52405 followed by another sos_start. We had about 15 exposures to re-reduce, so expect this to take about 45 mins to catch up. ---=== SPECTROGRAPHS COLLIMATION CHECK AND ADJUSTMENT ===--- Here is the check and correction of the spectrographs' collimation and focus. The initial flat-field exposure had Xsigma for r2 = 1.11 (flagged red by SoS) and r2 Wsigma = 0.95 (flagged yellow). The enclosure temperature was ~14.2C during the tests (the external ambient temperature was 17C). Progress messages omitted. Positions before adjustment: sp1> mechSend s Coll_motor_A -3475 Coll_motor_B -3475 Coll_motor_C -3475 sp1> sp2 sp2 sp2> mechSend s Coll_motor_A -4947 Coll_motor_B -4948 Coll_motor_C -4549 Collimation measurements in SOP and idlspec2d. Note that the predicted movements from idlspec2d are larger than those from SOP. sp1> goSpecFocus (all worked, progress messages deleted) Seq 14970 Exp 14972 Frame 14972 Seq 14970 Exp 14973 Frame 14973 sp1 red l-r = -0.494 pixels , tolerance = 0.15 pixels, not focused yet Issue: sp1; mechPiston 4520 sp2 red l-r = -0.581 pixels , tolerance = 0.15 pixels, not focused yet Issue: sp2; mechPiston 5316 We compared the changes recommended with those from the idlspec2d code: [observer@sos ~]$ setup idlspec2d [observer@sos ~]$ idl IDL> collimate, 14972 COLLIMATE: Min offset = -1.06 pix COLLIMATE: Max offset = 0.03 pix COLLIMATE: Mean offset = -0.53 +/- 0.30 pix COLLIMATE: Camera r1 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict (red) piston movement of 4826 steps for spectro-1 COLLIMATE: Issue SOP command: "sp1; mechPiston 4826" Log file = Collimate-52405-r1-00014972.log PostScript file = Collimate-52405-r1-00014972.ps COLLIMATE: Min offset = -1.24 pix COLLIMATE: Max offset = -0.28 pix COLLIMATE: Mean offset = -0.61 +/- 0.23 pix COLLIMATE: Camera r2 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict (red) piston movement of 5568 steps for spectro-2 COLLIMATE: Issue SOP command: "sp2; mechPiston 5568" Log file = Collimate-52405-r2-00014972.log PostScript file = Collimate-52405-r2-00014972.ps We followed the recommendations from idlspec2d for the red cameras, and issued: sp1> sp1; mechPiston 4826 sp1> sp2; mechPiston 5568 Positions after adjustment: sp1> mechSend s (non-position lines snipped) Coll_motor_A 1351 Coll_motor_B 1351 Coll_motor_C 1351 sp2> mechSend s Coll_motor_A 622 Coll_motor_B 619 Coll_motor_C 1029 We then ran the Hartmann exposure and calculation sequences again to check. sp1> goSpecFocus Seq 14970 Exp 14974 Frame 14974 Seq 14970 Exp 14975 Frame 14975 sp1 red l-r = -0.093 pixels, tolerance = 0.15 pixels, focused sp2 red l-r = -0.136 pixels, tolerance = 0.15 pixels, focused sp1 blue l-r = 0.074 pixels, tolerance = 0.15 pixels, focused sp2 blue l-r = 0.058 pixels, tolerance = 0.15 pixels, focused IDL> collimate, 14974 COLLIMATE: Camera b1 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Camera b2 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Camera r1 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Camera r2 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) Note that the actual movements from out of focus were ~ 4 times the residual movements in focus. We then took addition flat and arc exposures on plate 9999 to check SoS was reporting all values in spec. They were.