APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Monday February 9, 2004 (MJD 53045) ---=== OBSERVING TEAM ===--- Swing: Dan Long Night: Atsuko Nitta Steph Snedden Support: Dan Long Craig Loomis Connie Rockosi ---=== OBSERVING PLAN ===--- Shakedown - night 1 ---=== OBSERVING SUMMARY ===--- No pointing model because of questions about the center of the engineering camera CCD / instrument block. Will try again tomorrow. Successful shake items addressed: Watcher, IOP guiderMon.tcl change of Connie's, PR 5861, SEGUE IOP. Shake items tested that failed: MOP, IOP scrolling display additions Shake items that need further testing before passing; PR 5864 We are still getting M2 mirror temp warnings in Watcher. ---=== OBSERVING LOG ===--- Afternoon: ---------- There was some trouble getting iop going (sop had no trouble). The usual backplane communications whine. ptvmeReset didn't fix it. Rebooted the crates. The servers didn't come back on the first try even then. A third ptvmeReset seemed to do the trick. Here is the murmur output just before the third reset. 2004-02-09 23:00:11Z sdsshost TEXTONLY dscRun: executing /p/iop/v3_126_0/bin/iop.bin -cont 2004-02-09 23:00:13Z sdsshost v14_47 330 dsc_E_noShareMap Error mapping shared data area (memory) 2004-02-09 23:00:13Z sdsshost v14_47 330 dsc_E_mmapFail mmap failed for /dev/mmem for offset 0xB8000000 (0x00200000 bytes) 2004-02-09 23:00:13Z sdsshost v14_47 330 dsc_E_noShareMap Error mapping shared data area %s 2004-02-09 23:00:13Z sdsshost v14_47 330 dsc_E_MCunavail MC not available 2004-02-09 23:00:32Z sdsshost IDA 14679 TEXTONLY trace 1, 0 long original: {schedule read_output {file14 1} -60} /p at 100% ( 23MB free ). Should we be concerned? No spectrograph focus check since the engineering camera is on. It was checked last night in similar temperatures so it shouldn't be too far off. Mirrors homed and relaxed. All in spec. The laptop sirrobert needed about ten minutes for the screen to warm up enough to become legible after sitting all day in the enclosure with the Thermo-King on. The enclosure temperature was about 24 F. Night: ------ Clouds prevented us from starting the pointing model right away, so we turned our attention to other shake items. M2 Mirror Temps: Jon, we are still seeing M2 temp errors in Watcher, even with the new shielding. Watcher: PLC versions are read correctly as long as the interlocks are started from an IOP session. This works as advertised. IOP: SCD - The "All of r, i, u, z, g" works for r, i, u and z, but fails on "All of g". The stack trace is in the problems section. MOP: The new MOP version fails because it cannot talk to the master server. See the PT log for more info. stty: : Function not implemented 02:307Z - The clouds go away! We open for a pointing model. At first, we saw stars in the saoDisp, but gradually the errors got worse until by the 7th pointing star or so, we were seeing no stars. At first, we suspected the new azimuth fiducial table. We backed out to MCP version v5_24_0 to use the old fiducial tables, but still we lost stars after a few corrections. We also checked that the camera and shutter were still alive after we had seen several exposures without stars. We then called Dan. He suspected the instrument block, and asked us to check and make sure the corrections were moving the stars in the right direction. They were not (details in the problems section). Since no one has changed the instrument block since last shakedown, or worked on either the TCC or engcam that we know of, we are stumped as to why this happened. We next checked the center of the engcam CCD to see if the camera had moved. The center had moved slightly. We tried to home the stage, but couldn't connect. A phone call to Craig reminded us of the obvious...check the serial connections. We did so, and found that the connection to the junction box had not been made when the engcam was mounted during the afternoon... We homed the engcam galil and checked the center again. This time the chip center was even farther from the inst block center (46 unbinned pixels away from the instrument block center). We rehomed, just to double check. The center has indeed changed. Is it possible that the camera has been bumped? Before we go any further with this, we would like the engineering crew to check the mechanical center of the engineering camera, and make sure it is stable and solid. If all is well there, do we need to change the instrument block chip center? A PR was filed. We moved on to other shake tasks. We mounted cartridge 6 to test SOP changes. Connie's guiderMon.tcl change - Connie has written code to look for bad guide fibers from the guider flats. After clarifying the test instructions with Connie, we re-enabled fiber 11 on cartridge 6, turned gGuide on and did a guider flat. Sure enough, the bad fiber was detected and disabled. Excellent. PR 5864 - guider can silently fail if no new guiderFrame found 7.5s after it's due The fix was tested by taking several focus sweeps and waiting for errors. There was a timeout error on the very first sweep, but we saw no more timeout errors after that. We were unable to trigger the error by changing the instrument to imager and taking a guider frame (only the telrun session complained), or by increasing the focus step interval to lengthen the time between exposures. We will continue to test this PR tomorrow night. Atsuko's detailed notes about the first failed sweep are in the problems section. 10:25Z - We mount the imager to test IOP and get donuts. PR 5861 - scanStripe should apply offset after track command: This appears to work. The offset was applied after the slew finished. Here is the telrun session outout: 4 16 : MoveItems="YYYYYYYYY"; SlewEnd 0 0 I TCCStatus="SSS","NNN"; TCCPos=252.59,62.23,-15.49; AxePos=252.59,62.23,-15.49 0 0 I AxisErrCode="OK","OK","OK" 0 0 I CurrArcOff = 0.084025, 0.003987, 4583125693.28665, 0.026412, 0.001253, 4583125693.28665 0 0 I ObjNetPos = 157.296668, 0.004969, 4583125693.28665, 36.609055, 0.001249, 4583125693.28665 0 0 I ObjInstAng = 72.612788, 0.002963, 4583125693.28665 0 0 I SpiderInstAng = -15.253574, 0.000163, 4583125693.28665 0 0 I TCCStatus="TTT","NNN"; TCCPos=252.59,62.23,-15.49; AxePos=252.59,62.23,-15.49 0 0 I AxisErrCode="OK","OK","OK" 0 0 I CurrArcOff = 0.084025, 0.003987, 4583125693.28665, 0.026412, 0.001253, 4583125693.28665 0 0 I ObjNetPos = 157.296668, 0.004969, 4583125693.28665, 36.609055, 0.001249, 4583125693.28665 0 0 I ObjInstAng = 72.612788, 0.002963, 4583125693.28665 0 0 I SpiderInstAng = -15.253574, 0.000163, 4583125693.28665 5 16 > Started; Cmd="offset boresight 0.10483333333333333,0.,0.,0. /PAbsolute /Wait /Computed" SEGUE IOP - s1_6_0: We set up on stripe 205 with a startMu=150. A goDrift was issued, but hung because we forgot to start the proper servers. After fixing that, we started another drift successfully. We then tested the focus loop and skippy. Both worked! We then tried the SEGUE IOP on a standard survey stripe and checked the focus loop and skippy for that stripe as well. All was successful. Since I couldn't find a PR number, I didn't close the PR. During the SEGUE IOP tests, we saw that the seeing had deteriorated, so sorry, Jim...no donuts yet again. Maybe tomorrow night. Don't feel bad. Madonna called to say she can't get donuts for us today, either. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 4438 23:21Z 00:03Z 100 O -161.03 -150.55 83 ignore 4439 01:52Z 02:50Z 100 O -123.33 -108.83 34 ignore 4440 11:06Z 11:12Z 100 O 15.73 17.18 24 ignore 4441 11:20Z 11:34Z 205 N -34.48 -31.03 36 ignore 4442 11:37Z 12:02Z 26 N -21.47 -15.23 56 ignore ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- 4441 17 4 77 -15.900 0.134 -1.000 0.123 -0.01091 279 49 4441 17 4 77 -15.900 0.134 -1.000 0.123 -0.01091 279 49 4442 23 4 55 -14.200 0.117 5.500 0.143 -0.00694 291 49 4442 25 4 55 -14.300 0.152 5.300 0.143 -0.00711 291 49 ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ 4442 22 1 49 249 7 -0.877 0.195 1.891 0.290 -0.00636 4442 22 1 49 249 6 -1.014 0.170 2.112 0.277 -0.00710 4442 22 1 49 249 12 -1.112 0.214 2.132 0.260 -0.00717 4442 22 1 49 249 3 -1.283 0.172 1.968 0.344 -0.00662 4442 22 1 49 249 13 -1.392 0.235 1.926 0.321 -0.00648 4442 33 1 50 248 16 -0.453 0.257 1.714 0.128 -0.00576 4442 33 1 50 248 7 -0.474 0.226 1.777 0.178 -0.00598 4442 33 1 50 248 10 -0.537 0.334 1.725 0.197 -0.00580 4442 33 1 50 248 7 -0.679 0.301 1.800 0.137 -0.00605 4442 33 1 50 248 9 -0.731 0.234 1.745 0.208 -0.00587 ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): n QA Procedures Done (y/n): n UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 24947, plate -9999 ------- 22:45 24947 0.0 bias 00:33 24948 0.0 bias ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL N/A Stays: JL N/A ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 23:53Z 30 4 9 187 (S) -9999 - 00:25Z 30 6 12 212 (SSW) -9999 - 01:06Z 29 6 7 185 (S) -9999 - 01:38Z 29 6 10 185 (S) -9999 - 02:10Z 26 5 12 149 (SSE) -9999 - 02:46Z 26 5 13 144 (SE) -9999 - 03:23Z 26 6 11 163 (SSE) -9999 - 03:55Z 26 6 13 156 (SSE) -9999 - 04:27Z 27 7 10 178 (S) -9999 - 04:59Z 28 7 11 182 (S) -9999 - 05:32Z 27 7 6 181 (S) -9999 - 06:04Z 28 3 6 177 (S) -9999 - 06:40Z 30 4 1 174 (S) -9999 - 07:12Z 29 7 10 179 (S) -9999 - 07:45Z 28 7 1 168 (SSE) -9999 - 08:15Z 27 6 8 181 (S) -9999 - 08:45Z 26 7 10 143 (SE) -9999 - 09:18Z 27 8 9 169 (S) -9999 - 09:50Z 27 8 9 146 (SE) -9999 - 10:22Z 27 9 4 149 (SSE) -9999 - 10:55Z 28 10 6 179 (S) -9999 - 11:25Z 27 9 9 179 (S) -9999 - 11:57Z 27 10 10 147 (SSE) -9999 - 12:30Z 27 11 11 135 (SE) -9999 - ---=== TELESCOPE STATUS ===--- 23:50Z - Fans & blowers on 00:10Z - Doors opened. 02:30Z - Opened enclosure. 12:32Z - Enclosure closed. Imager still mounted. Email sent to Jon B. asking to have it unmounted. Spectro autofill connected. Counterweights at 720. 12:56Z - Spectro dewar weight: 251 12:56Z - Imager dewar weight: 213 ---=== SOFTWARE USED ===--- IOP/SOP: v3_127_0 Watcher: v2_29_0 MCP: v5_25_0 TPM: tpm_v2_37_0 AstroDa: v14_47 TCC: TCC 2.7.1 July 22 2003 sdssProcedures: v1_84 SoS: v4_10_7 hoggPT: v1_6_9 plate-mapper: v4_3_1 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.5380 0.5420 Axial B 0.6310 0.6390 Axial C 1.0930 1.0900 Trans D -8.4530 -8.4610 Lateral E 10.9093 10.8966 Lateral F 10.8712 10.8712 GALILS Commanded: 4600. -2850. 850. -9250. 18400. 17850. Actual: 4600. -2850. 850. -9250. 18400. 17850. SETMIR VALUES PriDesOrient: 0.00 0.00 15.20 829.90 1.60 PriOrient: 0.00 0.00 14.91 830.49 3.19 SECONDARY: ---------- Focus: 0.00 Air Temp.: -1.3 Alt.: 29.997866 MIGS TONIGHT NOMINAL Axial A 1.5160 1.5160 Axial B 1.1220 1.1180 Axial C 1.1180 1.1210 Trans D 0.0200 -0.0050 GALILS Commanded: 1628449. 1548172. 1574257. -3900. -8750. Actual: 1628650. 1548200. 1574500. -3900. -8750. SETMIR VALUES SecDesOrient: 1257.00 0.00 -25.00 0.00 146.27 SecOrient: 1256.88 0.06 -24.96 0.29 146.52 ---=== PROBLEMS IN DETAIL ===--- IOP - Scrolling Display: ----------------------- The "All of g" option fails with the following traceback: Unknown SCD configuration, allg while executing "error "Unknown SCD configuration, $type" " ("default" arm line 1) invoked from within "switch $type { "normal" { set offset 410 set photo_chips "1 0 1 0 1" } "all" { set offset 40; set all 1 set photo_chips ..." (procedure "setup_scd" line 4) invoked from within "setup_scd allg" invoked from within ".scd.bottom.setup.menu invoke active" ("uplevel" body line 1) invoked from within "uplevel #0 [list $w invoke active]" invoked from within "if {[$w type active] == "cascade"} { $w postcascade active set menu [$w entrycget active -menu] tkMenuFirstEntry $menu } elseif {[$w type activ ..." (procedure "tkMenuInvoke" line 14) invoked from within "tkMenuInvoke .scd.bottom.setup.menu 1" (command bound to event) Engineering Camera - Pointing Model Offsets Wrong: ------------------------------------------------- We saved guider frames of the offset corrections that moved the FK5 star away from the engineering camera center, instead of toward the center. The frames are on 'host in usrdevel/snedden/engcam/. Details follow: frame 58 - Position of FK5 star before correction: r=218, c=409 Az ,Alt offsets: -0.005574, -0.001767 to center the star frame 59 - New position after correction: r=183, c=431 frame 60, 61 - We then applied the offsets in the opposite direction (+0.005574, +0.001767) and the star was centered properly. Frame 60 shows the az correction, Frame 61, the alt correction. SOP - PR 5864: ------------- We then moved on to try to make the focus sweep fail. The first focus sweep indeed failed. IT was a 11 step, delta=50 focus sweep. The SOP session popped up a window informing the observe of the error and gave the following trace back. After one failed focus sweep, we successfully did 7 focus sweeps. We changed the number of steps and the amount of each step. Error from the first focus sweep: Expected new guiderImage did not arrive. while executing "error "Expected new guiderImage did not arrive."" invoked from within "if {$newflg<1} { error "Expected new guiderImage did not arrive."; murmur "Expected new guiderImage did not arrive."; }" (procedure "exposeAndWait" line 16) invoked from within "exposeAndWait $expt " ("foreach" body line 29) invoked from within "foreach focus $focusList { echo "Setting focus to $focus" tccPut set focus=$focus listenerToTCC # here we need to wait unti ..." (procedure "guiderFocus" line 33) invoked from within "guiderFocus $center $steps $delta $exp $fwhm" (procedure "GUIguiderFocus" line 29) invoked from within "GUIguiderFocus $GUIgfCenter $GUIgfSteps $GUIgfDelta $GUIgfExp $GUIgfFwhm" invoked from within ".guidergui.gfocus.a.button invoke" ("uplevel" body line 1) invoked from within "uplevel #0 [list $w invoke]" invoked from within "if {($w == $tkPriv(window)) && ([$w cget -state] != "disabled")} { uplevel #0 [list $w invoke] }" invoked from within "if {$w == $tkPriv(buttonWindow)} { set tkPriv(buttonWindow) "" $w config -relief $tkPriv(relief) if {($w == $tkPriv(window)) && ([$w cget -state] ..." (procedure "tkButtonUp" line 3) invoked from within "tkButtonUp .guidergui.gfocus.a.button" (command bound to event)