APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Tuesday December 15, 2009 (MJD 55181) ---=== OBSERVING TEAM ===--- Dan Oravetz Audrey Simmons Kaike Pan (afternoon) Steph Snedden (support) Jon Brinkmann (support) Kyle Dawson (support) ---=== OBSERVING PLAN ===--- BOSS Science! ---=== OBSERVING SUMMARY ===--- Clear all night, but we closed for ~30 minutes around 10:15Z due to high humidity. We could not observe the last hour or so due to problems with the sop actor hanging and not responding to any commands. It would not restart through stageManager. First the problem was with sop, then it was with stageManager not restarting sop and not reconnecting it (please see ticket #491 and the log for details). It would really be beneficial if we had more thorough documentation describing how to handle a variety of actor and stageManager problems similar to this. We had some SOS reduction problems that prevented us from seeing results on SOS. Kyle and Gary fixed the problem around 06:00Z and now SOS is displaying *most* everything it should. For some reason, it did not reduce SP2 cameras on plate 3647. Observed BOSS: cart 13, plate 3586 cart 14, plate 3606 cart 10, plate 3647 cart 12, plate 3659 cart 16, plate 3690 We discovered additional partially dropped columns within a few flat exposures. One MCP mismatch tonight. One telescope jump tonight - see guider images 652-653 ---=== OBSERVING LOG ===--- Afternoon: --------- Collimations were checked on cart 13. All systems were checked and found no problem. 22:45Z started ventilation. During afternoon, a few flat exposures were taken, and visual checks found that some column drop occured on r1, b2. They are not repeatable. But we did not find any spectra center shifted that David on expsore #104652. Night: ------ 00:00Z - Opened telescope to clear skies. 00:21Z - Biases 104916-17. 00:23Z - gotoField cart 13, BOSS 3586. 00:26Z - Dark 104918. 00:28Z - Hartmanns 104919-20. Moved collimators by what collimate asked for. 00:34Z - Hartmanns 104921-22. 00:45Z - Moved blue camera rings by what collimate suggested. 00:49Z - Hartmanns 104923-24. 00:51Z - Cals 104925-26. There are several partial dropped columns within b2 on the flat. 01:13Z - 15 min exposure 104927. Seeing 2.5" 01:30Z - 15 min exposure 104928. Seeing 1.7" 01:47Z - 15 min exposure 104929. 02:04Z - 15 min exposure 104930. 02:21Z - 15 min exposure 104931. 02:38Z - 15 min exposure 104932. 02:59Z - 15 min exposure 104933. 03:17Z - Cals 104934-35. We obtained 15.0 in b2 and 29.0 in r2. We are going to hope that this plate is DONE, because it is really close to the S/N^2 criteria. 03:25Z - Mounted cart 14, BOSS plate 3606. 03:27Z - Hartmanns 104936-37. 03:31:25Z - Sp1 motor c failed during collimator adjustment. The STUI actuator info doesn't change even when part of collimator move has completed and something fails later on. 03:35Z - Cals 104938-39. 03:42Z - 15 min exposure 104940. Seeing 1.8" 04:00Z - 15 min exposure 104941. 04:17Z - 15 min exposure 104942. 04:32Z - 15 min exposure 104943. 04:50Z - Cals 104944-45. We gathered 6.3 in b2 and 14.5 in r2. We did not finish this plate yet. 04:59Z - Mounted cart 10, BOSS plate 3647. 05:00Z - Hartmanns 104946-47. 05:07Z - Cals 104948-49. 05:12Z - 15 min exposure 104950. Seeing 1.8" 05:37Z - 15 min exposure 104951. 05:51Z - 15 min exposure 104952. 06:07Z - 15 min exposure 104953. 06:27Z - Cals 104954-55. We don't know what we got on this plate, but it has 5.9 in b2 and 10.8 in r2 for previous signal. Is it DONE? 06:28Z - Mounted cart 12, BOSS plate 3659. 06:40Z - Hartmanns 104956-57. 06:46Z - Cals 104958-59. Did not find stars until 30 minutes later. Increased exposure times, verifyed cartridge number, retook guider flat and dark, went back to the field, etc. Applied random 0.002 arc offset in RA, and was able to find stars. 07:24Z - Cals 104960-61. 07:28Z - MCP mismatch. Had to fix it with 'mcp info'. 07:30Z - 15 min exposure 104962. Seeing 1.5" 07:47Z - 15 min exposure 104963. 08:03Z - 15 min exposure 104964. 08:05Z - Telescope jump gimg.652-653. 08:26Z - Cals 104965-66. We collected 10.3 in b2 and 19.3 in r2, combined with previous signal, we finished this plate. 08:30Z - Mounted cart 16, BOSS 3690. 08:38Z - Hartmanns 104967-68. 08:49Z - Cals 104969-70. 08:55Z - 15 min exposure 104971. Seeing 1.5" 09:13Z - 15 min exposure 104972. 09:29Z - 15 min exposure 104973. 09:46Z - 15 min exposure 104974. 10:03Z - Cals 104975-76. We gathered 12.2 in b2 and 13.8 in r2. Sop timed out again with attempting and failed to close the FF screens. BOSS commands were issued instead. 10:11Z - Closed enclosure and stopped ventilating due to high humidity. There were several surfaces down at the telescope that felt wet and frosty. 10:55Z - Humidity drasically improved. Opened enclosure and started ventilating. Lost science time due to several sop timeouts from attempting to slew to the field (the azimuth brake engaged during the slew). One example of such: 11:10:21 APO.doravetz 248 sop f text="Failed to close screens, warm up lamps, and slew to field" During this time, we were unable to talk with sop. It would not respond to any commands. It was hung on slewing to the field and kept trying, even when it failed. Status read IDLE within the BOSS window the entire time. Attempted to restart sop through stageManager, but was unable to issue 'hub startNubs' and received these errors within STUI: 11:21:51 APO.doravetz 267 hub f NoTarget="the target named sop is not connected" 11:22:01 APO.doravetz 268 hub i text="(re-)starting nub sop" 11:22:01 APO.doravetz 268 hub w text="failed to start nub sop: [Errno 111] Connection refused" Could not restart SOP through stageManager because of 'no version declared current' and 'starting new sopActor... FAILED! sopActor is not running' messages. The tail current.log read: Traceback... File "sopActor_main.py", line 7, in ... import opscore.actor.model... ImportError: No module named opscore.actor.model 12:24Z - Twilight begins in a few minutes, so we decided to close and hopefully somebody will be able to check and fix this problem in the morning. 12:24Z - Closed enclosure and stopped ventilating. 12:31Z - endNight finished. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): QA Procedures Done (y/n): UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 01:05Z imager 22.66 0.0000 55.17 0.0000 198.89 0.0000 0.999911 04:22Z imager -28.86 0.0000 53.77 0.0000 156.08 0.0000 0.999911 05:29Z imager -40.15 0.0000 49.99 0.0010 147.20 0.0000 0.999690 07:29Z imager 126.16 0.0010 70.22 -0.0020 -67.26 0.0000 0.999992 09:06Z imager 164.33 -0.0010 68.96 -0.0030 -21.95 0.0000 0.999992 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 01:05Z unknown 0.99991 878 2443 550 23 55.2 -1.7 8 184 - 2.5 04:22Z imager 0.99991 878 2385 489 -29 53.8 -2.3 7 179 - 1.8 05:29Z engcam 0.99969 3060 4415 700 -40 50.0 -0.9 14 182 - 1.8 07:29Z unknown 0.99999 79 1643 446 126 70.2 -1.5 12 189 - -1.5 09:06Z unknown 0.99999 79 1683 484 164 69.0 -1.7 9 202 - 1.5 ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 22:24Z 35 24 4 190 (S) 311 - 22:56Z 34 23 6 187 (S) 284 - 23:28Z 33 21 11 146 (SE) 263 - 00:01Z 30 22 9 132 (SE) 245 - 00:41Z 29 17 13 149 (SSE) 235 - 01:13Z 29 17 11 179 (S) 224 - 01:45Z 29 17 9 184 (S) 224 - 02:17Z 29 17 10 183 (S) 206 - 02:49Z 28 17 9 175 (S) 231 - 03:22Z 29 16 5 190 (S) 240 - 03:54Z 29 16 6 185 (S) 232 - 04:26Z 28 15 6 179 (S) 230 - 04:58Z 29 15 13 190 (S) 181 - 05:30Z 30 19 13 182 (S) 243 - 06:02Z 30 21 14 182 (S) 530 - 06:34Z 30 21 12 184 (S) 485 - 07:06Z 29 22 11 184 (S) 518 - 07:39Z 29 23 10 185 (S) 410 - 08:11Z 29 23 9 186 (S) 513 - 08:43Z 29 23 10 191 (S) 478 - 09:15Z 29 23 9 213 (SSW) 466 - 09:47Z 28 23 8 224 (SW) 517 - 10:19Z 29 18 6 251 (WSW) 439 - 10:52Z 29 16 6 254 (WSW) 334 - 11:24Z 29 16 6 184 (S) 361 - 11:56Z 28 15 9 178 (S) 368 - 12:28Z 28 15 9 178 (S) 349 - ---=== TELESCOPE STATUS ===--- 23:25Z - Began ventilation. Doors and louvers opened, fans on. 00:00Z - Opened enclosure. 10:15Z - Enclosure is on, doors and louvers are closed, and fans are off. Hig humidity. 10:55Z - Opened enclosure. 12:23Z - Enclosure is on, doors and louvers are closed, and fans are off. No interlocks bypassed. Status at: 12:23Z Telescope stowed at: 30,121,00 mount Instrument mounted: Cartridge 16 Counterweights at: 250 lbs LN2 autofill systems: Connected and turned on. 180L LN2 dewar scales: Spectro 119.40 lbs, 21.41 psi ---=== SOFTWARE USED ===--- IOP/SOP: v4_40_0 Watcher: v2_36_0 MCP: v6_6_1 TPM: tpm_v3_5_0 AstroDa: v15_13_1 TCC: TCC 2.11.0 2009-06-24 sdssProcedures: SoS: plate-mapper: ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A -0.6980 -0.6940 Axial B -0.0370 -0.0410 Axial C -0.3770 -0.3710 Trans D 11.6060 11.6170 Lateral E 10.3759 10.3505 Lateral F 11.9507 11.9253 GALILS Commanded: 9900. 400. 5850. 1200. 2950. 2750. Actual: 9917. 401. 5832. 1205. 2954. 2757. SETMIR VALUES PriDesOrient: 0.00 -3.30 5.10 278.08 178.06 PriOrient: 0.00 -3.04 5.26 277.70 178.07 SECONDARY: ---------- Focus: 0.00 Air Temp.: 1.1 Alt.: 29.999989 MIGS TONIGHT NOMINAL Axial A 18.6320 18.6150 Axial B 15.6400 15.6120 Axial C 13.8620 13.8170 Trans D -11.7910 -11.7780 GALILS Commanded: 1600462. 1564739. 1585942. -7600. -12400. Actual: 1597721. 1566188. 1588095. -7597. -12380. SETMIR VALUES SecDesOrient: 1257.00 -4.00 -9.00 10.00 234.42 SecOrient: 1256.77 -3.28 -10.44 9.24 234.22 ---=== PROBLEMS IN DETAIL ===---