APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Tuesday September 5, 2006 (MJD 53984) ---=== OBSERVING TEAM ===--- Mike Harvanek Jian Ge Pengchang Guo Larry Carey (support) French Leger (support) ---=== OBSERVING PLAN ===--- ET science! ---=== OBSERVING SUMMARY ===--- The 75 and 90 degree alt fiducials and the 152 degree az fiducial did not read in either direction. Had a serious and potentially dangerous problem with the spectro autofill system. See the log for details. PR 7113 was filed. Lost much of the night to high humidity/clouds. Observed ET plate 8031 for ~2.5 hrs. ---=== OBSERVING LOG ===--- Afternoon: ---------- Alt fiducials at 75 and 90 degrees and the az fiducial at 152 degrees did not read in either direction. Night: ------ High humidity until around 04:30Z when it started to dry out a little. I waited for about half an hour to see if it was stable. However, in that time, clouds started to increase. Radar also showed a storm rapidly approaching from the north. So, I remained closed. Shortly thereafter, the humidity began to rise and by 05:30Z, the dewpoint depression had dropped back down to around 2 degrees again. The master server died around 06:53Z and needed to be restarted. Around the same time, id5 became unresponsive and the photo 2 crate needed to be rebooted. The following watcher error alerted me to a serious and potentially dangerous problem - dewar (Spectro Dewar Low Weight:: 2006-09-06 07:13:02Z). The spectro 10 l dewars were filling when they shouldn't have been filling and then the autofill failed to shut off. Based on the spectro dewar log, here's what happened: At 13:15Z on 09-05 the spectro dewars filled normally (~40 lbs). At 02:45Z on 09-06 (13.5 hrs later) the spectro dewars again filled normally (~40 lbs). At 06:30Z on 09-06 (3.75 hrs later) the spectro dewars took an additional ~16 lbs of LN2. At 07:00Z on 09-06 (0.5 hrs later) the spectro dewars again started filling. This time the fill dumped ~75 lbs of LN2 in 13 minutes at which time the watcher threw the low dewar weight warning. I had checked the dewar weights when I came on duty this evening and knew there was more than enough in both dewars to make it through the night. So, when I saw the low dewar weight warning, I knew there was a problem. I notified Larry Carey so he could act as a safety second. I opened the east door to air the enclosure out. After a few minutes, I opened the west door while Larry remained outside the enclosure. After waiting another few minutes for the enclosure to air out some more, I then went in to stop the fill. First I tried to close the dewar but it was frozen open. So, I stopped the fill by moving the auto/manual fill switch to the off position. After the dewar warmed up a bit, I was able to close the dewar liquid valve. The spectro dewar weight is currently at 72 lbs meaning we lost over 100 lbs of LN2 on that "fill". The fill system is currently shut off (the auto/manual fill switch is in the off position). Furthermore, when the excess LN2 was spilling out of the 10 l dewars, it was spilling onto the floor very near to the ET cables. Fortunately, they were not damaged but Jian Ge said they could have easily been destroyed by this event. PR 7113 was filed. Note that at no time during this event did the ODH sensors go off. Shortly after this, a sheriff drove on site looking for a couple fighting in a parked car. He drove up with headlights and spotlight a blazin'. We were not open at the time. At some point during all of this, the weather improved. The sky was mostly clear and the humidity was at an acceptable level so I started ventilating (08:10Z) and mounted a fresh cartridge. I was just about to open when I noticed clouds/fog below us. I decided to keep ventilating but remain closed. The humidity was bouncing around between 50% and 65%. Around 08:40Z, Jian Ge offered to post his student at the west rail to watch for fog rolling up the hill if we opened. I checked the conditions and visually checked the clouds/fog below us. They did not look bigger or closer so I agreed to open. We opened around 08:50Z. Within 10 minutes of opening, the clouds/fog below us began to break up and disperse. Good call Jian Ge. 09:02Z Cartridge 7, ET plate 8031. An fk5 was needed to acquire the field. No guide star in fiber 5 so fiber 5 was disabled. We spent about 2.5 hours on this plate. A nearly empty coffee pot left on the warmer too long had me searching for "that burning smell." At least it wasn't a fire. As we were heading down to close, the ODH alarm sounded. It went off for maybe 10-15 seconds and then stopped. We closed around 12:20Z. endNight rejoiced. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): YES QA Procedures Done (y/n): N/A UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 40585, plate -9999 ------- 01:03 40585 0.0 bias 09:01 40586 0.0 bias ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 10:19Z 7 8031 49.02 0.0004 73.37 0.0008 222.86 0.0000 1.000090 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL Stays: JL ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 10:20Z 7 8031 1.00009 -889 366 -25 49 73.4 12.0 10 65 ---- 1.6" 11:21Z 7 8031 1.00009 -889 410 25 -9 78.2 10.9 14 74 ---- 1.4" ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 23:13Z 52 51 5 200 (SSW) 220 - 23:45Z 51 50 5 188 (S) 2816 - 00:21Z 50 51 9 203 (SSW) 5956 - fog 00:53Z 50 50 1 188 (S) 9147 - partly cloudy 01:25Z 49 49 1 148 (SSE) 212 - partly cloudy 01:57Z 49 49 0 277 (W) 4659 - mostly cloudy 02:30Z 49 49 1 345 (NNW) 2868 - partly cloudy 03:02Z 48 48 3 38 (NE) 248 - mostly clear 03:34Z 48 48 5 10 (N) 315 - mostly clear 04:06Z 48 47 0 70 (ENE) 318 - partly cloudy 04:38Z 53 39 5 86 (E) 321 - partly cloudy 05:10Z 50 44 15 68 (ENE) 323 - mostly cloudy 05:43Z 49 46 16 70 (ENE) 415 - mostly cloudy 06:15Z 49 45 14 66 (ENE) 358 - partly cloudy 06:47Z 48 44 23 64 (ENE) 387 - mostly cloudy 07:19Z 47 41 21 70 (ENE) 238 - partly cloudy 07:51Z 48 40 11 83 (E) 279 - partly cloudy 08:24Z 50 38 0 24 (NNE) 338 - clear 08:56Z 50 38 0 351 (NNW) 320 - clear 09:26Z 51 34 0 285 (WNW) 438 - clear 09:57Z 53 33 0 297 (WNW) 598 - clear 10:28Z 53 31 10 53 (NE) 462 - clear 10:59Z 52 31 17 68 (ENE) 461 - clear 11:30Z 50 33 12 73 (ENE) 426 - clear 12:01Z 50 32 13 73 (ENE) 447 - clear 12:31Z 50 32 6 80 (E) 463 - clear ---=== TELESCOPE STATUS ===--- 08:10Z Enclosure doors opened, fans and blowers on, louvers opened. 08:50Z Enclosure off. 12:20Z Enclosure on, doors closed, fans and blowers off, louvers closed. Telescope stowed at 30 degrees with cartridge 2 (plate 8022) mounted. Counterweights at 250. Spectro auto-fill NOT connected and turned off. Spectro dewar 12:45Z 72 lb, 0 psi --> NOT CONNECTED Imager dewar 12:45Z 134 lb, 18 psi No interlocks bypassed. ---=== SOFTWARE USED ===--- IOP/SOP: v4_26_0 Watcher: v2_34_0 MCP: v6_3_2 TPM: tpm_v3_3_0 AstroDa: v15_12_1 TCC: TCC 2.7.2.1 August 6 2004 sdssProcedures: v2_19 SoS: v5_1_4 hoggPT: v1_6_9 plate-mapper: v4_3_1 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 24.6740 24.6750 Axial B 26.2530 26.2540 Axial C 25.3490 25.3490 Trans D 13.5740 13.5760 Lateral E 9.9568 9.9568 Lateral F 11.5316 11.5316 GALILS Commanded: 4600. -2000. 2650. -9300. -9800. -9800. Actual: 4600. -2000. 2650. -9300. -9800. -9800. SETMIR VALUES PriDesOrient: 0.00 0.00 0.00 0.00 0.00 PriOrient: 0.00 0.00 0.00 0.00 0.00 SECONDARY: ---------- Focus: 0.00 Air Temp.: 9.8 Alt.: 30.318750 MIGS TONIGHT NOMINAL Axial A 1.5570 1.5550 Axial B 1.1330 1.1290 Axial C 1.1930 1.1910 Trans D -0.0600 -0.0530 GALILS Commanded: 1614024. 1594685. 1543164. -10550. -5800. Actual: 1614053. 1594719. 1543082. -10535. -5822. SETMIR VALUES SecDesOrient: 1257.00 20.00 -10.00 -100.00 176.64 SecOrient: 1257.00 19.96 -9.99 -100.43 176.58 ---=== PROBLEMS IN DETAIL ===--- See the main log.