APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Friday September 9, 2005 (MJD 53623) ---=== OBSERVING TEAM ===--- Day and night: Howard Brewington Mike Harvanek Support: Eric Neilsen (by phone) Steve Kent (by phone) Steph Snedden (by phone) Rick Kessler (resident SN guy) Belated thanks to Margaret Votava for her support last night. Although she did not talk with the observers directly, her late evening support helped track down the problem with Murmur. Apologies for not acknowledging your effort last night Margaret. ---=== OBSERVING PLAN ===--- Supernova imaging; segue imaging and/or oblique scan if conditions permit. ---=== OBSERVING SUMMARY ===--- We got about 500 frames of supernova imaging. Run 5590, 82 N ran from lambda 114.8 to -172.4. However, the last 20 minutes or so were lost to clouds so this run is probably good to only about -177, frame 470. There are also isolated places scattered throughout the run (a few frames here, a few frames there) that are lost to clouds. We tried makeAllFieldLinks twice. Both times it appeared to hang. We waited 10-12 minutes each time but it just sat there. We killed it both times with a control C. We ran the supernova spooling script but it failed as well, probably because the makeAllFieldLinks script never completed. The spooling script said it could process up to 329 frames. We closed at about 08:00Z due to threatening clouds. It started to sprinkle as we were closing up. We got a few drops on the wind baffle but the mirrors appeared to stay dry. Clouds and humidity kept us closed for the remainder of the night. The light in the NW corner of the enclosure burned out. There is no hot water in the ops building kitchen. Focus loop problems; landru hung and needed to be rebooted; iop sessions from host2 on landru were taking 15 minutes to launch; teamster died; pointing needed correcting again (2nd night in a row; correction involved clearing offset corrections from previous night although we were pointed in almost the exact same place). ---=== OBSERVING LOG ===--- Afternoon --------- No problems with the checkout. We did check the spectro focus. Night ----- We did an on-telescope bias in the enclosure and then pulled the building around 01:50Z and began supernova imaging. Run 5587: on-telescope bias. Run 5588: 82 N lambda 110. Pointing problem? Skippy is giving wildly varying results. Maybe we are at the edge of the catalog? Run 5589: 82 N lambda 113. Still a pointing problem. Offsets from last night were cleared and this seemed to solve the problem. Run 5590: 82 N lambda 114. Everything good from frame 11, lambda 114.8. We ran for 496 frames before closing for threatening clouds and sprinkles. Ending lambda was -172.4 but there are definitely scattered parts of the drift (a few frames here and a few frames there) that are lost to clouds. The last 15-20 minutes (starting around lambda -177, frame 470) is probably all bad. Howard had trouble getting the focus loop to run. Twice it failed to start, giving a long error message but the 3rd time it succeeded. The error message would be included in the Problems section except that shortly after this, landru froze and Howard no longer had access to the message. Howard was animating the radar loop when landru froze. It appears that landru needs a full reboot as it does not even respond to a ping (Destination Host Unreachable). However, since we are in the middle of a SN scan started from landru, we are reluctant to mess with landru at this time. Howard logged into the necessary windows using monx3 and will run from there until we end the drift (run 5590). We will reboot landru after ending run 5590. Clouds started appearing around 05:45Z and were coming and going. Shortly before 08:00Z we decided to close. Although the radar showed nothing in the area, the clouds looked threatening and they were coming from our blind spot. Howard went down to the telescope to prepare to close while Mike ended the drift, closed the flat field screens and brought the telescope to stow. As the flat field screens were closing and the telescope was going to stow, Howard felt the first drop of rain. Fortunately, the rain was light and was coming from the back side of the telescope. We got a few sprinkles on the wind baffle but we believe the mirror was spared. We checked the mirror for water spots. We didn't see any but the white spots from the new coating made it difficult to determine if the mirror had indeed remained dry. Note that Howard sounded the alarm well before the rain sensor did. Landru was rebooted after run 5590 ended and it appeared to come back normally. Howard's iop sessions took a long time to open however. All 4 of his sessions hung at "defining help" for approximately 15 minutes before successfully launching. Howard had this same problem at the beginning of the night. Mike did not have this problem. Howard logged onto host2 from landru whereas Mike had logged onto host2 from host from commish. Humidity became a problem around 09:30Z but we were already closed for clouds. We found compressor #2 to be offline at around 10:30Z and put it back on-line. The light in the NW corner of the enclosure burned out tonight. There is no hot water in the ops building kitchen. endNight completed successfully. We ran makeAllFieldLinks . It sat there for 10-12 minutes without completing. The instructions say it should take a few minutes. We tried it again and it did the same thing. We thought we ran this script last night too but either we didn't or it failed silently. We ran the supernova spooling script. It failed too, probably because makeAllFieldLinks failed. It said it could process 329 frames. Watcher errors: --------------- We saw lots of poolNRGangTrim watcher errors tonight e.g., poolNRGangTrim (61: 2005-09-10 01:15:05Z) The teamster died and gang transfer stopped so we issued a "teamster 60 imager -force" to solve the problem. We got multiple occurrences of the following watcher warnings: TEXTONLY *** FATAL *** dscArch: While writing frames - 1 (TEXTONLY *** FATAL *** dscArch: While writing frames - 1 : 2005-09-10 06:34:41Z ) dsc E archWrFail Error archiving frame 5590 347.13: Failed to close file 13 (dsc_E_archWrFail Error archiving frame 5590 347.13: Failed to close file 13 : 2005-09-10 06:34:41Z ) We also saw: AxisStat:alt (clock_loss_signal: 2005-09-10 06:36:30Z) AxisStat:az (clock_loss_signal: 2005-09-10 06:36:30Z) AxisStat:rot (clock_loss_signal: 2005-09-10 06:36:30Z) ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 5586 00:03Z 00:24Z 100 O 59.07 64.34 41 ignore 5587 01:31Z 01:49Z 100 O 81.25 85.78 44 bias on-telescope 5588 02:27Z 02:42Z 82 N 110.40 114.24 40 ignore bad pointing 5589 02:45Z 03:04Z 82 N 113.35 118.15 46 ignore bad pointing 5590 03:06Z 07:59Z 82 N 114.31 -172.22 504 science some cloudy frames ---=== IMAGING RUN DETAILS ===--- See the main log. ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- 5588 11 4 4 52.600 0.835 -10.500 0.523 -0.00345 153 54 5589 15 4 8 21.600 1.606 -17.700 2.942 0.04399 156 55 5590 13 4 13 -13.900 2.004 -0.700 2.384 -0.02653 163 56 5590 99 4 112 -0.000 0.128 -1.400 0.119 0.00518 163 56 5590 151 4 103 0.100 0.144 -1.400 0.124 0.00487 163 56 5590 202 4 69 -0.100 0.156 -1.300 0.119 0.00546 163 56 5590 254 4 59 0.200 0.130 -1.200 0.110 0.00534 163 56 5590 306 4 39 0.300 0.126 -1.500 0.141 0.00529 163 56 5590 356 4 50 0.100 0.147 -2.000 0.117 0.00476 163 56 5590 408 4 58 0.500 0.144 -2.100 0.128 0.00431 163 56 5590 460 4 33 0.900 0.108 -1.600 0.151 0.00501 163 56 ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): YES QA Procedures Done (y/n): NO UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 33929, plate -9999 ------- 23:42 33929 0.0 bias 00:12 33930 0.0 bias ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 03:29Z imager 16.89 0.0000 56.18 0.0000 103.89 0.0000 1.000000 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL Stays: JL ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 03:29Z imager 1.00000 0 -378 -275 17 56.2 12.8 14 183 i-23 1.0" 05:02Z imager 1.00000 0 -378 -275 17 56.2 12.8 12 177 i-23 1.1" ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 23:39Z 61 41 13 214 (SW) -9999 - 00:10Z 60 40 10 220 (SW) -9999 - 00:40Z 59 40 9 259 (W) -9999 - 01:11Z 55 39 10 226 (SW) -9999 - 01:43Z 55 37 9 226 (SW) -9999 - clear 02:24Z 55 38 14 183 (S) -9999 - light clouds 02:55Z 55 37 17 181 (S) -9999 - clear 03:25Z 54 37 8 192 (SSW) -9999 - clear 03:56Z 54 36 11 189 (S) -9999 - light clouds 04:28Z 54 37 9 181 (S) -9999 - light clouds 05:00Z 55 36 12 179 (S) -9999 - light clouds 05:31Z 55 35 12 180 (S) -9999 - light clouds 06:01Z 56 36 13 185 (S) -9999 - clouds 06:32Z 55 35 14 202 (SSW) -9999 - clear 07:03Z 54 36 12 186 (S) -9999 - clouds 07:34Z 54 36 13 209 (SSW) -9999 - clouds 08:06Z 54 38 12 209 (SSW) -9999 - clouds 08:38Z 54 45 14 206 (SSW) -9999 - clouds 09:10Z 52 44 11 212 (SSW) -9999 - clouds 09:43Z 52 48 9 211 (SSW) -9999 - clouds 10:15Z 53 47 11 229 (SW) -9999 - clouds 10:47Z 52 46 11 234 (SW) -9999 - clouds 11:19Z 53 46 10 241 (WSW) -9999 - clouds 11:51Z 52 46 12 242 (WSW) -9999 - clouds ---=== TELESCOPE STATUS ===--- 01:00Z Enclosure doors opened, fans and blowers on, louvers opened. 01:50Z Enclosure off. 08:00Z Enclosure on, doors closed, fans and blowers off, louvers closed (clouds, rain sprinkles). Never reopened (clouds, rain, humidity). Telescope stowed at 30 degrees with cartridge 5 (plate 2256) mounted. Counterweights at 250. Spectro auto-fill connected. Imager dewar 11:58 292 lb, 22 psi Spectro dewar 11:58 131 lb, 21 psi No interlocks bypassed. ---=== SOFTWARE USED ===--- IOP/SOP: iop neilsen Watcher: watcher neilsen MCP: v5_26_0 TPM: tpm_v3_1_0 AstroDa: biery TCC: TCC 2.7.2.1 August 6 2004 sdssProcedures: v2_03 SoS: v4_10_7 hoggPT: v1_6_9 plate-mapper: v4_3_1 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A -5.8440 -5.8440 Axial B -5.6760 -5.6770 Axial C 0.9420 0.9420 Trans D -9.9890 -9.9940 Lateral E 10.3759 10.3759 Lateral F 11.9507 11.9380 GALILS Commanded: 4700. -2450. 3000. -7800. 3300. 3000. Actual: 4700. -2450. 3000. -7800. 3300. 3000. SETMIR VALUES PriDesOrient: 0.00 -1.68 7.05 385.25 92.02 PriOrient: 0.00 -1.52 7.02 384.99 93.81 SECONDARY: ---------- Focus: 0.00 Air Temp.: 16.8 Alt.: 30.401887 MIGS TONIGHT NOMINAL Axial A 1.5440 1.5420 Axial B 1.0490 1.0470 Axial C 1.1670 1.1650 Trans D -0.1060 -0.1240 GALILS Commanded: 1593035. 1604417. 1555466. -6800. -5900. Actual: 1593450. 1603650. 1554300. -6800. -5900. SETMIR VALUES SecDesOrient: 1257.00 15.87 0.00 -50.00 136.76 SecOrient: 1257.40 15.61 0.42 -49.28 136.91 ---=== PROBLEMS IN DETAIL ===---