APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Friday March 7, 2003 (MJD 52706) ---=== OBSERVING TEAM ===--- swing: Jurek Krzesinski night: Howard Brewington Steph Snedden Visitors: Korean film crew Gretchen Van Doren Scotsuko Nitta Kleinman ---=== OBSERVING PLAN ===--- Science! ---=== OBSERVING SUMMARY ===--- Clouds lingered longer than we expected. Two plates completed. Good data on one more. ---=== OBSERVING LOG ===--- Afternoon checkout: Servers started under test iop version (v3_109_4). DA-s - ok, spectro guider - ok, MIGs - ok, Spectrograph focus: Night: We opened at sunset because we were optimistic and the Korean public TV crew wanted shots of the building opening. We managed to get an FK5's photons down a fiber, but the clouds were too thick to see guide stars. We waited. We rolled the enclosure back on, but left the doors open & fans on. 05:55Z Clouds starting to thin. Opened back up & mounted the appropriate cartridge. 06:22Z Cartridge 4, plate 1007 Needed to use an FK5 star to set offsets for the field (we need a new pointing model, next dark run). After acquiring guide stars, the clouds thickened again. We waited some more. 07:19Z Clouds thin again. We start guiding again. The guiding looked poor, with the guide stars not centered well. We were using the old guiding algorithm and the scale was (reportedly) good. The efficiency was under 40 percent. We decided to try to start an exposure, and lo and behold Howard's preCal didn't work, then he couldn't get an saoMon to start. It looked like we were dealing with rogue iop processes left over from the server restart. We also lost some time while more clouds came and went. Steph started doing the science exposures while Howard cleaned up all processes, basically starting fresh. Guiding was good from Steph's SOP. 07:58Z Science begins. Plate 1007 completed after three exposures. 09:49Z Cartridge 7, plate 1017 It appeared photometric on the IRcam, and to the eye, but clouds were uncomfortably close on the satellite images. Seeing was also marginal. We decided to stick with spectroscopy. Howard's SOP behaved. Stars on slew, using offsets from previous plate. Declared done in three exposures. 11:20Z Cartridge 9, plate 1039 Stars on slew (did not clear offsets). Got two good exposures plus a good smear, but plate needs a little more. Not done. 13:11Z endNight -doSpectro -doTpm completed rejoicingly. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 3707 23:09Z 23:30Z 100 O -138.13 -133.02 47 ignore ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): QA Procedures Done (y/n): UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 19690, plate -9999 ------- 22:01 19690 0.0 bias ----- sequence 19691, plate 1264 ------- 23:11 19691 0.0 bias 23:25 19692 10.0 flat 23:28 19693 10.0 flat 23:33 19694 2.1 arc 23:36 19695 2.1 arc ----- sequence 19696, plate -9999 ------- 00:04 19696 0.0 bias ----- sequence 19697, plate -9999 ------- 03:41 19697 0.0 bias ----- sequence 19698, plate -9999 ------- 04:02 19698 0.0 bias ----- sequence 19699, plate 1264 ------- 04:07 19699 10.0 flat 04:09 19700 2.0 arc Spectro focus 04:13 19701 2.1 arc 04:16 19702 2.1 arc 04:31 19703 2.1 arc 04:34 19704 2.1 arc 04:49 19705 2.1 arc 04:52 19706 2.1 arc 05:00 19707 2.1 arc 05:02 19708 2.1 arc 05:38 19709 2.1 arc 05:41 19710 2.1 arc ----- sequence 19711, plate -9999 ------- 06:06 19711 0.0 bias 07:36 19712 0.0 bias ----- sequence 19713, plate 1007 ------- 07:47 19713 10.0 flat 07:50 19714 2.0 arc ----- sequence 19713, plate 1007 ------- 08:25 19715 1500.1 target 08:29 19716 0.0 bias 08:31 19717 0.0 bias 09:02 19718 1200.1 target 09:10 19719 240.0 smear 09:29 19720 870.7 target 09:32 19721 10.0 flat 09:35 19722 2.0 arc ----- sequence 19723, plate 1017 ------- 09:42 19723 0.0 bias 10:00 19724 10.0 flat 10:03 19725 2.0 arc 10:23 19726 900.1 target 10:41 19727 900.1 target 10:59 19728 800.1 target 11:07 19729 240.0 smear ----- sequence 19730, plate 1039 ------- 11:27 19730 10.0 flat 11:30 19731 2.0 arc 11:49 19732 900.1 target 12:07 19733 900.1 target 12:14 19734 240.0 smear ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 08:30Z 4 1007 224.51 0.0002 59.24 0.0031 67.91 -0.0116 1.000080 10:08Z 7 1017 216.15 -0.0003 58.10 0.0033 59.60 -0.0109 1.000130 10:08Z 7 1017 216.25 -0.0003 58.01 0.0033 59.83 -0.0109 1.000130 11:30Z 9 1039 218.81 -0.0001 56.17 0.0022 65.18 -0.0009 0.999910 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL6055 Stays: JL6056 ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 00:12Z 1 0 1.00000 0 1258 0 121 30.0 4.3 11 202 08:26Z 4 0 1.00008 -789 439 -60 224 59.8 2.6 13 291 08:30Z 4 1007 1.00008 -789 440 -60 225 59.3 2.8 13 302 - 1.9 08:59Z 4 1007 1.00007 -690 474 -120 227 54.8 2.7 13 289 - 1.5 10:08Z 7 1017 1.00013 -1283 -26 -120 216 58.1 2.3 12 317 - 1.8 11:30Z 9 1039 0.99991 888 1798 -100 219 56.2 2.8 14 322 - 1.9 ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 21:46Z 45 13 12 220 (SW) 186 - mcl 22:23Z 46 11 11 212 (SSW) 142 - mcl 22:53Z 44 12 8 195 (SSW) 166 - mcl 23:23Z 43 13 10 249 (WSW) 163 - mcl 23:59Z 40 8 10 208 (SSW) 106 - mcl 00:31Z 38 5 8 179 (S) 145 - mcl 01:04Z 37 1 9 180 (S) 161 - mcl 01:36Z 37 0 8 184 (S) 129 - mcl 02:09Z 35 7 9 179 (S) 187 - mcl 02:41Z 36 8 11 190 (S) 238 - mcl 03:15Z 36 4 16 217 (SW) 147 - mcl 03:47Z 38 2 13 255 (WSW) 129 - mcl 04:20Z 37 3 15 259 (W) 150 - mcl 04:51Z 38 3 11 270 (W) 134 - mcl 05:22Z 38 5 13 277 (W) 187 - mcl 05:53Z 38 5 10 269 (W) 199 - mcl 06:25Z 38 8 11 281 (W) 177 - pcl 06:56Z 38 9 11 272 (W) 173 - pcl 07:27Z 37 9 11 273 (W) 194 - mclear 07:58Z 37 10 12 288 (WNW) 222 - mclear 08:29Z 36 10 10 297 (WNW) 192 - mclear 09:00Z 37 10 13 286 (WNW) 253 - mclear 09:30Z 36 7 7 295 (WNW) 326 - mclear 10:02Z 36 11 12 308 (NW) 326 - mclear 10:32Z 36 11 12 324 (NW) 280 - mclear 11:02Z 36 10 8 357 (NNW) 253 - mclear 11:32Z 37 10 12 322 (NW) 226 - mclear 12:02Z 36 9 2 18 (NNE) 240 - mclear 12:33Z 37 8 9 4 (N) 201 - mclear 13:03Z 35 12 9 2 (N) 238 - 13:33Z 35 11 9 20 (NNE) 242 - ---=== TELESCOPE STATUS ===--- 00:22Z Opened doors, started blowers & fans 00:55Z Opened building 02:10Z Closed building, left doors open 05:55Z Opened again. 12:38Z Closed building. Blowers & fans off. Cartridge 9 mounted. Counterweights at 290. Stowed at 30 deg. alt with autofill connected. 12:49Z Imager & spectro 180-liter dewars swapped out. 13:09Z Spectro dewar weight: 306 13:09Z Imager dewar weight: 231 ---=== SOFTWARE USED ===--- IOP/SOP: v3_109_4 Watcher: v2_21_0 MCP: v5_18_0 TPM: tpm_v2_27_0 AstroDa: v14_47 TCC: TCC 2.6.8 November 13 2002 sdssProcedures: v1_64 SoS: v4_9_13 hoggPT: v1_6_6 plate-mapper: v4_2_0 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.0930 0.0790 Axial B 0.7960 0.7970 Axial C 0.8140 0.8240 Trans D -9.0790 -9.1130 Lateral E 1.8840 1.8870 Lateral F 0.0000 1.4300 GALILS Commanded: 5400. -3700. 900. -200. 31550. 30650. Actual: 5400. -3700. 900. -200. 31550. 30650. SETMIR VALUES PriDesOrient: 0.00 -11.80 23.00 1256.90 642.10 PriOrient: 0.00 -12.16 22.81 1257.41 642.19 SECONDARY: ---------- Focus: 0.00 Air Temp.: 5.9 Alt.: 30.011671 MIGS TONIGHT NOMINAL Axial A 1.5180 1.5170 Axial B 1.1580 1.1560 Axial C 1.1020 1.0990 Trans D -0.7030 -0.7070 GALILS Commanded: 1619680. 1555246. 1576386. -3400. -7900. Actual: 1619650. 1555250. 1576350. -3400. -7900. SETMIR VALUES SecDesOrient: 1257.00 0.00 -20.00 0.00 130.98 SecOrient: 1257.02 -0.02 -20.02 -0.49 130.90 ---=== PROBLEMS IN DETAIL ===--- Sever crash: ------------ 03:09Z We noticed that the servers had died. All nodes showed evidence of it (see the results for node 1 below). We restarted the servers. We were doing nothing at the time, using no instruments, waiting on clouds. The restart went without incident. PTVME test results: ptvmeTest -n 1 -a 0x18000000 PID: 101970 IOPB size: 64 bytes ping: no operation: read loopCnt: 1 (defaulted) 100 loops per dot LWordCnt: 0x00000001 (defaulted) controller: 0 (defaulted) remNode: 1 --> /dev/ptvme0n1 remAddr: 0x18000000 (defaulted) IxferMode: 0x00 (defaulted) TxferMode: 0x00 (defaulted) errno = 0: open ("/dev/ptvme0n1", O_RDWR) = 3 errno = 0: ioctl (fd=3, PT_IOCTL_GET_PRIORITY, &ptPriority) = 0 ptPriority = 0x03 errno = 0: ioctl (fd=3, PT_IOCTL_GET_VECTOR, &ptVector) = 0 ptVector = 0x01 errno = 0: ioctl (fd=3, PT_IOCTL_GET_TIMEOUT, &ptTimeoutSave) = 0 ptTimeout = 900 ticks errno = 0: ioctl (fd=3, PT_IOCTL_GET_OP_MODE, &ptOpMode) = 0 ptOpMode = 1 SINGLE_IO errno = 0: ioctl (fd=3, PT_IOCTL_GET_DEBUG_MODE, &ptDebugMode) = 0 ptDebugMode = 0 NORMAL errno = 0: ioctl (fd=3, PT_IOCTL_GET_DEBUG, &ptDebugSave) = 0 ptDebug = 0x00000000 errno = 5: ioctl (fd=3, PT_IOCTL_GET_NODE_ID, 0) = 0xFFFFFFFF (I/O error) . 0 3 ms 20:07 errno = 16: read (fd=3, 0x7F7F2CE0, 0x00000004) = -1 (Resource busy) CSB status: 0x00000000 AUX = (unknown) CMP = (Never Used) INNA = 0 errno = 0: lseek (fd=3, 0x00000000, SEEK_CUR) = 0x00000000 00000000: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F <................> 00000010: 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F <................> 00000020: 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F < !"#$%&'()*+,-./> 00000030: 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F <0123456789:;<=>?> ptvmeReset ptvmeMode: INFO: controller 0 reset (/dev/ptvme0n0) ptvmeMode: INFO: cur debug mode: 0 NORMAL ptvmeMode: INFO: cur debug: 0x00000000 ptvmeMode: INFO: cur timeout: 900 ticks ptvmeMode: INFO: def T/I xfer mode: 0x22 A32:D32 Program, Non-Privileged / A32:D32 Program, Non-Privileged ptvmeMode: INFO: op mode: 1 SINGLE_IO ptvmeMode: INFO: VMEbus priority: 0x03 ptvmeMode: INFO: VMEbus vector: 0x01 ptvmeMode: INFO: intern sem val: 1 ptvmeMode: INFO: user sem val: 1 ptvmeMode: INFO: PT-VME940 node: 0 Watcher Errors: --------------- MCP axes: altitude (calc_frames (end): Max accl. for altitude exceeded: 83995/10 (2003-03-08 01:07:04Z sdssmcp): 2003-03-08 01:07:22Z ) TPM dewar (MAJOR: 2003-03-08 09:36:56Z) while dewars were being changed. tracking (MINOR: 2003-03-08 08:29:22Z) ca (INVALID: 2003-03-08 11:35:55Z) Spectrograph focus check: -------------------------- sp1> Coll_motor_A -2434 Coll_motor_B -2435 Coll_motor_C -3639 sp2> Coll_motor_A 17 Coll_motor_B 13 Coll_motor_C 15