APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Saturday February 2, 2008 (MJD 54499) ---=== OBSERVING TEAM ===--- Kaike Pan Dmitry Bizyaev ---=== OBSERVING PLAN ===--- Science! ---=== OBSERVING SUMMARY ===--- Added signal to plate 2865 (evening seguefaint), just one good night requires to finish it. Finished plate 2912. During this plate exposure we got a problem with spectro crate and lost some 50 minites in total (including lost exposure). Added signal to plate 2943. Finished plate 2616. Added signal to 2657 (just one exposure left). ---=== OBSERVING LOG ===--- Afternoon: ---------- rebooted MCP as Steph suggested. all systems checked and working. Spectro collimation checked but no adjustment made. 23:55Z start ventilation. Mounted cartridge 9. ----- night: ----- 00:40Z enclosure off DIMM's software brings message about memory errors. Rebooted. Works fine after that. cartridge 9 plate 2865 is on fk5 did not found offsets cleared, then guide stars found at +5,+5 direction 01:23Z precal 01:30Z science 1700 --> 1800 02:03Z science 1800 postcal 02:44Z Cartridge 7 plate 2912 mounted fk5 star used. fucus loop employed; 30 sec exposure is used for the guider 03:01Z precal 03:13Z science 600 03:26Z science 600 03:39Z science 600 03:51Z science 600 04:04Z science 600 04:17Z science 300 04:26Z science 300 --> 400 Oops, 04:34Z - lost connection to sop during reading out. Spectro crate reseted. See problems section. 42 min was lost due to this problem. See details in "problems" section. This exposure is lost. 05:08Z science 360 postcal According to a message from Brian Yanny, this exposure can be declared as done. So be. Here we're getting yellow wind alert. DIMM results look hectic but it doesn't affect 2.5m. 05:31Z Plate 3 cartridge 2943 mounted 05:40Z precal 05:48Z science 1500 06:16Z science 1500 At the end efficiency dropped down to zero. postcal very cloudy after this 08:59Z cartridge 6 plate 2616 mounted fk5 used ... stars are not seen constantly due to clouds, efficiency ranges between 0 an 10% 09:57Z next attempt. precal 10:04Z science 1500 --> 1200 10:27Z science 1200 10:50Z science 1200 --> 1000 11:14Z Cartridge 5 plate 2657 mounted fk5 used 11:24Z precal 11:31Z science 1200 11:54Z science 1200 --> 1500 12:22Z science 1500 --> 1800 postcal 13:10Z closed telescope 13:20Z endNight rejoiced ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 7312 23:08Z 23:26Z 100 O -171.22 -166.78 42 ignore ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): QA Procedures Done (y/n): UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 49505, plate 2851 ------- 23:33 49505 10.0 flat 23:36 49506 2.0 arc 23:41 49507 3.0 arc 23:44 49508 3.0 arc ----- sequence 49509, plate -9999 ------- 00:27 49509 0.0 bias ----- sequence 49509, plate 2865 ------- 01:26 49510 10.0 flat 01:28 49511 2.0 arc 02:02 49512 1802.0 target 02:35 49513 1801.0 target 02:38 49514 10.0 flat 02:41 49515 2.0 arc ----- sequence 49516, plate 2912 ------- 03:03 49516 10.0 flat 03:06 49517 2.0 arc 03:25 49518 601.0 target 03:37 49519 600.0 target 03:50 49520 601.0 target 04:03 49521 600.0 target 04:16 49522 601.0 target 04:24 49523 300.0 target ----- sequence 49525, plate -9999 ------- 04:58 49525 0.0 bias ----- sequence 49527, plate 2912 ------- 05:03 49527 0.0 bias 05:14 49528 303.0 target 05:17 49529 10.0 flat 05:20 49530 2.0 arc ----- sequence 49531, plate 2943 ------- 05:43 49531 10.0 flat 05:45 49532 2.0 arc 06:15 49533 1500.0 target 06:35 49534 1014.0 target 06:41 49535 10.0 flat 06:43 49536 2.0 arc ----- sequence 49537, plate 2616 ------- 09:59 49537 10.0 flat 10:02 49538 2.0 arc 10:26 49539 1202.0 target 10:49 49540 1201.0 target 11:08 49541 1003.0 target ----- sequence 49542, plate 2657 ------- 11:26 49542 10.0 flat 11:28 49543 2.0 arc 11:52 49544 1200.0 target 12:20 49545 1501.0 target 12:54 49546 1802.0 target 12:59 49547 10.0 flat 13:01 49548 2.0 arc ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 01:39Z 9 2865 -35.51 -0.0013 40.94 0.0020 150.33 0.0180 1.000100 03:09Z 7 2912 62.56 0.0009 72.18 0.0034 234.41 0.0180 1.000170 05:48Z 3 2943 217.59 -0.0023 81.39 0.0012 41.64 0.0180 1.000150 10:22Z 6 2616 43.13 0.0014 72.79 0.0023 217.58 0.0180 1.000110 11:28Z 5 2657 -63.45 -0.0034 74.56 0.0007 123.91 0.0180 1.000120 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 01:39Z 9 2865 1.00010 -988 555 160 -35 41.0 0.0 16 222 -- 1.6" 03:09Z 7 2912 1.00017 -1679 -192 70 63 72.2 0.3 22 206 -- 1.3" 05:03Z 7 2912 1.00015 -1481 -24 80 -50 76.2 0.7 26 232 -- 1.3" 05:48Z 3 2943 1.00015 -1481 21 130 218 81.4 -0.4 21 242 -- 1.3" 10:22Z 6 2616 1.00011 -1086 478 250 43 72.8 -2.7 20 237 -- 1.3" 11:28Z 5 2657 1.00012 -1185 373 230 -63 74.6 -3.1 18 249 -- 1.6" ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 23:20Z 37 13 21 239 (WSW) 475 - 23:51Z 36 11 21 245 (WSW) 464 - clear 00:28Z 35 10 18 249 (WSW) 459 - clouds at horizon 01:00Z 33 11 15 257 (WSW) 524 - clear, photometric 01:33Z 33 11 16 225 (SW) 453 - '' 02:05Z 32 13 15 214 (SW) 522 - '' 02:35Z 32 13 17 213 (SSW) 609 - '' 03:06Z 32 13 21 214 (SW) 684 - '' 03:37Z 32 13 18 216 (SW) 662 - '' 04:09Z 33 13 22 222 (SW) 781 - '' 04:40Z 33 11 22 221 (SW) 975 - '' 05:11Z 32 11 19 231 (SW) 795 - '' 05:41Z 31 11 18 240 (WSW) 588 - clouds 06:12Z 30 11 21 250 (WSW) 556 - '' 06:42Z 30 12 23 242 (WSW) 566 - '' 07:12Z 30 12 15 246 (WSW) 541 - '' 07:43Z 30 13 19 262 (W) 573 - '' 08:15Z 29 12 22 256 (WSW) 576 - '' 08:48Z 28 12 17 267 (W) 564 - '' 09:22Z 28 12 14 262 (W) 568 - '' 09:52Z 27 11 18 252 (WSW) 608 - '' 10:27Z 27 11 20 237 (WSW) 635 - '' 11:00Z 27 11 18 236 (SW) 625 - '' 11:30Z 26 11 15 250 (WSW) 631 - '' 12:01Z 27 11 12 230 (SW) 610 - '' 12:31Z 26 11 12 253 (WSW) 475 - '' 13:01Z 27 10 23 232 (SW) 442 - '' ---=== TELESCOPE STATUS ===--- 23:55Z - Began ventilation. Doors and louvers opened, fans on. 00:40Z - Enclosure off. 13:10Z - Enclosure on, doors and louvers closed, fans off. No interlocks bypassed. Status at: 13:20Z Telescope stowed at: 30,121,00 mount Instrument mounted: Cartridge 5 Counterweights at: 270 lbs LN2 autofill systems: Connected and turned on. 180L LN2 dewar scales: Spectro 145 lbs, 16.8 psi Imager 178 lbs, 19.4 psi ---=== SOFTWARE USED ===--- IOP/SOP: v4_38_0 Watcher: v2_36_0 MCP: v6_4_0 TPM: tpm_v3_5_0 AstroDa: v15_13_1 TCC: TCC 2.7.2.1 August 6 2004 sdssProcedures: v2_38 SoS: v5_1_8 plate-mapper: v4_3_1 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 0.999920 MIGS TONIGHT NOMINAL Axial A -5.2160 -5.2160 Axial B -0.2143 -0.2142 Axial C -0.0015 -0.0016 Trans D 11.8370 11.8540 Lateral E 10.0711 10.0584 Lateral F 11.6586 11.6586 GALILS Commanded: 4600. -2000. 2650. -12450. -6500. -6550. Actual: 4600. -2000. 2650. -12439. -6477. -6546. SETMIR VALUES PriDesOrient: 0.00 0.00 0.00 100.00 -200.00 PriOrient: 0.00 0.00 0.00 99.59 -200.69 SECONDARY: ---------- Focus: 0.00 Air Temp.: 2.1 Alt.: 88.026177 MIGS TONIGHT NOMINAL Axial A 0.8025 0.8025 Axial B 0.6123 0.6120 Axial C 0.5475 0.5474 Trans D 0.6005 0.6008 GALILS Commanded: 1627391. 1652411. 1470786. -5900. -7800. Actual: 1627496. 1653168. 1470885. -5887. -7780. SETMIR VALUES SecDesOrient: 1257.00 60.00 -4.00 -20.00 118.90 SecOrient: 1256.75 59.85 -4.17 -20.03 119.38 ---=== PROBLEMS IN DETAIL ===--- Spectro Focus infor: --------------------- im> caget tpm_TM_PMSSAvg tpm_TM_TelairAvg.VALA tpm_TM_PMSSAvg 1.00571 tpm_TM_TelairAvg.VALA 2.6375 sp1> mechSend s Coll_motor_A -3273 01P-0000003273 Coll_motor_B -3541 02P-0000003541 Coll_motor_C -3691 01P-0000003691 Coll_motor_A_status 133 01S0133 Coll_motor_B_status 133 02S0133 Coll_motor_C_status 133 01S0133 Requested_exp.time 0 Exp_time_left 0 Last_exp.time 2 Last_open_transit.time 715 Last_close_transit.time 560 Humid_Hartmann 22.0 0xFD96 Humid_Cen_Optics 15.6 0xFDEA Temp_Median 5.0 Temp_Hartmann_Top 4.4 (0xFF46) Temp_Red_Cam_Bot 5.0 (0xFF30) Temp_Red_Cam_Top 5.2 (0xFF26) Temp_Blue_Cam_Bot 5.4 (0xFF20) Temp_Blue_Cam_Top 4.9 (0xFF31) AD 30335 1294 sp2> mechSend s spMechVersion v2_9_0 Bootup 870616738 SpectroID 2 SlitID 2 Air On Coll_motor_A 8314 01P0000008314 Coll_motor_B 7652 02P0000007652 Coll_motor_C 7963 01P0000007963 Coll_motor_A_status 132 01S0132 Coll_motor_B_status 133 02S0133 Coll_motor_C_status 133 01S0133 Requested_exp.time 0 Exp_time_left 0 Last_exp.time 3 Last_open_transit.time 720 Last_close_transit.time 635 Humid_Hartmann -24.6 0x0 Humid_Cen_Optics 15.7 0xFDE8 Temp_Median 3.0 Temp_Hartmann_Top 3.3 (0xFF74) Temp_Red_Cam_Bot 3.6 (0xFF68) Temp_Red_Cam_Top 2.7 (0xFF90) Temp_Blue_Cam_Bot 3.1 (0xFF80) Temp_Blue_Cam_Top 3.1 (0xFF80) AD 6975 1094 IDL 5.5a> collimate, 49507 Reading /data/spectro/54499/sdR-b1-00049507.fit Reading /data/spectro/54499/sdR-b1-00049508.fit COLLIMATE: Min offset = -0.65 pix COLLIMATE: Max offset = 1.25 pix COLLIMATE: RMS across CCD = 0.49 pix COLLIMATE: Mean offset = 0.30 pix COLLIMATE: Camera b1 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict blue camera ring movement of -10.9 degrees for spectro-1 COLLIMATE: (if red is already in focus) Log file = Collimate-54499-b1-00049507.log PostScript file = Collimate-54499-b1-00049507.ps Files not found Reading /data/spectro/54499/sdR-r1-00049507.fit Reading /data/spectro/54499/sdR-r1-00049508.fit COLLIMATE: Min offset = -0.80 pix COLLIMATE: Max offset = 0.09 pix COLLIMATE: RMS across CCD = 0.26 pix COLLIMATE: Mean offset = -0.40 pix COLLIMATE: Camera r1 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict (red) piston movement of 3655 steps for spectro-1 COLLIMATE: Issue SOP command: "sp1; mechPiston 3655" Log file = Collimate-54499-r1-00049507.log PostScript file = Collimate-54499-r1-00049507.ps Reading /data/spectro/54499/sdR-r2-00049507.fit Reading /data/spectro/54499/sdR-r2-00049508.fit COLLIMATE: Min offset = -0.91 pix COLLIMATE: Max offset = 0.02 pix COLLIMATE: RMS across CCD = 0.26 pix COLLIMATE: Mean offset = -0.29 pix COLLIMATE: Camera r2 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict (red) piston movement of 2609 steps for spectro-2 COLLIMATE: Issue SOP command: "sp2; mechPiston 2609" Log file = Collimate-54499-r2-00049507.log PostScript file = Collimate-54499-r2-00049507.ps --------- Problems: "Standard" CCD55 volts problem reported by watcher. ~04:34Z Landru's session to SOP cancelled by its own during read out. I tried to handle sop session on "monxmike" computer, but got the same error: sp1> grabInst -handlers 1300 IOP running as Backup Relay Server requestListen: listening 4 Access to shared data area at 0x18000000 lost (errno=0x71 No route to host) Read failed from TCP icc40 (sdsssd1) rmt=0x18000000 cnt=0x000B9688 ******************************************************************************** * * * Communication between the ICC/OAC is no longer possible. * * * * Perform the following steps until operations are possible. * * * * 1. Check network connectivity between the machines. * * 2. Reboot (reset) the ICC. * * 3. Restart this (OAC) program. * * 4. Power-cycle the VMEbus crates. * * 5. Reboot the local machine (host). * * * ******************************************************************************** [dmbiz@sdsshost2 dmbiz]$ [dmbiz@sdsshost2 dmbiz]$ [dmbiz@sdsshost2 dmbiz]$ [dmbiz@sdsshost2 dmbiz]$ setup iop [dmbiz@sdsshost2 dmbiz]$ sop dscRun using system v15_13_1 Executing commands in /linuxp/prd/dervish/v8_16_3/etc/dervishStartup.tcl: Executing commands in /linuxp/prd/astrotools/v5_19/etc/astrotoolsStartup.tcl astroda OAC version v15_13_1 After this SOP prompt was closed. Attempt to log in back crashes: --- [dmbiz@sdsshost2 dmbiz]$ setup iop [dmbiz@sdsshost2 dmbiz]$ sop dscRun using system v15_13_1 Executing commands in /linuxp/prd/dervish/v8_16_3/etc/dervishStartup.tcl: Executing commands in /linuxp/prd/astrotools/v5_19/etc/astrotoolsStartup.tcl astroda OAC version v15_13_1 ERROR: Initial test of TCP link failed: Bad ICC [node=sdsssd1, crate=4, board=0] Using HVE link to ICC Error mapping shared data area (memory) Cannot open /dev/mmem ******************************************************************************** * * * Backplane communications with the ICC are not possible. * * * * Observers! Check: * * * * o Murmur logs for indications of the source of the error. * * o Connections between the machines. * * - ping sdssid1 (or sdsssd1 for sop, sdssmd1 for mop) * * - If connection is down check power switch on crates and reboot. * * o ! Only if using PTVME communication ! * * - sdsshost>ptvmeTest -n 1 -a 0x18000000 * * - If all errno = 0 All is Ok. * * - Else file any non-zero errno as bug report, * * - kill all iop processes (ps -elf) * * - sdsshost> ptvmeReset * * o Restart this (OAC) program. (IOP, SOP or MOP) * * * ******************************************************************************** [dmbiz@sdsshost2 dmbiz]$ "Serial" error popped up in wather: "Camera serial line": server is not responding (Last packet: 2008-02-03 04:36:34Z) server is not responding (Last packet: 2008-02-03 04:36:34Z) ---- Servers (software) can not be restarted. Spectro crate looks standard. We gave a call to Jon. He recommended to reboot the spectro crate any way. Rebooted. Quite long time (> 10 min) until green lights FOX1 and LINE came up. Restarted software servers. Connection to sop can be establieshed now. Took spectro bias frame. It looks normal. --- Well. after the reset transfer from VME to unix stopped. "teamster" command applied, previous process killed. Transfer is normal now. just in a case, apo_plotbias again: IDL 5.5a> apo_plotbias, 49526 Filename 02% 05% 10% 50% 90% 95% 98% ------------------- ------ ------ ------ ------ ------ ------ ------ sdR-b2-00049526.fit -8.1 -5.9 -4.7 1.4 7.0 8.8 10.8 sdR-b2-00049526.fit -8.1 -5.9 -4.7 1.4 7.0 8.8 10.8 sdR-r2-00049526.fit -8.2 -6.2 -4.1 1.2 6.5 8.5 9.6 sdR-r2-00049526.fit -8.2 -6.2 -4.1 1.2 6.5 8.5 9.6 Done for 80 sec.