APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Friday August 6, 2004 (MJD 53224) ---=== OBSERVING TEAM ===--- Jurek Krzesinski (swing) Dan Long Steph Snedden Support: Jon Brinkmann Craig Loomis Jim Gunn ---=== OBSERVING PLAN ===--- Last night of shakedown ---=== OBSERVING SUMMARY ===--- Collimation adjusted (but still needs a final check). Spectro systems checked out. One plate done. New noise in the bias across all imager dewar 1 CCDs. IOP still needs to be tested on-sky for both survey and SEGUE imaging. IOP v3_140_0 is still test. ---=== OBSERVING LOG ===--- Afternoon checkout: ------------------- DA ok, spectrograps in spec., telescope ok. imager problem: While taking bias inside the dog house, we've seen a strange pattern on display 1 (dewar 1 all filters). We've seen that pattern a few times. We've pulled out the last frame (33) from dewar 1 where the pattern was present and confirmed that it was real. We've checked power supply on dewar 1, but everything was all right at that end. Jon was notified, We've run bias again and seen that pattern again. Since it wasn't just a glitch, we decided to replace dewar 1 power supply, followed by cc cold, iack and setImagerAmpsOn commands. This problem persisted...see the "Night" section below for a description of the tests we ran on power supplies. guider problem: We couldn't get a guider frame with gcam doread command. Problem was traced to a broken A2S converter inside the guider box. Craig replaced A2S with a spare one. After that we had to restart GCAMUSER process from tcc, then we got the guider working (see problem section for details). Mirrors: Primary mirror E,F secondary B,C axies were quite a bit out of spec. Mirrors were homed and relaxed. Why it went out of spec, no one knows. Dust counter: Never came back down from last night's dust incident. It is reading approx. 10X too high (and no, it isn't real). Night: ------ We delayed opening because of nearby clouds, lightning and climbing dust counts. 01:52Z: id 6-2 suffered a rimfire failure. errors are at the end of the log. We cleaned the tape drive as a matter of course, remounted and enabled the drive, but it failed again on a subsequent drift. This time, we replaced the tape itself. That didn't work, as id2 then failed to mount properly. We ended up resetting both photo crates. Everything was fine after that. 02:34Z: We went on-sky for spectro checkout with cartridge 3, plate 1659. We had guide stars on goToField and decent seeing (even without a final collimation tweak). The guider stopped spontaneously without error messages during the second exposure. Dan restarted the guider and it ran without problems. Finished the plate with a total exposure of 2700 s. 03:08Z: We kept running biases to monitor dewar 1 after the power supply swap, and unfortunately, after a while we saw the low frequency bias variation again. We called Jon and Jim. After talking to both, we put in the last working power supply. After running a bias for a while we saw the problem a third time. Our final test was to swap the power supplies to photo 1 and 2 to see if the bias noise stayed with dewar 1 or not. It appeared to, although the noise was much less. Frames with noise were archived to /data/photo/optics/53224 on sdsshost. Note that we also saw an odd/even pattern in some of the r1 frames. The fact that the noise did not migrate to dewar 2, but stayed with dewar 1 after the swap implies the problem is not in the power supplies themselves. Dan suggests that the problem may lie in the connecions to the photo 1 power supply...seems very reasonable that the connections should be checked next. A critical-high PR was filed. 06:50Z: Lightning to the east of us forced us to close. 08:10Z We opened again, and mounted the imager for collimation. Good collimation was achieved with tilts of -20 in x and y. Here are the details of the tilts and translation: Primary starting point: 0 -19.6 10.7 586.5 1069.1 After tilts to eliminate coma 0 -40 -10 586.5 1069.1 Converted back to new tilts and translation (new setmir coords) 0 -3.6 26.9 1470.4 198.0 At this point, clouds rolled in and we had to close. So, the new values still have to be checked on-sky to make sure the collimation is still good. We got to see a few donuts before the clouds got us and we believe that the collimation will be good. DIMM: ----- The new keyboard appears to work well. DIMM is ready to go for Polaris mode. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 4746 22:35Z 22:51Z 100 O 3.84 7.84 41 ignore lots of biases 4747 00:17Z 00:59Z 100 O 29.27 39.78 84 ignore in doghouse to 4748 01:29Z 02:08Z 100 O 47.33 57.32 81 ignore check for noise 4749 03:09Z 03:19Z 100 O 72.52 74.88 30 ignore 4750 04:24Z 04:43Z 100 O 91.31 96.05 46 ignore 4751 04:58Z 05:37Z 100 O 99.84 109.52 79 ignore 4752 06:02Z 07:13Z 100 O 115.90 133.67 133 ignore 4753 07:15Z 07:49Z 100 O 134.16 142.59 68 ignore 4754 08:19Z 09:32Z 82 N 150.40 170.39 135 engineering ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- 4754 13 4 52 -1.800 0.198 -1.900 0.158 0.00252 180 57 4754 18 5 62 -1.900 0.213 -2.000 0.713 0.00165 180 57 ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ 4754 22 1 57 0 23 0.516 0.217 -1.174 0.210 0.00395 4754 22 1 57 0 26 0.275 0.153 -1.383 0.393 0.00465 4754 22 1 57 0 24 0.266 0.267 -1.001 0.303 0.00336 4754 22 1 57 0 23 -0.284 0.182 -1.086 0.192 0.00365 4754 22 1 57 0 18 -0.573 0.314 -1.007 0.276 0.00339 ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n):y QA Procedures Done (y/n):y UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 27658, plate -9999 ------- 22:27 27658 0.0 bias ----- sequence 27659, plate 1659 ------- spectro focus check 22:46 27659 0.0 bias 22:50 27660 10.0 flat 22:53 27661 2.0 arc 23:06 27662 10.0 flat 23:17 27663 2.0 arc 23:23 27664 2.1 arc 23:25 27665 2.1 arc ----- sequence 27666, plate -9999 ------- 01:31 27666 0.0 bias ----- sequence 27667, plate 1659 ------- 02:54 27667 10.0 flat 02:56 27668 2.0 arc 03:15 27669 900.1 target 03:33 27670 900.1 target 03:51 27671 900.1 target 03:54 27672 10.0 flat 03:56 27673 2.0 arc apo_plotbias results: Filename 02% 05% 10% 50% 90% 95% 98% ------------------- ------ ------ ------ ------ ------ ------ ------ sdR-b1-00027666.fit -6.4 -4.2 -3.0 2.6 8.2 9.4 11.6 sdR-b1-00027666.fit -6.4 -4.2 -3.0 2.6 8.2 9.4 11.6 sdR-b2-00027666.fit -8.4 -7.0 -4.4 0.9 7.5 10.1 11.5 sdR-b2-00027666.fit -8.4 -7.0 -4.4 0.9 7.5 10.1 11.5 sdR-r1-00027666.fit -7.1 -5.4 -3.6 1.6 6.7 9.0 10.6 sdR-r1-00027666.fit -7.1 -5.4 -3.6 1.6 6.7 9.0 10.6 sdR-r2-00027666.fit -7.8 -6.3 -4.6 0.7 6.7 8.0 10.1 sdR-r2-00027666.fit -7.8 -6.3 -4.6 0.7 6.7 8.0 10.1 ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 02:52Z 3 1659 -20.40 -0.0022 77.11 0.0021 161.66 0.0227 1.000230 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL7678 Stays: JL7679 ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 02:52Z 3 1659 1.00023 -2271 -859 -100 -20 77.1 17.4 5 1 - 1.2 ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 22:16Z 70 42 12 273 (W) 90729 - pcl 22:47Z 72 43 17 298 (WNW) 92527 - " 23:17Z 73 45 13 265 (W) 133099 - " 23:48Z 72 43 12 272 (W) 131866 - " 00:20Z 69 43 6 269 (W) 145189 - " 00:52Z 70 45 9 267 (W) 136315 - " 01:25Z 66 44 7 275 (W) 137754 - " 02:02Z 64 45 4 353 (NNW) 148021 - " 02:35Z 63 45 3 20 (NNE) 145359 - " 03:05Z 63 45 5 336 (NNW) 142284 - " 03:35Z 63 45 5 352 (NNW) 139518 - " 04:07Z 63 45 9 10 (N) 147618 - " 04:40Z 62 45 6 4 (N) 138631 - " 05:12Z 63 46 3 1 (N) 141723 - " 05:44Z 62 46 4 9 (N) 135241 - " 06:16Z 62 43 11 12 (NNE) 138210 - " 06:49Z 62 42 9 4 (N) 132564 - " 07:22Z 60 44 13 17 (NNE) 132005 - " 07:53Z 61 40 16 33 (NNE) 128932 - " 08:25Z 61 40 11 41 (NE) 129019 - " 08:58Z 61 40 12 12 (NNE) 129495 - mcl 09:30Z 61 41 12 22 (NNE) 126687 - " 10:02Z 61 41 12 32 (NNE) 127956 - " 10:35Z 62 40 9 356 (NNW) 127314 - " ---=== TELESCOPE STATUS ===--- 01:30Z Started enclosure ventilation. 02:30Z Opened encosure 06:50Z Closed back up, but kept ventilating 08:10Z We opened back up again 10:00Z Closed enclosure. Fans off. Cartridge 5 mounted. Counterweights at 240. Spectro autofill system connected and fill forced. 10:53Z Spectro dewar weight: 147 10:53Z Imager dewar weight: 190 ---=== SOFTWARE USED ===--- IOP/SOP: v3_140_0 Watcher: v2_29_0 MCP: v5_25_0 TPM: tpm_v2_42_0 AstroDa: v14_47 TCC: TCC 2.7.2.1 August 6 2004 sdssProcedures: v1_92 SoS: v4_10_7 hoggPT: v1_6_9 plate-mapper: v4_3_1 ---=== MIRROR NUMBERS ===--- Before homing and relaxing M1: ----------------------------- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.4490 0.4450 Axial B 0.7010 0.6770 Axial C 1.1270 1.0980 Trans D -9.5490 -9.5280 Lateral E 10.6045 10.6172 out of spec! Lateral F 10.5791 10.6172 out of spec! GALILS Commanded: 5900. -3250. 2600. 7850. 10150. 9750. Actual: 5900. -3250. 2600. 7850. 10150. 9750. SETMIR VALUES PriDesOrient: 0.00 -19.60 10.70 586.50 1069.10 PriOrient: 0.00 -19.75 10.53 586.52 1069.84 SECONDARY: ---------- Focus: 0.00 Air Temp.: 20.6 Alt.: 70.000020 MIGS TONIGHT NOMINAL Axial A 1.5990 1.5100 Axial B 1.2060 1.0950 out of spec Axial C 1.1670 1.1110 out of spec Trans D 0.2400 0.0020 GALILS Commanded: 1498566. 1412815. 1454711. 5300. 500. Actual: 1470100. 1389650. 1415950. 5300. 500. SETMIR VALUES SecDesOrient: 1131.25 0.00 -25.00 0.00 -66.66 SecOrient: 1155.17 -4.78 -25.69 -0.74 -63.64 Mig values, after homing and relaxing M1: ----------------------------------------- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.4550 0.4450 Axial B 0.6800 0.6770 Axial C 1.1080 1.0980 Trans D -9.5460 -9.5280 Lateral E 10.5918 10.6172 Lateral F 10.5791 10.6172 GALILS Commanded: 5900. -3250. 2600. 7850. 10150. 9750. Actual: 5890. -3255. 2615. 7836. 10164. 9749. SETMIR VALUES PriDesOrient: 0.00 -19.60 10.70 586.50 1069.10 PriOrient: 0.00 -19.75 10.53 586.52 1069.84 SECONDARY: ---------- Focus: 0.00 Air Temp.: 18.7 Alt.: 48.634377 MIGS TONIGHT NOMINAL Axial A 1.4970 1.5100 Axial B 1.1090 1.0950 Axial C 1.0880 1.1110 Trans D 0.0700 0.0020 GALILS Commanded: 1628657. 1545677. 1572310. -3900. -8750. Actual: 1628641. 1548187. 1574504. -3899. -8726. SETMIR VALUES SecDesOrient: 1257.00 0.00 -25.00 0.00 146.27 SecOrient: 1255.76 0.18 -25.82 -0.28 146.44 After collimation tilts: PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.5260 0.4450 Axial B 0.7130 0.6770 Axial C 0.9990 1.0980 Trans D -8.6780 -9.5280 Lateral E 11.5062 10.6172 Lateral F 11.4808 10.6172 GALILS Commanded: 4850. -3650. 4050. -6150. 40200. 39150. Actual: 4850. -3650. 4050. -6150. 40200. 39150. SETMIR VALUES PriDesOrient: 0.00 -3.60 26.90 1470.40 198.00 PriOrient: 0.00 -3.80 26.76 1470.93 196.30 SECONDARY: ---------- Focus: -282.90 Air Temp.: 16.9 Alt.: 30.311346 MIGS TONIGHT NOMINAL Axial A 1.7810 1.5100 Axial B 1.3910 1.0950 Axial C 1.3710 1.1110 Trans D 0.0630 0.0020 GALILS Commanded: 1272236. 1191808. 1218100. -3950. -8750. Actual: 1272250. 1191800. 1218100. -3950. -8750. SETMIR VALUES SecDesOrient: 974.10 0.00 -25.00 0.00 146.27 SecOrient: 974.11 0.01 -24.99 -0.41 146.19 New nominal values applied: PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.5260 0.5260 Axial B 0.7130 0.7130 Axial C 0.9990 0.9990 Trans D -8.6780 -8.6780 Lateral E 11.5062 11.5062 Lateral F 11.4808 11.4808 GALILS Commanded: 4850. -3650. 4050. -6150. 40200. 39150. Actual: 4837. -3652. 4065. -6127. 40183. 39144. SETMIR VALUES PriDesOrient: 0.00 -3.60 26.90 1470.40 198.00 PriOrient: 0.00 -3.80 26.76 1470.93 196.30 SECONDARY: ---------- Focus: -282.90 Air Temp.: 16.9 Alt.: 30.311346 MIGS TONIGHT NOMINAL Axial A 1.7810 1.7810 Axial B 1.3910 1.3910 Axial C 1.3710 1.3710 Trans D 0.0630 0.0630 GALILS Commanded: 1272210. 1191751. 1217993. -3950. -8750. Actual: 1272233. 1191779. 1218096. -3935. -8772. SETMIR VALUES SecDesOrient: 974.10 0.00 -25.00 0.00 146.27 SecOrient: 974.06 0.03 -25.01 -0.42 146.17 ---=== PROBLEMS IN DETAIL ===--- Rimfire failure on id 6-2: ------------------------- dsc I archFailMsg Rimfire failure ID = 2 Status = 0xf10c0300 0x1194081 (dsc_I_archFailMsg Rimfire failure ID = 2 Status = 0xf10c0300 0x1194081 : 2004-08-07 01:52:14Z ) dsc E trcErr dscArch CheckPrevOp: rstatus = 0x800140fc, drive 2 (dsc_E_trcErr dscArch_CheckPrevOp: rstatus = 0x800140fc, drive 2 : 2004-08-07 01:52:14Z ) archDrvSts (drive 2: Error: 2004-08-07 01:53:04Z) archLastErr-2 (Drive 2 error: Medium Error: Write Error : isc=81: 2004-08-07 01:53:04Z ) dsc E archShutDown2 Archiver shutting down second drive (dsc_E_archShutDown2 Archiver shutting down second drive : 2004-08-07 01:52:14Z) dsc E archWrFail Tape write error while archiving frame 4748 40.46 (dsc_E_archWrFail Tape write error while archiving frame 4748 40.46 : 2004-08-07 01:52:14Z ) archUseDrv (drive 2: archUseDrv is 0: 2004-08-07 01:53:04Z) Spectrographs checkout: ----------------------- sp1> mechSend s Dropping at exit spMechVersion v2_5_0 Bootup 719771169 SpectroID 1 SlitID 35 Air On Shutter_open_sensor Off Shutter_closed_sensor On Left_open_sensor Off Left_closed_sensor On Right_open_sensor Off Right_closed_sensor On Coll_motor_A 2625 Coll_motor_B 3605 Coll_motor_C 2123 Requested_exp.time 0.0 Exp_time_left 0.0 Last_exp.time 2.1 Exp_state None Shutter_open_transit .70 Shutter_close_transit .52 [jurek@landrupe jurek]$ cat co collimation cor.ps [jurek@landrupe jurek]$ cat collimation sp1> mechSend s Dropping at exit spMechVersion v2_5_0 Bootup 719771169 SpectroID 1 SlitID 35 Air On Shutter_open_sensor Off Shutter_closed_sensor On Left_open_sensor Off Left_closed_sensor On Right_open_sensor Off Right_closed_sensor On Coll_motor_A 2625 Coll_motor_B 3605 Coll_motor_C 2123 Requested_exp.time 0.0 Exp_time_left 0.0 Last_exp.time 2.1 Exp_state None Shutter_open_transit .70 Shutter_close_transit .52 Coll_motor_A_status 0x85 Coll_motor_B_status 0x85 Coll_motor_C_status 0x85 sp2> mechSend s Dropping at exit spMechVersion v2_5_0 Bootup 719771169 SpectroID 2 SlitID 3 Air On Shutter_open_sensor Off Shutter_closed_sensor On Left_open_sensor Off Left_closed_sensor On Right_open_sensor Off Right_closed_sensor On Coll_motor_A 2859 Coll_motor_B 650 Coll_motor_C -1236 Requested_exp.time 0.0 Exp_time_left 0.0 Last_exp.time 2.1 Exp_state None Shutter_open_transit .70 Shutter_close_transit .61 Coll_motor_A_status 0x85 Coll_motor_B_status 0x85 Coll_motor_C_status 0x85 im> caget tpm_TM_PMSSAvg tpm_TM_TelairAvg.VALA tpm_TM_PMSSAvg 16.9521 tpm_TM_TelairAvg.VALA 16.1425 All NGOODFIBERs are in yellow. XMIN for b2,r2 in yellow but I'm correcting anyway (6 pixels to the right) sp2> mechMoveColl -600 -600 600 XMAX for r1 in yellow sp1> mechMoveColl 200 200 -200 Now XMIN & XMAX are black. Arcs in spec (all black) Collimation - all in spec, no changes were necessary: ----------------------------------------------------- IDL 5.5a> collimate, 27664 Reading /data/spectro/53224/sdR-b1-00027664.fit SPHDRFIX1: Changing QUALITY=excellent -> test Reading /data/spectro/53224/sdR-b1-00027665.fit COLLIMATE: Min offset = -0.67 pix COLLIMATE: Max offset = 0.30 pix COLLIMATE: RMS across CCD = 0.23 pix COLLIMATE: Mean offset = -0.10 pix COLLIMATE: Camera b1 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Predict blue camera ring movement of 3.5 degrees for spectro-1 COLLIMATE: (if red is already in focus) Log file = Collimate-53224-b1-00027664.log PostScript file = Collimate-53224-b1-00027664.ps Reading /data/spectro/53224/sdR-b2-00027664.fit SPHDRFIX1: Changing QUALITY=excellent -> test Reading /data/spectro/53224/sdR-b2-00027665.fit COLLIMATE: Min offset = -0.39 pix COLLIMATE: Max offset = 0.61 pix COLLIMATE: RMS across CCD = 0.27 pix COLLIMATE: Mean offset = 0.15 pix COLLIMATE: Camera b2 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict blue camera ring movement of -5.4 degrees for spectro-2 COLLIMATE: (if red is already in focus) Log file = Collimate-53224-b2-00027664.log PostScript file = Collimate-53224-b2-00027664.ps Reading /data/spectro/53224/sdR-r1-00027664.fit SPHDRFIX1: Changing QUALITY=excellent -> test Reading /data/spectro/53224/sdR-r1-00027665.fit COLLIMATE: Min offset = -0.42 pix COLLIMATE: Max offset = 0.73 pix COLLIMATE: RMS across CCD = 0.32 pix COLLIMATE: Mean offset = 0.05 pix COLLIMATE: Camera r1 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Predict (red) piston movement of -458 steps for spectro-1 COLLIMATE: Issue SOP command: "sp1; mechPiston -458" Log file = Collimate-53224-r1-00027664.log PostScript file = Collimate-53224-r1-00027664.ps Reading /data/spectro/53224/sdR-r2-00027664.fit SPHDRFIX1: Changing QUALITY=excellent -> test Reading /data/spectro/53224/sdR-r2-00027665.fit COLLIMATE: Min offset = -0.66 pix COLLIMATE: Max offset = 0.14 pix COLLIMATE: RMS across CCD = 0.23 pix COLLIMATE: Mean offset = -0.15 pix COLLIMATE: Camera r2 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict (red) piston movement of 1396 steps for spectro-2 COLLIMATE: Issue SOP command: "sp2; mechPiston 1396" Log file = Collimate-53224-r2-00027664.log PostScript file = Collimate-53224-r2-00027664.ps -------------------------------------------------------------- Guider problem: --------------- While trying to get a guider image we saw an error: DeadProc="T_GCAMUSER" We traced the problem to dead A2S converter at the telescope guider box (yellow LED was on on that A2S which indicates that something is bad with the converter). Craig replaced the converter with a spare one (after switching the new A2S on we've seen green LED). Issuing gcam doread command from tcc still showed an error: DeadProc="T_GCAMUSER" but Craig said that we have to restart GCAMUSER: Login to tcc do not start telrun, then: $ restart gcamuser you'll see: STOP T_GCAMUSER %SYSTEM-W-NONEXPR, nonexistent process %RUN-S-PROC_ID, identification of created process is 000022B1 now from telrun we issued gcam doread command and we've got the guider working. ------------------------------------------------------------------------------