APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Wednesday September 9, 2009 (MJD 55084) ---=== OBSERVING TEAM ===--- Observers: Dmitry Bizyaev Howard Brewington Support: Craig Loomis ---=== OBSERVING PLAN ===--- More guider and spectro tests! ---=== OBSERVING SUMMARY ===--- The spectro dewar needs to be changed; it last filled at 4:30 a.m. Problems with cart 10; see log. Had a problem during Kyle's SOP tests; see log. Paul Harding sent a link for the current status of the guider, viz., https://trac.sdss3.org/wiki/INFRASTRUCTURE/Guider/55084 ---=== OBSERVING LOG ===--- Afternoon: ---------- 23:20Z Started the checkout 23:30Z Restarted the the servers 23:40Z Did an imager bias 23:50Z Filesystem Size Used Avail Use% Mounted on /dev/sda3 142G 123G 13G 92% / /dev/sda1 99M 30M 64M 33% /boot /dev/sdb1 917G 828G 43G 96% /data sdssfiles1p:/data1/data 1.5T 766G 627G 56% /files hub25mp:/home 92G 47G 45G 52% /home_hub25m 00:27Z Verified that the STUI actors were happy; also checked the status of hub25m and perms. 00:30Z Checked telescope, opened doors, and started fans. 00:45Z We mounted cart 11 to prepare for spectro colilmation check. However, we could not do a loadCart. After talking with Paul Harding, we learned the database was under construction. We postponed this task. 01:00Z Paul Harding asked that we do a test on fibers 10 and 11 of cart 06. While illuminating the each fiber, we slide a piece of paper from the fiber's index pin and partly occulted the fiber's end. Then we watch the result at the guider block for each fiber. With fiber 10, the shadow moved from seven o'clock to one o'clock. With fiber 11, the shadow moved from five o'clock to eleven o'clock. Night: ------- 01:17Z Sunset with clouds and radar returns nearby; we waited. 02:41Z Twilight end 04:16Z Storms to the north kept us closed, so we worked on spectro collimation with cart 10. When we looked for cart 10 in the bay, we noticed that it had no ID numbers on its north face when resting in the cart bay. And, when we tried to mount cart 10, we heard a metallic clicking sound as the cart touched the bottom of the telescope. The noise seemed to be coming from the south-latch area of the cart. Therefore, we decided not to try latching cart 10 and moved on to cart 11 instead. 04:40Z started the collimation procedure with cart 11. The collimate numbers are in the Problems Section. 05:00Z Started Kyle's flat/arc tests. 06:49Z During Kyle's tests, we got errors that the 'BOSS ICC was dead' and the Sp1 micro rebooted. We used stageManger stop/start and an IACK to fix the problem. Craig emailed some notes on the problem. The logs have actually gotten harder to read, but the sp1 reboot boiled down to this. 2009-09-10 06:26:25.498Z root 10 Forth.py:152 sent 'DOREDERASE' 2009-09-10 06:26:25.820Z root 10 TDS.py:119 line=' ' 2009-09-10 06:26:29.099Z root 10 TDS.py:119 line='< REDERON ok >' 2009-09-10 06:26:29.579Z root 10 TDS.py:119 line='< RR˙˙' 2009-09-10 06:26:29.580Z root 10 TDS.py:119 line='' 2009-09-10 06:26:29.581Z root 10 TDS.py:119 line='' 2009-09-10 06:26:29.581Z root 10 TDS.py:119 line='' 2009-09-10 06:26:29.582Z root 10 TDS.py:119 line='' 2009-09-10 06:26:29.582Z root 10 TDS.py:119 line='' 2009-09-10 06:26:29.660Z root 10 TDS.py:119 line=' BOSS CCD CONTROLLER' 2009-09-10 06:26:29.660Z root 10 TDS.py:119 line='' A successful one shows up as: 2009-09-10 06:23:22.057Z root 10 Forth.py:152 sent 'DOREDERASE' 2009-09-10 06:23:22.379Z root 10 TDS.py:119 line=' ' 2009-09-10 06:23:25.659Z root 10 TDS.py:119 line='< REDERON ok >' 2009-09-10 06:23:25.660Z root 10 TDS.py:119 line='< REDEROFF ok > ok' 2009-09-10 06:23:26.457Z root 10 Forth.py:152 sent 'DOREDPURGE' 2009-09-10 06:23:26.778Z root 10 TDS.py:119 line=' ' 2009-09-10 06:23:28.539Z root 10 TDS.py:119 line='< PURGEON ok >' 2009-09-10 06:23:28.617Z root 10 TDS.py:119 line='< PURGEOFF ok > ok' The reboot was not noticed by the ICC, and thing went worse from there. Craig 07:06Z Started Kyle's ditheredFlats and got another error that the 'BOSS ICC was dead.' Craig set us another email about the problem. sp1 rebooted twice more. Once when the readout was being configured, and once during the iack: 2009-09-10T07:18:43.log:2009-09-10 08:53:19.844Z root 10 Forth.py:152 sent 'SFWD' 2009-09-10T07:18:43.log:2009-09-10 08:53:20.487Z root 10 Forth.py:265 got flag='i', response=' ˙˙' 2009-09-10T07:18:43.log:2009-09-10 08:53:20.488Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 08:53:20.488Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 08:53:20.489Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 08:53:20.489Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 08:53:20.490Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 08:53:20.565Z root 10 Forth.py:265 got flag='i', response=' BOSS CCD CONTROLLER' 2009-09-10T07:18:43.log:2009-09-10 09:06:56.813Z root 10 Forth.py:152 sent '2 NORMRT ! 10 WARMRT !' 2009-09-10T07:18:43.log:2009-09-10 09:06:57.053Z root 10 Forth.py:265 got flag=':', response='' 2009-09-10T07:18:43.log:2009-09-10 09:06:57.853Z root 10 Forth.py:152 sent 'PIACK IACK' 2009-09-10T07:18:43.log:2009-09-10 09:06:58.497Z root 10 Forth.py:265 got flag='i', response=' ˙˙' 2009-09-10T07:18:43.log:2009-09-10 09:06:58.497Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 09:06:58.498Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 09:06:58.498Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 09:06:58.499Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 09:06:58.500Z root 10 Forth.py:265 got flag='i', response='' 2009-09-10T07:18:43.log:2009-09-10 09:06:58.573Z root 10 Forth.py:265 got flag='i', response=' BOSS CCD CONTROLLER' - craig 08:04Z We got yet another 'BOSS ICC was dead' error. Craig thinks this one is result of the camCheck routine. 07:27Z Closed doors and turned off fans as humidity increased. 09:06Z Did another BOSS IACK after ditheredFlats failed on sp2. 10:02Z At Craig's request, we did not try any more ditheredFlats. We did a bias instead; however, it failed also. Craig fixed the problem, and we were able to take bias frames. 11:21Z Here's a list of the night's tests for Kyle ------------------------------------- Frames Flavor Exptime Comment ------------------------------------- 100955 ignore wrong, lamps off 100956 flat 1 100957 flat 2 100958 flat 3 100959 flat 4 100960 flat 15 100961 flat 24 100962 flat 32 100963 arc 1 100964 arc 1 100965 arc 1 100966 arc 2 100967 arc 2 100968 arc 2 100969 arc 4 100970 arc 4 100971 arc 4 100972 arc 8 100973 arc 8 100974 arc 8 100975 dead, no such exposure, iack-ed 100976 arc 16 100977 arc 16 100978 arc 16 100879-984 dFlat 1s -- attemp to make ditheredFlat for SP1, expTime = 1s Some images look useful, but the batch was interrupted by error "boss ICC is dead..." 100985 - dFlat 1s default sequence of dithered flats for SP1 move 101006 101007 - dflat 1s attempt to do dithered flats for SP2 mech. move 101011 interrupted by errors "boss ICC is dead..." 101012 - 101022 - if exist, results of Craig experiments. Ignore. 101023 - 101042 bias 0 sequence of 20 biases 11:20Z Morning twilight 11:32Z endNight finished and was happy! 12:43Z Sunrise ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 8026 23:41Z 00:13Z 100 O 53.69 61.66 57 ignore ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): QA Procedures Done (y/n): UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 22:22Z 55 47 9 18 (NNE) 802 - 22:54Z 57 48 7 72 (ENE) 682 - 23:26Z 59 45 6 93 (E) 652 - 23:58Z 59 44 7 90 (E) 720 - 00:31Z 60 44 8 89 (E) 680 - 01:03Z 58 43 6 84 (E) 737 - 01:35Z 56 43 2 97 (E) 729 - 02:07Z 56 43 1 68 (ENE) 768 - 02:39Z 56 43 20 73 (ENE) 865 - 03:11Z 55 45 15 80 (E) 1236 - 03:44Z 55 47 14 78 (ENE) 1259 - 04:16Z 53 46 14 67 (ENE) 1154 - 04:48Z 53 46 15 77 (ENE) 1107 - 05:20Z 53 45 16 73 (ENE) 994 - 05:52Z 52 45 16 68 (ENE) 960 - 06:24Z 52 45 19 73 (ENE) 843 - 06:57Z 51 45 18 70 (ENE) 798 - 07:29Z 49 47 10 359 (NNW) 1178 - 08:01Z 51 46 13 48 (NE) 934 - 08:33Z 48 48 2 29 (NNE) 1290 - 09:05Z 49 47 8 74 (ENE) 1015 - 09:37Z 49 47 20 75 (ENE) 934 - 10:10Z 49 47 19 80 (E) 965 - 10:42Z 48 47 13 79 (E) 893 - 11:14Z 48 46 17 63 (ENE) 937 - ---=== TELESCOPE STATUS ===--- 00:30Z Doors and louvers opened, fans on. 07:27Z Enclosure on, doors and louvers closed, fans off. No interlocks bypassed. Status at: 11:55Z Telescope stowed at: 30,121,00 mount Instrument mounted: 11 Cartridge Counterweights at: 250 unit LN2 autofill systems: Connected and turned on. 180L LN2 dewar scales: Spectro 42lbs, 21 psi Imager 188lbs, 19 psi ---=== SOFTWARE USED ===--- IOP/SOP: v4_40_0 Watcher: v2_36_0 MCP: NOSVN TPM: tpm_v3_5_0 AstroDa: v15_13_1 TCC: TCC 2.11.0 2009-06-24 sdssProcedures: v2_44 SoS: v5_1_8 plate-mapper: v4_3_1 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A -0.6930 -0.6940 Axial B -0.0300 -0.0410 Axial C -0.3700 -0.3710 Trans D 11.6140 11.6170 Lateral E 10.3759 10.3505 Lateral F 11.9380 11.9253 GALILS Commanded: 9900. 400. 5850. 1200. 2950. 2750. Actual: 9900. 400. 5850. 1200. 2950. 2750. SETMIR VALUES PriDesOrient: 0.00 -3.30 5.10 278.08 178.06 PriOrient: 0.00 -3.04 5.26 277.70 178.07 SECONDARY: ---------- Focus: 0.00 Air Temp.: 16.2 Alt.: 30.001358 MIGS TONIGHT NOMINAL Axial A 18.6210 18.6150 Axial B 15.6270 15.6120 Axial C 13.8420 13.8170 Trans D -11.7910 -11.7780 GALILS Commanded: 1597576. 1566200. 1588134. -7600. -12400. Actual: 1597700. 1566200. 1588100. -7600. -12400. SETMIR VALUES SecDesOrient: 1257.00 -4.00 -9.00 10.00 234.42 SecOrient: 1256.97 -4.03 -8.95 10.22 234.71 PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A -0.6930 -0.6940 Axial B -0.0300 -0.0410 Axial C -0.3710 -0.3710 Trans D 11.6130 11.6170 Lateral E 10.3759 10.3505 Lateral F 11.9380 11.9253 GALILS Commanded: 9900. 400. 5850. 1200. 2950. 2750. Actual: 9917. 401. 5832. 1205. 2954. 2757. SETMIR VALUES PriDesOrient: 0.00 -3.30 5.10 278.08 178.06 PriOrient: 0.00 -3.04 5.26 277.70 178.07 SECONDARY: ---------- Focus: 0.00 Air Temp.: 15.2 Alt.: 29.999993 MIGS TONIGHT NOMINAL Axial A 18.6200 18.6150 Axial B 15.6270 15.6120 Axial C 13.8420 13.8170 Trans D -11.7900 -11.7780 GALILS Commanded: 1597658. 1566282. 1588128. -7600. -12400. Actual: 1597721. 1566188. 1588095. -7597. -12380. SETMIR VALUES SecDesOrient: 1257.00 -4.00 -9.00 10.00 234.42 SecOrient: 1257.02 -4.00 -8.95 10.22 234.69 ---=== PROBLEMS IN DETAIL ===--- IDL> collimate, 100950 Reading /data/spectro/55084/sdR-b1-00100950.fit SDSSPROC: Read noise = 1.69728 1.74019 1.73973 1.79123 electrons Reading /data/spectro/55084/sdR-b1-00100951.fit SDSSPROC: Read noise = 1.69788 1.74027 1.73656 1.79910 electrons COLLIMATE: Min offset = -2.60 pix COLLIMATE: Max offset = 0.80 pix COLLIMATE: RMS across CCD = 0.71 pix COLLIMATE: Mean offset = -0.54 pix COLLIMATE: Camera b1 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict blue camera ring movement of 19.2 degrees for spectro-1 COLLIMATE: (if red is already in focus) Log file = Collimate-55084-b1-00100950.log PostScript file = Collimate-55084-b1-00100950.ps Reading /data/spectro/55084/sdR-b2-00100950.fit SDSSPROC: Read noise = 1.72099 1.94751 2.01757 1.99879 electrons Reading /data/spectro/55084/sdR-b2-00100951.fit SDSSPROC: Read noise = 1.73898 1.93740 1.98779 2.00931 electrons COLLIMATE: Min offset = -1.20 pix COLLIMATE: Max offset = 1.20 pix COLLIMATE: RMS across CCD = 0.64 pix COLLIMATE: Mean offset = -0.32 pix COLLIMATE: Camera b2 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict blue camera ring movement of 11.6 degrees for spectro-2 COLLIMATE: (if red is already in focus) Log file = Collimate-55084-b2-00100950.log PostScript file = Collimate-55084-b2-00100950.ps Reading /data/spectro/55084/sdR-r1-00100950.fit SDSSPROC: Read noise = 1.93613 2.20802 2.15765 8.68700 electrons Reading /data/spectro/55084/sdR-r1-00100951.fit SDSSPROC: Read noise = 1.90218 2.24470 2.10725 8.66940 electrons COLLIMATE: Min offset = -1.60 pix COLLIMATE: Max offset = 1.00 pix COLLIMATE: RMS across CCD = 0.70 pix COLLIMATE: Mean offset = 0.02 pix COLLIMATE: Camera r1 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Predict (red) piston movement of -142 steps for spectro-1 COLLIMATE: Issue SOP command: "sp1; mechPiston -142" Log file = Collimate-55084-r1-00100950.log PostScript file = Collimate-55084-r1-00100950.ps Reading /data/spectro/55084/sdR-r2-00100950.fit SDSSPROC: Read noise = 2.41375 2.59730 2.54552 2.60022 electrons Reading /data/spectro/55084/sdR-r2-00100951.fit SDSSPROC: Read noise = 2.44746 2.59982 2.55053 2.63900 electrons COLLIMATE: Min offset = -1.20 pix COLLIMATE: Max offset = 2.80 pix COLLIMATE: RMS across CCD = 0.51 pix COLLIMATE: Mean offset = -0.10 pix COLLIMATE: Camera r2 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Predict (red) piston movement of 886 steps for spectro-2 COLLIMATE: Issue SOP command: "sp2; mechPiston 886" Log file = Collimate-55084-r2-00100950.log PostScript file = Collimate-55084-r2-00100950.ps % Program caused arithmetic error: Floating illegal operand IDL>