APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Friday September 26, 2003 (MJD 52909) ---=== OBSERVING TEAM ===--- Atsuko Nitta Pete Newman Mike Harvanek (swing) ---=== OBSERVING PLAN ===--- Science! ---=== OBSERVING SUMMARY ===--- Segue 1220N 276 --> 328 Segue 1220S 296 --> 340 We had a rotator aborta the end of 1220S scan. We do not know what caused the abort and would like to know why this happened. 82S -158--> -126 This data is probably not good. We had problem getting skippy to run and we could not get skyGang to work to verify our position. We did not notice the AZ's "too large to correct fiducial" error when we slewed the telescope (we saw it in murmur log, but we do not remember seeing it in watcher). By the time we found this out, we did not have enough time before twilight to re-do the scan. ---=== OBSERVING LOG ===--- Afternoon: Everything went smoothly. Spectro collimation was not checked because the imager was mounted. Evening: 01:27Z The servers died and murmurlog complained about not being able to get gangs for id1. ptvmeReset fixed the problem. 01:55Z Pete mostly humbly asked the Qball if we will be imaging all night long. Qball's devine words were that only if Pete believes in himself. We now know whom to blaim if anything ANYTHING that will prevent us from having all night long imaging tonight. 02:00Z Run 4134: Segue 1220N ------------------------------- The stripeAxisRate shows that the ROT and AZ will be going quite fast (several times the siderial rate) few hours in the scan. So we will have to stop when the axis rate becomes too high. Frame 13: skippy says all is well. Frame 23: In focus. Frame 53 SEGlong=274: Ramp up starts here. Cloud cameara shows that the std dev is around 7.0, going in and out of being photometric/non-photometric boundary. Although the seeing run claims that the seeing is in yellow at frame 53, the seeing values are below 1.5" and therfore we believe the data is good from the ramp-up. Around 02:40Z: Cloud camera says we are definitely photometric. Frame 544 dewar 5 has a very distorted galaxy behind another galaxy. ~4:45Z to 5:50Z: Cloud camera shows the std dev is slightly above 7.0 again. We saw no clouds. Frame 415 SEGLon=328: endDrift. The AZ is velocity isat 47"/s and ROT is moving at -39.0"/s. And moving faster as we scan. We decided to go and do the South part of this strip, but from the middle (SEGLong=296) of ths stripe instead of the beginning. This will be equivalent to scanning this stripe from the beginning at 03:50Z which is within Steve Kent's designated start the scan time period. We should be able to finish the second half of 1220S and avoid having to go through zenith and having AZ and ROT move too fast. 06:10Z Run 4135 Sgue 1220S start SEGLong 296 ---------------------------------------------- When starting focus loop for this run, Pete saw an error message which did not seem to have any effect on the focus loop. (please see problem section for the eroror message). Frame 20 SEGLong=298 :start of science. skippy and seeing are good from the start of the run. 09:15Z Frame 317 SEGLong=342.6 Within few degrees of finishing this scan, we got an ROT abort. We were at rot~ 101 therefore we were not close to the wrap limit. BTW, the interlock display did not show anything in red when this abort took place -- why? The murmurlog message that seems to be related to this abort event is in the problem section. We would like to know for sure what caused this problem and how we could have avoided this. During this scan, the axis scan rate said we did not exceed 45"/s and when the abort happened, we were well below the maximum rate the telescope tracked during this scan. PR was filed. We ended this drift and went to the next scan. Run 4136 82S Lambda=-158 -------------------------- In focus, seeing are all good from the beginning of this run. We found that skippy has been failing after the first(ran on frame 60) one until frame 135. We note that when skipy ran successfully, there were only 3 stars that it used to calculate positions. We ran ltMatch by hand (it has not been running automatically) and found that the rotator position is in spec. We noticed that right before the slew ended, AZ had a too large to correct message in the murmur log although we did not see it in the watcher (could we both have missed/ignored it?). We probably missed the murmur message because we were both trying to find out why the rotator aborted at the last scan when we started this scan. We tried running it on 10 frames instead of the default 4 frame. It still only found 2 stars or failed. The nuErr reported by skippy when it runs were widly different (from -1" to -17.8"). Considering the fact that one of the nuErr were close to allowed upper limit and we saw the fiducial too large to correct error, we are too close to being off or completely off in nu which is making it hard for skippy to run. We tried to run skyGang to check the position, but it failed (PR filed). The current astrom, previous version of astrom, previous version of astrotools were setup to try to get skyGang to run, but we could not get it to work. We are afraid that this data might not be good. Probably we have position error. At this point, we have not much time before twilight and no point in redoing any of this scan. The last 13min of the run is in twilight, but the watcher did not complain on the sky background level. Frame 214 lambda=-126: end of science Frame 225 lambda=-124.4: endDrift Run 4137 donuts ------------------ Frame 10 focus=-371 in-focus Frame 14 focus=-771 -400 out of focus Frame 17 focus=29 +400 out of focus The frames are saved in the usual place. Before closing, we forced the fiducials and found that indeed the AZ was off and this 82S scan is not good. 12:00Z Closed. endNight is running and we will not wait for it to complete. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 4131 23:18Z 23:35Z 100 O 64.12 68.48 43 bias 4133 01:01Z 01:04Z 1220 N 91.01 91.72 19 ignore 4134 02:00Z 06:04Z 1220 N 82.01 143.25 423 science 4135 06:09Z 09:20Z 1220 S 111.11 158.83 331 science 4136 09:33Z 11:44Z 82 S -157.10 -124.21 233 science 4137 11:47Z 11:54Z 82 S -123.47 -121.71 26 engineering ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- 4134 13 4 98 -7.100 0.153 2.400 0.196 -0.00216 345 41 4134 33 4 96 -6.300 0.384 2.200 0.405 -0.00286 348 41 4134 66 4 125 -5.800 0.248 1.900 0.163 -0.00245 353 43 4134 118 4 155 -5.700 0.373 1.300 0.334 -0.00280 0 46 4134 170 4 122 -4.800 0.233 0.700 0.267 -0.00285 7 50 4134 224 4 94 -3.900 1.301 -0.100 0.746 -0.00231 14 56 4134 277 4 88 -3.700 0.977 -0.800 0.770 -0.00002 21 62 4134 330 4 117 -3.800 0.481 -2.300 0.137 0.00250 28 69 4134 383 4 147 -2.800 0.525 -2.100 0.502 0.01519 36 77 4135 12 4 87 -8.100 1.092 -0.100 0.547 -0.00363 350 54 4135 17 4 66 -8.300 0.683 -0.400 0.863 -0.00351 350 55 4135 68 4 52 -9.400 1.540 -2.600 0.415 0.00011 345 61 4135 120 4 53 -4.600 1.409 -5.400 1.276 0.00067 333 67 4135 170 4 128 -2.300 0.714 -7.400 0.172 0.00705 313 69 4135 223 4 130 2.200 0.153 -7.600 0.154 0.00669 290 68 4135 275 4 113 5.300 0.459 -7.100 0.448 0.00272 273 63 4136 60 4 3 31.200 0.472 7.300 1.494 0.02092 207 54 4136 135 4 3 -18.500 0.006 -17.800 0.033 0.12520 207 54 4136 126 10 2 -17.000 0.000 -1.000 0.000 0.05205 207 54 4136 214 4 2 4.100 0.000 -2.800 0.000 -0.03696 207 54 ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ 4134 23 1 41 193 20 0.818 0.431 2.116 0.279 -0.00711 4134 23 1 41 193 19 0.589 0.359 2.276 0.454 -0.00765 4134 23 1 41 193 21 0.858 0.422 2.243 0.294 -0.00754 4134 23 1 41 193 21 0.561 0.426 2.271 0.336 -0.00764 4134 23 1 41 193 26 0.379 0.566 2.139 0.279 -0.00719 4134 40 1 42 190 34 0.166 0.313 1.933 0.484 -0.00650 4134 40 1 42 190 23 -0.107 0.255 2.377 0.207 -0.00799 4134 40 1 42 190 24 0.199 0.359 2.482 0.212 -0.00835 4134 40 1 42 190 34 -0.030 0.249 2.402 0.371 -0.00807 4134 40 1 42 190 32 -0.140 0.232 2.502 0.286 -0.00841 4134 76 1 44 185 37 0.601 0.304 2.362 0.251 -0.00794 4134 76 1 44 185 32 0.485 0.389 2.434 0.455 -0.00818 4134 76 1 44 185 52 0.514 0.336 2.399 0.542 -0.00807 4134 76 1 44 185 31 0.334 0.294 2.433 0.328 -0.00818 4134 76 1 44 185 41 0.063 0.344 2.445 0.339 -0.00822 4134 95 1 45 182 31 -0.184 0.388 0.523 0.196 -0.00176 4134 95 1 45 182 39 -0.347 0.381 0.512 0.264 -0.00172 4134 95 1 45 182 51 -0.384 0.355 0.376 0.238 -0.00126 4134 95 1 45 182 40 -0.851 0.406 0.635 0.300 -0.00214 4134 95 1 45 182 63 -0.928 0.376 0.576 0.301 -0.00194 4134 129 1 47 177 32 1.735 0.167 2.030 0.160 -0.00683 4134 129 1 47 177 53 1.581 0.218 2.061 0.143 -0.00693 4134 129 1 47 177 27 1.649 0.291 2.081 0.194 -0.00700 4134 129 1 47 177 40 1.412 0.211 2.188 0.199 -0.00736 4134 129 1 47 177 51 1.171 0.298 1.961 0.207 -0.00659 4134 147 1 49 175 32 -1.395 0.295 -0.112 0.164 0.00038 4134 147 1 49 175 45 -1.281 0.141 -0.042 0.166 0.00014 4134 147 1 49 175 57 -1.353 0.171 0.242 0.178 -0.00081 4134 147 1 49 175 58 -1.425 0.269 0.140 0.157 -0.00047 4134 147 1 49 175 45 -1.830 0.330 0.138 0.166 -0.00046 4134 181 1 52 170 73 -0.033 0.143 1.560 0.104 -0.00524 4134 181 1 52 170 70 -0.179 0.129 1.574 0.084 -0.00529 4134 181 1 52 170 71 -0.092 0.147 1.765 0.145 -0.00593 4134 181 1 52 170 78 -0.343 0.132 1.796 0.134 -0.00604 4134 181 1 52 170 75 -0.428 0.181 1.572 0.194 -0.00529 4134 199 1 54 168 81 1.502 0.247 1.581 0.126 -0.00532 4134 199 1 54 168 81 1.332 0.183 1.586 0.173 -0.00533 4134 199 1 54 168 87 1.234 0.152 1.615 0.260 -0.00543 4134 199 1 54 168 75 1.164 0.253 1.728 0.237 -0.00581 4134 199 1 54 168 65 1.023 0.293 1.819 0.261 -0.00611 4134 234 1 57 164 75 1.103 0.227 1.097 0.179 -0.00369 4134 234 1 57 164 65 0.977 0.170 1.175 0.115 -0.00395 4134 234 1 57 164 70 0.936 0.131 1.243 0.242 -0.00418 4134 234 1 57 164 71 0.793 0.121 1.203 0.182 -0.00404 4134 234 1 57 164 73 0.548 0.185 1.175 0.321 -0.00395 4134 252 1 60 161 68 0.398 0.303 1.310 0.141 -0.00440 4134 252 1 60 161 66 0.229 0.193 1.260 0.355 -0.00424 4134 252 1 60 161 69 0.100 0.185 1.111 0.408 -0.00374 4134 252 1 60 161 73 -0.057 0.234 1.125 0.341 -0.00378 4134 252 1 60 161 74 -0.285 0.232 1.087 0.174 -0.00366 4134 287 1 64 157 66 0.759 0.241 0.497 0.373 -0.00167 4134 287 1 64 157 68 0.580 0.175 0.667 0.384 -0.00224 4134 287 1 64 157 70 0.532 0.133 0.680 0.427 -0.00229 4134 287 1 64 157 78 0.301 0.219 0.745 0.330 -0.00250 4134 287 1 64 157 76 0.112 0.388 0.699 0.221 -0.00235 4134 306 1 67 154 68 1.158 0.392 -1.214 0.233 0.00408 4134 306 1 67 154 66 0.786 0.262 -1.244 0.207 0.00418 4134 306 1 67 154 71 0.659 0.236 -1.165 0.258 0.00392 4134 306 1 67 154 76 0.420 0.176 -1.257 0.327 0.00423 4134 306 1 67 154 74 0.308 0.192 -1.240 0.256 0.00417 4134 342 1 72 149 69 2.339 0.151 -0.702 0.171 0.00236 4134 342 1 72 149 66 2.034 0.137 -0.663 0.141 0.00223 4134 342 1 72 149 72 1.811 0.140 -0.505 0.140 0.00170 4134 342 1 72 149 81 1.511 0.142 -0.353 0.153 0.00119 4134 342 1 72 149 81 1.260 0.157 -0.454 0.140 0.00153 4134 358 1 74 147 76 1.357 0.139 -2.188 0.157 0.00736 4134 358 1 74 147 78 1.016 0.122 -2.204 0.140 0.00741 4134 358 1 74 147 77 0.762 0.134 -2.028 0.170 0.00682 4134 358 1 74 147 80 0.377 0.104 -1.947 0.115 0.00654 4134 358 1 74 147 81 0.054 0.111 -2.012 0.114 0.00676 4134 394 1 80 139 45 1.603 0.151 -7.076 0.098 0.02379 4134 394 1 80 139 63 0.960 0.172 -7.110 0.148 0.02390 4134 394 1 80 139 68 0.306 0.154 -7.072 0.147 0.02378 4134 394 1 80 139 61 -0.428 0.098 -6.977 0.102 0.02346 4134 394 1 80 139 56 -1.033 0.173 -6.947 0.096 0.02336 4136 137 1 54 333 11 0.296 0.093 7.739 0.206 -0.02602 4136 137 1 54 333 29 0.561 0.102 7.920 0.105 -0.02663 4136 137 1 54 333 18 0.842 0.193 7.860 0.132 -0.02643 4136 137 1 54 333 16 1.233 0.185 7.899 0.111 -0.02656 4136 137 1 54 333 17 1.534 0.144 7.779 0.144 -0.02615 4136 156 1 54 333 19 0.730 0.090 8.457 0.118 -0.02843 4136 156 1 54 333 20 0.990 0.091 8.575 0.142 -0.02883 4136 156 1 54 333 25 1.439 0.086 8.523 0.137 -0.02866 4136 156 1 54 333 20 1.644 0.091 8.523 0.112 -0.02865 4136 156 1 54 333 14 1.835 0.145 8.473 0.095 -0.02849 ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): QA Procedures Done (y/n): UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 22447, plate -9999 ------- 23:08 22447 0.0 bias ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL6737-6742, 6749 Stays: JL6743-6748, 6750 ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 22:45Z 63 35 9 311 (NW) 542 - 23:18Z 63 32 11 289 (WNW) 490 - 00:06Z 63 28 14 327 (NNW) 478 - 00:38Z 62 25 10 311 (NW) 479 - 01:11Z 61 23 6 0 (N) 571 - 01:42Z 60 24 9 349 (NNW) 492 - 02:14Z 60 24 5 353 (NNW) 582 - 02:46Z 60 24 3 0 (N) 520 - 03:19Z 60 23 0 305 (NW) 441 - 03:51Z 60 23 0 58 (ENE) 513 - 04:24Z 62 22 5 77 (ENE) 465 - 04:56Z 60 22 2 354 (NNW) 487 - 05:28Z 60 22 1 301 (WNW) 514 - 06:01Z 60 22 0 270 (W) 497 - 06:34Z 60 23 1 241 (WSW) 485 - 07:06Z 60 21 0 259 (W) 468 - 07:39Z 60 21 1 266 (W) 511 - 08:11Z 59 23 0 273 (W) 484 - 08:43Z 59 22 1 313 (NW) 444 - 09:15Z 60 23 0 244 (WSW) 502 - 09:52Z 61 18 0 289 (WNW) 467 - 10:24Z 61 20 1 307 (NW) 460 - 10:57Z 60 23 0 337 (NNW) 473 - 11:29Z 59 19 3 357 (NNW) 441 - 12:01Z 59 18 4 356 (NNW) 422 - 12:34Z 59 18 5 0 (N) 410 - ---=== TELESCOPE STATUS ===--- 00:15Z Enclosure doors opened, fans and blowers on, louvers opened. 01:05Z Open 12:00Z Closed, fans off. No interlocks bypassed. Telescops is at stow with imager mounted. Spectro autofill connected. Dewar: Pete changed imager dewar before going to bed. spectro@160 imager@315 ---=== SOFTWARE USED ===--- IOP/SOP: v3_119_0 Watcher: v2_24_0 MCP: v5_20_0 TPM: tpm_v2_32_0 AstroDa: v14_47 TCC: 2.7.1 sdssProcedures: v1_74 tccMon: v2_2 SoS: v4_9_13 hoggPT: v1_6_9 plate-mapper: v4_3_1 Segue iop s1_4_0 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.5470 0.5460 Axial B 0.6350 0.6290 Axial C 1.0910 1.0900 Trans D -8.5740 -8.5750 Lateral E 10.8839 10.9093 Lateral F 10.8585 10.8712 GALILS Commanded: 4600. -2850. 850. -7250. 19000. 18450. Actual: 4600. -2866. 866. -7275. 19011. 18425. SETMIR VALUES PriDesOrient: 0.00 0.00 15.20 829.90 1.60 PriOrient: 0.00 0.00 14.91 830.49 3.19 SECONDARY: ---------- Focus: 0.00 Air Temp.: 17.2 Alt.: 29.985608 MIGS TONIGHT NOMINAL Axial A 1.5140 1.5100 Axial B 1.1060 1.0990 Axial C 1.1120 1.1160 Trans D -0.8250 -0.8210 GALILS Commanded: 1628638. 1548153. 1574471. -3900. -8750. Actual: 1628641. 1548187. 1574504. -3899. -8726. SETMIR VALUES SecDesOrient: 1257.00 0.00 -25.00 0.00 146.27 SecOrient: 1256.98 0.00 -25.01 0.26 146.56 ---=== PROBLEMS IN DETAIL ===--- Confused server?? ----------------- We saw the following errors couple times before sunset and we are not sure why this was triggered: server is not responding (Last packet: 2003-09-26 23:53:45Z) 01:27Z Servers died. We noted the following message in the murmurlog 2003-09-27 01:27:49Z sdsshost TMS 135184 TEXTONLY getGangs failed: ICC id1 (crate 1, board 0) is not available Bad ICC focusLoop error message ------------------------ On the second scan, after the scanStripe and goDrift, I did: im> stopFocusLoop im> startFocusLoop im> setFocusNkeep 500 Error: can't read "focusData(frameList,ad1)": no such element in array and I don't know what that error means. However, the focus loop seems to be running: im> jobs command arguments interval (s) ------- --------- ------------ checkMakeRunLog {} 30.0 focusLoop 0 0 60.0 kill_zombies {} 60.0 listenerToTCC {} 1.0 plotFocus -quiet 40.0 read_output file17 1 60.0 tapeAdjust {id1 id2 id3 id4 id5 id6 ad1 ad2 ad3 ad4} 300.0 Variable show_scheduled_errors is not set TPM Watcher errors ------------------- m1 temp (Excessive M1 - PMSS Temperature Differential:Excessive Telescope - Ambient Temperature Differential:Excessive Temperature Radial Gradient:: 2003-09-27 01:11:11Z ) tracking (Excessive Az Tracking Error:: 2003-09-27 01:11:56Z) m2 mig (Communications Fault: 2003-09-27 00:09:38Z) m2 temp (Excessive Front-Back Temperature Differential: 2003-09-27 05:40:09Z) ca (Channel Access Failure: TPM Down?: 2003-09-27 04:31:39Z) m2 temp (Excessive Front-Back Temperature Differential: 2003-09-27 05:40:09Z) ROT abort -------------- 09:15Z Rotator aborted and the watcher gave us the following message which we interpreted as we hit the ROT wrap limit. Is this correct? MCP watcher error: rotator (MGT: bad rot state ABORT_EVENT: ID_ERROR_LIMIT (2003-09-27 09:15:50Z sdssmcp): 2003-09-27 09:15:52Z ) TCC watcher error: AxisStat:rot (out_closed_loop: 2003-09-27 09:16:02Z) Here are the messages in the murmurlog regarding this abort: 2003-09-27 09:12:22Z sdssmcp MEI_DC dsc_I_trcInfo Encoders for axis rotator differ by -163408090 2003-09-27 09:13:31Z sdssmcp MEI_DC dsc_I_trcInfo Encoders for axis rotator differ by -163408082 2003-09-27 09:14:39Z sdssmcp MEI_DC dsc_I_trcInfo Encoders for axis rotator differ by -163408095 2003-09-27 09:15:47Z sdssmcp MEI_DC dsc_I_trcInfo Encoders for axis rotator differ by -163408072 2003-09-27 09:15:50Z sdssmcp tmRot dsc_I_trcInfo rotator frame_break: frames left=2 2003-09-27 09:15:51Z sdssmcp tmRot dsc_I_trcInfo Repositioning rotator by TCC cmd 2003-09-27 09:15:51Z sdssmcp tmRot dsc_I_trcInfo Done repositioning rotator to 17082938