APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Thursday February 7, 2002 (MJD 52313) ---=== OBSERVING TEAM ===--- Scot Kleinman Atsuko Nitta Kleinman Swinging, sometimes soaring, Jurek Krzesinski Visiting Dan the Mac Man Visiting Happy Mike Harvanek ---=== OBSERVING PLAN ===--- Science! ---=== OBSERVING SUMMARY ===--- 6+ hours of good imaging and some donuts for Jim. We finished one last exposure on one spectro plate (333) and had some nasty IOP problems (PR 3166) that seemed awfully similar to last night's endNight problem. Please, could someone take a look at what is going on? See the Problems section. ---=== OBSERVING LOG ===--- 0124Z We start with imaging because clouds are not present and we have no reason to expect bad seeing. We start with stripe 82S well before twilight, starting 10degrees before the startLambda, so we shouldn't need to re-slew once twilight starts. Focus was about -300 from the "formula". The initial skippy showed adequate pointing. No pointing corrections were made. Seeing was excellent. We also passed a VERY BRIGHT star at frame 61 in chip 11. 0205Z Twilight ; lambda=-100.4 ; frame=67. We passed through some very cool nebulosity in Orion in dewar 4 around frame 86 in g. We noted the automatic skippy and ltMatch jobs appeared not to be run/scheduled with this scan. 0246Z endDrift at frame 137; lambda=-90. 0249Z On to 30S, starting at lambda=-59.5. Initial skippy results were good. 0258Z Frame 20 - lambda=-57 - seeing, pointing, focus, all OK! For this run, skippy and ltMatch were getting run automatically. 0357Z We noticed a cloud on the horizon (not near where the telescope was pointing). (Lambda=-42; frame ~119.) The weather map shows that a thin wisp is passing by. We are probably in cloud about right at the end of a scan at 0414Z lambda=-38. We'll start the next scan an extra few degrees early as it looks like this wisp of clouds is isolated. 0422Z endDrift at frame 160; lambda=-36. 0429Z start scanning 30N at startLambda=-51. Clouds still present. skippy good at start. 0447Z : frame 38, lambda=-45.9 - start of science. Clouds came in again around 0455Z. The satellite image still looked promising and seeing was still excellent, so we decided to hang in there a while longer. Once again, ltMatch and skippy were not getting run automatically. 0541Z Clouds away- so we restart the scan. We called the last run ignore/bad. 0544Z start drifting again 30 N startLambda=-48. 0550Z frame 17 - arrived at science startLambda=-46. Skippy ok, although the rotator error is high -it was near 0 at the last scan... ltMatch, however, claimed the error was only 1/2 that skippy claimed. Later, the two estimates agreed more closely. 0925Z We cross lambda=+8 - the endLambda in the science plan, but continue as the next scan in the list continues from lambda=8. 1129Z moonrise at frame 584 , lambda= 39 start taking JEG's donuts: frame 588 focus=nominal -300 frame 595 focus=nominal +300 (donuts look OK) data transfered to /data/photo/optics/52313 endDrift: frame 597. makeRunLog for this run failed and IOP hung on the getAllFrames - see Problems section below. 1137Z Off to instChange for spectroscopy - cart 2 plate 333. Found a few guide stars on the gotoField. All hell broke loose trying to remedy the above problem -see Problems section. 1224Z We finally start a science exposure! (Twilight is in 7 minutes). This exposure ends and we rely on the smear from last night. 1241Z We finish our one-needed exposure and start putting the telescope to bed. 1330Z endNight continues as I finish this log ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 2958 2353Z 0013Z 100 O -154.57 -149.50 48 ignore bad bias 2959 0113Z 0125Z 100 O -134.40 -131.45 34 bias good bias 2960 0129Z 0247Z 82 S -109.40 -89.78 145 science Good data 2961 0250Z 0422Z 30 S -58.86 -35.74 168 science Good data 2962 0425Z 0426Z 20 N -50.42 -50.16 16 ignore issued wrong scanStripe 2963 0429Z 0542Z 30 N -50.47 -32.16 136 ignore clouds 2964 0544Z 1137Z 30 N -47.57 40.98 605 science Good data end at frame 584. ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot --------------------------------------------------------------- 2960 11 4 104 3.300 0.108 -2.900 0.101 0.00238 2960 64 4 107 3.100 0.583 -1.300 0.541 0.00118 2961 9 4 121 4.000 0.270 -2.300 0.363 -0.00101 2961 10 4 119 3.900 0.276 -2.300 0.407 -0.00098 2961 19 4 115 3.700 0.134 -2.000 0.146 -0.00205 2961 43 4 115 4.500 0.124 -1.000 0.177 -0.00204 2961 78 4 119 5.200 0.127 -0.100 0.140 -0.00217 2961 112 4 99 5.100 0.135 1.100 0.177 -0.00343 2961 147 4 53 4.300 0.119 3.000 0.143 -0.00310 2963 13 4 131 5.500 0.620 0.000 0.254 0.00186 2963 67 4 75 4.500 0.256 2.700 0.202 -0.00077 2964 9 4 103 7.100 0.155 -0.700 0.255 0.03558 2964 23 4 91 6.300 0.326 1.700 0.493 0.03110 2964 45 4 82 4.700 0.429 4.500 0.383 0.01631 2964 59 4 54 3.000 0.123 5.500 0.166 0.00753 2964 78 4 60 1.900 0.167 6.700 0.143 0.00046 2964 113 4 46 0.900 0.133 6.700 0.134 -0.00569 2964 128 4 36 0.000 0.144 6.500 0.108 -0.00750 2964 148 4 51 -0.900 0.372 6.400 0.909 -0.00634 2964 182 4 47 -0.800 0.333 6.600 0.616 -0.00805 2964 217 4 40 -0.800 0.131 6.600 0.112 -0.00956 2964 250 4 37 -0.600 0.108 6.600 0.102 -0.00930 2964 285 4 46 0.100 0.695 6.600 0.501 -0.01173 2964 320 4 45 -0.200 0.088 6.400 0.086 -0.00990 2964 354 4 51 0.400 0.660 6.300 0.368 -0.01098 2964 388 4 55 -0.600 0.147 6.200 0.111 -0.00946 2964 424 4 48 -0.300 0.125 6.200 0.099 -0.00905 2964 459 4 45 0.000 0.110 6.000 0.119 -0.00862 2964 493 4 54 1.000 0.300 5.500 0.709 -0.00707 2964 527 4 44 0.600 0.111 5.100 0.136 -0.00778 2964 562 4 53 1.200 0.116 4.900 0.103 -0.00633 2964 575 4 66 1.500 0.107 4.800 0.122 -0.00622 2964 575 4 66 1.500 0.107 4.800 0.122 -0.00622 2964 575 4 66 1.500 0.107 4.800 0.122 -0.00622 ---=== LTMATCH RESULTS ===--- Run Field nFields nGood rowMean rowSig colMean colSig rot --------------------------------------------------------------- 2960 86 1 76 -1.133 1.136 1.001 1.019 -0.00337 2960 86 1 45 -0.925 0.930 0.908 0.920 -0.00305 2960 86 1 61 -0.808 0.819 0.843 0.862 -0.00283 2960 86 1 52 -0.538 0.552 0.833 0.838 -0.00280 2960 86 1 70 -0.352 0.376 0.784 0.798 -0.00263 2961 14 1 42 -0.044 0.182 2.543 2.551 -0.00855 2961 14 1 53 0.249 0.305 2.423 2.432 -0.00815 2961 14 1 61 0.277 0.325 2.200 2.215 -0.00739 2961 14 1 35 0.542 0.570 2.089 2.092 -0.00702 2961 14 1 32 0.667 0.691 1.932 1.939 -0.00649 2961 20 1 34 -0.390 0.403 3.086 3.092 -0.01038 2961 20 1 52 -0.243 0.294 2.877 2.880 -0.00967 2961 20 1 42 -0.052 0.228 2.746 2.753 -0.00923 2961 20 1 36 0.124 0.226 2.581 2.588 -0.00868 2961 20 1 48 0.446 0.468 2.575 2.585 -0.00866 2961 54 1 32 -0.334 0.369 2.505 2.515 -0.00842 2961 54 1 20 0.016 0.218 2.501 2.508 -0.00841 2961 54 1 36 0.009 0.192 2.196 2.206 -0.00739 2961 54 1 34 0.329 0.392 2.207 2.218 -0.00742 2961 54 1 29 0.627 0.661 2.131 2.146 -0.00716 2961 88 1 22 0.213 0.299 3.312 3.327 -0.01113 2961 88 1 24 0.384 0.461 3.153 3.158 -0.01060 2961 88 1 24 0.526 0.560 2.997 3.003 -0.01008 2961 88 1 21 0.678 0.692 2.895 2.896 -0.00973 2961 88 1 21 0.900 0.911 2.641 2.645 -0.00888 2961 122 1 13 -1.532 1.539 3.501 3.534 -0.01177 2961 122 1 22 -1.467 1.477 3.564 3.585 -0.01198 2961 122 1 15 -1.274 1.287 3.495 3.536 -0.01175 2961 122 1 23 -0.909 0.922 3.345 3.385 -0.01125 2961 122 1 20 -0.653 0.672 3.170 3.193 -0.01066 2963 60 1 29 0.232 0.258 2.158 2.168 -0.00726 2963 60 1 21 -0.105 0.232 2.104 2.113 -0.00707 2963 60 1 21 -0.308 0.328 2.232 2.237 -0.00750 2963 60 1 20 -0.351 0.375 2.103 2.115 -0.00707 2963 60 1 32 -0.590 0.640 2.127 2.139 -0.00715 2963 82 1 21 0.041 0.145 3.014 3.016 -0.01013 2963 82 1 18 -0.271 0.313 2.920 2.924 -0.00982 2963 82 1 22 -0.552 0.563 2.846 2.854 -0.00957 2963 82 1 20 -0.775 0.786 2.756 2.759 -0.00926 2963 82 1 14 -0.817 0.827 2.761 2.763 -0.00928 2964 15 1 28 -0.593 0.602 -4.690 4.694 0.01577 2964 15 1 34 -0.573 0.580 -4.630 4.636 0.01557 2964 15 1 26 -0.483 0.498 -4.716 4.728 0.01586 2964 15 1 24 -0.394 0.407 -4.853 4.865 0.01632 2964 15 1 36 -0.247 0.279 -4.802 4.811 0.01614 2964 20 1 31 -1.912 1.914 -4.010 4.012 0.01348 2964 20 1 41 -1.794 1.796 -4.031 4.032 0.01355 2964 20 1 30 -1.671 1.676 -4.142 4.145 0.01392 2964 20 1 19 -1.412 1.415 -4.179 4.180 0.01405 2964 20 1 14 -1.328 1.333 -4.263 4.264 0.01433 2964 48 1 22 -3.352 3.353 -1.283 1.290 0.00431 2964 48 1 18 -3.066 3.066 -1.253 1.265 0.00421 2964 48 1 23 -2.820 2.821 -1.456 1.468 0.00490 2964 48 1 26 -2.571 2.572 -1.372 1.382 0.00461 2964 48 1 23 -2.302 2.304 -1.259 1.263 0.00423 2964 54 1 24 -2.292 2.293 -0.427 0.445 0.00144 2964 54 1 17 -2.074 2.078 -0.496 0.528 0.00167 2964 54 1 15 -1.943 1.949 -0.718 0.735 0.00242 2964 54 1 21 -1.788 1.794 -0.672 0.701 0.00226 2964 54 1 21 -1.543 1.549 -0.624 0.643 0.00210 2964 88 1 15 -0.915 0.918 1.075 1.088 -0.00361 2964 88 1 25 -1.004 1.012 1.001 1.017 -0.00336 2964 88 1 13 -1.121 1.128 0.991 1.007 -0.00333 2964 88 1 16 -1.230 1.240 1.022 1.027 -0.00344 2964 88 1 20 -1.257 1.268 0.812 0.827 -0.00273 2964 113 1 21 -0.055 0.146 1.260 1.271 -0.00424 2964 113 1 18 -0.318 0.345 1.231 1.248 -0.00414 2964 113 1 17 -0.421 0.432 1.116 1.126 -0.00375 2964 113 1 19 -0.631 0.658 1.130 1.139 -0.00380 2964 113 1 18 -0.820 0.841 1.074 1.078 -0.00361 2964 123 1 12 0.973 0.980 2.405 2.408 -0.00809 2964 123 1 21 0.687 0.714 2.323 2.326 -0.00781 2964 123 1 26 0.494 0.526 2.367 2.372 -0.00796 2964 123 1 11 0.465 0.490 2.441 2.444 -0.00821 2964 123 1 19 0.173 0.306 2.499 2.502 -0.00840 2964 157 1 15 0.359 0.410 2.583 2.585 -0.00869 2964 157 1 17 0.189 0.308 2.499 2.502 -0.00840 2964 157 1 10 -0.162 0.283 2.478 2.480 -0.00833 2964 157 1 14 -0.530 0.560 2.319 2.322 -0.00780 2964 157 1 18 -0.814 0.827 2.291 2.296 -0.00770 2964 191 1 15 1.777 1.779 3.048 3.054 -0.01025 2964 191 1 19 1.533 1.538 3.012 3.024 -0.01013 2964 191 1 21 1.263 1.267 2.915 2.921 -0.00980 2964 191 1 13 1.024 1.031 2.879 2.884 -0.00968 2964 191 1 14 0.798 0.802 2.967 2.973 -0.00997 2964 227 1 8 1.932 1.942 2.302 2.304 -0.00774 2964 227 1 17 1.676 1.684 2.454 2.457 -0.00825 2964 227 1 18 1.381 1.385 2.388 2.397 -0.00803 2964 227 1 14 1.118 1.123 2.301 2.311 -0.00774 2964 227 1 15 1.015 1.018 2.712 2.730 -0.00912 2964 262 1 17 1.536 1.541 2.864 2.869 -0.00963 2964 262 1 12 1.291 1.298 2.938 2.941 -0.00988 2964 262 1 23 0.892 0.907 2.761 2.769 -0.00928 2964 262 1 13 0.705 0.711 2.873 2.881 -0.00966 2964 262 1 10 0.507 0.515 2.880 2.885 -0.00968 2964 295 1 17 1.291 1.309 2.837 2.840 -0.00954 2964 295 1 15 0.897 0.912 2.800 2.803 -0.00941 2964 295 1 10 0.738 0.769 2.632 2.637 -0.00885 2964 295 1 16 0.433 0.463 2.678 2.681 -0.00900 2964 295 1 11 0.100 0.243 2.631 2.638 -0.00885 2964 330 1 14 0.929 0.941 2.295 2.298 -0.00772 2964 330 1 23 0.634 0.646 2.422 2.426 -0.00814 2964 330 1 14 0.343 0.375 2.411 2.418 -0.00811 2964 330 1 10 0.091 0.149 2.327 2.333 -0.00783 2964 330 1 10 -0.072 0.107 2.350 2.352 -0.00790 2964 364 1 13 1.517 1.520 2.522 2.525 -0.00848 2964 364 1 11 1.202 1.205 2.407 2.411 -0.00809 2964 364 1 7 0.925 0.936 2.328 2.333 -0.00783 2964 364 1 15 0.634 0.641 2.452 2.455 -0.00824 2964 364 1 13 0.474 0.488 2.418 2.422 -0.00813 2964 400 1 16 1.798 1.804 2.569 2.582 -0.00864 2964 400 1 16 1.617 1.621 2.723 2.728 -0.00915 2964 400 1 15 1.249 1.259 2.744 2.758 -0.00923 2964 400 1 10 1.060 1.078 2.663 2.674 -0.00895 2964 400 1 10 0.804 0.815 2.783 2.798 -0.00936 2964 435 1 11 1.664 1.673 2.028 2.034 -0.00682 2964 435 1 22 1.345 1.354 2.045 2.048 -0.00687 2964 435 1 12 0.997 1.027 2.014 2.020 -0.00677 2964 435 1 13 0.666 0.680 2.133 2.139 -0.00717 2964 435 1 19 0.540 0.584 2.060 2.064 -0.00693 2964 469 1 13 1.902 1.905 1.743 1.757 -0.00586 2964 469 1 16 1.628 1.636 1.926 1.932 -0.00647 2964 469 1 12 1.373 1.379 1.874 1.882 -0.00630 2964 469 1 17 1.103 1.111 1.901 1.913 -0.00639 2964 469 1 13 0.851 0.861 1.987 1.995 -0.00668 2964 503 1 10 1.968 1.976 1.453 1.473 -0.00488 2964 503 1 16 1.648 1.654 1.393 1.401 -0.00468 2964 503 1 15 1.193 1.213 1.493 1.509 -0.00502 2964 503 1 16 1.031 1.041 1.302 1.326 -0.00438 2964 503 1 14 0.776 0.786 1.455 1.477 -0.00489 2964 537 1 17 1.945 1.953 1.353 1.379 -0.00455 2964 537 1 16 1.550 1.555 1.412 1.424 -0.00475 2964 537 1 14 1.161 1.174 1.090 1.105 -0.00366 2964 537 1 16 1.074 1.089 1.384 1.388 -0.00465 2964 537 1 18 0.744 0.779 1.413 1.424 -0.00475 2964 568 1 14 1.978 1.989 1.152 1.181 -0.00387 2964 568 1 19 1.747 1.756 1.162 1.184 -0.00391 2964 568 1 17 1.311 1.319 1.003 1.018 -0.00337 2964 568 1 17 0.994 0.998 0.943 0.957 -0.00317 2964 568 1 19 0.767 0.802 1.221 1.239 -0.00411 2964 568 1 14 1.978 1.989 1.152 1.181 -0.00387 2964 568 1 19 1.747 1.756 1.162 1.184 -0.00391 2964 568 1 17 1.311 1.319 1.003 1.018 -0.00337 2964 568 1 17 0.994 0.998 0.943 0.957 -0.00317 2964 568 1 19 0.767 0.802 1.221 1.239 -0.00411 ---=== SPECTROSCOPY DATA SUMMARY ===--- UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 12995, plate -9999 ------- 23:57 12995 0.0 bias 00:04 12997 0.0 bias ----- sequence 13000, plate 333 ------- 12:14 13000 0.0 bias 12:19 13001 10.0 flat 12:22 13002 2.0 arc 12:40 13003 900.0 target Plate done with data from last night! ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ 0138Z imager 35.37 0.0017 51.53 0.0003 118.98 -0.0096 1.000000 0303Z imager 89.10 0.0017 53.87 0.0003 113.93 -0.0096 1.000000 0851Z imager 146.77 0.0017 67.81 0.0003 223.35 -0.0096 1.000000 1237Z 2 333 -52.43 0.0006 40.18 0.0008 138.11 -0.0192 0.999930 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL4986 JL4987 JL4988 JL4989 JL4990 JL4991 JL4998 Stays: JL some others ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 18:37 imager 1.00000 0 -350 -261 35 51.5 0.6 6 275 i'-23 1.4 19:05 imager 1.00000 0 -358 -269 35 51.5 0.9 3 267 i'-23 0.81 20:03 imager 1.00000 0 -425 -329 89 53.9 1.0 4 262 i'-23 1.14 20:41 imager 1.00000 0 -461 -356 99 57.1 1.3 1 214 i'-23 1.06 21:41 imager 1.00000 0 -501 -366 92 70.5 0.4 8 211 i'-23 1.02 22:22 imager 1.00000 0 -498 -362 112 71.0 -0.3 9 231 i'-23 0.91 23:20 imager 1.00000 0 -525 -376 130 83.2 0.7 5 192 i'-23 1.14 00:00 imager 1.00000 0 -506 -360 151 78.7 -1.1 3 227 i'-23 1.18 00:50 imager 1.00000 0 -547 -408 154 73.0 0.5 8 264 i'-23 1.3 01:28 imager 1.00000 0 -525 -392 150 69.5 -0.1 8 268 i'-23 0.94 02:16 imager 1.00000 0 -521 -394 142 66.3 0.6 9 270 i'-23 0.99 02:50 imager 1.00000 0 -545 -421 135 64.9 0.8 11 290 i'-23 1.0 03:11 imager 1.00000 0 -551 -429 130 64.4 1.1 10 272 i'-23 1.14 03:54 imager 1.00000 0 -520 -398 119 63.9 1.1 10 290 i'-23 1.39 05:38 2 333 0.99993 690 1688 -100 -53 40.1 -0.3 9 290 gdr 1.4 ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 16:47 32 8 5 269 (W) 86 - 17:19 32 4 1 261 (W) 95 - 17:51 32 -2 3 272 (W) 43 - 18:23 33 -7 5 271 (W) 70 - 19:00 33 -8 3 267 (W) 63 - 19:32 33 -9 2 260 (W) 47 - 20:03 33 -8 4 262 (W) 56 - 20:36 34 -9 1 258 (WSW) 40 - 21:13 33 -9 5 179 (S) 37 - 21:45 32 -8 6 190 (S) 55 - 22:17 32 -8 8 196 (SSW) 33 - 22:53 32 -9 6 231 (SW) 30 - 23:26 32 -10 6 193 (SSW) 35 - 23:58 30 -8 4 241 (WSW) 29 - 00:30 33 -13 6 262 (W) 25 - 01:03 32 -15 8 265 (W) 33 - 01:35 31 -18 7 267 (W) 36 - 02:07 33 -21 9 268 (W) 22 - 02:39 32 -21 11 284 (WNW) 26 - 03:11 33 -8 10 272 (W) 35 - 03:52 33 -19 9 275 (W) 27 - 04:24 33 -9 8 277 (W) 25 - 04:55 33 -8 9 273 (W) 34 - 05:26 31 -7 10 276 (W) 29 - 05:59 32 -8 12 273 (W) 18 - ---=== TELESCOPE STATUS ===--- Doors Open/Fans On: ~23:15Z Enclosure Off: ~01:00Z Interlocks bypassed slip detection, but velocity door bumpswitches building interlocks Enclosure on: ~13:10Z Cart 3 mounted on telescope ---=== SOFTWARE USED ===--- IOP/SOP: v3_66_5a -new today bug fix from last night and old astroda dependent Watcher: v2_15_0 MCP: v5_9_6 TPM: v2_4_6 AstroDa: v14_36 - reverted today TCC: 2.6 sdssProcedures: v1_39 tccMon: v1_8 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.0790 0.0750 Axial B 0.7990 0.8020 Axial C 0.0650 0.0710 Trans D -1.5210 -1.4860 Lateral E 1.8760 1.8540 Lateral F 1.4310 1.4240 GALILS Commanded: 5400. -3700. 900. -200. 31550. 30650. Actual: 5400. -3700. 900. -200. 31550. 30650. SETMIR VALUES PriDesOrient: 0.00 -11.80 23.00 1256.90 642.10 PriOrient: 0.00 -12.16 22.81 1257.41 642.19 SECONDARY: ---------- Focus: 0.00 Air Temp.: 0.1 Alt.: 30.003287 MIGS TONIGHT NOMINAL Axial A 1.5580 1.5500 Axial B 1.2440 1.2260 Axial C 1.2620 1.2480 Trans D 0.2010 0.2320 GALILS Commanded: 1599580. 1565262. 1586351. -10400. -9750. Actual: 1599600. 1565250. 1586350. -10400. -9750. SETMIR VALUES SecDesOrient: 1257.00 -3.40 -10.00 0.00 133.82 SecOrient: 1256.99 -3.40 -9.98 -0.23 133.70 ---=== PROBLEMS IN DETAIL ===--- MCP Watcher errors: ------------------- altitude (addoffset: Max accl. for altitude exceeded: 442/1000 (Feb 07 21:23:27): Feb 07 21:23:45 MST ) azimuth (addoffset: Max accl. for azimuth exceeded: 269/1000 (Feb 07 21:23:27): Feb 07 21:23:45 MST ) rotator (addoffset: Max accl. for rotator exceeded: 419/1000 (Feb 07 21:23:28): Feb 07 21:23:45 MST ) VME Watcher errors: ------------------- 2 ad nodes: dead (Frame=9 Row=1361: Feb 07 21:26:46 MST) IOP problem ending run 2964 and with getAllFrames: --------------------------------------------------- im> getAllFrames 2964 588 /data/photo/optics/52313 file /data/photo/optics/52313/idR-002964-g1-0588.fit file /data/photo/optics/52313/idR-002964-g2-0588.fit file /data/photo/optics/52313/idR-002964-i3-0588.fit makeRunLog failed for run 2964. I have the following diagnostics plus check the murmur log and your screen history: ipVectorGetFromRegion: row/col out of range file /data/photo/optics/52313/idR-002964-g3-0588.fit file /data/photo/optics/52313/idR-002964-g4-0588.fit file /data/photo/optics/52313/idR-002964-g5-0588.fit file /data/photo/optics/52313/idR-002964-g6-0588.fit file /data/photo/optics/52313/idR-002964-o5-0588.fit Waiting for command response from ad2 (crate 3, ICC 1), data server (client 1) The last message repeated continuously. We rebooted the astrometric crate and I lost my controlling IOP and SOP, but Atsuko kept hers and my logTool and murmurLog IOPs also remained alive! Attempt to restart SOP failed, ptvmeTest revealed: ... errno = 0: ioctl (fd=3, PT_IOCTL_GET_DEBUG, &ptDebugSave) = 0 ptDebug = 0x00000000 errno = 5: ioctl (fd=3, PT_IOCTL_GET_NODE_ID, 0) = 0xFFFFFFFF (I/O error) . 0 2 ms 05:05 errno = 16: read (fd=3, 0x7F7F2B10, 0x00000004) = -1 (Resource busy) CSB status: 0x00000000 AUX = (unknown) CMP = (Never Used) INNA = 0 errno = 0: lseek (fd=3, 0x00000000, SEEK_CUR) = 0x00000000 00000000: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F <................> 00000010: 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F <................> 00000020: 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F < !"#$%&'()*+,-./> 00000030: 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F <0123456789:;<=>?> I issued a ptvmeReset and started SOP again - successfully, this time, but it came up with an error: --> can't read "processInformation": no such variable sp1> tb can't read "processInformation": no such variable while executing "lindex $processInformation..." invoked from within "set serverExecutable [lindex $processInformation..." invoked from within "if {[file exists $lockFileName]} { set lockFile [open $lockFileName r] gets $lockFile lockFileContents close $lockFile set process ..." ("foreach" body line 4) invoked from within "foreach portName $portNameList { set portNumber [keylget portNumberList $portName] set lockFileName ${lockFileLocation}/server_port_$portNumber if ..." (procedure "checkServers" line 26) invoked from within "checkServers -warn" (file "/p/iop/v3_66_5a/etc/iopStartup.tcl" line 555) invoked from within "source $env(IOP_DIR)/etc/iopStartup.tcl" (file "/p/iop/v3_66_5a/etc/sop/sopStartup.tcl" line 1) invoked from within "source /p/iop/v3_66_5a/etc/sop/sopStartup.tcl" I exited SOP and tried again. Same error. I did a new ptvmeReset and started SOP again. This time, things came up OK. Meanwhile, I lost the guide stars and had to re-acquire with a new gotoField. Whatever happened sure smells like whatever happened last night with endNight. Later attempts to getAllFrames worked. A PR was filed.