APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Friday October 28, 2005 (MJD 53672) ---=== OBSERVING TEAM ===--- Howard Brewington (swing) Olena Malanushenko (swing helper) Jurek Krzesinski (night) Kaike Pan (night) Viktor Malanushenko ---=== OBSERVING PLAN ===--- superNova Science! ---=== OBSERVING SUMMARY ===--- High humidity and thick clouds, we only open for about half hour and imaged on 82 N for supernova. We had several problems: mirror temperature sensor; crate 3 crashed two times, and endNight complained about miss frames and gangs. see problem sections. We could not get proper temperatures from TPM. We do not know if it is mirror temperature sensor problem or TPM problem. Could someone look into it, please? ---=== OBSERVING LOG ===--- Afternoon: ------------- We had to reboot photo-two during the afternoon checkout but had no more problems with the imager bias. Our first attempt with specFlats 15 failed and issued the following error message in the sop session, i.e., Finished with specFlat 1 of 15. Setting up Collimator 2005-10-28 22:03:05Z: Original command was !ma100! 2005-10-28 22:03:05Z: expected reply was !ma100 OK! motor a: Mechanicals did not respond correctly; response was ma100 Error: problem with motors (see previous error message) Original motor positions: {a -9642} {b -9645} {c -7742} Current motor positions: {a -9541} {b -9645} {c -7742} sp1> IOP running as Backup Relay Server requestListen: listening Night: -------------- 00:40Z The imager was mounted. But thick dark clouds and high humidity prevent us from opening. 01:30Z Run 5789 on-telescope dark bias. 05:00Z Run 5790 (SN Stripe 82 N, startLambda=144). Skippy was happy from the begining. It was in the focus at frame 19 (lambda 146.8). Being ramp at frame 26 (lambda 146). Seeing was about 1.7". 5:20Z ended the drift due to a high humidity. The endnight complained about missing frames and gangs (see problem section for detail) 12:30Z endNight rejoced. ---=== IMAGING RUN SUMMARY ===--- Run Time Stripe Lambda Last Flavor Comments Start End Begin End Frame ------------------------------------------------------------------------- 5787 22:06Z 22:15Z 100 O 78.16 80.24 21 ignore 5788 22:17Z 23:04Z 100 O 80.79 92.54 93 ignore 5789 01:31Z 01:54Z 100 O 129.51 135.29 49 bias 5790 05:01Z 05:17Z 82 N 144.39 148.52 41 ignore ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot az el --------------------------------------------------------------------------- 5790 17 4 49 1.600 0.122 -2.100 0.124 -0.01246 235 42 5790 20 4 38 1.700 0.106 -2.200 0.124 -0.01284 235 42 5790 29 4 51 1.600 0.148 -2.300 0.165 -0.01217 235 42 5790 29 4 51 1.600 0.148 -2.300 0.165 -0.01217 235 42 ---=== LTMATCH RESULTS ===--- Run Field nFields alt az nGood rowMean rowSig colMean colSig rot ------------------------------------------------------------------------ ---=== SPECTROSCOPY DATA SUMMARY ===--- Summary Checked (y/n): QA Procedures Done (y/n): UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 34377, plate 2265 ------- 21:38 34377 0.0 bias 21:42 34378 10.0 flat 21:44 34379 2.0 arc 21:48 34380 2.1 arc 21:50 34381 2.1 arc 22:02 34382 10.0 flat 22:31 34383 10.0 flat 22:35 34384 10.0 flat 22:39 34385 10.0 flat 22:43 34386 10.0 flat 22:46 34387 10.0 flat 22:50 34388 10.0 flat 22:54 34389 10.0 flat 22:58 34390 10.0 flat 23:02 34391 10.0 flat 23:06 34392 10.0 flat 23:10 34393 10.0 flat 23:14 34394 10.0 flat 23:18 34395 10.0 flat ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument Az Alt Rot Scale pos offset pos offset pos offset ------------------------------------------------------------------------------ ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL Stays: JL ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 21:35Z 52 36 15 213 (SSW) -9999 - 22:05Z 50 36 16 187 (S) -9999 - 22:35Z 50 36 -9999 -9999 (N) -9999 - 23:07Z 47 36 11 183 (S) 502 - 01:08Z 45 37 16 182 (S) 594 - 01:40Z 44 38 13 166 (SSE) 615 - 02:12Z 44 38 12 183 (S) 0 - 02:44Z 44 35 11 182 (S) 0 - 03:17Z 41 40 14 151 (SSE) 0 - 03:49Z 43 38 11 179 (S) 0 - 04:21Z 44 38 18 188 (S) 0 - 04:57Z 44 38 17 175 (S) 0 - 05:29Z 44 40 9 198 (SSW) 0 - 06:01Z 44 40 14 200 (SSW) 0 - 06:33Z 44 42 20 193 (SSW) 0 - 07:06Z 44 42 16 195 (SSW) 0 - 07:38Z 43 42 18 198 (SSW) 0 - 08:10Z 43 42 16 201 (SSW) 0 - 08:44Z 42 42 16 189 (S) 0 - 09:16Z 42 42 15 214 (SW) 0 - 09:48Z 42 43 15 205 (SSW) 0 - 10:20Z 42 43 18 206 (SSW) 0 - 10:58Z 42 43 14 214 (SW) 0 - 11:30Z 42 42 14 206 (SSW) 0 - 12:02Z 41 42 15 221 (SW) 0 - 12:35Z 41 41 21 224 (SW) 0 - ---=== TELESCOPE STATUS ===--- 00:20Z Enclosure doors opened, fans and blowers on, louvers opened. 02:30Z Enclosure doors closed, fans and blowers off due to high humidity. 04:10Z Enclosure doors opened, fans and blowers on, louvers opened. 04:30Z Enclosure off. 05:35Z Enclosure on, doors closed, fans and blowers off, louvers closed (clouds, high humidity). We never reopened. Telescope stowed at 30 degrees with cartridge 3 mounted. Counterweights at 245. Spectro auto-fill connected. Spectro dewar 12:40 272 lb. 25 psi Imager dewar 12:40 206 lb, 23 psi No interlocks bypassed. ---=== SOFTWARE USED ===--- IOP/SOP: v4_6_0b Watcher: v2_30_2 MCP: v5_26_0 TPM: tpm_v3_1_0 AstroDa: v15_6 TCC: TCC 2.7.2.1 August 6 2004 sdssProcedures: SoS: v4_10_7 hoggPT: v1_6_9 plate-mapper: v4_3_1 ---=== MIRROR NUMBERS ===--- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A -5.8410 -5.8440 Axial B -5.6840 -5.6770 Axial C 0.9410 0.9420 Trans D -9.9850 -9.9940 Lateral E 10.3759 10.3759 Lateral F 11.9507 11.9380 GALILS Commanded: 4700. -2450. 3000. -7800. 3300. 3000. Actual: 4700. -2450. 3000. -7800. 3300. 3000. SETMIR VALUES PriDesOrient: 0.00 -1.68 7.05 385.25 92.02 PriOrient: 0.00 -1.52 7.02 384.99 93.81 SECONDARY: ---------- Focus: 0.00 Air Temp.: 10.7 Alt.: 30.000767 MIGS TONIGHT NOMINAL Axial A 1.5580 1.5420 Axial B 1.0630 1.0470 Axial C 1.2050 1.1650 Trans D -0.1060 -0.1240 GALILS Commanded: 1596021. 1601696. 1554150. -6800. -5900. Actual: 1593450. 1603650. 1554300. -6800. -5900. SETMIR VALUES SecDesOrient: 1257.00 15.87 0.00 -50.00 136.76 SecOrient: 1257.12 15.77 -1.38 -50.47 136.81 ---=== PROBLEMS IN DETAIL ===--- ---=== Misc Watcher Messages ==--- m1 temp (Excessive M1 - PMSS Difference:Excessive Telescope - Ambient Difference:Excessive Radial Gradient:Excessive Front-Back Difference (Entire Mirror):Excessive Front-Back Difference (Left Half):Excessive Front-Back Difference (Right Half):Excessive Front-Back Difference (Lower Half):Excessive Front-Back Difference (Upper Half): 2005-10-28 22:14:42Z ) m1 mig (Communications Fault: 2005-10-28 22:14:42Z) tmicro (Disconnected Sensor in M2B4:: 2005-10-28 22:05:46Z) m2 galil (Communications Fault: 2005-10-28 22:14:42Z) m1 galil (Communications Fault: 2005-10-28 22:14:42Z) m2 mig (Communications Fault: 2005-10-28 22:14:42Z) ---== Spectro Focus/Collimation Numbers ===--- IDL 5.5a> collimate, 34380 Reading /data/spectro/53672/sdR-b1-00034380.fit Reading /data/spectro/53672/sdR-b1-00034381.fit COLLIMATE: Min offset = -0.53 pix COLLIMATE: Max offset = 0.92 pix COLLIMATE: RMS across CCD = 0.38 pix COLLIMATE: Mean offset = 0.23 pix COLLIMATE: Camera b1 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix) COLLIMATE: Predict blue camera ring movement of -8.0 degrees for spectro-1 COLLIMATE: (if red is already in focus) Log file = Collimate-53672-b1-00034380.log PostScript file = Collimate-53672-b1-00034380.ps Reading /data/spectro/53672/sdR-b2-00034380.fit Reading /data/spectro/53672/sdR-b2-00034381.fit COLLIMATE: Min offset = -0.37 pix COLLIMATE: Max offset = 0.34 pix COLLIMATE: RMS across CCD = 0.18 pix COLLIMATE: Mean offset = -0.01 pix COLLIMATE: Camera b2 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Predict blue camera ring movement of 0.2 degrees for spectro-2 COLLIMATE: (if red is already in focus) Log file = Collimate-53672-b2-00034380.log PostScript file = Collimate-53672-b2-00034380.ps Reading /data/spectro/53672/sdR-r1-00034380.fit Reading /data/spectro/53672/sdR-r1-00034381.fit COLLIMATE: Min offset = -0.48 pix COLLIMATE: Max offset = 0.36 pix COLLIMATE: RMS across CCD = 0.23 pix COLLIMATE: Mean offset = -0.11 pix COLLIMATE: Camera r1 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Predict (red) piston movement of 1026 steps for spectro-1 COLLIMATE: Issue SOP command: "sp1; mechPiston 1026" Log file = Collimate-53672-r1-00034380.log PostScript file = Collimate-53672-r1-00034380.ps Reading /data/spectro/53672/sdR-r2-00034380.fit Reading /data/spectro/53672/sdR-r2-00034381.fit COLLIMATE: Min offset = -0.57 pix COLLIMATE: Max offset = 0.14 pix COLLIMATE: RMS across CCD = 0.17 pix COLLIMATE: Mean offset = -0.08 pix COLLIMATE: Camera r2 appears to be IN-FOCUS (|mean| < 0.15 pix) COLLIMATE: Predict (red) piston movement of 750 steps for spectro-2 COLLIMATE: Issue SOP command: "sp2; mechPiston 750" Log file = Collimate-53672-r2-00034380.log PostScript file = Collimate-53672-r2-00034380.ps IDL 5.5a> ad3, ad1 crashed ---------- While we were waiting for weather improving, ad3 crashed, we got follwing messages from the watcher and the murmur log: no response (not responding: 2005-10-29 07:57:43Z) (from the watcher); TMS 30124 TEXTONLY getGangs failed: ICC ad3 (crate 3, board 2) is not available Bad ICC 2005-10-29 08:00:06Z sdsshost2 TMS 30124 TEXTONLY updating idAstroline files 2005-10-29 08:00:10Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:13Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:16Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:16Z sdsshost2 IDA 18343 TEXTONLY Opening MCP port -1: Address already in use 2005-10-29 08:00:29Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:30Z sdsshost2 TMS 12123 TEXTONLY transfering frames to unix 2005-10-29 08:00:32Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:35Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:35Z sdsshost2 IDA 18343 TEXTONLY Opening MCP port -1: Address already in use 2005-10-29 08:00:39Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:42Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:45Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:48Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} 2005-10-29 08:00:51Z sdsshost2 IDA 18343 TEXTONLY {ad3 {ICC ad3 (crate 3, board 2) is not available Bad ICC}} we rebooted the crate 3. The same thing happened to ad1 at 10:53Z. Follow is the related message from murmur: 05-10-29 10:52:32Z sdsshost2 IOP 29349 TEXTONLY {ad1 {ICC ad1 (crate 3, board 0) is not available Bad ICC}} 2005-10-29 10:52:35Z sdsshost2 IOP 29349 TEXTONLY {ad1 {ICC ad1 (crate 3, board 0) is not available Bad ICC}} 2005-10-29 10:52:38Z sdsshost2 IOP 29349 TEXTONLY {ad1 {ICC ad1 (crate 3, board 0) is not available Bad ICC}} 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY trace 1, 0 jb original: {schedule astrolineHandler {} -120} 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY trace 1, 0 postsubstitution: {schedule astrolineHandler {} -120} 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY memBytesInUse: 304, memNumBlocksInUse: 1 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY memTotalBytes: 135872, memActualBytes: 135872 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY Checking gang transfer for node ad3 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY trace 1, 0 jb original: {schedule serverHandler 29349 -120} 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY trace 1, 0 postsubstitution: {schedule serverHandler 29349 -120} 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY memBytesInUse: 304, memNumBlocksInUse: 1 2005-10-29 10:52:39Z sdsshost2 IOP 29349 TEXTONLY memTotalBytes: 135872, memActualBytes: 135872 2005-10-29 10:52:41Z sdsshost2 IOP 29349 TEXTONLY trace 1, 0 jb original: {schedule camHandler {} -60} 2005-10-29 10:52:41Z sdsshost2 IOP 29349 TEXTONLY trace 1, 0 postsubstitution: {schedule camHandler {} -60} 2005-10-29 10:52:41Z sdsshost2 IOP 29349 TEXTONLY memBytesInUse: 304, memNumBlocksInUse: 1 2005-10-29 10:53:33Z sdsshost2 IOP 29349 TEXTONLY {ad1 {ICC ad1 (crate 3, board 0) is not available Bad ICC}} 2005-10-29 10:53:36Z sdsshost2 IOP 29349 TEXTONLY {ad1 {ICC ad1 (crate 3, board 0) is not available Bad ICC}} Skippy was running after the crate 3 was rebooted. servers died --------------- server is not responding (Last packet: 2005-10-29 07:54:55Z) servers died at about 07:55Z, we tried to restart serveral with "start_servers -K", and got: you do not have the power to kill observer's 18343 (30307: master) you do not have the power to kill observer's 23665 (30310: spectro). after two attempts, we exited the server windows, and tried the above command, it work this time. The thermo King unit --------------------- At 08:45Z, we found that the humidity inside 2.5m enclosure was high, 87% (surfaces inside enclosure were dry), we tried to turn on the thermo King unit, but could not. So we turned on a heater borrowed from 3.5m dom. endNight --------- The endnight complained about missing frames and gangs Continue? y Run 5787 has 30 frames missing from disk. Run 5787 has 5 gangs missing from disk. Note that binned scans are not supposed to have gangs, but I'm not yot smart enough to distinguish binned and unbinned scans. If files that really should be there are missing, perhaps you should stop here, run copyUnstoredFilesInIdReport, and rerun endNight. Continue with endNight? since 5787 is a bias in a doghouse and we did not have any useful data we did not care about missing frames: 5787 22:06Z 22:15Z 100 O 78.16 80.24 21 ignore 5788 22:17Z 23:04Z 100 O 80.79 92.54 93 ignore 5789 01:31Z 01:54Z 100 O 129.51 135.29 49 bias