astrolog

Introduction

This directory contains log files produced by SDSS telescope control systems, imaging camera and spectroscopic instrument.

To jump to links to the individual MJD directories, click here.

These data should be regarded as the source material for metadata used by the SDSS imaging and spectroscopic processing pipeleines (photoop and idlspec2d). However, these data are not the actual metadata used those pipelines. That metadata can be found in the photolog and speclog products, respectively. In other words, the photolog and speclog products are the curated versions of the raw data found here.

During the summer of 2015, as part of the transition from SDSS-III to SDSS-IV, astrolog data was transferred from APO to long-term archiving. However, there were already two other versions of these data extant. Here is a brief description of the three versions:

apo_engineering
The version obtained with the APO engineering data in 2015.
apo_staging
The version derived from daily APO transfers. This version contains symbolic links.
boss_spectro
The version obtained from the boss/spectro directory. This was the precursor to the speclog_ product.

In addition there is some duplication of files with the oplogs data. These files are left alone.

The MJD directories in the same directory as this file contain a deduplicated sample from these sources. The deduplication followed the procedure outlined below.

Deduplication

The removal of duplicate files followed this procedure. For each MJD directory in each of the three sources listed above:

  1. If there is only one version of the MJD directory, it is unconditionally moved to the 'unique' directory.
  2. The presence of any subdirectory forces a by-hand resolution.
  3. Within a non-unique MJD directory, any unique file is unconditionally moved to the unique directory.
  4. For any set of non-unique files with a given MJD, the files' size and mtime are compared. If these values are all the same, one file is moved to the unique directory. The rest are deleted.
  5. Any remaining sets of non-unique files are compared using checksums. If the checksums are all the same, one file is moved to the unique directory. The rest are deleted.
  6. For any-remaining non-unique files, the 'boss_spectro' version is considered preferred and is moved to the unique directory. Other versions are left in place.
  7. Any remaining non-unique files (that did not have a preferred version) trigger a warning message, and are left in place.
  8. The contents of the unique directory are moved up one level to become the official astrolog data.

The code that implements this procedure is in python/deduplicate.py in the sas product.

The log showing the files and directories that were moved is here.

After deduplication, the remaining contents of apo_engineering and apo_staging were mothballed:

Subdirectories

The following subdirectories were discovered:

boss_spectro/53656/RCS
Merged with apo_engineering version of the same directory.
boss_spectro/53819/testLog
The only contents of this directory is the subdirectory 53819.
boss_spectro/53819/testLog/53819
Don't know why this was ever here. Just leaving it alone.
boss_spectro/54029/RCS
This RCS directory has no other possibly conflicting versions. Just leaving it alone.
boss_spectro/54423/temp
Don't know why this was ever here. Just leaving it alone.
apo_engineering/53656/RCS
Contained non-overlapping files with boss_spectro/53656/RCS, listed above. Merged into a single RCS directory.
apo_engineering/53995/RCS
The contents could be moved unambiguously to the boss_spectro/53995 directory.
apo_engineering/53996/RCS
Moved to boss_spectro/53996. Could not resolve differences in the idReport file. Leaving it alone.

Historical

The original README file in this directory is preserved below.

51615/plPlugMapM-0323-51612-01.par
  Changed the SPECTROPHOTO_STD stars to those suggested by Hogg

51579/plPlugMapM-0318-51578-01.par
  Changed the SPECTROPHOTO_STD stars to the following two:
   FIBERID=255  RA=132.55750  DEC=11.356539
   FIBERID=558  RA=133.01796  DEC=11.911908a
  These changes were made by changing the u,g,i,z magnitudes such that these
  stars will be selected.

  The existing SPECTROPHOTO_STD objects were renamed to UNKNOWN.

  Are the SKY fibers really sky??

51580/plPlugMapM-0318-51578-01.par
  Same changes as for 51579/plPlugMapM-0318-51578-01.par

51614/plPlugMapM-0318-51598-01.par
  Same changes as for 51579/plPlugMapM-0318-51578-01.par

51629/plPlugMapM-0318-51626-01.par
  Same changes as for 51579/plPlugMapM-0318-51578-01.par

51951
  Replace the plugmap files for 0413-51925-01 with 0413-51942-01.

51959
  Replace the plugmap files for 0550-51954-01 with 0550-51954-02.

51959
  Replace the plugmap files for 0547-51954-02 with 0547-51954-03.

51984
  Replace the plugmap files for 0542-51984-01 with 0542-51984-02.

51984
  Replace the plugmap files for 0279-51977-01 with 0279-51977-02.

51985
  Replace the plugmap files for 0542-51984-01 with 0542-51984-02.

51991
  Replace the plugmap files for 0542-51984-01 with 0542-51984-02.

51993
  Replace the plugmap files for 0542-51984-01 with 0542-51984-02.

51994
  Replace the plugmap files for 0503-51976-01 with 0503-51991-01.
------------------------------------------------------------------------------
On 13 Nov 02, I have done an rsync from sdsshost.apo:/astrolog
for MJD = [52510, 52592] (today).  This is to get the imaging log files
for these dates, which were not copied by SOS.

On 13 Nov 02, I have done an rsync from sdsshost.apo:/skycat/oplogs/astrolog
for MJD = [52312, 52510).  This is to get the imaging log files
for these dates, which were not copied by SOS.
--> Run /scr0/data/astrolog/rsync.13nov02 !!!???

On 13 Nov 02, I have copied the Fermi golden directories at Princeton
for MJD 50991 to 51259 (pre-spectroscopy days, which started on 51433).
Since I'm using the Princeton version, this includes the Finkbeiner-edits
to the idFrameLog files for the Orion scans.

We still need the idAstroline files for runs 745->756,
corresponding to MJD 51257->51259, which aren't even in the Fermi
golden directory.

The following directories existed in the golden directory,
but not in astrolog.  I'll copy from the golden directory ???
       50991
       50992
       51437
       51458
       51459
       51463
       51464
       51465
       51466
       51467
       51568
       51569
       51570
       51574
       51576
       51577
       51586
       51606
       51607
       51618
       51631
       51632
       51638
       51639
       51640
       51659
       51661
       51664
       51667
       51668
       51669
       51670
       51674
       51675
       51685
       51686
       51698
       51781
       51801
       51808
       51814
       51815
       51887
       51888
       51891
       51892

The following directories existed in the golden directory,
but not in astrolog.  On 15 Nov 02, I have done an rsync from
sdsshost.apo:/skycat/oplogs/astrolog:
       51819
       51853
       51854
       51855
       51856
       51862
       51863
       51864
       51865
       51866
       51867
       51870
       51874
       51875
       51880
       51881
       51886
       51894
       51895
       51896
       51922
       51933
       52308

Index of astrolog/