sdssdp1> export OO_FD_BOOT=/opdb/d1/data/opdb/photoFor "csh" users:
sdssdp4> setenv OO_FD_BOOT /opdb/d1/data/opdb/photoYou should check with Survey Operations for the correct value to set the variable to. To access the OPDB, one must setup and run the "ts" product.
sdssdp4> setup ts sdssdp4> ts ts>One must then start a database session, and initialize access to the OPDB. Set the maximum number of page buffers to a comfortable number.
ts> dbInit -nMaxPages 5000 ts> sessionBegin ts> photoInitThe above pair of commands grants update-access to the OPDB. For read-only access (a nice safety precaution if you're only browsing):
ts> sessionBegin -readonly -mrow ts> photoInit -readonlyAfter performing all operations, the database session must be closed before quitting the "ts" executable.
ts> sessionEnd ts> quit sdssdp4>
ts> set stats [setObjectStatus 94 1 0 -authors adelman]If there is only one Frames Pipeline Run for this camCol of this run, then it is not necessary to specify the rerun number. An optional comment may also be entered:
ts> set stats [setObjectStatus 94 1 -authors adelman -comment "Blame Yanny"]A default value is used for the radius when matching objects in adjacent fields (see here for the default value, or type "help setObjectStatus" in the "ts" shell). A different value (in pixels) may be specified using the "-matchRadius" flag. Thus, to match with a radius of 3 pixels:
ts> set stats [setObjectStatus 94 1 -authors adelman -matchRadius 3]The statistics are on a per field basis. They may be binned over fields to increase the number of pairs used for each set of statistics using the mergeQABin proc. This proc returns a TSMATCHSTATS chain. For example, to bin over 20 fields:
ts> set stats2 [mergeQABin $stats 20]Plots based on the returned statistics chain (binned or unbinned) may be displayed using the dbMatchPlot proc.
ts> dbMatchPlot $statsTo plot to a postscript file, rather than the screen, use the "-file" option:
ts> dbMatchPlot $stats2 -file plots.psThe statistics can be printed to the screen using the dbMatchPrint proc. For example, to print the differences in positions:
ts> dbMatchPrint $stats pos
The same statistics can also be generated for a Frames Pipeline Run which has already been through "setObjectStatus" by using the selfMergeQA task. For example, to generated the statistics for run 94, camCol 1:
ts> set run [framesRunGet 94 1] ts> set stats [selfMergeQA $run -noBlends -nSigma 5] ts> objDel $runFor a full description of the statistics chain and routines to examine it, see here. When done with the stats chain, it may be destroyed with "taGenericChainDestroy":
ts> taGenericChainDestroy $stats
ts> runMerge 94 1 0 -authors adelmanThe rerun argument is optional if there is only one Frames Pipeline Run for the specified run and camCol. The "-authors" argument is required. A default value is used for the matching radius (see here for the default value, or type "help runMerge" in the "ts" shell). Use the "-matchRadius" argument to specify a different value (in arcsecs). Thus, to merge with a matching radius of 1.5 arcsecs:
ts> runMerge 94 1 -authors adelman -matchRadius 1.5The current astrometric calibration is used by default. A different astrometric calibration may be specified with the "-astroCalib" argument. DCR corrections are applied using cosmic colors, since a final photometric calibration can not be guaranteed to exist for the run at this time.
ts> set run1 [framesRunGet 94 1 0] ts> set run2 [framesRunGet 125 2 1] ts> set stats [runMergeQA $run1 $run2 -noBlends] ts> objDel $run1 ts> objDel $run2In "framesRunGet", the rerun is optional if only one Frames Pipeline Run for that run and camCol have been stuffed. If the range of fields in the first run that intersects the second run is known ahead of time, it may be specified to speed the task:
ts> set stats [runMergeQA $run1 $run2 -noBlends -firstField $field1 \
-lastField $field2]
By default, the current astrometric and photometric calibrations are used;
the arguments "-astro1", "-photo1", "-astro2", and "-photo2" may be used to
specify different calibrations, useful for comparing various calibrations.
runMergeQA returns a chain of TSMATCHSTATS statistics, one entry for each field in the first run, based on the pairs of objects matched between the two runs. The "-noBlends" flag will cause objects blended in the first run (either flag BLENDED or CHILD set) to be excluded in the matching statistics. By default, matching statistics are compiled for objects in a default magnitude range in a default filter: these values may be changed using the arguments "-min", "-max", and "-filter". For example, to examine matched pairs where the object in run 1 in the pair is between 17th and 19th magnitude in g':
ts> set stats [runMergeQA $run1 $run2 -noBlends -min 17 -max 19 -filter g]The statistics are on a per field basis. They may be binned over fields to increase the number of pairs used for each set of statistics using the mergeQABin proc. This proc returns a TSMATCHSTATS chain. For example, to bin over 20 fields:
ts> set stats2 [mergeQABin $stats 20]Plots based on the returned statistics chain may be displayed using the dbMatchPlot proc.
ts> dbMatchPlot $stats -calib -logThe "-calib" arguments tells the proc that calibrated quantities are being plotted, rather than instrumental quantities. The "-log" argument causes counts to be plotted as log counts. To plot to a postscript file, rather than the screen, use the "-file" option:
ts> dbMatchPlot $stats -calib -log -file plots.psA TCL proc is provided to create plots for all overlapping columns between two runs. For example, to do so for runs 1140 and 1231:
ts> runMatchPlot 1140 1231 30 mag -file all.ps -calibThe statistics can be printed to the screen using the dbMatchPrint proc. For example, to print the differences in positions:
ts> dbMatchPrint $stats posFor a full description of the statistics chain and routines to examine it, see here.
runMergeQA can also return chains containing each matched pair, with a separate chain for stars and galaxies. Use the "-obj" flag for this feature:
ts> set list [runMergeQA $run1 $run2 -noBlends -obj] ts> set stats [lindex $list 0] ts> set stars [lindex $list 1] ts> set galaxies [lindex $list 2]For each matched pair, the great circle longitude and latitude (mu and nu, in degrees), the magnitude in each filter (PSF magnitude for stars, petrosian magnitude for galaxies), and the model colors are given for the object in the first run, along with the differences in those quantities between the two runs (in the sense of run 1 quantity minus run 2 quantity, in arcsecs for dmu and dnu, mag for dmag and). dcolor). The photo flags for each object in the pair are also given. For example, to plot the difference in mu (in arcsecs) versus mu (in degrees) for stars:
ts> set pg [pgstateNew] ts> pgstateOpen $pg ts> ePlot $pg $stars.mu $stars.dmuIf the "-full" flag is specified, then both entire calibrated objects are also included:
ts> set list [runMergeQA $run1 $run2 -noBlends -obj -full] ts> set stats [lindex $list 0] ts> set stars [lindex $list 1] ts> set galaxies [lindex $list 2]For example, to plot the difference in row velocity versus r' petrosian magnitude for galaxies:
ts> ePlot $pg $galaxies.mag<2> \
$galaxies.obj1->rowv.val-$galaxies.obj2->rowv.val
The "-full" flag slows things down, as it must read a much bigger chunk of the
OPDB to fetch the full objects.
When done with the stats chain, it may be destroyed with
"taGenericChainDestroy":
ts> taGenericChainDestroy $statsThe matched-pairs chains must be destroyed with "matchedPairChainDel":
ts> matchedPairChainDel $stars ts> matchedPairChainDel $galaxiesAn alternate way to examine match-ups between Frames Pipeline Runs is to use the GUI. Select one Frames Pipeline Run with the left mouse button. Then select another with the right mouse button. The matching statistics will be displayed using the GUI's tool.
ts> fieldQualitySet 94 1 good 23 248 -rerun 0The "-rerun" argument is required only if there is more than one Frames Pipeline Run stuffed for that run and camCol. If a starting and ending field aren't specified, then all fields in the Frames Pipeline Run are set to that quality. For example, to set all fields to "bad" for run 125, camCol 3:
ts> fieldQualitySet 125 3 badTo set fields 244 to the end of the run to acceptable for run 125, camCol 3:
ts> fieldQualitySet 125 4 acceptable 244
sdssdp4> ts -useTk ts> tsGuiThe GUI should now display all segments, color coded as indicated by the legend on the left. The southern stripes are displayed first, then the northern stripes. "Primary", "secondary", and "bad" segments have already been resolved. "Unresolved ok" and "unresolved bad" segments are unresolved pieces of Frames Pipeline Runs which have had a quality assigned ("unresolved bad" corresponding to quality "bad", and "unresolved ok" corresponding to quality "good" or "acceptable"), and which have been processed through the setObjectStatus and merge tasks. "Undetermined" segments are unresolved pieces of Frames Pipeline Runs which have been processed through setObjectStatus and merge, but have not had a quality assigned. "Unprocessed" segments are unresolved pieces of Frames Pipeline Runs which have not been processed through either or both setObjectStatus and merge. Only segments which have been processed through both setObjectStatus and merge, which have had a quality assigned (bad, acceptable, or good), and which have not previously been resolved are eligible to be resolved (that is, only "unresolved ok" and "unresolved bad" segments).
Many tasks not directly related to resolving are available from the GUI, and are briefly summarized in this paragraph. Clicking with the middle mouse button on a segment will list identifying information on it in the lower left panel. Clicking with the left mouse button on a segment "selects" it (indicated by it's border changing colors). The buttons on the left perform the tasks listed by clicking with the left mouse button (additional tasks are available via the "Actions" menu). Clicking the "set object status" button will process the selected segment through the setObjectStatus task (using the default matching radius). Clicking the "merge" button will process the selected segment through the merge task (using the default matching radius). Clicking on a second segment with the right mouse button will generate matching statistics between it and the selected segment, and display those statistics with the GUI's QA plotting tool (as well as give the user the option to set the quality over ranges of fields for the selected segment). The plot may be manipulated using the "QA" menu.
To resolve, we must first display those regions of the sky which are ready to resolve. A resolvable piece of sky is a continuous range in survey longitude of a stripe which is completely covered by "unresolved ok" segments. Press the "plot chunks-to-be" button with the left mouse button. A purple rectangle will be plotted delimiting each piece of a stripe which can be resolved. Click the border on one of the purple boxes. The border will turn gold, indicating you've selected it. Then press the "resolve" button with the left mouse button. You will be prompted for a list of runs to use for the primary segments. You may specify a list of runs to use. For example, if you wish to use runs 259 and 273 as the primary segments, then entery "259 273". If nothing is entered, then the lowest numbered runs will be used as the primary segments. That is, for a given scanline, the good segment with the lowest run number will be used as the primary segment over its full extent, with the rest of the scanline filled by the good segment with the next lowest run number and so on. The chunk will now be resolved. This takes quite a while; progress will be indicated in the xterm in which each field will be printed as it is resolved. After the chunk has been resolved, the purple box will disappear, and the newly resolved segments will change color to indicate their resolved status.
You can get some feedback on whether things work by using fieldPlot. This is primarily a debugging tool to visualize whether the object status flags were properly set. It plots up GOOD objects in a set of contiguous fields (spans of three fields are nice). For example:
ts> set run [framesRunGet 745 5] ts> fieldPlot $run -first 142 -last 145 -resThe meaning of the various symbols and lines is given in the task documentation.
After a chunk of sky has been resolved, there may be additional scans which go through the same chunk of sky. We would like to be able to resolve this additional data, setting all objects in the portion overlapping the previously resolved chunk to SECONDARY (since PRIMARIES have already been selected in that piece of sky). When this situation arises, the following steps may be performed. First, fetch a list of all segments (both resolved and unresolved), using the proc segmentsGet:
ts> set segments [segmentsGet]Next, call the proc repeatSegmentsGet, which will parse the list of segments and return a list of all segments which have not been resolved, have an acceptable survey quality, and are completely overlapped by previously resolved primary segments:
ts> set repeats [repeatSegmentsGet]A TCL list of segments is returned, where each element in the list is itself a 5-element TCL list, consisting of the run number, camera column, rerun number, starting great circle longitude (in integer arcsecs), and ending great circle longitude (in integer arcsecs) of the segment. If desired, one can remove segments from the list that you don't wish to resolve (using the TCL command "lreplace"). Finally, resolve that set of repeat segments using the repeatResolve proc:
ts> repeatResolve $repeats yanny
ts> dbTargetSelect tsPlan.par {adelman yanny}
The first argument is the name of the plan file, and the second argument the
list of folks who are running target selection. If this chunk has already
been run through target selection, you may force it to be rerun through
target selection again using the "-force" argument (all effects of the
previous target selection will be undone):
ts> dbTargetSelect tsPlan.par {adelman yanny} -force
The "target" product provides a routine, "taDiagnostics", to generate various
QA files after target selection runs off of flatfiles. A simple TCL proc,
tsDiagnostics, is provided to
provide the same functionality out of the OPDB. It only works if flatfiles
were exported when target selection ran. It simply generates a "tsChunk" file
and then calls "taDiagnostics" on the flatfiles. Just run the routine on
the same plan file used to run target selection:
ts> tsDiagnostics tsPlan.parYou can optionally specify a directory for the output files. For example, to create all output files in the directory "output":
ts> tsDiagnostics tsPlan.par output
ts> manualSelect tsManual.par
ts> untiledToFitsThe file, named "tiInputTargets.par", will be created in the current directory. Optionally, the output directory for the file may be specified. For example, to create the "tiInputTargets.par" file in the directory "output":
ts> untiledToFits outputThe "tile" postprocessor is described in the "photoSch" product.
ts> completeTilePreThe files, named "tiTileCenters.par" and "ctObjs.fit", are created in the current directory. Optionally, the output directory for the file may be specified. For example, to create the files in the directory "/data/dp1.9/data/completeTile":
ts> completeTilePre /data/dp1.9/data/completeTileThe "completeTile" postprocessor is described in the "photoSch" product.
ts> set run [framesRunGet 94 1] ts> unsetObjectStatus $run ts> objDel $run
ts> set run [framesRunGet 94 1] ts> runUnmerge $run ts> objDel $run
ts> fieldQualitySet 94 1 undetermined 120 248
ts> set chunk [queryGetOne arChunk $photo_segments \
"stripe_ == 15 && startMu_ == 65378"]
ts> chunkUnresolve $chunk
ts> set chunk [queryGetOne arChunk $photo_segments \
"stripe_ == 15 && startMu_ == 65378"]
ts> dbTargetUnselectFromChunk $chunk
ts> objDel $chunk