Usage

Target Selection (target) contains C code to select spectroscopic targets from a set of FITS files containing calibrated objects. It is intended to provided a test bed for Working Groups to develop target selection code outside the environment of the Operational Database. The actual target selection performed within the Operational Database links to the C library contained in this product.

Definitions

First some definitions:
Calibrated object
A calibrated object is very similar to an object produced by the Frames Pipeline, but now all quantites are calibrated; e.g., magnitudes rather than counts, arcsecs rather than pixels, etc.
Primary object
A primary object is an object which belongs to the primary science sample; i.e., it is eligible for consideration as a spectroscopic target for the primary target selection criteria. While a given object in the sky may be detected multiple times by the survey, there will only be one primary object associated with it.
Segment
A segment is a piece of a single Frames Pipeline Run, delimited by a starting (inclusive) and ending (exclusive) great circle longitude (expressed in integer arcsecs), and is considered to consist of all objects detected by that Frames Pipeline Run which fall within that great circle range. It is the smallest unit by which we divide up the sky, mapping survey objects to the sky. Segments have three flavors:

primary
This segment contains all primary objects for the portion of sky to which it maps (although not all objects contained in the segment are necessarily primary objects). No other primary segments on the same scanline can overlap this segment. The primary survey is defined by the union of all primary segments.
secondary
This segment does not contain any primary objects for the portion of sky to which it maps, although the data is acceptable. It may overlap any other segments on the same scanline, and in fact will be completely overlapped by the union of one or more primary segments.
bad
This segment does not contain any primary objects for the portion of sky to which it maps. The data is not considered of acceptable quality, and thus should not be used. The segment may overlap any other segments on the same scanline.
Chunk
A chunk is a piece of a stripe, delimited by a starting (inclusive) and ending (exclusive) great circle longitude (expressed in integer arcsecs), and consists of a set of segments whose union covers the longitude range in all twelve scanlines in the stripe, including a set of primary segments whose non-overlapping union exactly covers the chunk. All segments belong to a chunk. Chunks are the fundamental unit for tracking the survey coverage of the sky.

Inputs

The preceding definitions are necessary to understand the format of the input data. In normal operations, target selection is performed on all primary and secondary objects in a single chunk. For each chunk, the following input files exist:
tsChunk-ss-mmmmmm.par
where ss is the stripe number, and mmmmmm is the starting great circle longitude (mu) of the chunk in integer arcsecs. This file lists all segments contained in the chunk.
tsObj-rrrrrr-c-v-ffff.fit
where rrrrrr is the imaging run number, c is the camera column (1-6), v is the rerun number, and ffff is the field number for the first field in the file. This is a FITS file, containing a FITS binary table HDU for each field in the segment, where each table contains all calibrated objects (possibly limited to primary objects only) in the field. There is one such file for each segment. Additionally there are numerous standard survey tunable parameter files, which will be discussed below.

Target Selection Algorithms

Each object may satisfy numerous target selection criteria. The different types of targets are enumerated in the file arTargetType.h (this file must NOT be edited by anyone but the Keeper Of The Data Model). These types are grouped into two classes: primary - those which are considered primary survey samples and thus whose selection algorithms are the responsibility of the Working Groups; and secondary - those which are more operational in nature and not the responsibility of the Working Groups. Target types are subgrouped further into natural divisions to facilitate code development. Thus, primary target types are subgrouped by Working Groups. All secondary target types are grouped under the heading "standards". The following Working Groups are responsible for developing and maintaining target selection algorithms: Additionally, for purposes of code development we will also refer here to a "Standards" Working Group, which is responsible for the secondary target types, and the "Rosat" Working Group, which is responsible for the ROSAT raget flags. No such Working Groups exists, but the code pretends they do. Each working group must maintain the following four files (the file names are given for the Galaxies Working Group; substitute "Galaxies" with the name of your Working Group below):
include/taGalaxies.h
This should define a single tunable parameters structure, TA_GALAXIES_PARAMS, listing all tunable parameters for your selection algorithm (i.e., all parameters whose values are set by an FTCL par file and which must be recorded in the Operational Database).
src/galaxies.c
This must define the function: 
     int
     taGalaxiesTargetSelect(TA_CALIB_OBJ *obj,
	  	            /* MOD: calibrated object to test */
		            const TA_GALAXIES_PARAMS *params
		            /* IN: tunable parameters*/)
This function tests the input calibrated object (TA_CALIB_OBJ) against all target selection criteria which are the responsibility of the Working Group. For each criterion the object satisfies, the function must set that target type's bit in the target bit mask (for all Working Groups, the target bit mask is obj->primTarget. For example, if the object satisifies the selection criterion for a red galaxy: 
     obj->primTarget |= AR_TARGET_GALAXY_RED:
The function should assume that all target bits which it is responsible for have been unset before the function is called. The function must not change anything else in the calibrated object structure. The function returns the target selection priority for the object, which must be 0 if the target met none of this module's target selection criteria, and in the range 1-255 if it did (higher number = higher priority).
etc/galaxiesParamsRead.tcl
This contains a single TCL proc, galaxiesParamsRead, which reads the Galaxies Working Group's tunable parameters file into a TA_GALAXIES_PARAMS structure and returns a handle to the structure.
examples/tsGalaxiesParams.par
An up-to-date example of the tunable parameters file. Be sure to forward any changes to Brian Yanny.

Matching Algorithms

Target Selection requires that object lists be matched against three different external catalogs; FIRST, ROSAT, and PMM. Matching occurs as part of Target Selection, and is done before actual selection of targets. Whoever is responsible for these matching algorithms must modify the following files: (structure and subroutine names are given for the FIRST catalog; substitute FIRST with ROSAT or PMM for the other catalogs):
include/taCatalogs.h
This defines the structures for each catalog. Each catalog is defined by two structures. The first defines a single catalog entry (TA_FIRST). The second defines the catalog (TA_FIRST_CATALOG), and include the number of objects in the catalog, an array of the objects, and any parameters that control the matching algorithm.
src/tclCatalogs.c
C code to read the catalogs from FITS files into the C structures. Each catalog has a single C routine (i.e., tclFirstRead).
src/catalogs.c
The matching algorithm, captured in a single subroutine for each catalog (i.e., taObjMatchAgainstFirst()).
examples/tsMatchParams.tcl
An up-to-date example of the tunable parameters file which control matching. There is one file containing the parameters for all three catalogs. Be sure to forward any changes to Brian Yanny.
etc/targetSelect.tcl
The tunable parameters read from the tunable parameters file are copied to the C structures for the catalogs in this file, just after the catalogs are read. Search for the string "$firstMatch == 1".

How to run TARGET

The executable is named target. Selection may be performed on units of chunks, segments, CHAINs of calibrated objects, or a single calibrated object. Examples of each are given here. In all cases, the output consists of merely changing the target bit masks (obj->primTarget and obj->secTarget in a TA_CALIB_OBJ structure, fields primTarget and secTarget in a FITS calibrated object file), the target bit (AR_OBJECT_STATUS_TARGET) in the status bit mask (obj->status in a TA_CALIB_OBJ structure, field status in a FITS calibrated object file), and the FIRST, ROSAT, and PMM fields. The target bit masks and target bit in the status bit mask are first unset. Then, for each object, it is first matched against the selected catalogs, then for each tested selection criterion which the object satisfies, the relevant bit in the target bit masks is set. If the object satisfies one or more tested selection criteria, the target bit in the status bit mask is set.

Target selection on chunks

In normal operations, target slection will be performed in units of chunks. In product, this works off a standard SDSS "plan" file. A sample plan file may be found in the "examples" directory. For example, if our chunk file is called "tsChunk-25-647905.par", and we're matching against all catalogs and selecting targets in all classes, the plan file may look like: 
inputDir		.
outputDir		.
chunkFile		tsChunk-25-647905.par.par
parametersDir		.

matchParameters         tsMatchParams.par
starsParameters		tsStarsParams.par
quasarsParameters	tsQuasarsParams.par
galaxiesParameters	tsGalaxiesParams.par
serendipityParameters	tsSerendipityParams.par
standardsParameters	tsStandardsParams.par

firstCatalog /data/dp1.16/data/first/first_south.fit
rosatCatalog /data/dp1.16/data/rosat/rosat.fit
To run target selection, then enter the following command: 
target> targetSelectFromChunk tsPlan.par
Target selection will be performed using only those criteria which have a tunable parameters file specified.

Each object file (e.g., tsObj-0000582-1-0-0004.fit) will be rewritten as an exact copy (with "tsObj" in the name substituted with "tsObjOut") except that the target bit masks, target bit in the status bit mask, and FIRST, ROSAT, and PMM fields rewritten.

There are two flags which allow you to control the output: -ascii and -fits. They each take an integer which specifies whether there should be no output (0), only targets output (1), or all objects output (2). These flags are independent of each other, so you could for instance ouput all the objects in a fits file and only the targets in an ascii file by using "-ascii 1 -fits 2". The default is "-ascii 0 -fits 2".

Target selection on segments

To perform target selection on all primary and secondary objects in a single segment: 
target> set quasars [quasarsParamsRead tsQuasarsParams.par]
target> set stars [starsParamsRead tsStarsParams.par]
target> targetSelectFromSegment tsObj-000582-1-0-0004.fit \
	     -quasars $quasars -stars $stars
The first argument is the "tsObj" file for a single segment. The flags -quasars, -stars, etc are used to specify handles to the tunable parameter structures (e.g., TA_QUASARS_PARAMS, TA_STARS_PARAMS) to use for each subgroup of target types. Target selection will be performed using only those criteria which have a tunable parameters structure specified. Thus, in the above example, target selection will be performed for quasars and stars only.

The object file (e.g., tsObj-0000582-1-0-0004.fit) will be rewritten as an exact copy (with "tsObj" in the name substituted "tsObjOut") except that the target bit masks and the target bit in the status bit mask are rewritten.

Target selection on CHAINs of calibrated object and single objects

To perform target selection on all objects in, for example, the first field in a segment: 
target> set quasars [quasarsParamsRead tsQuasarsParams.par]
target> set stars [starsParamsRead tsStarsParams.par]
target> set tbl [fitsRead [tblColNew] tsObj-000582-1-0-0004.fit -hdu 1]
target> set fieldInfo [genericNew TA_FIELD_INFO]
target> handleSet $fieldInfo.field 4
target> handleSet $fieldInfo.seeing<0> [hdrGetAsDbl $tbl.hdr SEEING_U]
target> handleSet $fieldInfo.seeing<1> [hdrGetAsDbl $tbl.hdr SEEING_G]
target> handleSet $fieldInfo.seeing<2> [hdrGetAsDbl $tbl.hdr SEEING_R]
target> handleSet $fieldInfo.seeing<3> [hdrGetAsDbl $tbl.hdr SEEING_I]
target> handleSet $fieldInfo.seeing<4> [hdrGetAsDbl $tbl.hdr SEEING_Z]
target> set objChain [calibObjsReadFromTbl $tbl -field $fieldInfo]
target> targetSelect $objChain -quasars $quasars -stars $stars
To perform target selection on the first object in the first field, substitute the last line above with: 
target> set obj [chainElementGetByPos $objChain HEAD]
target> targetSelect $obj -quasars $quasars -stars $stars
The first argument is a handle to either a CHAIN of TA_CALIB_OBJ or a single TA_CALIB_OBJ. The flags -quasars, -stars, etc are used to specify handles to the tunable parameter structures (e.g., TA_QUASARS_PARAMS, TA_STARS_PARAMS) to use for each subgroup of target types. Target selection will be performed using only those criteria which have a tunable parameters structure specified. Thus, in the above example, target selection will be performed for quasars and stars only.

The target bit masks and the target bit in the status bit mask for all objects in the CHAIN (or for the single object) will be rewritten in place.

How to run calibrate

A routine is provided to take the raw pipeline flatfiles and convert them into calibrated "tsObj" files. Besides applying the photometric and astrometric calbrations, the object status flags are set as if processing through "resolve" was accomplished (i.e., setting all status flags up through PRIMARY as appropriate). This task essentially performs all functions of the Operational Database, except no merging or resolving at boundaries is performed.

The routine is set up to work by default with the current data layout at Fermilab (i.e., data is routed in the directory /data/dp3.b/data). If data is located elsewhere, optional arguments are available to specify the location of the raw data. For example, to create the "tsObj" files for run 94, column 1, fields 11 to 545 in the current directory, assuming the inputs are found in the default directory structure: 

target> calibrate 94 1 11 545
See the documentation for the procedure for a description of the optional arguments.