============================================================================ ============================================================================ Developer's guide for mksurfdata_map ============================================================================ ============================================================================ ============================================================================ Table of Contents ============================================================================ I. Adding a new raw data file II. Adding mapping files for a raw data file with a new grid / landmask ============================================================================ I. Adding a new raw data file ============================================================================ Here is what you need to change when adding a new raw data file, with one or more fields that need to be remapped to the CLM resolution. Note that additional steps are needed (beyond what's listed here) when the field you're adding specifies something about the subgrid breakdown (into landunits, columns & pfts): for those fields, additional code is needed to ensure that percentages add to 100%. Note: The following assumes that the new file uses an existing set of mapping files, or that you have already done everything necessary to add a new set of mapping files. If your mapping file has a new grid, or a new landmask on an existing grid, see the instructions for adding mapping files in a separate section of this document. - Add a new module in mksurfdata_map/src that provides a routine for remapping your new field(s). Note that there is generally one module per input file; multiple fields can be regridded using a single subroutine or multiple subroutines. - Add your new file in mksurfdata_map/src/Srcfiles - Add new namelist options in mksurfdata_map/src/mkvarctl.F90; e.g., for a new field xxx: - mksrf_fxxx - map_fxxx - Add output calls in mksurfdata_map/src/mkfileMod.F90; you need to add calls in 3 places: - raw data file name attribute (nf_put_att_text call) - mapping file name attribute (nf_put_att_text call) - variable definition (ncd_defvar calls) Make sure this goes in an 'if (.not dynlanduse)' conditional, if appropriate - Add code in mksurfdata_map/src/mksurfdat.F90; you need to add the following: - add a 'use' statement to use your new regridding routine(s) - declare array(s) to hold data on the output grid - add your new mksrf_fxxx and map_fxxx variables to the 'namelist /clmexp/' declaration - document your new mksrf_fxxx and map_fxxx variables in the long comment following the 'namelist /clmexp/' declaration - add your new array(s) to the allocation statement under the heading "Allocate and initialize dynamic memory" - initialize your new array(s) in the initialization section following the allocation - add output to ndiag (the log file) documenting your new mksrf_fxxx and map_fxxx variables - add call(s) to your new subroutine(s) for regridding the data - add calls to nf_inq_varid & nf_put_var_double (or nf_put_var_int) for each new output variable; make sure to put these calls in the section for dynlanduse = false and/or true, as appropriate - add a deallocation statement for each new output variable - Add your new file in bld/namelist_files/namelist_definition_clm4_5.xml; e.g. (replace xxx with your new field): XXX dataset for mksurfdata - Add your new mksrf_fxxx variable to the list of valid_values for mksrf_filename in bld/namelist_files/namelist_definition_clm4_5.xml - Add defaults in bld/namelist_files/namelist_defaults_clm4_5_tools.xml; note that the "type" attribute is a short code that can be used in mksurfdata.pl, and doesn't have to match the "xxx" that is used elsewhere - lmask - hgrid - mksrf_filename - mksrf_fxxx (including hgrid and lmask attributes) - Add hooks to your new files in mksurfdata_map/mksurfdata.pl: - add new string in the list following 'foreach my $typ' - add the new mapping file to clmexp, as in: map_fxxx = '$map{'xxx'}' - add the new raw data file to clmexp, as in: mksrf_fxxx = '$datfil{'xxx'}' - Add new raw data file to the inputdata repository: lnd/clm2/rawdata - locally - check in to the inputdata svn repository - Add documentation for your new mksrf_fxxx in doc/UsersGuide/tools.xml ============================================================================ II. Adding mapping files for a raw data file with a new grid / landmask ============================================================================ If your raw data file is on a new grid, or just has a new landmask on an existing grid, you will need to perform a number of additional steps, as laid out here. - First, move your data file to the inputdata directory and give it its final name. (This will ensure that the appropriate metadata is put in the SCRIP grid file.) - Make a scrip grid file from your data file using mkmapgrids, and move it to the inputdata directory - Add a scripgriddata entry for the new scrip grid file in bld/namelist_files/namelist_defaults_clm4_5_tools.xml - If necessary, add other entries in namelist_defaults_clm4_5_tools.xml giving information about your scrip grid file: - If this is a high resolution grid (e.g., 3min or higher), add a scripgriddata_lrgfile_needed entry, saying we need 64bit_offset (or netcdf4) support for mapping files made with this scrip grid file - If the grid file is in UGRID format rather than SCRIP grid format, add scripgriddata_type and scripgriddata_meshname entries. If you don't know what I'm talking about, then your grid file is in SCRIP format and you can ignore this. - If necessary, add new grid and/or landmask to lists of valid values for hgrid, res and lmask in bld/namelist_files/namelist_definition_clm4_5.xml - Note that a new resolution currently needs to be added to both the hgrid and res lists of valid values, although in the future this should probably be changed so that these raw data grids just appear in hgrid - Add the new grid-landmask combo to the 'mapgrids' list in bld/namelist_files/checkmapfiles.ncl - Add the new grid-landmask combo to the 'grids' list in tools/shared/mkmapdata/mkmapdata.sh (in the clm4_5 branch of the conditional) - Make mapping files, from tools/shared/mkmapdata - Modify mkmapdata.sh: - edit the grids list so it only contains your new grid - Modify regridbatch.sh as desired, e.g.: - project number - number of processors (BSUB -n line, span, and the regrid_num_proc setting) - wall-clock limit - if ESMFBIN_PATH is in your environment, you may want to unset it; this can be important to allow mkmapdata.sh choose a different executable for mpi vs serial runs - if you renamed the mkmapdata.sh script, be sure to call the renamed script at the bottom of regridbatch.sh - Submit regridbatch.sh - When mapping files have all been created, run createXMLEntries.pl from tools/shared/mkmapdata (usage: just run the script with no arguments) - Cut and paste the xml entries from mapping_entries.txt (created by createXMLEntries.pl) into bld/namelist_files/namelist_defaults_clm4_5.xml, in the correct locations - Move mapping files to correct location, either using mv_cmds.sh created by createXMLEntries.pl, or using tools/shared/mkmapdata/mvNimport.sh. - Note that the latter also imports to the inputdata directory; if you don't use that, you'll need to add the files to the inputdata directory yourself