================================================================================ This file describes changes made to the PTCLM tool ================================================================================ ================================================================================ Originator: sacks Date: May/21/2014 Tag: PTCLM2_140521 One-line: rename fpftdyn to flanduse_timeseries, and do some other related variable renames Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in dynlu_crops_n03_addclm50bld_n06_clm4_5_72 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values This is meant to go with an upcoming CLM tag, currently slated for clm4_5_74 A + PTCLM_sitedata/cnvrt_trnsyrs2_landuse_timeseries_txtfile.pl D PTCLM_sitedata/cnvrt_trnsyrs2_pftdyntxtfile.pl M PTCLMmkdata M README M test/compdirs/global_US-UMB/user_nl_clm M test/compdirs/rcp26_US-Dk2/user_nl_clm M test/compdirs/rcp45_US-Dk3/user_nl_clm M test/compdirs/rcp6_US-IB1/user_nl_clm M test/compdirs/rcp85_US-Me4/user_nl_clm M test/compdirs/trans_20th_US-Ne3/user_nl_clm M test/compdirs/trans_clm45_US-Ha1/user_nl_clm ================================================================================ Originator: erik Date: Apr/23/2014 Tag: PTCLM2_140423 One-line: Fix documentation, set DIN_LOC_ROOT_CLMFORC, correct call to mkmapdata.sh allow release_tags in version, add hopper/edison to machine list. Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_71 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values A PTCLMsublist --------- Script to submit a list of sites to the batch queue A PTCLMsublist_prog.py - Python code to handle the submision list A batchque.py ---------- Python code to handle batch command submission for single processor M PTCLMmkdata -- Set DIN_LOC_ROOT_CLMFORC, Correct call to mkmapdata.sh Also allow release_tags in version find. M README ------- Correct compset used. M buildtools --- Add edison/hopper to list of machines will work for change frankfurt to goldbach, update netcdf version for yellowstone M test/compdirs/* -------- Update the comparison files M test/listings/sitelist - Update sitelist M test/listings/help ----- Update help M test/listings/list ----- Update list M test/PTCLMtestlist.py ----- Change result codes to be more consistent with other testing mechanisms M test/PTCLMtesting_prog.py - Change result codes, and set sdate/map_gdate so sdate won't change with the day testing was run. M test/PTCLMtestlist.xml ---- Add new tests for new sites ------> Add new sites for Rosie, RF-Bra,LBA-Cax,LTER-Sev ------> And Jinyun US-Bol M PTCLM_sitedata/PTCLMDATA_pftdata.txt M PTCLM_sitedata/PTCLMDATA_sitedata.txt M PTCLM_sitedata/PTCLMDATA_soildata.txt ------> Add new comparison files for testing A test/compdirs/std_RF-Bra A test/compdirs/std_RF-Bra/user_nl_clm A test/compdirs/std_RF-Bra/run.log A test/compdirs/std_RF-Bra/xmlchange_cmnds A test/compdirs/std_RF-Bra/README.PTCLM A test/compdirs/std_LBA-Cax A test/compdirs/std_LBA-Cax/user_nl_clm A test/compdirs/std_LBA-Cax/run.log A test/compdirs/std_LBA-Cax/xmlchange_cmnds A test/compdirs/std_LBA-Cax/README.PTCLM A test/compdirs/std_LTER-Sev A test/compdirs/std_LTER-Sev/user_nl_clm A test/compdirs/std_LTER-Sev/run.log A test/compdirs/std_LTER-Sev/xmlchange_cmnds A test/compdirs/std_LTER-Sev/README.PTCLM ================================================================================ Originator: erik Date: Feb/04/2014 Tag: PTCLM2_140204 One-line: Add testing directory, some reworking, and bug fixes, remove -mach Remove --mach, --sitegroupname options add --map_gdate options. Testing: Ran PTCLM tests in test directory, and test_driver.sh tests on yellowstone. Add test directory and XML based test list A test A test/compdirs A test/compdirs/* ----------- Compare directory files A test/PTCLMtestlist.py ----- Class for reading the test list A test/run_PTCLM_tests ------ Main test script A test/PTCLMtestlist.xml ---- XML test list A test/PTCLMtesting_prog.py - Main class for test script A test/listings ------------- List option compare directory A test/listings/sitelist ---- Site list compare A test/listings/help -------- Help compare A test/listings/list -------- List compare A test/README M buildtools ---- Remove *.o files for gen_domain and gmake clean everything else M PTCLMmkdata --- STARTDATE and ALIGNYR only set if using tower years. Add more metadata to filenames. More work on debug option so works well. Also run mksurfdata.pl in debug mode with --debug and --allownofile so will test it's usage. Remove -mach option, add --map_gdate option so can use old maps easier. Add function to get filenames of created files, and touch the file in debug mode. Fix some of the use-case logic. Fix directory logic, and allow being able to run from a different directory. Exercise more commands in system function for debug mode. Remove -m option M mydatafiles/1x1pt_US-UMB/user_nl_clm M mydatafiles/1x1pt_US-UMB/xmlchange_cmnds M mydatafiles/1x1pt_US-UMB/README.PTCLM M README --- Remove -m option Add some test sites for testing: TS-Ts1, TS-Ts2, TS-Ts3 M PTCLM_sitedata/PTCLMDATA_sitedata.txt ----- Add test sites, change align year as first year to use when cycle_forcing used This is usually the first year, unless it's a leap year and it's the year after that. If the site has less than 3 years it's "1". M PTCLM_sitedata/PTCLMDATA_soildata.txt M PTCLM_sitedata/PTCLMDATA_pftdata.txt ================================================================================ Originator: erik Date: Nov/22/2013 Tag: PTCLM2_131122c One-line: Add some example files Testing: limited setup for US-UMB case on yellowstone A mydatafiles/1x1pt_US-UMB/user_nl_clm A mydatafiles/1x1pt_US-UMB/xmlchange_cmnds A mydatafiles/1x1pt_US-UMB/README.PTCLM ================================================================================ Originator: erik Date: Nov/22/2013 Tag: PTCLM2_131122b One-line: Rename a bunch of files, add US-UMB sample data Testing: limited setup for US-UMB case on yellowstone Rename files to new names: A + PTCLMmkdata A + PTCLM_sitedata/PTCLMDATA_pftdata.txt A + PTCLM_sitedata/PTCLMDATA_sitedata.txt A + PTCLM_sitedata/PTCLMDATA_soildata.txt A mydatafiles/1x1pt_US-UMB ---- sample directory for US-UMB site D PTCLM.py D PTCLM_sitedata/EXAMPLE_sitedata.txt D PTCLM_sitedata/EXAMPLE_soildata.txt D PTCLM_sitedata/EXAMPLE_pftdata.txt >>>>> Remove other site data files as we already have a lot of sample data in the PTCLMDATA file set. D PTCLM_sitedata/LBA_pftdata.txt D PTCLM_sitedata/LBA_sitedata.txt D PTCLM_sitedata/LBA_soildata.txt D PTCLM_sitedata/Fluxnet-Canada_sitedata.txt D PTCLM_sitedata/Fluxnet-Canada_soildata.txt D PTCLM_sitedata/Fluxnet-Canada_pftdata.txt D PTCLM_sitedata/AmeriFlux_sitedata.txt D PTCLM_sitedata/AmeriFlux_soildata.txt D PTCLM_sitedata/AmeriFlux_pftdata.txt D PTCLM_sitedata/CarboEurope_pftdata.txt D PTCLM_sitedata/CarboEurope_sitedata.txt D PTCLM_sitedata/CarboEurope_soildata.txt >>>>>> Update documentation M KnownBugs M README ================================================================================ Originator: erik Date: Nov/22/2013 Tag: PTCLM2_131122 One-line: Add more documentation, remove some case settings not needed, make work a bit cleaner Testing: limited setup for US-UMB case on yellowstone >>>> Add README file for the sample mydatafiles directory A https://svn-ccsm-models.cgd.ucar.edu/PTCLM/trunk_tags/PTCLM2_131119/mydatafiles/README M https://svn-ccsm-models.cgd.ucar.edu/PTCLM/trunk_tags/PTCLM2_131119/PTCLM.py -- remove some options not needed. Fix directory, and setup user_nl_clm and xmlchange_cmnds files. M https://svn-ccsm-models.cgd.ucar.edu/PTCLM/trunk_tags/PTCLM2_131119/README -- update doc... ================================================================================ Originator: erik Date: Nov/19/2013 Tag: PTCLM2_131119 One-line: Remove create_newcase call from PTCLM, just create datasets Testing: limited setup for US-UMB case on yellowstone Requires: scripts4_131119 Change PTCLM to JUST be a tool that creates datasets rather than also creates a case for you. Running create_newcase is now a seperate step that you use the -user_mods_dir option with. There are some issues with this version. But, it has basic functionality. The xmlchange_xcmnds file that is created has many fields multiple times. The dataset location is also screwy. Change name for CHATS datasite to US-CHATS so name is more similar to the other FluxNet sites. CHATS is an experiment that NCAR ran. A buildtools -- new script to build all the tools needed for PTCLM M PTCLM.py ---- Change so that only creates datasets, does NOT run create_newcase M README ------ Update documentation M PTCLM_sitedata/EXAMPLE_sitedata.txt -- Add campaign name to end M PTCLM_sitedata/EXAMPLE_soildata.txt -- Change CHATS to US-CHATS M PTCLM_sitedata/EXAMPLE_pftdata.txt --- Change CHATS to US-CHATS ================================================================================ Originator: erik Date: Sep/29/2013 Tag: PTCLM1_130929 cesm_baseline: scripts4_130929 One-line: default compset: I1PTCLM45, files go under site subdir, add script to rename maps Testing: limited ran mexicocity, US-UMB, US-Ha1, and BE-Vie cases M PTCLM.py -------------------- Make default compset: I1PTCLM45, make files in resolution subdiretory (under mydatadir) M README ---------------------- Update documentation A mydatafiles/renamemapfiles -- Script to rename mapping files creation date, so don't have to recreate mapping files again. Remove directory NOT needed anymore... D usr_files/mkgriddata.TEMPLATE D usr_files ================================================================================ Originator: erik Date: Sep/23/2013 Tag: PTCLM1_130923 cesm_baseline: scripts4_130916 One-line: Add --cycle_forcing, and --mydatadir options, remove useQIAN, rename --QIAN_tower_yrs to --use_tower_yrs Testing: limited ran a couple cases M PTCLM.py -------- Changes from Keith Oleson to cycle_forcing, check what model version is being used (clm4_0 or clm4_5) set time-step. D testcases.csh --- Remove as not working well on yellowstone ================================================================================ Originator: erik Date: Sep/20/2013 Tag: PTCLM1_130920 cesm_baseline: scripts4_130920 One-line: Make datafiles in mydatafiles directory Testing: Ran basic US-UMB, US-NR1 and 1x1_mexicocityMEX cases M PTCLM.py ---------------- File changes from Keith Oleson M PTCLM_sitedata/Fluxnet-Canada_sitedata.txt M PTCLM_sitedata/EXAMPLE_sitedata.txt M PTCLM_sitedata/Fluxnet-Canada_soildata.txt M PTCLM_sitedata/AmeriFlux_sitedata.txt M PTCLM_sitedata/EXAMPLE_soildata.txt M PTCLM_sitedata/AmeriFlux_soildata.txt M PTCLM_sitedata/Fluxnet-Canada_pftdata.txt M PTCLM_sitedata/EXAMPLE_pftdata.txt M PTCLM_sitedata/AmeriFlux_pftdata.txt ================================================================================ Originator: erik Date: Sep/10/2013 Tag: PTCLM1_130910 cesm_baseline: scripts4_130910 One-line: Get rid of pft-phys file copy, fix mapping file names, correct domain file Testing: Limited, basic test setup M PTCLM.py --------- Remove pft-phys file copy, fix mapping file name, correct domain file path M testcases.csh ---- change compsets testing, unzip log file to check status ================================================================================ Originator: erik Date: Jul/24/2013 Tag: PTCLM1_130724 cesm_baseline: scripts4_130724 One-line: Add some more error checking, more changes to get things working Testing: Limited, basic test setup M PTCLM.py ------- error checking, correct mapfile name, get gen_domain working, set ATM/LND_DOMAIN_FILE ================================================================================ Originator: erik Date: May/29/2013 Tag: PTCLM1_130529 cesm_baseline: scripts4_130529 One-line: Get PTCLM working with the new CLM tools Testing: Limited, test mexicocity, and 1x1_US-UMB basic case setup M PTCLM.py ------- Remove spinup options, get working with new tools M testcases.csh -- Remove spinup tests, get working again ================================================================================ Originator: erik Date: Feb/16/2013 Tag: PTCLM1_130216 cesm_baseline: scripts4_130130 One-line: Get supported compsets working and some progress with tools Testing: Limited -- but surported res work on ys M PTCLM.py M testcases.csh ================================================================================ Originator: erik Date: Jan/30/2013 Tag: PTCLM1_130130 cesm_baseline: scripts4_130130 One-line: Initial changes to get PTCLM working with new scripts and tools Testing: Limited testing Initial add of yellowstone to testcases and switch frankfurt for edinburgh. Remove generic stuff for userdefined. Remove ndep/aer grid. M PTCLM.py M testcases.csh D usr_files/mkdatadomain.TEMPLATE ================================================================================ Originator: erik Date: Jan/11/2013 Tag: PTCLM1_130111 cesm_baseline: scripts4_120125 One-line: Add some new sitedata from Keith Oleson Testing: none A PTCLM_sitedata/LBA_pftdata.txt M PTCLM_sitedata/Fluxnet-Canada_sitedata.txt M PTCLM_sitedata/EXAMPLE_sitedata.txt A PTCLM_sitedata/CarboEurope_pftdata.txt M PTCLM_sitedata/AmeriFlux_sitedata.txt M PTCLM_sitedata/Fluxnet-Canada_soildata.txt M PTCLM_sitedata/EXAMPLE_soildata.txt M PTCLM_sitedata/AmeriFlux_soildata.txt A PTCLM_sitedata/LBA_sitedata.txt A PTCLM_sitedata/LBA_soildata.txt A PTCLM_sitedata/CarboEurope_sitedata.txt A PTCLM_sitedata/CarboEurope_soildata.txt M PTCLM_sitedata/Fluxnet-Canada_pftdata.txt M PTCLM_sitedata/EXAMPLE_pftdata.txt M PTCLM_sitedata/AmeriFlux_pftdata.txt ================================================================================ Originator: mvertens Date: Jan/25/2012 Tag: PTCLM1_120125 cesm_baseline: scripts4_120125 One-line: Remove USE_MPISERIAL replace with MPILIB ================================================================================ Originator: erik Date: Nov/29/2011 Tag: PTCLM1_111129 cesm_baseline: clm4_0_39 / scripts4_111129 One-line: Use CLM_USRDAT for resolution or supported single-point res name so will work with latest scripts version Bugs-fixed: Known Bugs: (See KnownBugs file for details) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: Limited testing bluefire testing: All PASS M PTCLM.py -- Set res in create_newcase to either supported 1pt name or to CLM_USRDAT. ================================================================================ Originator: erik Date: Nov/14/2011 Tag: PTCLM1_111114 cesm_baseline: clm4_0_38 / scripts4_111108 One-line: Update path for NetCDF4 Bugs-fixed: Known Bugs: (See KnownBugs file for details) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: Limited testing bluefire testing: All PASS Update path for NetCDF for bluefire, jaguar, lynx, and mirage. M testcases.csh ================================================================================ Originator: erik Date: Sep/02/2011 Tag: PTCLM1_110902 cesm_baseline: clm4_0_35 / scripts4_110902 One-line: Change name of mksurfdata to mksurfdata_map, fix soil texture of US-UMB Bugs-fixed: 1392 (US-UMB site has some incorrect data) Known Bugs: (See KnownBugs file for details) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: Limited testing bluefire testing: All PASS Get basic script working with non2D grid version of clm. mksurfdata directory was changed to mksurfdata_map, and tools directories now have a "src" subdirectory to build the code in them. Also fix soil texture for US-UMB site (bug 1392). M PTCLM.py M testcases.csh M PTCLM_sitedata/EXAMPLE_soildata.txt M PTCLM_sitedata/AmeriFlux_soildata.txt ================================================================================ Originator: erik Date: Jul/26/2011 Tag: PTCLM1_110726 cesm_baseline: clm4_0_33 / scripts4_110724 One-line: Test if should use skip_rundb option or not, fix transient problem Bugs-fixed: 1368 (PTCLM for US-UMB spins up with zero GPP) 1361 (Problem with transient compsets for PTCLM) Known Bugs: (See KnownBugs file for details) 1392 (US-UMB site has some incorrect data) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: All tests pass up to the spinup tests where they all fail bluefire testing: PTCLM.*_US-UMB_ICN_exit_spinup.PTCLM jaguar testing: PTCLM._US_US-UMB_ICN_exit_spinup.PTCLM edinburgh testing: PTCLM.*_US-UMB_ICN_exit_spinup.PTCLM yong/intel testing: PTCLM.*_US-UMB_ICN_exit_spinup.PTCLM M PTCLM.py ------ Add check for finidat file, check if should use -skip_rundb option, fix transient issue M testcases.csh - Change order of arguments for spinup tests, so more likely to die on an error with finidat file M KnownBugs ----- Update list of bugs ================================================================================ Originator: erik Date: May/04/2011 Tag: PTCLM1_110504 cesm_baseline: clm4_0_28 / scripts4_110428a One-line: Move PTCLM to own external, and fix cnvrt script so can take year-range allow it to use older versions of Python Bugs-fixed: 1279 (Latest version of PTCLM requires python2.5) 1248 (PTCLM can only run until 2005) Known Bugs: (See KnownBugs file for details) 1392 (US-UMB site has some incorrect data) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: bluefire testing: All PASS jaguar testing: All PASS edinburgh testing: All PASS except 4 PTCLM.8696_US-UMB_I_1850.PTCLM FAIL 0 5 PTCLM.8696_US-UMB_I20TR.PTCLM FAIL 0 6 PTCLM.8696_US-UMB_I20TRCN.PTCLM FAIL 0 7 PTCLM.8696_US-UMB_ICN.PTCLM FAIL 0 8 PTCLM.8696_US-UMB_I1850CN.PTCLM FAIL 0 9 PTCLM.8696_US-UMB_IRCP85CN.PTCLM FAIL 0 10 PTCLM.8696_US-UMB_I.PTCLM FAIL 0 11 PTCLM.8696_US-UMB_I_QIAN.PTCLM FAIL 0 12 PTCLM.8696_US-UMB_I.PTCLM FAIL 0 14 PTCLM.8696_US-UMB_ICN_exit_spinup.PTCLM FAIL 0 15 PTCLM.8696_US-UMB_ICN_final_spinup.PTCLM FAIL 0 yong/intel testing: All PASS Send a year range to the convert script and have years that start at the beginning of the year range and go to the end of it. Add a DEBUG option to testcases.csh so can just run tests quickly to debug test script. M PTCLM.py M PTCLM_sitedata/cnvrt_trnsyrs2_pftdyntxtfile.pl M testcases.csh M KnownBugs M PTCLM_sitedata/EXAMPLE_sitedata.txt - Remove all but US-UMB M PTCLM_sitedata/EXAMPLE_soildata.txt - Remove all but US-UMB M PTCLM_sitedata/EXAMPLE_pftdata.txt -- Remove all but US-UMB ================================================================================ Originator: erik Date: Feb/03/2011 Tag: clm4_0_23 / scripts4_110111 One-line: Fix --list and --cesm_root options, add --scratchroot, fix bugs update use-cases and compset names, add CESM_ROOT/CLM_SOFF to testcases.csh Bugs-fixed: 1256 (fix PTCLM testcases.csh test script on jaguar to use netcdf/3.6.2) 1254 (pft-physiology copy doesn't have .nc extension/can't handle new files) 1250 (add ability to set scratchroot) 1224 (Fix -aerdepgrid/ndepgrid options in PTCLM.py) Known Bugs: (See KnownBugs file for details) 1392 (US-UMB site has some incorrect data) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) 1248 (PTCLM can only run until 2005) Testing: bluefire testing: All PASS edinburgh testing: All Fail (Python is too old 2.4 when needs 2.5) yong/intel testing: All PASS except... myPTCLMtests_US-Ha1_I_1850.PTCLM FAIL 0 myPTCLMtests_US-Ha1_I20TR.PTCLM FAIL 0 M PTCLM.py ------- Fix bugs, get --list and --cesm_root options working, add --scratchroot, update CLM use-cases and how queried add ability to add directory to caseprefix M testcases.csh -- Add ability to use env vars CESM_ROOT and CLM_SOFF set scratchroot for generic machines. Put case directories under the PTCLM directory, so can delete it easier M KnownBugs ================================================================================ Originator: erik Date: Jan/12/2011 Tag: clm4_0_21 / scripts4_110108 One-line: Create ChangeLog file for PTCLM Testing: bluefire testing: All PASS edinburgh testing: All PASS up to ... myPTCLMtests_US-Ha1_I_1850.PTCLM FAIL 0 yong/intel testing: All PASS up to... myPTCLMtests_US-Ha1_I_QIAN.PTCLM PASS ================================================================================ Originator: erik Date: Dec/06/2010 Tag: scripts4_101206 One-line: Add some documentation files M ccsm_utils/Tools/lnd/clm/PTCLM/testcases.csh Add USER_CC setting to yong M ccsm_utils/Tools/lnd/clm/PTCLM/README ------- Add note about aerdepgrid/ndepgrid A ccsm_utils/Tools/lnd/clm/PTCLM/KnownBugs ---- Add file with list of bugs ================================================================================ Originator: erik Date: Dec/02/2010 Tag: scripts4_101202 One-line: Updates for script changes >>>>>>>>>>>> Get PTCLM working with changes >>>>>>>>>>>> PTCLM updates from mpiserial branch >>>>>>>>>>>> Add PTCLM tests for yong (Mac OS-X laptop) M ccsm_utils/Tools/lnd/clm/PTCLM/PTCLM.py M ccsm_utils/Tools/lnd/clm/PTCLM/testcases.csh M ccsm_utils/Tools/lnd/clm/PTCLM/README ================================================================================ Originator: erik Date: Aug/30/2010 Tag: scripts4_100830 One-line: Bring PTCLM branch to scripts trunk ================================================================================