Public Member Functions | |
| def | __init__ |
| def | cmdline |
| def | known_input_type |
| def | add_input_file |
| def | configure_python_features |
| def | configure_ipython |
| def | configure_framework |
| def | configure_random |
| def | configure_run |
| def | configure_job |
| def | configure_geometry |
| def | configure_optmods |
| def | configure_args |
| def | configure_user |
| def | configure_post_user |
| def | configure_visualization |
| def | configure |
| def | spade_files |
| def | run_post_user |
| def | run |
| def | finalize |
Public Attributes | |
| opts_parser | |
| DATETIME_FORMAT | |
| post_user_algs | |
| spadeSvc | |
| opts | |
| args | |
| use_AES | |
| AES_window | |
| theApp | |
| input_files | |
| modules | |
This is the main program to run NuWa offline jobs.
It provides a job with a minimal, standard setup. Non standard
behavior can made using command line options or providing additional
configuration in the form of python files or modules to load.
Usage:
nuwa.py [options] [-m|--module "mod.ule --mod-arg ..."] \\
[config1.py config2.py ...] \\
[mod.ule1 mod.ule2 ...] \\
[[input1.root input2.root ...] or [input1.data ...]] \\
Python modules can be specified with -m|--module options and may
include any per-module arguments by enclosing them in shell quotes
as in the above usage. Modules that do not take arguments may
also be listed as non-option arguments. Modules may supply the
following functions:
configure(argv=[]) - if exists, executed at configuration time
run(theApp) - if exists, executed at run time with theApp set to
the AppMgr.
Additionally, python job scripts may be specified.
Modules and scripts are loaded in the order they are specified on
the command line.
Finally, input ROOT files may be specified. These will be read in
the order they are specified and will be assigned to supplying
streams not specificially specified in any input-stream map.
The listing of modules, job scripts and/or ROOT files may be
interspersed but must follow all options.
Definition at line 7 of file Control.py.
| def DybPython::Control::NuWa::__init__ | ( | self | ) |
Definition at line 10 of file Control.py.
00015 : 00016 00017 nuwa.py [options] [-m|--module "mod.ule --mod-arg ..."] \\ 00018 [config1.py config2.py ...] \\ 00019 [mod.ule1 mod.ule2 ...] \\ 00020 [[input1.root input2.root ...] or [input1.data ...]] \\ 00021 00022 Python modules can be specified with -m|--module options and may 00023 include any per-module arguments by enclosing them in shell quotes 00024 as in the above usage. Modules that do not take arguments may 00025 also be listed as non-option arguments. Modules may supply the 00026 following functions: 00027 00028 configure(argv=[]) - if exists, executed at configuration time 00029 00030 run(theApp) - if exists, executed at run time with theApp set to 00031 the AppMgr. 00032 00033 Additionally, python job scripts may be specified. 00034 00035 Modules and scripts are loaded in the order they are specified on 00036 the command line. 00037 00038 Finally, input ROOT files may be specified. These will be read in 00039 the order they are specified and will be assigned to supplying 00040 streams not specificially specified in any input-stream map. 00041 00042 The listing of modules, job scripts and/or ROOT files may be 00043 interspersed but must follow all options. 00044 00045 """ 00046 00047 def __init__(self): 00048 ' Create a NuWa instance.' 00049 from optparse import OptionParser 00050 from DybPython.hostid import hostid 00051 parser = OptionParser(usage=self.__doc__) 00052 00053 parser.add_option("-A", "--aes-trim-window",default="0",type="string", 00054 help="'None' for no AES. Turn on AES with zero or any positive value and use that for AES trimming window (in second).") 00055 00056 parser.add_option("-l", "--log-level",default=3,type="int", 00057 help="Set output log level.") 00058 00059 parser.add_option("-M","--msg-stream-level", action="append", 00060 help="Set individual MsgStream levels using form: 'StreamName,LevelNumber'") 00061 00062 parser.add_option("-L", "--file-list", default=None, type="string", 00063 help="Input file list") 00064 00065 parser.add_option("-C", "--color",default="no", 00066 help="Use colored logs assuming given background ('light' or 'dark')") 00067 00068 parser.add_option("-i", "--interactive",action="store_true",default=False, 00069 help="Enter interactive ipython shell after the run completes (def is batch).") 00070 00071 parser.add_option("-s", "--show-includes", action="store_true",default=False, 00072 help="Show printout of included files.") 00073 00074 parser.add_option("-m", "--module", action="append", 00075 help="Load given module and pass optional argument list") 00076 00077 parser.add_option("-n", "--executions", default=0, type="int", 00078 help="Number of times to execute list of top level algorithms.") 00079 00080 parser.add_option("-o", "--output",default="", 00081 help="Output filename") 00082 00083 parser.add_option("-O", "--output-streams",default="{}", 00084 help="Output file map") 00085 00086 parser.add_option("-I", "--input-streams",default="{}", 00087 help="Input file map") 00088 00089 parser.add_option("-H", "--hostid",default=hostid(), type="int", 00090 help="Force given hostid") 00091 00092 parser.add_option("-R", "--run",default=0, type="int", 00093 help="Set run number") 00094 00095 parser.add_option("-T", "--time",default="0", type="string", 00096 help="Set wall time") 00097 00098 parser.add_option("", "--dbconf",default=None , type="string", 00099 help="Sets DBCONF envvar identifying section of ~/.my.cnf to use for DB connection parameters and triggering DBI to use this config (DbiCascader.cxx) ") 00100 00101 parser.add_option("-N", "--execution",default=0, type="int", 00102 help="Set the starting execution number") 00103 00104 parser.add_option("-V", "--visualize", action="store_true", default = False, 00105 help="Run in visualize mode") 00106 00107 parser.add_option("-E", "--external-loop", type='string', default='', 00108 help="Delegate to an external loop, name a module with a 'start(app)' method defined.") 00109 00110 parser.add_option("-G", "--detector",default= "", 00111 help="Specify a non-default, top-level geometry file") 00112 00113 parser.add_option("-K", "--leak-check-execute",action="store_true",default= False, 00114 help="Use Hephaestus memory tracker") 00115 parser.add_option("","--leak-check-method",action="append", 00116 default=[], 00117 help="Add a method for the leak checker to focus on"); 00118 00119 parser.add_option("","--no-history",action='store_const',dest='history', 00120 const='off',help='Same as --history=off') parser.add_option("","--history",default="on",
| def DybPython::Control::NuWa::cmdline | ( | self, | ||
| argv | ||||
| ) |
Definition at line 121 of file Control.py.
00134 : {'file1':'histograms.root'}") 00135 00136 parser.add_option("", "--input-stats",default=None, 00137 help="Input file map for histograms. Ex: {'file1':'histograms.root'}") 00138 parser.add_option("", "--force-readout", action="store_true",default=False, 00139 help="Force Readout to be written to output file.") 00140 00141 import uuid 00142 parser.add_option("", "--jobid",default=str(uuid.uuid1()), 00143 help="Set the job ID for bookkeeping") 00144 00145 parser.add_option("", "--job-info",default="{}", 00146 help="Add or override job bookkeeping information") 00147 00148 00149 parser.disable_interspersed_args() 00150 self.opts_parser = parser 00151 self.DATETIME_FORMAT = '%Y-%m-%dT%H:%M:%S' 00152 self.post_user_algs = [] 00153 00154 # If the spade service (SpadeAlgo) is used it will register itself. 00155 self.spadeSvc = None; 00156 return 00157 00158 def cmdline(self,argv): 00159 'Parse command line' 00160 (options,args) = self.opts_parser.parse_args(args=argv) 00161 00162 self.opts = options 00163 self.args = args 00164 00165 # convert string to dictionary object
| def DybPython::Control::NuWa::known_input_type | ( | self, | ||
| fname | ||||
| ) |
Definition at line 166 of file Control.py.
00169 : 00170 self.opts.output_stats = mapify(self.opts.output_stats) 00171 if self.opts.input_stats: 00172 self.opts.input_stats = mapify(self.opts.input_stats) 00173 self.opts.job_info = mapify(self.opts.job_info)
| def DybPython::Control::NuWa::add_input_file | ( | self, | ||
| fname | ||||
| ) |
Add file name or list of file names to self.input_files, expanding if it is a .list file.
Definition at line 174 of file Control.py.
00177 : 00178 self.opts.time = int(mktime(strptime(self.opts.time, 00179 self.DATETIME_FORMAT)) 00180 -timezone) 00181 # Note: mktime adds the timezone, so we need to subtract it. 00182 else: 00183 self.opts.time = int(self.opts.time) 00184 00185 00186 from Tools import mapify, unitify 00187 if self.opts.aes_trim_window.lower() == "none": 00188 self.use_AES = False 00189 else: 00190 self.use_AES = True 00191 self.AES_window = unitify(self.opts.aes_trim_window,"second") 00192 if self.AES_window >=0 : 00193 print "AES is turned on and trimming window is set to ",\ 00194 self.AES_window/units.second," seconds" 00195 # valid 00196 pass 00197 else : 00198 print "error: invalid AES trimming window" 00199 exit() 00200 00201 return 00202 00203 def known_input_type(self,fname): 00204 'Return True if file name has a recognized extension.' 00205 for ext in ['.root','.data','.rraw','.list']: 00206 if fname[-len(ext):] == ext: 00207 return True 00208 continue 00209 return False 00210 00211 def add_input_file(self,fname): 00212 '''Add file name or list of file names to self.input_files, 00213 expanding if it is a .list file.'''
| def DybPython::Control::NuWa::configure_python_features | ( | self | ) |
Definition at line 214 of file Control.py.
00215 : return 00216 00217 #print 'add_input_file(%s)'%fname 00218 00219 # recurse if list 00220 if isinstance(fname,list): 00221 for name in fname: 00222 self.add_input_file(name) 00223 continue 00224 return 00225 00226 if not self.known_input_type(fname): 00227 msg = 'Got unknown input file type: "%s"'%fname 00228 print msg 00229 raise TypeError,msg 00230 00231 if fname[-5:] != '.list': 00232 self.input_files.append(fname) 00233 return 00234 00235 # process as a list file 00236 fp = open(fname) 00237 for line in fp.readlines(): 00238 line = line.strip() 00239 if len(line) == 0: continue 00240 if line[0] == '#': pass 00241 if not line: pass 00242 if not self.known_input_type(line): 00243 print ' unknown file:',line 00244 continue 00245 self.add_input_file(line) 00246 continue 00247 fp.close() 00248 return 00249 00250 00251 def configure_python_features(self): 00252 'Set up python features' 00253 if not os.getcwd() in sys.path: 00254 sys.path = [ os.getcwd() ] + sys.path 00255 00256 if not '' in sys.path: 00257 sys.path = [ '' ] + sys.path 00258 00259 sys.ps1 = 'nuwa> ' 00260 sys.ps2 = '. ... ' 00261 00262 00263 if self.opts.interactive: 00264 try: 00265 import rlcompleter, readline, atexit 00266 except ImportError: 00267 print 'No readline available.' 00268 else: 00269 readline.parse_and_bind('tab: complete') 00270 readline.parse_and_bind('set show-all-if-ambiguous On') 00271
| def DybPython::Control::NuWa::configure_ipython | ( | self | ) |
If ipython not available or are already inside ipython, setup a dummy
embedded ipython ipshell function, otherwise setup the real thing.
Definition at line 272 of file Control.py.
00273 : 00274 readline.read_history_file(fhistory) 00275 readline.set_history_length(1024) 00276 atexit.register(readline.write_history_file,fhistory) 00277 else: # batch 00278 if os.isatty(sys.stdin.fileno()): 00279 os.close(sys.stdin.fileno()) 00280 pass 00281 00282 00283 if self.opts.leak_check_execute: 00284 try: 00285 import Hephaestus.MemoryTracker as memtrack 00286 except ImportError: 00287 print "ERROR: Hephaestus is not available." 00288 sys.exit(1) 00289 else: 00290 methods = [] 00291 if "Algorithm::sysExecute" not in self.opts.leak_check_method: 00292 methods.append("Algorithm::sysExecute") 00293 methods += self.opts.leak_check_method 00294 for m in methods: 00295 print "Will leak check in",m 00296 memtrack.trace(m) 00297 memtrack.start() 00298 if self.opts.dbconf:
| def DybPython::Control::NuWa::configure_framework | ( | self | ) |
Definition at line 299 of file Control.py.
00299 : 00300 os.environ.update( DBCONF=self.opts.dbconf ) 00301 print "Defining DBCONF to %s " % os.environ.get('DBCONF') 00302 00303 # to do: 00304 # * set up python side logging 00305 00306 return 00307 00308 00309 def configure_ipython(self): 00310 """ 00311 If ipython not available or are already inside ipython, setup a dummy 00312 embedded ipython ipshell function, otherwise setup the real thing. 00313 """ 00314 if not self.opts.interactive: 00315 return 00316 00317 def ipshell(*args): pass 00318 self.ipshell = ipshell 00319 00320 try: 00321 import IPython 00322 except ImportError: 00323 print "No ipython available." 00324 return 00325 00326 try: 00327 __IPYTHON__ 00328 except NameError: 00329 from IPython.Shell import IPShellEmbed 00330 irgs = [''] 00331 banner = "entering ipython embedded shell, within the scope of the NuWa instance... try the locals() command " 00332 self.ipshell = IPShellEmbed(irgs, banner=banner, exit_msg="exiting ipython" ) 00333 00334 00335 00336 def configure_framework(self): 00337 'Set up framework level defaults' 00338 00339 # Fudge to make sure the right TimeStamp definition is loaded for MDC10b 00340 from GaudiPython import gbl 00341 t = gbl.TimeStamp() 00342 # end fudge 00343 00344 from Gaudi.Configuration import ApplicationMgr 00345 self.theApp = ApplicationMgr() 00346 self.theApp.OutputLevel = self.opts.log_level 00347 00348 # Setup history service 00349 if self.opts.history.lower() not in ["off","no","none","n","false"] \ 00350 and self.opts.executions != 0: 00351 from GaudiSvc.GaudiSvcConf import HistorySvc 00352 hs = HistorySvc() 00353 hs.OutputLevel = 1 00354 self.theApp.ExtSvc.append(hs) 00355 self.theApp.ActivateHistory = True 00356 if self.opts.history.lower() in ["on","yes","y","true"]: 00357 hs.Dump = True 00358 else: 00359 hs.OutputFile = self.opts.history 00360 00361 # Use our event data service which will in turn use the 00362 # standard EvtDataSvc as an Archive Event Service by default 00363 import DybEventMgr 00364 DybEventMgr.Configure(self.use_AES) # double negative 00365 00366 # Deal with I/O. 00367 try: 00368 import DybPython.Catalog as Catalog 00369 print 'Embedded Catalog is available' 00370 except ImportError: 00371 print 'No Embedded Catalog available' 00372 00373 # Input files can be determined in several ways 00374 self.input_files = [] 00375 00376 # First add files that are specified on the command line 00377 left_overs = [] 00378 for arg in self.args: 00379 if not self.known_input_type(arg): 00380 left_overs.append(arg) 00381 continue 00382 self.add_input_file(arg) 00383 continue 00384 self.args = left_overs 00385 del(left_overs) 00386 00387 # Next, WTF is this code supposed to do? The first half 00388 # requires a Python file to exist but then strips off the .py 00389 # and loads it as a module. This unnecessarily limiting use 00390 # to a .py file in the current directory and won't allow use 00391 # of modules that are found via sys.path. The second half 00392 # duplicates the functionality of just specifying input files 00393 # on the command line. 00394 if self.opts.file_list: 00395 if (os.path.exists(self.opts.file_list) and self.opts.file_list[-3:] == '.py'): 00396 # If python file, assume it is a List module 00397 listModule = self.opts.file_list[:-3] 00398 try: 00399 exec('import ' + listModule) 00400 except Exception,err: 00401 import traceback 00402 traceback.print_exc() 00403 print 'Failed to load', listModule 00404 sys.exit(1) 00405 print "Loaded \"%s\""%listModule 00406 self.add_input_file(eval(listModule + '.list')) 00407 00408 else: 00409 # Otherwise get list by evaluation the expression. 00410 fnames = eval(str(self.opts.file_list)) 00411 # if evaluation is not a real list try and force it to be one. 00412 if not isinstance(fnames, type([])): 00413 fnames = eval(str(fnames)) 00414 self.add_input_file(fnames) 00415 00416 00417 # Check that all input files are of the same type 00418 inputFiletype = None 00419 if self.input_files: 00420 firstFile = self.input_files[0] 00421 inputFiletype = firstFile[-5:] 00422 for filename in self.input_files[1:]: 00423 if filename[-5:] != inputFiletype: 00424 msg = "nuwa.py ERROR: Incorrect file type:",file 00425 raise TypeError,msg 00426 pass 00427 continue 00428 pass 00429 00430 # Should check that the files exist unless they are 00431 # xrootd files 00432 for fname in self.input_files: 00433 if fname[:7]=='root://' : 00434 if self.input_files.index( fname ) < 1 : 00435 print 'Cannot test for existence of xrootd file. Here is first file',fname 00436 else: 00437 if not os.path.exists(fname): 00438 msg = 'Warning: input file does not exist "%s"'%fname 00439 raise RuntimeError,msg 00440 continue 00441 00442 # For ROOT input, prepare streams 00443 if self.input_files and inputFiletype == '.root': 00444 try: 00445 default = self.opts.input_streams['default'] 00446 except KeyError: 00447 default = self.input_files 00448 self.opts.input_streams['default'] = default 00449 pass 00450 00451 if self.opts.input_streams and inputFiletype == '.root': 00452 # Enable reading of Run Data from ROOT input file 00453 from RunDataSvc.RunDataSvcConf import RunDataSvc 00454 runDataSvc = RunDataSvc() 00455 runDataSvc.ReadFromFile = True 00456 self.theApp.ExtSvc.append( runDataSvc ) 00457 00458 # Enable reading of Job Info from ROOT input file 00459 from JobInfoSvc.JobInfoSvcConf import JobInfoSvc 00460 jobInfoSvc = JobInfoSvc() 00461 jobInfoSvc.ReadFromFile = True 00462 self.theApp.ExtSvc.append( jobInfoSvc ) 00463 00464 # Merge default output files if needed 00465 if self.opts.output: 00466 self.opts.output_streams['default'] = self.opts.output 00467 00468 if self.opts.output_streams: 00469 # Add Run Data to ROOT output file 00470 from RunDataSvc.RunDataSvcConf import RunDataWriterAlg 00471 runDataWriterAlg = RunDataWriterAlg() 00472 runDataWriterAlg.WriteToFile = True 00473 00474 # Add Job Info to ROOT output file 00475 from JobInfoSvc.JobInfoSvcConf import JobInfoWriterAlg 00476 jobInfoWriterAlg = JobInfoWriterAlg() 00477 jobInfoWriterAlg.WriteToFile = True 00478 00479 import RootIOSvc 00480 self.io = RootIOSvc.Configure(self.opts.input_streams, 00481 self.opts.output_streams, 00482 self.opts.force_readout) 00483 00484 # Set up DaqFormat handling 00485 import DaqReadoutSvc 00486 daqSvc=DaqReadoutSvc.DaqReadoutSvc() 00487 if self.opts.daq == 'on': 00488 daqSvc.Generating=True 00489 if self.opts.daq == 'off': 00490 daqSvc.Generating=False 00491 daqSvc.RunNumber = self.opts.run 00492 00493 # Enable Raw File I/O if given raw data files. 00494 if inputFiletype == '.data' or inputFiletype == '.rraw': 00495 if (self.opts.daq == 'off' and self.opts.raw_load == 'off') or self.opts.raw_load == 'raw': 00496 import RawDataIO 00497 self.rawIo = RawDataIO.Configure() 00498 self.rawIo.InputSvc.RawDataFiles = self.input_files 00499 elif self.opts.daq == 'on' or self.opts.raw_load == 'daq': 00500 import DaqFormatIO 00501 self.daqIO = DaqFormatIO.eventSelector() 00502 self.daqIO.InputFiles = self.input_files 00503 self.daqIO.SamplingInterval = 1 00504 # For ActionOnInvalid: 0 = exit, 1 = skip, 2 = process 00505 self.daqIO.ActionOnInvalid = 1 00506 elif self.opts.raw_load.startswith('daq-'): 00507 from DaqFormatModules.DaqFormatModulesConf import DybDaq__DaqFormatInput 00508 self.rawLoad = DybDaq__DaqFormatInput() 00509 self.theApp.TopAlg += [self.rawLoad] 00510 self.rawLoad.InputFiles = self.input_files 00511 if self.opts.raw_load =='daq-repackage': 00512 self.rawLoad.Repackage = True 00513 else: 00514 self.rawLoad.Repackage = False 00515 if self.opts.raw_load in ['daq-both','daq-packed']: 00516 self.rawLoad.UsePacked = True 00517 else: 00518 self.rawLoad.UsePacked = False 00519 if self.opts.raw_load in ['daq-both','daq-unpacked']: 00520 self.rawLoad.UseUnpacked = True 00521 else: 00522 self.rawLoad.UseUnpacked = False 00523 else: from DataDebug.DataDebugConf import DataLoaderAlg
| def DybPython::Control::NuWa::configure_random | ( | self | ) |
Definition at line 524 of file Control.py.
00528 : 00529 self.rawLoad.UsePacked = True 00530 else: 00531 self.rawLoad.UsePacked = False 00532 if self.opts.raw_load in ['both','unpacked']: self.rawLoad.UseUnpacked = True
| def DybPython::Control::NuWa::configure_run | ( | self | ) |
Definition at line 533 of file Control.py.
00534 : 00535 self.rawLoad.UseUnpacked = False 00536 00537 # Configure Statistics Service for histogram I/O 00538 if self.opts.output_stats or self.opts.input_stats: 00539 print 'output_stats=',self.opts.output_stats 00540 print 'input_stats=',self.opts.input_stats 00541 from StatisticsSvc.StatisticsSvcConf import StatisticsSvc 00542 statsSvc = StatisticsSvc() if self.opts.output_stats:
| def DybPython::Control::NuWa::configure_job | ( | self | ) |
Definition at line 543 of file Control.py.
00543 : 00544 statsSvc.Output = self.opts.output_stats 00545 if self.opts.input_stats: 00546 statsSvc.Input = self.opts.input_stats 00547 self.theApp.ExtSvc.append( statsSvc ) 00548 pass 00549 00550 ninput = len(self.input_files) 00551 for k,v in self.opts.input_streams.iteritems(): 00552 ninput += len(v) 00553 if self.opts.executions == -1 and ninput == 0: 00554 err = 'Unbounded running with no input files, probably not what you wanted.\n' 00555 sys.stderr.write(err) 00556 sys.exit(1) 00557 pass 00558 00559 return 00560 00561 def configure_random(self): 00562 00563 if self.opts.random == 'on': 00564 from DybAlg import RandomSeeder 00565 rs = RandomSeeder(hostid = self.opts.hostid, 00566 run = self.opts.run, 00567 execcountoffset = self.opts.execution-1); 00568 return 00569 00570 def configure_run(self): 00571 # Catch command-line run number and wall time for simulation jobs 00572 # Send run number and wall time to RunDataSvc 00573 from RunDataSvc.RunDataSvcConf import RunDataSvc 00574 runDataSvc = RunDataSvc() runDataSvc.SimRunNumber = self.opts.run
| def DybPython::Control::NuWa::configure_geometry | ( | self | ) |
Definition at line 575 of file Control.py.
00580 : 00581 # Catch job configuration info and feed to Job Info Service 00582 from JobInfoSvc.JobInfoSvcConf import JobInfoSvc 00583 jobInfoSvc = JobInfoSvc() 00584 # Initialize current job info 00585 import sys, time, os nuwaPath = os.path.realpath(os.path.expandvars('$DYBRELEASEROOT/../../'))
| def DybPython::Control::NuWa::configure_optmods | ( | self | ) |
load and configure() "-m" modules here
Definition at line 586 of file Control.py.
00590 : 00591 revisionFile = open(revPath) 00592 if revisionFile: 00593 revLine = revisionFile.readline() 00594 if revLine != "": 00595 revision = revLine.strip() 00596 jobConfigList = {'command':' '.join(sys.argv), 00597 'revision':revision, 00598 'nuwaPath':nuwaPath, 00599 'cmtConfig':os.getenv('CMTCONFIG'), 00600 'hostid':str(self.opts.hostid), 00601 'jobTime':time.strftime("%a, %d %b %Y %H:%M:%S +0000", time.gmtime()), 00602 'username':os.getenv('LOGNAME')} 00603 jobInfoSvc.JobId = self.opts.jobid 00604 # Process user-defined job information 00605 for key in self.opts.job_info.keys(): 00606 print "User-defined job info: "+key+" = "+self.opts.job_info[key] 00607 jobConfigList[key] = self.opts.job_info[key] 00608 jobInfoSvc.JobConfig = jobConfigList 00609 self.theApp.ExtSvc.append( jobInfoSvc ) 00610 return 00611 00612 def configure_geometry(self): 00613 import XmlDetDesc 00614 if self.opts.detector: 00615 XmlDetDesc.Configure(self.opts.detector) 00616 print "Note: using non-default geometry in " \ 00617 + self.opts.detector + " is loaded." 00618 else: 00619 XmlDetDesc.Configure() 00620 return
| def DybPython::Control::NuWa::configure_args | ( | self | ) |
spin over all non-option arguments
Definition at line 621 of file Control.py.
00623 : 00624 """ load and configure() "-m" modules here """ 00625 if self.opts.module == None: return 00626 00627 from DybPython.cmdline import quotedParse 00628 00629 for modcmd in self.opts.module: 00630 argv = quotedParse(modcmd) 00631 modname = argv[0] 00632 if modname[-3:] == '.py': 00633 print 'Fixing assumed typo in module name from "%s" to "%s"'%(modname,modname[:-3]) 00634 modname = modname[:-3] 00635 modargs = argv[1:] 00636 print "Importing modules",modname,"["," ".join(modargs),"]" 00637 exec('import %s'%modname) # Don't catch potential exception! 00638 mod = eval(modname) 00639 self.modules.append(mod) 00640 00641 print "Trying to call configure() on",modname 00642 try: 00643 if modargs: 00644 mod.configure(argv=modargs) 00645 else: 00646 mod.configure() 00647 00648 except SyntaxError,err: 00649 print "Got SyntaxError:",str(err) 00650 print "Assuming",cfg,"has not configure() function" 00651 pass 00652 except AttributeError,err: 00653 if "object has no attribute 'configure" not in str(err): 00654 raise 00655 pass 00656 00657 def configure_args(self):
| def DybPython::Control::NuWa::configure_user | ( | self | ) |
Definition at line 658 of file Control.py.
00658 : 00659 """ spin over all non-option arguments """ 00660 00661 # At this point there should be no input files left in the args. 00662 for cfg in self.args: 00663 00664 if cfg[-3:] == '.py': 00665 cfgf = open(cfg) 00666 try: 00667 exec(cfgf) 00668 except Exception,err: 00669 import traceback 00670 traceback.print_exc() 00671 print 'Failed to load',cfg 00672 sys.exit(1) print "Loaded \"%s\""%cfg
| def DybPython::Control::NuWa::configure_post_user | ( | self | ) |
Definition at line 673 of file Control.py.
00676 : # assume this is a module 00677 print "Importing module",cfg 00678 exec('import %s'%cfg) # Don't catch potential exception! 00679 00680 mod = eval(cfg) 00681 self.modules.append(mod) 00682 00683 # if it has a configure() method, call it now 00684 print "Trying to call configure() on",cfg 00685 try: 00686 mod.configure() 00687 except AttributeError,err: 00688 if "object has no attribute 'configure'" not in str(err): 00689 raise 00690 pass 00691 pass 00692 continue 00693 return 00694 00695 def configure_user(self): 00696 from DybPython.Include import include, IncludeError 00697 include.setShowIncludes(self.opts.show_includes) 00698 00699 try: 00700 include("$HOME/.nuwarc") 00701 print "Included ~/.nuwarc" 00702 except IncludeError,err: 00703 pass 00704 00705 self.modules = [] 00706 self.configure_optmods() 00707 self.configure_args() 00708 return 00709 00710 def configure_post_user(self):
| def DybPython::Control::NuWa::configure_visualization | ( | self | ) |
Definition at line 711 of file Control.py.
00711 : 00712 if self.opts.output_streams: 00713 print "Using DybStoreAlg to trigger output." 00714 from RootIOTest.RootIOTestConf import DybStoreAlg 00715 dsa = DybStoreAlg() 00716 self.post_user_algs.append(dsa) 00717 else: 00718 if self.opts.executions != 0: 00719 print "Using DybTrimIO to trigger output and do AES trimming" 00720 # add DybTrimIO 00721 from DybAlg.DybAlgConf import DybTrimIO 00722 trimIO = DybTrimIO() 00723 self.post_user_algs.append(trimIO) 00724 # add Trimming service 00725 from DybEventMgr.DybEventMgrConf import ArchiveTrimSvc 00726 trimSvc = ArchiveTrimSvc() 00727 self.theApp.ExtSvc.append(trimSvc) 00728 trimSvc.DefaultWindowSeconds = self.AES_window/units.second 00729 00730 # check whether io is involed 00731 if self.opts.output_streams: 00732 # add SaveB4TrimAesSvc 00733 from DybEventMgr.DybEventMgrConf import SaveB4TrimAesSvc 00734 svb4t = SaveB4TrimAesSvc() 00735 self.theApp.ExtSvc.append(svb4t) 00736 00737 trimIO.SaveFlag=True; else:
| def DybPython::Control::NuWa::configure | ( | self | ) |
Definition at line 738 of file Control.py.
00738 : 00739 trimIO.SaveFlag=False; 00740 # Add Run Data Writer to the end of the list of algorithms 00741 from RunDataSvc.RunDataSvcConf import RunDataWriterAlg 00742 self.post_user_algs.append(RunDataWriterAlg()) 00743 from JobInfoSvc.JobInfoSvcConf import JobInfoWriterAlg 00744 self.post_user_algs.append(JobInfoWriterAlg()) 00745 return 00746 00747 00748 def configure_visualization(self): 00749 'Configure for "quanjing/panoramix" visualization' 00750 gui = os.getenv("QUANJINGROOT") if not gui:
| def DybPython::Control::NuWa::spade_files | ( | self | ) |
Definition at line 751 of file Control.py.
00751 : 00752 print "Warning: no QUANJINGROOT variable, no default gui, failure likely." 00753 return 00754 gui += "/xml/gui.onx"
| def DybPython::Control::NuWa::run_post_user | ( | self, | ||
| app | ||||
| ) |
Definition at line 755 of file Control.py.
00757 : 00758 from VisSvc.VisSvcConf import VisualizationSvc 00759 vis = VisualizationSvc() 00760 vis.ColorDbLocation = os.path.dirname(gui) + '/gui/vis/colors.xml'
| def DybPython::Control::NuWa::run | ( | self | ) |
Definition at line 761 of file Control.py.
00771 {'Runable':'OnXSvc'} ) 00772 appMgr.config() 00773 return 00774 00775 def configure(self): 00776 self.configure_python_features() 00777 self.configure_ipython() 00778 self.configure_framework() 00779 self.configure_random() 00780 self.configure_run() 00781 self.configure_job() 00782 self.configure_geometry() 00783 self.configure_user() 00784 self.configure_post_user() 00785 return 00786 00787 00788 def spade_files(self): 00789 if not self.spadeSvc: return [] 00790 return self.spadeSvc.marked 00791 00792 def run_post_user(self, app): 00793 # Add framework algorithms after user algorithms 00794 for alg in self.post_user_algs: 00795 app.addAlgorithm(alg.getFullJobOptName()) 00796 return 00797 00798 def run(self): 00799 00800 if self.opts.visualize: 00801 self.configure_visualization() 00802 00803 from GaudiPython import AppMgr 00804 appMgr = AppMgr() 00805 00806 if self.opts.color == "dark": 00807 #print 'setting up for dark color terminals' 00808 msg = appMgr.property('MessageSvc') 00809 00810 msg.OutputLevel = self.opts.log_level 00811 msg.useColors=True 00812 msg.fatalColorCode=['red','white'] 00813 msg.errorColorCode=['red'] 00814 msg.warningColorCode=['yellow'] 00815 msg.debugColorCode=['blue'] 00816 msg.verboseColorCode=['cyan'] 00817 pass 00818 if self.opts.color == "light": 00819 #print 'setting up for light color terminals' 00820 msg = appMgr.property('MessageSvc') 00821 msg.OutputLevel = self.opts.log_level 00822 msg.useColors=True 00823 msg.fatalColorCode=['red','white'] 00824 msg.errorColorCode=['red'] 00825 msg.warningColorCode=['yellow'] 00826 msg.debugColorCode=['blue'] 00827 msg.verboseColorCode=['cyan'] 00828 pass 00829 00830 # Muck with the msg formatting to give more room to show component names 00831 msg = appMgr.property('MessageSvc') 00832 msg.Format="%% F%%%dW%%S%%7W%%R%%T %%0W%%M"%35 00833 00834 msgSvc = appMgr.service('MessageSvc','IMessageSvc') 00835 if self.opts.msg_stream_level: 00836 for namelevel in self.opts.msg_stream_level: 00837 name,level = namelevel.split(',') 00838 msgSvc.setOutputLevel(name,int(level)) 00839 continue 00840 pass 00841 00842 # Run any modules with a run() function
| def DybPython::Control::NuWa::finalize | ( | self | ) |
Definition at line 843 of file Control.py.
00843 : 00844 try: 00845 print "Trying to call run() on",mod.__name__ 00846 mod.run(appMgr) 00847 except AttributeError,err: 00848 if "object has no attribute 'run'" not in str(err): 00849 raise 00850 pass 00851 00852 self.run_post_user(appMgr) 00853 00854 n = self.opts.executions 00855 appMgr.EvtMax = n 00856 #appMgr.initialize() 00857 00858 print 'Starting run with these top algorithms:',appMgr.TopAlg
Definition at line 113 of file Control.py.
Definition at line 114 of file Control.py.
Definition at line 115 of file Control.py.
Definition at line 118 of file Control.py.
Definition at line 125 of file Control.py.
Definition at line 126 of file Control.py.
Definition at line 151 of file Control.py.
Definition at line 154 of file Control.py.
Definition at line 308 of file Control.py.
Definition at line 337 of file Control.py.
Definition at line 668 of file Control.py.
1.4.7