Started working on galaxy IC4408_GROUP at Sat Feb  8 08:20:42 2020
RA, Dec=215.304539, 29.993514: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:20:43.858157
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.30485368345146', '29.987209711224114', '--width', '773', '--height', '773', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.30485368345146 29.987209711224114
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 38710 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38711 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38712 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 38713 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38714 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 38715 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 38716 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 38717 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:20:46.960072
Keeping 17 CCDs:
   c4d_150408_055941_ooi_z_ls9.fits.fz c4d_150408_055941_ooi_z_ls9-S14 z exptime 109.0 propid 2014B-0404 seeing 1.17 MJD 57120.248 object DECaLS_22714_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S19 r exptime 72.0 propid 2014B-0404 seeing 1.25 MJD 57865.209 object DECaLS_22695_r
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.77 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S19 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S14 g exptime 148.0 propid 2014B-0404 seeing 1.45 MJD 58255.108 object DECaLS_22714_g
   c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S14 r exptime 95.0 propid 2014B-0404 seeing 1.16 MJD 57864.240 object DECaLS_22714_r
   c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S14 g exptime 175.0 propid 2014B-0404 seeing 1.48 MJD 57893.143 object DECaLS_22714_g
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S18 g exptime 105.0 propid 2014B-0404 seeing 1.13 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S18 z exptime 87.0 propid 2014B-0404 seeing 1.35 MJD 58547.300 object DECaLS_38567_z
   c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S8 g exptime 113.0 propid 2014B-0404 seeing 1.63 MJD 57952.982 object DECaLS_6842_g
   c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S8 z exptime 109.0 propid 2014B-0404 seeing 1.04 MJD 58158.384 object DECaLS_6842_z
   c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S8 z exptime 88.0 propid 2014B-0404 seeing 1.28 MJD 57123.240 object DECaLS_6842_z
   c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S8 r exptime 63.0 propid 2014B-0404 seeing 1.18 MJD 57841.291 object DECaLS_6842_r
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S18 z exptime 80.0 propid 2014B-0404 seeing 1.11 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S18 r exptime 47.0 propid 2014B-0404 seeing 0.90 MJD 58171.364 object DECaLS_38567_r
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S13 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S13 r exptime 72.0 propid 2014B-0404 seeing 1.22 MJD 57865.209 object DECaLS_22695_r
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy IC4408_GROUP; please check the logfile.
Finished galaxy IC4408_GROUP in 0.133 minutes.
Started working on galaxy IC4408_GROUP at Sat Feb  8 08:43:24 2020
RA, Dec=215.304539, 29.993514: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:43:25.237374
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.30485368345146', '29.987209711224114', '--width', '773', '--height', '773', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.30485368345146 29.987209711224114
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 43449 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 43450 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 43451 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 43452 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 43453 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 43454 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 43455 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 43456 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:43:27.278529
Keeping 17 CCDs:
   c4d_150408_055941_ooi_z_ls9.fits.fz c4d_150408_055941_ooi_z_ls9-S14 z exptime 109.0 propid 2014B-0404 seeing 1.17 MJD 57120.248 object DECaLS_22714_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S19 r exptime 72.0 propid 2014B-0404 seeing 1.25 MJD 57865.209 object DECaLS_22695_r
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.77 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S19 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S14 g exptime 148.0 propid 2014B-0404 seeing 1.45 MJD 58255.108 object DECaLS_22714_g
   c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S14 r exptime 95.0 propid 2014B-0404 seeing 1.16 MJD 57864.240 object DECaLS_22714_r
   c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S14 g exptime 175.0 propid 2014B-0404 seeing 1.48 MJD 57893.143 object DECaLS_22714_g
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S18 g exptime 105.0 propid 2014B-0404 seeing 1.13 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S18 z exptime 87.0 propid 2014B-0404 seeing 1.35 MJD 58547.300 object DECaLS_38567_z
   c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S8 g exptime 113.0 propid 2014B-0404 seeing 1.63 MJD 57952.982 object DECaLS_6842_g
   c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S8 z exptime 109.0 propid 2014B-0404 seeing 1.04 MJD 58158.384 object DECaLS_6842_z
   c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S8 z exptime 88.0 propid 2014B-0404 seeing 1.28 MJD 57123.240 object DECaLS_6842_z
   c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S8 r exptime 63.0 propid 2014B-0404 seeing 1.18 MJD 57841.291 object DECaLS_6842_r
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S18 z exptime 80.0 propid 2014B-0404 seeing 1.11 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S18 r exptime 47.0 propid 2014B-0404 seeing 0.90 MJD 58171.364 object DECaLS_38567_r
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S13 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S13 r exptime 72.0 propid 2014B-0404 seeing 1.22 MJD 57865.209 object DECaLS_22695_r
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180516/c4d_180517_
023606_ooi_g_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy IC4408_GROUP; please check the logfile.
Finished galaxy IC4408_GROUP in 0.106 minutes.
Started working on galaxy IC4408_GROUP at Sat Feb  8 12:47:34 2020
RA, Dec=215.304539, 29.993514: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T12:47:35.147303
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.30485368345146', '29.987209711224114', '--width', '773', '--height', '773', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.30485368345146 29.987209711224114
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 54852 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 54853 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 54854 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 54855 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 54856 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 54857 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 54858 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 54859 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T12:47:43.716496
Keeping 17 CCDs:
   c4d_150408_055941_ooi_z_ls9.fits.fz c4d_150408_055941_ooi_z_ls9-S14 z exptime 109.0 propid 2014B-0404 seeing 1.17 MJD 57120.248 object DECaLS_22714_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S19 r exptime 72.0 propid 2014B-0404 seeing 1.25 MJD 57865.209 object DECaLS_22695_r
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.77 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S19 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S14 g exptime 148.0 propid 2014B-0404 seeing 1.45 MJD 58255.108 object DECaLS_22714_g
   c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S14 r exptime 95.0 propid 2014B-0404 seeing 1.16 MJD 57864.240 object DECaLS_22714_r
   c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S14 g exptime 175.0 propid 2014B-0404 seeing 1.48 MJD 57893.143 object DECaLS_22714_g
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S18 g exptime 105.0 propid 2014B-0404 seeing 1.13 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S18 z exptime 87.0 propid 2014B-0404 seeing 1.35 MJD 58547.300 object DECaLS_38567_z
   c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S8 g exptime 113.0 propid 2014B-0404 seeing 1.63 MJD 57952.982 object DECaLS_6842_g
   c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S8 z exptime 109.0 propid 2014B-0404 seeing 1.04 MJD 58158.384 object DECaLS_6842_z
   c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S8 z exptime 88.0 propid 2014B-0404 seeing 1.28 MJD 57123.240 object DECaLS_6842_z
   c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S8 r exptime 63.0 propid 2014B-0404 seeing 1.18 MJD 57841.291 object DECaLS_6842_r
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S18 z exptime 80.0 propid 2014B-0404 seeing 1.11 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S18 r exptime 47.0 propid 2014B-0404 seeing 0.90 MJD 58171.364 object DECaLS_38567_r
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S13 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S13 r exptime 72.0 propid 2014B-0404 seeing 1.22 MJD 57865.209 object DECaLS_22695_r
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy IC4408_GROUP; please check the logfile.
Finished galaxy IC4408_GROUP in 0.210 minutes.
Started working on galaxy IC4408_GROUP at Sun Feb  9 09:30:20 2020
RA, Dec=215.304539, 29.993514: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T09:30:21.135270
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.30485368345146', '29.987209711224114', '--width', '773', '--height', '773', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.30485368345146 29.987209711224114
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 32704 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071168 MB
Starting process 32705 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071168 MB
Starting process 32706 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071168 MB
Starting process 32707 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071168 MB
Starting process 32708 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071172 MB
Starting process 32709 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071176 MB
Starting process 32710 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071176 MB
Starting process 32711 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07118 MB
Starting process 32712 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07118 MB
Starting process 32713 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07118 MB
Starting process 32714 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071184 MB
Starting process 32715 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071188 MB
Starting process 32716 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071188 MB
Starting process 32717 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071188 MB
Starting process 32718 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071188 MB
Starting process 32719 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071188 MB
Starting process 32720 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 141 MB, maxrss: 0.071192 MB
Starting process 32721 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 141 MB, maxrss: 0.072204 MB
Starting process 32722 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 141 MB, maxrss: 0.072204 MB
Starting process 32723 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071912 MB
Starting process 32724 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071908 MB
Starting process 32725 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071544 MB
Starting process 32726 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32727 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32728 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32729 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32730 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32731 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32732 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32733 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 32734 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071484 MB
Starting process 32735 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071484 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T09:30:22.430666
Keeping 17 CCDs:
   c4d_150408_055941_ooi_z_ls9.fits.fz c4d_150408_055941_ooi_z_ls9-S14 z exptime 109.0 propid 2014B-0404 seeing 1.17 MJD 57120.248 object DECaLS_22714_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S19 r exptime 72.0 propid 2014B-0404 seeing 1.25 MJD 57865.209 object DECaLS_22695_r
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.77 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S19 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S14 g exptime 148.0 propid 2014B-0404 seeing 1.45 MJD 58255.108 object DECaLS_22714_g
   c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S14 r exptime 95.0 propid 2014B-0404 seeing 1.16 MJD 57864.240 object DECaLS_22714_r
   c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S14 g exptime 175.0 propid 2014B-0404 seeing 1.48 MJD 57893.143 object DECaLS_22714_g
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S18 g exptime 105.0 propid 2014B-0404 seeing 1.13 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S18 z exptime 87.0 propid 2014B-0404 seeing 1.35 MJD 58547.300 object DECaLS_38567_z
   c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S8 g exptime 113.0 propid 2014B-0404 seeing 1.63 MJD 57952.982 object DECaLS_6842_g
   c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S8 z exptime 109.0 propid 2014B-0404 seeing 1.04 MJD 58158.384 object DECaLS_6842_z
   c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S8 z exptime 88.0 propid 2014B-0404 seeing 1.28 MJD 57123.240 object DECaLS_6842_z
   c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S8 r exptime 63.0 propid 2014B-0404 seeing 1.18 MJD 57841.291 object DECaLS_6842_r
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S18 z exptime 80.0 propid 2014B-0404 seeing 1.11 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S18 r exptime 47.0 propid 2014B-0404 seeing 0.90 MJD 58171.364 object DECaLS_38567_r
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S13 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S13 r exptime 72.0 propid 2014B-0404 seeing 1.22 MJD 57865.209 object DECaLS_22695_r
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy IC4408_GROUP; please check the logfile.
Finished galaxy IC4408_GROUP in 0.099 minutes.
Started working on galaxy IC4408_GROUP at Sun Feb  9 10:23:16 2020
RA, Dec=215.304539, 29.993514: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T10:23:17.439626
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.30485368345146', '29.987209711224114', '--width', '773', '--height', '773', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.30485368345146 29.987209711224114
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 43860 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07116 MB
Starting process 43861 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 43862 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 43863 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 43864 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 43865 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 43866 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 43867 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 43868 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 43869 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 43870 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 43871 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 43872 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 43873 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 43874 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 43875 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 43876 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 43877 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB
Starting process 43878 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB
Starting process 43879 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071908 MB
Starting process 43880 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071904 MB
Starting process 43881 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 43882 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071476 MB
Starting process 43883 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 43884 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071476 MB
Starting process 43885 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 43886 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071476 MB
Starting process 43887 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 43888 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 43889 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 43890 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 43891 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T10:23:19.511511
Keeping 17 CCDs:
   c4d_150408_055941_ooi_z_ls9.fits.fz c4d_150408_055941_ooi_z_ls9-S14 z exptime 109.0 propid 2014B-0404 seeing 1.17 MJD 57120.248 object DECaLS_22714_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S19 r exptime 72.0 propid 2014B-0404 seeing 1.25 MJD 57865.209 object DECaLS_22695_r
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.77 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S19 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S14 g exptime 148.0 propid 2014B-0404 seeing 1.45 MJD 58255.108 object DECaLS_22714_g
   c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S14 r exptime 95.0 propid 2014B-0404 seeing 1.16 MJD 57864.240 object DECaLS_22714_r
   c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S14 g exptime 175.0 propid 2014B-0404 seeing 1.48 MJD 57893.143 object DECaLS_22714_g
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S18 g exptime 105.0 propid 2014B-0404 seeing 1.13 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S18 z exptime 87.0 propid 2014B-0404 seeing 1.35 MJD 58547.300 object DECaLS_38567_z
   c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S8 g exptime 113.0 propid 2014B-0404 seeing 1.63 MJD 57952.982 object DECaLS_6842_g
   c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S8 z exptime 109.0 propid 2014B-0404 seeing 1.04 MJD 58158.384 object DECaLS_6842_z
   c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S8 z exptime 88.0 propid 2014B-0404 seeing 1.28 MJD 57123.240 object DECaLS_6842_z
   c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S8 r exptime 63.0 propid 2014B-0404 seeing 1.18 MJD 57841.291 object DECaLS_6842_r
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S18 z exptime 80.0 propid 2014B-0404 seeing 1.11 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S18 r exptime 47.0 propid 2014B-0404 seeing 0.90 MJD 58171.364 object DECaLS_38567_r
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S13 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S13 r exptime 72.0 propid 2014B-0404 seeing 1.22 MJD 57865.209 object DECaLS_22695_r
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy IC4408_GROUP; please check the logfile.
Finished galaxy IC4408_GROUP in 0.081 minutes.
Started working on galaxy IC4408_GROUP at Mon Feb 10 06:17:06 2020
RA, Dec=215.304539, 29.993514: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-10T06:17:07.141649
legacypipe git version: dr9e-pre1-9-g4a51be93
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.30485368345146', '29.987209711224114', '--width', '773', '--height', '773', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.30485368345146 29.987209711224114 --width 773 --height 773 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/IC4408_GROUP/IC4408_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.30485368345146 29.987209711224114
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 15033 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15034 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15035 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15036 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15037 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 15038 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 15039 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 15040 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15041 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15042 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15043 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 15044 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15045 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15046 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15047 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15048 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15049 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 15050 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072156 MB
Starting process 15051 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072156 MB
Starting process 15052 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071868 MB
Starting process 15053 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071864 MB
Starting process 15054 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 15055 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15056 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0715 MB
Starting process 15057 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0715 MB
Starting process 15058 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071504 MB
Starting process 15059 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15060 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15061 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15062 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15063 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 15064 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-10T06:17:08.430049
Keeping 17 CCDs:
   c4d_150408_055941_ooi_z_ls9.fits.fz c4d_150408_055941_ooi_z_ls9-S14 z exptime 109.0 propid 2014B-0404 seeing 1.17 MJD 57120.248 object DECaLS_22714_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S19 r exptime 72.0 propid 2014B-0404 seeing 1.25 MJD 57865.209 object DECaLS_22695_r
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.77 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S19 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S14 g exptime 148.0 propid 2014B-0404 seeing 1.45 MJD 58255.108 object DECaLS_22714_g
   c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S14 r exptime 95.0 propid 2014B-0404 seeing 1.16 MJD 57864.240 object DECaLS_22714_r
   c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S14 g exptime 175.0 propid 2014B-0404 seeing 1.48 MJD 57893.143 object DECaLS_22714_g
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S18 g exptime 105.0 propid 2014B-0404 seeing 1.13 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S18 z exptime 87.0 propid 2014B-0404 seeing 1.35 MJD 58547.300 object DECaLS_38567_z
   c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S8 g exptime 113.0 propid 2014B-0404 seeing 1.63 MJD 57952.982 object DECaLS_6842_g
   c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S8 z exptime 109.0 propid 2014B-0404 seeing 1.04 MJD 58158.384 object DECaLS_6842_z
   c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S8 z exptime 88.0 propid 2014B-0404 seeing 1.28 MJD 57123.240 object DECaLS_6842_z
   c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S8 r exptime 63.0 propid 2014B-0404 seeing 1.18 MJD 57841.291 object DECaLS_6842_r
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S18 z exptime 80.0 propid 2014B-0404 seeing 1.11 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S18 r exptime 47.0 propid 2014B-0404 seeing 0.90 MJD 58171.364 object DECaLS_38567_r
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S13 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S13 r exptime 72.0 propid 2014B-0404 seeing 1.22 MJD 57865.209 object DECaLS_22695_r
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S18,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S13,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055701_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054506_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T091225_oi[S8,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy IC4408_GROUP; please check the logfile.
Finished galaxy IC4408_GROUP in 0.060 minutes.