Started working on galaxy PGC1934503 at Sat Feb  8 08:36:00 2020
RA, Dec=220.029172, 31.163108: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:36:01.382671
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '220.0291725', '31.1631078', '--width', '435', '--height', '435', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 220.0291725 31.1631078
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 36010 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 36011 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 36012 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 36013 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 36014 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 36015 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 36016 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 36017 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:36:02.707193
Keeping 15 CCDs:
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.29 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.42 MJD 57843.287 object DECaLS_22744_g
   c4d_160229_082453_ooi_z_ls9.fits.fz c4d_160229_082453_ooi_z_ls9-S1 z exptime 80.0 propid 2014B-0404 seeing 1.03 MJD 57447.349 object DECaLS_250_z
   c4d_170329_073043_ooi_r_ls9.fits.fz c4d_170329_073043_ooi_r_ls9-S1 r exptime 63.0 propid 2014B-0404 seeing 1.32 MJD 57841.312 object DECaLS_250_r
   c4d_170422_054608_ooi_g_ls9.fits.fz c4d_170422_054608_ooi_g_ls9-S1 g exptime 122.0 propid 2014B-0404 seeing 1.25 MJD 57865.238 object DECaLS_250_g
   c4d_180222_090307_ooi_r_ls9.fits.fz c4d_180222_090307_ooi_r_ls9-S5 r exptime 51.0 propid 2014B-0404 seeing 0.93 MJD 58171.377 object DECaLS_38616_r
   c4d_180222_090427_ooi_g_ls9.fits.fz c4d_180222_090427_ooi_g_ls9-S5 g exptime 121.0 propid 2014B-0404 seeing 1.15 MJD 58171.378 object DECaLS_38616_g
   c4d_160226_091110_ooi_z_ls9.fits.fz c4d_160226_091110_ooi_z_ls9-S5 z exptime 155.0 propid 2014B-0404 seeing 0.96 MJD 57444.380 object DECaLS_38616_z
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S6 z exptime 142.0 propid 2014B-0404 seeing 1.10 MJD 57444.347 object DECaLS_22744_z
   c4d_170329_070606_ooi_r_ls9.fits.fz c4d_170329_070606_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.32 MJD 57841.295 object DECaLS_6872_r
   c4d_160228_083710_ooi_z_ls9.fits.fz c4d_160228_083710_ooi_z_ls9-S6 z exptime 141.0 propid 2014B-0404 seeing 0.92 MJD 57446.357 object DECaLS_6872_z
   c4d_170422_050829_ooi_g_ls9.fits.fz c4d_170422_050829_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57865.212 object DECaLS_6872_g
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S5 r exptime 62.0 propid 2014B-0404 seeing 1.31 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S5 g exptime 175.0 propid 2014B-0404 seeing 1.43 MJD 57843.287 object DECaLS_22744_g
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S5 z exptime 142.0 propid 2014B-0404 seeing 1.14 MJD 57444.347 object DECaLS_22744_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T090745_oi[S5,noinherit][2046,4094][real]: DECaLS_3
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T083400_oi[S6,noinherit][2046,4094][real]: DECaLS_6
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T090745_oi[S5,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170330/c4d_170331_0
65648_ooi_g_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1934503; please check the logfile.
Finished galaxy PGC1934503 in 0.093 minutes.
Started working on galaxy PGC1934503 at Sat Feb  8 12:45:34 2020
RA, Dec=220.029172, 31.163108: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T12:45:35.369078
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '220.0291725', '31.1631078', '--width', '435', '--height', '435', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 220.0291725 31.1631078
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 16614 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 16615 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 16616 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 16617 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 16618 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 16619 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 16620 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 16621 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T12:45:36.646859
Keeping 15 CCDs:
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.29 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.42 MJD 57843.287 object DECaLS_22744_g
   c4d_160229_082453_ooi_z_ls9.fits.fz c4d_160229_082453_ooi_z_ls9-S1 z exptime 80.0 propid 2014B-0404 seeing 1.03 MJD 57447.349 object DECaLS_250_z
   c4d_170329_073043_ooi_r_ls9.fits.fz c4d_170329_073043_ooi_r_ls9-S1 r exptime 63.0 propid 2014B-0404 seeing 1.32 MJD 57841.312 object DECaLS_250_r
   c4d_170422_054608_ooi_g_ls9.fits.fz c4d_170422_054608_ooi_g_ls9-S1 g exptime 122.0 propid 2014B-0404 seeing 1.25 MJD 57865.238 object DECaLS_250_g
   c4d_180222_090307_ooi_r_ls9.fits.fz c4d_180222_090307_ooi_r_ls9-S5 r exptime 51.0 propid 2014B-0404 seeing 0.93 MJD 58171.377 object DECaLS_38616_r
   c4d_180222_090427_ooi_g_ls9.fits.fz c4d_180222_090427_ooi_g_ls9-S5 g exptime 121.0 propid 2014B-0404 seeing 1.15 MJD 58171.378 object DECaLS_38616_g
   c4d_160226_091110_ooi_z_ls9.fits.fz c4d_160226_091110_ooi_z_ls9-S5 z exptime 155.0 propid 2014B-0404 seeing 0.96 MJD 57444.380 object DECaLS_38616_z
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S6 z exptime 142.0 propid 2014B-0404 seeing 1.10 MJD 57444.347 object DECaLS_22744_z
   c4d_170329_070606_ooi_r_ls9.fits.fz c4d_170329_070606_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.32 MJD 57841.295 object DECaLS_6872_r
   c4d_160228_083710_ooi_z_ls9.fits.fz c4d_160228_083710_ooi_z_ls9-S6 z exptime 141.0 propid 2014B-0404 seeing 0.92 MJD 57446.357 object DECaLS_6872_z
   c4d_170422_050829_ooi_g_ls9.fits.fz c4d_170422_050829_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57865.212 object DECaLS_6872_g
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S5 r exptime 62.0 propid 2014B-0404 seeing 1.31 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S5 g exptime 175.0 propid 2014B-0404 seeing 1.43 MJD 57843.287 object DECaLS_22744_g
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S5 z exptime 142.0 propid 2014B-0404 seeing 1.14 MJD 57444.347 object DECaLS_22744_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T083400_oi[S6,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T090745_oi[S5,noinherit][2046,4094][real]: DECaLS_3
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T083400_oi[S6,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170330/c4d_170331_0
65648_ooi_g_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1934503; please check the logfile.
Finished galaxy PGC1934503 in 0.079 minutes.
Started working on galaxy PGC1934503 at Sun Feb  9 09:57:24 2020
RA, Dec=220.029172, 31.163108: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T09:57:24.773900
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '220.0291725', '31.1631078', '--width', '435', '--height', '435', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 220.0291725 31.1631078
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 37559 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 37560 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 37561 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 37562 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 37563 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 37564 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 37565 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 37566 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 37567 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 37568 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 37569 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 37570 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 37571 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 37572 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 37573 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 37574 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0712 MB
Starting process 37575 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0712 MB
Starting process 37576 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072216 MB
Starting process 37577 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072216 MB
Starting process 37578 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071924 MB
Starting process 37579 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071924 MB
Starting process 37580 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07156 MB
Starting process 37581 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB
Starting process 37582 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB
Starting process 37583 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB
Starting process 37584 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB
Starting process 37585 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB
Starting process 37586 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB
Starting process 37587 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB
Starting process 37588 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB
Starting process 37589 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB
Starting process 37590 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T09:57:26.029386
Keeping 15 CCDs:
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.29 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.42 MJD 57843.287 object DECaLS_22744_g
   c4d_160229_082453_ooi_z_ls9.fits.fz c4d_160229_082453_ooi_z_ls9-S1 z exptime 80.0 propid 2014B-0404 seeing 1.03 MJD 57447.349 object DECaLS_250_z
   c4d_170329_073043_ooi_r_ls9.fits.fz c4d_170329_073043_ooi_r_ls9-S1 r exptime 63.0 propid 2014B-0404 seeing 1.32 MJD 57841.312 object DECaLS_250_r
   c4d_170422_054608_ooi_g_ls9.fits.fz c4d_170422_054608_ooi_g_ls9-S1 g exptime 122.0 propid 2014B-0404 seeing 1.25 MJD 57865.238 object DECaLS_250_g
   c4d_180222_090307_ooi_r_ls9.fits.fz c4d_180222_090307_ooi_r_ls9-S5 r exptime 51.0 propid 2014B-0404 seeing 0.93 MJD 58171.377 object DECaLS_38616_r
   c4d_180222_090427_ooi_g_ls9.fits.fz c4d_180222_090427_ooi_g_ls9-S5 g exptime 121.0 propid 2014B-0404 seeing 1.15 MJD 58171.378 object DECaLS_38616_g
   c4d_160226_091110_ooi_z_ls9.fits.fz c4d_160226_091110_ooi_z_ls9-S5 z exptime 155.0 propid 2014B-0404 seeing 0.96 MJD 57444.380 object DECaLS_38616_z
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S6 z exptime 142.0 propid 2014B-0404 seeing 1.10 MJD 57444.347 object DECaLS_22744_z
   c4d_170329_070606_ooi_r_ls9.fits.fz c4d_170329_070606_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.32 MJD 57841.295 object DECaLS_6872_r
   c4d_160228_083710_ooi_z_ls9.fits.fz c4d_160228_083710_ooi_z_ls9-S6 z exptime 141.0 propid 2014B-0404 seeing 0.92 MJD 57446.357 object DECaLS_6872_z
   c4d_170422_050829_ooi_g_ls9.fits.fz c4d_170422_050829_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57865.212 object DECaLS_6872_g
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S5 r exptime 62.0 propid 2014B-0404 seeing 1.31 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S5 g exptime 175.0 propid 2014B-0404 seeing 1.43 MJD 57843.287 object DECaLS_22744_g
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S5 z exptime 142.0 propid 2014B-0404 seeing 1.14 MJD 57444.347 object DECaLS_22744_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T083400_oi[S6,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T090745_oi[S5,noinherit][2046,4094][real]: DECaLS_3
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T090745_oi[S5,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T083400_oi[S6,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170330/c4d_170331_0
65648_ooi_g_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1934503; please check the logfile.
Finished galaxy PGC1934503 in 0.053 minutes.
Started working on galaxy PGC1934503 at Mon Feb 10 06:13:22 2020
RA, Dec=220.029172, 31.163108: run=decam (2 north CCDs, 15 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-10T06:13:22.908966
legacypipe git version: dr9e-pre1-9-g4a51be93
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '220.0291725', '31.1631078', '--width', '435', '--height', '435', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 220.0291725 31.1631078 --width 435 --height 435 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/220/PGC1934503/PGC1934503-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 220.0291725 31.1631078
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 12114 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 12115 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 12116 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 12117 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 12118 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 12119 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 12120 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 12121 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 12122 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 12123 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 12124 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 12125 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 12126 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 12127 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 12128 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 12129 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 12130 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.07116 MB
Starting process 12131 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072176 MB
Starting process 12132 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072176 MB
Starting process 12133 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07188 MB
Starting process 12134 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07188 MB
Starting process 12135 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12136 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12137 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12138 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12139 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12140 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12141 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12142 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12143 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12144 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Starting process 12145 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-10T06:13:24.157733
Keeping 15 CCDs:
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.29 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.42 MJD 57843.287 object DECaLS_22744_g
   c4d_160229_082453_ooi_z_ls9.fits.fz c4d_160229_082453_ooi_z_ls9-S1 z exptime 80.0 propid 2014B-0404 seeing 1.03 MJD 57447.349 object DECaLS_250_z
   c4d_170329_073043_ooi_r_ls9.fits.fz c4d_170329_073043_ooi_r_ls9-S1 r exptime 63.0 propid 2014B-0404 seeing 1.32 MJD 57841.312 object DECaLS_250_r
   c4d_170422_054608_ooi_g_ls9.fits.fz c4d_170422_054608_ooi_g_ls9-S1 g exptime 122.0 propid 2014B-0404 seeing 1.25 MJD 57865.238 object DECaLS_250_g
   c4d_180222_090307_ooi_r_ls9.fits.fz c4d_180222_090307_ooi_r_ls9-S5 r exptime 51.0 propid 2014B-0404 seeing 0.93 MJD 58171.377 object DECaLS_38616_r
   c4d_180222_090427_ooi_g_ls9.fits.fz c4d_180222_090427_ooi_g_ls9-S5 g exptime 121.0 propid 2014B-0404 seeing 1.15 MJD 58171.378 object DECaLS_38616_g
   c4d_160226_091110_ooi_z_ls9.fits.fz c4d_160226_091110_ooi_z_ls9-S5 z exptime 155.0 propid 2014B-0404 seeing 0.96 MJD 57444.380 object DECaLS_38616_z
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S6 z exptime 142.0 propid 2014B-0404 seeing 1.10 MJD 57444.347 object DECaLS_22744_z
   c4d_170329_070606_ooi_r_ls9.fits.fz c4d_170329_070606_ooi_r_ls9-S6 r exptime 62.0 propid 2014B-0404 seeing 1.32 MJD 57841.295 object DECaLS_6872_r
   c4d_160228_083710_ooi_z_ls9.fits.fz c4d_160228_083710_ooi_z_ls9-S6 z exptime 141.0 propid 2014B-0404 seeing 0.92 MJD 57446.357 object DECaLS_6872_z
   c4d_170422_050829_ooi_g_ls9.fits.fz c4d_170422_050829_ooi_g_ls9-S6 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57865.212 object DECaLS_6872_g
   c4d_170331_065325_ooi_r_ls9.fits.fz c4d_170331_065325_ooi_r_ls9-S5 r exptime 62.0 propid 2014B-0404 seeing 1.31 MJD 57843.286 object DECaLS_22744_r
   c4d_170331_065648_ooi_g_ls9.fits.fz c4d_170331_065648_ooi_g_ls9-S5 g exptime 175.0 propid 2014B-0404 seeing 1.43 MJD 57843.287 object DECaLS_22744_g
   c4d_160226_082233_ooi_z_ls9.fits.fz c4d_160226_082233_ooi_z_ls9-S5 z exptime 142.0 propid 2014B-0404 seeing 1.14 MJD 57444.347 object DECaLS_22744_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T083400_oi[S6,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T090745_oi[S5,noinherit][2046,4094][real]: DECaLS_3
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S5,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081923_oi[S6,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T090745_oi[S5,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T083400_oi[S6,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170330/c4d_170331_0
65648_ooi_g_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1934503; please check the logfile.
Finished galaxy PGC1934503 in 0.055 minutes.