Started working on galaxy PGC1894069 at Sat Feb  8 08:02:52 2020
RA, Dec=215.098344, 30.195911: run=decam (1 north CCDs, 10 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:02:53.244228
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.09834400000003', '30.195910700000002', '--width', '467', '--height', '467', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.09834400000003 30.195910700000002
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 27252 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 27253 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 27254 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 27255 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 27256 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 27257 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 27258 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 27259 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:02:54.482050
Keeping 10 CCDs:
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S24 r exptime 72.0 propid 2014B-0404 seeing 1.20 MJD 57865.209 object DECaLS_22695_r
   c4d_170406_064656_ooi_r_ls9.fits.fz c4d_170406_064656_ooi_r_ls9-S19 r exptime 85.0 propid 2014B-0404 seeing 1.27 MJD 57849.281 object DECaLS_6823_r
   c4d_150411_054325_ooi_z_ls9.fits.fz c4d_150411_054325_ooi_z_ls9-S19 z exptime 88.0 propid 2014B-0404 seeing 1.07 MJD 57123.237 object DECaLS_6823_z
   c4d_180213_085746_ooi_g_ls9.fits.fz c4d_180213_085746_ooi_g_ls9-S19 g exptime 117.0 propid 2014B-0404 seeing 1.22 MJD 58162.373 object DECaLS_6823_g
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S23 z exptime 80.0 propid 2014B-0404 seeing 1.10 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S23 r exptime 47.0 propid 2014B-0404 seeing 0.93 MJD 58171.364 object DECaLS_38567_r
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S23 g exptime 105.0 propid 2014B-0404 seeing 1.14 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S23 z exptime 87.0 propid 2014B-0404 seeing 1.33 MJD 58547.300 object DECaLS_38567_z
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S24 g exptime 175.0 propid 2014B-0404 seeing 1.37 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S24 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170405/c4d_170406_0
64656_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1894069; please check the logfile.
Finished galaxy PGC1894069 in 0.102 minutes.
Started working on galaxy PGC1894069 at Sat Feb  8 08:48:25 2020
RA, Dec=215.098344, 30.195911: run=decam (1 north CCDs, 10 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:48:26.735885
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.09834400000003', '30.195910700000002', '--width', '467', '--height', '467', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.09834400000003 30.195910700000002
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 64268 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 64269 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 64270 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 64271 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 64272 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 64273 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 64274 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 64275 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:48:27.990008
Keeping 10 CCDs:
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S24 r exptime 72.0 propid 2014B-0404 seeing 1.20 MJD 57865.209 object DECaLS_22695_r
   c4d_170406_064656_ooi_r_ls9.fits.fz c4d_170406_064656_ooi_r_ls9-S19 r exptime 85.0 propid 2014B-0404 seeing 1.27 MJD 57849.281 object DECaLS_6823_r
   c4d_150411_054325_ooi_z_ls9.fits.fz c4d_150411_054325_ooi_z_ls9-S19 z exptime 88.0 propid 2014B-0404 seeing 1.07 MJD 57123.237 object DECaLS_6823_z
   c4d_180213_085746_ooi_g_ls9.fits.fz c4d_180213_085746_ooi_g_ls9-S19 g exptime 117.0 propid 2014B-0404 seeing 1.22 MJD 58162.373 object DECaLS_6823_g
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S23 z exptime 80.0 propid 2014B-0404 seeing 1.10 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S23 r exptime 47.0 propid 2014B-0404 seeing 0.93 MJD 58171.364 object DECaLS_38567_r
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S23 g exptime 105.0 propid 2014B-0404 seeing 1.14 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S23 z exptime 87.0 propid 2014B-0404 seeing 1.33 MJD 58547.300 object DECaLS_38567_z
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S24 g exptime 175.0 propid 2014B-0404 seeing 1.37 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S24 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_
084443_ooi_g_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1894069; please check the logfile.
Finished galaxy PGC1894069 in 0.101 minutes.
Started working on galaxy PGC1894069 at Sat Feb  8 12:46:04 2020
RA, Dec=215.098344, 30.195911: run=decam (1 north CCDs, 10 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T12:46:05.287819
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.09834400000003', '30.195910700000002', '--width', '467', '--height', '467', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.09834400000003 30.195910700000002
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 1810 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 1811 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 1812 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 1813 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 1814 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 1815 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 1816 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 1817 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T12:46:06.600801
Keeping 10 CCDs:
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S24 r exptime 72.0 propid 2014B-0404 seeing 1.20 MJD 57865.209 object DECaLS_22695_r
   c4d_170406_064656_ooi_r_ls9.fits.fz c4d_170406_064656_ooi_r_ls9-S19 r exptime 85.0 propid 2014B-0404 seeing 1.27 MJD 57849.281 object DECaLS_6823_r
   c4d_150411_054325_ooi_z_ls9.fits.fz c4d_150411_054325_ooi_z_ls9-S19 z exptime 88.0 propid 2014B-0404 seeing 1.07 MJD 57123.237 object DECaLS_6823_z
   c4d_180213_085746_ooi_g_ls9.fits.fz c4d_180213_085746_ooi_g_ls9-S19 g exptime 117.0 propid 2014B-0404 seeing 1.22 MJD 58162.373 object DECaLS_6823_g
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S23 z exptime 80.0 propid 2014B-0404 seeing 1.10 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S23 r exptime 47.0 propid 2014B-0404 seeing 0.93 MJD 58171.364 object DECaLS_38567_r
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S23 g exptime 105.0 propid 2014B-0404 seeing 1.14 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S23 z exptime 87.0 propid 2014B-0404 seeing 1.33 MJD 58547.300 object DECaLS_38567_z
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S24 g exptime 175.0 propid 2014B-0404 seeing 1.37 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S24 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1894069; please check the logfile.
Finished galaxy PGC1894069 in 0.090 minutes.
Started working on galaxy PGC1894069 at Sun Feb  9 10:01:45 2020
RA, Dec=215.098344, 30.195911: run=decam (1 north CCDs, 10 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T10:01:45.624178
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.09834400000003', '30.195910700000002', '--width', '467', '--height', '467', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.09834400000003 30.195910700000002
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 38972 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38973 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38974 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38975 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38976 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 38977 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 38978 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 38979 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 38980 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 38981 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 38982 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 38983 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 38984 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 38985 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 38986 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 38987 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 38988 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 38989 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072208 MB
Starting process 38990 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072212 MB
Starting process 38991 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071928 MB
Starting process 38992 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07192 MB
Starting process 38993 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 38994 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB
Starting process 38995 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB
Starting process 38996 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB
Starting process 38997 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB
Starting process 38998 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Starting process 38999 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Starting process 39000 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Starting process 39001 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Starting process 39002 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB
Starting process 39003 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T10:01:46.905394
Keeping 10 CCDs:
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S24 r exptime 72.0 propid 2014B-0404 seeing 1.20 MJD 57865.209 object DECaLS_22695_r
   c4d_170406_064656_ooi_r_ls9.fits.fz c4d_170406_064656_ooi_r_ls9-S19 r exptime 85.0 propid 2014B-0404 seeing 1.27 MJD 57849.281 object DECaLS_6823_r
   c4d_150411_054325_ooi_z_ls9.fits.fz c4d_150411_054325_ooi_z_ls9-S19 z exptime 88.0 propid 2014B-0404 seeing 1.07 MJD 57123.237 object DECaLS_6823_z
   c4d_180213_085746_ooi_g_ls9.fits.fz c4d_180213_085746_ooi_g_ls9-S19 g exptime 117.0 propid 2014B-0404 seeing 1.22 MJD 58162.373 object DECaLS_6823_g
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S23 z exptime 80.0 propid 2014B-0404 seeing 1.10 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S23 r exptime 47.0 propid 2014B-0404 seeing 0.93 MJD 58171.364 object DECaLS_38567_r
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S23 g exptime 105.0 propid 2014B-0404 seeing 1.14 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S23 z exptime 87.0 propid 2014B-0404 seeing 1.33 MJD 58547.300 object DECaLS_38567_z
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S24 g exptime 175.0 propid 2014B-0404 seeing 1.37 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S24 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1894069; please check the logfile.
Finished galaxy PGC1894069 in 0.085 minutes.
Started working on galaxy PGC1894069 at Mon Feb 10 06:14:04 2020
RA, Dec=215.098344, 30.195911: run=decam (1 north CCDs, 10 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-10T06:14:05.186940
legacypipe git version: dr9e-pre1-9-g4a51be93
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.09834400000003', '30.195910700000002', '--width', '467', '--height', '467', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.09834400000003 30.195910700000002 --width 467 --height 467 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1894069/PGC1894069-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 215.09834400000003 30.195910700000002
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 12767 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 12768 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 12769 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 12770 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 12771 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 12772 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 12773 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 12774 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 12775 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 12776 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 12777 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 12778 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 12779 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 12780 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 12781 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 12782 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 12783 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 12784 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072164 MB
Starting process 12785 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072168 MB
Starting process 12786 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072044 MB
Starting process 12787 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0721 MB
Starting process 12788 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 12789 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12790 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12791 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12792 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12793 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12794 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12795 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12796 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12797 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 12798 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-10T06:14:06.467848
Keeping 10 CCDs:
   c4d_170422_050255_ooi_r_ls9.fits.fz c4d_170422_050255_ooi_r_ls9-S24 r exptime 72.0 propid 2014B-0404 seeing 1.20 MJD 57865.209 object DECaLS_22695_r
   c4d_170406_064656_ooi_r_ls9.fits.fz c4d_170406_064656_ooi_r_ls9-S19 r exptime 85.0 propid 2014B-0404 seeing 1.27 MJD 57849.281 object DECaLS_6823_r
   c4d_150411_054325_ooi_z_ls9.fits.fz c4d_150411_054325_ooi_z_ls9-S19 z exptime 88.0 propid 2014B-0404 seeing 1.07 MJD 57123.237 object DECaLS_6823_z
   c4d_180213_085746_ooi_g_ls9.fits.fz c4d_180213_085746_ooi_g_ls9-S19 g exptime 117.0 propid 2014B-0404 seeing 1.22 MJD 58162.373 object DECaLS_6823_g
   c4d_170317_072842_ooi_z_ls9.fits.fz c4d_170317_072842_ooi_z_ls9-S23 z exptime 80.0 propid 2014B-0404 seeing 1.10 MJD 57829.310 object DECaLS_38567_z
   c4d_180222_084327_ooi_r_ls9.fits.fz c4d_180222_084327_ooi_r_ls9-S23 r exptime 47.0 propid 2014B-0404 seeing 0.93 MJD 58171.364 object DECaLS_38567_r
   c4d_180222_084443_ooi_g_ls9.fits.fz c4d_180222_084443_ooi_g_ls9-S23 g exptime 105.0 propid 2014B-0404 seeing 1.14 MJD 58171.364 object DECaLS_38567_g
   c4d_190305_071119_ooi_z_ls9.fits.fz c4d_190305_071119_ooi_z_ls9-S23 z exptime 87.0 propid 2014B-0404 seeing 1.33 MJD 58547.300 object DECaLS_38567_z
   c4d_170522_025302_ooi_g_ls9.fits.fz c4d_170522_025302_ooi_g_ls9-S24 g exptime 175.0 propid 2014B-0404 seeing 1.37 MJD 57895.118 object DECaLS_22695_g
   c4d_150408_055503_ooi_z_ls9.fits.fz c4d_150408_055503_ooi_z_ls9-S24 z exptime 109.0 propid 2014B-0404 seeing 1.16 MJD 57120.245 object DECaLS_22695_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9577b3f-z-20150408T055224_oi[S24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95351a0-z-20190305T071119_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9510adb-z-20170317T072630_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956d285-z-20150411T054106_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0
50255_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1894069; please check the logfile.
Finished galaxy PGC1894069 in 0.095 minutes.