Started working on galaxy PGC1952328 at Sat Feb  8 08:43:11 2020
RA, Dec=216.583215, 31.544953: run=decam (4 north CCDs, 18 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:43:12.051368
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.583215', '31.5449528', '--width', '349', '--height', '349', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 216.583215 31.5449528
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 9167 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 9168 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 9169 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 9170 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 9171 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 9172 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 9173 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 9174 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:43:13.507816
Keeping 18 CCDs:
   c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-S19 z exptime 111.0 propid 2014B-0404 seeing 1.00 MJD 58158.368 object DECaLS_38587_z
   c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-S19 r exptime 46.0 propid 2014B-0404 seeing 1.04 MJD 58171.374 object DECaLS_38587_r
   c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-S19 z exptime 213.0 propid 2014B-0404 seeing 1.25 MJD 57864.254 object DECaLS_38587_z
   c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-S19 g exptime 161.0 propid 2014B-0404 seeing 1.45 MJD 58170.371 object DECaLS_38587_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S15 z exptime 115.0 propid 2014B-0404 seeing 0.97 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S15 g exptime 149.0 propid 2014B-0404 seeing 1.36 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S15 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S15 g exptime 175.0 propid 2014B-0404 seeing 1.64 MJD 57849.282 object DECaLS_6859_g
   c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57895.120 object DECaLS_22715_g
   c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-S19 z exptime 133.0 propid 2014B-0404 seeing 1.00 MJD 57444.339 object DECaLS_22715_z
   c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-S19 r exptime 125.0 propid 2014B-0404 seeing 1.59 MJD 57894.135 object DECaLS_22715_r
   c4d_170503_050055_ooi_r_ls9.fits.fz c4d_170503_050055_ooi_r_ls9-S14 r exptime 112.0 propid 2014B-0404 seeing 1.33 MJD 57876.207 object DECaLS_22731_r
   c4d_160226_081655_ooi_z_ls9.fits.fz c4d_160226_081655_ooi_z_ls9-S14 z exptime 135.0 propid 2014B-0404 seeing 1.12 MJD 57444.343 object DECaLS_22731_z
   c4d_170422_051737_ooi_g_ls9.fits.fz c4d_170422_051737_ooi_g_ls9-S14 g exptime 121.0 propid 2014B-0404 seeing 1.35 MJD 57865.219 object DECaLS_22731_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S9 z exptime 115.0 propid 2014B-0404 seeing 0.90 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S9 g exptime 149.0 propid 2014B-0404 seeing 1.31 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S9 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S9 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57849.282 object DECaLS_6859_g
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081350_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T080822_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95130c6-z-20170421T060621_oi[S19,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081350_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_
085753_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1952328; please check the logfile.
Finished galaxy PGC1952328 in 0.100 minutes.
Started working on galaxy PGC1952328 at Sat Feb  8 12:51:21 2020
RA, Dec=216.583215, 31.544953: run=decam (4 north CCDs, 18 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T12:51:22.057089
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.583215', '31.5449528', '--width', '349', '--height', '349', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 216.583215 31.5449528
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 31833 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31834 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31835 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31836 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31837 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31838 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 31839 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 31840 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T12:51:23.292359
Keeping 18 CCDs:
   c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-S19 z exptime 111.0 propid 2014B-0404 seeing 1.00 MJD 58158.368 object DECaLS_38587_z
   c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-S19 r exptime 46.0 propid 2014B-0404 seeing 1.04 MJD 58171.374 object DECaLS_38587_r
   c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-S19 z exptime 213.0 propid 2014B-0404 seeing 1.25 MJD 57864.254 object DECaLS_38587_z
   c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-S19 g exptime 161.0 propid 2014B-0404 seeing 1.45 MJD 58170.371 object DECaLS_38587_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S15 z exptime 115.0 propid 2014B-0404 seeing 0.97 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S15 g exptime 149.0 propid 2014B-0404 seeing 1.36 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S15 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S15 g exptime 175.0 propid 2014B-0404 seeing 1.64 MJD 57849.282 object DECaLS_6859_g
   c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57895.120 object DECaLS_22715_g
   c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-S19 z exptime 133.0 propid 2014B-0404 seeing 1.00 MJD 57444.339 object DECaLS_22715_z
   c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-S19 r exptime 125.0 propid 2014B-0404 seeing 1.59 MJD 57894.135 object DECaLS_22715_r
   c4d_170503_050055_ooi_r_ls9.fits.fz c4d_170503_050055_ooi_r_ls9-S14 r exptime 112.0 propid 2014B-0404 seeing 1.33 MJD 57876.207 object DECaLS_22731_r
   c4d_160226_081655_ooi_z_ls9.fits.fz c4d_160226_081655_ooi_z_ls9-S14 z exptime 135.0 propid 2014B-0404 seeing 1.12 MJD 57444.343 object DECaLS_22731_z
   c4d_170422_051737_ooi_g_ls9.fits.fz c4d_170422_051737_ooi_g_ls9-S14 g exptime 121.0 propid 2014B-0404 seeing 1.35 MJD 57865.219 object DECaLS_22731_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S9 z exptime 115.0 propid 2014B-0404 seeing 0.90 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S9 g exptime 149.0 propid 2014B-0404 seeing 1.31 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S9 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S9 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57849.282 object DECaLS_6859_g
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95130c6-z-20170421T060621_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081350_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T080822_oi[S19,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T080822_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95130c6-z-20170421T060621_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_
085753_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1952328; please check the logfile.
Finished galaxy PGC1952328 in 0.086 minutes.
Started working on galaxy PGC1952328 at Sun Feb  9 09:46:59 2020
RA, Dec=216.583215, 31.544953: run=decam (4 north CCDs, 18 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T09:47:00.219729
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.583215', '31.5449528', '--width', '349', '--height', '349', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 216.583215 31.5449528
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 33994 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 33995 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 33996 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 33997 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 33998 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 33999 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 34000 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 34001 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 34002 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 34003 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 34004 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 34005 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 34006 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 34007 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 34008 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 34009 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 34010 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071196 MB
Starting process 34011 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072208 MB
Starting process 34012 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072208 MB
Starting process 34013 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07192 MB
Starting process 34014 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071916 MB
Starting process 34015 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 34016 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 34017 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 34018 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 34019 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 34020 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 34021 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 34022 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 34023 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 34024 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Starting process 34025 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T09:47:01.471850
Keeping 18 CCDs:
   c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-S19 z exptime 111.0 propid 2014B-0404 seeing 1.00 MJD 58158.368 object DECaLS_38587_z
   c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-S19 r exptime 46.0 propid 2014B-0404 seeing 1.04 MJD 58171.374 object DECaLS_38587_r
   c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-S19 z exptime 213.0 propid 2014B-0404 seeing 1.25 MJD 57864.254 object DECaLS_38587_z
   c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-S19 g exptime 161.0 propid 2014B-0404 seeing 1.45 MJD 58170.371 object DECaLS_38587_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S15 z exptime 115.0 propid 2014B-0404 seeing 0.97 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S15 g exptime 149.0 propid 2014B-0404 seeing 1.36 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S15 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S15 g exptime 175.0 propid 2014B-0404 seeing 1.64 MJD 57849.282 object DECaLS_6859_g
   c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57895.120 object DECaLS_22715_g
   c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-S19 z exptime 133.0 propid 2014B-0404 seeing 1.00 MJD 57444.339 object DECaLS_22715_z
   c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-S19 r exptime 125.0 propid 2014B-0404 seeing 1.59 MJD 57894.135 object DECaLS_22715_r
   c4d_170503_050055_ooi_r_ls9.fits.fz c4d_170503_050055_ooi_r_ls9-S14 r exptime 112.0 propid 2014B-0404 seeing 1.33 MJD 57876.207 object DECaLS_22731_r
   c4d_160226_081655_ooi_z_ls9.fits.fz c4d_160226_081655_ooi_z_ls9-S14 z exptime 135.0 propid 2014B-0404 seeing 1.12 MJD 57444.343 object DECaLS_22731_z
   c4d_170422_051737_ooi_g_ls9.fits.fz c4d_170422_051737_ooi_g_ls9-S14 g exptime 121.0 propid 2014B-0404 seeing 1.35 MJD 57865.219 object DECaLS_22731_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S9 z exptime 115.0 propid 2014B-0404 seeing 0.90 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S9 g exptime 149.0 propid 2014B-0404 seeing 1.31 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S9 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S9 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57849.282 object DECaLS_6859_g
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T080822_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081350_oi[S14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95130c6-z-20170421T060621_oi[S19,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T080822_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95130c6-z-20170421T060621_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081350_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_
085753_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1952328; please check the logfile.
Finished galaxy PGC1952328 in 0.055 minutes.
Started working on galaxy PGC1952328 at Mon Feb 10 06:10:30 2020
RA, Dec=216.583215, 31.544953: run=decam (4 north CCDs, 18 south CCDs).
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-10T06:10:31.536016
legacypipe git version: dr9e-pre1-9-g4a51be93
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.583215', '31.5449528', '--width', '349', '--height', '349', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.583215 31.5449528 --width 349 --height 349 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1952328/PGC1952328-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 216.583215 31.5449528
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 10170 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 10171 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 10172 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 10173 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 10174 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 10175 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 10176 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 10177 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 10178 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 10179 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 10180 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 10181 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 10182 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 10183 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 10184 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 10185 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 10186 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 10187 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072168 MB
Starting process 10188 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072168 MB
Starting process 10189 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071876 MB
Starting process 10190 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071872 MB
Starting process 10191 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 10192 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 10200 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 10201 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 10202 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 10207 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 10208 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 10209 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071448 MB
Starting process 10210 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071448 MB
Starting process 10211 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB
Starting process 10212 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-10T06:10:32.846527
Keeping 18 CCDs:
   c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-S19 z exptime 111.0 propid 2014B-0404 seeing 1.00 MJD 58158.368 object DECaLS_38587_z
   c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-S19 r exptime 46.0 propid 2014B-0404 seeing 1.04 MJD 58171.374 object DECaLS_38587_r
   c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-S19 z exptime 213.0 propid 2014B-0404 seeing 1.25 MJD 57864.254 object DECaLS_38587_z
   c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-S19 g exptime 161.0 propid 2014B-0404 seeing 1.45 MJD 58170.371 object DECaLS_38587_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S15 z exptime 115.0 propid 2014B-0404 seeing 0.97 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S15 g exptime 149.0 propid 2014B-0404 seeing 1.36 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S15 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S15 g exptime 175.0 propid 2014B-0404 seeing 1.64 MJD 57849.282 object DECaLS_6859_g
   c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-S19 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57895.120 object DECaLS_22715_g
   c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-S19 z exptime 133.0 propid 2014B-0404 seeing 1.00 MJD 57444.339 object DECaLS_22715_z
   c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-S19 r exptime 125.0 propid 2014B-0404 seeing 1.59 MJD 57894.135 object DECaLS_22715_r
   c4d_170503_050055_ooi_r_ls9.fits.fz c4d_170503_050055_ooi_r_ls9-S14 r exptime 112.0 propid 2014B-0404 seeing 1.33 MJD 57876.207 object DECaLS_22731_r
   c4d_160226_081655_ooi_z_ls9.fits.fz c4d_160226_081655_ooi_z_ls9-S14 z exptime 135.0 propid 2014B-0404 seeing 1.12 MJD 57444.343 object DECaLS_22731_z
   c4d_170422_051737_ooi_g_ls9.fits.fz c4d_170422_051737_ooi_g_ls9-S14 g exptime 121.0 propid 2014B-0404 seeing 1.35 MJD 57865.219 object DECaLS_22731_g
   c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-S9 z exptime 115.0 propid 2014B-0404 seeing 0.90 MJD 57446.348 object DECaLS_6859_z
   c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-S9 g exptime 149.0 propid 2014B-0404 seeing 1.31 MJD 58255.138 object DECaLS_6859_g
   c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-S9 r exptime 62.0 propid 2014B-0404 seeing 1.25 MJD 57841.294 object DECaLS_6859_r
   c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-S9 g exptime 175.0 propid 2014B-0404 seeing 1.35 MJD 57849.282 object DECaLS_6859_g
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95130c6-z-20170421T060621_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T080822_oi[S19,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081350_oi[S14,noinherit][2046,4094][real]: DECaLS_
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95130c6-z-20170421T060621_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9527115-z-20180209T085020_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T080822_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95508ff-z-20160226T081350_oi[S14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954f856-z-20160228T082024_oi[S9,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: FITSIO status = 104: could not open the named file
failed to find or open the following file: (ffopen)
/global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_
085753_ooi_r_ls9.fits.fz

Something went wrong; please check the logfile.
ERROR: galaxy PGC1952328; please check the logfile.
Finished galaxy PGC1952328 in 0.094 minutes.