Started working on galaxy IC4422 at Sat Feb 8 08:36:09 2020 RA, Dec=216.496390, 30.473782: run=decam (1 north CCDs, 18 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T08:36:10.424336 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4963905', '30.473782399999997', '--width', '507', '--height', '507', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4963905 30.473782399999997 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 9926 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 9927 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 9928 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 9929 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 9930 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 9931 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 9932 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 9933 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T08:36:11.735990 Keeping 18 CCDs: c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N28 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N28 r exptime 46.0 propid 2014B-0404 seeing 0.93 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N28 z exptime 213.0 propid 2014B-0404 seeing 1.24 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N28 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S30 g exptime 148.0 propid 2014B-0404 seeing 1.62 MJD 58255.108 object DECaLS_22714_g c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S30 r exptime 95.0 propid 2014B-0404 seeing 1.30 MJD 57864.240 object DECaLS_22714_r c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S30 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57893.143 object DECaLS_22714_g c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S28 g exptime 113.0 propid 2014B-0404 seeing 1.62 MJD 57952.982 object DECaLS_6842_g c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S28 r exptime 63.0 propid 2014B-0404 seeing 1.20 MJD 57841.291 object DECaLS_6842_r c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S28 z exptime 109.0 propid 2014B-0404 seeing 1.10 MJD 58158.384 object DECaLS_6842_z c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S28 z exptime 88.0 propid 2014B-0404 seeing 1.27 MJD 57123.240 object DECaLS_6842_z c4d_170306_093442_ooi_z_ls9.fits.fz c4d_170306_093442_ooi_z_ls9-S30 z exptime 101.0 propid 2014B-0404 seeing 1.04 MJD 57818.397 object DECaLS_38586_z c4d_180222_085300_ooi_r_ls9.fits.fz c4d_180222_085300_ooi_r_ls9-S30 r exptime 49.0 propid 2014B-0404 seeing 1.00 MJD 58171.370 object DECaLS_38586_r c4d_180222_085419_ooi_g_ls9.fits.fz c4d_180222_085419_ooi_g_ls9-S30 g exptime 185.0 propid 2014B-0404 seeing 1.39 MJD 58171.371 object DECaLS_38586_g c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-N25 z exptime 115.0 propid 2014B-0404 seeing 0.95 MJD 57446.348 object DECaLS_6859_z c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-N25 g exptime 149.0 propid 2014B-0404 seeing 1.29 MJD 58255.138 object DECaLS_6859_g c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-N25 r exptime 62.0 propid 2014B-0404 seeing 1.26 MJD 57841.294 object DECaLS_6859_r c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-N25 g exptime 175.0 propid 2014B-0404 seeing 1.57 MJD 57849.282 object DECaLS_6859_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180220/c4d_180221_ 085437_ooi_g_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy IC4422; please check the logfile. Finished galaxy IC4422 in 0.105 minutes. Started working on galaxy IC4422 at Sat Feb 8 12:46:07 2020 RA, Dec=216.496390, 30.473782: run=decam (1 north CCDs, 18 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T12:46:08.510027 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4963905', '30.473782399999997', '--width', '507', '--height', '507', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4963905 30.473782399999997 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 63556 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 63557 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 63558 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 63559 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 63560 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 63561 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 63562 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 63563 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T12:46:09.807227 Keeping 18 CCDs: c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N28 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N28 r exptime 46.0 propid 2014B-0404 seeing 0.93 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N28 z exptime 213.0 propid 2014B-0404 seeing 1.24 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N28 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S30 g exptime 148.0 propid 2014B-0404 seeing 1.62 MJD 58255.108 object DECaLS_22714_g c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S30 r exptime 95.0 propid 2014B-0404 seeing 1.30 MJD 57864.240 object DECaLS_22714_r c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S30 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57893.143 object DECaLS_22714_g c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S28 g exptime 113.0 propid 2014B-0404 seeing 1.62 MJD 57952.982 object DECaLS_6842_g c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S28 r exptime 63.0 propid 2014B-0404 seeing 1.20 MJD 57841.291 object DECaLS_6842_r c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S28 z exptime 109.0 propid 2014B-0404 seeing 1.10 MJD 58158.384 object DECaLS_6842_z c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S28 z exptime 88.0 propid 2014B-0404 seeing 1.27 MJD 57123.240 object DECaLS_6842_z c4d_170306_093442_ooi_z_ls9.fits.fz c4d_170306_093442_ooi_z_ls9-S30 z exptime 101.0 propid 2014B-0404 seeing 1.04 MJD 57818.397 object DECaLS_38586_z c4d_180222_085300_ooi_r_ls9.fits.fz c4d_180222_085300_ooi_r_ls9-S30 r exptime 49.0 propid 2014B-0404 seeing 1.00 MJD 58171.370 object DECaLS_38586_r c4d_180222_085419_ooi_g_ls9.fits.fz c4d_180222_085419_ooi_g_ls9-S30 g exptime 185.0 propid 2014B-0404 seeing 1.39 MJD 58171.371 object DECaLS_38586_g c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-N25 z exptime 115.0 propid 2014B-0404 seeing 0.95 MJD 57446.348 object DECaLS_6859_z c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-N25 g exptime 149.0 propid 2014B-0404 seeing 1.29 MJD 58255.138 object DECaLS_6859_g c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-N25 r exptime 62.0 propid 2014B-0404 seeing 1.26 MJD 57841.294 object DECaLS_6859_r c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-N25 g exptime 175.0 propid 2014B-0404 seeing 1.57 MJD 57849.282 object DECaLS_6859_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_ 085753_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy IC4422; please check the logfile. Finished galaxy IC4422 in 0.065 minutes. Started working on galaxy IC4422 at Sun Feb 9 08:13:32 2020 RA, Dec=216.496390, 30.473782: run=decam (1 north CCDs, 18 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-09T08:13:33.093497 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4963905', '30.473782399999997', '--width', '507', '--height', '507', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4963905 30.473782399999997 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 27707 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07116 MB Starting process 27708 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 27709 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 27710 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 27711 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 27712 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 27713 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 27714 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 27715 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 27716 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 27717 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 27718 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 27719 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 27720 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 27721 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 27722 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 27723 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 27724 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB Starting process 27725 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB Starting process 27726 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071904 MB Starting process 27727 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071904 MB Starting process 27728 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27729 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27730 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27731 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27732 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27733 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27734 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27735 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB Starting process 27736 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071472 MB Starting process 27737 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB Starting process 27738 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071476 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-09T08:13:34.324904 Keeping 18 CCDs: c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N28 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N28 r exptime 46.0 propid 2014B-0404 seeing 0.93 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N28 z exptime 213.0 propid 2014B-0404 seeing 1.24 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N28 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S30 g exptime 148.0 propid 2014B-0404 seeing 1.62 MJD 58255.108 object DECaLS_22714_g c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S30 r exptime 95.0 propid 2014B-0404 seeing 1.30 MJD 57864.240 object DECaLS_22714_r c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S30 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57893.143 object DECaLS_22714_g c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S28 g exptime 113.0 propid 2014B-0404 seeing 1.62 MJD 57952.982 object DECaLS_6842_g c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S28 r exptime 63.0 propid 2014B-0404 seeing 1.20 MJD 57841.291 object DECaLS_6842_r c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S28 z exptime 109.0 propid 2014B-0404 seeing 1.10 MJD 58158.384 object DECaLS_6842_z c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S28 z exptime 88.0 propid 2014B-0404 seeing 1.27 MJD 57123.240 object DECaLS_6842_z c4d_170306_093442_ooi_z_ls9.fits.fz c4d_170306_093442_ooi_z_ls9-S30 z exptime 101.0 propid 2014B-0404 seeing 1.04 MJD 57818.397 object DECaLS_38586_z c4d_180222_085300_ooi_r_ls9.fits.fz c4d_180222_085300_ooi_r_ls9-S30 r exptime 49.0 propid 2014B-0404 seeing 1.00 MJD 58171.370 object DECaLS_38586_r c4d_180222_085419_ooi_g_ls9.fits.fz c4d_180222_085419_ooi_g_ls9-S30 g exptime 185.0 propid 2014B-0404 seeing 1.39 MJD 58171.371 object DECaLS_38586_g c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-N25 z exptime 115.0 propid 2014B-0404 seeing 0.95 MJD 57446.348 object DECaLS_6859_z c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-N25 g exptime 149.0 propid 2014B-0404 seeing 1.29 MJD 58255.138 object DECaLS_6859_g c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-N25 r exptime 62.0 propid 2014B-0404 seeing 1.26 MJD 57841.294 object DECaLS_6859_r c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-N25 g exptime 175.0 propid 2014B-0404 seeing 1.57 MJD 57849.282 object DECaLS_6859_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_ 085753_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy IC4422; please check the logfile. Finished galaxy IC4422 in 0.080 minutes. Started working on galaxy IC4422 at Sun Feb 9 10:04:05 2020 RA, Dec=216.496390, 30.473782: run=decam (1 north CCDs, 18 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-09T10:04:05.892211 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4963905', '30.473782399999997', '--width', '507', '--height', '507', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4963905 30.473782399999997 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 39633 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 39634 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 39635 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 39636 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 39637 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 39638 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 39639 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 39640 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 39641 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 39642 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 39643 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 39644 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 39645 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 39646 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 39647 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 39648 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39649 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39650 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072204 MB Starting process 39651 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072204 MB Starting process 39652 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071908 MB Starting process 39653 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071908 MB Starting process 39654 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 39655 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 39656 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 39657 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 39658 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 39659 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 39660 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 39661 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB Starting process 39662 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB Starting process 39663 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB Starting process 39664 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-09T10:04:07.262498 Keeping 18 CCDs: c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N28 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N28 r exptime 46.0 propid 2014B-0404 seeing 0.93 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N28 z exptime 213.0 propid 2014B-0404 seeing 1.24 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N28 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S30 g exptime 148.0 propid 2014B-0404 seeing 1.62 MJD 58255.108 object DECaLS_22714_g c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S30 r exptime 95.0 propid 2014B-0404 seeing 1.30 MJD 57864.240 object DECaLS_22714_r c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S30 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57893.143 object DECaLS_22714_g c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S28 g exptime 113.0 propid 2014B-0404 seeing 1.62 MJD 57952.982 object DECaLS_6842_g c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S28 r exptime 63.0 propid 2014B-0404 seeing 1.20 MJD 57841.291 object DECaLS_6842_r c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S28 z exptime 109.0 propid 2014B-0404 seeing 1.10 MJD 58158.384 object DECaLS_6842_z c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S28 z exptime 88.0 propid 2014B-0404 seeing 1.27 MJD 57123.240 object DECaLS_6842_z c4d_170306_093442_ooi_z_ls9.fits.fz c4d_170306_093442_ooi_z_ls9-S30 z exptime 101.0 propid 2014B-0404 seeing 1.04 MJD 57818.397 object DECaLS_38586_z c4d_180222_085300_ooi_r_ls9.fits.fz c4d_180222_085300_ooi_r_ls9-S30 r exptime 49.0 propid 2014B-0404 seeing 1.00 MJD 58171.370 object DECaLS_38586_r c4d_180222_085419_ooi_g_ls9.fits.fz c4d_180222_085419_ooi_g_ls9-S30 g exptime 185.0 propid 2014B-0404 seeing 1.39 MJD 58171.371 object DECaLS_38586_g c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-N25 z exptime 115.0 propid 2014B-0404 seeing 0.95 MJD 57446.348 object DECaLS_6859_z c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-N25 g exptime 149.0 propid 2014B-0404 seeing 1.29 MJD 58255.138 object DECaLS_6859_g c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-N25 r exptime 62.0 propid 2014B-0404 seeing 1.26 MJD 57841.294 object DECaLS_6859_r c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-N25 g exptime 175.0 propid 2014B-0404 seeing 1.57 MJD 57849.282 object DECaLS_6859_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_ 085753_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy IC4422; please check the logfile. Finished galaxy IC4422 in 0.068 minutes. Started working on galaxy IC4422 at Mon Feb 10 06:14:32 2020 RA, Dec=216.496390, 30.473782: run=decam (1 north CCDs, 18 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-10T06:14:33.016905 legacypipe git version: dr9e-pre1-9-g4a51be93 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4963905', '30.473782399999997', '--width', '507', '--height', '507', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4963905 30.473782399999997 --width 507 --height 507 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/IC4422/IC4422-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4963905 30.473782399999997 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 13095 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071128 MB Starting process 13096 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071128 MB Starting process 13097 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071128 MB Starting process 13098 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071128 MB Starting process 13099 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071128 MB Starting process 13100 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071132 MB Starting process 13101 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071136 MB Starting process 13102 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.07114 MB Starting process 13103 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07114 MB Starting process 13104 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.07114 MB Starting process 13105 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071144 MB Starting process 13106 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071144 MB Starting process 13107 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071144 MB Starting process 13108 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071148 MB Starting process 13109 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071148 MB Starting process 13110 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 141 MB, maxrss: 0.071152 MB Starting process 13111 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 141 MB, maxrss: 0.071152 MB Starting process 13112 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.072164 MB Starting process 13113 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.072164 MB Starting process 13114 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071872 MB Starting process 13115 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071868 MB Starting process 13116 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071508 MB Starting process 13117 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071508 MB Starting process 13118 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071508 MB Starting process 13119 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071508 MB Starting process 13120 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071508 MB Starting process 13121 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071444 MB Starting process 13122 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071444 MB Starting process 13123 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071444 MB Starting process 13124 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071444 MB Starting process 13125 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071448 MB Starting process 13126 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071448 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-10T06:14:34.284570 Keeping 18 CCDs: c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N28 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N28 r exptime 46.0 propid 2014B-0404 seeing 0.93 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N28 z exptime 213.0 propid 2014B-0404 seeing 1.24 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N28 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g c4d_180517_023606_ooi_g_ls9.fits.fz c4d_180517_023606_ooi_g_ls9-S30 g exptime 148.0 propid 2014B-0404 seeing 1.62 MJD 58255.108 object DECaLS_22714_g c4d_170421_054800_ooi_r_ls9.fits.fz c4d_170421_054800_ooi_r_ls9-S30 r exptime 95.0 propid 2014B-0404 seeing 1.30 MJD 57864.240 object DECaLS_22714_r c4d_170520_033017_ooi_g_ls9.fits.fz c4d_170520_033017_ooi_g_ls9-S30 g exptime 175.0 propid 2014B-0404 seeing 1.66 MJD 57893.143 object DECaLS_22714_g c4d_170718_233712_ooi_g_ls9.fits.fz c4d_170718_233712_ooi_g_ls9-S28 g exptime 113.0 propid 2014B-0404 seeing 1.62 MJD 57952.982 object DECaLS_6842_g c4d_170329_070133_ooi_r_ls9.fits.fz c4d_170329_070133_ooi_r_ls9-S28 r exptime 63.0 propid 2014B-0404 seeing 1.20 MJD 57841.291 object DECaLS_6842_r c4d_180209_091225_ooi_z_ls9.fits.fz c4d_180209_091225_ooi_z_ls9-S28 z exptime 109.0 propid 2014B-0404 seeing 1.10 MJD 58158.384 object DECaLS_6842_z c4d_150411_054724_ooi_z_ls9.fits.fz c4d_150411_054724_ooi_z_ls9-S28 z exptime 88.0 propid 2014B-0404 seeing 1.27 MJD 57123.240 object DECaLS_6842_z c4d_170306_093442_ooi_z_ls9.fits.fz c4d_170306_093442_ooi_z_ls9-S30 z exptime 101.0 propid 2014B-0404 seeing 1.04 MJD 57818.397 object DECaLS_38586_z c4d_180222_085300_ooi_r_ls9.fits.fz c4d_180222_085300_ooi_r_ls9-S30 r exptime 49.0 propid 2014B-0404 seeing 1.00 MJD 58171.370 object DECaLS_38586_r c4d_180222_085419_ooi_g_ls9.fits.fz c4d_180222_085419_ooi_g_ls9-S30 g exptime 185.0 propid 2014B-0404 seeing 1.39 MJD 58171.371 object DECaLS_38586_g c4d_160228_082310_ooi_z_ls9.fits.fz c4d_160228_082310_ooi_z_ls9-N25 z exptime 115.0 propid 2014B-0404 seeing 0.95 MJD 57446.348 object DECaLS_6859_z c4d_180517_031813_ooi_g_ls9.fits.fz c4d_180517_031813_ooi_g_ls9-N25 g exptime 149.0 propid 2014B-0404 seeing 1.29 MJD 58255.138 object DECaLS_6859_g c4d_170329_070435_ooi_r_ls9.fits.fz c4d_170329_070435_ooi_r_ls9-N25 r exptime 62.0 propid 2014B-0404 seeing 1.26 MJD 57841.294 object DECaLS_6859_r c4d_170406_065019_ooi_g_ls9.fits.fz c4d_170406_065019_ooi_g_ls9-N25 g exptime 175.0 propid 2014B-0404 seeing 1.57 MJD 57849.282 object DECaLS_6859_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956d285-z-20150411T054506_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091225_oi[S28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093211_oi[S30,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082024_oi[N25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180221/c4d_180222_ 085753_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy IC4422; please check the logfile. Finished galaxy IC4422 in 0.058 minutes.