Started working on galaxy PGC1895214 at Sat Feb 8 08:10:57 2020 RA, Dec=218.182904, 30.224799: run=decam (3 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T08:10:57.579669 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '218.1829035', '30.2247993', '--width', '469', '--height', '469', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 218.1829035 30.2247993 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 23242 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07116 MB Starting process 23243 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 23244 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 23245 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 23246 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 23247 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 23248 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 23249 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T08:10:58.845080 Keeping 14 CCDs: c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S22 r exptime 113.0 propid 2014B-0404 seeing 1.52 MJD 57952.033 object DECaLS_22730_r c4d_170421_055123_ooi_g_ls9.fits.fz c4d_170421_055123_ooi_g_ls9-S22 g exptime 175.0 propid 2014B-0404 seeing 1.44 MJD 57864.241 object DECaLS_22730_g c4d_150408_060636_ooi_z_ls9.fits.fz c4d_150408_060636_ooi_z_ls9-S22 z exptime 109.0 propid 2014B-0404 seeing 1.25 MJD 57120.253 object DECaLS_22730_z c4d_160226_090529_ooi_z_ls9.fits.fz c4d_160226_090529_ooi_z_ls9-N31 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.377 object DECaLS_38603_z c4d_180224_084701_ooi_r_ls9.fits.fz c4d_180224_084701_ooi_r_ls9-N31 r exptime 86.0 propid 2014B-0404 seeing 1.17 MJD 58173.366 object DECaLS_38603_r c4d_180221_090109_ooi_g_ls9.fits.fz c4d_180221_090109_ooi_g_ls9-N31 g exptime 129.0 propid 2014B-0404 seeing 1.28 MJD 58170.376 object DECaLS_38603_g c4d_180210_082602_ooi_z_ls9.fits.fz c4d_180210_082602_ooi_z_ls9-S22 z exptime 138.0 propid 2014B-0404 seeing 1.40 MJD 58159.351 object DECaLS_6858_z c4d_180209_091446_ooi_z_ls9.fits.fz c4d_180209_091446_ooi_z_ls9-S22 z exptime 113.0 propid 2014B-0404 seeing 1.13 MJD 58158.385 object DECaLS_6858_z c4d_170329_070735_ooi_r_ls9.fits.fz c4d_170329_070735_ooi_r_ls9-S22 r exptime 60.0 propid 2014B-0404 seeing 1.31 MJD 57841.296 object DECaLS_6858_r c4d_170329_071015_ooi_g_ls9.fits.fz c4d_170329_071015_ooi_g_ls9-S22 g exptime 132.0 propid 2014B-0404 seeing 1.51 MJD 57841.297 object DECaLS_6858_g c4d_170306_093651_ooi_z_ls9.fits.fz c4d_170306_093651_ooi_z_ls9-S25 z exptime 99.0 propid 2014B-0404 seeing 0.90 MJD 57818.399 object DECaLS_38602_z c4d_180225_081201_ooi_r_ls9.fits.fz c4d_180225_081201_ooi_r_ls9-S25 r exptime 88.0 propid 2014B-0404 seeing 1.19 MJD 58174.342 object DECaLS_38602_r c4d_180224_084906_ooi_g_ls9.fits.fz c4d_180224_084906_ooi_g_ls9-S25 g exptime 128.0 propid 2014B-0404 seeing 1.42 MJD 58173.367 object DECaLS_38602_g c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S26 r exptime 113.0 propid 2014B-0404 seeing 1.54 MJD 57952.033 object DECaLS_22730_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093421_oi[S25,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170717/c4d_170718_0 05053_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1895214; please check the logfile. Finished galaxy PGC1895214 in 0.086 minutes. Started working on galaxy PGC1895214 at Sat Feb 8 08:35:00 2020 RA, Dec=218.182904, 30.224799: run=decam (3 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T08:35:01.088677 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '218.1829035', '30.2247993', '--width', '469', '--height', '469', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 218.1829035 30.2247993 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 35061 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35062 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35063 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35064 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35065 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35066 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 35067 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 35068 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T08:35:02.409438 Keeping 14 CCDs: c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S22 r exptime 113.0 propid 2014B-0404 seeing 1.52 MJD 57952.033 object DECaLS_22730_r c4d_170421_055123_ooi_g_ls9.fits.fz c4d_170421_055123_ooi_g_ls9-S22 g exptime 175.0 propid 2014B-0404 seeing 1.44 MJD 57864.241 object DECaLS_22730_g c4d_150408_060636_ooi_z_ls9.fits.fz c4d_150408_060636_ooi_z_ls9-S22 z exptime 109.0 propid 2014B-0404 seeing 1.25 MJD 57120.253 object DECaLS_22730_z c4d_160226_090529_ooi_z_ls9.fits.fz c4d_160226_090529_ooi_z_ls9-N31 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.377 object DECaLS_38603_z c4d_180224_084701_ooi_r_ls9.fits.fz c4d_180224_084701_ooi_r_ls9-N31 r exptime 86.0 propid 2014B-0404 seeing 1.17 MJD 58173.366 object DECaLS_38603_r c4d_180221_090109_ooi_g_ls9.fits.fz c4d_180221_090109_ooi_g_ls9-N31 g exptime 129.0 propid 2014B-0404 seeing 1.28 MJD 58170.376 object DECaLS_38603_g c4d_180210_082602_ooi_z_ls9.fits.fz c4d_180210_082602_ooi_z_ls9-S22 z exptime 138.0 propid 2014B-0404 seeing 1.40 MJD 58159.351 object DECaLS_6858_z c4d_180209_091446_ooi_z_ls9.fits.fz c4d_180209_091446_ooi_z_ls9-S22 z exptime 113.0 propid 2014B-0404 seeing 1.13 MJD 58158.385 object DECaLS_6858_z c4d_170329_070735_ooi_r_ls9.fits.fz c4d_170329_070735_ooi_r_ls9-S22 r exptime 60.0 propid 2014B-0404 seeing 1.31 MJD 57841.296 object DECaLS_6858_r c4d_170329_071015_ooi_g_ls9.fits.fz c4d_170329_071015_ooi_g_ls9-S22 g exptime 132.0 propid 2014B-0404 seeing 1.51 MJD 57841.297 object DECaLS_6858_g c4d_170306_093651_ooi_z_ls9.fits.fz c4d_170306_093651_ooi_z_ls9-S25 z exptime 99.0 propid 2014B-0404 seeing 0.90 MJD 57818.399 object DECaLS_38602_z c4d_180225_081201_ooi_r_ls9.fits.fz c4d_180225_081201_ooi_r_ls9-S25 r exptime 88.0 propid 2014B-0404 seeing 1.19 MJD 58174.342 object DECaLS_38602_r c4d_180224_084906_ooi_g_ls9.fits.fz c4d_180224_084906_ooi_g_ls9-S25 g exptime 128.0 propid 2014B-0404 seeing 1.42 MJD 58173.367 object DECaLS_38602_g c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S26 r exptime 113.0 propid 2014B-0404 seeing 1.54 MJD 57952.033 object DECaLS_22730_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093421_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093421_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180223/c4d_180224_ 084701_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1895214; please check the logfile. Finished galaxy PGC1895214 in 0.089 minutes. Started working on galaxy PGC1895214 at Sat Feb 8 12:49:38 2020 RA, Dec=218.182904, 30.224799: run=decam (3 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T12:49:40.824210 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '218.1829035', '30.2247993', '--width', '469', '--height', '469', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 218.1829035 30.2247993 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 11700 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 11701 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 11702 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 11703 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 11704 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 11705 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 11706 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 11707 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T12:49:42.084416 Keeping 14 CCDs: c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S22 r exptime 113.0 propid 2014B-0404 seeing 1.52 MJD 57952.033 object DECaLS_22730_r c4d_170421_055123_ooi_g_ls9.fits.fz c4d_170421_055123_ooi_g_ls9-S22 g exptime 175.0 propid 2014B-0404 seeing 1.44 MJD 57864.241 object DECaLS_22730_g c4d_150408_060636_ooi_z_ls9.fits.fz c4d_150408_060636_ooi_z_ls9-S22 z exptime 109.0 propid 2014B-0404 seeing 1.25 MJD 57120.253 object DECaLS_22730_z c4d_160226_090529_ooi_z_ls9.fits.fz c4d_160226_090529_ooi_z_ls9-N31 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.377 object DECaLS_38603_z c4d_180224_084701_ooi_r_ls9.fits.fz c4d_180224_084701_ooi_r_ls9-N31 r exptime 86.0 propid 2014B-0404 seeing 1.17 MJD 58173.366 object DECaLS_38603_r c4d_180221_090109_ooi_g_ls9.fits.fz c4d_180221_090109_ooi_g_ls9-N31 g exptime 129.0 propid 2014B-0404 seeing 1.28 MJD 58170.376 object DECaLS_38603_g c4d_180210_082602_ooi_z_ls9.fits.fz c4d_180210_082602_ooi_z_ls9-S22 z exptime 138.0 propid 2014B-0404 seeing 1.40 MJD 58159.351 object DECaLS_6858_z c4d_180209_091446_ooi_z_ls9.fits.fz c4d_180209_091446_ooi_z_ls9-S22 z exptime 113.0 propid 2014B-0404 seeing 1.13 MJD 58158.385 object DECaLS_6858_z c4d_170329_070735_ooi_r_ls9.fits.fz c4d_170329_070735_ooi_r_ls9-S22 r exptime 60.0 propid 2014B-0404 seeing 1.31 MJD 57841.296 object DECaLS_6858_r c4d_170329_071015_ooi_g_ls9.fits.fz c4d_170329_071015_ooi_g_ls9-S22 g exptime 132.0 propid 2014B-0404 seeing 1.51 MJD 57841.297 object DECaLS_6858_g c4d_170306_093651_ooi_z_ls9.fits.fz c4d_170306_093651_ooi_z_ls9-S25 z exptime 99.0 propid 2014B-0404 seeing 0.90 MJD 57818.399 object DECaLS_38602_z c4d_180225_081201_ooi_r_ls9.fits.fz c4d_180225_081201_ooi_r_ls9-S25 r exptime 88.0 propid 2014B-0404 seeing 1.19 MJD 58174.342 object DECaLS_38602_r c4d_180224_084906_ooi_g_ls9.fits.fz c4d_180224_084906_ooi_g_ls9-S25 g exptime 128.0 propid 2014B-0404 seeing 1.42 MJD 58173.367 object DECaLS_38602_g c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S26 r exptime 113.0 propid 2014B-0404 seeing 1.54 MJD 57952.033 object DECaLS_22730_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093421_oi[S25,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180223/c4d_180224_ 084701_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1895214; please check the logfile. Finished galaxy PGC1895214 in 0.847 minutes. Started working on galaxy PGC1895214 at Sun Feb 9 10:01:50 2020 RA, Dec=218.182904, 30.224799: run=decam (3 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-09T10:01:50.748120 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '218.1829035', '30.2247993', '--width', '469', '--height', '469', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 218.1829035 30.2247993 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 39018 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 39019 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 39020 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 39021 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 39022 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 39023 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 39024 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 39025 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 39026 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 39027 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 39028 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39029 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39030 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39031 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39032 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39033 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 39034 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071196 MB Starting process 39035 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072208 MB Starting process 39036 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072208 MB Starting process 39037 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07192 MB Starting process 39038 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07192 MB Starting process 39039 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 39040 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39041 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39042 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39043 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39044 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39045 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39046 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39047 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39048 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Starting process 39049 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071552 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-09T10:01:52.027639 Keeping 14 CCDs: c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S22 r exptime 113.0 propid 2014B-0404 seeing 1.52 MJD 57952.033 object DECaLS_22730_r c4d_170421_055123_ooi_g_ls9.fits.fz c4d_170421_055123_ooi_g_ls9-S22 g exptime 175.0 propid 2014B-0404 seeing 1.44 MJD 57864.241 object DECaLS_22730_g c4d_150408_060636_ooi_z_ls9.fits.fz c4d_150408_060636_ooi_z_ls9-S22 z exptime 109.0 propid 2014B-0404 seeing 1.25 MJD 57120.253 object DECaLS_22730_z c4d_160226_090529_ooi_z_ls9.fits.fz c4d_160226_090529_ooi_z_ls9-N31 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.377 object DECaLS_38603_z c4d_180224_084701_ooi_r_ls9.fits.fz c4d_180224_084701_ooi_r_ls9-N31 r exptime 86.0 propid 2014B-0404 seeing 1.17 MJD 58173.366 object DECaLS_38603_r c4d_180221_090109_ooi_g_ls9.fits.fz c4d_180221_090109_ooi_g_ls9-N31 g exptime 129.0 propid 2014B-0404 seeing 1.28 MJD 58170.376 object DECaLS_38603_g c4d_180210_082602_ooi_z_ls9.fits.fz c4d_180210_082602_ooi_z_ls9-S22 z exptime 138.0 propid 2014B-0404 seeing 1.40 MJD 58159.351 object DECaLS_6858_z c4d_180209_091446_ooi_z_ls9.fits.fz c4d_180209_091446_ooi_z_ls9-S22 z exptime 113.0 propid 2014B-0404 seeing 1.13 MJD 58158.385 object DECaLS_6858_z c4d_170329_070735_ooi_r_ls9.fits.fz c4d_170329_070735_ooi_r_ls9-S22 r exptime 60.0 propid 2014B-0404 seeing 1.31 MJD 57841.296 object DECaLS_6858_r c4d_170329_071015_ooi_g_ls9.fits.fz c4d_170329_071015_ooi_g_ls9-S22 g exptime 132.0 propid 2014B-0404 seeing 1.51 MJD 57841.297 object DECaLS_6858_g c4d_170306_093651_ooi_z_ls9.fits.fz c4d_170306_093651_ooi_z_ls9-S25 z exptime 99.0 propid 2014B-0404 seeing 0.90 MJD 57818.399 object DECaLS_38602_z c4d_180225_081201_ooi_r_ls9.fits.fz c4d_180225_081201_ooi_r_ls9-S25 r exptime 88.0 propid 2014B-0404 seeing 1.19 MJD 58174.342 object DECaLS_38602_r c4d_180224_084906_ooi_g_ls9.fits.fz c4d_180224_084906_ooi_g_ls9-S25 g exptime 128.0 propid 2014B-0404 seeing 1.42 MJD 58173.367 object DECaLS_38602_g c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S26 r exptime 113.0 propid 2014B-0404 seeing 1.54 MJD 57952.033 object DECaLS_22730_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093421_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093421_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170420/c4d_170421_0 55123_ooi_g_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1895214; please check the logfile. Finished galaxy PGC1895214 in 0.057 minutes. Started working on galaxy PGC1895214 at Mon Feb 10 06:14:10 2020 RA, Dec=218.182904, 30.224799: run=decam (3 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-10T06:14:11.000447 legacypipe git version: dr9e-pre1-9-g4a51be93 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '218.1829035', '30.2247993', '--width', '469', '--height', '469', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 218.1829035 30.2247993 --width 469 --height 469 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/218/PGC1895214/PGC1895214-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 218.1829035 30.2247993 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 12814 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 12815 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 12816 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 12817 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 12818 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 12819 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB Starting process 12820 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB Starting process 12821 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071148 MB Starting process 12822 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071148 MB Starting process 12823 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB Starting process 12824 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 12825 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 12826 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 12827 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 12828 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 12829 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 12830 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.07116 MB Starting process 12831 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072172 MB Starting process 12832 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072172 MB Starting process 12833 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07188 MB Starting process 12834 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07188 MB Starting process 12835 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07116 MB Starting process 12836 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 12837 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 12838 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 12839 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 12840 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 12841 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 12842 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 12843 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 12844 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 12845 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-10T06:14:12.264809 Keeping 14 CCDs: c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S22 r exptime 113.0 propid 2014B-0404 seeing 1.52 MJD 57952.033 object DECaLS_22730_r c4d_170421_055123_ooi_g_ls9.fits.fz c4d_170421_055123_ooi_g_ls9-S22 g exptime 175.0 propid 2014B-0404 seeing 1.44 MJD 57864.241 object DECaLS_22730_g c4d_150408_060636_ooi_z_ls9.fits.fz c4d_150408_060636_ooi_z_ls9-S22 z exptime 109.0 propid 2014B-0404 seeing 1.25 MJD 57120.253 object DECaLS_22730_z c4d_160226_090529_ooi_z_ls9.fits.fz c4d_160226_090529_ooi_z_ls9-N31 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.377 object DECaLS_38603_z c4d_180224_084701_ooi_r_ls9.fits.fz c4d_180224_084701_ooi_r_ls9-N31 r exptime 86.0 propid 2014B-0404 seeing 1.17 MJD 58173.366 object DECaLS_38603_r c4d_180221_090109_ooi_g_ls9.fits.fz c4d_180221_090109_ooi_g_ls9-N31 g exptime 129.0 propid 2014B-0404 seeing 1.28 MJD 58170.376 object DECaLS_38603_g c4d_180210_082602_ooi_z_ls9.fits.fz c4d_180210_082602_ooi_z_ls9-S22 z exptime 138.0 propid 2014B-0404 seeing 1.40 MJD 58159.351 object DECaLS_6858_z c4d_180209_091446_ooi_z_ls9.fits.fz c4d_180209_091446_ooi_z_ls9-S22 z exptime 113.0 propid 2014B-0404 seeing 1.13 MJD 58158.385 object DECaLS_6858_z c4d_170329_070735_ooi_r_ls9.fits.fz c4d_170329_070735_ooi_r_ls9-S22 r exptime 60.0 propid 2014B-0404 seeing 1.31 MJD 57841.296 object DECaLS_6858_r c4d_170329_071015_ooi_g_ls9.fits.fz c4d_170329_071015_ooi_g_ls9-S22 g exptime 132.0 propid 2014B-0404 seeing 1.51 MJD 57841.297 object DECaLS_6858_g c4d_170306_093651_ooi_z_ls9.fits.fz c4d_170306_093651_ooi_z_ls9-S25 z exptime 99.0 propid 2014B-0404 seeing 0.90 MJD 57818.399 object DECaLS_38602_z c4d_180225_081201_ooi_r_ls9.fits.fz c4d_180225_081201_ooi_r_ls9-S25 r exptime 88.0 propid 2014B-0404 seeing 1.19 MJD 58174.342 object DECaLS_38602_r c4d_180224_084906_ooi_g_ls9.fits.fz c4d_180224_084906_ooi_g_ls9-S25 g exptime 128.0 propid 2014B-0404 seeing 1.42 MJD 58173.367 object DECaLS_38602_g c4d_170718_005053_ooi_r_ls9.fits.fz c4d_170718_005053_ooi_r_ls9-S26 r exptime 113.0 propid 2014B-0404 seeing 1.54 MJD 57952.033 object DECaLS_22730_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950ea1d-z-20170306T093421_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T090219_oi[N31,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T091446_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9577b3f-z-20150408T060358_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180210T082602_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170717/c4d_170718_0 05053_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1895214; please check the logfile. Finished galaxy PGC1895214 in 0.058 minutes.