Started working on galaxy PGC1921154 at Sat Feb 8 08:07:37 2020 RA, Dec=215.973249, 30.860960: run=decam (1 north CCDs, 16 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T08:07:38.527482 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.97324899999998', '30.8609602', '--width', '345', '--height', '345', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 215.97324899999998 30.8609602 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 57518 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 57519 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 57520 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071236 MB Starting process 57521 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071236 MB Starting process 57522 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 57523 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07124 MB Starting process 57524 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 57525 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T08:07:49.777037 Keeping 16 CCDs: c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N19 r exptime 64.0 propid 2014B-0404 seeing 1.22 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N19 g exptime 147.0 propid 2014B-0404 seeing 1.22 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N19 z exptime 131.0 propid 2014B-0404 seeing 1.07 MJD 57446.346 object DECaLS_6843_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N18 g exptime 175.0 propid 2014B-0404 seeing 1.65 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N18 r exptime 125.0 propid 2014B-0404 seeing 1.53 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N18 z exptime 133.0 propid 2014B-0404 seeing 0.98 MJD 57444.339 object DECaLS_22715_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N12 g exptime 175.0 propid 2014B-0404 seeing 0.96 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N12 r exptime 125.0 propid 2014B-0404 seeing 1.13 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N12 z exptime 133.0 propid 2014B-0404 seeing 0.96 MJD 57444.339 object DECaLS_22715_z c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N18 r exptime 64.0 propid 2014B-0404 seeing 1.21 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N18 g exptime 147.0 propid 2014B-0404 seeing 1.21 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N18 z exptime 131.0 propid 2014B-0404 seeing 0.98 MJD 57446.346 object DECaLS_6843_z c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N11 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N11 r exptime 46.0 propid 2014B-0404 seeing 0.92 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N11 z exptime 213.0 propid 2014B-0404 seeing 1.19 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N11 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170520/c4d_170521_0 31658_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1921154; please check the logfile. Finished galaxy PGC1921154 in 0.264 minutes. Started working on galaxy PGC1921154 at Sat Feb 8 08:34:45 2020 RA, Dec=215.973249, 30.860960: run=decam (1 north CCDs, 16 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T08:34:46.297791 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.97324899999998', '30.8609602', '--width', '345', '--height', '345', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 215.97324899999998 30.8609602 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 26828 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 26829 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 26830 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 26831 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 26832 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 26833 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 26834 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 26835 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T08:34:47.570601 Keeping 16 CCDs: c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N19 r exptime 64.0 propid 2014B-0404 seeing 1.22 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N19 g exptime 147.0 propid 2014B-0404 seeing 1.22 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N19 z exptime 131.0 propid 2014B-0404 seeing 1.07 MJD 57446.346 object DECaLS_6843_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N18 g exptime 175.0 propid 2014B-0404 seeing 1.65 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N18 r exptime 125.0 propid 2014B-0404 seeing 1.53 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N18 z exptime 133.0 propid 2014B-0404 seeing 0.98 MJD 57444.339 object DECaLS_22715_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N12 g exptime 175.0 propid 2014B-0404 seeing 0.96 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N12 r exptime 125.0 propid 2014B-0404 seeing 1.13 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N12 z exptime 133.0 propid 2014B-0404 seeing 0.96 MJD 57444.339 object DECaLS_22715_z c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N18 r exptime 64.0 propid 2014B-0404 seeing 1.21 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N18 g exptime 147.0 propid 2014B-0404 seeing 1.21 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N18 z exptime 131.0 propid 2014B-0404 seeing 0.98 MJD 57446.346 object DECaLS_6843_z c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N11 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N11 r exptime 46.0 propid 2014B-0404 seeing 0.92 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N11 z exptime 213.0 propid 2014B-0404 seeing 1.19 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N11 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0 52449_ooi_g_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1921154; please check the logfile. Finished galaxy PGC1921154 in 0.102 minutes. Started working on galaxy PGC1921154 at Sat Feb 8 12:46:42 2020 RA, Dec=215.973249, 30.860960: run=decam (1 north CCDs, 16 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T12:46:43.391548 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.97324899999998', '30.8609602', '--width', '345', '--height', '345', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 215.97324899999998 30.8609602 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 29759 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 29760 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 29761 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 29762 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 29763 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 29764 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 29765 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 29766 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T12:46:44.691702 Keeping 16 CCDs: c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N19 r exptime 64.0 propid 2014B-0404 seeing 1.22 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N19 g exptime 147.0 propid 2014B-0404 seeing 1.22 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N19 z exptime 131.0 propid 2014B-0404 seeing 1.07 MJD 57446.346 object DECaLS_6843_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N18 g exptime 175.0 propid 2014B-0404 seeing 1.65 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N18 r exptime 125.0 propid 2014B-0404 seeing 1.53 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N18 z exptime 133.0 propid 2014B-0404 seeing 0.98 MJD 57444.339 object DECaLS_22715_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N12 g exptime 175.0 propid 2014B-0404 seeing 0.96 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N12 r exptime 125.0 propid 2014B-0404 seeing 1.13 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N12 z exptime 133.0 propid 2014B-0404 seeing 0.96 MJD 57444.339 object DECaLS_22715_z c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N18 r exptime 64.0 propid 2014B-0404 seeing 1.21 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N18 g exptime 147.0 propid 2014B-0404 seeing 1.21 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N18 z exptime 131.0 propid 2014B-0404 seeing 0.98 MJD 57446.346 object DECaLS_6843_z c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N11 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N11 r exptime 46.0 propid 2014B-0404 seeing 0.92 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N11 z exptime 213.0 propid 2014B-0404 seeing 1.19 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N11 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0 52449_ooi_g_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1921154; please check the logfile. Finished galaxy PGC1921154 in 0.097 minutes. Started working on galaxy PGC1921154 at Sun Feb 9 08:14:05 2020 RA, Dec=215.973249, 30.860960: run=decam (1 north CCDs, 16 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-09T08:14:06.598218 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.97324899999998', '30.8609602', '--width', '345', '--height', '345', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 215.97324899999998 30.8609602 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 28076 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 28077 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 28078 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 28079 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 28080 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 28081 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 28082 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 28083 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 28084 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 28085 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 28086 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 28087 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 28088 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 28089 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 28090 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 28091 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 28092 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 28093 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB Starting process 28094 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB Starting process 28095 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07214 MB Starting process 28096 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07214 MB Starting process 28097 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 28098 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 28099 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB Starting process 28100 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 28101 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB Starting process 28102 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 28103 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 28104 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 28105 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 28106 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Starting process 28107 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-09T08:14:07.848955 Keeping 16 CCDs: c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N19 r exptime 64.0 propid 2014B-0404 seeing 1.22 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N19 g exptime 147.0 propid 2014B-0404 seeing 1.22 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N19 z exptime 131.0 propid 2014B-0404 seeing 1.07 MJD 57446.346 object DECaLS_6843_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N18 g exptime 175.0 propid 2014B-0404 seeing 1.65 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N18 r exptime 125.0 propid 2014B-0404 seeing 1.53 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N18 z exptime 133.0 propid 2014B-0404 seeing 0.98 MJD 57444.339 object DECaLS_22715_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N12 g exptime 175.0 propid 2014B-0404 seeing 0.96 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N12 r exptime 125.0 propid 2014B-0404 seeing 1.13 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N12 z exptime 133.0 propid 2014B-0404 seeing 0.96 MJD 57444.339 object DECaLS_22715_z c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N18 r exptime 64.0 propid 2014B-0404 seeing 1.21 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N18 g exptime 147.0 propid 2014B-0404 seeing 1.21 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N18 z exptime 131.0 propid 2014B-0404 seeing 0.98 MJD 57446.346 object DECaLS_6843_z c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N11 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N11 r exptime 46.0 propid 2014B-0404 seeing 0.92 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N11 z exptime 213.0 propid 2014B-0404 seeing 1.19 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N11 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0 52449_ooi_g_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1921154; please check the logfile. Finished galaxy PGC1921154 in 0.056 minutes. Started working on galaxy PGC1921154 at Sun Feb 9 09:45:33 2020 RA, Dec=215.973249, 30.860960: run=decam (1 north CCDs, 16 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-09T09:45:34.980884 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.97324899999998', '30.8609602', '--width', '345', '--height', '345', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 215.97324899999998 30.8609602 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 33592 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071176 MB Starting process 33593 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071184 MB Starting process 33594 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071184 MB Starting process 33595 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071184 MB Starting process 33596 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071184 MB Starting process 33597 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071184 MB Starting process 33598 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071188 MB Starting process 33599 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071188 MB Starting process 33600 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071188 MB Starting process 33601 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071196 MB Starting process 33602 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071196 MB Starting process 33603 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.0712 MB Starting process 33605 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33606 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33607 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33608 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33609 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33610 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.0715 MB Starting process 33611 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33612 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33613 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071204 MB Starting process 33614 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071208 MB Starting process 33615 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071208 MB Starting process 33616 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071212 MB Starting process 33617 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071212 MB Starting process 33618 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071216 MB Starting process 33619 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07122 MB Starting process 33620 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07122 MB Starting process 33621 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071224 MB Starting process 33622 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071224 MB Starting process 33623 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071224 MB Starting process 33624 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071224 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-09T09:45:36.241325 Keeping 16 CCDs: c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N19 r exptime 64.0 propid 2014B-0404 seeing 1.22 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N19 g exptime 147.0 propid 2014B-0404 seeing 1.22 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N19 z exptime 131.0 propid 2014B-0404 seeing 1.07 MJD 57446.346 object DECaLS_6843_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N18 g exptime 175.0 propid 2014B-0404 seeing 1.65 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N18 r exptime 125.0 propid 2014B-0404 seeing 1.53 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N18 z exptime 133.0 propid 2014B-0404 seeing 0.98 MJD 57444.339 object DECaLS_22715_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N12 g exptime 175.0 propid 2014B-0404 seeing 0.96 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N12 r exptime 125.0 propid 2014B-0404 seeing 1.13 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N12 z exptime 133.0 propid 2014B-0404 seeing 0.96 MJD 57444.339 object DECaLS_22715_z c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N18 r exptime 64.0 propid 2014B-0404 seeing 1.21 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N18 g exptime 147.0 propid 2014B-0404 seeing 1.21 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N18 z exptime 131.0 propid 2014B-0404 seeing 0.98 MJD 57446.346 object DECaLS_6843_z c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N11 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N11 r exptime 46.0 propid 2014B-0404 seeing 0.92 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N11 z exptime 213.0 propid 2014B-0404 seeing 1.19 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N11 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170328/c4d_170329_0 70305_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1921154; please check the logfile. Finished galaxy PGC1921154 in 0.084 minutes. Started working on galaxy PGC1921154 at Mon Feb 10 06:10:15 2020 RA, Dec=215.973249, 30.860960: run=decam (1 north CCDs, 16 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-10T06:10:16.899429 legacypipe git version: dr9e-pre1-9-g4a51be93 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '215.97324899999998', '30.8609602', '--width', '345', '--height', '345', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 215.97324899999998 30.8609602 --width 345 --height 345 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/215/PGC1921154/PGC1921154-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 215.97324899999998 30.8609602 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 10074 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071156 MB Starting process 10075 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071156 MB Starting process 10076 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071156 MB Starting process 10077 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071156 MB Starting process 10078 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071156 MB Starting process 10079 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071156 MB Starting process 10080 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07116 MB Starting process 10081 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07116 MB Starting process 10082 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071164 MB Starting process 10083 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071168 MB Starting process 10084 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071172 MB Starting process 10085 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071172 MB Starting process 10086 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071176 MB Starting process 10087 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071176 MB Starting process 10088 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071176 MB Starting process 10089 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071176 MB Starting process 10090 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 10091 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071472 MB Starting process 10092 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 10093 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 10094 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 10095 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 10096 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 10097 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071184 MB Starting process 10098 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071188 MB Starting process 10099 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071188 MB Starting process 10100 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071192 MB Starting process 10101 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071196 MB Starting process 10102 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071196 MB Starting process 10103 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071196 MB Starting process 10104 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071196 MB Starting process 10105 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.0712 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-10T06:10:19.670952 Keeping 16 CCDs: c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N19 r exptime 64.0 propid 2014B-0404 seeing 1.22 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N19 g exptime 147.0 propid 2014B-0404 seeing 1.22 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N19 z exptime 131.0 propid 2014B-0404 seeing 1.07 MJD 57446.346 object DECaLS_6843_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N18 g exptime 175.0 propid 2014B-0404 seeing 1.65 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N18 r exptime 125.0 propid 2014B-0404 seeing 1.53 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N18 z exptime 133.0 propid 2014B-0404 seeing 0.98 MJD 57444.339 object DECaLS_22715_z c4d_170522_025630_ooi_g_ls9.fits.fz c4d_170522_025630_ooi_g_ls9-N12 g exptime 175.0 propid 2014B-0404 seeing 0.96 MJD 57895.120 object DECaLS_22715_g c4d_170521_031658_ooi_r_ls9.fits.fz c4d_170521_031658_ooi_r_ls9-N12 r exptime 125.0 propid 2014B-0404 seeing 1.13 MJD 57894.135 object DECaLS_22715_r c4d_160226_081124_ooi_z_ls9.fits.fz c4d_160226_081124_ooi_z_ls9-N12 z exptime 133.0 propid 2014B-0404 seeing 0.96 MJD 57444.339 object DECaLS_22715_z c4d_170329_070305_ooi_r_ls9.fits.fz c4d_170329_070305_ooi_r_ls9-N18 r exptime 64.0 propid 2014B-0404 seeing 1.21 MJD 57841.292 object DECaLS_6843_r c4d_170422_052449_ooi_g_ls9.fits.fz c4d_170422_052449_ooi_g_ls9-N18 g exptime 147.0 propid 2014B-0404 seeing 1.21 MJD 57865.223 object DECaLS_6843_g c4d_160228_082044_ooi_z_ls9.fits.fz c4d_160228_082044_ooi_z_ls9-N18 z exptime 131.0 propid 2014B-0404 seeing 0.98 MJD 57446.346 object DECaLS_6843_z c4d_180209_085020_ooi_z_ls9.fits.fz c4d_180209_085020_ooi_z_ls9-N11 z exptime 111.0 propid 2014B-0404 seeing 0.84 MJD 58158.368 object DECaLS_38587_z c4d_180222_085753_ooi_r_ls9.fits.fz c4d_180222_085753_ooi_r_ls9-N11 r exptime 46.0 propid 2014B-0404 seeing 0.92 MJD 58171.374 object DECaLS_38587_r c4d_170421_061044_ooi_z_ls9.fits.fz c4d_170421_061044_ooi_z_ls9-N11 z exptime 213.0 propid 2014B-0404 seeing 1.19 MJD 57864.254 object DECaLS_38587_z c4d_180221_085437_ooi_g_ls9.fits.fz c4d_180221_085437_ooi_g_ls9-N11 g exptime 161.0 propid 2014B-0404 seeing 1.31 MJD 58170.371 object DECaLS_38587_g The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95130c6-z-20170421T060621_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T081743_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9527115-z-20180209T085020_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T080822_oi[N12,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2/CP20170421/c4d_170422_0 52449_ooi_g_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1921154; please check the logfile. Finished galaxy PGC1921154 in 0.157 minutes.