Started working on galaxy PGC1993316 at Sat Feb 8 07:59:35 2020 RA, Dec=216.435818, 32.360914: run=decam (11 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T07:59:36.575438 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4358175', '32.3609144', '--width', '379', '--height', '379', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4358175 32.3609144 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 14296 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071176 MB Starting process 14297 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 14298 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 14299 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 14300 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 14302 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 14301 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.07118 MB Starting process 14303 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 731 MB, VmSize: 731 MB, VmRSS: 70 MB, VmData: 143 MB, maxrss: 0.071184 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T07:59:37.929472 Keeping 14 CCDs: c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N15 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.375 object DECaLS_38604_z c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N15 r exptime 166.0 propid 2014B-0404 seeing 1.19 MJD 58251.135 object DECaLS_38604_r c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N15 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N9 r exptime 49.0 propid 2014B-0404 seeing 1.56 MJD 58549.301 object DECaLS_38604_r c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N9 r exptime 166.0 propid 2014B-0404 seeing 1.89 MJD 58251.135 object DECaLS_38604_r c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N9 z exptime 140.0 propid 2014B-0404 seeing 0.97 MJD 57444.375 object DECaLS_38604_z c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N9 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_180213_090013_ooi_r_ls9.fits.fz c4d_180213_090013_ooi_r_ls9-N17 r exptime 52.0 propid 2014B-0404 seeing 0.96 MJD 58162.375 object DECaLS_6860_r c4d_170422_052010_ooi_g_ls9.fits.fz c4d_170422_052010_ooi_g_ls9-N17 g exptime 125.0 propid 2014B-0404 seeing 1.30 MJD 57865.220 object DECaLS_6860_g c4d_160228_083134_ooi_z_ls9.fits.fz c4d_160228_083134_ooi_z_ls9-N17 z exptime 147.0 propid 2014B-0404 seeing 1.00 MJD 57446.353 object DECaLS_6860_z c4d_170717_235713_ooi_g_ls9.fits.fz c4d_170717_235713_ooi_g_ls9-N16 g exptime 169.0 propid 2014B-0404 seeing 1.04 MJD 57951.996 object DECaLS_22732_g c4d_170717_235355_ooi_r_ls9.fits.fz c4d_170717_235355_ooi_r_ls9-N16 r exptime 96.0 propid 2014B-0404 seeing 1.39 MJD 57951.994 object DECaLS_22732_r c4d_160226_081409_ooi_z_ls9.fits.fz c4d_160226_081409_ooi_z_ls9-N16 z exptime 135.0 propid 2014B-0404 seeing 1.02 MJD 57444.341 object DECaLS_22732_z c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N15 r exptime 49.0 propid 2014B-0404 seeing 1.50 MJD 58549.301 object DECaLS_38604_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180512/c4d_180513_ 031444_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1993316; please check the logfile. Finished galaxy PGC1993316 in 0.233 minutes. Started working on galaxy PGC1993316 at Sat Feb 8 08:50:51 2020 RA, Dec=216.435818, 32.360914: run=decam (11 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T08:50:52.527881 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4358175', '32.3609144', '--width', '379', '--height', '379', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4358175 32.3609144 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 53029 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 53030 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 53031 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 53032 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 53033 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 53034 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 53035 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 53036 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T08:50:53.907376 Keeping 14 CCDs: c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N15 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.375 object DECaLS_38604_z c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N15 r exptime 166.0 propid 2014B-0404 seeing 1.19 MJD 58251.135 object DECaLS_38604_r c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N15 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N9 r exptime 49.0 propid 2014B-0404 seeing 1.56 MJD 58549.301 object DECaLS_38604_r c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N9 r exptime 166.0 propid 2014B-0404 seeing 1.89 MJD 58251.135 object DECaLS_38604_r c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N9 z exptime 140.0 propid 2014B-0404 seeing 0.97 MJD 57444.375 object DECaLS_38604_z c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N9 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_180213_090013_ooi_r_ls9.fits.fz c4d_180213_090013_ooi_r_ls9-N17 r exptime 52.0 propid 2014B-0404 seeing 0.96 MJD 58162.375 object DECaLS_6860_r c4d_170422_052010_ooi_g_ls9.fits.fz c4d_170422_052010_ooi_g_ls9-N17 g exptime 125.0 propid 2014B-0404 seeing 1.30 MJD 57865.220 object DECaLS_6860_g c4d_160228_083134_ooi_z_ls9.fits.fz c4d_160228_083134_ooi_z_ls9-N17 z exptime 147.0 propid 2014B-0404 seeing 1.00 MJD 57446.353 object DECaLS_6860_z c4d_170717_235713_ooi_g_ls9.fits.fz c4d_170717_235713_ooi_g_ls9-N16 g exptime 169.0 propid 2014B-0404 seeing 1.04 MJD 57951.996 object DECaLS_22732_g c4d_170717_235355_ooi_r_ls9.fits.fz c4d_170717_235355_ooi_r_ls9-N16 r exptime 96.0 propid 2014B-0404 seeing 1.39 MJD 57951.994 object DECaLS_22732_r c4d_160226_081409_ooi_z_ls9.fits.fz c4d_160226_081409_ooi_z_ls9-N16 z exptime 135.0 propid 2014B-0404 seeing 1.02 MJD 57444.341 object DECaLS_22732_z c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N15 r exptime 49.0 propid 2014B-0404 seeing 1.50 MJD 58549.301 object DECaLS_38604_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180220/c4d_180221_ 085747_ooi_g_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1993316; please check the logfile. Finished galaxy PGC1993316 in 0.116 minutes. Started working on galaxy PGC1993316 at Sat Feb 8 12:47:06 2020 RA, Dec=216.435818, 32.360914: run=decam (11 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T12:47:07.569177 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4358175', '32.3609144', '--width', '379', '--height', '379', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4358175 32.3609144 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 10936 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 10937 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 10938 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 10939 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 10940 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 10941 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 10942 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 10943 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T12:47:08.987831 Keeping 14 CCDs: c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N15 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.375 object DECaLS_38604_z c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N15 r exptime 166.0 propid 2014B-0404 seeing 1.19 MJD 58251.135 object DECaLS_38604_r c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N15 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N9 r exptime 49.0 propid 2014B-0404 seeing 1.56 MJD 58549.301 object DECaLS_38604_r c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N9 r exptime 166.0 propid 2014B-0404 seeing 1.89 MJD 58251.135 object DECaLS_38604_r c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N9 z exptime 140.0 propid 2014B-0404 seeing 0.97 MJD 57444.375 object DECaLS_38604_z c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N9 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_180213_090013_ooi_r_ls9.fits.fz c4d_180213_090013_ooi_r_ls9-N17 r exptime 52.0 propid 2014B-0404 seeing 0.96 MJD 58162.375 object DECaLS_6860_r c4d_170422_052010_ooi_g_ls9.fits.fz c4d_170422_052010_ooi_g_ls9-N17 g exptime 125.0 propid 2014B-0404 seeing 1.30 MJD 57865.220 object DECaLS_6860_g c4d_160228_083134_ooi_z_ls9.fits.fz c4d_160228_083134_ooi_z_ls9-N17 z exptime 147.0 propid 2014B-0404 seeing 1.00 MJD 57446.353 object DECaLS_6860_z c4d_170717_235713_ooi_g_ls9.fits.fz c4d_170717_235713_ooi_g_ls9-N16 g exptime 169.0 propid 2014B-0404 seeing 1.04 MJD 57951.996 object DECaLS_22732_g c4d_170717_235355_ooi_r_ls9.fits.fz c4d_170717_235355_ooi_r_ls9-N16 r exptime 96.0 propid 2014B-0404 seeing 1.39 MJD 57951.994 object DECaLS_22732_r c4d_160226_081409_ooi_z_ls9.fits.fz c4d_160226_081409_ooi_z_ls9-N16 z exptime 135.0 propid 2014B-0404 seeing 1.02 MJD 57444.341 object DECaLS_22732_z c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N15 r exptime 49.0 propid 2014B-0404 seeing 1.50 MJD 58549.301 object DECaLS_38604_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180512/c4d_180513_ 031444_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1993316; please check the logfile. Finished galaxy PGC1993316 in 0.092 minutes. Started working on galaxy PGC1993316 at Sun Feb 9 09:49:54 2020 RA, Dec=216.435818, 32.360914: run=decam (11 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-09T09:49:55.145788 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4358175', '32.3609144', '--width', '379', '--height', '379', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4358175 32.3609144 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 35259 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35260 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35261 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35262 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35263 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 35264 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 35265 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 35266 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 35267 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 35268 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 35269 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB Starting process 35270 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 35271 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 35272 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 35273 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB Starting process 35274 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0712 MB Starting process 35275 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0712 MB Starting process 35276 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072212 MB Starting process 35277 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072212 MB Starting process 35278 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071924 MB Starting process 35279 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071924 MB Starting process 35280 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0712 MB Starting process 35281 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 35282 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07156 MB Starting process 35283 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07156 MB Starting process 35284 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 35285 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 35286 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 35287 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 35288 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 35289 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 35290 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-09T09:49:56.467585 Keeping 14 CCDs: c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N15 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.375 object DECaLS_38604_z c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N15 r exptime 166.0 propid 2014B-0404 seeing 1.19 MJD 58251.135 object DECaLS_38604_r c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N15 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N9 r exptime 49.0 propid 2014B-0404 seeing 1.56 MJD 58549.301 object DECaLS_38604_r c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N9 r exptime 166.0 propid 2014B-0404 seeing 1.89 MJD 58251.135 object DECaLS_38604_r c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N9 z exptime 140.0 propid 2014B-0404 seeing 0.97 MJD 57444.375 object DECaLS_38604_z c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N9 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_180213_090013_ooi_r_ls9.fits.fz c4d_180213_090013_ooi_r_ls9-N17 r exptime 52.0 propid 2014B-0404 seeing 0.96 MJD 58162.375 object DECaLS_6860_r c4d_170422_052010_ooi_g_ls9.fits.fz c4d_170422_052010_ooi_g_ls9-N17 g exptime 125.0 propid 2014B-0404 seeing 1.30 MJD 57865.220 object DECaLS_6860_g c4d_160228_083134_ooi_z_ls9.fits.fz c4d_160228_083134_ooi_z_ls9-N17 z exptime 147.0 propid 2014B-0404 seeing 1.00 MJD 57446.353 object DECaLS_6860_z c4d_170717_235713_ooi_g_ls9.fits.fz c4d_170717_235713_ooi_g_ls9-N16 g exptime 169.0 propid 2014B-0404 seeing 1.04 MJD 57951.996 object DECaLS_22732_g c4d_170717_235355_ooi_r_ls9.fits.fz c4d_170717_235355_ooi_r_ls9-N16 r exptime 96.0 propid 2014B-0404 seeing 1.39 MJD 57951.994 object DECaLS_22732_r c4d_160226_081409_ooi_z_ls9.fits.fz c4d_160226_081409_ooi_z_ls9-N16 z exptime 135.0 propid 2014B-0404 seeing 1.02 MJD 57444.341 object DECaLS_22732_z c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N15 r exptime 49.0 propid 2014B-0404 seeing 1.50 MJD 58549.301 object DECaLS_38604_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180512/c4d_180513_ 031444_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1993316; please check the logfile. Finished galaxy PGC1993316 in 0.061 minutes. Started working on galaxy PGC1993316 at Mon Feb 10 06:12:02 2020 RA, Dec=216.435818, 32.360914: run=decam (11 north CCDs, 14 south CCDs). python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-10T06:12:02.780972 legacypipe git version: dr9e-pre1-9-g4a51be93 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.4358175', '32.3609144', '--width', '379', '--height', '379', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', 'decam', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.4358175 32.3609144 --width 379 --height 379 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run decam --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/216/PGC1993316/PGC1993316-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 216.4358175 32.3609144 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 11102 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 11103 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 11104 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 11105 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 11106 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 11107 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB Starting process 11108 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB Starting process 11109 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB Starting process 11110 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB Starting process 11111 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB Starting process 11112 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB Starting process 11113 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 11114 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 11115 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 11116 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 11117 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07116 MB Starting process 11118 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.07116 MB Starting process 11119 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072172 MB Starting process 11120 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072172 MB Starting process 11121 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07188 MB Starting process 11122 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07188 MB Starting process 11123 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 11124 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11125 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11126 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11127 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11128 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11129 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11130 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11131 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11132 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071452 MB Starting process 11133 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071456 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-10T06:12:04.021660 Keeping 14 CCDs: c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N15 z exptime 140.0 propid 2014B-0404 seeing 0.96 MJD 57444.375 object DECaLS_38604_z c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N15 r exptime 166.0 propid 2014B-0404 seeing 1.19 MJD 58251.135 object DECaLS_38604_r c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N15 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N9 r exptime 49.0 propid 2014B-0404 seeing 1.56 MJD 58549.301 object DECaLS_38604_r c4d_180513_031444_ooi_r_ls9.fits.fz c4d_180513_031444_ooi_r_ls9-N9 r exptime 166.0 propid 2014B-0404 seeing 1.89 MJD 58251.135 object DECaLS_38604_r c4d_160226_090239_ooi_z_ls9.fits.fz c4d_160226_090239_ooi_z_ls9-N9 z exptime 140.0 propid 2014B-0404 seeing 0.97 MJD 57444.375 object DECaLS_38604_z c4d_180221_085747_ooi_g_ls9.fits.fz c4d_180221_085747_ooi_g_ls9-N9 g exptime 173.0 propid 2014B-0404 seeing 1.35 MJD 58170.373 object DECaLS_38604_g c4d_180213_090013_ooi_r_ls9.fits.fz c4d_180213_090013_ooi_r_ls9-N17 r exptime 52.0 propid 2014B-0404 seeing 0.96 MJD 58162.375 object DECaLS_6860_r c4d_170422_052010_ooi_g_ls9.fits.fz c4d_170422_052010_ooi_g_ls9-N17 g exptime 125.0 propid 2014B-0404 seeing 1.30 MJD 57865.220 object DECaLS_6860_g c4d_160228_083134_ooi_z_ls9.fits.fz c4d_160228_083134_ooi_z_ls9-N17 z exptime 147.0 propid 2014B-0404 seeing 1.00 MJD 57446.353 object DECaLS_6860_z c4d_170717_235713_ooi_g_ls9.fits.fz c4d_170717_235713_ooi_g_ls9-N16 g exptime 169.0 propid 2014B-0404 seeing 1.04 MJD 57951.996 object DECaLS_22732_g c4d_170717_235355_ooi_r_ls9.fits.fz c4d_170717_235355_ooi_r_ls9-N16 r exptime 96.0 propid 2014B-0404 seeing 1.39 MJD 57951.994 object DECaLS_22732_r c4d_160226_081409_ooi_z_ls9.fits.fz c4d_160226_081409_ooi_z_ls9-N16 z exptime 135.0 propid 2014B-0404 seeing 1.02 MJD 57444.341 object DECaLS_22732_z c4d_190307_071335_ooi_r_ls9.fits.fz c4d_190307_071335_ooi_r_ls9-N15 r exptime 49.0 propid 2014B-0404 seeing 1.50 MJD 58549.301 object DECaLS_38604_r The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T081104_oi[N16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954f856-z-20160228T082816_oi[N17,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N9,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95508ff-z-20160226T085930_oi[N15,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value OSError: FITSIO status = 104: could not open the named file failed to find or open the following file: (ffopen) /global/cscratch1/sd/desimpp/dr9e/images/decam/CP/V4.8.2a/CP20180512/c4d_180513_ 031444_ooi_r_ls9.fits.fz Something went wrong; please check the logfile. ERROR: galaxy PGC1993316; please check the logfile. Finished galaxy PGC1993316 in 0.059 minutes.