Started working on galaxy PGC3097828 at Sat Feb 8 12:47:26 2020 python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.291706 68.7649669 --width 619 --height 619 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-08T12:47:26.789882 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '146.291706', '68.7649669', '--width', '619', '--height', '619', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.291706 68.7649669 --width 619 --height 619 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 146.291706 68.7649669 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 16942 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB Starting process 16943 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 16944 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 16945 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 16946 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB Starting process 16947 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 16948 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 16949 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-08T12:47:28.094499 Keeping 16 CCDs: ksb_180410_063935_ooi_g_ls9.fits.fz ksb_180410_063935_ooi_g_ls9-CCD3 g exptime 224.25 propid BASS seeing 1.48 MJD 58218.277 object 44302 ksb_190131_115514_ooi_r_ls9.fits.fz ksb_190131_115514_ooi_r_ls9-CCD3 r exptime 200.6 propid NOTBASS seeing 1.59 MJD 58514.497 object ksb_190131_114353_ooi_r_ls9.fits.fz ksb_190131_114353_ooi_r_ls9-CCD4 r exptime 139.363 propid NOTBASS seeing 1.48 MJD 58514.489 object ksb_180411_072612_ooi_g_ls9.fits.fz ksb_180411_072612_ooi_g_ls9-CCD4 g exptime 215.859 propid BASS seeing 1.54 MJD 58219.310 object 45082 ksb_190131_113812_ooi_r_ls9.fits.fz ksb_190131_113812_ooi_r_ls9-CCD1 r exptime 116.707 propid NOTBASS seeing 1.59 MJD 58514.485 object ksb_180411_071357_ooi_g_ls9.fits.fz ksb_180411_071357_ooi_g_ls9-CCD1 g exptime 219.186 propid BASS seeing 1.69 MJD 58219.301 object 44292 ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD2 g exptime 124.63 propid NOTBASS seeing 1.48 MJD 58495.427 object ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD2 r exptime 163.914 propid NOTBASS seeing 1.35 MJD 58511.457 object ksb_190128_063416_ooi_g_ls9.fits.fz ksb_190128_063416_ooi_g_ls9-CCD3 g exptime 110.595 propid NOTBASS seeing 1.49 MJD 58511.274 object ksb_190209_094735_ooi_r_ls9.fits.fz ksb_190209_094735_ooi_r_ls9-CCD3 r exptime 129.274 propid NOTBASS seeing 1.56 MJD 58523.408 object ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD1 g exptime 118.43 propid NOTBASS seeing 1.46 MJD 58511.267 object ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD1 r exptime 140.178 propid NOTBASS seeing 1.67 MJD 58523.391 object k4m_160125_091939_ooi_zd_ls9.fits.fz k4m_160125_091939_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 0.99 MJD 57412.389 object MzLS_710852_z k4m_160125_092739_ooi_zd_ls9.fits.fz k4m_160125_092739_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016B-0453 seeing 1.13 MJD 57412.394 object MzLS_710853_z k4m_160320_060607_ooi_zd_ls9.fits.fz k4m_160320_060607_ooi_zd_ls9-CCD2 z exptime 94.104 propid 2016A-0453 seeing 0.88 MJD 57467.254 object MzLS_710390_z k4m_171028_104055_ooi_zd_ls9.fits.fz k4m_171028_104055_ooi_zd_ls9-CCD1 z exptime 135.306 propid 2016A-0453 seeing 1.02 MJD 58054.445 object MzLS_711310_z Faked up EXPNUM 84950057 Faked up EXPNUM 84950057 Faked up EXPNUM 84950057 90prime get_fwhm: self.fwhm = 3.27 WARNING: EXPNUM None!=160125092739 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092739_ooi_zd_ls9.fits.fz header Faked up EXPNUM 85110243 Faked up EXPNUM 85110243 Faked up EXPNUM 85110243 90prime get_fwhm: self.fwhm = 2.97 WARNING: EXPNUM None!=160125091939 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_091939_ooi_zd_ls9.fits.fz header Faked up EXPNUM 85140225 Faked up EXPNUM 85140225 Faked up EXPNUM 85140225 90prime get_fwhm: self.fwhm = 3.51 Faked up EXPNUM 85110144 Faked up EXPNUM 85110144 Faked up EXPNUM 85110144 90prime get_fwhm: self.fwhm = 3.29 Faked up EXPNUM 85140227 Faked up EXPNUM 85140227 Faked up EXPNUM 85140227 90prime get_fwhm: self.fwhm = 3.27 Faked up EXPNUM 85110140 Faked up EXPNUM 85110140 Faked up EXPNUM 85110140 90prime get_fwhm: self.fwhm = 3.23 Faked up EXPNUM 82190101 Faked up EXPNUM 82190101 Faked up EXPNUM 82190101 90prime get_fwhm: self.fwhm = 3.39 Faked up EXPNUM 85230154 Faked up EXPNUM 85230154 Faked up EXPNUM 85230154 90prime get_fwhm: self.fwhm = 3.69 Faked up EXPNUM 85140230 Faked up EXPNUM 85140230 Faked up EXPNUM 85140230 90prime get_fwhm: self.fwhm = 3.51 mosaic get_fwhm: self.fwhm = 3.39 Faked up EXPNUM 82190098 Faked up EXPNUM 82190098 Faked up EXPNUM 82190098 90prime get_fwhm: self.fwhm = 3.73 Faked up EXPNUM 85230163 Faked up EXPNUM 85230163 Faked up EXPNUM 85230163 90prime get_fwhm: self.fwhm = 3.45 Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value AssertionError Something went wrong; please check the logfile. ERROR: galaxy PGC3097828; please check the logfile. Finished galaxy PGC3097828 in 0.072 minutes. Started working on galaxy PGC3097828 at Sun Feb 9 10:13:52 2020 python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.291706 68.7649669 --width 619 --height 619 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-09T10:13:52.824803 legacypipe git version: dr9e-pre1-7-g54c974a3 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '146.291706', '68.7649669', '--width', '619', '--height', '619', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.291706 68.7649669 --width 619 --height 619 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 146.291706 68.7649669 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 42435 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 42436 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 42437 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 42438 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 42439 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB Starting process 42440 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB Starting process 42441 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB Starting process 42442 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 42443 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 42444 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB Starting process 42445 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB Starting process 42446 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB Starting process 42447 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB Starting process 42448 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB Starting process 42449 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB Starting process 42450 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071196 MB Starting process 42451 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0712 MB Starting process 42452 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072212 MB Starting process 42453 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072212 MB Starting process 42454 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07192 MB Starting process 42455 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071916 MB Starting process 42456 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0712 MB Starting process 42457 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 42458 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 42459 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 42460 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 42461 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 42462 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB Starting process 42463 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB Starting process 42464 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB Starting process 42465 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071556 MB Starting process 42466 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071492 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-09T10:13:54.101738 Keeping 16 CCDs: ksb_180410_063935_ooi_g_ls9.fits.fz ksb_180410_063935_ooi_g_ls9-CCD3 g exptime 224.25 propid BASS seeing 1.48 MJD 58218.277 object 44302 ksb_190131_115514_ooi_r_ls9.fits.fz ksb_190131_115514_ooi_r_ls9-CCD3 r exptime 200.6 propid NOTBASS seeing 1.59 MJD 58514.497 object ksb_190131_114353_ooi_r_ls9.fits.fz ksb_190131_114353_ooi_r_ls9-CCD4 r exptime 139.363 propid NOTBASS seeing 1.48 MJD 58514.489 object ksb_180411_072612_ooi_g_ls9.fits.fz ksb_180411_072612_ooi_g_ls9-CCD4 g exptime 215.859 propid BASS seeing 1.54 MJD 58219.310 object 45082 ksb_190131_113812_ooi_r_ls9.fits.fz ksb_190131_113812_ooi_r_ls9-CCD1 r exptime 116.707 propid NOTBASS seeing 1.59 MJD 58514.485 object ksb_180411_071357_ooi_g_ls9.fits.fz ksb_180411_071357_ooi_g_ls9-CCD1 g exptime 219.186 propid BASS seeing 1.69 MJD 58219.301 object 44292 ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD2 g exptime 124.63 propid NOTBASS seeing 1.48 MJD 58495.427 object ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD2 r exptime 163.914 propid NOTBASS seeing 1.35 MJD 58511.457 object ksb_190128_063416_ooi_g_ls9.fits.fz ksb_190128_063416_ooi_g_ls9-CCD3 g exptime 110.595 propid NOTBASS seeing 1.49 MJD 58511.274 object ksb_190209_094735_ooi_r_ls9.fits.fz ksb_190209_094735_ooi_r_ls9-CCD3 r exptime 129.274 propid NOTBASS seeing 1.56 MJD 58523.408 object ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD1 g exptime 118.43 propid NOTBASS seeing 1.46 MJD 58511.267 object ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD1 r exptime 140.178 propid NOTBASS seeing 1.67 MJD 58523.391 object k4m_160125_091939_ooi_zd_ls9.fits.fz k4m_160125_091939_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 0.99 MJD 57412.389 object MzLS_710852_z k4m_160125_092739_ooi_zd_ls9.fits.fz k4m_160125_092739_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016B-0453 seeing 1.13 MJD 57412.394 object MzLS_710853_z k4m_160320_060607_ooi_zd_ls9.fits.fz k4m_160320_060607_ooi_zd_ls9-CCD2 z exptime 94.104 propid 2016A-0453 seeing 0.88 MJD 57467.254 object MzLS_710390_z k4m_171028_104055_ooi_zd_ls9.fits.fz k4m_171028_104055_ooi_zd_ls9-CCD1 z exptime 135.306 propid 2016A-0453 seeing 1.02 MJD 58054.445 object MzLS_711310_z WARNING: EXPNUM None!=160125091939 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_091939_ooi_zd_ls9.fits.fz header WARNING: EXPNUM None!=160125092739 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092739_ooi_zd_ls9.fits.fz header Faked up EXPNUM 85140225 Faked up EXPNUM 85140225 Faked up EXPNUM 85140225 90prime get_fwhm: self.fwhm = 3.51 Faked up EXPNUM 82190098 Faked up EXPNUM 82190098 Faked up EXPNUM 82190098 90prime get_fwhm: self.fwhm = 3.73 Faked up EXPNUM 85230154 Faked up EXPNUM 85230154 Faked up EXPNUM 85230154 90prime get_fwhm: self.fwhm = 3.69 Faked up EXPNUM 82190101 Faked up EXPNUM 82190101 Faked up EXPNUM 82190101 90prime get_fwhm: self.fwhm = 3.39 Faked up EXPNUM 85110140 Faked up EXPNUM 85110140 Faked up EXPNUM 85110140 90prime get_fwhm: self.fwhm = 3.23 Faked up EXPNUM 85140227 Faked up EXPNUM 85140227 Faked up EXPNUM 85140227 90prime get_fwhm: self.fwhm = 3.27 Faked up EXPNUM 84950057 Faked up EXPNUM 84950057 Faked up EXPNUM 84950057 90prime get_fwhm: self.fwhm = 3.27 Faked up EXPNUM 85110243 Faked up EXPNUM 85110243 Faked up EXPNUM 85110243 90prime get_fwhm: self.fwhm = 2.97 Faked up EXPNUM 85230163 Faked up EXPNUM 85230163 Faked up EXPNUM 85230163 90prime get_fwhm: self.fwhm = 3.45 Faked up EXPNUM 85110144 Faked up EXPNUM 85110144 Faked up EXPNUM 85110144 90prime get_fwhm: self.fwhm = 3.29 Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value AssertionError Something went wrong; please check the logfile. ERROR: galaxy PGC3097828; please check the logfile. Finished galaxy PGC3097828 in 0.061 minutes. Started working on galaxy PGC3097828 at Mon Feb 10 06:16:22 2020 python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.291706 68.7649669 --width 619 --height 619 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p --unwise-coadds runbrick.py starting at 2020-02-10T06:16:22.501825 legacypipe git version: dr9e-pre1-9-g4a51be93 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '146.291706', '68.7649669', '--width', '619', '--height', '619', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p', '--unwise-coadds'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.291706 68.7649669 --width 619 --height 619 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/PGC3097828/PGC3097828-runbrick-%%(stage)s.p --unwise-coadds WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky Parsed RA,Dec 146.291706 68.7649669 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data] Starting process 14523 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB Starting process 14524 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 14525 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 14526 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 14527 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB Starting process 14528 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB Starting process 14529 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB Starting process 14530 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB Starting process 14531 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB Starting process 14532 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB Starting process 14533 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB Starting process 14534 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 14535 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 14536 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 14537 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB Starting process 14538 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 14539 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 14540 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072168 MB Starting process 14541 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072172 MB Starting process 14542 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072172 MB Starting process 14543 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072168 MB Starting process 14544 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB Starting process 14545 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 14546 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 14547 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB Starting process 14548 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 14549 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071516 MB Starting process 14550 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB Starting process 14551 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB Starting process 14552 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB Starting process 14553 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB Starting process 14554 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage largegalaxies Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-02-10T06:16:23.795720 Keeping 16 CCDs: ksb_180410_063935_ooi_g_ls9.fits.fz ksb_180410_063935_ooi_g_ls9-CCD3 g exptime 224.25 propid BASS seeing 1.48 MJD 58218.277 object 44302 ksb_190131_115514_ooi_r_ls9.fits.fz ksb_190131_115514_ooi_r_ls9-CCD3 r exptime 200.6 propid NOTBASS seeing 1.59 MJD 58514.497 object ksb_190131_114353_ooi_r_ls9.fits.fz ksb_190131_114353_ooi_r_ls9-CCD4 r exptime 139.363 propid NOTBASS seeing 1.48 MJD 58514.489 object ksb_180411_072612_ooi_g_ls9.fits.fz ksb_180411_072612_ooi_g_ls9-CCD4 g exptime 215.859 propid BASS seeing 1.54 MJD 58219.310 object 45082 ksb_190131_113812_ooi_r_ls9.fits.fz ksb_190131_113812_ooi_r_ls9-CCD1 r exptime 116.707 propid NOTBASS seeing 1.59 MJD 58514.485 object ksb_180411_071357_ooi_g_ls9.fits.fz ksb_180411_071357_ooi_g_ls9-CCD1 g exptime 219.186 propid BASS seeing 1.69 MJD 58219.301 object 44292 ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD2 g exptime 124.63 propid NOTBASS seeing 1.48 MJD 58495.427 object ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD2 r exptime 163.914 propid NOTBASS seeing 1.35 MJD 58511.457 object ksb_190128_063416_ooi_g_ls9.fits.fz ksb_190128_063416_ooi_g_ls9-CCD3 g exptime 110.595 propid NOTBASS seeing 1.49 MJD 58511.274 object ksb_190209_094735_ooi_r_ls9.fits.fz ksb_190209_094735_ooi_r_ls9-CCD3 r exptime 129.274 propid NOTBASS seeing 1.56 MJD 58523.408 object ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD1 g exptime 118.43 propid NOTBASS seeing 1.46 MJD 58511.267 object ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD1 r exptime 140.178 propid NOTBASS seeing 1.67 MJD 58523.391 object k4m_160125_091939_ooi_zd_ls9.fits.fz k4m_160125_091939_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 0.99 MJD 57412.389 object MzLS_710852_z k4m_160125_092739_ooi_zd_ls9.fits.fz k4m_160125_092739_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016B-0453 seeing 1.13 MJD 57412.394 object MzLS_710853_z k4m_160320_060607_ooi_zd_ls9.fits.fz k4m_160320_060607_ooi_zd_ls9-CCD2 z exptime 94.104 propid 2016A-0453 seeing 0.88 MJD 57467.254 object MzLS_710390_z k4m_171028_104055_ooi_zd_ls9.fits.fz k4m_171028_104055_ooi_zd_ls9-CCD1 z exptime 135.306 propid 2016A-0453 seeing 1.02 MJD 58054.445 object MzLS_711310_z WARNING: EXPNUM None!=160125091939 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_091939_ooi_zd_ls9.fits.fz header WARNING: EXPNUM None!=160125092739 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092739_ooi_zd_ls9.fits.fz header Faked up EXPNUM 85110140 Faked up EXPNUM 85110140 Faked up EXPNUM 85110140 90prime get_fwhm: self.fwhm = 3.23 Faked up EXPNUM 85140227 Faked up EXPNUM 85140227 Faked up EXPNUM 85140227 90prime get_fwhm: self.fwhm = 3.27 Faked up EXPNUM 85230154 Faked up EXPNUM 85230154 Faked up EXPNUM 85230154 90prime get_fwhm: self.fwhm = 3.69 Faked up EXPNUM 85140225 Faked up EXPNUM 85140225 Faked up EXPNUM 85140225 90prime get_fwhm: self.fwhm = 3.51 Faked up EXPNUM 82190098 Faked up EXPNUM 82190098 Faked up EXPNUM 82190098 90prime get_fwhm: self.fwhm = 3.73 Faked up EXPNUM 84950057 Faked up EXPNUM 84950057 Faked up EXPNUM 84950057 90prime get_fwhm: self.fwhm = 3.27 Faked up EXPNUM 82190101 Faked up EXPNUM 82190101 Faked up EXPNUM 82190101 90prime get_fwhm: self.fwhm = 3.39 Faked up EXPNUM 85110243 Faked up EXPNUM 85110243 Faked up EXPNUM 85110243 90prime get_fwhm: self.fwhm = 2.97 Faked up EXPNUM 85230163 Faked up EXPNUM 85230163 Faked up EXPNUM 85230163 90prime get_fwhm: self.fwhm = 3.45 Faked up EXPNUM 85110144 Faked up EXPNUM 85110144 Faked up EXPNUM 85110144 90prime get_fwhm: self.fwhm = 3.29 Faked up EXPNUM 82180085 Faked up EXPNUM 82180085 Faked up EXPNUM 82180085 90prime get_fwhm: self.fwhm = 3.27 Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) [Previous line repeated 6 more times] File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims tims = list(mp.map(read_one_tim, args)) File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map return self.pool.map(f, args, cs) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value AssertionError Something went wrong; please check the logfile. ERROR: galaxy PGC3097828; please check the logfile. Finished galaxy PGC3097828 in 0.066 minutes.