Started working on galaxy PGC2362941 at Wed Aug 5 20:00:19 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.22127900000004 50.105148299999996 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T20:00:22.383297 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.22127900000004', '50.105148299999996', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.22127900000004 50.105148299999996 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 277.22127900000004 50.105148299999996 Starting process 46358 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 46359 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 46360 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 46361 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 46362 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059536 MB Starting process 46363 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB Starting process 46364 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB Starting process 46365 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 46366 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 46367 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 46368 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 46369 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 46371 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 46372 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 46370 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 46373 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T20:00:23.451906 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: k4m_160523_101309_ooi_zd_ls9-CCD3 z expnum 90951 exptime 84.104 propid 2016A-0453 seeing 0.97 MJD 57531.426 object MzLS_651131_z mosaic/CP/V4.3/CP20160522/k4m_160523_101309_ooi_zd_ls9.fits.fz k4m_160524_094908_ooi_zd_ls9-CCD1 z expnum 91253 exptime 80.104 propid 2016A-0453 seeing 1.05 MJD 57532.409 object MzLS_651132_z mosaic/CP/V4.3/CP20160523/k4m_160524_094908_ooi_zd_ls9.fits.fz k4m_160520_100205_ooi_zd_ls9-CCD3 z expnum 90521 exptime 80.103 propid 2016A-0453 seeing 0.91 MJD 57528.418 object MzLS_650328_z mosaic/CP/V4.3/CP20160519/k4m_160520_100205_ooi_zd_ls9.fits.fz k4m_170831_065611_ooi_zd_ls9-CCD4 z expnum 137469 exptime 250.101 propid 2016A-0453 seeing 1.38 MJD 57996.289 object MzLS_649520_z mosaic/CP/V4.3/CP20170830/k4m_170831_065611_ooi_zd_ls9.fits.fz ksb_160612_110533_ooi_r_ls9-CCD2 r expnum 75510139 exptime 99.5 propid BASS seeing 1.24 MJD 57551.462 object 27212 90prime/CP/V2.3/CP20160611/ksb_160612_110533_ooi_r_ls9.fits.fz ksb_160706_100058_ooi_g_ls9-CCD2 g expnum 75750119 exptime 199.0 propid BASS seeing 1.85 MJD 57575.417 object 27212 90prime/CP/V2.3/CP20160705/ksb_160706_100058_ooi_g_ls9.fits.fz ksb_160709_101405_ooi_g_ls9-CCD3 g expnum 75780122 exptime 177.2 propid BASS seeing 2.00 MJD 57578.426 object 27223 90prime/CP/V2.3/CP20160708/ksb_160709_101405_ooi_g_ls9.fits.fz ksb_180518_105622_ooi_r_ls9-CCD3 r expnum 82560213 exptime 86.915 propid NOTBASS seeing 1.26 MJD 58256.456 object 27223 90prime/CP/V2.3/CP20180517/ksb_180518_105622_ooi_r_ls9.fits.fz ksb_160611_110505_ooi_r_ls9-CCD1 r expnum 75500190 exptime 69.6 propid BASS seeing 1.20 MJD 57550.462 object 27211 90prime/CP/V2.3/CP20160610/ksb_160611_110505_ooi_r_ls9.fits.fz ksb_180515_104449_ooi_g_ls9-CCD1 g expnum 82530168 exptime 109.824 propid NOTBASS seeing 1.52 MJD 58253.448 object 27211 90prime/CP/V2.3/CP20180514/ksb_180515_104449_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 536 804 and amp range 0 2005 and image y 296 567 and amp range 0 2039 Range in image: x 0 269 , y 0 272 (with image size 269 x 272) dzp -0.011066288 -> scaling image by 0.989859339076806 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 90prime get_fwhm: self.fwhm = 3.35 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3163 3314 and amp range 2016 4031 and image y 1613 1765 and amp range 0 2047 Range in image: x 0 152 , y 0 153 (with image size 152 x 153) dzp -0.019046262 -> scaling image by 0.9826107140833663 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3292 3550 and amp range 2006 4053 and image y 1050 1309 and amp range 0 2039 Range in image: x 0 259 , y 0 260 (with image size 259 x 260) dzp -0.00064781494 -> scaling image by 0.9994035183523652 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 563 828 and amp range 0 2005 and image y 4050 4072 and amp range 2040 4078 Range in image: x 0 266 , y 0 23 (with image size 266 x 23) dzp 0.0022078273 -> scaling image by 1.0020355530120029 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 90prime get_fwhm: self.fwhm = 2.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3149 3301 and amp range 2016 4031 and image y 1604 1755 and amp range 0 2047 Range in image: x 0 153 , y 0 152 (with image size 153 x 152) dzp -0.0041296566 -> scaling image by 0.9962036700684744 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 992 1142 and amp range 0 2015 and image y 3580 3730 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp 0.016961161 -> scaling image by 1.0157444649385616 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 90prime get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1226 1376 and amp range 0 2015 and image y 3146 3297 and amp range 2048 4095 Range in image: x 0 151 , y 0 152 (with image size 151 x 152) dzp -0.0045482004 -> scaling image by 0.9958197144381092 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 5.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1727 1991 and amp range 0 2047 and image y 2652 2914 and amp range 2022 4060 Range in image: x 0 265 , y 0 263 (with image size 265 x 263) dzp -0.016192675 -> scaling image by 0.9851966584456431 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 90prime get_fwhm: self.fwhm = 2.79 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1245 1395 and amp range 0 2015 and image y 3164 3314 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp -0.038361903 -> scaling image by 0.9652842912941227 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 90prime get_fwhm: self.fwhm = 4.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 999 1148 and amp range 0 2015 and image y 3592 3742 and amp range 2048 4095 Range in image: x 0 150 , y 0 151 (with image size 150 x 151) dzp 0.01693853 -> scaling image by 1.015723292931903 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 9.74 s, CPU: 1.96 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 175 MB, VmData: 207 MB, maxrss: 0.179336 MB Stage tims finished: 2020-08-05T20:00:33.194053 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.442 s worker CPU, 43.857 s worker Wall, Wall: 9.744 s, Cores in use: 0.76, Total efficiency (on 16 cores): 4.7 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T20:00:33.233544 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01379.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01486.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/reference-custom-277221p50105.fits Stage refs : Wall: 16.71 s, CPU: 6.21 s, VmPeak: 1030 MB, VmSize: 1025 MB, VmRSS: 245 MB, VmData: 275 MB, maxrss: 0.251356 MB Stage refs finished: 2020-08-05T20:00:49.941573 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 16.710 s, Cores in use: 0.37, Total efficiency (on 16 cores): 2.3 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T20:00:49.966946 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outlier-mask-custom-277221p50105.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-pre-custom-277221p50105.jpg Image ksb_160706_100058_ooi_g_ls9-CCD2 g : masking 627 positive outlier pixels and 0 negative outlier pixels Image ksb_180518_105622_ooi_r_ls9-CCD3 r : masking 82 positive outlier pixels and 0 negative outlier pixels Image k4m_160520_100205_ooi_zd_ls9-CCD3 z : masking 445 positive outlier pixels and 0 negative outlier pixels Image k4m_170831_065611_ooi_zd_ls9-CCD4 z : masking 2556 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outlier-mask-custom-277221p50105.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-post-custom-277221p50105.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-masked-pos-custom-277221p50105.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-masked-neg-custom-277221p50105.jpg Stage outliers : Wall: 1.85 s, CPU: 1.47 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 259 MB, VmData: 286 MB, maxrss: 0.265292 MB Stage outliers finished: 2020-08-05T20:00:51.813381 Resources for stage outliers : 0.723 s worker CPU, 0.837 s worker Wall, Wall: 1.848 s, Cores in use: 1.19, Total efficiency (on 16 cores): 7.4 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T20:00:51.899473 Stage halos : Wall: 0.62 s, CPU: 0.60 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 259 MB, VmData: 286 MB, maxrss: 0.26558 MB Stage halos finished: 2020-08-05T20:00:52.519665 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.621 s, Cores in use: 0.96, Total efficiency (on 16 cores): 6.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T20:00:52.546713 PSF sigmas in coadd pixels: ['3.01', '3.24', '2.47'] Using average PSF sigma 2.906776493387067 Gaussian PSF norm 0.09704729359675415 vs pixelized 0.07774058 average tim pixel scale / coadd scale: 1.7293943056680319 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.01', '2.05', '1.95'] Using average PSF sigma 2.004306305485882 Gaussian PSF norm 0.14074435179980788 vs pixelized 0.11695165 average tim pixel scale / coadd scale: 1.7295298717546326 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.57', '1.71', '1.48', '2.24'] Using average PSF sigma 1.749919649473247 Gaussian PSF norm 0.1612044255053614 vs pixelized 0.12437621 average tim pixel scale / coadd scale: 0.9909144364775492 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.16 s, CPU: 0.15 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 261 MB, VmData: 286 MB, maxrss: 0.267124 MB Stage fit_on_coadds finished: 2020-08-05T20:00:52.709980 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.177 s worker CPU, 0.177 s worker Wall, Wall: 0.165 s, Cores in use: 1.97, Total efficiency (on 16 cores): 12.3 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-05T20:00:52.720916 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 26 potential peaks Of 26 potential peaks: 5 in veto map, 2 cut by saddle test, 0 cut by aper test, 19 kept SED z : 19 new peaks Found 27 potential peaks Of 27 potential peaks: 22 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 29 potential peaks Of 29 potential peaks: 24 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 38 potential peaks Of 38 potential peaks: 33 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 24 new sources Median coadd sky for g : 2.9595509e-05 Median coadd sky for r : 6.372848e-05 Median coadd sky for z : -0.00017500823 Stage srcs : Wall: 1.72 s, CPU: 1.64 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 261 MB, VmData: 286 MB, maxrss: 0.267624 MB Stage srcs finished: 2020-08-05T20:00:54.442956 Resources for stage srcs : 0.091 s worker CPU, 0.091 s worker Wall, Wall: 1.723 s, Cores in use: 1.00, Total efficiency (on 16 cores): 6.3 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T20:00:54.459984 Blob 1 of 14, id: 3, sources: 10, size: 169x163, npix 13204, brick X: 48,217, Y: 56,219, one pixel: 87 56 Blob 2 of 14, id: 0, sources: 3, size: 84x62, npix 3200, brick X: 61,145, Y: 0,62, one pixel: 68 0 Blob 3 of 14, id: 11, sources: 4, size: 62x56, npix 2344, brick X: 188,250, Y: 201,257, one pixel: 203 201 Blob 4 of 14, id: 10, sources: 2, size: 41x44, npix 1226, brick X: 159,200, Y: 167,211, one pixel: 169 167 Blob 5 of 14, id: 1, sources: 3, size: 42x41, npix 1123, brick X: 183,225, Y: 0,41, one pixel: 185 0 Blob 6 of 14, id: 4, sources: 2, size: 33x43, npix 862, brick X: 224,257, Y: 62,105, one pixel: 234 62 Blob 7 of 14, id: 13, sources: 1, size: 32x29, npix 690, brick X: 0,32, Y: 228,257, one pixel: 6 228 Blob 8 of 14, id: 7, sources: 1, size: 25x27, npix 468, brick X: 0,25, Y: 110,137, one pixel: 8 110 Blob 9 of 14, id: 5, sources: 1, size: 23x29, npix 459, brick X: 0,23, Y: 75,104, one pixel: 5 75 Blob 10 of 14, id: 6, sources: 1, size: 25x24, npix 375, brick X: 19,44, Y: 88,112, one pixel: 30 88 Blob 11 of 14, id: 2, sources: 1, size: 20x20, npix 238, brick X: 226,246, Y: 22,42, one pixel: 234 22 Blob 12 of 14, id: 12, sources: 1, size: 20x19, npix 209, brick X: 96,116, Y: 225,244, one pixel: 105 225 Blob 13 of 14, id: 9, sources: 1, size: 18x19, npix 189, brick X: 14,32, Y: 143,162, one pixel: 22 143 Blob 14 of 14, id: 8, sources: 1, size: 13x21, npix 176, brick X: 0,13, Y: 143,164, one pixel: 2 143 Blob 12 finished, total: Wall: 0.05 s, CPU: 0.05 s, VmPeak: 807 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150668 MB Blob 13 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151476 MB Blob 11 finished, total: Wall: 0.52 s, CPU: 0.52 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 93 MB, VmData: 141 MB, maxrss: 0.09514 MB Blob 8 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094392 MB Blob 14 finished, total: Wall: 0.53 s, CPU: 0.53 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 138 MB, maxrss: 0.09446 MB Blob 10 finished, total: Wall: 0.63 s, CPU: 0.62 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151412 MB Blob 6 finished, total: Wall: 0.73 s, CPU: 0.73 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 138 MB, maxrss: 0.094188 MB Blob 7 finished, total: Wall: 0.83 s, CPU: 0.83 s, VmPeak: 816 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151536 MB Blob 9 finished, total: Wall: 0.88 s, CPU: 0.88 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096208 MB Blob 4 finished, total: Wall: 1.08 s, CPU: 1.08 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.096896 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21283, 50.09575), pm (-3.6, 3.2), parallax 1.172 with NanoMaggies: g=19.1, r=18, z=17 Reset brightness to NanoMaggies: g=19.3, r=18, z=17.1 Blob 5 finished, total: Wall: 1.19 s, CPU: 1.19 s, VmPeak: 808 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150628 MB Measuring 1 iterative sources Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.22557, 50.09463), pm (-12.9, -4.1), parallax 1.741 with NanoMaggies: g=17.3, r=15.6, z=16.2 Reset brightness to NanoMaggies: g=17.4, r=16.1, z=15.4 Measuring 1 iterative sources Blob 3 finished, total: Wall: 1.69 s, CPU: 1.69 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151468 MB Blob 2 finished, total: Wall: 2.23 s, CPU: 2.23 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 149 MB, VmData: 184 MB, maxrss: 0.152888 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.58318, ee1=-0.419685, ee2=0.561108 Measuring 1 iterative sources Blob 1 finished, total: Wall: 8.06 s, CPU: 8.06 s, VmPeak: 810 MB, VmSize: 732 MB, VmRSS: 150 MB, VmData: 184 MB, maxrss: 0.153692 MB Old catalog: 40 New catalog: 31 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/blobs-custom-277221p50105.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/all-models-custom-277221p50105.fits Stage fitblobs : Wall: 8.94 s, CPU: 0.82 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 262 MB, VmData: 286 MB, maxrss: 0.267788 MB Stage fitblobs finished: 2020-08-05T20:01:03.398461 Resources for stage fitblobs : 19.414 s worker CPU, 19.533 s worker Wall, Wall: 8.940 s, Cores in use: 2.26, Total efficiency (on 16 cores): 14.1 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T20:01:03.416378 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-resid.jpg Stage coadds : Wall: 3.19 s, CPU: 0.86 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 262 MB, VmData: 286 MB, maxrss: 0.268112 MB Stage coadds finished: 2020-08-05T20:01:06.610798 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 8.683 s worker CPU, 13.109 s worker Wall, Wall: 3.196 s, Cores in use: 2.99, Total efficiency (on 16 cores): 18.7 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T20:01:06.631206 Cut to 1 unWISE tiles Reading WISE tile 2763p499 band 1 Reading WISE tile 2763p499 band 2 Reading WISE tile 2763p499 band 3 Reading WISE tile 2763p499 band 4 Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 807 MB, VmSize: 804 MB, VmRSS: 144 MB, VmData: 172 MB, maxrss: 0.208588 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 806 MB, VmSize: 804 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.190512 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 813 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 171 MB, maxrss: 0.208724 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 810 MB, VmSize: 808 MB, VmRSS: 150 MB, VmData: 175 MB, maxrss: 0.214196 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 15.51 s, CPU: 0.93 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 262 MB, VmData: 286 MB, maxrss: 0.268624 MB Stage wise_forced finished: 2020-08-05T20:01:22.143928 Resources for stage wise_forced : 2.913 s worker CPU, 22.484 s worker Wall, Wall: 15.514 s, Cores in use: 0.25, Total efficiency (on 16 cores): 1.5 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T20:01:22.157938 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/tractor-i/cus/tractor-custom-277221p50105.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/tractor/cus/tractor-custom-277221p50105.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/tractor/cus/brick-custom-277221p50105.sha256sum Stage writecat : Wall: 2.06 s, CPU: 0.61 s, VmPeak: 1167 MB, VmSize: 1039 MB, VmRSS: 264 MB, VmData: 286 MB, maxrss: 0.401504 MB Stage writecat finished: 2020-08-05T20:01:24.219955 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 2.064 s, Cores in use: 0.30, Total efficiency (on 16 cores): 1.9 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-writecat.p All done: 37.446 s worker CPU, 100.092 s worker Wall, Wall: 60.795 s, Cores in use: 0.87, Total efficiency (on 16 cores): 5.4 % Finished galaxy PGC2362941 in 1.088 minutes. Started working on galaxy PGC2362941 at Wed Aug 5 21:27:14 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.22127900000004 50.105148299999996 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T21:27:16.215553 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.22127900000004', '50.105148299999996', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.22127900000004 50.105148299999996 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 277.22127900000004 50.105148299999996 Starting process 63473 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 63474 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 63475 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 63476 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 63477 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059536 MB Starting process 63478 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB Starting process 63479 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB Starting process 63480 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 63481 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 63482 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 63483 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 63484 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 63485 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 63486 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 63487 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 63488 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 63489 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 63490 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 63491 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 63492 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 63493 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 63494 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 63495 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 63496 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 63497 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 63498 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 63499 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 63500 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 63501 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 63502 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 63503 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 63504 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T21:27:18.160667 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: k4m_160523_101309_ooi_zd_ls9-CCD3 z expnum 90951 exptime 84.104 propid 2016A-0453 seeing 0.97 MJD 57531.426 object MzLS_651131_z mosaic/CP/V4.3/CP20160522/k4m_160523_101309_ooi_zd_ls9.fits.fz k4m_160524_094908_ooi_zd_ls9-CCD1 z expnum 91253 exptime 80.104 propid 2016A-0453 seeing 1.05 MJD 57532.409 object MzLS_651132_z mosaic/CP/V4.3/CP20160523/k4m_160524_094908_ooi_zd_ls9.fits.fz k4m_160520_100205_ooi_zd_ls9-CCD3 z expnum 90521 exptime 80.103 propid 2016A-0453 seeing 0.91 MJD 57528.418 object MzLS_650328_z mosaic/CP/V4.3/CP20160519/k4m_160520_100205_ooi_zd_ls9.fits.fz k4m_170831_065611_ooi_zd_ls9-CCD4 z expnum 137469 exptime 250.101 propid 2016A-0453 seeing 1.38 MJD 57996.289 object MzLS_649520_z mosaic/CP/V4.3/CP20170830/k4m_170831_065611_ooi_zd_ls9.fits.fz ksb_160612_110533_ooi_r_ls9-CCD2 r expnum 75510139 exptime 99.5 propid BASS seeing 1.24 MJD 57551.462 object 27212 90prime/CP/V2.3/CP20160611/ksb_160612_110533_ooi_r_ls9.fits.fz ksb_160706_100058_ooi_g_ls9-CCD2 g expnum 75750119 exptime 199.0 propid BASS seeing 1.85 MJD 57575.417 object 27212 90prime/CP/V2.3/CP20160705/ksb_160706_100058_ooi_g_ls9.fits.fz ksb_160709_101405_ooi_g_ls9-CCD3 g expnum 75780122 exptime 177.2 propid BASS seeing 2.00 MJD 57578.426 object 27223 90prime/CP/V2.3/CP20160708/ksb_160709_101405_ooi_g_ls9.fits.fz ksb_180518_105622_ooi_r_ls9-CCD3 r expnum 82560213 exptime 86.915 propid NOTBASS seeing 1.26 MJD 58256.456 object 27223 90prime/CP/V2.3/CP20180517/ksb_180518_105622_ooi_r_ls9.fits.fz ksb_160611_110505_ooi_r_ls9-CCD1 r expnum 75500190 exptime 69.6 propid BASS seeing 1.20 MJD 57550.462 object 27211 90prime/CP/V2.3/CP20160610/ksb_160611_110505_ooi_r_ls9.fits.fz ksb_180515_104449_ooi_g_ls9-CCD1 g expnum 82530168 exptime 109.824 propid NOTBASS seeing 1.52 MJD 58253.448 object 27211 90prime/CP/V2.3/CP20180514/ksb_180515_104449_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 536 804 and amp range 0 2005 and image y 296 567 and amp range 0 2039 Range in image: x 0 269 , y 0 272 (with image size 269 x 272) dzp -0.011066288 -> scaling image by 0.989859339076806 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3292 3550 and amp range 2006 4053 and image y 1050 1309 and amp range 0 2039 Range in image: x 0 259 , y 0 260 (with image size 259 x 260) dzp -0.00064781494 -> scaling image by 0.9994035183523652 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 90prime get_fwhm: self.fwhm = 2.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3149 3301 and amp range 2016 4031 and image y 1604 1755 and amp range 0 2047 Range in image: x 0 153 , y 0 152 (with image size 153 x 152) dzp -0.0041296566 -> scaling image by 0.9962036700684744 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 90prime get_fwhm: self.fwhm = 3.35 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3163 3314 and amp range 2016 4031 and image y 1613 1765 and amp range 0 2047 Range in image: x 0 152 , y 0 153 (with image size 152 x 153) dzp -0.019046262 -> scaling image by 0.9826107140833663 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 90prime get_fwhm: self.fwhm = 2.79 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1245 1395 and amp range 0 2015 and image y 3164 3314 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp -0.038361903 -> scaling image by 0.9652842912941227 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 90prime get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1226 1376 and amp range 0 2015 and image y 3146 3297 and amp range 2048 4095 Range in image: x 0 151 , y 0 152 (with image size 151 x 152) dzp -0.0045482004 -> scaling image by 0.9958197144381092 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 5.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1727 1991 and amp range 0 2047 and image y 2652 2914 and amp range 2022 4060 Range in image: x 0 265 , y 0 263 (with image size 265 x 263) dzp -0.016192675 -> scaling image by 0.9851966584456431 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 563 828 and amp range 0 2005 and image y 4050 4072 and amp range 2040 4078 Range in image: x 0 266 , y 0 23 (with image size 266 x 23) dzp 0.0022078273 -> scaling image by 1.0020355530120029 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 992 1142 and amp range 0 2015 and image y 3580 3730 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp 0.016961161 -> scaling image by 1.0157444649385616 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 90prime get_fwhm: self.fwhm = 4.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 999 1148 and amp range 0 2015 and image y 3592 3742 and amp range 2048 4095 Range in image: x 0 150 , y 0 151 (with image size 150 x 151) dzp 0.01693853 -> scaling image by 1.015723292931903 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 9.66 s, CPU: 2.20 s, VmPeak: 958 MB, VmSize: 958 MB, VmRSS: 175 MB, VmData: 207 MB, maxrss: 0.178932 MB Stage tims finished: 2020-08-05T21:27:27.818803 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.337 s worker CPU, 34.824 s worker Wall, Wall: 9.661 s, Cores in use: 0.78, Total efficiency (on 32 cores): 2.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T21:27:27.893624 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01379.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01486.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/reference-custom-277221p50105.fits Stage refs : Wall: 12.34 s, CPU: 5.71 s, VmPeak: 1030 MB, VmSize: 1025 MB, VmRSS: 245 MB, VmData: 275 MB, maxrss: 0.250592 MB Stage refs finished: 2020-08-05T21:27:40.235186 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 12.344 s, Cores in use: 0.46, Total efficiency (on 32 cores): 1.4 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T21:27:40.381781 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outlier-mask-custom-277221p50105.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-pre-custom-277221p50105.jpg Image ksb_160706_100058_ooi_g_ls9-CCD2 g : masking 627 positive outlier pixels and 0 negative outlier pixels Image ksb_180518_105622_ooi_r_ls9-CCD3 r : masking 82 positive outlier pixels and 0 negative outlier pixels Image k4m_170831_065611_ooi_zd_ls9-CCD4 z : masking 2556 positive outlier pixels and 0 negative outlier pixels Image k4m_160520_100205_ooi_zd_ls9-CCD3 z : masking 445 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outlier-mask-custom-277221p50105.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-post-custom-277221p50105.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-masked-pos-custom-277221p50105.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/outliers-masked-neg-custom-277221p50105.jpg Stage outliers : Wall: 3.53 s, CPU: 2.59 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 260 MB, VmData: 287 MB, maxrss: 0.26632 MB Stage outliers finished: 2020-08-05T21:27:43.909884 Resources for stage outliers : 0.906 s worker CPU, 1.344 s worker Wall, Wall: 3.530 s, Cores in use: 0.99, Total efficiency (on 32 cores): 3.1 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T21:27:43.960606 Stage halos : Wall: 2.32 s, CPU: 2.27 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 260 MB, VmData: 287 MB, maxrss: 0.266568 MB Stage halos finished: 2020-08-05T21:27:46.282216 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 2.324 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T21:27:46.417741 PSF sigmas in coadd pixels: ['3.01', '3.24', '2.47'] Using average PSF sigma 2.906776493387067 Gaussian PSF norm 0.09704729359675415 vs pixelized 0.07774058 average tim pixel scale / coadd scale: 1.7293943056680319 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.01', '2.05', '1.95'] Using average PSF sigma 2.004306305485882 Gaussian PSF norm 0.14074435179980788 vs pixelized 0.11695165 average tim pixel scale / coadd scale: 1.7295298717546326 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.57', '1.71', '1.48', '2.24'] Using average PSF sigma 1.749919649473247 Gaussian PSF norm 0.1612044255053614 vs pixelized 0.12437621 average tim pixel scale / coadd scale: 0.9909144364775492 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.20 s, CPU: 0.17 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 262 MB, VmData: 287 MB, maxrss: 0.268036 MB Stage fit_on_coadds finished: 2020-08-05T21:27:46.613095 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.196 s worker CPU, 0.223 s worker Wall, Wall: 0.197 s, Cores in use: 1.85, Total efficiency (on 32 cores): 5.8 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-05T21:27:46.637907 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 26 potential peaks Of 26 potential peaks: 5 in veto map, 2 cut by saddle test, 0 cut by aper test, 19 kept SED z : 19 new peaks Found 27 potential peaks Of 27 potential peaks: 22 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 29 potential peaks Of 29 potential peaks: 24 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 38 potential peaks Of 38 potential peaks: 33 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 24 new sources Median coadd sky for g : 2.9595509e-05 Median coadd sky for r : 6.3728556e-05 Median coadd sky for z : -0.00017500823 Stage srcs : Wall: 2.06 s, CPU: 2.02 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 262 MB, VmData: 287 MB, maxrss: 0.268536 MB Stage srcs finished: 2020-08-05T21:27:48.698375 Resources for stage srcs : 0.081 s worker CPU, 0.081 s worker Wall, Wall: 2.063 s, Cores in use: 1.02, Total efficiency (on 32 cores): 3.2 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T21:27:48.713481 Blob 1 of 14, id: 3, sources: 10, size: 169x163, npix 13204, brick X: 48,217, Y: 56,219, one pixel: 87 56 Blob 2 of 14, id: 0, sources: 3, size: 84x62, npix 3200, brick X: 61,145, Y: 0,62, one pixel: 68 0 Blob 3 of 14, id: 11, sources: 4, size: 62x56, npix 2344, brick X: 188,250, Y: 201,257, one pixel: 203 201 Blob 4 of 14, id: 10, sources: 2, size: 41x44, npix 1226, brick X: 159,200, Y: 167,211, one pixel: 169 167 Blob 5 of 14, id: 1, sources: 3, size: 42x41, npix 1123, brick X: 183,225, Y: 0,41, one pixel: 185 0 Blob 6 of 14, id: 4, sources: 2, size: 33x43, npix 862, brick X: 224,257, Y: 62,105, one pixel: 234 62 Blob 7 of 14, id: 13, sources: 1, size: 32x29, npix 690, brick X: 0,32, Y: 228,257, one pixel: 6 228 Blob 8 of 14, id: 7, sources: 1, size: 25x27, npix 468, brick X: 0,25, Y: 110,137, one pixel: 8 110 Blob 9 of 14, id: 5, sources: 1, size: 23x29, npix 459, brick X: 0,23, Y: 75,104, one pixel: 5 75 Blob 10 of 14, id: 6, sources: 1, size: 25x24, npix 375, brick X: 19,44, Y: 88,112, one pixel: 30 88 Blob 11 of 14, id: 2, sources: 1, size: 20x20, npix 238, brick X: 226,246, Y: 22,42, one pixel: 234 22 Blob 12 of 14, id: 12, sources: 1, size: 20x19, npix 209, brick X: 96,116, Y: 225,244, one pixel: 105 225 Blob 13 of 14, id: 9, sources: 1, size: 18x19, npix 189, brick X: 14,32, Y: 143,162, one pixel: 22 143 Blob 14 of 14, id: 8, sources: 1, size: 13x21, npix 176, brick X: 0,13, Y: 143,164, one pixel: 2 143 Blob 12 finished, total: Wall: 0.11 s, CPU: 0.09 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 86 MB, VmData: 138 MB, maxrss: 0.087924 MB Blob 13 finished, total: Wall: 0.23 s, CPU: 0.21 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092788 MB Blob 11 finished, total: Wall: 0.40 s, CPU: 0.38 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096252 MB Blob 10 finished, total: Wall: 0.39 s, CPU: 0.38 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092828 MB Blob 8 finished, total: Wall: 0.50 s, CPU: 0.49 s, VmPeak: 808 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150352 MB Blob 14 finished, total: Wall: 0.50 s, CPU: 0.49 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.0945 MB Blob 7 finished, total: Wall: 0.62 s, CPU: 0.60 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094464 MB Blob 6 finished, total: Wall: 0.62 s, CPU: 0.61 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094064 MB Blob 9 finished, total: Wall: 0.69 s, CPU: 0.68 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096252 MB Blob 4 finished, total: Wall: 0.94 s, CPU: 0.93 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093004 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21283, 50.09575), pm (-3.6, 3.2), parallax 1.172 with NanoMaggies: g=19.1, r=18, z=17 Reset brightness to NanoMaggies: g=19.3, r=18, z=17.1 Blob 5 finished, total: Wall: 1.03 s, CPU: 1.02 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.09466 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.22557, 50.09463), pm (-12.9, -4.1), parallax 1.741 with NanoMaggies: g=17.3, r=15.6, z=16.2 Reset brightness to NanoMaggies: g=17.4, r=16.1, z=15.4 Measuring 1 iterative sources Measuring 1 iterative sources Blob 3 finished, total: Wall: 1.87 s, CPU: 1.86 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096704 MB Blob 2 finished, total: Wall: 2.15 s, CPU: 2.15 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 98 MB, VmData: 142 MB, maxrss: 0.100188 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.58318, ee1=-0.419685, ee2=0.561109 Measuring 1 iterative sources Blob 1 finished, total: Wall: 8.16 s, CPU: 8.15 s, VmPeak: 807 MB, VmSize: 732 MB, VmRSS: 150 MB, VmData: 184 MB, maxrss: 0.153916 MB Old catalog: 40 New catalog: 31 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/blobs-custom-277221p50105.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/metrics/cus/all-models-custom-277221p50105.fits Stage fitblobs : Wall: 10.08 s, CPU: 1.81 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 262 MB, VmData: 287 MB, maxrss: 0.2687 MB Stage fitblobs finished: 2020-08-05T21:27:58.793857 Resources for stage fitblobs : 18.271 s worker CPU, 19.667 s worker Wall, Wall: 10.083 s, Cores in use: 1.99, Total efficiency (on 32 cores): 6.2 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T21:27:58.811202 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-resid.jpg Stage coadds : Wall: 4.07 s, CPU: 1.85 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 263 MB, VmData: 287 MB, maxrss: 0.269024 MB Stage coadds finished: 2020-08-05T21:28:02.881287 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 25.805 s worker CPU, 43.801 s worker Wall, Wall: 4.072 s, Cores in use: 6.79, Total efficiency (on 32 cores): 21.2 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T21:28:02.895854 Cut to 1 unWISE tiles Reading WISE tile 2763p499 band 2 Reading WISE tile 2763p499 band 4 Reading WISE tile 2763p499 band 3 Reading WISE tile 2763p499 band 1 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 811 MB, VmSize: 807 MB, VmRSS: 149 MB, VmData: 179 MB, maxrss: 0.1907 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 809 MB, VmSize: 807 MB, VmRSS: 149 MB, VmData: 174 MB, maxrss: 0.21326 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 807 MB, VmSize: 804 MB, VmRSS: 144 MB, VmData: 172 MB, maxrss: 0.208248 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 811 MB, VmSize: 799 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.204644 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 4.67 s, CPU: 1.43 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 263 MB, VmData: 287 MB, maxrss: 0.269536 MB Stage wise_forced finished: 2020-08-05T21:28:07.568378 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.368 s worker CPU, 6.769 s worker Wall, Wall: 4.675 s, Cores in use: 1.03, Total efficiency (on 32 cores): 3.2 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-wise_forced.p Running stage writecat WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage writecat at 2020-08-05T21:28:07.583874 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/tractor-i/cus/tractor-custom-277221p50105.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/tractor/cus/tractor-custom-277221p50105.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/tractor/cus/brick-custom-277221p50105.sha256sum Stage writecat : Wall: 1.70 s, CPU: 1.50 s, VmPeak: 1167 MB, VmSize: 1039 MB, VmRSS: 265 MB, VmData: 287 MB, maxrss: 0.402416 MB Stage writecat finished: 2020-08-05T21:28:09.285912 Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 1.705 s, Cores in use: 0.88, Total efficiency (on 32 cores): 2.8 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2362941/PGC2362941-largegalaxy-writecat.p All done: 53.971 s worker CPU, 106.717 s worker Wall, Wall: 51.149 s, Cores in use: 1.48, Total efficiency (on 32 cores): 4.6 % Finished galaxy PGC2362941 in 0.918 minutes. Started working on galaxy PGC2362941 at Fri Aug 7 09:39:23 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T09:39:25.241812 legacypipe git version: DR9.5.11-watershed-3-gd509eaaf Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.2211564669914', '50.105411514107395', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 277.2211564669914 50.105411514107395 Starting process 48419 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 48420 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 48421 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 48422 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 48423 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059412 MB Starting process 48424 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05942 MB Starting process 48425 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05942 MB Starting process 48426 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 48427 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 48428 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 48429 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 48430 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 48431 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 48432 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 48433 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 48434 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T09:39:25.727742 Keeping 10 CCDs: k4m_160523_101309_ooi_zd_ls9-CCD3 z expnum 90951 exptime 84.104 propid 2016A-0453 seeing 0.97 MJD 57531.426 object MzLS_651131_z mosaic/CP/V4.3/CP20160522/k4m_160523_101309_ooi_zd_ls9.fits.fz k4m_160524_094908_ooi_zd_ls9-CCD1 z expnum 91253 exptime 80.104 propid 2016A-0453 seeing 1.05 MJD 57532.409 object MzLS_651132_z mosaic/CP/V4.3/CP20160523/k4m_160524_094908_ooi_zd_ls9.fits.fz k4m_160520_100205_ooi_zd_ls9-CCD3 z expnum 90521 exptime 80.103 propid 2016A-0453 seeing 0.91 MJD 57528.418 object MzLS_650328_z mosaic/CP/V4.3/CP20160519/k4m_160520_100205_ooi_zd_ls9.fits.fz k4m_170831_065611_ooi_zd_ls9-CCD4 z expnum 137469 exptime 250.101 propid 2016A-0453 seeing 1.38 MJD 57996.289 object MzLS_649520_z mosaic/CP/V4.3/CP20170830/k4m_170831_065611_ooi_zd_ls9.fits.fz ksb_160612_110533_ooi_r_ls9-CCD2 r expnum 75510139 exptime 99.5 propid BASS seeing 1.24 MJD 57551.462 object 27212 90prime/CP/V2.3/CP20160611/ksb_160612_110533_ooi_r_ls9.fits.fz ksb_160706_100058_ooi_g_ls9-CCD2 g expnum 75750119 exptime 199.0 propid BASS seeing 1.85 MJD 57575.417 object 27212 90prime/CP/V2.3/CP20160705/ksb_160706_100058_ooi_g_ls9.fits.fz ksb_160709_101405_ooi_g_ls9-CCD3 g expnum 75780122 exptime 177.2 propid BASS seeing 2.00 MJD 57578.426 object 27223 90prime/CP/V2.3/CP20160708/ksb_160709_101405_ooi_g_ls9.fits.fz ksb_180518_105622_ooi_r_ls9-CCD3 r expnum 82560213 exptime 86.915 propid NOTBASS seeing 1.26 MJD 58256.456 object 27223 90prime/CP/V2.3/CP20180517/ksb_180518_105622_ooi_r_ls9.fits.fz ksb_160611_110505_ooi_r_ls9-CCD1 r expnum 75500190 exptime 69.6 propid BASS seeing 1.20 MJD 57550.462 object 27211 90prime/CP/V2.3/CP20160610/ksb_160611_110505_ooi_r_ls9.fits.fz ksb_180515_104449_ooi_g_ls9-CCD1 g expnum 82530168 exptime 109.824 propid NOTBASS seeing 1.52 MJD 58253.448 object 27211 90prime/CP/V2.3/CP20180514/ksb_180515_104449_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 540 808 and amp range 0 2005 and image y 295 565 and amp range 0 2039 Range in image: x 0 269 , y 0 271 (with image size 269 x 271) dzp -0.011066288 -> scaling image by 0.989859339076806 mosaic get_fwhm: self.fwhm = 5.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1730 1994 and amp range 0 2047 and image y 2650 2912 and amp range 2022 4060 Range in image: x 0 265 , y 0 263 (with image size 265 x 263) dzp -0.016192675 -> scaling image by 0.9851966584456431 mosaic get_fwhm: self.fwhm = 3.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 566 832 and amp range 0 2005 and image y 4049 4072 and amp range 2040 4078 Range in image: x 0 267 , y 0 24 (with image size 267 x 24) dzp 0.0022078273 -> scaling image by 1.0020355530120029 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 90prime get_fwhm: self.fwhm = 3.35 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3161 3312 and amp range 2016 4031 and image y 1614 1765 and amp range 0 2047 Range in image: x 0 152 , y 0 152 (with image size 152 x 152) dzp -0.019046262 -> scaling image by 0.9826107140833663 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 90prime get_fwhm: self.fwhm = 4.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 997 1146 and amp range 0 2015 and image y 3592 3742 and amp range 2048 4095 Range in image: x 0 150 , y 0 151 (with image size 150 x 151) dzp 0.01693853 -> scaling image by 1.015723292931903 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 90prime get_fwhm: self.fwhm = 2.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3147 3299 and amp range 2016 4031 and image y 1604 1756 and amp range 0 2047 Range in image: x 0 153 , y 0 153 (with image size 153 x 153) dzp -0.0041296566 -> scaling image by 0.9962036700684744 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 90prime get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1223 1374 and amp range 0 2015 and image y 3147 3297 and amp range 2048 4095 Range in image: x 0 152 , y 0 151 (with image size 152 x 151) dzp -0.0045482004 -> scaling image by 0.9958197144381092 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 90prime get_fwhm: self.fwhm = 2.79 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1243 1393 and amp range 0 2015 and image y 3164 3315 and amp range 2048 4095 Range in image: x 0 151 , y 0 152 (with image size 151 x 152) dzp -0.038361903 -> scaling image by 0.9652842912941227 mosaic get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3295 3554 and amp range 2006 4053 and image y 1049 1308 and amp range 0 2039 Range in image: x 0 260 , y 0 260 (with image size 260 x 260) dzp -0.00064781494 -> scaling image by 0.9994035183523652 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 990 1140 and amp range 0 2015 and image y 3581 3731 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp 0.016961161 -> scaling image by 1.0157444649385616 Stage tims : Wall: 225.96 s, CPU: 1.07 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 173 MB, VmData: 207 MB, maxrss: 0.177332 MB Stage tims finished: 2020-08-07T09:43:11.688155 Resources for stage tims : 4.924 s worker CPU, 750.181 s worker Wall, Wall: 225.962 s, Cores in use: 0.03, Total efficiency (on 16 cores): 0.2 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T09:43:14.757582 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01379.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01486.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/reference-custom-277221p50105.fits Stage refs : Wall: 15.46 s, CPU: 4.68 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 243 MB, VmData: 274 MB, maxrss: 0.248848 MB Stage refs finished: 2020-08-07T09:43:30.220452 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 15.464 s, Cores in use: 0.30, Total efficiency (on 16 cores): 1.9 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T09:43:30.268344 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/outlier-mask-custom-277221p50105.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/outliers-pre-custom-277221p50105.jpg Image ksb_160706_100058_ooi_g_ls9-CCD2 g : masking 686 positive outlier pixels and 0 negative outlier pixels Image ksb_180518_105622_ooi_r_ls9-CCD3 r : masking 89 positive outlier pixels and 0 negative outlier pixels Image k4m_160520_100205_ooi_zd_ls9-CCD3 z : masking 446 positive outlier pixels and 0 negative outlier pixels Image k4m_170831_065611_ooi_zd_ls9-CCD4 z : masking 2593 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/outlier-mask-custom-277221p50105.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/outliers-post-custom-277221p50105.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/outliers-masked-pos-custom-277221p50105.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/outliers-masked-neg-custom-277221p50105.jpg Stage outliers : Wall: 1.55 s, CPU: 0.60 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 258 MB, VmData: 287 MB, maxrss: 0.264452 MB Stage outliers finished: 2020-08-07T09:43:31.815455 Resources for stage outliers : 0.695 s worker CPU, 0.787 s worker Wall, Wall: 1.548 s, Cores in use: 0.84, Total efficiency (on 16 cores): 5.2 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T09:43:31.846169 Stage halos : Wall: 0.31 s, CPU: 0.30 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 258 MB, VmData: 287 MB, maxrss: 0.2647 MB Stage halos finished: 2020-08-07T09:43:32.160062 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.315 s, Cores in use: 0.97, Total efficiency (on 16 cores): 6.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T09:43:32.247505 PSF sigmas in coadd pixels: ['3.01', '3.24', '2.47'] Using average PSF sigma 2.906776493387067 Gaussian PSF norm 0.09704729359675415 vs pixelized 0.077739716 average tim pixel scale / coadd scale: 1.7293943056680319 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.01', '2.05', '1.95'] Using average PSF sigma 2.004306305485882 Gaussian PSF norm 0.14074435179980788 vs pixelized 0.11695378 average tim pixel scale / coadd scale: 1.7295298717546326 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.57', '1.71', '1.48', '2.24'] Using average PSF sigma 1.749919649473247 Gaussian PSF norm 0.1612044255053614 vs pixelized 0.124374904 average tim pixel scale / coadd scale: 0.9909144364775492 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.15 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 260 MB, VmData: 287 MB, maxrss: 0.266168 MB Stage fit_on_coadds finished: 2020-08-07T09:43:32.427092 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.166 s worker CPU, 0.166 s worker Wall, Wall: 0.181 s, Cores in use: 1.73, Total efficiency (on 16 cores): 10.8 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-07T09:43:32.457055 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 27 potential peaks Of 27 potential peaks: 6 in veto map, 1 cut by saddle test, 0 cut by aper test, 20 kept SED z : 20 new peaks Found 24 potential peaks Of 24 potential peaks: 21 in veto map, 1 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 38 potential peaks Of 38 potential peaks: 33 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 25 new sources Median coadd sky for g : 3.4408953e-05 Median coadd sky for r : 8.198482e-05 Median coadd sky for z : -0.00020729686 Stage srcs : Wall: 0.50 s, CPU: 0.45 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 260 MB, VmData: 287 MB, maxrss: 0.266652 MB Stage srcs finished: 2020-08-07T09:43:32.952906 Resources for stage srcs : 0.092 s worker CPU, 0.092 s worker Wall, Wall: 0.497 s, Cores in use: 1.09, Total efficiency (on 16 cores): 6.8 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T09:43:32.964199 Blob 1 of 15, id: 3, sources: 10, size: 170x163, npix 13143, brick X: 46,216, Y: 52,215, one pixel: 87 52 Blob 2 of 15, id: 0, sources: 2, size: 83x58, npix 2840, brick X: 61,144, Y: 0,58, one pixel: 66 0 Blob 3 of 15, id: 11, sources: 4, size: 62x60, npix 2438, brick X: 187,249, Y: 197,257, one pixel: 203 197 Blob 4 of 15, id: 10, sources: 2, size: 41x45, npix 1235, brick X: 158,199, Y: 163,208, one pixel: 170 163 Blob 5 of 15, id: 1, sources: 3, size: 40x37, npix 936, brick X: 184,224, Y: 0,37, one pixel: 187 0 Blob 6 of 15, id: 4, sources: 2, size: 35x43, npix 891, brick X: 222,257, Y: 59,102, one pixel: 232 59 Blob 7 of 15, id: 13, sources: 1, size: 31x32, npix 746, brick X: 0,31, Y: 225,257, one pixel: 5 225 Blob 8 of 15, id: 7, sources: 1, size: 24x27, npix 467, brick X: 0,24, Y: 106,133, one pixel: 8 106 Blob 9 of 15, id: 14, sources: 2, size: 39x18, npix 467, brick X: 145,184, Y: 239,257, one pixel: 153 239 Blob 10 of 15, id: 5, sources: 1, size: 22x28, npix 424, brick X: 0,22, Y: 72,100, one pixel: 5 72 Blob 11 of 15, id: 6, sources: 1, size: 25x23, npix 363, brick X: 18,43, Y: 85,108, one pixel: 28 85 Blob 12 of 15, id: 2, sources: 1, size: 20x20, npix 219, brick X: 225,245, Y: 18,38, one pixel: 234 18 Blob 13 of 15, id: 12, sources: 1, size: 19x19, npix 208, brick X: 96,115, Y: 222,241, one pixel: 104 222 Blob 14 of 15, id: 8, sources: 1, size: 18x19, npix 189, brick X: 13,31, Y: 139,158, one pixel: 21 139 Blob 15 of 15, id: 9, sources: 1, size: 11x17, npix 109, brick X: 0,11, Y: 141,158, one pixel: 2 141 Blob 13 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.1467 MB Blob 14 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 138 MB, maxrss: 0.094148 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21688, 50.11484), pm (-0.5, -8.3), parallax 1.142 with NanoMaggies: g=22.3, r=21, z=18.7 Reset brightness to NanoMaggies: g=22, r=20.5, z=19.4 Blob 9 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093656 MB Blob 8 finished, total: Wall: 0.43 s, CPU: 0.43 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147432 MB Blob 12 finished, total: Wall: 0.42 s, CPU: 0.42 s, VmPeak: 808 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.146764 MB Blob 11 finished, total: Wall: 0.49 s, CPU: 0.48 s, VmPeak: 792 MB, VmSize: 711 MB, VmRSS: 130 MB, VmData: 169 MB, maxrss: 0.133336 MB Blob 15 finished, total: Wall: 0.51 s, CPU: 0.51 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092836 MB Blob 10 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.096948 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21283, 50.09575), pm (-3.6, 3.2), parallax 1.172 with NanoMaggies: g=19.5, r=18.2, z=17.1 Reset brightness to NanoMaggies: g=19.3, r=18, z=17.1 Blob 5 finished, total: Wall: 0.66 s, CPU: 0.65 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146912 MB Blob 7 finished, total: Wall: 0.68 s, CPU: 0.67 s, VmPeak: 816 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147064 MB Blob 6 finished, total: Wall: 0.84 s, CPU: 0.83 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.094868 MB Blob 4 finished, total: Wall: 1.04 s, CPU: 1.03 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.09526 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 1.66 s, CPU: 1.65 s, VmPeak: 807 MB, VmSize: 725 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148752 MB Blob 3 finished, total: Wall: 1.86 s, CPU: 1.86 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147864 MB Exception in one_blob: brick custom-277221p50105, iblob 3 Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1637, in _bounce_one_blob result = one_blob(X) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 124, in one_blob B = ob.run(B, reoptimize=reoptimize, iterative_detection=iterative) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 376, in run Ibright = _argsort_by_brightness(cat, self.bands, ref_first=True) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 1989, in _argsort_by_brightness br = src.getBrightness() AttributeError: 'NoneType' object has no attribute 'getBrightness' multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 121, in worker result = (True, func(*args, **kwds)) File "/usr/local/lib/python/astrometry/util/timingpool.py", line 269, in __call__ R = self.func(*args, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1637, in _bounce_one_blob result = one_blob(X) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 124, in one_blob B = ob.run(B, reoptimize=reoptimize, iterative_detection=iterative) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 376, in run Ibright = _argsort_by_brightness(cat, self.bands, ref_first=True) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 1989, in _argsort_by_brightness br = src.getBrightness() AttributeError: 'NoneType' object has no attribute 'getBrightness' """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3771, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3739, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3295, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3284, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1131, in stage_fitblobs r = Riter.next(timeout) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 748, in next raise value AttributeError: 'NoneType' object has no attribute 'getBrightness' Something went wrong; please check the logfile. ERROR: galaxy PGC2362941; please check the logfile. Finished galaxy PGC2362941 in 4.217 minutes. Started working on galaxy PGC2362941 at Fri Aug 7 12:03:51 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T12:03:52.586945 legacypipe git version: DR9.5.11-watershed-4-gcc619972 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.2211564669914', '50.105411514107395', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 277.2211564669914 50.105411514107395 Starting process 12143 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 12144 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 12145 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 12146 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 12147 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05932 MB Starting process 12148 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 12149 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 12150 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 12151 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 12152 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 12153 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 12154 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 12155 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 12156 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 12157 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 12158 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 12159 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 12160 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 12161 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 12162 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 12163 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 12164 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 12165 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 12166 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 12167 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 12168 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 12169 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 12170 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 12171 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 12172 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 12173 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 12174 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Ignoring pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p and forcing stage srcs Runstage fit_on_coadds Ignoring pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p and forcing stage fit_on_coadds Runstage halos Ignoring pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p and forcing stage halos Runstage outliers Ignoring pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p and forcing stage outliers Runstage refs Ignoring pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p and forcing stage refs Runstage tims Ignoring pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p and forcing stage tims Running stage tims Running stage tims at 2020-08-07T12:03:53.165658 Keeping 10 CCDs: k4m_160523_101309_ooi_zd_ls9-CCD3 z expnum 90951 exptime 84.104 propid 2016A-0453 seeing 0.97 MJD 57531.426 object MzLS_651131_z mosaic/CP/V4.3/CP20160522/k4m_160523_101309_ooi_zd_ls9.fits.fz k4m_160524_094908_ooi_zd_ls9-CCD1 z expnum 91253 exptime 80.104 propid 2016A-0453 seeing 1.05 MJD 57532.409 object MzLS_651132_z mosaic/CP/V4.3/CP20160523/k4m_160524_094908_ooi_zd_ls9.fits.fz k4m_160520_100205_ooi_zd_ls9-CCD3 z expnum 90521 exptime 80.103 propid 2016A-0453 seeing 0.91 MJD 57528.418 object MzLS_650328_z mosaic/CP/V4.3/CP20160519/k4m_160520_100205_ooi_zd_ls9.fits.fz k4m_170831_065611_ooi_zd_ls9-CCD4 z expnum 137469 exptime 250.101 propid 2016A-0453 seeing 1.38 MJD 57996.289 object MzLS_649520_z mosaic/CP/V4.3/CP20170830/k4m_170831_065611_ooi_zd_ls9.fits.fz ksb_160612_110533_ooi_r_ls9-CCD2 r expnum 75510139 exptime 99.5 propid BASS seeing 1.24 MJD 57551.462 object 27212 90prime/CP/V2.3/CP20160611/ksb_160612_110533_ooi_r_ls9.fits.fz ksb_160706_100058_ooi_g_ls9-CCD2 g expnum 75750119 exptime 199.0 propid BASS seeing 1.85 MJD 57575.417 object 27212 90prime/CP/V2.3/CP20160705/ksb_160706_100058_ooi_g_ls9.fits.fz ksb_160709_101405_ooi_g_ls9-CCD3 g expnum 75780122 exptime 177.2 propid BASS seeing 2.00 MJD 57578.426 object 27223 90prime/CP/V2.3/CP20160708/ksb_160709_101405_ooi_g_ls9.fits.fz ksb_180518_105622_ooi_r_ls9-CCD3 r expnum 82560213 exptime 86.915 propid NOTBASS seeing 1.26 MJD 58256.456 object 27223 90prime/CP/V2.3/CP20180517/ksb_180518_105622_ooi_r_ls9.fits.fz ksb_160611_110505_ooi_r_ls9-CCD1 r expnum 75500190 exptime 69.6 propid BASS seeing 1.20 MJD 57550.462 object 27211 90prime/CP/V2.3/CP20160610/ksb_160611_110505_ooi_r_ls9.fits.fz ksb_180515_104449_ooi_g_ls9-CCD1 g expnum 82530168 exptime 109.824 propid NOTBASS seeing 1.52 MJD 58253.448 object 27211 90prime/CP/V2.3/CP20180514/ksb_180515_104449_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 3.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 566 832 and amp range 0 2005 and image y 4049 4072 and amp range 2040 4078 Range in image: x 0 267 , y 0 24 (with image size 267 x 24) dzp 0.0022078273 -> scaling image by 1.0020355530120029 mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 540 808 and amp range 0 2005 and image y 295 565 and amp range 0 2039 Range in image: x 0 269 , y 0 271 (with image size 269 x 271) dzp -0.011066288 -> scaling image by 0.989859339076806 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 90prime get_fwhm: self.fwhm = 3.35 Found 4 relevant rows in amp-corrections file. mosaic get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3161 3312 and amp range 2016 4031 and image y 1614 1765 and amp range 0 2047 Range in image: x 0 152 , y 0 152 (with image size 152 x 152) dzp -0.019046262 -> scaling image by 0.9826107140833663 Found overlap: image x 3295 3554 and amp range 2006 4053 and image y 1049 1308 and amp range 0 2039 Range in image: x 0 260 , y 0 260 (with image size 260 x 260) dzp -0.00064781494 -> scaling image by 0.9994035183523652 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 90prime get_fwhm: self.fwhm = 2.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3147 3299 and amp range 2016 4031 and image y 1604 1756 and amp range 0 2047 Range in image: x 0 153 , y 0 153 (with image size 153 x 153) dzp -0.0041296566 -> scaling image by 0.9962036700684744 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 90prime get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1223 1374 and amp range 0 2015 and image y 3147 3297 and amp range 2048 4095 Range in image: x 0 152 , y 0 151 (with image size 152 x 151) dzp -0.0045482004 -> scaling image by 0.9958197144381092 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 90prime get_fwhm: self.fwhm = 2.79 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1243 1393 and amp range 0 2015 and image y 3164 3315 and amp range 2048 4095 Range in image: x 0 151 , y 0 152 (with image size 151 x 152) dzp -0.038361903 -> scaling image by 0.9652842912941227 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 90prime get_fwhm: self.fwhm = 4.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 997 1146 and amp range 0 2015 and image y 3592 3742 and amp range 2048 4095 Range in image: x 0 150 , y 0 151 (with image size 150 x 151) dzp 0.01693853 -> scaling image by 1.015723292931903 mosaic get_fwhm: self.fwhm = 5.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1730 1994 and amp range 0 2047 and image y 2650 2912 and amp range 2022 4060 Range in image: x 0 265 , y 0 263 (with image size 265 x 263) dzp -0.016192675 -> scaling image by 0.9851966584456431 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 990 1140 and amp range 0 2015 and image y 3581 3731 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp 0.016961161 -> scaling image by 1.0157444649385616 Stage tims : Wall: 2.33 s, CPU: 0.82 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 173 MB, VmData: 207 MB, maxrss: 0.177396 MB Stage tims finished: 2020-08-07T12:03:55.490784 Resources for stage tims : 5.091 s worker CPU, 11.531 s worker Wall, Wall: 2.328 s, Cores in use: 2.54, Total efficiency (on 32 cores): 7.9 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T12:03:55.553627 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01379.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01486.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/reference-custom-277221p50105.fits Stage refs : Wall: 4.93 s, CPU: 4.59 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 243 MB, VmData: 274 MB, maxrss: 0.249304 MB Stage refs finished: 2020-08-07T12:04:00.484540 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 4.933 s, Cores in use: 0.93, Total efficiency (on 32 cores): 2.9 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T12:04:00.507732 Stage outliers : Wall: 0.36 s, CPU: 0.33 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 244 MB, VmData: 275 MB, maxrss: 0.249872 MB Stage outliers finished: 2020-08-07T12:04:00.867052 Resources for stage outliers : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.361 s, Cores in use: 0.92, Total efficiency (on 32 cores): 2.9 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T12:04:00.890772 Stage halos : Wall: 0.25 s, CPU: 0.22 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 247 MB, VmData: 278 MB, maxrss: 0.252736 MB Stage halos finished: 2020-08-07T12:04:01.137357 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.249 s, Cores in use: 0.91, Total efficiency (on 32 cores): 2.8 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T12:04:03.349596 PSF sigmas in coadd pixels: ['3.01', '3.24', '2.47'] Using average PSF sigma 2.906776493387067 Gaussian PSF norm 0.09704729359675415 vs pixelized 0.077739716 average tim pixel scale / coadd scale: 1.7293943056680319 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.01', '2.05', '1.95'] Using average PSF sigma 2.004306305485882 Gaussian PSF norm 0.14074435179980788 vs pixelized 0.11695378 average tim pixel scale / coadd scale: 1.7295298717546326 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.57', '1.71', '1.48', '2.24'] Using average PSF sigma 1.749919649473247 Gaussian PSF norm 0.1612044255053614 vs pixelized 0.124374904 average tim pixel scale / coadd scale: 0.9909144364775492 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-z.fits.fz Stage fit_on_coadds : Wall: 7.54 s, CPU: 0.16 s, VmPeak: 1024 MB, VmSize: 1024 MB, VmRSS: 258 MB, VmData: 287 MB, maxrss: 0.26428 MB Stage fit_on_coadds finished: 2020-08-07T12:04:10.890127 Resources for stage fit_on_coadds : 0.209 s worker CPU, 0.339 s worker Wall, Wall: 7.543 s, Cores in use: 0.05, Total efficiency (on 32 cores): 0.2 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-07T12:04:10.902540 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 27 potential peaks Of 27 potential peaks: 6 in veto map, 1 cut by saddle test, 0 cut by aper test, 20 kept SED z : 20 new peaks Found 24 potential peaks Of 24 potential peaks: 21 in veto map, 1 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 38 potential peaks Of 38 potential peaks: 33 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 25 new sources Median coadd sky for g : 3.4408957e-05 Median coadd sky for r : 8.198478e-05 Median coadd sky for z : -0.00020729704 Stage srcs : Wall: 0.59 s, CPU: 0.53 s, VmPeak: 1024 MB, VmSize: 1024 MB, VmRSS: 259 MB, VmData: 287 MB, maxrss: 0.264956 MB Stage srcs finished: 2020-08-07T12:04:11.492771 Resources for stage srcs : 0.110 s worker CPU, 0.124 s worker Wall, Wall: 0.592 s, Cores in use: 1.09, Total efficiency (on 32 cores): 3.4 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T12:04:12.477252 Blob 1 of 15, id: 3, sources: 10, size: 170x163, npix 13143, brick X: 46,216, Y: 52,215, one pixel: 87 52 Blob 2 of 15, id: 0, sources: 2, size: 83x58, npix 2840, brick X: 61,144, Y: 0,58, one pixel: 66 0 Blob 3 of 15, id: 11, sources: 4, size: 62x60, npix 2438, brick X: 187,249, Y: 197,257, one pixel: 203 197 Blob 4 of 15, id: 10, sources: 2, size: 41x45, npix 1235, brick X: 158,199, Y: 163,208, one pixel: 170 163 Blob 5 of 15, id: 1, sources: 3, size: 40x37, npix 936, brick X: 184,224, Y: 0,37, one pixel: 187 0 Blob 6 of 15, id: 4, sources: 2, size: 35x43, npix 891, brick X: 222,257, Y: 59,102, one pixel: 232 59 Blob 7 of 15, id: 13, sources: 1, size: 31x32, npix 746, brick X: 0,31, Y: 225,257, one pixel: 5 225 Blob 8 of 15, id: 7, sources: 1, size: 24x27, npix 467, brick X: 0,24, Y: 106,133, one pixel: 8 106 Blob 9 of 15, id: 14, sources: 2, size: 39x18, npix 467, brick X: 145,184, Y: 239,257, one pixel: 153 239 Blob 10 of 15, id: 5, sources: 1, size: 22x28, npix 424, brick X: 0,22, Y: 72,100, one pixel: 5 72 Blob 11 of 15, id: 6, sources: 1, size: 25x23, npix 363, brick X: 18,43, Y: 85,108, one pixel: 28 85 Blob 12 of 15, id: 2, sources: 1, size: 20x20, npix 219, brick X: 225,245, Y: 18,38, one pixel: 234 18 Blob 13 of 15, id: 12, sources: 1, size: 19x19, npix 208, brick X: 96,115, Y: 222,241, one pixel: 104 222 Blob 14 of 15, id: 8, sources: 1, size: 18x19, npix 189, brick X: 13,31, Y: 139,158, one pixel: 21 139 Blob 15 of 15, id: 9, sources: 1, size: 11x17, npix 109, brick X: 0,11, Y: 141,158, one pixel: 2 141 Blob 13 finished, total: Wall: 0.11 s, CPU: 0.06 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 82 MB, VmData: 136 MB, maxrss: 0.084004 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21688, 50.11484), pm (-0.5, -8.3), parallax 1.142 with NanoMaggies: g=24.5, r=23, z=20.4 Reset brightness to NanoMaggies: g=22, r=20.5, z=19.4 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21283, 50.09575), pm (-3.6, 3.2), parallax 1.172 with NanoMaggies: g=21.2, r=19.1, z=17.8 Reset brightness to NanoMaggies: g=19.3, r=18, z=17.1 Blob 9 finished, total: Wall: 0.16 s, CPU: 0.16 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 85 MB, VmData: 137 MB, maxrss: 0.087296 MB Blob 14 finished, total: Wall: 0.20 s, CPU: 0.20 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145372 MB Blob 12 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.09274 MB Blob 11 finished, total: Wall: 0.34 s, CPU: 0.34 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.14512 MB Blob 15 finished, total: Wall: 0.38 s, CPU: 0.37 s, VmPeak: 808 MB, VmSize: 724 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.146468 MB Blob 8 finished, total: Wall: 0.45 s, CPU: 0.45 s, VmPeak: 806 MB, VmSize: 724 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145752 MB Blob 5 finished, total: Wall: 0.57 s, CPU: 0.57 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145168 MB Blob 7 finished, total: Wall: 0.64 s, CPU: 0.60 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 88 MB, VmData: 137 MB, maxrss: 0.090172 MB Blob 6 finished, total: Wall: 0.66 s, CPU: 0.62 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 88 MB, VmData: 138 MB, maxrss: 0.090588 MB Blob 10 finished, total: Wall: 0.75 s, CPU: 0.70 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 88 MB, VmData: 137 MB, maxrss: 0.089844 MB Blob 4 finished, total: Wall: 1.01 s, CPU: 0.97 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 89 MB, VmData: 138 MB, maxrss: 0.091252 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 1.58 s, CPU: 1.55 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.096408 MB Blob 3 finished, total: Wall: 1.73 s, CPU: 1.73 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145432 MB Exception in one_blob: brick custom-277221p50105, iblob 3 Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1641, in _bounce_one_blob result = one_blob(X) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 124, in one_blob B = ob.run(B, reoptimize=reoptimize, iterative_detection=iterative) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 380, in run Ibright = _argsort_by_brightness(cat, self.bands, ref_first=True) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 1994, in _argsort_by_brightness br = src.getBrightness() AttributeError: 'NoneType' object has no attribute 'getBrightness' multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 121, in worker result = (True, func(*args, **kwds)) File "/usr/local/lib/python/astrometry/util/timingpool.py", line 269, in __call__ R = self.func(*args, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1641, in _bounce_one_blob result = one_blob(X) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 124, in one_blob B = ob.run(B, reoptimize=reoptimize, iterative_detection=iterative) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 380, in run Ibright = _argsort_by_brightness(cat, self.bands, ref_first=True) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 1994, in _argsort_by_brightness br = src.getBrightness() AttributeError: 'NoneType' object has no attribute 'getBrightness' """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3775, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3743, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3299, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3288, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1135, in stage_fitblobs r = Riter.next(timeout) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 748, in next raise value AttributeError: 'NoneType' object has no attribute 'getBrightness' Something went wrong; please check the logfile. ERROR: galaxy PGC2362941; please check the logfile. Finished galaxy PGC2362941 in 0.393 minutes. Started working on galaxy PGC2362941 at Fri Aug 7 12:31:50 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T12:31:51.199765 legacypipe git version: DR9.5.11-watershed-4-gcc619972 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.2211564669914', '50.105411514107395', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 277.2211564669914 50.105411514107395 Starting process 15982 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 15983 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 15984 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 15985 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 15986 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05954 MB Starting process 15987 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059552 MB Starting process 15988 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059552 MB Starting process 15989 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 15990 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 15991 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 15992 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 15993 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 15994 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 15995 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 15996 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 15997 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 15998 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 15999 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 16000 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 16001 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 16002 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 16003 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 16004 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 16005 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 16006 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 16007 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 16008 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 16009 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 16010 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 16011 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 16012 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 16013 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T12:31:51.635865 Keeping 10 CCDs: k4m_160523_101309_ooi_zd_ls9-CCD3 z expnum 90951 exptime 84.104 propid 2016A-0453 seeing 0.97 MJD 57531.426 object MzLS_651131_z mosaic/CP/V4.3/CP20160522/k4m_160523_101309_ooi_zd_ls9.fits.fz k4m_160524_094908_ooi_zd_ls9-CCD1 z expnum 91253 exptime 80.104 propid 2016A-0453 seeing 1.05 MJD 57532.409 object MzLS_651132_z mosaic/CP/V4.3/CP20160523/k4m_160524_094908_ooi_zd_ls9.fits.fz k4m_160520_100205_ooi_zd_ls9-CCD3 z expnum 90521 exptime 80.103 propid 2016A-0453 seeing 0.91 MJD 57528.418 object MzLS_650328_z mosaic/CP/V4.3/CP20160519/k4m_160520_100205_ooi_zd_ls9.fits.fz k4m_170831_065611_ooi_zd_ls9-CCD4 z expnum 137469 exptime 250.101 propid 2016A-0453 seeing 1.38 MJD 57996.289 object MzLS_649520_z mosaic/CP/V4.3/CP20170830/k4m_170831_065611_ooi_zd_ls9.fits.fz ksb_160612_110533_ooi_r_ls9-CCD2 r expnum 75510139 exptime 99.5 propid BASS seeing 1.24 MJD 57551.462 object 27212 90prime/CP/V2.3/CP20160611/ksb_160612_110533_ooi_r_ls9.fits.fz ksb_160706_100058_ooi_g_ls9-CCD2 g expnum 75750119 exptime 199.0 propid BASS seeing 1.85 MJD 57575.417 object 27212 90prime/CP/V2.3/CP20160705/ksb_160706_100058_ooi_g_ls9.fits.fz ksb_160709_101405_ooi_g_ls9-CCD3 g expnum 75780122 exptime 177.2 propid BASS seeing 2.00 MJD 57578.426 object 27223 90prime/CP/V2.3/CP20160708/ksb_160709_101405_ooi_g_ls9.fits.fz ksb_180518_105622_ooi_r_ls9-CCD3 r expnum 82560213 exptime 86.915 propid NOTBASS seeing 1.26 MJD 58256.456 object 27223 90prime/CP/V2.3/CP20180517/ksb_180518_105622_ooi_r_ls9.fits.fz ksb_160611_110505_ooi_r_ls9-CCD1 r expnum 75500190 exptime 69.6 propid BASS seeing 1.20 MJD 57550.462 object 27211 90prime/CP/V2.3/CP20160610/ksb_160611_110505_ooi_r_ls9.fits.fz ksb_180515_104449_ooi_g_ls9-CCD1 g expnum 82530168 exptime 109.824 propid NOTBASS seeing 1.52 MJD 58253.448 object 27211 90prime/CP/V2.3/CP20180514/ksb_180515_104449_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 3.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 566 832 and amp range 0 2005 and image y 4049 4072 and amp range 2040 4078 Range in image: x 0 267 , y 0 24 (with image size 267 x 24) dzp 0.0022078273 -> scaling image by 1.0020355530120029 mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 540 808 and amp range 0 2005 and image y 295 565 and amp range 0 2039 Range in image: x 0 269 , y 0 271 (with image size 269 x 271) dzp -0.011066288 -> scaling image by 0.989859339076806 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 90prime get_fwhm: self.fwhm = 2.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3147 3299 and amp range 2016 4031 and image y 1604 1756 and amp range 0 2047 Range in image: x 0 153 , y 0 153 (with image size 153 x 153) dzp -0.0041296566 -> scaling image by 0.9962036700684744 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 90prime get_fwhm: self.fwhm = 3.35 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3161 3312 and amp range 2016 4031 and image y 1614 1765 and amp range 0 2047 Range in image: x 0 152 , y 0 152 (with image size 152 x 152) dzp -0.019046262 -> scaling image by 0.9826107140833663 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 90prime get_fwhm: self.fwhm = 2.79 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1243 1393 and amp range 0 2015 and image y 3164 3315 and amp range 2048 4095 Range in image: x 0 151 , y 0 152 (with image size 151 x 152) dzp -0.038361903 -> scaling image by 0.9652842912941227 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 90prime get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1223 1374 and amp range 0 2015 and image y 3147 3297 and amp range 2048 4095 Range in image: x 0 152 , y 0 151 (with image size 152 x 151) dzp -0.0045482004 -> scaling image by 0.9958197144381092 mosaic get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3295 3554 and amp range 2006 4053 and image y 1049 1308 and amp range 0 2039 Range in image: x 0 260 , y 0 260 (with image size 260 x 260) dzp -0.00064781494 -> scaling image by 0.9994035183523652 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 90prime get_fwhm: self.fwhm = 4.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 997 1146 and amp range 0 2015 and image y 3592 3742 and amp range 2048 4095 Range in image: x 0 150 , y 0 151 (with image size 150 x 151) dzp 0.01693853 -> scaling image by 1.015723292931903 mosaic get_fwhm: self.fwhm = 5.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1730 1994 and amp range 0 2047 and image y 2650 2912 and amp range 2022 4060 Range in image: x 0 265 , y 0 263 (with image size 265 x 263) dzp -0.016192675 -> scaling image by 0.9851966584456431 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 990 1140 and amp range 0 2015 and image y 3581 3731 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp 0.016961161 -> scaling image by 1.0157444649385616 Stage tims : Wall: 2.46 s, CPU: 0.78 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 173 MB, VmData: 207 MB, maxrss: 0.177384 MB Stage tims finished: 2020-08-07T12:31:54.095777 Resources for stage tims : 5.060 s worker CPU, 12.790 s worker Wall, Wall: 2.462 s, Cores in use: 2.37, Total efficiency (on 32 cores): 7.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T12:31:54.115539 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01379.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01486.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/reference-custom-277221p50105.fits Stage refs : Wall: 5.03 s, CPU: 4.61 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 243 MB, VmData: 274 MB, maxrss: 0.248888 MB Stage refs finished: 2020-08-07T12:31:59.142330 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 5.029 s, Cores in use: 0.92, Total efficiency (on 32 cores): 2.9 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T12:31:59.163394 Stage outliers : Wall: 0.27 s, CPU: 0.25 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 244 MB, VmData: 275 MB, maxrss: 0.249864 MB Stage outliers finished: 2020-08-07T12:31:59.435977 Resources for stage outliers : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.274 s, Cores in use: 0.93, Total efficiency (on 32 cores): 2.9 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T12:31:59.576849 Stage halos : Wall: 0.21 s, CPU: 0.20 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 247 MB, VmData: 278 MB, maxrss: 0.25272 MB Stage halos finished: 2020-08-07T12:31:59.787429 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.213 s, Cores in use: 0.95, Total efficiency (on 32 cores): 3.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T12:31:59.811959 PSF sigmas in coadd pixels: ['3.01', '3.24', '2.47'] Using average PSF sigma 2.906776493387067 Gaussian PSF norm 0.09704729359675415 vs pixelized 0.077739716 average tim pixel scale / coadd scale: 1.7293943056680319 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.01', '2.05', '1.95'] Using average PSF sigma 2.004306305485882 Gaussian PSF norm 0.14074435179980788 vs pixelized 0.116953775 average tim pixel scale / coadd scale: 1.7295298717546326 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.57', '1.71', '1.48', '2.24'] Using average PSF sigma 1.749919649473247 Gaussian PSF norm 0.1612044255053614 vs pixelized 0.124374904 average tim pixel scale / coadd scale: 0.9909144364775492 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.17 s, CPU: 0.16 s, VmPeak: 1024 MB, VmSize: 1024 MB, VmRSS: 257 MB, VmData: 286 MB, maxrss: 0.263192 MB Stage fit_on_coadds finished: 2020-08-07T12:31:59.977428 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.182 s worker CPU, 0.255 s worker Wall, Wall: 0.168 s, Cores in use: 2.03, Total efficiency (on 32 cores): 6.3 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-07T12:31:59.995497 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 27 potential peaks Of 27 potential peaks: 6 in veto map, 1 cut by saddle test, 0 cut by aper test, 20 kept SED z : 20 new peaks Found 24 potential peaks Of 24 potential peaks: 21 in veto map, 1 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 38 potential peaks Of 38 potential peaks: 33 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 25 new sources Median coadd sky for g : 3.4408953e-05 Median coadd sky for r : 8.198466e-05 Median coadd sky for z : -0.00020729704 Stage srcs : Wall: 0.43 s, CPU: 0.37 s, VmPeak: 1024 MB, VmSize: 1024 MB, VmRSS: 258 MB, VmData: 286 MB, maxrss: 0.264032 MB Stage srcs finished: 2020-08-07T12:32:00.424153 Resources for stage srcs : 0.112 s worker CPU, 0.128 s worker Wall, Wall: 0.431 s, Cores in use: 1.13, Total efficiency (on 32 cores): 3.5 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T12:32:00.446857 Blob 1 of 15, id: 3, sources: 10, size: 170x163, npix 13143, brick X: 46,216, Y: 52,215, one pixel: 87 52 Blob 2 of 15, id: 0, sources: 2, size: 83x58, npix 2840, brick X: 61,144, Y: 0,58, one pixel: 66 0 Blob 3 of 15, id: 11, sources: 4, size: 62x60, npix 2438, brick X: 187,249, Y: 197,257, one pixel: 203 197 Blob 4 of 15, id: 10, sources: 2, size: 41x45, npix 1235, brick X: 158,199, Y: 163,208, one pixel: 170 163 Blob 5 of 15, id: 1, sources: 3, size: 40x37, npix 936, brick X: 184,224, Y: 0,37, one pixel: 187 0 Blob 6 of 15, id: 4, sources: 2, size: 35x43, npix 891, brick X: 222,257, Y: 59,102, one pixel: 232 59 Blob 7 of 15, id: 13, sources: 1, size: 31x32, npix 746, brick X: 0,31, Y: 225,257, one pixel: 5 225 Blob 8 of 15, id: 7, sources: 1, size: 24x27, npix 467, brick X: 0,24, Y: 106,133, one pixel: 8 106 Blob 9 of 15, id: 14, sources: 2, size: 39x18, npix 467, brick X: 145,184, Y: 239,257, one pixel: 153 239 Blob 10 of 15, id: 5, sources: 1, size: 22x28, npix 424, brick X: 0,22, Y: 72,100, one pixel: 5 72 Blob 11 of 15, id: 6, sources: 1, size: 25x23, npix 363, brick X: 18,43, Y: 85,108, one pixel: 28 85 Blob 12 of 15, id: 2, sources: 1, size: 20x20, npix 219, brick X: 225,245, Y: 18,38, one pixel: 234 18 Blob 13 of 15, id: 12, sources: 1, size: 19x19, npix 208, brick X: 96,115, Y: 222,241, one pixel: 104 222 Blob 14 of 15, id: 8, sources: 1, size: 18x19, npix 189, brick X: 13,31, Y: 139,158, one pixel: 21 139 Blob 15 of 15, id: 9, sources: 1, size: 11x17, npix 109, brick X: 0,11, Y: 141,158, one pixel: 2 141 Blob 13 finished, total: Wall: 0.09 s, CPU: 0.06 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 82 MB, VmData: 137 MB, maxrss: 0.084168 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21688, 50.11484), pm (-0.5, -8.3), parallax 1.142 with NanoMaggies: g=24.5, r=23, z=20.4 Reset brightness to NanoMaggies: g=22, r=20.5, z=19.4 Blob 9 finished, total: Wall: 0.13 s, CPU: 0.13 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145652 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21283, 50.09575), pm (-3.6, 3.2), parallax 1.172 with NanoMaggies: g=21.2, r=19.1, z=17.8 Reset brightness to NanoMaggies: g=19.3, r=18, z=17.1 Blob 14 finished, total: Wall: 0.24 s, CPU: 0.21 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 87 MB, VmData: 137 MB, maxrss: 0.089596 MB Blob 12 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 792 MB, VmSize: 711 MB, VmRSS: 128 MB, VmData: 169 MB, maxrss: 0.130584 MB Blob 11 finished, total: Wall: 0.32 s, CPU: 0.32 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092504 MB Blob 8 finished, total: Wall: 0.45 s, CPU: 0.44 s, VmPeak: 808 MB, VmSize: 724 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.145992 MB Blob 15 finished, total: Wall: 0.45 s, CPU: 0.43 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 87 MB, VmData: 137 MB, maxrss: 0.089588 MB Blob 6 finished, total: Wall: 0.55 s, CPU: 0.54 s, VmPeak: 806 MB, VmSize: 724 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.14596 MB Blob 7 finished, total: Wall: 0.55 s, CPU: 0.55 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145516 MB Blob 5 finished, total: Wall: 0.56 s, CPU: 0.55 s, VmPeak: 816 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145144 MB Blob 10 finished, total: Wall: 0.69 s, CPU: 0.69 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.14564 MB Blob 4 finished, total: Wall: 0.93 s, CPU: 0.89 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 89 MB, VmData: 138 MB, maxrss: 0.091348 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 1.58 s, CPU: 1.54 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.096092 MB Blob 3 finished, total: Wall: 1.80 s, CPU: 1.76 s, VmPeak: 668 MB, VmSize: 667 MB, VmRSS: 89 MB, VmData: 138 MB, maxrss: 0.092824 MB Exception in one_blob: brick custom-277221p50105, iblob 3 Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1641, in _bounce_one_blob result = one_blob(X) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 124, in one_blob B = ob.run(B, reoptimize=reoptimize, iterative_detection=iterative) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 380, in run Ibright = _argsort_by_brightness(cat, self.bands, ref_first=True) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 1994, in _argsort_by_brightness br = src.getBrightness() AttributeError: 'NoneType' object has no attribute 'getBrightness' multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 121, in worker result = (True, func(*args, **kwds)) File "/usr/local/lib/python/astrometry/util/timingpool.py", line 269, in __call__ R = self.func(*args, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1641, in _bounce_one_blob result = one_blob(X) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 124, in one_blob B = ob.run(B, reoptimize=reoptimize, iterative_detection=iterative) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 380, in run Ibright = _argsort_by_brightness(cat, self.bands, ref_first=True) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/oneblob.py", line 1994, in _argsort_by_brightness br = src.getBrightness() AttributeError: 'NoneType' object has no attribute 'getBrightness' """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3775, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3743, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3299, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage write=write, initial_args=initial_args, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3288, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 1135, in stage_fitblobs r = Riter.next(timeout) File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 748, in next raise value AttributeError: 'NoneType' object has no attribute 'getBrightness' Something went wrong; please check the logfile. ERROR: galaxy PGC2362941; please check the logfile. Finished galaxy PGC2362941 in 0.215 minutes. Started working on galaxy PGC2362941 at Fri Aug 7 13:31:20 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T13:31:21.510871 legacypipe git version: DR9.5.11-watershed-5-g77ea3463 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.2211564669914', '50.105411514107395', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.2211564669914 50.105411514107395 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 277.2211564669914 50.105411514107395 Starting process 19668 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 19669 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 19670 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 19671 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 19672 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05948 MB Starting process 19673 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059488 MB Starting process 19674 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059488 MB Starting process 19675 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 19676 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 19677 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 19678 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 19679 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 19680 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 19681 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 19682 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 19683 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 19684 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 19685 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 19686 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 19687 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 19688 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 19689 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 19690 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 19691 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 19692 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 19693 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 19694 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 19695 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 19696 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 19697 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 19698 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 19699 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T13:31:22.159352 Keeping 10 CCDs: k4m_160523_101309_ooi_zd_ls9-CCD3 z expnum 90951 exptime 84.104 propid 2016A-0453 seeing 0.97 MJD 57531.426 object MzLS_651131_z mosaic/CP/V4.3/CP20160522/k4m_160523_101309_ooi_zd_ls9.fits.fz k4m_160524_094908_ooi_zd_ls9-CCD1 z expnum 91253 exptime 80.104 propid 2016A-0453 seeing 1.05 MJD 57532.409 object MzLS_651132_z mosaic/CP/V4.3/CP20160523/k4m_160524_094908_ooi_zd_ls9.fits.fz k4m_160520_100205_ooi_zd_ls9-CCD3 z expnum 90521 exptime 80.103 propid 2016A-0453 seeing 0.91 MJD 57528.418 object MzLS_650328_z mosaic/CP/V4.3/CP20160519/k4m_160520_100205_ooi_zd_ls9.fits.fz k4m_170831_065611_ooi_zd_ls9-CCD4 z expnum 137469 exptime 250.101 propid 2016A-0453 seeing 1.38 MJD 57996.289 object MzLS_649520_z mosaic/CP/V4.3/CP20170830/k4m_170831_065611_ooi_zd_ls9.fits.fz ksb_160612_110533_ooi_r_ls9-CCD2 r expnum 75510139 exptime 99.5 propid BASS seeing 1.24 MJD 57551.462 object 27212 90prime/CP/V2.3/CP20160611/ksb_160612_110533_ooi_r_ls9.fits.fz ksb_160706_100058_ooi_g_ls9-CCD2 g expnum 75750119 exptime 199.0 propid BASS seeing 1.85 MJD 57575.417 object 27212 90prime/CP/V2.3/CP20160705/ksb_160706_100058_ooi_g_ls9.fits.fz ksb_160709_101405_ooi_g_ls9-CCD3 g expnum 75780122 exptime 177.2 propid BASS seeing 2.00 MJD 57578.426 object 27223 90prime/CP/V2.3/CP20160708/ksb_160709_101405_ooi_g_ls9.fits.fz ksb_180518_105622_ooi_r_ls9-CCD3 r expnum 82560213 exptime 86.915 propid NOTBASS seeing 1.26 MJD 58256.456 object 27223 90prime/CP/V2.3/CP20180517/ksb_180518_105622_ooi_r_ls9.fits.fz ksb_160611_110505_ooi_r_ls9-CCD1 r expnum 75500190 exptime 69.6 propid BASS seeing 1.20 MJD 57550.462 object 27211 90prime/CP/V2.3/CP20160610/ksb_160611_110505_ooi_r_ls9.fits.fz ksb_180515_104449_ooi_g_ls9-CCD1 g expnum 82530168 exptime 109.824 propid NOTBASS seeing 1.52 MJD 58253.448 object 27211 90prime/CP/V2.3/CP20180514/ksb_180515_104449_ooi_g_ls9.fits.fz Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 Faked up EXPNUM 75780122 90prime get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1223 1374 and amp range 0 2015 and image y 3147 3297 and amp range 2048 4095 Range in image: x 0 152 , y 0 151 (with image size 152 x 151) dzp -0.0045482004 -> scaling image by 0.9958197144381092 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 Faked up EXPNUM 82530168 90prime get_fwhm: self.fwhm = 3.35 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3161 3312 and amp range 2016 4031 and image y 1614 1765 and amp range 0 2047 Range in image: x 0 152 , y 0 152 (with image size 152 x 152) dzp -0.019046262 -> scaling image by 0.9826107140833663 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 Faked up EXPNUM 75500190 90prime get_fwhm: self.fwhm = 2.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3147 3299 and amp range 2016 4031 and image y 1604 1756 and amp range 0 2047 Range in image: x 0 153 , y 0 153 (with image size 153 x 153) dzp -0.0041296566 -> scaling image by 0.9962036700684744 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 Faked up EXPNUM 82560213 90prime get_fwhm: self.fwhm = 2.79 Found 4 relevant rows in amp-corrections file. mosaic get_fwhm: self.fwhm = 5.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1243 1393 and amp range 0 2015 and image y 3164 3315 and amp range 2048 4095 Range in image: x 0 151 , y 0 152 (with image size 151 x 152) Found overlap: image x 1730 1994 and amp range 0 2047 and image y 2650 2912 and amp range 2022 4060 dzp -0.038361903 -> scaling image by 0.9652842912941227 Range in image: x 0 265 , y 0 263 (with image size 265 x 263) dzp -0.016192675 -> scaling image by 0.9851966584456431 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 Faked up EXPNUM 75750119 90prime get_fwhm: self.fwhm = 4.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 997 1146 and amp range 0 2015 and image y 3592 3742 and amp range 2048 4095 Range in image: x 0 150 , y 0 151 (with image size 150 x 151) dzp 0.01693853 -> scaling image by 1.015723292931903 mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 540 808 and amp range 0 2005 and image y 295 565 and amp range 0 2039 Range in image: x 0 269 , y 0 271 (with image size 269 x 271) dzp -0.011066288 -> scaling image by 0.989859339076806 mosaic get_fwhm: self.fwhm = 3.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 566 832 and amp range 0 2005 and image y 4049 4072 and amp range 2040 4078 Range in image: x 0 267 , y 0 24 (with image size 267 x 24) dzp 0.0022078273 -> scaling image by 1.0020355530120029 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 Faked up EXPNUM 75510139 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 990 1140 and amp range 0 2015 and image y 3581 3731 and amp range 2048 4095 Range in image: x 0 151 , y 0 151 (with image size 151 x 151) dzp 0.016961161 -> scaling image by 1.0157444649385616 mosaic get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3295 3554 and amp range 2006 4053 and image y 1049 1308 and amp range 0 2039 Range in image: x 0 260 , y 0 260 (with image size 260 x 260) dzp -0.00064781494 -> scaling image by 0.9994035183523652 Stage tims : Wall: 5.96 s, CPU: 0.84 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 173 MB, VmData: 207 MB, maxrss: 0.177608 MB Stage tims finished: 2020-08-07T13:31:28.121177 Resources for stage tims : 4.888 s worker CPU, 16.464 s worker Wall, Wall: 5.964 s, Cores in use: 0.96, Total efficiency (on 32 cores): 3.0 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T13:31:28.227391 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01379.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01486.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/reference-custom-277221p50105.fits Stage refs : Wall: 5.07 s, CPU: 4.60 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 243 MB, VmData: 275 MB, maxrss: 0.249184 MB Stage refs finished: 2020-08-07T13:31:33.293931 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 5.069 s, Cores in use: 0.91, Total efficiency (on 32 cores): 2.8 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T13:31:33.445738 Stage outliers : Wall: 0.30 s, CPU: 0.28 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 244 MB, VmData: 275 MB, maxrss: 0.25016 MB Stage outliers finished: 2020-08-07T13:31:33.745418 Resources for stage outliers : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.302 s, Cores in use: 0.95, Total efficiency (on 32 cores): 3.0 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T13:31:33.779212 Stage halos : Wall: 0.32 s, CPU: 0.31 s, VmPeak: 1024 MB, VmSize: 1019 MB, VmRSS: 247 MB, VmData: 278 MB, maxrss: 0.253008 MB Stage halos finished: 2020-08-07T13:31:34.101100 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.324 s, Cores in use: 0.96, Total efficiency (on 32 cores): 3.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T13:31:34.124860 PSF sigmas in coadd pixels: ['3.01', '3.24', '2.47'] Using average PSF sigma 2.906776493387067 Gaussian PSF norm 0.09704729359675415 vs pixelized 0.077739716 average tim pixel scale / coadd scale: 1.7293943056680319 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.01', '2.05', '1.95'] Using average PSF sigma 2.004306305485882 Gaussian PSF norm 0.14074435179980788 vs pixelized 0.11695378 average tim pixel scale / coadd scale: 1.7295298717546326 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.57', '1.71', '1.48', '2.24'] Using average PSF sigma 1.749919649473247 Gaussian PSF norm 0.1612044255053614 vs pixelized 0.124374904 average tim pixel scale / coadd scale: 0.9909144364775492 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.22 s, CPU: 0.15 s, VmPeak: 1024 MB, VmSize: 1024 MB, VmRSS: 258 MB, VmData: 287 MB, maxrss: 0.2642 MB Stage fit_on_coadds finished: 2020-08-07T13:31:34.341285 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.273 s worker CPU, 0.401 s worker Wall, Wall: 0.218 s, Cores in use: 1.96, Total efficiency (on 32 cores): 6.1 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-07T13:31:34.352407 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 27 potential peaks Of 27 potential peaks: 6 in veto map, 1 cut by saddle test, 0 cut by aper test, 20 kept SED z : 20 new peaks Found 24 potential peaks Of 24 potential peaks: 21 in veto map, 1 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 38 potential peaks Of 38 potential peaks: 33 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 25 new sources Median coadd sky for g : 3.4408942e-05 Median coadd sky for r : 8.198478e-05 Median coadd sky for z : -0.00020729704 Stage srcs : Wall: 0.62 s, CPU: 0.42 s, VmPeak: 1024 MB, VmSize: 1024 MB, VmRSS: 259 MB, VmData: 287 MB, maxrss: 0.264944 MB Stage srcs finished: 2020-08-07T13:31:34.972033 Resources for stage srcs : 0.088 s worker CPU, 0.106 s worker Wall, Wall: 0.621 s, Cores in use: 0.82, Total efficiency (on 32 cores): 2.6 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T13:31:34.984991 Blob 1 of 15, id: 3, sources: 10, size: 170x163, npix 13143, brick X: 46,216, Y: 52,215, one pixel: 87 52 Blob 2 of 15, id: 0, sources: 2, size: 83x58, npix 2840, brick X: 61,144, Y: 0,58, one pixel: 66 0 Blob 3 of 15, id: 11, sources: 4, size: 62x60, npix 2438, brick X: 187,249, Y: 197,257, one pixel: 203 197 Blob 4 of 15, id: 10, sources: 2, size: 41x45, npix 1235, brick X: 158,199, Y: 163,208, one pixel: 170 163 Blob 5 of 15, id: 1, sources: 3, size: 40x37, npix 936, brick X: 184,224, Y: 0,37, one pixel: 187 0 Blob 6 of 15, id: 4, sources: 2, size: 35x43, npix 891, brick X: 222,257, Y: 59,102, one pixel: 232 59 Blob 7 of 15, id: 13, sources: 1, size: 31x32, npix 746, brick X: 0,31, Y: 225,257, one pixel: 5 225 Blob 8 of 15, id: 7, sources: 1, size: 24x27, npix 467, brick X: 0,24, Y: 106,133, one pixel: 8 106 Blob 9 of 15, id: 14, sources: 2, size: 39x18, npix 467, brick X: 145,184, Y: 239,257, one pixel: 153 239 Blob 10 of 15, id: 5, sources: 1, size: 22x28, npix 424, brick X: 0,22, Y: 72,100, one pixel: 5 72 Blob 11 of 15, id: 6, sources: 1, size: 25x23, npix 363, brick X: 18,43, Y: 85,108, one pixel: 28 85 Blob 12 of 15, id: 2, sources: 1, size: 20x20, npix 219, brick X: 225,245, Y: 18,38, one pixel: 234 18 Blob 13 of 15, id: 12, sources: 1, size: 19x19, npix 208, brick X: 96,115, Y: 222,241, one pixel: 104 222 Blob 14 of 15, id: 8, sources: 1, size: 18x19, npix 189, brick X: 13,31, Y: 139,158, one pixel: 21 139 Blob 15 of 15, id: 9, sources: 1, size: 11x17, npix 109, brick X: 0,11, Y: 141,158, one pixel: 2 141 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21283, 50.09575), pm (-3.6, 3.2), parallax 1.172 with NanoMaggies: g=21.2, r=19.1, z=17.8 Reset brightness to NanoMaggies: g=19.3, r=18, z=17.1 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (277.21688, 50.11484), pm (-0.5, -8.3), parallax 1.142 with NanoMaggies: g=24.5, r=23, z=20.4 Reset brightness to NanoMaggies: g=22, r=20.5, z=19.4 Blob 13 finished, total: Wall: 0.22 s, CPU: 0.09 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 83 MB, VmData: 137 MB, maxrss: 0.084596 MB Blob 9 finished, total: Wall: 0.23 s, CPU: 0.15 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 83 MB, VmData: 137 MB, maxrss: 0.084704 MB Blob 14 finished, total: Wall: 0.30 s, CPU: 0.24 s, VmPeak: 806 MB, VmSize: 724 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.145988 MB Blob 12 finished, total: Wall: 0.33 s, CPU: 0.27 s, VmPeak: 816 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.1451 MB Blob 11 finished, total: Wall: 0.39 s, CPU: 0.32 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.09418 MB Blob 15 finished, total: Wall: 0.43 s, CPU: 0.40 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.14512 MB Blob 8 finished, total: Wall: 0.56 s, CPU: 0.51 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 88 MB, VmData: 137 MB, maxrss: 0.089904 MB Blob 5 finished, total: Wall: 0.62 s, CPU: 0.57 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145156 MB Blob 7 finished, total: Wall: 0.69 s, CPU: 0.58 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 88 MB, VmData: 137 MB, maxrss: 0.090012 MB Blob 6 finished, total: Wall: 0.75 s, CPU: 0.62 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 88 MB, VmData: 138 MB, maxrss: 0.090572 MB Blob 10 finished, total: Wall: 0.78 s, CPU: 0.68 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 88 MB, VmData: 137 MB, maxrss: 0.08998 MB Blob 4 finished, total: Wall: 1.03 s, CPU: 0.92 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 89 MB, VmData: 138 MB, maxrss: 0.0913 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 1.64 s, CPU: 1.53 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 142 MB, maxrss: 0.096364 MB Blob 3 finished, total: Wall: 1.85 s, CPU: 1.78 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.1455 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.025, ee1=-0.310793, ee2=0.454796 Measuring 1 iterative sources Blob 1 finished, total: Wall: 8.88 s, CPU: 8.88 s, VmPeak: 694 MB, VmSize: 688 MB, VmRSS: 114 MB, VmData: 156 MB, maxrss: 0.1223 MB Old catalog: 41 New catalog: 30 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/blobs-custom-277221p50105.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/metrics/cus/all-models-custom-277221p50105.fits Stage fitblobs : Wall: 9.51 s, CPU: 0.62 s, VmPeak: 1024 MB, VmSize: 1024 MB, VmRSS: 259 MB, VmData: 287 MB, maxrss: 0.265112 MB Stage fitblobs finished: 2020-08-07T13:31:44.495736 Resources for stage fitblobs : 17.754 s worker CPU, 19.004 s worker Wall, Wall: 9.513 s, Cores in use: 1.93, Total efficiency (on 32 cores): 6.0 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-07T13:31:44.510377 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-resid.jpg Stage coadds : Wall: 2.07 s, CPU: 0.55 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 260 MB, VmData: 287 MB, maxrss: 0.266312 MB Stage coadds finished: 2020-08-07T13:31:46.579403 Resources for stage coadds : 25.854 s worker CPU, 26.710 s worker Wall, Wall: 2.071 s, Cores in use: 12.75, Total efficiency (on 32 cores): 39.8 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-07T13:31:48.447509 Cut to 1 unWISE tiles Reading WISE tile 2763p499 band 2 Reading WISE tile 2763p499 band 3 Reading WISE tile 2763p499 band 4 Reading WISE tile 2763p499 band 1 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 811 MB, VmSize: 802 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.187912 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 811 MB, VmSize: 792 MB, VmRSS: 136 MB, VmData: 170 MB, maxrss: 0.198432 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 792 MB, VmSize: 792 MB, VmRSS: 135 MB, VmData: 170 MB, maxrss: 0.193536 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2763p499/unwise-2763p499-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 813 MB, VmSize: 798 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.204672 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/tmp-legacysurvey-custom-277221p50105-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 3.20 s, CPU: 0.57 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 261 MB, VmData: 287 MB, maxrss: 0.267112 MB Stage wise_forced finished: 2020-08-07T13:31:51.650647 Resources for stage wise_forced : 3.107 s worker CPU, 4.884 s worker Wall, Wall: 3.205 s, Cores in use: 1.15, Total efficiency (on 32 cores): 3.6 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-07T13:31:51.665827 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/coadd/cus/custom-277221p50105/legacysurvey-custom-277221p50105-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/tractor-i/cus/tractor-custom-277221p50105.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/tractor/cus/tractor-custom-277221p50105.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/tractor/cus/brick-custom-277221p50105.sha256sum Stage writecat : Wall: 0.77 s, CPU: 0.59 s, VmPeak: 1160 MB, VmSize: 1032 MB, VmRSS: 263 MB, VmData: 287 MB, maxrss: 0.399996 MB Stage writecat finished: 2020-08-07T13:31:52.435120 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.771 s, Cores in use: 0.76, Total efficiency (on 32 cores): 2.4 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2362941/PGC2362941-largegalaxy-writecat.p All done: 51.970 s worker CPU, 67.575 s worker Wall, Wall: 30.298 s, Cores in use: 2.02, Total efficiency (on 32 cores): 6.3 % Finished galaxy PGC2362941 in 0.541 minutes.