Started working on galaxy PGC2407584 at Wed Aug 5 19:49:41 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 272.585736 52.135887100000005 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:49:42.596685 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '272.585736', '52.135887100000005', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 272.585736 52.135887100000005 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 272.585736 52.135887100000005 Starting process 20761 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 20762 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 20763 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 20764 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 20765 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059404 MB Starting process 20766 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059352 MB Starting process 20767 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059352 MB Starting process 20768 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 20769 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 20770 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 20771 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 20772 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 20773 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 20774 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 20775 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 20776 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:49:43.670240 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 9 CCDs: k4m_160526_091718_ooi_zd_ls9-CCD2 z expnum 91635 exptime 119.103 propid 2016A-0453 seeing 1.67 MJD 57534.387 object MzLS_658212_z mosaic/CP/V4.3/CP20160525/k4m_160526_091718_ooi_zd_ls9.fits.fz k4m_170901_071139_ooi_zd_ls9-CCD1 z expnum 137641 exptime 156.103 propid 2016A-0453 seeing 1.09 MJD 57997.300 object MzLS_658983_z mosaic/CP/V4.3/CP20170831/k4m_170901_071139_ooi_zd_ls9.fits.fz k4m_160520_094855_ooi_zd_ls9-CCD2 z expnum 90515 exptime 80.103 propid 2016A-0453 seeing 1.08 MJD 57528.409 object MzLS_657439_z mosaic/CP/V4.3/CP20160519/k4m_160520_094855_ooi_zd_ls9.fits.fz ksb_160707_091929_ooi_g_ls9-CCD2 g expnum 75760094 exptime 229.0 propid BASS seeing 2.13 MJD 57576.389 object 29452 90prime/CP/V2.3/CP20160706/ksb_160707_091929_ooi_g_ls9.fits.fz ksb_160712_064128_ooi_r_ls9-CCD2 r expnum 75810082 exptime 138.0 propid BASS seeing 1.64 MJD 57581.279 object 29452 90prime/CP/V2.3/CP20160711/ksb_160712_064128_ooi_r_ls9.fits.fz ksb_170624_062822_ooi_r_ls9-CCD3 r expnum 79280073 exptime 76.476 propid BASS seeing 1.13 MJD 57928.270 object 29463 90prime/CP/V2.3/CP20170623/ksb_170624_062822_ooi_r_ls9.fits.fz ksb_170618_061845_ooi_g_ls9-CCD3 g expnum 79220074 exptime 121.071 propid BASS seeing 1.42 MJD 57922.263 object 29463 90prime/CP/V2.3/CP20170617/ksb_170618_061845_ooi_g_ls9.fits.fz ksb_160603_083839_ooi_g_ls9-CCD3 g expnum 75420121 exptime 87.9 propid BASS seeing 1.37 MJD 57542.360 object 29461 90prime/CP/V2.3/CP20160602/ksb_160603_083839_ooi_g_ls9.fits.fz ksb_170405_113253_ooi_r_ls9-CCD3 r expnum 78480180 exptime 56.095 propid BASS seeing 1.24 MJD 57848.481 object 29461 90prime/CP/V2.3/CP20170404/ksb_170405_113253_ooi_r_ls9.fits.fz Faked up EXPNUM 78480180 Faked up EXPNUM 78480180 Faked up EXPNUM 78480180 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2969 3111 and amp range 2016 4031 and image y 2452 2595 and amp range 2048 4095 Range in image: x 0 143 , y 0 144 (with image size 143 x 144) dzp -0.035697103 -> scaling image by 0.9676563679910417 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75420121 Faked up EXPNUM 75420121 Faked up EXPNUM 75420121 90prime get_fwhm: self.fwhm = 3.03 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2953 3095 and amp range 2016 4031 and image y 2451 2594 and amp range 2048 4095 Range in image: x 0 143 , y 0 144 (with image size 143 x 144) dzp -0.0012536051 -> scaling image by 0.9988460533227724 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75810082 Faked up EXPNUM 75810082 Faked up EXPNUM 75810082 90prime get_fwhm: self.fwhm = 3.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 723 867 and amp range 0 2015 and image y 911 1056 and amp range 0 2047 Range in image: x 0 145 , y 0 146 (with image size 145 x 146) dzp -0.006045583 -> scaling image by 0.9944472857505693 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 4.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1613 1863 and amp range 0 2005 and image y 2225 2472 and amp range 2040 4078 Range in image: x 0 251 , y 0 248 (with image size 251 x 248) dzp 0.014419514 -> scaling image by 1.0133694453441033 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75760094 Faked up EXPNUM 75760094 Faked up EXPNUM 75760094 90prime get_fwhm: self.fwhm = 4.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 723 867 and amp range 0 2015 and image y 897 1042 and amp range 0 2047 Range in image: x 0 145 , y 0 146 (with image size 145 x 146) dzp -0.008523628 -> scaling image by 0.992180183483648 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 4.15 Swapping in SKY_JOHN values for 1 splinesky cells; 46.982933 -> 46.94934 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2751 3002 and amp range 2048 4095 and image y 2111 2361 and amp range 2040 4078 Range in image: x 0 252 , y 0 251 (with image size 252 x 251) dzp -0.009154885 -> scaling image by 0.9916034885139223 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 79280073 Faked up EXPNUM 79280073 Faked up EXPNUM 79280073 90prime get_fwhm: self.fwhm = 2.49 Found 4 relevant rows in amp-corrections file. Found overlap: image x 981 1124 and amp range 0 2015 and image y 465 608 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp -0.043123975 -> scaling image by 0.961059796099719 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 79220074 Faked up EXPNUM 79220074 Faked up EXPNUM 79220074 90prime get_fwhm: self.fwhm = 3.13 Found 4 relevant rows in amp-corrections file. Found overlap: image x 987 1130 and amp range 0 2015 and image y 468 611 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp -0.036130503 -> scaling image by 0.9672701794860779 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 6.45 Found 4 relevant rows in amp-corrections file. Found overlap: image x 51 297 and amp range 0 2047 and image y 3518 3764 and amp range 2040 4078 Range in image: x 0 247 , y 0 247 (with image size 247 x 247) dzp 0.008550294 -> scaling image by 1.0079062018164673 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 9.01 s, CPU: 1.22 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 207 MB, maxrss: 0.17828 MB Stage tims finished: 2020-08-05T19:49:52.681743 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 4.535 s worker CPU, 45.572 s worker Wall, Wall: 9.013 s, Cores in use: 0.64, Total efficiency (on 16 cores): 4.0 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T19:49:52.972223 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01378.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01275.fits Matched 7 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/reference-custom-272585p52135.fits Stage refs : Wall: 7.33 s, CPU: 3.95 s, VmPeak: 1005 MB, VmSize: 1000 MB, VmRSS: 219 MB, VmData: 249 MB, maxrss: 0.224604 MB Stage refs finished: 2020-08-05T19:50:00.306747 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 7.336 s, Cores in use: 0.54, Total efficiency (on 16 cores): 3.4 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:50:00.418668 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outlier-mask-custom-272585p52135.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-pre-custom-272585p52135.jpg Image ksb_160603_083839_ooi_g_ls9-CCD3 g : masking 163 positive outlier pixels and 0 negative outlier pixels Image ksb_170624_062822_ooi_r_ls9-CCD3 r : masking 142 positive outlier pixels and 0 negative outlier pixels Image k4m_160526_091718_ooi_zd_ls9-CCD2 z : masking 325 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outlier-mask-custom-272585p52135.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-post-custom-272585p52135.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-masked-pos-custom-272585p52135.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-masked-neg-custom-272585p52135.jpg Stage outliers : Wall: 1.34 s, CPU: 1.04 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 233 MB, VmData: 260 MB, maxrss: 0.238376 MB Stage outliers finished: 2020-08-05T19:50:01.757908 Resources for stage outliers : 0.676 s worker CPU, 0.752 s worker Wall, Wall: 1.341 s, Cores in use: 1.28, Total efficiency (on 16 cores): 8.0 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:50:01.777571 Stage halos : Wall: 0.83 s, CPU: 0.81 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 233 MB, VmData: 260 MB, maxrss: 0.238624 MB Stage halos finished: 2020-08-05T19:50:02.612162 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.836 s, Cores in use: 0.98, Total efficiency (on 16 cores): 6.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T19:50:02.635949 PSF sigmas in coadd pixels: ['3.47', '2.30', '2.23'] Using average PSF sigma 2.6660326936969154 Gaussian PSF norm 0.10581070233715138 vs pixelized 0.08075449 average tim pixel scale / coadd scale: 1.7290613223009437 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.67', '1.83', '2.01'] Using average PSF sigma 2.170793607651196 Gaussian PSF norm 0.12995007483880763 vs pixelized 0.11714631 average tim pixel scale / coadd scale: 1.7292568733816156 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-r.fits.fz PSF sigmas in coadd pixels: ['2.72', '1.78', '1.75'] Using average PSF sigma 2.0817441364226625 Gaussian PSF norm 0.1355088681833105 vs pixelized 0.118383765 average tim pixel scale / coadd scale: 0.9909674648785579 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.16 s, CPU: 0.13 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 234 MB, VmData: 260 MB, maxrss: 0.240088 MB Stage fit_on_coadds finished: 2020-08-05T19:50:02.797593 Resources for stage fit_on_coadds : 0.139 s worker CPU, 0.139 s worker Wall, Wall: 0.163 s, Cores in use: 1.67, Total efficiency (on 16 cores): 10.4 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-05T19:50:02.810329 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 20 potential peaks Of 20 potential peaks: 4 in veto map, 2 cut by saddle test, 0 cut by aper test, 14 kept SED z : 14 new peaks Found 22 potential peaks Of 22 potential peaks: 17 in veto map, 2 cut by saddle test, 1 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 17 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 29 potential peaks Of 29 potential peaks: 24 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept SED Flat : 3 new peaks Found 30 potential peaks Of 30 potential peaks: 25 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 23 new sources Median coadd sky for g : 0.0001318332 Median coadd sky for r : 0.00030450846 Median coadd sky for z : -9.575677e-05 Stage srcs : Wall: 0.57 s, CPU: 0.55 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 235 MB, VmData: 260 MB, maxrss: 0.240588 MB Stage srcs finished: 2020-08-05T19:50:03.384485 Resources for stage srcs : 0.066 s worker CPU, 0.066 s worker Wall, Wall: 0.575 s, Cores in use: 1.06, Total efficiency (on 16 cores): 6.7 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:50:03.398212 Blob 1 of 16, id: 4, sources: 7, size: 156x114, npix 10961, brick X: 68,224, Y: 57,171, one pixel: 203 57 Blob 2 of 16, id: 2, sources: 3, size: 69x52, npix 2151, brick X: 90,159, Y: 17,69, one pixel: 145 17 Blob 3 of 16, id: 14, sources: 2, size: 47x40, npix 1476, brick X: 139,186, Y: 205,245, one pixel: 161 205 Blob 4 of 16, id: 5, sources: 3, size: 32x64, npix 1080, brick X: 35,67, Y: 89,153, one pixel: 46 89 Blob 5 of 16, id: 3, sources: 2, size: 42x29, npix 768, brick X: 155,197, Y: 42,71, one pixel: 164 42 Blob 6 of 16, id: 15, sources: 2, size: 28x34, npix 766, brick X: 217,245, Y: 211,245, one pixel: 227 211 Blob 7 of 16, id: 6, sources: 1, size: 32x33, npix 753, brick X: 0,32, Y: 110,143, one pixel: 14 110 Blob 8 of 16, id: 13, sources: 1, size: 28x28, npix 525, brick X: 185,213, Y: 182,210, one pixel: 197 182 Blob 9 of 16, id: 7, sources: 1, size: 27x26, npix 465, brick X: 184,211, Y: 153,179, one pixel: 196 153 Blob 10 of 16, id: 10, sources: 1, size: 25x26, npix 416, brick X: 2,27, Y: 175,201, one pixel: 12 175 Blob 11 of 16, id: 12, sources: 1, size: 22x26, npix 361, brick X: 117,139, Y: 181,207, one pixel: 126 181 Blob 12 of 16, id: 1, sources: 1, size: 21x27, npix 351, brick X: 184,205, Y: 12,39, one pixel: 195 12 Blob 13 of 16, id: 0, sources: 1, size: 22x22, npix 294, brick X: 57,79, Y: 5,27, one pixel: 66 5 Blob 14 of 16, id: 9, sources: 1, size: 15x24, npix 264, brick X: 230,245, Y: 169,193, one pixel: 239 169 Blob 15 of 16, id: 8, sources: 1, size: 20x20, npix 219, brick X: 171,191, Y: 169,189, one pixel: 180 169 Blob 16 of 16, id: 11, sources: 1, size: 19x19, npix 217, brick X: 98,117, Y: 176,195, one pixel: 106 176 Blob 15 finished, total: Wall: 0.09 s, CPU: 0.09 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 86 MB, VmData: 138 MB, maxrss: 0.088412 MB Blob 12 finished, total: Wall: 0.10 s, CPU: 0.09 s, VmPeak: 809 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149484 MB Blob 16 finished, total: Wall: 0.09 s, CPU: 0.09 s, VmPeak: 809 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149716 MB Blob 11 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09264 MB Blob 13 finished, total: Wall: 0.31 s, CPU: 0.31 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092284 MB Blob 10 finished, total: Wall: 0.45 s, CPU: 0.45 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.0946 MB Blob 14 finished, total: Wall: 0.46 s, CPU: 0.46 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092308 MB Blob 9 finished, total: Wall: 0.57 s, CPU: 0.57 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 89 MB, VmData: 137 MB, maxrss: 0.091628 MB Blob 8 finished, total: Wall: 0.66 s, CPU: 0.66 s, VmPeak: 804 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149352 MB Measuring 1 iterative sources Blob 5 finished, total: Wall: 0.79 s, CPU: 0.79 s, VmPeak: 810 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150164 MB Blob 4 finished, total: Wall: 0.84 s, CPU: 0.84 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149796 MB Blob 3 finished, total: Wall: 0.89 s, CPU: 0.89 s, VmPeak: 810 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149716 MB Blob 7 finished, total: Wall: 0.88 s, CPU: 0.88 s, VmPeak: 806 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149196 MB Measuring 1 iterative sources Blob 6 finished, total: Wall: 1.34 s, CPU: 1.34 s, VmPeak: 805 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150604 MB Measuring 2 iterative sources Blob 2 finished, total: Wall: 1.74 s, CPU: 1.74 s, VmPeak: 810 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.1505 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.21853, ee1=-0.0405982, ee2=-0.0748727 Measuring 3 iterative sources Blob 1 finished, total: Wall: 6.51 s, CPU: 6.51 s, VmPeak: 684 MB, VmSize: 684 MB, VmRSS: 111 MB, VmData: 155 MB, maxrss: 0.113716 MB Old catalog: 31 New catalog: 33 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/blobs-custom-272585p52135.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/all-models-custom-272585p52135.fits Stage fitblobs : Wall: 6.90 s, CPU: 0.38 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 235 MB, VmData: 260 MB, maxrss: 0.240752 MB Stage fitblobs finished: 2020-08-05T19:50:10.301096 Resources for stage fitblobs : 16.236 s worker CPU, 16.344 s worker Wall, Wall: 6.904 s, Cores in use: 2.41, Total efficiency (on 16 cores): 15.0 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:50:10.316459 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-resid.jpg Stage coadds : Wall: 3.12 s, CPU: 1.02 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 235 MB, VmData: 260 MB, maxrss: 0.24108 MB Stage coadds finished: 2020-08-05T19:50:13.437928 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 9.231 s worker CPU, 16.255 s worker Wall, Wall: 3.123 s, Cores in use: 3.28, Total efficiency (on 16 cores): 20.5 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T19:50:13.457106 Cut to 1 unWISE tiles Reading WISE tile 2717p514 band 2 Reading WISE tile 2717p514 band 4 Reading WISE tile 2717p514 band 1 Reading WISE tile 2717p514 band 3 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 809 MB, VmSize: 809 MB, VmRSS: 150 MB, VmData: 177 MB, maxrss: 0.197012 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 810 MB, VmSize: 803 MB, VmRSS: 144 MB, VmData: 171 MB, maxrss: 0.206956 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 808 MB, VmSize: 806 MB, VmRSS: 147 MB, VmData: 174 MB, maxrss: 0.210152 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 813 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 171 MB, maxrss: 0.209292 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 5.38 s, CPU: 0.98 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 236 MB, VmData: 260 MB, maxrss: 0.241656 MB Stage wise_forced finished: 2020-08-05T19:50:18.837683 Resources for stage wise_forced : 3.149 s worker CPU, 8.161 s worker Wall, Wall: 5.382 s, Cores in use: 0.77, Total efficiency (on 16 cores): 4.8 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:50:18.854557 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/tractor-i/cus/tractor-custom-272585p52135.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/tractor/cus/tractor-custom-272585p52135.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/tractor/cus/brick-custom-272585p52135.sha256sum Stage writecat : Wall: 1.76 s, CPU: 1.24 s, VmPeak: 1142 MB, VmSize: 1014 MB, VmRSS: 238 MB, VmData: 260 MB, maxrss: 0.374536 MB Stage writecat finished: 2020-08-05T19:50:20.616097 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.763 s, Cores in use: 0.70, Total efficiency (on 16 cores): 4.4 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-writecat.p All done: 34.035 s worker CPU, 87.295 s worker Wall, Wall: 36.970 s, Cores in use: 1.23, Total efficiency (on 16 cores): 7.7 % Finished galaxy PGC2407584 in 0.669 minutes. Started working on galaxy PGC2407584 at Wed Aug 5 20:38:32 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 272.585736 52.135887100000005 --width 245 --height 245 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T20:38:33.846771 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '272.585736', '52.135887100000005', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 272.585736 52.135887100000005 --width 245 --height 245 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 272.585736 52.135887100000005 Starting process 30574 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 30575 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 30576 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 30577 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 30578 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059472 MB Starting process 30579 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059484 MB Starting process 30580 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059484 MB Starting process 30581 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 30582 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 30583 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 30584 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 30585 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 30586 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 30587 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 30588 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 30589 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 30590 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 30591 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 30592 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 30593 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 30594 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 30595 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 30596 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 30597 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 30598 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 30599 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 30600 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 30601 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 30602 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 30604 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 30605 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 30603 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T20:38:36.704707 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 9 CCDs: k4m_160526_091718_ooi_zd_ls9-CCD2 z expnum 91635 exptime 119.103 propid 2016A-0453 seeing 1.67 MJD 57534.387 object MzLS_658212_z mosaic/CP/V4.3/CP20160525/k4m_160526_091718_ooi_zd_ls9.fits.fz k4m_170901_071139_ooi_zd_ls9-CCD1 z expnum 137641 exptime 156.103 propid 2016A-0453 seeing 1.09 MJD 57997.300 object MzLS_658983_z mosaic/CP/V4.3/CP20170831/k4m_170901_071139_ooi_zd_ls9.fits.fz k4m_160520_094855_ooi_zd_ls9-CCD2 z expnum 90515 exptime 80.103 propid 2016A-0453 seeing 1.08 MJD 57528.409 object MzLS_657439_z mosaic/CP/V4.3/CP20160519/k4m_160520_094855_ooi_zd_ls9.fits.fz ksb_160707_091929_ooi_g_ls9-CCD2 g expnum 75760094 exptime 229.0 propid BASS seeing 2.13 MJD 57576.389 object 29452 90prime/CP/V2.3/CP20160706/ksb_160707_091929_ooi_g_ls9.fits.fz ksb_160712_064128_ooi_r_ls9-CCD2 r expnum 75810082 exptime 138.0 propid BASS seeing 1.64 MJD 57581.279 object 29452 90prime/CP/V2.3/CP20160711/ksb_160712_064128_ooi_r_ls9.fits.fz ksb_170624_062822_ooi_r_ls9-CCD3 r expnum 79280073 exptime 76.476 propid BASS seeing 1.13 MJD 57928.270 object 29463 90prime/CP/V2.3/CP20170623/ksb_170624_062822_ooi_r_ls9.fits.fz ksb_170618_061845_ooi_g_ls9-CCD3 g expnum 79220074 exptime 121.071 propid BASS seeing 1.42 MJD 57922.263 object 29463 90prime/CP/V2.3/CP20170617/ksb_170618_061845_ooi_g_ls9.fits.fz ksb_160603_083839_ooi_g_ls9-CCD3 g expnum 75420121 exptime 87.9 propid BASS seeing 1.37 MJD 57542.360 object 29461 90prime/CP/V2.3/CP20160602/ksb_160603_083839_ooi_g_ls9.fits.fz ksb_170405_113253_ooi_r_ls9-CCD3 r expnum 78480180 exptime 56.095 propid BASS seeing 1.24 MJD 57848.481 object 29461 90prime/CP/V2.3/CP20170404/ksb_170405_113253_ooi_r_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1613 1863 and amp range 0 2005 and image y 2225 2472 and amp range 2040 4078 Range in image: x 0 251 , y 0 248 (with image size 251 x 248) dzp 0.014419514 -> scaling image by 1.0133694453441033 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 79280073 Faked up EXPNUM 79280073 Faked up EXPNUM 79280073 90prime get_fwhm: self.fwhm = 2.49 Found 4 relevant rows in amp-corrections file. Found overlap: image x 981 1124 and amp range 0 2015 and image y 465 608 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp -0.043123975 -> scaling image by 0.961059796099719 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 78480180 Faked up EXPNUM 78480180 Faked up EXPNUM 78480180 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2969 3111 and amp range 2016 4031 and image y 2452 2595 and amp range 2048 4095 Range in image: x 0 143 , y 0 144 (with image size 143 x 144) dzp -0.035697103 -> scaling image by 0.9676563679910417 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75810082 Faked up EXPNUM 75810082 Faked up EXPNUM 75810082 90prime get_fwhm: self.fwhm = 3.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 723 867 and amp range 0 2015 and image y 911 1056 and amp range 0 2047 Range in image: x 0 145 , y 0 146 (with image size 145 x 146) dzp -0.006045583 -> scaling image by 0.9944472857505693 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75420121 Faked up EXPNUM 75420121 Faked up EXPNUM 75420121 90prime get_fwhm: self.fwhm = 3.03 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2953 3095 and amp range 2016 4031 and image y 2451 2594 and amp range 2048 4095 Range in image: x 0 143 , y 0 144 (with image size 143 x 144) dzp -0.0012536051 -> scaling image by 0.9988460533227724 Faked up EXPNUM 75760094 Faked up EXPNUM 75760094 Faked up EXPNUM 75760094 90prime get_fwhm: self.fwhm = 4.71 Found 4 relevant rows in amp-corrections file. mosaic get_fwhm: self.fwhm = 6.45 Found 4 relevant rows in amp-corrections file. Found overlap: image x 723 867 and amp range 0 2015 and image y 897 1042 and amp range 0 2047 Range in image: x 0 145 , y 0 146 (with image size 145 x 146) dzp -0.008523628 -> scaling image by 0.992180183483648 Found overlap: image x 51 297 and amp range 0 2047 and image y 3518 3764 and amp range 2040 4078 Range in image: x 0 247 , y 0 247 (with image size 247 x 247) dzp 0.008550294 -> scaling image by 1.0079062018164673 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 79220074 Faked up EXPNUM 79220074 Faked up EXPNUM 79220074 90prime get_fwhm: self.fwhm = 3.13 Found 4 relevant rows in amp-corrections file. Found overlap: image x 987 1130 and amp range 0 2015 and image y 468 611 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp -0.036130503 -> scaling image by 0.9672701794860779 mosaic get_fwhm: self.fwhm = 4.15 Swapping in SKY_JOHN values for 1 splinesky cells; 46.982933 -> 46.94934 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Found 4 relevant rows in amp-corrections file. Found overlap: image x 2751 3002 and amp range 2048 4095 and image y 2111 2361 and amp range 2040 4078 Range in image: x 0 252 , y 0 251 (with image size 252 x 251) dzp -0.009154885 -> scaling image by 0.9916034885139223 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 5.89 s, CPU: 2.28 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 207 MB, maxrss: 0.17862 MB Stage tims finished: 2020-08-05T20:38:42.597304 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.506 s worker CPU, 18.666 s worker Wall, Wall: 5.895 s, Cores in use: 1.32, Total efficiency (on 32 cores): 4.1 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T20:38:42.617910 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01378.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01275.fits Matched 7 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/reference-custom-272585p52135.fits Stage refs : Wall: 12.20 s, CPU: 4.23 s, VmPeak: 1005 MB, VmSize: 1000 MB, VmRSS: 219 MB, VmData: 249 MB, maxrss: 0.224676 MB Stage refs finished: 2020-08-05T20:38:54.813826 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 12.198 s, Cores in use: 0.35, Total efficiency (on 32 cores): 1.1 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-refs.p Running stage outliers WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage outliers at 2020-08-05T20:38:54.909315 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outlier-mask-custom-272585p52135.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-pre-custom-272585p52135.jpg Image ksb_160603_083839_ooi_g_ls9-CCD3 g : masking 163 positive outlier pixels and 0 negative outlier pixels Image ksb_170624_062822_ooi_r_ls9-CCD3 r : masking 142 positive outlier pixels and 0 negative outlier pixels Image k4m_160526_091718_ooi_zd_ls9-CCD2 z : masking 325 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outlier-mask-custom-272585p52135.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-post-custom-272585p52135.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-masked-pos-custom-272585p52135.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/outliers-masked-neg-custom-272585p52135.jpg Stage outliers : Wall: 1.06 s, CPU: 0.64 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 233 MB, VmData: 260 MB, maxrss: 0.238372 MB Stage outliers finished: 2020-08-05T20:38:55.965094 Resources for stage outliers : 0.840 s worker CPU, 1.133 s worker Wall, Wall: 1.058 s, Cores in use: 1.41, Total efficiency (on 32 cores): 4.4 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T20:38:56.036081 Stage halos : Wall: 0.62 s, CPU: 0.61 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 233 MB, VmData: 260 MB, maxrss: 0.238436 MB Stage halos finished: 2020-08-05T20:38:56.659299 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.625 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T20:38:56.775257 PSF sigmas in coadd pixels: ['3.47', '2.30', '2.23'] Using average PSF sigma 2.6660326936969154 Gaussian PSF norm 0.10581070233715138 vs pixelized 0.08075449 average tim pixel scale / coadd scale: 1.7290613223009437 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.67', '1.83', '2.01'] Using average PSF sigma 2.170793607651196 Gaussian PSF norm 0.12995007483880763 vs pixelized 0.11714631 average tim pixel scale / coadd scale: 1.7292568733816156 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-r.fits.fz PSF sigmas in coadd pixels: ['2.72', '1.78', '1.75'] Using average PSF sigma 2.0817441364226625 Gaussian PSF norm 0.1355088681833105 vs pixelized 0.11838376 average tim pixel scale / coadd scale: 0.9909674648785579 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.16 s, CPU: 0.14 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 234 MB, VmData: 260 MB, maxrss: 0.24004 MB Stage fit_on_coadds finished: 2020-08-05T20:38:56.939927 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.132 s worker CPU, 0.132 s worker Wall, Wall: 0.167 s, Cores in use: 1.66, Total efficiency (on 32 cores): 5.2 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T20:38:56.952791 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 20 potential peaks Of 20 potential peaks: 4 in veto map, 2 cut by saddle test, 0 cut by aper test, 14 kept SED z : 14 new peaks Found 22 potential peaks Of 22 potential peaks: 17 in veto map, 2 cut by saddle test, 1 cut by aper test, 2 kept SED r : 2 new peaks Found 22 potential peaks Of 22 potential peaks: 17 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 29 potential peaks Of 29 potential peaks: 24 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept SED Flat : 3 new peaks Found 30 potential peaks Of 30 potential peaks: 25 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 23 new sources Median coadd sky for g : 0.0001318332 Median coadd sky for r : 0.0003045085 Median coadd sky for z : -9.5756855e-05 Stage srcs : Wall: 1.63 s, CPU: 1.57 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 235 MB, VmData: 260 MB, maxrss: 0.24054 MB Stage srcs finished: 2020-08-05T20:38:58.581654 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage srcs : 0.086 s worker CPU, 0.087 s worker Wall, Wall: 1.631 s, Cores in use: 1.02, Total efficiency (on 32 cores): 3.2 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T20:38:58.598719 Blob 1 of 16, id: 4, sources: 7, size: 156x114, npix 10961, brick X: 68,224, Y: 57,171, one pixel: 203 57 Blob 2 of 16, id: 2, sources: 3, size: 69x52, npix 2151, brick X: 90,159, Y: 17,69, one pixel: 145 17 Blob 3 of 16, id: 14, sources: 2, size: 47x40, npix 1476, brick X: 139,186, Y: 205,245, one pixel: 161 205 Blob 4 of 16, id: 5, sources: 3, size: 32x64, npix 1080, brick X: 35,67, Y: 89,153, one pixel: 46 89 Blob 5 of 16, id: 3, sources: 2, size: 42x29, npix 768, brick X: 155,197, Y: 42,71, one pixel: 164 42 Blob 6 of 16, id: 15, sources: 2, size: 28x34, npix 766, brick X: 217,245, Y: 211,245, one pixel: 227 211 Blob 7 of 16, id: 6, sources: 1, size: 32x33, npix 753, brick X: 0,32, Y: 110,143, one pixel: 14 110 Blob 8 of 16, id: 13, sources: 1, size: 28x28, npix 525, brick X: 185,213, Y: 182,210, one pixel: 197 182 Blob 9 of 16, id: 7, sources: 1, size: 27x26, npix 465, brick X: 184,211, Y: 153,179, one pixel: 196 153 Blob 10 of 16, id: 10, sources: 1, size: 25x26, npix 416, brick X: 2,27, Y: 175,201, one pixel: 12 175 Blob 11 of 16, id: 12, sources: 1, size: 22x26, npix 361, brick X: 117,139, Y: 181,207, one pixel: 126 181 Blob 12 of 16, id: 1, sources: 1, size: 21x27, npix 351, brick X: 184,205, Y: 12,39, one pixel: 195 12 Blob 13 of 16, id: 0, sources: 1, size: 22x22, npix 294, brick X: 57,79, Y: 5,27, one pixel: 66 5 Blob 14 of 16, id: 9, sources: 1, size: 15x24, npix 264, brick X: 230,245, Y: 169,193, one pixel: 239 169 Blob 15 of 16, id: 8, sources: 1, size: 20x20, npix 219, brick X: 171,191, Y: 169,189, one pixel: 180 169 Blob 16 of 16, id: 11, sources: 1, size: 19x19, npix 217, brick X: 98,117, Y: 176,195, one pixel: 106 176 Blob 12 finished, total: Wall: 0.07 s, CPU: 0.06 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 85 MB, VmData: 137 MB, maxrss: 0.087032 MB Blob 15 finished, total: Wall: 0.10 s, CPU: 0.08 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 86 MB, VmData: 139 MB, maxrss: 0.088032 MB Blob 16 finished, total: Wall: 0.10 s, CPU: 0.07 s, VmPeak: 813 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149524 MB Blob 11 finished, total: Wall: 0.26 s, CPU: 0.24 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 89 MB, VmData: 137 MB, maxrss: 0.0913 MB Blob 13 finished, total: Wall: 0.28 s, CPU: 0.26 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.09278 MB Blob 10 finished, total: Wall: 0.38 s, CPU: 0.35 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092044 MB Blob 9 finished, total: Wall: 0.41 s, CPU: 0.39 s, VmPeak: 809 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149792 MB Blob 14 finished, total: Wall: 0.43 s, CPU: 0.42 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09256 MB Blob 8 finished, total: Wall: 0.45 s, CPU: 0.43 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093036 MB Measuring 1 iterative sources Blob 7 finished, total: Wall: 0.70 s, CPU: 0.70 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092784 MB Blob 4 finished, total: Wall: 0.70 s, CPU: 0.67 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.0928 MB Blob 5 finished, total: Wall: 0.73 s, CPU: 0.70 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.09304 MB Blob 3 finished, total: Wall: 0.77 s, CPU: 0.76 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 95 MB, VmData: 142 MB, maxrss: 0.096828 MB Measuring 1 iterative sources Blob 6 finished, total: Wall: 1.32 s, CPU: 1.31 s, VmPeak: 805 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150364 MB Measuring 2 iterative sources Blob 2 finished, total: Wall: 1.80 s, CPU: 1.78 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.09454 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.21853, ee1=-0.0405983, ee2=-0.0748726 Measuring 3 iterative sources Blob 1 finished, total: Wall: 6.69 s, CPU: 6.69 s, VmPeak: 684 MB, VmSize: 684 MB, VmRSS: 111 MB, VmData: 155 MB, maxrss: 0.113584 MB Old catalog: 31 New catalog: 33 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/blobs-custom-272585p52135.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/metrics/cus/all-models-custom-272585p52135.fits Stage fitblobs : Wall: 8.81 s, CPU: 2.05 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 235 MB, VmData: 260 MB, maxrss: 0.240704 MB Stage fitblobs finished: 2020-08-05T20:39:07.409937 Resources for stage fitblobs : 15.152 s worker CPU, 16.598 s worker Wall, Wall: 8.813 s, Cores in use: 1.95, Total efficiency (on 32 cores): 6.1 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T20:39:07.428747 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-resid.jpg Stage coadds : Wall: 4.30 s, CPU: 1.50 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 235 MB, VmData: 260 MB, maxrss: 0.241028 MB Stage coadds finished: 2020-08-05T20:39:11.732550 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 25.583 s worker CPU, 51.369 s worker Wall, Wall: 4.306 s, Cores in use: 6.29, Total efficiency (on 32 cores): 19.7 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T20:39:11.753502 Cut to 1 unWISE tiles Reading WISE tile 2717p514 band 1 Reading WISE tile 2717p514 band 2 Reading WISE tile 2717p514 band 4 Reading WISE tile 2717p514 band 3 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 808 MB, VmSize: 808 MB, VmRSS: 148 MB, VmData: 176 MB, maxrss: 0.188772 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 804 MB, VmSize: 802 MB, VmRSS: 143 MB, VmData: 174 MB, maxrss: 0.205556 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 809 MB, VmSize: 802 MB, VmRSS: 144 MB, VmData: 174 MB, maxrss: 0.206216 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 813 MB, VmSize: 803 MB, VmRSS: 143 MB, VmData: 171 MB, maxrss: 0.206604 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 9.52 s, CPU: 2.01 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 236 MB, VmData: 260 MB, maxrss: 0.241604 MB Stage wise_forced finished: 2020-08-05T20:39:21.274635 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.101 s worker CPU, 11.264 s worker Wall, Wall: 9.523 s, Cores in use: 0.54, Total efficiency (on 32 cores): 1.7 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-wise_forced.p Running stage writecat WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage writecat at 2020-08-05T20:39:21.295749 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/tractor-i/cus/tractor-custom-272585p52135.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/tractor/cus/tractor-custom-272585p52135.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/tractor/cus/brick-custom-272585p52135.sha256sum Stage writecat : Wall: 2.22 s, CPU: 1.53 s, VmPeak: 1142 MB, VmSize: 1014 MB, VmRSS: 238 MB, VmData: 260 MB, maxrss: 0.374484 MB Stage writecat finished: 2020-08-05T20:39:23.511283 Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 2.218 s, Cores in use: 0.69, Total efficiency (on 32 cores): 2.2 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/272/PGC2407584/PGC2407584-largegalaxy-writecat.p All done: 50.407 s worker CPU, 99.257 s worker Wall, Wall: 46.843 s, Cores in use: 1.43, Total efficiency (on 32 cores): 4.5 % Finished galaxy PGC2407584 in 0.858 minutes. Started working on galaxy PGC2407584 at Fri Aug 7 09:20:42 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 272.585125044969 52.13573457248111 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T09:20:43.765140 legacypipe git version: DR9.5.11-watershed-3-gd509eaaf Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '272.585125044969', '52.13573457248111', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 272.585125044969 52.13573457248111 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 272.585125044969 52.13573457248111 Starting process 18767 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 18768 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 18769 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 18770 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 18771 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB Starting process 18772 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059552 MB Starting process 18773 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059556 MB Starting process 18774 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 18775 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 18776 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 18777 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 18778 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 18779 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 18780 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 18781 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 18782 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T09:20:44.350533 Keeping 9 CCDs: k4m_160526_091718_ooi_zd_ls9-CCD2 z expnum 91635 exptime 119.103 propid 2016A-0453 seeing 1.67 MJD 57534.387 object MzLS_658212_z mosaic/CP/V4.3/CP20160525/k4m_160526_091718_ooi_zd_ls9.fits.fz k4m_170901_071139_ooi_zd_ls9-CCD1 z expnum 137641 exptime 156.103 propid 2016A-0453 seeing 1.09 MJD 57997.300 object MzLS_658983_z mosaic/CP/V4.3/CP20170831/k4m_170901_071139_ooi_zd_ls9.fits.fz k4m_160520_094855_ooi_zd_ls9-CCD2 z expnum 90515 exptime 80.103 propid 2016A-0453 seeing 1.08 MJD 57528.409 object MzLS_657439_z mosaic/CP/V4.3/CP20160519/k4m_160520_094855_ooi_zd_ls9.fits.fz ksb_160707_091929_ooi_g_ls9-CCD2 g expnum 75760094 exptime 229.0 propid BASS seeing 2.13 MJD 57576.389 object 29452 90prime/CP/V2.3/CP20160706/ksb_160707_091929_ooi_g_ls9.fits.fz ksb_160712_064128_ooi_r_ls9-CCD2 r expnum 75810082 exptime 138.0 propid BASS seeing 1.64 MJD 57581.279 object 29452 90prime/CP/V2.3/CP20160711/ksb_160712_064128_ooi_r_ls9.fits.fz ksb_170624_062822_ooi_r_ls9-CCD3 r expnum 79280073 exptime 76.476 propid BASS seeing 1.13 MJD 57928.270 object 29463 90prime/CP/V2.3/CP20170623/ksb_170624_062822_ooi_r_ls9.fits.fz ksb_170618_061845_ooi_g_ls9-CCD3 g expnum 79220074 exptime 121.071 propid BASS seeing 1.42 MJD 57922.263 object 29463 90prime/CP/V2.3/CP20170617/ksb_170618_061845_ooi_g_ls9.fits.fz ksb_160603_083839_ooi_g_ls9-CCD3 g expnum 75420121 exptime 87.9 propid BASS seeing 1.37 MJD 57542.360 object 29461 90prime/CP/V2.3/CP20160602/ksb_160603_083839_ooi_g_ls9.fits.fz ksb_170405_113253_ooi_r_ls9-CCD3 r expnum 78480180 exptime 56.095 propid BASS seeing 1.24 MJD 57848.481 object 29461 90prime/CP/V2.3/CP20170404/ksb_170405_113253_ooi_r_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1611 1861 and amp range 0 2005 and image y 2219 2467 and amp range 2040 4078 Range in image: x 0 251 , y 0 249 (with image size 251 x 249) dzp 0.014419514 -> scaling image by 1.0133694453441033 mosaic get_fwhm: self.fwhm = 4.15 Swapping in SKY_JOHN values for 1 splinesky cells; 46.982933 -> 46.94934 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2749 3000 and amp range 2048 4095 and image y 2106 2356 and amp range 2040 4078 Range in image: x 0 252 , y 0 251 (with image size 252 x 251) dzp -0.009154885 -> scaling image by 0.9916034885139223 mosaic get_fwhm: self.fwhm = 6.45 Found 4 relevant rows in amp-corrections file. Found overlap: image x 49 295 and amp range 0 2047 and image y 3513 3759 and amp range 2040 4078 Range in image: x 0 247 , y 0 247 (with image size 247 x 247) dzp 0.008550294 -> scaling image by 1.0079062018164673 Faked up EXPNUM 79220074 Faked up EXPNUM 79220074 Faked up EXPNUM 79220074 90prime get_fwhm: self.fwhm = 3.13 Found 4 relevant rows in amp-corrections file. Found overlap: image x 988 1131 and amp range 0 2015 and image y 471 614 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp -0.036130503 -> scaling image by 0.9672701794860779 Faked up EXPNUM 75420121 Faked up EXPNUM 75420121 Faked up EXPNUM 75420121 90prime get_fwhm: self.fwhm = 3.03 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2954 3096 and amp range 2016 4031 and image y 2454 2597 and amp range 2048 4095 Range in image: x 0 143 , y 0 144 (with image size 143 x 144) dzp -0.0012536051 -> scaling image by 0.9988460533227724 Faked up EXPNUM 75760094 Faked up EXPNUM 75760094 Faked up EXPNUM 75760094 90prime get_fwhm: self.fwhm = 4.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 724 868 and amp range 0 2015 and image y 900 1045 and amp range 0 2047 Range in image: x 0 145 , y 0 146 (with image size 145 x 146) dzp -0.008523628 -> scaling image by 0.992180183483648 Faked up EXPNUM 75810082 Faked up EXPNUM 75810082 Faked up EXPNUM 75810082 90prime get_fwhm: self.fwhm = 3.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 724 868 and amp range 0 2015 and image y 914 1059 and amp range 0 2047 Range in image: x 0 145 , y 0 146 (with image size 145 x 146) dzp -0.006045583 -> scaling image by 0.9944472857505693 Faked up EXPNUM 78480180 Faked up EXPNUM 78480180 Faked up EXPNUM 78480180 90prime get_fwhm: self.fwhm = 2.73 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2970 3112 and amp range 2016 4031 and image y 2455 2598 and amp range 2048 4095 Range in image: x 0 143 , y 0 144 (with image size 143 x 144) dzp -0.035697103 -> scaling image by 0.9676563679910417 Faked up EXPNUM 79280073 Faked up EXPNUM 79280073 Faked up EXPNUM 79280073 90prime get_fwhm: self.fwhm = 2.49 Found 4 relevant rows in amp-corrections file. Found overlap: image x 982 1125 and amp range 0 2015 and image y 468 611 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp -0.043123975 -> scaling image by 0.961059796099719 Stage tims : Wall: 80.60 s, CPU: 0.98 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 172 MB, VmData: 206 MB, maxrss: 0.176124 MB Stage tims finished: 2020-08-07T09:22:04.949305 Resources for stage tims : 4.382 s worker CPU, 447.084 s worker Wall, Wall: 80.600 s, Cores in use: 0.07, Total efficiency (on 16 cores): 0.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T09:22:05.054275 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01378.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01275.fits Matched 7 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/reference-custom-272585p52135.fits Stage refs : Wall: 4.86 s, CPU: 3.90 s, VmPeak: 999 MB, VmSize: 994 MB, VmRSS: 217 MB, VmData: 248 MB, maxrss: 0.222424 MB Stage refs finished: 2020-08-07T09:22:09.913492 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 4.861 s, Cores in use: 0.80, Total efficiency (on 16 cores): 5.0 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T09:22:09.939063 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/outlier-mask-custom-272585p52135.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/outliers-pre-custom-272585p52135.jpg Image ksb_160603_083839_ooi_g_ls9-CCD3 g : masking 166 positive outlier pixels and 0 negative outlier pixels Image ksb_170624_062822_ooi_r_ls9-CCD3 r : masking 143 positive outlier pixels and 0 negative outlier pixels Image k4m_160526_091718_ooi_zd_ls9-CCD2 z : masking 320 positive outlier pixels and 302 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/outlier-mask-custom-272585p52135.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/outliers-post-custom-272585p52135.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/outliers-masked-pos-custom-272585p52135.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/outliers-masked-neg-custom-272585p52135.jpg Stage outliers : Wall: 0.81 s, CPU: 0.54 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 230 MB, VmData: 259 MB, maxrss: 0.235668 MB Stage outliers finished: 2020-08-07T09:22:10.746575 Resources for stage outliers : 0.623 s worker CPU, 0.680 s worker Wall, Wall: 0.809 s, Cores in use: 1.44, Total efficiency (on 16 cores): 9.0 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T09:22:10.849817 Stage halos : Wall: 0.31 s, CPU: 0.30 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 230 MB, VmData: 259 MB, maxrss: 0.235868 MB Stage halos finished: 2020-08-07T09:22:11.162378 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.314 s, Cores in use: 0.97, Total efficiency (on 16 cores): 6.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T09:22:11.292484 PSF sigmas in coadd pixels: ['3.47', '2.30', '2.23'] Using average PSF sigma 2.6660326936969154 Gaussian PSF norm 0.10581070233715138 vs pixelized 0.08075363 average tim pixel scale / coadd scale: 1.7290613223009437 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.67', '1.83', '2.01'] Using average PSF sigma 2.170793607651196 Gaussian PSF norm 0.12995007483880763 vs pixelized 0.11713946 average tim pixel scale / coadd scale: 1.7292568733816156 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-r.fits.fz PSF sigmas in coadd pixels: ['2.72', '1.78', '1.75'] Using average PSF sigma 2.0817441364226625 Gaussian PSF norm 0.1355088681833105 vs pixelized 0.118383296 average tim pixel scale / coadd scale: 0.9909674648785579 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-copsf-z.fits.fz Stage fit_on_coadds : Wall: 1.05 s, CPU: 0.14 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 232 MB, VmData: 259 MB, maxrss: 0.237392 MB Stage fit_on_coadds finished: 2020-08-07T09:22:12.339719 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.160 s worker CPU, 0.160 s worker Wall, Wall: 1.049 s, Cores in use: 0.29, Total efficiency (on 16 cores): 1.8 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-07T09:22:12.349819 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 20 potential peaks Of 20 potential peaks: 5 in veto map, 2 cut by saddle test, 0 cut by aper test, 13 kept SED z : 13 new peaks Found 21 potential peaks Of 21 potential peaks: 17 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 23 potential peaks Of 23 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 30 potential peaks Of 30 potential peaks: 24 in veto map, 2 cut by saddle test, 0 cut by aper test, 4 kept SED Flat : 4 new peaks Found 32 potential peaks Of 32 potential peaks: 26 in veto map, 4 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 23 new sources Median coadd sky for g : 0.00012660158 Median coadd sky for r : 0.00031703603 Median coadd sky for z : -9.217054e-05 Stage srcs : Wall: 0.48 s, CPU: 0.44 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 232 MB, VmData: 259 MB, maxrss: 0.237884 MB Stage srcs finished: 2020-08-07T09:22:12.832898 Resources for stage srcs : 0.079 s worker CPU, 0.079 s worker Wall, Wall: 0.485 s, Cores in use: 1.08, Total efficiency (on 16 cores): 6.8 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T09:22:12.843636 Blob 1 of 18, id: 5, sources: 6, size: 156x115, npix 10955, brick X: 63,219, Y: 59,174, one pixel: 198 59 Blob 2 of 18, id: 2, sources: 3, size: 69x52, npix 2166, brick X: 85,154, Y: 19,71, one pixel: 140 19 Blob 3 of 18, id: 16, sources: 2, size: 47x38, npix 1397, brick X: 134,181, Y: 207,245, one pixel: 156 207 Blob 4 of 18, id: 17, sources: 2, size: 33x33, npix 891, brick X: 212,245, Y: 212,245, one pixel: 235 212 Blob 5 of 18, id: 4, sources: 2, size: 43x29, npix 754, brick X: 149,192, Y: 44,73, one pixel: 159 44 Blob 6 of 18, id: 6, sources: 2, size: 29x40, npix 705, brick X: 30,59, Y: 92,132, one pixel: 39 92 Blob 7 of 18, id: 7, sources: 1, size: 27x32, npix 630, brick X: 0,27, Y: 113,145, one pixel: 7 113 Blob 8 of 18, id: 15, sources: 1, size: 28x28, npix 513, brick X: 180,208, Y: 184,212, one pixel: 193 184 Blob 9 of 18, id: 9, sources: 1, size: 27x27, npix 459, brick X: 179,206, Y: 155,182, one pixel: 191 155 Blob 10 of 18, id: 10, sources: 1, size: 21x28, npix 418, brick X: 224,245, Y: 169,197, one pixel: 236 169 Blob 11 of 18, id: 12, sources: 1, size: 22x26, npix 383, brick X: 0,22, Y: 177,203, one pixel: 8 177 Blob 12 of 18, id: 14, sources: 1, size: 23x26, npix 369, brick X: 111,134, Y: 183,209, one pixel: 122 183 Blob 13 of 18, id: 1, sources: 1, size: 20x27, npix 359, brick X: 179,199, Y: 14,41, one pixel: 190 14 Blob 14 of 18, id: 8, sources: 1, size: 24x23, npix 352, brick X: 38,62, Y: 132,155, one pixel: 48 132 Blob 15 of 18, id: 0, sources: 1, size: 22x22, npix 294, brick X: 52,74, Y: 7,29, one pixel: 61 7 Blob 16 of 18, id: 3, sources: 1, size: 16x22, npix 241, brick X: 229,245, Y: 40,62, one pixel: 238 40 Blob 17 of 18, id: 11, sources: 1, size: 20x20, npix 219, brick X: 166,186, Y: 171,191, one pixel: 175 171 Blob 18 of 18, id: 13, sources: 1, size: 19x19, npix 217, brick X: 93,112, Y: 178,197, one pixel: 101 178 Blob 13 finished, total: Wall: 0.10 s, CPU: 0.09 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 90 MB, VmData: 142 MB, maxrss: 0.091784 MB Blob 17 finished, total: Wall: 0.09 s, CPU: 0.09 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 90 MB, VmData: 142 MB, maxrss: 0.091952 MB Blob 14 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 809 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145908 MB Blob 12 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.145944 MB Blob 18 finished, total: Wall: 0.09 s, CPU: 0.09 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 90 MB, VmData: 142 MB, maxrss: 0.091952 MB Blob 15 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 137 MB, maxrss: 0.091784 MB Blob 11 finished, total: Wall: 0.41 s, CPU: 0.41 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146348 MB Blob 9 finished, total: Wall: 0.46 s, CPU: 0.46 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093016 MB Blob 16 finished, total: Wall: 0.57 s, CPU: 0.57 s, VmPeak: 806 MB, VmSize: 724 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145332 MB Blob 10 finished, total: Wall: 0.62 s, CPU: 0.62 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.092848 MB Measuring 1 iterative sources Blob 8 finished, total: Wall: 0.69 s, CPU: 0.69 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146172 MB Blob 6 finished, total: Wall: 0.74 s, CPU: 0.74 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09242 MB Blob 7 finished, total: Wall: 0.83 s, CPU: 0.83 s, VmPeak: 809 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.145968 MB Blob 3 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145524 MB Blob 5 finished, total: Wall: 0.90 s, CPU: 0.89 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.094896 MB Measuring 1 iterative sources Blob 4 finished, total: Wall: 1.50 s, CPU: 1.50 s, VmPeak: 805 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146908 MB Measuring 2 iterative sources Blob 2 finished, total: Wall: 1.85 s, CPU: 1.84 s, VmPeak: 804 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.1459 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=0.633837, ee1=-0.0446822, ee2=-0.0720099 Measuring 2 iterative sources Blob 1 finished, total: Wall: 5.59 s, CPU: 5.59 s, VmPeak: 687 MB, VmSize: 687 MB, VmRSS: 112 MB, VmData: 155 MB, maxrss: 0.1143 MB Old catalog: 31 New catalog: 31 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/blobs-custom-272585p52135.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/metrics/cus/all-models-custom-272585p52135.fits Stage fitblobs : Wall: 5.93 s, CPU: 0.37 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 232 MB, VmData: 259 MB, maxrss: 0.238048 MB Stage fitblobs finished: 2020-08-07T09:22:18.769368 Resources for stage fitblobs : 16.379 s worker CPU, 16.476 s worker Wall, Wall: 5.927 s, Cores in use: 2.83, Total efficiency (on 16 cores): 17.7 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-07T09:22:18.780996 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-resid.jpg Stage coadds : Wall: 14.06 s, CPU: 0.58 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 233 MB, VmData: 259 MB, maxrss: 0.238372 MB Stage coadds finished: 2020-08-07T09:22:32.837342 Resources for stage coadds : 9.006 s worker CPU, 10.360 s worker Wall, Wall: 14.058 s, Cores in use: 0.68, Total efficiency (on 16 cores): 4.3 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-07T09:22:32.875612 Cut to 1 unWISE tiles Reading WISE tile 2717p514 band 1 Reading WISE tile 2717p514 band 2 Reading WISE tile 2717p514 band 4 Reading WISE tile 2717p514 band 3 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 805 MB, VmSize: 805 MB, VmRSS: 150 MB, VmData: 179 MB, maxrss: 0.1914 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 864 MB, VmSize: 864 MB, VmRSS: 208 MB, VmData: 238 MB, maxrss: 0.212532 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 862 MB, VmSize: 862 MB, VmRSS: 207 MB, VmData: 236 MB, maxrss: 0.212148 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/271/2717p514/unwise-2717p514-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 809 MB, VmSize: 797 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.203968 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/tmp-legacysurvey-custom-272585p52135-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 4.82 s, CPU: 0.56 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 233 MB, VmData: 259 MB, maxrss: 0.238952 MB Stage wise_forced finished: 2020-08-07T09:22:37.692060 Resources for stage wise_forced : 2.946 s worker CPU, 13.452 s worker Wall, Wall: 4.818 s, Cores in use: 0.73, Total efficiency (on 16 cores): 4.5 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-07T09:22:37.704699 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/coadd/cus/custom-272585p52135/legacysurvey-custom-272585p52135-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/tractor-i/cus/tractor-custom-272585p52135.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/tractor/cus/tractor-custom-272585p52135.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/tractor/cus/brick-custom-272585p52135.sha256sum Stage writecat : Wall: 0.71 s, CPU: 0.53 s, VmPeak: 1135 MB, VmSize: 1007 MB, VmRSS: 235 MB, VmData: 259 MB, maxrss: 0.3719 MB Stage writecat finished: 2020-08-07T09:22:38.413432 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.710 s, Cores in use: 0.75, Total efficiency (on 16 cores): 4.7 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/272/PGC2407584/PGC2407584-largegalaxy-writecat.p All done: 33.578 s worker CPU, 488.296 s worker Wall, Wall: 114.080 s, Cores in use: 0.37, Total efficiency (on 16 cores): 2.3 % Finished galaxy PGC2407584 in 1.933 minutes.