Started working on galaxy PGC2054868 at Wed Aug 5 19:38:23 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.401568 34.809166 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:38:26.357643 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.401568', '34.809166', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.401568 34.809166 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 276.401568 34.809166 Starting process 31868 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 31869 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 31870 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 31871 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 31872 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB Starting process 31873 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059496 MB Starting process 31874 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059496 MB Starting process 31875 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 31876 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 31877 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 31878 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 31879 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 31880 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 31881 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 31882 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 31883 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:38:27.945051 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: k4m_160415_115838_ooi_zd_ls9-CCD3 z expnum 82536 exptime 91.102 propid 2016A-0453 seeing 1.05 MJD 57493.499 object MzLS_579415_z mosaic/CP/V4.3/CP20160414/k4m_160415_115838_ooi_zd_ls9.fits.fz k4m_160429_110622_ooi_zd_ls9-CCD4 z expnum 85811 exptime 80.103 propid 2016A-0453 seeing 1.24 MJD 57507.463 object MzLS_577359_z mosaic/CP/V4.3/CP20160428/k4m_160429_110622_ooi_zd_ls9.fits.fz k4m_160413_120439_ooi_zd_ls9-CCD3 z expnum 82031 exptime 80.102 propid 2016A-0453 seeing 0.82 MJD 57491.503 object MzLS_578388_z mosaic/CP/V4.3/CP20160412/k4m_160413_120439_ooi_zd_ls9.fits.fz k4m_160415_115629_ooi_zd_ls9-CCD4 z expnum 82535 exptime 85.104 propid 2016A-0453 seeing 0.99 MJD 57493.498 object MzLS_576327_z mosaic/CP/V4.3/CP20160414/k4m_160415_115629_ooi_zd_ls9.fits.fz k4m_160413_121059_ooi_zd_ls9-CCD1 z expnum 82034 exptime 81.102 propid 2016A-0453 seeing 0.83 MJD 57491.508 object MzLS_578389_z mosaic/CP/V4.3/CP20160412/k4m_160413_121059_ooi_zd_ls9.fits.fz k4m_160415_120352_ooi_zd_ls9-CCD2 z expnum 82538 exptime 149.104 propid 2016A-0453 seeing 1.02 MJD 57493.503 object MzLS_576328_z mosaic/CP/V4.3/CP20160414/k4m_160415_120352_ooi_zd_ls9.fits.fz ksb_160602_085948_ooi_g_ls9-CCD4 g expnum 75410141 exptime 119.0 propid BASS seeing 1.19 MJD 57541.375 object 08282 90prime/CP/V2.3/CP20160601/ksb_160602_085948_ooi_g_ls9.fits.fz ksb_160511_110505_ooi_r_ls9-CCD4 r expnum 75190253 exptime 60.2 propid BASS seeing 1.23 MJD 57519.462 object 08282 90prime/CP/V2.3/CP20160510/ksb_160511_110505_ooi_r_ls9.fits.fz ksb_180509_110248_ooi_r_ls9-CCD1 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.15 MJD 58247.460 object 08283 90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz ksb_160702_081612_ooi_g_ls9-CCD1 g expnum 75710100 exptime 134.2 propid BASS seeing 2.06 MJD 57571.345 object 08283 90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 3.93 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3776 4029 and amp range 2048 4095 and image y 313 564 and amp range 0 2039 Range in image: x 0 254 , y 0 252 (with image size 254 x 252) dzp 0.0072022504 -> scaling image by 1.0066555682883527 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 82470196 Faked up EXPNUM 82470196 Faked up EXPNUM 82470196 90prime get_fwhm: self.fwhm = 2.53 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3529 3669 and amp range 2016 4031 and image y 2229 2369 and amp range 2048 4095 Range in image: x 0 141 , y 0 141 (with image size 141 x 141) dzp 0.015744466 -> scaling image by 1.0146068417155982 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75410141 Faked up EXPNUM 75410141 Faked up EXPNUM 75410141 90prime get_fwhm: self.fwhm = 2.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3380 3521 and amp range 2016 4031 and image y 1761 1900 and amp range 0 2047 Range in image: x 0 142 , y 0 140 (with image size 142 x 140) dzp 0.010226944 -> scaling image by 1.0094638657706572 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 4.77 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1039 1279 and amp range 0 2047 and image y 254 494 and amp range 0 2021 Range in image: x 0 241 , y 0 241 (with image size 241 x 241) dzp 0.005139143 -> scaling image by 1.0047445456970268 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75710100 Faked up EXPNUM 75710100 Faked up EXPNUM 75710100 90prime get_fwhm: self.fwhm = 4.55 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3530 3670 and amp range 2016 4031 and image y 2259 2399 and amp range 2048 4095 Range in image: x 0 141 , y 0 141 (with image size 141 x 141) dzp 0.009677861 -> scaling image by 1.0089534840651213 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 0 147 and amp range 0 2005 and image y 1706 1951 and amp range 0 2039 Range in image: x 0 148 , y 0 246 (with image size 148 x 246) dzp -0.005856539 -> scaling image by 0.9946204500110131 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75190253 Faked up EXPNUM 75190253 Faked up EXPNUM 75190253 90prime get_fwhm: self.fwhm = 2.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3385 3526 and amp range 2016 4031 and image y 1787 1926 and amp range 0 2047 Range in image: x 0 142 , y 0 140 (with image size 142 x 140) dzp 0.0047082007 -> scaling image by 1.0043458289369953 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.19 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2617 2860 and amp range 2006 4053 and image y 374 620 and amp range 0 2039 Range in image: x 0 244 , y 0 247 (with image size 244 x 247) dzp 0.001388291 -> scaling image by 1.0012794811153327 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.81 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3814 4063 and amp range 2048 4095 and image y 4012 4054 and amp range 2022 4060 Range in image: x 0 250 , y 0 43 (with image size 250 x 43) dzp -0.0028835079 -> scaling image by 0.9973477146455455 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 13.39 s, CPU: 1.52 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 206 MB, maxrss: 0.178208 MB Stage tims finished: 2020-08-05T19:38:41.336951 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.043 s worker CPU, 53.694 s worker Wall, Wall: 13.394 s, Cores in use: 0.49, Total efficiency (on 16 cores): 3.1 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T19:38:41.358767 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/reference-custom-276401p34809.fits Stage refs : Wall: 16.62 s, CPU: 8.76 s, VmPeak: 1070 MB, VmSize: 1065 MB, VmRSS: 284 MB, VmData: 314 MB, maxrss: 0.29116 MB Stage refs finished: 2020-08-05T19:38:57.975303 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 16.618 s, Cores in use: 0.53, Total efficiency (on 16 cores): 3.3 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:38:57.997369 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outlier-mask-custom-276401p34809.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-pre-custom-276401p34809.jpg Image ksb_160702_081612_ooi_g_ls9-CCD1 g : masking 174 positive outlier pixels and 0 negative outlier pixels Image ksb_160602_085948_ooi_g_ls9-CCD4 g : masking 161 positive outlier pixels and 0 negative outlier pixels Image ksb_160511_110505_ooi_r_ls9-CCD4 r : masking 189 positive outlier pixels and 0 negative outlier pixels Image ksb_180509_110248_ooi_r_ls9-CCD1 r : masking 0 positive outlier pixels and 130 negative outlier pixels Image k4m_160413_121059_ooi_zd_ls9-CCD1 z : masking 1552 positive outlier pixels and 0 negative outlier pixels Image k4m_160415_115838_ooi_zd_ls9-CCD3 z : masking 0 positive outlier pixels and 1454 negative outlier pixels Image k4m_160415_115629_ooi_zd_ls9-CCD4 z : masking 103 positive outlier pixels and 0 negative outlier pixels Image k4m_160415_120352_ooi_zd_ls9-CCD2 z : masking 2599 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outlier-mask-custom-276401p34809.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-post-custom-276401p34809.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-masked-pos-custom-276401p34809.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-masked-neg-custom-276401p34809.jpg Stage outliers : Wall: 1.19 s, CPU: 0.77 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 296 MB, VmData: 324 MB, maxrss: 0.303392 MB Stage outliers finished: 2020-08-05T19:38:59.183129 Resources for stage outliers : 0.692 s worker CPU, 0.774 s worker Wall, Wall: 1.187 s, Cores in use: 1.23, Total efficiency (on 16 cores): 7.7 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:38:59.205789 Stage halos : Wall: 0.93 s, CPU: 0.91 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 296 MB, VmData: 324 MB, maxrss: 0.303544 MB Stage halos finished: 2020-08-05T19:39:00.135582 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.931 s, Cores in use: 0.98, Total efficiency (on 16 cores): 6.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T19:39:00.211639 PSF sigmas in coadd pixels: ['1.94', '3.35'] Using average PSF sigma 2.641509650306382 Gaussian PSF norm 0.1067930195678667 vs pixelized 0.08212437 average tim pixel scale / coadd scale: 1.7293010956522445 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.00', '1.86'] Using average PSF sigma 1.9286637892988572 Gaussian PSF norm 0.14626436880241858 vs pixelized 0.13213196 average tim pixel scale / coadd scale: 1.729900245768025 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.71', '2.01', '1.61', '1.35', '1.66'] Using average PSF sigma 1.6656765399163167 Gaussian PSF norm 0.16935748629085606 vs pixelized 0.1336563 average tim pixel scale / coadd scale: 0.9909729568380151 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.15 s, CPU: 0.12 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 299 MB, VmData: 325 MB, maxrss: 0.305756 MB Stage fit_on_coadds finished: 2020-08-05T19:39:00.366119 Resources for stage fit_on_coadds : 0.174 s worker CPU, 0.174 s worker Wall, Wall: 0.156 s, Cores in use: 1.88, Total efficiency (on 16 cores): 11.7 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-05T19:39:00.379979 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 47 potential peaks Of 47 potential peaks: 14 in veto map, 5 cut by saddle test, 0 cut by aper test, 28 kept SED z : 28 new peaks Found 49 potential peaks Of 49 potential peaks: 44 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 41 potential peaks Of 41 potential peaks: 36 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 69 potential peaks Of 69 potential peaks: 58 in veto map, 9 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 68 potential peaks Of 68 potential peaks: 58 in veto map, 8 cut by saddle test, 1 cut by aper test, 1 kept SED Red : 1 new peaks Adding 33 new sources Median coadd sky for g : 0.0006869103 Median coadd sky for r : 0.00092201383 Median coadd sky for z : -0.0007761282 Stage srcs : Wall: 0.84 s, CPU: 0.80 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 299 MB, VmData: 325 MB, maxrss: 0.306292 MB Stage srcs finished: 2020-08-05T19:39:01.222381 Resources for stage srcs : 0.080 s worker CPU, 0.080 s worker Wall, Wall: 0.844 s, Cores in use: 1.05, Total efficiency (on 16 cores): 6.5 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:39:01.236203 Blob 1 of 10, id: 5, sources: 33, size: 239x205, npix 26941, brick X: 0,239, Y: 34,239, one pixel: 5 34 Blob 2 of 10, id: 8, sources: 4, size: 48x45, npix 1421, brick X: 0,48, Y: 188,233, one pixel: 12 188 Blob 3 of 10, id: 9, sources: 3, size: 58x32, npix 1279, brick X: 52,110, Y: 207,239, one pixel: 71 207 Blob 4 of 10, id: 1, sources: 1, size: 38x38, npix 992, brick X: 106,144, Y: 0,38, one pixel: 122 0 Blob 5 of 10, id: 4, sources: 1, size: 29x32, npix 660, brick X: 65,94, Y: 33,65, one pixel: 77 33 Blob 6 of 10, id: 2, sources: 2, size: 31x22, npix 498, brick X: 208,239, Y: 0,22, one pixel: 211 0 Blob 7 of 10, id: 6, sources: 1, size: 27x27, npix 481, brick X: 208,235, Y: 81,108, one pixel: 219 81 Blob 8 of 10, id: 3, sources: 1, size: 22x24, npix 308, brick X: 178,200, Y: 24,48, one pixel: 186 24 Blob 9 of 10, id: 7, sources: 1, size: 22x23, npix 295, brick X: 9,31, Y: 151,174, one pixel: 19 151 Blob 10 of 10, id: 0, sources: 1, size: 22x17, npix 221, brick X: 84,106, Y: 0,17, one pixel: 90 0 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39198, 34.80005), pm (2.6, -6.9), parallax 0.322 with NanoMaggies: g=18.3, r=17, z=16.7 Reset brightness to NanoMaggies: g=17.1, r=16.6, z=16.4 Blob 6 finished, total: Wall: 0.15 s, CPU: 0.15 s, VmPeak: 816 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.150012 MB Blob 8 finished, total: Wall: 0.17 s, CPU: 0.17 s, VmPeak: 792 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.136988 MB Blob 10 finished, total: Wall: 0.29 s, CPU: 0.29 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.09384 MB Blob 5 finished, total: Wall: 0.41 s, CPU: 0.40 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093588 MB Blob 9 finished, total: Wall: 0.46 s, CPU: 0.45 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.094756 MB Blob 7 finished, total: Wall: 0.51 s, CPU: 0.50 s, VmPeak: 791 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.13688 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.40489, 34.81802), pm (-3.5, 16.1), parallax 1.627 with NanoMaggies: g=17.1, r=15.9, z=15.3 Reset brightness to NanoMaggies: g=17, r=15.9, z=15.3 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.40477, 34.82073), pm (-3.7, -4.8), parallax 0.068 with NanoMaggies: g=19.3, r=18.8, z=18.6 Reset brightness to NanoMaggies: g=19.3, r=18.8, z=18.6 Measuring 1 iterative sources Blob 3 finished, total: Wall: 0.74 s, CPU: 0.74 s, VmPeak: 806 MB, VmSize: 731 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148852 MB Blob 4 finished, total: Wall: 0.91 s, CPU: 0.91 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.093756 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.41505, 34.81535), pm (3.7, 13.1), parallax 0.736 with NanoMaggies: g=15.5, r=15, z=14.8 Reset brightness to NanoMaggies: g=15.5, r=15, z=14.8 Measuring 1 iterative sources Blob 2 finished, total: Wall: 1.74 s, CPU: 1.74 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150276 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.35518, 34.82551), pm (4.3, 3.8), parallax 1.896 with NanoMaggies: g=11.7, r=11.4, z=11.4 Reset brightness to NanoMaggies: g=11.7, r=11.4, z=11.4 Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.33595, ee1=0.00823371, ee2=0.152249 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39846, 34.82012), pm (0.4, -7.7), parallax 0.474 with NanoMaggies: g=18.6, r=17.9, z=13.3 Reset brightness to NanoMaggies: g=18.6, r=17.9, z=17.7 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39199, 34.81808), pm (-6.3, -12.0), parallax 0.095 with NanoMaggies: g=22.2, r=20.8, z=20 Reset brightness to NanoMaggies: g=20.4, r=19.1, z=18.3 Measuring 7 iterative sources Blob 1 finished, total: Wall: 19.70 s, CPU: 19.70 s, VmPeak: 806 MB, VmSize: 742 MB, VmRSS: 165 MB, VmData: 193 MB, maxrss: 0.168956 MB Old catalog: 58 New catalog: 50 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/blobs-custom-276401p34809.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/all-models-custom-276401p34809.fits Stage fitblobs : Wall: 20.26 s, CPU: 0.50 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 301 MB, VmData: 326 MB, maxrss: 0.308468 MB Stage fitblobs finished: 2020-08-05T19:39:21.498871 Resources for stage fitblobs : 25.217 s worker CPU, 25.317 s worker Wall, Wall: 20.264 s, Cores in use: 1.27, Total efficiency (on 16 cores): 7.9 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:39:21.519350 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-resid.jpg Stage coadds : Wall: 4.37 s, CPU: 1.63 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 302 MB, VmData: 326 MB, maxrss: 0.308792 MB Stage coadds finished: 2020-08-05T19:39:25.884932 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 9.636 s worker CPU, 19.021 s worker Wall, Wall: 4.367 s, Cores in use: 2.58, Total efficiency (on 16 cores): 16.1 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T19:39:26.586672 Cut to 1 unWISE tiles Reading WISE tile 2759p348 band 3 Reading WISE tile 2759p348 band 2 Reading WISE tile 2759p348 band 1 Reading WISE tile 2759p348 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 810 MB, VmSize: 810 MB, VmRSS: 150 MB, VmData: 177 MB, maxrss: 0.191556 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 867 MB, VmSize: 867 MB, VmRSS: 206 MB, VmData: 235 MB, maxrss: 0.21104 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 814 MB, VmSize: 812 MB, VmRSS: 158 MB, VmData: 180 MB, maxrss: 0.222944 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 813 MB, VmSize: 805 MB, VmRSS: 146 MB, VmData: 172 MB, maxrss: 0.209024 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 3.66 s, CPU: 1.62 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 302 MB, VmData: 326 MB, maxrss: 0.309304 MB Stage wise_forced finished: 2020-08-05T19:39:30.250834 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.312 s worker CPU, 5.649 s worker Wall, Wall: 3.666 s, Cores in use: 1.35, Total efficiency (on 16 cores): 8.4 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:39:30.284611 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/tractor-i/cus/tractor-custom-276401p34809.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/tractor/cus/tractor-custom-276401p34809.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/tractor/cus/brick-custom-276401p34809.sha256sum Stage writecat : Wall: 4.04 s, CPU: 2.32 s, VmPeak: 1207 MB, VmSize: 1079 MB, VmRSS: 304 MB, VmData: 326 MB, maxrss: 0.442184 MB Stage writecat finished: 2020-08-05T19:39:34.325492 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 4.043 s, Cores in use: 0.57, Total efficiency (on 16 cores): 3.6 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-writecat.p All done: 44.159 s worker CPU, 104.715 s worker Wall, Wall: 66.409 s, Cores in use: 0.95, Total efficiency (on 16 cores): 6.0 % Finished galaxy PGC2054868 in 1.193 minutes. Started working on galaxy PGC2054868 at Wed Aug 5 19:40:03 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.401568 34.809166 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:40:05.649363 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.401568', '34.809166', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.401568 34.809166 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 276.401568 34.809166 Starting process 32330 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 32331 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 32332 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 32333 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 32334 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05942 MB Starting process 32335 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB Starting process 32336 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB Starting process 32337 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 32338 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 32339 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 32340 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 32341 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 32342 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 32343 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 32344 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 32345 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 32346 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 32347 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 32348 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 32349 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 32350 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 32351 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 32352 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 32354 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 32355 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 32353 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 32357 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Starting process 32356 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 32358 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB Starting process 32359 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB Starting process 32360 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB Starting process 32361 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05932 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:40:07.397669 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: k4m_160415_115838_ooi_zd_ls9-CCD3 z expnum 82536 exptime 91.102 propid 2016A-0453 seeing 1.05 MJD 57493.499 object MzLS_579415_z mosaic/CP/V4.3/CP20160414/k4m_160415_115838_ooi_zd_ls9.fits.fz k4m_160429_110622_ooi_zd_ls9-CCD4 z expnum 85811 exptime 80.103 propid 2016A-0453 seeing 1.24 MJD 57507.463 object MzLS_577359_z mosaic/CP/V4.3/CP20160428/k4m_160429_110622_ooi_zd_ls9.fits.fz k4m_160413_120439_ooi_zd_ls9-CCD3 z expnum 82031 exptime 80.102 propid 2016A-0453 seeing 0.82 MJD 57491.503 object MzLS_578388_z mosaic/CP/V4.3/CP20160412/k4m_160413_120439_ooi_zd_ls9.fits.fz k4m_160415_115629_ooi_zd_ls9-CCD4 z expnum 82535 exptime 85.104 propid 2016A-0453 seeing 0.99 MJD 57493.498 object MzLS_576327_z mosaic/CP/V4.3/CP20160414/k4m_160415_115629_ooi_zd_ls9.fits.fz k4m_160413_121059_ooi_zd_ls9-CCD1 z expnum 82034 exptime 81.102 propid 2016A-0453 seeing 0.83 MJD 57491.508 object MzLS_578389_z mosaic/CP/V4.3/CP20160412/k4m_160413_121059_ooi_zd_ls9.fits.fz k4m_160415_120352_ooi_zd_ls9-CCD2 z expnum 82538 exptime 149.104 propid 2016A-0453 seeing 1.02 MJD 57493.503 object MzLS_576328_z mosaic/CP/V4.3/CP20160414/k4m_160415_120352_ooi_zd_ls9.fits.fz ksb_160602_085948_ooi_g_ls9-CCD4 g expnum 75410141 exptime 119.0 propid BASS seeing 1.19 MJD 57541.375 object 08282 90prime/CP/V2.3/CP20160601/ksb_160602_085948_ooi_g_ls9.fits.fz ksb_160511_110505_ooi_r_ls9-CCD4 r expnum 75190253 exptime 60.2 propid BASS seeing 1.23 MJD 57519.462 object 08282 90prime/CP/V2.3/CP20160510/ksb_160511_110505_ooi_r_ls9.fits.fz ksb_180509_110248_ooi_r_ls9-CCD1 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.15 MJD 58247.460 object 08283 90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz ksb_160702_081612_ooi_g_ls9-CCD1 g expnum 75710100 exptime 134.2 propid BASS seeing 2.06 MJD 57571.345 object 08283 90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz Faked up EXPNUM 82470196 Faked up EXPNUM 82470196 Faked up EXPNUM 82470196 90prime get_fwhm: self.fwhm = 2.53 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3529 3669 and amp range 2016 4031 and image y 2229 2369 and amp range 2048 4095 Range in image: x 0 141 , y 0 141 (with image size 141 x 141) dzp 0.015744466 -> scaling image by 1.0146068417155982 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.93 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3776 4029 and amp range 2048 4095 and image y 313 564 and amp range 0 2039 Range in image: x 0 254 , y 0 252 (with image size 254 x 252) dzp 0.0072022504 -> scaling image by 1.0066555682883527 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75410141 Faked up EXPNUM 75410141 Faked up EXPNUM 75410141 90prime get_fwhm: self.fwhm = 2.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3380 3521 and amp range 2016 4031 and image y 1761 1900 and amp range 0 2047 Range in image: x 0 142 , y 0 140 (with image size 142 x 140) dzp 0.010226944 -> scaling image by 1.0094638657706572 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75710100 Faked up EXPNUM 75710100 Faked up EXPNUM 75710100 90prime get_fwhm: self.fwhm = 4.55 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3530 3670 and amp range 2016 4031 and image y 2259 2399 and amp range 2048 4095 Range in image: x 0 141 , y 0 141 (with image size 141 x 141) dzp 0.009677861 -> scaling image by 1.0089534840651213 mosaic get_fwhm: self.fwhm = 4.77 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1039 1279 and amp range 0 2047 and image y 254 494 and amp range 0 2021 Range in image: x 0 241 , y 0 241 (with image size 241 x 241) dzp 0.005139143 -> scaling image by 1.0047445456970268 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 0 147 and amp range 0 2005 and image y 1706 1951 and amp range 0 2039 Range in image: x 0 148 , y 0 246 (with image size 148 x 246) dzp -0.005856539 -> scaling image by 0.9946204500110131 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75190253 Faked up EXPNUM 75190253 Faked up EXPNUM 75190253 90prime get_fwhm: self.fwhm = 2.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3385 3526 and amp range 2016 4031 and image y 1787 1926 and amp range 0 2047 Range in image: x 0 142 , y 0 140 (with image size 142 x 140) dzp 0.0047082007 -> scaling image by 1.0043458289369953 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.19 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2617 2860 and amp range 2006 4053 and image y 374 620 and amp range 0 2039 Range in image: x 0 244 , y 0 247 (with image size 244 x 247) dzp 0.001388291 -> scaling image by 1.0012794811153327 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.81 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3814 4063 and amp range 2048 4095 and image y 4012 4054 and amp range 2022 4060 Range in image: x 0 250 , y 0 43 (with image size 250 x 43) dzp -0.0028835079 -> scaling image by 0.9973477146455455 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 25.56 s, CPU: 1.63 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 175 MB, VmData: 207 MB, maxrss: 0.178768 MB Stage tims finished: 2020-08-05T19:40:32.953993 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 4.920 s worker CPU, 81.913 s worker Wall, Wall: 25.559 s, Cores in use: 0.26, Total efficiency (on 32 cores): 0.8 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T19:40:32.981470 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/reference-custom-276401p34809.fits Stage refs : Wall: 24.36 s, CPU: 8.82 s, VmPeak: 1070 MB, VmSize: 1065 MB, VmRSS: 285 MB, VmData: 315 MB, maxrss: 0.291836 MB Stage refs finished: 2020-08-05T19:40:57.338687 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 24.359 s, Cores in use: 0.36, Total efficiency (on 32 cores): 1.1 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:40:57.367777 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outlier-mask-custom-276401p34809.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-pre-custom-276401p34809.jpg Image ksb_160602_085948_ooi_g_ls9-CCD4 g : masking 161 positive outlier pixels and 0 negative outlier pixels Image ksb_160702_081612_ooi_g_ls9-CCD1 g : masking 174 positive outlier pixels and 0 negative outlier pixels Image ksb_160511_110505_ooi_r_ls9-CCD4 r : masking 189 positive outlier pixels and 0 negative outlier pixels Image ksb_180509_110248_ooi_r_ls9-CCD1 r : masking 0 positive outlier pixels and 130 negative outlier pixels Image k4m_160415_115629_ooi_zd_ls9-CCD4 z : masking 103 positive outlier pixels and 0 negative outlier pixels Image k4m_160415_115838_ooi_zd_ls9-CCD3 z : masking 0 positive outlier pixels and 1454 negative outlier pixels Image k4m_160415_120352_ooi_zd_ls9-CCD2 z : masking 2599 positive outlier pixels and 0 negative outlier pixels Image k4m_160413_121059_ooi_zd_ls9-CCD1 z : masking 1552 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outlier-mask-custom-276401p34809.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-post-custom-276401p34809.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-masked-pos-custom-276401p34809.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/outliers-masked-neg-custom-276401p34809.jpg Stage outliers : Wall: 1.13 s, CPU: 0.67 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 297 MB, VmData: 325 MB, maxrss: 0.304296 MB Stage outliers finished: 2020-08-05T19:40:58.499359 Resources for stage outliers : 0.916 s worker CPU, 1.172 s worker Wall, Wall: 1.134 s, Cores in use: 1.40, Total efficiency (on 32 cores): 4.4 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:40:58.699578 Stage halos : Wall: 0.36 s, CPU: 0.35 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 297 MB, VmData: 325 MB, maxrss: 0.304296 MB Stage halos finished: 2020-08-05T19:40:59.063439 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.366 s, Cores in use: 0.97, Total efficiency (on 32 cores): 3.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T19:40:59.106541 PSF sigmas in coadd pixels: ['1.94', '3.35'] Using average PSF sigma 2.641509650306382 Gaussian PSF norm 0.1067930195678667 vs pixelized 0.08212437 average tim pixel scale / coadd scale: 1.7293010956522445 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.00', '1.86'] Using average PSF sigma 1.9286637892988572 Gaussian PSF norm 0.14626436880241858 vs pixelized 0.13213196 average tim pixel scale / coadd scale: 1.729900245768025 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.71', '2.01', '1.61', '1.35', '1.66'] Using average PSF sigma 1.6656765399163167 Gaussian PSF norm 0.16935748629085606 vs pixelized 0.1336563 average tim pixel scale / coadd scale: 0.9909729568380151 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.14 s, CPU: 0.13 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 299 MB, VmData: 325 MB, maxrss: 0.305848 MB Stage fit_on_coadds finished: 2020-08-05T19:40:59.246684 Resources for stage fit_on_coadds : 0.148 s worker CPU, 0.148 s worker Wall, Wall: 0.142 s, Cores in use: 1.96, Total efficiency (on 32 cores): 6.1 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T19:40:59.259858 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 47 potential peaks Of 47 potential peaks: 14 in veto map, 5 cut by saddle test, 0 cut by aper test, 28 kept SED z : 28 new peaks Found 49 potential peaks Of 49 potential peaks: 44 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 41 potential peaks Of 41 potential peaks: 36 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 69 potential peaks Of 69 potential peaks: 58 in veto map, 9 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 68 potential peaks Of 68 potential peaks: 58 in veto map, 8 cut by saddle test, 1 cut by aper test, 1 kept SED Red : 1 new peaks Adding 33 new sources Median coadd sky for g : 0.0006869103 Median coadd sky for r : 0.00092201383 Median coadd sky for z : -0.00077612785 Stage srcs : Wall: 0.63 s, CPU: 0.60 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 299 MB, VmData: 325 MB, maxrss: 0.306348 MB Stage srcs finished: 2020-08-05T19:40:59.892686 Resources for stage srcs : 0.072 s worker CPU, 0.072 s worker Wall, Wall: 0.635 s, Cores in use: 1.07, Total efficiency (on 32 cores): 3.3 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:40:59.909056 Blob 1 of 10, id: 5, sources: 33, size: 239x205, npix 26941, brick X: 0,239, Y: 34,239, one pixel: 5 34 Blob 2 of 10, id: 8, sources: 4, size: 48x45, npix 1421, brick X: 0,48, Y: 188,233, one pixel: 12 188 Blob 3 of 10, id: 9, sources: 3, size: 58x32, npix 1279, brick X: 52,110, Y: 207,239, one pixel: 71 207 Blob 4 of 10, id: 1, sources: 1, size: 38x38, npix 992, brick X: 106,144, Y: 0,38, one pixel: 122 0 Blob 5 of 10, id: 4, sources: 1, size: 29x32, npix 660, brick X: 65,94, Y: 33,65, one pixel: 77 33 Blob 6 of 10, id: 2, sources: 2, size: 31x22, npix 498, brick X: 208,239, Y: 0,22, one pixel: 211 0 Blob 7 of 10, id: 6, sources: 1, size: 27x27, npix 481, brick X: 208,235, Y: 81,108, one pixel: 219 81 Blob 8 of 10, id: 3, sources: 1, size: 22x24, npix 308, brick X: 178,200, Y: 24,48, one pixel: 186 24 Blob 9 of 10, id: 7, sources: 1, size: 22x23, npix 295, brick X: 9,31, Y: 151,174, one pixel: 19 151 Blob 10 of 10, id: 0, sources: 1, size: 22x17, npix 221, brick X: 84,106, Y: 0,17, one pixel: 90 0 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39198, 34.80005), pm (2.6, -6.9), parallax 0.322 with NanoMaggies: g=18.3, r=17, z=16.7 Reset brightness to NanoMaggies: g=17.1, r=16.6, z=16.4 Blob 6 finished, total: Wall: 0.12 s, CPU: 0.12 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 87 MB, VmData: 140 MB, maxrss: 0.0893 MB Blob 8 finished, total: Wall: 0.16 s, CPU: 0.16 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.09346 MB Blob 10 finished, total: Wall: 0.21 s, CPU: 0.20 s, VmPeak: 813 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149888 MB Blob 9 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092496 MB Blob 5 finished, total: Wall: 0.44 s, CPU: 0.44 s, VmPeak: 791 MB, VmSize: 719 MB, VmRSS: 133 MB, VmData: 170 MB, maxrss: 0.136456 MB Blob 7 finished, total: Wall: 0.45 s, CPU: 0.45 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092684 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.40489, 34.81802), pm (-3.5, 16.1), parallax 1.627 with NanoMaggies: g=17.1, r=15.9, z=15.3 Reset brightness to NanoMaggies: g=17, r=15.9, z=15.3 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.40477, 34.82073), pm (-3.7, -4.8), parallax 0.068 with NanoMaggies: g=19.3, r=18.8, z=18.6 Reset brightness to NanoMaggies: g=19.3, r=18.8, z=18.6 Measuring 1 iterative sources Blob 3 finished, total: Wall: 0.78 s, CPU: 0.78 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09262 MB Blob 4 finished, total: Wall: 0.95 s, CPU: 0.95 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092644 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.41505, 34.81535), pm (3.7, 13.1), parallax 0.736 with NanoMaggies: g=15.5, r=15, z=14.8 Reset brightness to NanoMaggies: g=15.5, r=15, z=14.8 Measuring 1 iterative sources Blob 2 finished, total: Wall: 1.73 s, CPU: 1.73 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.1502 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.35518, 34.82551), pm (4.3, 3.8), parallax 1.896 with NanoMaggies: g=11.7, r=11.4, z=11.4 Reset brightness to NanoMaggies: g=11.7, r=11.4, z=11.4 Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.33595, ee1=0.00823389, ee2=0.152249 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39846, 34.82012), pm (0.4, -7.7), parallax 0.474 with NanoMaggies: g=18.6, r=17.9, z=13.3 Reset brightness to NanoMaggies: g=18.6, r=17.9, z=17.7 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39199, 34.81808), pm (-6.3, -12.0), parallax 0.095 with NanoMaggies: g=22.2, r=20.8, z=20 Reset brightness to NanoMaggies: g=20.4, r=19.1, z=18.3 Measuring 7 iterative sources Blob 1 finished, total: Wall: 21.07 s, CPU: 21.07 s, VmPeak: 704 MB, VmSize: 700 MB, VmRSS: 133 MB, VmData: 171 MB, maxrss: 0.136208 MB Old catalog: 58 New catalog: 50 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/blobs-custom-276401p34809.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/metrics/cus/all-models-custom-276401p34809.fits Stage fitblobs : Wall: 21.62 s, CPU: 0.51 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 301 MB, VmData: 326 MB, maxrss: 0.308524 MB Stage fitblobs finished: 2020-08-05T19:41:21.531669 Resources for stage fitblobs : 26.415 s worker CPU, 26.472 s worker Wall, Wall: 21.625 s, Cores in use: 1.25, Total efficiency (on 32 cores): 3.9 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:41:21.551675 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-resid.jpg Stage coadds : Wall: 5.13 s, CPU: 1.04 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 302 MB, VmData: 326 MB, maxrss: 0.308848 MB Stage coadds finished: 2020-08-05T19:41:26.686523 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 27.393 s worker CPU, 65.100 s worker Wall, Wall: 5.138 s, Cores in use: 5.54, Total efficiency (on 32 cores): 17.3 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T19:41:26.714087 Cut to 1 unWISE tiles Reading WISE tile 2759p348 band 3 Reading WISE tile 2759p348 band 2 Reading WISE tile 2759p348 band 1 Reading WISE tile 2759p348 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 813 MB, VmSize: 810 MB, VmRSS: 151 MB, VmData: 177 MB, maxrss: 0.192108 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 809 MB, VmSize: 802 MB, VmRSS: 143 MB, VmData: 174 MB, maxrss: 0.205744 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 799 MB, VmSize: 799 MB, VmRSS: 138 MB, VmData: 170 MB, maxrss: 0.198024 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 799 MB, VmSize: 799 MB, VmRSS: 140 MB, VmData: 171 MB, maxrss: 0.202712 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/tmp-legacysurvey-custom-276401p34809-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 4.46 s, CPU: 2.27 s, VmPeak: 1075 MB, VmSize: 1075 MB, VmRSS: 302 MB, VmData: 326 MB, maxrss: 0.30936 MB Stage wise_forced finished: 2020-08-05T19:41:31.176447 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.578 s worker CPU, 5.510 s worker Wall, Wall: 4.465 s, Cores in use: 1.31, Total efficiency (on 32 cores): 4.1 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:41:31.197521 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/coadd/cus/custom-276401p34809/legacysurvey-custom-276401p34809-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/tractor-i/cus/tractor-custom-276401p34809.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/tractor/cus/tractor-custom-276401p34809.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/tractor/cus/brick-custom-276401p34809.sha256sum Stage writecat : Wall: 1.01 s, CPU: 0.88 s, VmPeak: 1207 MB, VmSize: 1079 MB, VmRSS: 304 MB, VmData: 326 MB, maxrss: 0.44224 MB Stage writecat finished: 2020-08-05T19:41:32.209064 Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.014 s, Cores in use: 0.87, Total efficiency (on 32 cores): 2.7 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2054868/PGC2054868-largegalaxy-writecat.p All done: 63.450 s worker CPU, 180.395 s worker Wall, Wall: 84.839 s, Cores in use: 0.95, Total efficiency (on 32 cores): 3.0 % Finished galaxy PGC2054868 in 1.483 minutes. Started working on galaxy PGC2054868 at Fri Aug 7 09:28:42 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.40122507828056 34.808958998181694 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T09:28:43.165555 legacypipe git version: DR9.5.11-watershed-3-gd509eaaf Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.40122507828056', '34.808958998181694', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.40122507828056 34.808958998181694 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 276.40122507828056 34.808958998181694 Starting process 22124 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 22125 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 22126 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 22127 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 22128 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05934 MB Starting process 22129 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059348 MB Starting process 22130 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059348 MB Starting process 22131 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 22132 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 22133 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 22134 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 22135 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 22136 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 22137 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 22138 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 22139 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T09:28:43.619217 Keeping 10 CCDs: k4m_160415_115838_ooi_zd_ls9-CCD3 z expnum 82536 exptime 91.102 propid 2016A-0453 seeing 1.05 MJD 57493.499 object MzLS_579415_z mosaic/CP/V4.3/CP20160414/k4m_160415_115838_ooi_zd_ls9.fits.fz k4m_160429_110622_ooi_zd_ls9-CCD4 z expnum 85811 exptime 80.103 propid 2016A-0453 seeing 1.24 MJD 57507.463 object MzLS_577359_z mosaic/CP/V4.3/CP20160428/k4m_160429_110622_ooi_zd_ls9.fits.fz k4m_160413_120439_ooi_zd_ls9-CCD3 z expnum 82031 exptime 80.102 propid 2016A-0453 seeing 0.82 MJD 57491.503 object MzLS_578388_z mosaic/CP/V4.3/CP20160412/k4m_160413_120439_ooi_zd_ls9.fits.fz k4m_160415_115629_ooi_zd_ls9-CCD4 z expnum 82535 exptime 85.104 propid 2016A-0453 seeing 0.99 MJD 57493.498 object MzLS_576327_z mosaic/CP/V4.3/CP20160414/k4m_160415_115629_ooi_zd_ls9.fits.fz k4m_160413_121059_ooi_zd_ls9-CCD1 z expnum 82034 exptime 81.102 propid 2016A-0453 seeing 0.83 MJD 57491.508 object MzLS_578389_z mosaic/CP/V4.3/CP20160412/k4m_160413_121059_ooi_zd_ls9.fits.fz k4m_160415_120352_ooi_zd_ls9-CCD2 z expnum 82538 exptime 149.104 propid 2016A-0453 seeing 1.02 MJD 57493.503 object MzLS_576328_z mosaic/CP/V4.3/CP20160414/k4m_160415_120352_ooi_zd_ls9.fits.fz ksb_160602_085948_ooi_g_ls9-CCD4 g expnum 75410141 exptime 119.0 propid BASS seeing 1.19 MJD 57541.375 object 08282 90prime/CP/V2.3/CP20160601/ksb_160602_085948_ooi_g_ls9.fits.fz ksb_160511_110505_ooi_r_ls9-CCD4 r expnum 75190253 exptime 60.2 propid BASS seeing 1.23 MJD 57519.462 object 08282 90prime/CP/V2.3/CP20160510/ksb_160511_110505_ooi_r_ls9.fits.fz ksb_180509_110248_ooi_r_ls9-CCD1 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.15 MJD 58247.460 object 08283 90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz ksb_160702_081612_ooi_g_ls9-CCD1 g expnum 75710100 exptime 134.2 propid BASS seeing 2.06 MJD 57571.345 object 08283 90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 0 144 and amp range 0 2005 and image y 1702 1947 and amp range 0 2039 Range in image: x 0 145 , y 0 246 (with image size 145 x 246) dzp -0.005856539 -> scaling image by 0.9946204500110131 mosaic get_fwhm: self.fwhm = 4.77 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1036 1276 and amp range 0 2047 and image y 250 490 and amp range 0 2021 Range in image: x 0 241 , y 0 241 (with image size 241 x 241) dzp 0.005139143 -> scaling image by 1.0047445456970268 mosaic get_fwhm: self.fwhm = 3.81 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3811 4060 and amp range 2048 4095 and image y 4008 4054 and amp range 2022 4060 Range in image: x 0 250 , y 0 47 (with image size 250 x 47) dzp -0.0028835079 -> scaling image by 0.9973477146455455 mosaic get_fwhm: self.fwhm = 3.19 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2615 2858 and amp range 2006 4053 and image y 370 616 and amp range 0 2039 Range in image: x 0 244 , y 0 247 (with image size 244 x 247) dzp 0.001388291 -> scaling image by 1.0012794811153327 Faked up EXPNUM 82470196 Faked up EXPNUM 82470196 Faked up EXPNUM 82470196 90prime get_fwhm: self.fwhm = 2.53 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3530 3670 and amp range 2016 4031 and image y 2231 2371 and amp range 2048 4095 Range in image: x 0 141 , y 0 141 (with image size 141 x 141) dzp 0.015744466 -> scaling image by 1.0146068417155982 Faked up EXPNUM 75710100 Faked up EXPNUM 75710100 Faked up EXPNUM 75710100 90prime get_fwhm: self.fwhm = 4.55 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3532 3672 and amp range 2016 4031 and image y 2261 2401 and amp range 2048 4095 Range in image: x 0 141 , y 0 141 (with image size 141 x 141) dzp 0.009677861 -> scaling image by 1.0089534840651213 Faked up EXPNUM 75410141 Faked up EXPNUM 75410141 Faked up EXPNUM 75410141 90prime get_fwhm: self.fwhm = 2.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3382 3522 and amp range 2016 4031 and image y 1763 1902 and amp range 0 2047 Range in image: x 0 141 , y 0 140 (with image size 141 x 140) dzp 0.010226944 -> scaling image by 1.0094638657706572 Faked up EXPNUM 75190253 Faked up EXPNUM 75190253 Faked up EXPNUM 75190253 90prime get_fwhm: self.fwhm = 2.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3387 3528 and amp range 2016 4031 and image y 1789 1928 and amp range 0 2047 Range in image: x 0 142 , y 0 140 (with image size 142 x 140) dzp 0.0047082007 -> scaling image by 1.0043458289369953 mosaic get_fwhm: self.fwhm = 3.93 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3774 4026 and amp range 2048 4095 and image y 309 560 and amp range 0 2039 Range in image: x 0 253 , y 0 252 (with image size 253 x 252) dzp 0.0072022504 -> scaling image by 1.0066555682883527 Stage tims : Wall: 36.61 s, CPU: 0.88 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 173 MB, VmData: 206 MB, maxrss: 0.176744 MB Stage tims finished: 2020-08-07T09:29:20.229553 mosaic get_fwhm: self.fwhm = 3.15 Resources for stage tims : 4.676 s worker CPU, 174.823 s worker Wall, Wall: 36.612 s, Cores in use: 0.15, Total efficiency (on 16 cores): 0.9 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T09:29:20.262016 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/reference-custom-276401p34808.fits Stage refs : Wall: 9.48 s, CPU: 8.56 s, VmPeak: 1064 MB, VmSize: 1059 MB, VmRSS: 283 MB, VmData: 314 MB, maxrss: 0.289912 MB Stage refs finished: 2020-08-07T09:29:29.741464 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 9.481 s, Cores in use: 0.90, Total efficiency (on 16 cores): 5.6 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T09:29:29.796197 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/outlier-mask-custom-276401p34808.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/outliers-pre-custom-276401p34808.jpg Image ksb_160702_081612_ooi_g_ls9-CCD1 g : masking 277 positive outlier pixels and 0 negative outlier pixels Image ksb_160602_085948_ooi_g_ls9-CCD4 g : masking 160 positive outlier pixels and 0 negative outlier pixels Image ksb_160511_110505_ooi_r_ls9-CCD4 r : masking 57 positive outlier pixels and 0 negative outlier pixels Image k4m_160415_115629_ooi_zd_ls9-CCD4 z : masking 100 positive outlier pixels and 0 negative outlier pixels Image k4m_160415_115838_ooi_zd_ls9-CCD3 z : masking 0 positive outlier pixels and 1463 negative outlier pixels Image k4m_160413_121059_ooi_zd_ls9-CCD1 z : masking 1496 positive outlier pixels and 0 negative outlier pixels Image k4m_160415_120352_ooi_zd_ls9-CCD2 z : masking 2256 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/outlier-mask-custom-276401p34808.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/outliers-post-custom-276401p34808.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/outliers-masked-pos-custom-276401p34808.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/outliers-masked-neg-custom-276401p34808.jpg Stage outliers : Wall: 0.86 s, CPU: 0.53 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 295 MB, VmData: 324 MB, maxrss: 0.301616 MB Stage outliers finished: 2020-08-07T09:29:30.656428 Resources for stage outliers : 0.621 s worker CPU, 0.684 s worker Wall, Wall: 0.862 s, Cores in use: 1.34, Total efficiency (on 16 cores): 8.4 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T09:29:30.744076 Stage halos : Wall: 0.22 s, CPU: 0.21 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 295 MB, VmData: 324 MB, maxrss: 0.301708 MB Stage halos finished: 2020-08-07T09:29:30.966397 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.224 s, Cores in use: 0.96, Total efficiency (on 16 cores): 6.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T09:29:30.999711 PSF sigmas in coadd pixels: ['1.94', '3.35'] Using average PSF sigma 2.641509650306382 Gaussian PSF norm 0.1067930195678667 vs pixelized 0.082122825 average tim pixel scale / coadd scale: 1.7293010956522445 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.00', '1.86'] Using average PSF sigma 1.9286637892988572 Gaussian PSF norm 0.14626436880241858 vs pixelized 0.13212746 average tim pixel scale / coadd scale: 1.729900245768025 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.71', '2.01', '1.61', '1.35', '1.66'] Using average PSF sigma 1.6656765399163167 Gaussian PSF norm 0.16935748629085606 vs pixelized 0.13365503 average tim pixel scale / coadd scale: 0.9909729568380151 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.22 s, CPU: 0.14 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 297 MB, VmData: 325 MB, maxrss: 0.303808 MB Stage fit_on_coadds finished: 2020-08-07T09:29:31.224381 Resources for stage fit_on_coadds : 0.158 s worker CPU, 0.158 s worker Wall, Wall: 0.226 s, Cores in use: 1.31, Total efficiency (on 16 cores): 8.2 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-07T09:29:31.257290 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 45 potential peaks Of 45 potential peaks: 14 in veto map, 4 cut by saddle test, 0 cut by aper test, 27 kept SED z : 27 new peaks Found 48 potential peaks Of 48 potential peaks: 43 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 41 potential peaks Of 41 potential peaks: 35 in veto map, 6 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 70 potential peaks Of 70 potential peaks: 58 in veto map, 10 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 70 potential peaks Of 70 potential peaks: 60 in veto map, 8 cut by saddle test, 1 cut by aper test, 1 kept SED Red : 1 new peaks Adding 32 new sources Median coadd sky for g : 0.00066860195 Median coadd sky for r : 0.00089750794 Median coadd sky for z : -0.0007645822 Stage srcs : Wall: 0.51 s, CPU: 0.48 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 297 MB, VmData: 325 MB, maxrss: 0.30454 MB Stage srcs finished: 2020-08-07T09:29:31.767496 Resources for stage srcs : 0.077 s worker CPU, 0.077 s worker Wall, Wall: 0.512 s, Cores in use: 1.08, Total efficiency (on 16 cores): 6.8 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T09:29:31.801899 Blob 1 of 10, id: 5, sources: 25, size: 228x169, npix 23994, brick X: 0,228, Y: 38,207, one pixel: 4 38 Blob 2 of 10, id: 8, sources: 7, size: 111x44, npix 2859, brick X: 128,239, Y: 195,239, one pixel: 156 195 Blob 3 of 10, id: 7, sources: 3, size: 44x45, npix 1273, brick X: 0,44, Y: 191,236, one pixel: 8 191 Blob 4 of 10, id: 2, sources: 1, size: 38x38, npix 992, brick X: 102,140, Y: 3,41, one pixel: 117 3 Blob 5 of 10, id: 9, sources: 3, size: 50x29, npix 982, brick X: 50,100, Y: 210,239, one pixel: 67 210 Blob 6 of 10, id: 4, sources: 1, size: 29x32, npix 661, brick X: 61,90, Y: 36,68, one pixel: 73 36 Blob 7 of 10, id: 1, sources: 2, size: 36x25, npix 657, brick X: 203,239, Y: 0,25, one pixel: 205 0 Blob 8 of 10, id: 6, sources: 1, size: 27x28, npix 486, brick X: 205,232, Y: 83,111, one pixel: 217 83 Blob 9 of 10, id: 3, sources: 1, size: 22x25, npix 311, brick X: 174,196, Y: 26,51, one pixel: 183 26 Blob 10 of 10, id: 0, sources: 1, size: 22x19, npix 271, brick X: 80,102, Y: 0,19, one pixel: 87 0 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39198, 34.80005), pm (2.6, -6.9), parallax 0.322 with NanoMaggies: g=17.3, r=16.8, z=16.6 Reset brightness to NanoMaggies: g=17.1, r=16.6, z=16.4 Blob 7 finished, total: Wall: 0.12 s, CPU: 0.11 s, VmPeak: 791 MB, VmSize: 711 MB, VmRSS: 129 MB, VmData: 169 MB, maxrss: 0.131924 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.40489, 34.81802), pm (-3.5, 16.1), parallax 1.627 with NanoMaggies: g=17.4, r=15.7, z=15.4 Reset brightness to NanoMaggies: g=17, r=15.9, z=15.3 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.40477, 34.82073), pm (-3.7, -4.8), parallax 0.068 with NanoMaggies: g=19.3, r=18.8, z=18.6 Reset brightness to NanoMaggies: g=19.3, r=18.8, z=18.6 Measuring 1 iterative sources Blob 9 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092768 MB Blob 10 finished, total: Wall: 0.30 s, CPU: 0.29 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092316 MB Blob 6 finished, total: Wall: 0.40 s, CPU: 0.40 s, VmPeak: 806 MB, VmSize: 724 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145528 MB Blob 5 finished, total: Wall: 0.41 s, CPU: 0.40 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093432 MB Blob 8 finished, total: Wall: 0.43 s, CPU: 0.43 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146188 MB Blob 4 finished, total: Wall: 0.96 s, CPU: 0.96 s, VmPeak: 806 MB, VmSize: 724 MB, VmRSS: 141 MB, VmData: 182 MB, maxrss: 0.144768 MB Blob 3 finished, total: Wall: 1.54 s, CPU: 1.53 s, VmPeak: 792 MB, VmSize: 712 MB, VmRSS: 130 MB, VmData: 170 MB, maxrss: 0.133016 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39846, 34.82012), pm (0.4, -7.7), parallax 0.474 with NanoMaggies: g=18.6, r=17.9, z=17.7 Reset brightness to NanoMaggies: g=18.6, r=17.9, z=17.7 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.39199, 34.81808), pm (-6.3, -12.0), parallax 0.095 with NanoMaggies: g=20.3, r=18.7, z=18.2 Reset brightness to NanoMaggies: g=20.4, r=19.1, z=18.3 Blob 2 finished, total: Wall: 2.33 s, CPU: 2.33 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147756 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=0.876744, ee1=0.00501655, ee2=0.130234 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.41188, 34.80432), pm (-2.5, -8.7), parallax 0.592 with NanoMaggies: g=22.7, r=(flux -0.455), z=19.8 Reset brightness to NanoMaggies: g=20.2, r=18.9, z=18 Measuring 5 iterative sources Blob 1 finished, total: Wall: 14.73 s, CPU: 14.72 s, VmPeak: 809 MB, VmSize: 729 MB, VmRSS: 152 MB, VmData: 184 MB, maxrss: 0.155988 MB Old catalog: 57 New catalog: 46 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/blobs-custom-276401p34808.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/metrics/cus/all-models-custom-276401p34808.fits Stage fitblobs : Wall: 16.03 s, CPU: 0.37 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 299 MB, VmData: 325 MB, maxrss: 0.305828 MB Stage fitblobs finished: 2020-08-07T09:29:47.834369 Resources for stage fitblobs : 21.640 s worker CPU, 21.694 s worker Wall, Wall: 16.034 s, Cores in use: 1.37, Total efficiency (on 16 cores): 8.6 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-07T09:29:47.920463 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/tmp-legacysurvey-custom-276401p34808-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/tmp-legacysurvey-custom-276401p34808-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/tmp-legacysurvey-custom-276401p34808-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/tmp-legacysurvey-custom-276401p34808-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-resid.jpg Stage coadds : Wall: 2.32 s, CPU: 0.57 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 299 MB, VmData: 325 MB, maxrss: 0.306152 MB Stage coadds finished: 2020-08-07T09:29:50.241003 Resources for stage coadds : 9.117 s worker CPU, 9.539 s worker Wall, Wall: 2.322 s, Cores in use: 4.17, Total efficiency (on 16 cores): 26.1 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-07T09:29:50.275573 Cut to 1 unWISE tiles Reading WISE tile 2759p348 band 1 Reading WISE tile 2759p348 band 2 Reading WISE tile 2759p348 band 3 Reading WISE tile 2759p348 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 812 MB, VmSize: 803 MB, VmRSS: 147 MB, VmData: 177 MB, maxrss: 0.188356 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 799 MB, VmSize: 797 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.202516 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 805 MB, VmSize: 803 MB, VmRSS: 147 MB, VmData: 177 MB, maxrss: 0.209884 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 810 MB, VmSize: 808 MB, VmRSS: 155 MB, VmData: 179 MB, maxrss: 0.218996 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/tmp-legacysurvey-custom-276401p34808-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/tmp-legacysurvey-custom-276401p34808-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 15.27 s, CPU: 0.50 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 299 MB, VmData: 325 MB, maxrss: 0.306664 MB Stage wise_forced finished: 2020-08-07T09:30:05.543509 Resources for stage wise_forced : 3.016 s worker CPU, 24.583 s worker Wall, Wall: 15.270 s, Cores in use: 0.23, Total efficiency (on 16 cores): 1.4 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-07T09:30:05.573539 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/coadd/cus/custom-276401p34808/legacysurvey-custom-276401p34808-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/tractor-i/cus/tractor-custom-276401p34808.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/tractor/cus/tractor-custom-276401p34808.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/tractor/cus/brick-custom-276401p34808.sha256sum Stage writecat : Wall: 2.21 s, CPU: 0.67 s, VmPeak: 1201 MB, VmSize: 1073 MB, VmRSS: 301 MB, VmData: 325 MB, maxrss: 0.439548 MB Stage writecat finished: 2020-08-07T09:30:07.780494 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 2.209 s, Cores in use: 0.31, Total efficiency (on 16 cores): 1.9 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2054868/PGC2054868-largegalaxy-writecat.p All done: 39.310 s worker CPU, 231.562 s worker Wall, Wall: 84.215 s, Cores in use: 0.62, Total efficiency (on 16 cores): 3.9 % Finished galaxy PGC2054868 in 1.452 minutes.