Started working on galaxy PGC2099102 at Wed Aug 5 19:54:00 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.679542 37.36794879999999 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:54:01.980626 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '273.679542', '37.36794879999999', '--width', '251', '--height', '251', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.679542 37.36794879999999 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 273.679542 37.36794879999999 Starting process 32085 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 32087 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 32089 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 32091 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 32093 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059412 MB Starting process 32095 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05942 MB Starting process 32098 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05942 MB Starting process 32100 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 32102 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 32103 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 32105 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 32107 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 32109 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 32113 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 32111 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 32115 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:54:02.791051 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: k4m_160414_115749_ooi_zd_ls9-CCD2 z expnum 82276 exptime 80.102 propid 2016A-0453 seeing 0.84 MJD 57492.498 object MzLS_590546_z mosaic/CP/V4.3/CP20160413/k4m_160414_115749_ooi_zd_ls9.fits.fz k4m_160427_113004_ooi_zd_ls9-CCD1 z expnum 85294 exptime 80.103 propid 2016A-0453 seeing 0.73 MJD 57505.479 object MzLS_592537_z mosaic/CP/V4.3/CP20160426/k4m_160427_113004_ooi_zd_ls9.fits.fz k4m_160430_104744_ooi_zd_ls9-CCD1 z expnum 86079 exptime 80.104 propid 2016A-0453 seeing 0.83 MJD 57508.450 object MzLS_591543_z mosaic/CP/V4.3/CP20160429/k4m_160430_104744_ooi_zd_ls9.fits.fz k4m_160427_110333_ooi_zd_ls9-CCD4 z expnum 85282 exptime 80.104 propid 2016A-0453 seeing 0.70 MJD 57505.461 object MzLS_589546_z mosaic/CP/V4.3/CP20160426/k4m_160427_110333_ooi_zd_ls9.fits.fz ksb_160703_073031_ooi_g_ls9-CCD3 g expnum 75720046 exptime 71.8 propid BASS seeing 1.16 MJD 57572.313 object 10942 90prime/CP/V2.3/CP20160702/ksb_160703_073031_ooi_g_ls9.fits.fz ksb_160513_103534_ooi_r_ls9-CCD3 r expnum 75210210 exptime 62.6 propid BASS seeing 1.21 MJD 57521.441 object 10942 90prime/CP/V2.3/CP20160512/ksb_160513_103534_ooi_r_ls9.fits.fz ksb_170430_112929_ooi_r_ls9-CCD2 r expnum 78730270 exptime 161.859 propid BASS seeing 1.62 MJD 57873.479 object 12273 90prime/CP/V2.3/CP20170429/ksb_170430_112929_ooi_r_ls9.fits.fz ksb_160705_070952_ooi_g_ls9-CCD2 g expnum 75740098 exptime 115.6 propid BASS seeing 1.84 MJD 57574.299 object 12273 90prime/CP/V2.3/CP20160704/ksb_160705_070952_ooi_g_ls9.fits.fz ksb_160430_103438_ooi_r_ls9-CCD1 r expnum 75080183 exptime 130.6 propid BASS seeing 1.38 MJD 57508.441 object 10941 90prime/CP/V2.3/CP20160429/ksb_160430_103438_ooi_r_ls9.fits.fz ksb_160504_101953_ooi_g_ls9-CCD1 g expnum 75120196 exptime 91.7 propid BASS seeing 1.54 MJD 57512.430 object 10941 90prime/CP/V2.3/CP20160503/ksb_160504_101953_ooi_g_ls9.fits.fz Faked up EXPNUM 75080183 Faked up EXPNUM 75080183 Faked up EXPNUM 75080183 90prime get_fwhm: self.fwhm = 3.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1095 1243 and amp range 0 2015 and image y 2843 2990 and amp range 2048 4095 Range in image: x 0 149 , y 0 148 (with image size 149 x 148) dzp -0.00015519658 -> scaling image by 0.9998570688798908 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75720046 Faked up EXPNUM 75720046 Faked up EXPNUM 75720046 90prime get_fwhm: self.fwhm = 2.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3110 3256 and amp range 2016 4031 and image y 363 508 and amp range 0 2047 Range in image: x 0 147 , y 0 146 (with image size 147 x 146) dzp -0.0022715968 -> scaling image by 0.9979099691834846 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75740098 Faked up EXPNUM 75740098 Faked up EXPNUM 75740098 90prime get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2866 3013 and amp range 2016 4031 and image y 1881 2029 and amp range 0 2047 Range in image: x 0 148 , y 0 149 (with image size 148 x 149) dzp -0.008960029 -> scaling image by 0.9917814665476246 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75210210 Faked up EXPNUM 75210210 Faked up EXPNUM 75210210 90prime get_fwhm: self.fwhm = 2.67 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3127 3273 and amp range 2016 4031 and image y 417 563 and amp range 0 2047 Range in image: x 0 147 , y 0 147 (with image size 147 x 147) dzp 0.004245165 -> scaling image by 1.0039175950705865 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2855 3107 and amp range 2006 4053 and image y 3952 4072 and amp range 2040 4078 Range in image: x 0 253 , y 0 121 (with image size 253 x 121) dzp -0.006092005 -> scaling image by 0.9944047680378836 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.23 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1290 1543 and amp range 0 2047 and image y 3938 4072 and amp range 2040 4078 Range in image: x 0 254 , y 0 135 (with image size 254 x 135) dzp 0.008550294 -> scaling image by 1.0079062018164673 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 2.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 4036 4088 and amp range 2048 4095 and image y 2795 3054 and amp range 2022 4060 Range in image: x 0 53 , y 0 260 (with image size 53 x 260) dzp -0.0028835079 -> scaling image by 0.9973477146455455 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75120196 Faked up EXPNUM 75120196 Faked up EXPNUM 75120196 90prime get_fwhm: self.fwhm = 3.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1099 1247 and amp range 0 2015 and image y 2845 2992 and amp range 2048 4095 Range in image: x 0 149 , y 0 148 (with image size 149 x 148) dzp -0.0051191514 -> scaling image by 0.9952961850385206 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 78730270 Faked up EXPNUM 78730270 Faked up EXPNUM 78730270 90prime get_fwhm: self.fwhm = 3.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2863 3010 and amp range 2016 4031 and image y 1883 2030 and amp range 0 2047 Range in image: x 0 148 , y 0 148 (with image size 148 x 148) dzp 0.032776505 -> scaling image by 1.0306485625912574 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 2.81 Found 4 relevant rows in amp-corrections file. Found overlap: image x 137 396 and amp range 0 2005 and image y 3106 3360 and amp range 2040 4078 Range in image: x 0 260 , y 0 255 (with image size 260 x 255) dzp -0.005990375 -> scaling image by 0.994497853444703 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 9.85 s, CPU: 0.99 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 206 MB, maxrss: 0.178624 MB Stage tims finished: 2020-08-05T19:54:12.639612 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.653 s worker CPU, 51.295 s worker Wall, Wall: 9.850 s, Cores in use: 0.67, Total efficiency (on 16 cores): 4.2 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T19:54:12.658809 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02466.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02465.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02337.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/reference-custom-273679p37367.fits Stage refs : Wall: 10.78 s, CPU: 6.77 s, VmPeak: 1037 MB, VmSize: 1031 MB, VmRSS: 251 MB, VmData: 281 MB, maxrss: 0.25672 MB Stage refs finished: 2020-08-05T19:54:23.434841 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 10.778 s, Cores in use: 0.63, Total efficiency (on 16 cores): 3.9 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:54:23.458288 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outlier-mask-custom-273679p37367.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-pre-custom-273679p37367.jpg Image k4m_160427_110333_ooi_zd_ls9-CCD4 z : masking 2223 positive outlier pixels and 0 negative outlier pixels Image k4m_160414_115749_ooi_zd_ls9-CCD2 z : masking 6812 positive outlier pixels and 0 negative outlier pixels Image k4m_160427_113004_ooi_zd_ls9-CCD1 z : masking 130 positive outlier pixels and 825 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outlier-mask-custom-273679p37367.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-post-custom-273679p37367.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-masked-pos-custom-273679p37367.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-masked-neg-custom-273679p37367.jpg Stage outliers : Wall: 1.24 s, CPU: 0.90 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 263 MB, VmData: 290 MB, maxrss: 0.2692 MB Stage outliers finished: 2020-08-05T19:54:24.700750 Resources for stage outliers : 0.715 s worker CPU, 1.055 s worker Wall, Wall: 1.244 s, Cores in use: 1.30, Total efficiency (on 16 cores): 8.1 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:54:24.726626 Stage halos : Wall: 0.47 s, CPU: 0.46 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 263 MB, VmData: 290 MB, maxrss: 0.269456 MB Stage halos finished: 2020-08-05T19:54:25.193046 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.468 s, Cores in use: 0.98, Total efficiency (on 16 cores): 6.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T19:54:25.300041 PSF sigmas in coadd pixels: ['1.89', '2.98', '2.51'] Using average PSF sigma 2.460048803979209 Gaussian PSF norm 0.11467040463489207 vs pixelized 0.09825789 average tim pixel scale / coadd scale: 1.7291134425957921 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.96', '2.63', '2.24'] Using average PSF sigma 2.2793192950983054 Gaussian PSF norm 0.12376273582228048 vs pixelized 0.1006384 average tim pixel scale / coadd scale: 1.7296827001895272 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.36', '1.18', '1.35', '1.14'] Using average PSF sigma 1.260845348359652 Gaussian PSF norm 0.2237346492501089 vs pixelized 0.18630978 average tim pixel scale / coadd scale: 0.9909641990288808 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.16 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 265 MB, VmData: 290 MB, maxrss: 0.27118 MB Stage fit_on_coadds finished: 2020-08-05T19:54:25.478164 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.214 s worker CPU, 0.214 s worker Wall, Wall: 0.180 s, Cores in use: 2.09, Total efficiency (on 16 cores): 13.1 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T19:54:25.494904 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 42 potential peaks Of 42 potential peaks: 15 in veto map, 4 cut by saddle test, 0 cut by aper test, 23 kept SED z : 23 new peaks Found 27 potential peaks Of 27 potential peaks: 23 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 21 potential peaks Of 21 potential peaks: 19 in veto map, 1 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 43 potential peaks Of 43 potential peaks: 39 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept SED Flat : 0 new peaks Found 72 potential peaks Of 72 potential peaks: 62 in veto map, 7 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 28 new sources Median coadd sky for g : -1.1149298e-05 Median coadd sky for r : 9.977016e-05 Median coadd sky for z : 0.0002512724 Stage srcs : Wall: 0.78 s, CPU: 0.74 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 265 MB, VmData: 290 MB, maxrss: 0.271644 MB Stage srcs finished: 2020-08-05T19:54:26.272320 Resources for stage srcs : 0.082 s worker CPU, 0.082 s worker Wall, Wall: 0.779 s, Cores in use: 1.06, Total efficiency (on 16 cores): 6.6 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:54:26.287296 Blob 1 of 15, id: 0, sources: 20, size: 145x245, npix 18933, brick X: 29,174, Y: 0,245, one pixel: 86 0 Blob 2 of 15, id: 10, sources: 3, size: 75x49, npix 2076, brick X: 9,84, Y: 168,217, one pixel: 73 168 Blob 3 of 15, id: 12, sources: 2, size: 41x46, npix 1208, brick X: 157,198, Y: 194,240, one pixel: 177 194 Blob 4 of 15, id: 6, sources: 3, size: 40x49, npix 1084, brick X: 5,45, Y: 75,124, one pixel: 14 75 Blob 5 of 15, id: 7, sources: 1, size: 30x35, npix 793, brick X: 221,251, Y: 104,139, one pixel: 236 104 Blob 6 of 15, id: 4, sources: 1, size: 33x33, npix 768, brick X: 149,182, Y: 42,75, one pixel: 162 42 Blob 7 of 15, id: 3, sources: 1, size: 30x29, npix 575, brick X: 217,247, Y: 30,59, one pixel: 230 30 Blob 8 of 15, id: 1, sources: 1, size: 26x26, npix 419, brick X: 15,41, Y: 3,29, one pixel: 28 3 Blob 9 of 15, id: 11, sources: 1, size: 25x24, npix 377, brick X: 195,220, Y: 184,208, one pixel: 205 184 Blob 10 of 15, id: 9, sources: 1, size: 22x22, npix 312, brick X: 46,68, Y: 141,163, one pixel: 54 141 Blob 11 of 15, id: 5, sources: 1, size: 23x24, npix 304, brick X: 210,233, Y: 74,98, one pixel: 219 74 Blob 12 of 15, id: 13, sources: 1, size: 21x20, npix 249, brick X: 46,67, Y: 207,227, one pixel: 55 207 Blob 13 of 15, id: 8, sources: 1, size: 20x19, npix 218, brick X: 196,216, Y: 137,156, one pixel: 204 137 Blob 14 of 15, id: 14, sources: 1, size: 19x18, npix 189, brick X: 226,245, Y: 219,237, one pixel: 235 219 Blob 15 of 15, id: 2, sources: 1, size: 18x17, npix 162, brick X: 130,148, Y: 30,47, one pixel: 138 30 Blob 15 finished, total: Wall: 0.10 s, CPU: 0.09 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 85 MB, VmData: 138 MB, maxrss: 0.087104 MB Blob 13 finished, total: Wall: 0.25 s, CPU: 0.25 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 137 MB, maxrss: 0.092356 MB Blob 10 finished, total: Wall: 0.28 s, CPU: 0.27 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093148 MB Blob 12 finished, total: Wall: 0.31 s, CPU: 0.30 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149984 MB Blob 14 finished, total: Wall: 0.46 s, CPU: 0.45 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150164 MB Blob 11 finished, total: Wall: 0.52 s, CPU: 0.52 s, VmPeak: 791 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.137112 MB Blob 8 finished, total: Wall: 0.61 s, CPU: 0.61 s, VmPeak: 790 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.137176 MB Blob 7 finished, total: Wall: 0.82 s, CPU: 0.81 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092656 MB Blob 6 finished, total: Wall: 0.91 s, CPU: 0.90 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093484 MB Blob 9 finished, total: Wall: 0.97 s, CPU: 0.97 s, VmPeak: 805 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.1505 MB Blob 5 finished, total: Wall: 1.04 s, CPU: 1.04 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149576 MB Blob 2 finished, total: Wall: 1.38 s, CPU: 1.38 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149788 MB Blob 3 finished, total: Wall: 1.39 s, CPU: 1.39 s, VmPeak: 790 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.136952 MB Blob 4 finished, total: Wall: 1.48 s, CPU: 1.47 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149976 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.27296, ee1=0.125153, ee2=0.179404 Measuring 8 iterative sources Blob 1 finished, total: Wall: 20.60 s, CPU: 20.58 s, VmPeak: 810 MB, VmSize: 735 MB, VmRSS: 158 MB, VmData: 187 MB, maxrss: 0.161748 MB Old catalog: 46 New catalog: 44 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/blobs-custom-273679p37367.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/all-models-custom-273679p37367.fits Stage fitblobs : Wall: 21.38 s, CPU: 0.59 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 266 MB, VmData: 291 MB, maxrss: 0.272588 MB Stage fitblobs finished: 2020-08-05T19:54:47.663613 Resources for stage fitblobs : 31.294 s worker CPU, 31.529 s worker Wall, Wall: 21.378 s, Cores in use: 1.49, Total efficiency (on 16 cores): 9.3 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:54:47.681389 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-resid.jpg Stage coadds : Wall: 3.41 s, CPU: 1.26 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 267 MB, VmData: 291 MB, maxrss: 0.272912 MB Stage coadds finished: 2020-08-05T19:54:51.092032 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 9.608 s worker CPU, 15.396 s worker Wall, Wall: 3.412 s, Cores in use: 3.19, Total efficiency (on 16 cores): 19.9 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-05T19:54:51.108445 Cut to 1 unWISE tiles Reading WISE tile 2728p378 band 1 Reading WISE tile 2728p378 band 4 Reading WISE tile 2728p378 band 3 Reading WISE tile 2728p378 band 2 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 811 MB, VmSize: 809 MB, VmRSS: 151 MB, VmData: 177 MB, maxrss: 0.1918 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 805 MB, VmSize: 803 MB, VmRSS: 144 MB, VmData: 171 MB, maxrss: 0.20752 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 808 MB, VmSize: 806 MB, VmRSS: 147 MB, VmData: 174 MB, maxrss: 0.210232 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 812 MB, VmSize: 805 MB, VmRSS: 146 MB, VmData: 173 MB, maxrss: 0.20946 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 3.06 s, CPU: 1.17 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 267 MB, VmData: 291 MB, maxrss: 0.273424 MB Stage wise_forced finished: 2020-08-05T19:54:54.164517 Resources for stage wise_forced : 3.292 s worker CPU, 5.136 s worker Wall, Wall: 3.058 s, Cores in use: 1.46, Total efficiency (on 16 cores): 9.1 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:54:54.183150 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/tractor-i/cus/tractor-custom-273679p37367.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/tractor/cus/tractor-custom-273679p37367.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/tractor/cus/brick-custom-273679p37367.sha256sum Stage writecat : Wall: 1.95 s, CPU: 1.08 s, VmPeak: 1173 MB, VmSize: 1045 MB, VmRSS: 269 MB, VmData: 291 MB, maxrss: 0.406304 MB Stage writecat finished: 2020-08-05T19:54:56.133364 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.952 s, Cores in use: 0.55, Total efficiency (on 16 cores): 3.5 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-writecat.p All done: 50.862 s worker CPU, 104.711 s worker Wall, Wall: 53.423 s, Cores in use: 1.22, Total efficiency (on 16 cores): 7.6 % Finished galaxy PGC2099102 in 0.946 minutes. Started working on galaxy PGC2099102 at Wed Aug 5 20:49:35 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.679542 37.36794879999999 --width 251 --height 251 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T20:49:36.671710 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '273.679542', '37.36794879999999', '--width', '251', '--height', '251', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.679542 37.36794879999999 --width 251 --height 251 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 273.679542 37.36794879999999 Starting process 43470 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 43471 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 43472 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 43473 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 43474 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05934 MB Starting process 43475 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059328 MB Starting process 43476 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059328 MB Starting process 43477 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 43478 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 43479 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 43480 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 43481 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 43482 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 43483 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 43484 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 43485 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 43486 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 43487 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 43488 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 43489 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 43490 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 43491 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 43492 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 43493 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 43495 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 43496 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 43497 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 43498 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 43499 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 43501 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 43500 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 43502 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T20:49:37.249145 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: k4m_160414_115749_ooi_zd_ls9-CCD2 z expnum 82276 exptime 80.102 propid 2016A-0453 seeing 0.84 MJD 57492.498 object MzLS_590546_z mosaic/CP/V4.3/CP20160413/k4m_160414_115749_ooi_zd_ls9.fits.fz k4m_160427_113004_ooi_zd_ls9-CCD1 z expnum 85294 exptime 80.103 propid 2016A-0453 seeing 0.73 MJD 57505.479 object MzLS_592537_z mosaic/CP/V4.3/CP20160426/k4m_160427_113004_ooi_zd_ls9.fits.fz k4m_160430_104744_ooi_zd_ls9-CCD1 z expnum 86079 exptime 80.104 propid 2016A-0453 seeing 0.83 MJD 57508.450 object MzLS_591543_z mosaic/CP/V4.3/CP20160429/k4m_160430_104744_ooi_zd_ls9.fits.fz k4m_160427_110333_ooi_zd_ls9-CCD4 z expnum 85282 exptime 80.104 propid 2016A-0453 seeing 0.70 MJD 57505.461 object MzLS_589546_z mosaic/CP/V4.3/CP20160426/k4m_160427_110333_ooi_zd_ls9.fits.fz ksb_160703_073031_ooi_g_ls9-CCD3 g expnum 75720046 exptime 71.8 propid BASS seeing 1.16 MJD 57572.313 object 10942 90prime/CP/V2.3/CP20160702/ksb_160703_073031_ooi_g_ls9.fits.fz ksb_160513_103534_ooi_r_ls9-CCD3 r expnum 75210210 exptime 62.6 propid BASS seeing 1.21 MJD 57521.441 object 10942 90prime/CP/V2.3/CP20160512/ksb_160513_103534_ooi_r_ls9.fits.fz ksb_170430_112929_ooi_r_ls9-CCD2 r expnum 78730270 exptime 161.859 propid BASS seeing 1.62 MJD 57873.479 object 12273 90prime/CP/V2.3/CP20170429/ksb_170430_112929_ooi_r_ls9.fits.fz ksb_160705_070952_ooi_g_ls9-CCD2 g expnum 75740098 exptime 115.6 propid BASS seeing 1.84 MJD 57574.299 object 12273 90prime/CP/V2.3/CP20160704/ksb_160705_070952_ooi_g_ls9.fits.fz ksb_160430_103438_ooi_r_ls9-CCD1 r expnum 75080183 exptime 130.6 propid BASS seeing 1.38 MJD 57508.441 object 10941 90prime/CP/V2.3/CP20160429/ksb_160430_103438_ooi_r_ls9.fits.fz ksb_160504_101953_ooi_g_ls9-CCD1 g expnum 75120196 exptime 91.7 propid BASS seeing 1.54 MJD 57512.430 object 10941 90prime/CP/V2.3/CP20160503/ksb_160504_101953_ooi_g_ls9.fits.fz Faked up EXPNUM 78730270 Faked up EXPNUM 78730270 Faked up EXPNUM 78730270 90prime get_fwhm: self.fwhm = 3.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2863 3010 and amp range 2016 4031 and image y 1883 2030 and amp range 0 2047 Range in image: x 0 148 , y 0 148 (with image size 148 x 148) dzp 0.032776505 -> scaling image by 1.0306485625912574 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75080183 Faked up EXPNUM 75080183 Faked up EXPNUM 75080183 90prime get_fwhm: self.fwhm = 3.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1095 1243 and amp range 0 2015 and image y 2843 2990 and amp range 2048 4095 Range in image: x 0 149 , y 0 148 (with image size 149 x 148) dzp -0.00015519658 -> scaling image by 0.9998570688798908 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 2.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 4036 4088 and amp range 2048 4095 and image y 2795 3054 and amp range 2022 4060 Range in image: x 0 53 , y 0 260 (with image size 53 x 260) dzp -0.0028835079 -> scaling image by 0.9973477146455455 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2855 3107 and amp range 2006 4053 and image y 3952 4072 and amp range 2040 4078 Range in image: x 0 253 , y 0 121 (with image size 253 x 121) dzp -0.006092005 -> scaling image by 0.9944047680378836 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75720046 Faked up EXPNUM 75720046 Faked up EXPNUM 75720046 90prime get_fwhm: self.fwhm = 2.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3110 3256 and amp range 2016 4031 and image y 363 508 and amp range 0 2047 Range in image: x 0 147 , y 0 146 (with image size 147 x 146) dzp -0.0022715968 -> scaling image by 0.9979099691834846 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75740098 Faked up EXPNUM 75740098 Faked up EXPNUM 75740098 90prime get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2866 3013 and amp range 2016 4031 and image y 1881 2029 and amp range 0 2047 Range in image: x 0 148 , y 0 149 (with image size 148 x 149) dzp -0.008960029 -> scaling image by 0.9917814665476246 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75120196 Faked up EXPNUM 75120196 Faked up EXPNUM 75120196 90prime get_fwhm: self.fwhm = 3.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1099 1247 and amp range 0 2015 and image y 2845 2992 and amp range 2048 4095 Range in image: x 0 149 , y 0 148 (with image size 149 x 148) dzp -0.0051191514 -> scaling image by 0.9952961850385206 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.23 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1290 1543 and amp range 0 2047 and image y 3938 4072 and amp range 2040 4078 Range in image: x 0 254 , y 0 135 (with image size 254 x 135) dzp 0.008550294 -> scaling image by 1.0079062018164673 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75210210 Faked up EXPNUM 75210210 Faked up EXPNUM 75210210 90prime get_fwhm: self.fwhm = 2.67 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3127 3273 and amp range 2016 4031 and image y 417 563 and amp range 0 2047 Range in image: x 0 147 , y 0 147 (with image size 147 x 147) dzp 0.004245165 -> scaling image by 1.0039175950705865 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 2.81 Found 4 relevant rows in amp-corrections file. Found overlap: image x 137 396 and amp range 0 2005 and image y 3106 3360 and amp range 2040 4078 Range in image: x 0 260 , y 0 255 (with image size 260 x 255) dzp -0.005990375 -> scaling image by 0.994497853444703 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 11.15 s, CPU: 0.85 s, VmPeak: 958 MB, VmSize: 958 MB, VmRSS: 175 MB, VmData: 207 MB, maxrss: 0.178912 MB Stage tims finished: 2020-08-05T20:49:48.403187 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.289 s worker CPU, 31.915 s worker Wall, Wall: 11.157 s, Cores in use: 0.55, Total efficiency (on 32 cores): 1.7 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T20:49:48.423289 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02466.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02465.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02337.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/reference-custom-273679p37367.fits Stage refs : Wall: 14.21 s, CPU: 7.19 s, VmPeak: 1037 MB, VmSize: 1031 MB, VmRSS: 251 MB, VmData: 281 MB, maxrss: 0.257328 MB Stage refs finished: 2020-08-05T20:50:02.637876 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 14.217 s, Cores in use: 0.51, Total efficiency (on 32 cores): 1.6 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T20:50:02.668211 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outlier-mask-custom-273679p37367.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-pre-custom-273679p37367.jpg Image k4m_160414_115749_ooi_zd_ls9-CCD2 z : masking 6812 positive outlier pixels and 0 negative outlier pixels Image k4m_160427_110333_ooi_zd_ls9-CCD4 z : masking 2223 positive outlier pixels and 0 negative outlier pixels Image k4m_160427_113004_ooi_zd_ls9-CCD1 z : masking 130 positive outlier pixels and 825 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outlier-mask-custom-273679p37367.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-post-custom-273679p37367.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-masked-pos-custom-273679p37367.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/outliers-masked-neg-custom-273679p37367.jpg Stage outliers : Wall: 1.64 s, CPU: 1.20 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 264 MB, VmData: 291 MB, maxrss: 0.27054 MB Stage outliers finished: 2020-08-05T20:50:04.309812 Resources for stage outliers : 0.769 s worker CPU, 1.134 s worker Wall, Wall: 1.644 s, Cores in use: 1.20, Total efficiency (on 32 cores): 3.7 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T20:50:04.341850 Stage halos : Wall: 0.63 s, CPU: 0.62 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 264 MB, VmData: 291 MB, maxrss: 0.270828 MB Stage halos finished: 2020-08-05T20:50:04.971807 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.632 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T20:50:05.075092 PSF sigmas in coadd pixels: ['1.89', '2.98', '2.51'] Using average PSF sigma 2.460048803979209 Gaussian PSF norm 0.11467040463489207 vs pixelized 0.09825789 average tim pixel scale / coadd scale: 1.7291134425957921 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.96', '2.63', '2.24'] Using average PSF sigma 2.2793192950983054 Gaussian PSF norm 0.12376273582228048 vs pixelized 0.1006384 average tim pixel scale / coadd scale: 1.7296827001895272 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.36', '1.18', '1.35', '1.14'] Using average PSF sigma 1.260845348359652 Gaussian PSF norm 0.2237346492501089 vs pixelized 0.18630978 average tim pixel scale / coadd scale: 0.9909641990288808 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.16 s, CPU: 0.14 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 266 MB, VmData: 291 MB, maxrss: 0.272404 MB Stage fit_on_coadds finished: 2020-08-05T20:50:05.238633 Warning: Stellar halo subtraction is only implemented for DECam Resources for stage fit_on_coadds : 0.159 s worker CPU, 0.159 s worker Wall, Wall: 0.166 s, Cores in use: 1.83, Total efficiency (on 32 cores): 5.7 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-05T20:50:05.257093 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 42 potential peaks Of 42 potential peaks: 15 in veto map, 4 cut by saddle test, 0 cut by aper test, 23 kept SED z : 23 new peaks Found 27 potential peaks Of 27 potential peaks: 23 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 21 potential peaks Of 21 potential peaks: 19 in veto map, 1 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 43 potential peaks Of 43 potential peaks: 39 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept SED Flat : 0 new peaks Found 72 potential peaks Of 72 potential peaks: 62 in veto map, 7 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 28 new sources Median coadd sky for g : -1.1149298e-05 Median coadd sky for r : 9.977016e-05 Median coadd sky for z : 0.0002512724 Stage srcs : Wall: 0.88 s, CPU: 0.84 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 266 MB, VmData: 291 MB, maxrss: 0.272868 MB Stage srcs finished: 2020-08-05T20:50:06.140272 Resources for stage srcs : 0.087 s worker CPU, 0.087 s worker Wall, Wall: 0.885 s, Cores in use: 1.05, Total efficiency (on 32 cores): 3.3 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T20:50:06.157341 Blob 1 of 15, id: 0, sources: 20, size: 145x245, npix 18933, brick X: 29,174, Y: 0,245, one pixel: 86 0 Blob 2 of 15, id: 10, sources: 3, size: 75x49, npix 2076, brick X: 9,84, Y: 168,217, one pixel: 73 168 Blob 3 of 15, id: 12, sources: 2, size: 41x46, npix 1208, brick X: 157,198, Y: 194,240, one pixel: 177 194 Blob 4 of 15, id: 6, sources: 3, size: 40x49, npix 1084, brick X: 5,45, Y: 75,124, one pixel: 14 75 Blob 5 of 15, id: 7, sources: 1, size: 30x35, npix 793, brick X: 221,251, Y: 104,139, one pixel: 236 104 Blob 6 of 15, id: 4, sources: 1, size: 33x33, npix 768, brick X: 149,182, Y: 42,75, one pixel: 162 42 Blob 7 of 15, id: 3, sources: 1, size: 30x29, npix 575, brick X: 217,247, Y: 30,59, one pixel: 230 30 Blob 8 of 15, id: 1, sources: 1, size: 26x26, npix 419, brick X: 15,41, Y: 3,29, one pixel: 28 3 Blob 9 of 15, id: 11, sources: 1, size: 25x24, npix 377, brick X: 195,220, Y: 184,208, one pixel: 205 184 Blob 10 of 15, id: 9, sources: 1, size: 22x22, npix 312, brick X: 46,68, Y: 141,163, one pixel: 54 141 Blob 11 of 15, id: 5, sources: 1, size: 23x24, npix 304, brick X: 210,233, Y: 74,98, one pixel: 219 74 Blob 12 of 15, id: 13, sources: 1, size: 21x20, npix 249, brick X: 46,67, Y: 207,227, one pixel: 55 207 Blob 13 of 15, id: 8, sources: 1, size: 20x19, npix 218, brick X: 196,216, Y: 137,156, one pixel: 204 137 Blob 14 of 15, id: 14, sources: 1, size: 19x18, npix 189, brick X: 226,245, Y: 219,237, one pixel: 235 219 Blob 15 of 15, id: 2, sources: 1, size: 18x17, npix 162, brick X: 130,148, Y: 30,47, one pixel: 138 30 Blob 15 finished, total: Wall: 0.12 s, CPU: 0.10 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 85 MB, VmData: 138 MB, maxrss: 0.087424 MB Blob 10 finished, total: Wall: 0.18 s, CPU: 0.16 s, VmPeak: 811 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.14962 MB Blob 13 finished, total: Wall: 0.21 s, CPU: 0.19 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09246 MB Blob 12 finished, total: Wall: 0.21 s, CPU: 0.19 s, VmPeak: 790 MB, VmSize: 719 MB, VmRSS: 133 MB, VmData: 170 MB, maxrss: 0.136424 MB Blob 14 finished, total: Wall: 0.38 s, CPU: 0.37 s, VmPeak: 812 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.14958 MB Blob 11 finished, total: Wall: 0.41 s, CPU: 0.38 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092068 MB Blob 8 finished, total: Wall: 0.45 s, CPU: 0.42 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092084 MB Blob 7 finished, total: Wall: 0.64 s, CPU: 0.63 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092292 MB Blob 9 finished, total: Wall: 0.66 s, CPU: 0.64 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09194 MB Blob 5 finished, total: Wall: 0.79 s, CPU: 0.78 s, VmPeak: 805 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149784 MB Blob 6 finished, total: Wall: 0.79 s, CPU: 0.79 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.093992 MB Blob 3 finished, total: Wall: 1.13 s, CPU: 1.11 s, VmPeak: 790 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.13734 MB Blob 2 finished, total: Wall: 1.16 s, CPU: 1.15 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.093704 MB Blob 4 finished, total: Wall: 1.39 s, CPU: 1.38 s, VmPeak: 791 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.136784 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.27296, ee1=0.125153, ee2=0.179404 Measuring 8 iterative sources Blob 1 finished, total: Wall: 19.80 s, CPU: 19.76 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 125 MB, VmData: 164 MB, maxrss: 0.129272 MB Old catalog: 46 New catalog: 44 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/blobs-custom-273679p37367.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/metrics/cus/all-models-custom-273679p37367.fits Stage fitblobs : Wall: 20.82 s, CPU: 0.97 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 267 MB, VmData: 292 MB, maxrss: 0.273812 MB Stage fitblobs finished: 2020-08-05T20:50:26.973734 Resources for stage fitblobs : 28.245 s worker CPU, 29.444 s worker Wall, Wall: 20.819 s, Cores in use: 1.40, Total efficiency (on 32 cores): 4.4 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T20:50:26.991941 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-resid.jpg Stage coadds : Wall: 4.38 s, CPU: 0.92 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 268 MB, VmData: 292 MB, maxrss: 0.274136 MB Stage coadds finished: 2020-08-05T20:50:31.372185 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 25.437 s worker CPU, 47.551 s worker Wall, Wall: 4.383 s, Cores in use: 6.01, Total efficiency (on 32 cores): 18.8 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T20:50:31.390045 Cut to 1 unWISE tiles Reading WISE tile 2728p378 band 1 Reading WISE tile 2728p378 band 2 Reading WISE tile 2728p378 band 3 Reading WISE tile 2728p378 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 811 MB, VmSize: 804 MB, VmRSS: 144 MB, VmData: 172 MB, maxrss: 0.207508 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 809 MB, VmSize: 809 MB, VmRSS: 150 MB, VmData: 177 MB, maxrss: 0.194804 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 813 MB, VmSize: 799 MB, VmRSS: 140 MB, VmData: 171 MB, maxrss: 0.202592 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 813 MB, VmSize: 800 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.203992 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/tmp-legacysurvey-custom-273679p37367-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.67 s, CPU: 0.67 s, VmPeak: 1041 MB, VmSize: 1041 MB, VmRSS: 268 MB, VmData: 292 MB, maxrss: 0.274648 MB Stage wise_forced finished: 2020-08-05T20:50:34.057744 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.366 s worker CPU, 5.194 s worker Wall, Wall: 2.670 s, Cores in use: 1.51, Total efficiency (on 32 cores): 4.7 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-wise_forced.p Running stage writecat WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage writecat at 2020-08-05T20:50:34.129928 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/coadd/cus/custom-273679p37367/legacysurvey-custom-273679p37367-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/tractor-i/cus/tractor-custom-273679p37367.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/tractor/cus/tractor-custom-273679p37367.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/tractor/cus/brick-custom-273679p37367.sha256sum Stage writecat : Wall: 1.24 s, CPU: 0.65 s, VmPeak: 1173 MB, VmSize: 1045 MB, VmRSS: 270 MB, VmData: 292 MB, maxrss: 0.407528 MB Stage writecat finished: 2020-08-05T20:50:35.366197 Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.239 s, Cores in use: 0.53, Total efficiency (on 32 cores): 1.7 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2099102/PGC2099102-largegalaxy-writecat.p All done: 63.359 s worker CPU, 115.492 s worker Wall, Wall: 58.140 s, Cores in use: 1.33, Total efficiency (on 32 cores): 4.2 % Finished galaxy PGC2099102 in 1.015 minutes. Started working on galaxy PGC2099102 at Fri Aug 7 09:35:37 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.67985741669423 37.3681810494684 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T09:35:39.465842 legacypipe git version: DR9.5.11-watershed-3-gd509eaaf Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '273.67985741669423', '37.3681810494684', '--width', '251', '--height', '251', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.67985741669423 37.3681810494684 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 273.67985741669423 37.3681810494684 Starting process 23419 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 23420 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 23421 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 23422 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 23423 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 23424 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 23425 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 23426 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 23427 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 23428 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 23429 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 23430 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 23432 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 23433 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 23434 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 23431 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T09:35:39.960521 Keeping 10 CCDs: k4m_160414_115749_ooi_zd_ls9-CCD2 z expnum 82276 exptime 80.102 propid 2016A-0453 seeing 0.84 MJD 57492.498 object MzLS_590546_z mosaic/CP/V4.3/CP20160413/k4m_160414_115749_ooi_zd_ls9.fits.fz k4m_160427_113004_ooi_zd_ls9-CCD1 z expnum 85294 exptime 80.103 propid 2016A-0453 seeing 0.73 MJD 57505.479 object MzLS_592537_z mosaic/CP/V4.3/CP20160426/k4m_160427_113004_ooi_zd_ls9.fits.fz k4m_160430_104744_ooi_zd_ls9-CCD1 z expnum 86079 exptime 80.104 propid 2016A-0453 seeing 0.83 MJD 57508.450 object MzLS_591543_z mosaic/CP/V4.3/CP20160429/k4m_160430_104744_ooi_zd_ls9.fits.fz k4m_160427_110333_ooi_zd_ls9-CCD4 z expnum 85282 exptime 80.104 propid 2016A-0453 seeing 0.70 MJD 57505.461 object MzLS_589546_z mosaic/CP/V4.3/CP20160426/k4m_160427_110333_ooi_zd_ls9.fits.fz ksb_160703_073031_ooi_g_ls9-CCD3 g expnum 75720046 exptime 71.8 propid BASS seeing 1.16 MJD 57572.313 object 10942 90prime/CP/V2.3/CP20160702/ksb_160703_073031_ooi_g_ls9.fits.fz ksb_160513_103534_ooi_r_ls9-CCD3 r expnum 75210210 exptime 62.6 propid BASS seeing 1.21 MJD 57521.441 object 10942 90prime/CP/V2.3/CP20160512/ksb_160513_103534_ooi_r_ls9.fits.fz ksb_170430_112929_ooi_r_ls9-CCD2 r expnum 78730270 exptime 161.859 propid BASS seeing 1.62 MJD 57873.479 object 12273 90prime/CP/V2.3/CP20170429/ksb_170430_112929_ooi_r_ls9.fits.fz ksb_160705_070952_ooi_g_ls9-CCD2 g expnum 75740098 exptime 115.6 propid BASS seeing 1.84 MJD 57574.299 object 12273 90prime/CP/V2.3/CP20160704/ksb_160705_070952_ooi_g_ls9.fits.fz ksb_160430_103438_ooi_r_ls9-CCD1 r expnum 75080183 exptime 130.6 propid BASS seeing 1.38 MJD 57508.441 object 10941 90prime/CP/V2.3/CP20160429/ksb_160430_103438_ooi_r_ls9.fits.fz ksb_160504_101953_ooi_g_ls9-CCD1 g expnum 75120196 exptime 91.7 propid BASS seeing 1.54 MJD 57512.430 object 10941 90prime/CP/V2.3/CP20160503/ksb_160504_101953_ooi_g_ls9.fits.fz Faked up EXPNUM 75080183 Faked up EXPNUM 75080183 Faked up EXPNUM 75080183 90prime get_fwhm: self.fwhm = 3.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1093 1241 and amp range 0 2015 and image y 2841 2988 and amp range 2048 4095 Range in image: x 0 149 , y 0 148 (with image size 149 x 148) dzp -0.00015519658 -> scaling image by 0.9998570688798908 mosaic get_fwhm: self.fwhm = 3.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2858 3110 and amp range 2006 4053 and image y 3956 4072 and amp range 2040 4078 Range in image: x 0 253 , y 0 117 (with image size 253 x 117) dzp -0.006092005 -> scaling image by 0.9944047680378836 mosaic get_fwhm: self.fwhm = 3.23 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1293 1546 and amp range 0 2047 and image y 3942 4072 and amp range 2040 4078 Range in image: x 0 254 , y 0 131 (with image size 254 x 131) dzp 0.008550294 -> scaling image by 1.0079062018164673 Faked up EXPNUM 75740098 Faked up EXPNUM 75740098 Faked up EXPNUM 75740098 90prime get_fwhm: self.fwhm = 4.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2864 3011 and amp range 2016 4031 and image y 1879 2027 and amp range 0 2047 Range in image: x 0 148 , y 0 149 (with image size 148 x 149) dzp -0.008960029 -> scaling image by 0.9917814665476246 Faked up EXPNUM 78730270 Faked up EXPNUM 78730270 Faked up EXPNUM 78730270 90prime get_fwhm: self.fwhm = 3.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2862 3008 and amp range 2016 4031 and image y 1881 2028 and amp range 0 2047 Range in image: x 0 147 , y 0 148 (with image size 147 x 148) dzp 0.032776505 -> scaling image by 1.0306485625912574 Faked up EXPNUM 75210210 Faked up EXPNUM 75210210 Faked up EXPNUM 75210210 90prime get_fwhm: self.fwhm = 2.67 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3125 3271 and amp range 2016 4031 and image y 415 561 and amp range 0 2047 Range in image: x 0 147 , y 0 147 (with image size 147 x 147) dzp 0.004245165 -> scaling image by 1.0039175950705865 Faked up EXPNUM 75120196 Faked up EXPNUM 75120196 Faked up EXPNUM 75120196 90prime get_fwhm: self.fwhm = 3.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1097 1245 and amp range 0 2015 and image y 2843 2990 and amp range 2048 4095 Range in image: x 0 149 , y 0 148 (with image size 149 x 148) dzp -0.0051191514 -> scaling image by 0.9952961850385206 mosaic get_fwhm: self.fwhm = 2.71 Found 4 relevant rows in amp-corrections file. Found overlap: image x 4039 4088 and amp range 2048 4095 and image y 2799 3057 and amp range 2022 4060 Range in image: x 0 50 , y 0 259 (with image size 50 x 259) dzp -0.0028835079 -> scaling image by 0.9973477146455455 mosaic get_fwhm: self.fwhm = 2.81 Found 4 relevant rows in amp-corrections file. Found overlap: image x 140 399 and amp range 0 2005 and image y 3110 3364 and amp range 2040 4078 Range in image: x 0 260 , y 0 255 (with image size 260 x 255) dzp -0.005990375 -> scaling image by 0.994497853444703 Faked up EXPNUM 75720046 Faked up EXPNUM 75720046 Faked up EXPNUM 75720046 90prime get_fwhm: self.fwhm = 2.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3108 3254 and amp range 2016 4031 and image y 361 507 and amp range 0 2047 Range in image: x 0 147 , y 0 147 (with image size 147 x 147) dzp -0.0022715968 -> scaling image by 0.9979099691834846 Stage tims : Wall: 140.14 s, CPU: 0.97 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 172 MB, VmData: 206 MB, maxrss: 0.176584 MB Stage tims finished: 2020-08-07T09:38:00.101719 Resources for stage tims : 5.020 s worker CPU, 757.983 s worker Wall, Wall: 140.143 s, Cores in use: 0.04, Total efficiency (on 16 cores): 0.3 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T09:38:00.148711 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02466.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02465.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02337.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/reference-custom-273679p37368.fits Stage refs : Wall: 13.82 s, CPU: 6.68 s, VmPeak: 1030 MB, VmSize: 1025 MB, VmRSS: 249 MB, VmData: 280 MB, maxrss: 0.25458 MB Stage refs finished: 2020-08-07T09:38:13.965361 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 13.818 s, Cores in use: 0.48, Total efficiency (on 16 cores): 3.0 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T09:38:14.684076 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/outlier-mask-custom-273679p37368.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/outliers-pre-custom-273679p37368.jpg Image k4m_160414_115749_ooi_zd_ls9-CCD2 z : masking 6848 positive outlier pixels and 0 negative outlier pixels Image k4m_160427_110333_ooi_zd_ls9-CCD4 z : masking 2195 positive outlier pixels and 0 negative outlier pixels Image k4m_160427_113004_ooi_zd_ls9-CCD1 z : masking 138 positive outlier pixels and 891 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/outlier-mask-custom-273679p37368.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/outliers-post-custom-273679p37368.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/outliers-masked-pos-custom-273679p37368.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/outliers-masked-neg-custom-273679p37368.jpg Stage outliers : Wall: 0.92 s, CPU: 0.62 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 261 MB, VmData: 289 MB, maxrss: 0.267124 MB Stage outliers finished: 2020-08-07T09:38:15.605083 Resources for stage outliers : 0.653 s worker CPU, 0.702 s worker Wall, Wall: 0.923 s, Cores in use: 1.38, Total efficiency (on 16 cores): 8.7 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T09:38:15.691311 Stage halos : Wall: 0.28 s, CPU: 0.28 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 261 MB, VmData: 289 MB, maxrss: 0.267212 MB Stage halos finished: 2020-08-07T09:38:15.975221 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.285 s, Cores in use: 0.97, Total efficiency (on 16 cores): 6.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T09:38:16.002811 PSF sigmas in coadd pixels: ['1.89', '2.98', '2.51'] Using average PSF sigma 2.460048803979209 Gaussian PSF norm 0.11467040463489207 vs pixelized 0.09826019 average tim pixel scale / coadd scale: 1.7291134425957921 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.96', '2.63', '2.24'] Using average PSF sigma 2.2793192950983054 Gaussian PSF norm 0.12376273582228048 vs pixelized 0.10064063 average tim pixel scale / coadd scale: 1.7296827001895272 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.36', '1.18', '1.35', '1.14'] Using average PSF sigma 1.260845348359652 Gaussian PSF norm 0.2237346492501089 vs pixelized 0.18630779 average tim pixel scale / coadd scale: 0.9909641990288808 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.19 s, CPU: 0.14 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 262 MB, VmData: 289 MB, maxrss: 0.268712 MB Stage fit_on_coadds finished: 2020-08-07T09:38:16.192161 Resources for stage fit_on_coadds : 0.192 s worker CPU, 0.192 s worker Wall, Wall: 0.191 s, Cores in use: 1.72, Total efficiency (on 16 cores): 10.8 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-07T09:38:16.239239 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 46 potential peaks Of 46 potential peaks: 17 in veto map, 6 cut by saddle test, 0 cut by aper test, 23 kept SED z : 23 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 22 potential peaks Of 22 potential peaks: 20 in veto map, 1 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 45 potential peaks Of 45 potential peaks: 41 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept SED Flat : 0 new peaks Found 72 potential peaks Of 72 potential peaks: 63 in veto map, 6 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 28 new sources Median coadd sky for g : -1.1640005e-05 Median coadd sky for r : 9.05863e-05 Median coadd sky for z : 0.00012737185 Stage srcs : Wall: 0.57 s, CPU: 0.54 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 263 MB, VmData: 289 MB, maxrss: 0.2692 MB Stage srcs finished: 2020-08-07T09:38:16.811151 Resources for stage srcs : 0.073 s worker CPU, 0.073 s worker Wall, Wall: 0.573 s, Cores in use: 1.07, Total efficiency (on 16 cores): 6.7 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T09:38:16.832663 Blob 1 of 17, id: 1, sources: 19, size: 145x242, npix 18941, brick X: 32,177, Y: 0,242, one pixel: 76 0 Blob 2 of 17, id: 11, sources: 1, size: 47x48, npix 1534, brick X: 12,59, Y: 166,214, one pixel: 36 166 Blob 3 of 17, id: 13, sources: 2, size: 41x46, npix 1210, brick X: 161,202, Y: 191,237, one pixel: 180 191 Blob 4 of 17, id: 5, sources: 3, size: 41x48, npix 1087, brick X: 8,49, Y: 72,120, one pixel: 17 72 Blob 5 of 17, id: 3, sources: 1, size: 34x34, npix 761, brick X: 152,186, Y: 38,72, one pixel: 167 38 Blob 6 of 17, id: 7, sources: 1, size: 27x35, npix 723, brick X: 224,251, Y: 101,136, one pixel: 239 101 Blob 7 of 17, id: 2, sources: 1, size: 30x29, npix 565, brick X: 220,250, Y: 27,56, one pixel: 233 27 Blob 8 of 17, id: 10, sources: 2, size: 28x36, npix 558, brick X: 60,88, Y: 165,201, one pixel: 76 165 Blob 9 of 17, id: 0, sources: 1, size: 26x26, npix 437, brick X: 18,44, Y: 0,26, one pixel: 31 0 Blob 10 of 17, id: 12, sources: 1, size: 24x24, npix 383, brick X: 199,223, Y: 180,204, one pixel: 210 180 Blob 11 of 17, id: 9, sources: 1, size: 23x22, npix 304, brick X: 49,72, Y: 138,160, one pixel: 58 138 Blob 12 of 17, id: 4, sources: 1, size: 24x24, npix 297, brick X: 213,237, Y: 71,95, one pixel: 222 71 Blob 13 of 17, id: 16, sources: 1, size: 26x15, npix 294, brick X: 3,29, Y: 236,251, one pixel: 12 236 Blob 14 of 17, id: 6, sources: 1, size: 15x27, npix 288, brick X: 0,15, Y: 99,126, one pixel: 3 99 Blob 15 of 17, id: 14, sources: 1, size: 21x20, npix 249, brick X: 50,71, Y: 204,224, one pixel: 58 204 Blob 16 of 17, id: 8, sources: 1, size: 20x19, npix 218, brick X: 199,219, Y: 134,153, one pixel: 208 134 Blob 17 of 17, id: 15, sources: 1, size: 18x18, npix 180, brick X: 230,248, Y: 216,234, one pixel: 238 216 Blob 11 finished, total: Wall: 0.25 s, CPU: 0.25 s, VmPeak: 790 MB, VmSize: 712 MB, VmRSS: 129 MB, VmData: 169 MB, maxrss: 0.132544 MB Blob 16 finished, total: Wall: 0.33 s, CPU: 0.23 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145284 MB Blob 13 finished, total: Wall: 0.34 s, CPU: 0.34 s, VmPeak: 790 MB, VmSize: 711 MB, VmRSS: 129 MB, VmData: 169 MB, maxrss: 0.132404 MB Blob 15 finished, total: Wall: 0.39 s, CPU: 0.35 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145848 MB Blob 8 finished, total: Wall: 0.57 s, CPU: 0.57 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093216 MB Blob 12 finished, total: Wall: 0.57 s, CPU: 0.51 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146104 MB Blob 9 finished, total: Wall: 0.67 s, CPU: 0.67 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092132 MB Blob 7 finished, total: Wall: 0.79 s, CPU: 0.78 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 137 MB, maxrss: 0.0918 MB Blob 17 finished, total: Wall: 0.55 s, CPU: 0.55 s, VmPeak: 790 MB, VmSize: 712 MB, VmRSS: 129 MB, VmData: 169 MB, maxrss: 0.132544 MB Blob 5 finished, total: Wall: 0.90 s, CPU: 0.83 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092488 MB Blob 2 finished, total: Wall: 0.95 s, CPU: 0.94 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.14578 MB Blob 6 finished, total: Wall: 0.95 s, CPU: 0.94 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092644 MB Blob 10 finished, total: Wall: 1.02 s, CPU: 0.88 s, VmPeak: 791 MB, VmSize: 712 MB, VmRSS: 129 MB, VmData: 169 MB, maxrss: 0.1322 MB Blob 14 finished, total: Wall: 1.18 s, CPU: 1.17 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 91 MB, VmData: 137 MB, maxrss: 0.093016 MB Blob 3 finished, total: Wall: 1.29 s, CPU: 1.29 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.14556 MB Blob 4 finished, total: Wall: 1.63 s, CPU: 1.62 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145824 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.03496, ee1=0.139695, ee2=0.158963 Measuring 11 iterative sources Blob 1 finished, total: Wall: 22.07 s, CPU: 22.06 s, VmPeak: 805 MB, VmSize: 729 MB, VmRSS: 153 MB, VmData: 184 MB, maxrss: 0.156168 MB Old catalog: 45 New catalog: 46 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/blobs-custom-273679p37368.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/metrics/cus/all-models-custom-273679p37368.fits Stage fitblobs : Wall: 29.03 s, CPU: 0.49 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 264 MB, VmData: 290 MB, maxrss: 0.270132 MB Stage fitblobs finished: 2020-08-07T09:38:45.861416 Resources for stage fitblobs : 34.262 s worker CPU, 35.147 s worker Wall, Wall: 29.031 s, Cores in use: 1.20, Total efficiency (on 16 cores): 7.5 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-07T09:38:45.916458 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/tmp-legacysurvey-custom-273679p37368-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/tmp-legacysurvey-custom-273679p37368-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/tmp-legacysurvey-custom-273679p37368-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/tmp-legacysurvey-custom-273679p37368-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-resid.jpg Stage coadds : Wall: 3.00 s, CPU: 0.68 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 264 MB, VmData: 290 MB, maxrss: 0.270456 MB Stage coadds finished: 2020-08-07T09:38:48.916164 Resources for stage coadds : 9.096 s worker CPU, 9.426 s worker Wall, Wall: 3.001 s, Cores in use: 3.26, Total efficiency (on 16 cores): 20.4 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-07T09:38:48.939576 Cut to 1 unWISE tiles Reading WISE tile 2728p378 band 1 Reading WISE tile 2728p378 band 3 Reading WISE tile 2728p378 band 2 Reading WISE tile 2728p378 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 813 MB, VmSize: 803 MB, VmRSS: 150 MB, VmData: 177 MB, maxrss: 0.190448 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 798 MB, VmSize: 796 MB, VmRSS: 139 MB, VmData: 170 MB, maxrss: 0.20092 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 812 MB, VmSize: 797 MB, VmRSS: 140 MB, VmData: 171 MB, maxrss: 0.2035 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/272/2728p378/unwise-2728p378-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 800 MB, VmSize: 798 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.2031 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/tmp-legacysurvey-custom-273679p37368-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/tmp-legacysurvey-custom-273679p37368-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 69.14 s, CPU: 0.61 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 265 MB, VmData: 290 MB, maxrss: 0.270968 MB Stage wise_forced finished: 2020-08-07T09:39:58.083308 Resources for stage wise_forced : 3.043 s worker CPU, 139.961 s worker Wall, Wall: 69.145 s, Cores in use: 0.05, Total efficiency (on 16 cores): 0.3 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-07T09:39:58.097858 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/coadd/cus/custom-273679p37368/legacysurvey-custom-273679p37368-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/tractor-i/cus/tractor-custom-273679p37368.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/tractor/cus/tractor-custom-273679p37368.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/tractor/cus/brick-custom-273679p37368.sha256sum Stage writecat : Wall: 0.98 s, CPU: 0.57 s, VmPeak: 1167 MB, VmSize: 1039 MB, VmRSS: 266 MB, VmData: 290 MB, maxrss: 0.403852 MB Stage writecat finished: 2020-08-07T09:39:59.078898 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.983 s, Cores in use: 0.58, Total efficiency (on 16 cores): 3.6 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2099102/PGC2099102-largegalaxy-writecat.p All done: 52.343 s worker CPU, 943.488 s worker Wall, Wall: 259.140 s, Cores in use: 0.25, Total efficiency (on 16 cores): 1.5 % Finished galaxy PGC2099102 in 4.366 minutes.