Started working on galaxy PGC2663521 at Mon Aug 3 06:40:42 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 253.90257150000002 64.0196294 --width 269 --height 269 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-03T06:40:43.602822 legacypipe git version: DR9.5.10-2-ge83842ef Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '253.90257150000002', '64.0196294', '--width', '269', '--height', '269', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 253.90257150000002 64.0196294 --width 269 --height 269 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 253.90257150000002 64.0196294 Starting process 22037 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 57 MB, VmData: 128 MB, maxrss: 0.058876 MB Starting process 22038 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 57 MB, VmData: 128 MB, maxrss: 0.058876 MB Starting process 22039 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05888 MB Starting process 22040 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.059048 MB Starting process 22041 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.059112 MB Starting process 22042 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.059112 MB Starting process 22043 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058884 MB Starting process 22044 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058884 MB Starting process 22045 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058892 MB Starting process 22046 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058892 MB Starting process 22047 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22048 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22049 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22050 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22051 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22052 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058904 MB Starting process 22053 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058908 MB Starting process 22054 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058908 MB Starting process 22055 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB Starting process 22056 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05892 MB Starting process 22057 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05892 MB Starting process 22058 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058924 MB Starting process 22059 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058928 MB Starting process 22060 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058928 MB Starting process 22061 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058928 MB Starting process 22062 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058928 MB Starting process 22063 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058928 MB Starting process 22064 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058932 MB Starting process 22065 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058932 MB Starting process 22066 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05894 MB Starting process 22067 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05894 MB Starting process 22068 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05894 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-03T06:40:46.599074 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 9 CCDs: k4m_170616_094923_ooi_zd_ls9-CCD1 z expnum 133722 exptime 101.102 propid 2016A-0453 seeing 0.95 MJD 57920.409 object MzLS_698762_z mosaic/CP/V4.3/CP20170615/k4m_170616_094923_ooi_zd_ls9.fits.fz k4m_170417_114757_ooi_zd_ls9-CCD2 z expnum 122694 exptime 80.105 propid 2016A-0453 seeing 1.04 MJD 57860.492 object MzLS_698206_z mosaic/CP/V4.3/CP20170416/k4m_170417_114757_ooi_zd_ls9.fits.fz k4m_170521_100103_ooi_zd_ls9-CCD1 z expnum 129255 exptime 99.104 propid 2016A-0453 seeing 1.09 MJD 57894.417 object MzLS_699314_z mosaic/CP/V4.3/CP20170520/k4m_170521_100103_ooi_zd_ls9.fits.fz ksb_180421_112921_ooi_r_ls9-CCD2 r expnum 82290183 exptime 157.818 propid BASS seeing 2.04 MJD 58229.479 object 41382 90prime/CP/V2.3/CP20180420/ksb_180421_112921_ooi_r_ls9.fits.fz ksb_170403_112240_ooi_g_ls9-CCD2 g expnum 78460209 exptime 118.444 propid BASS seeing 1.45 MJD 57846.474 object 41382 90prime/CP/V2.3/CP20170402/ksb_170403_112240_ooi_g_ls9.fits.fz ksb_180520_080206_ooi_r_ls9-CCD3 r expnum 82580074 exptime 140.88 propid NOTBASS seeing 1.95 MJD 58258.335 object 41393 90prime/CP/V2.3/CP20180519/ksb_180520_080206_ooi_r_ls9.fits.fz ksb_170619_044002_ooi_g_ls9-CCD3 g expnum 79230061 exptime 166.371 propid BASS seeing 1.59 MJD 57923.194 object 41393 90prime/CP/V2.3/CP20170618/ksb_170619_044002_ooi_g_ls9.fits.fz ksb_180317_115658_ooi_g_ls9-CCD3 g expnum 81940137 exptime 109.519 propid BASS seeing 1.56 MJD 58194.498 object 41391 90prime/CP/V2.3/CP20180316/ksb_180317_115658_ooi_g_ls9.fits.fz ksb_180423_105032_ooi_r_ls9-CCD3 r expnum 82310158 exptime 93.947 propid BASS seeing 1.40 MJD 58231.452 object 41391 90prime/CP/V2.3/CP20180422/ksb_180423_105032_ooi_r_ls9.fits.fz Faked up EXPNUM 82310158 Faked up EXPNUM 82310158 Faked up EXPNUM 82310158 90prime get_fwhm: self.fwhm = 3.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3946 4016 and amp range 2016 4031 and image y 2373 2529 and amp range 2048 4095 Faked up EXPNUM 82290183 Faked up EXPNUM 82290183 Faked up EXPNUM 82290183 90prime get_fwhm: self.fwhm = 4.49 Found 4 relevant rows in amp-corrections file. Range in image: x 0 71 , y 0 157 (with image size 71 x 157) dzp -0.0072602304 -> scaling image by 0.9933353884052984 Found overlap: image x 1606 1766 and amp range 0 2015 and image y 920 1081 and amp range 0 2047 Range in image: x 0 161 , y 0 162 (with image size 161 x 162) dzp -0.003875085 -> scaling image by 0.9964372765054066 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 79230061 Faked up EXPNUM 79230061 Faked up EXPNUM 79230061 90prime get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Faked up EXPNUM 82580074 Faked up EXPNUM 82580074 Faked up EXPNUM 82580074 90prime get_fwhm: self.fwhm = 4.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1910 2066 and amp range 0 2015 and image y 441 597 and amp range 0 2047 Range in image: x 0 106 , y 0 157 (with image size 157 x 157) Found overlap: image x 1904 2061 and amp range 0 2015 and image y 455 612 and amp range 0 2047 dzp -0.036130503 -> scaling image by 0.9672701794860779 Range in image: x 0 112 , y 0 158 (with image size 158 x 158) dzp -0.039144866 -> scaling image by 0.9645884414840357 Found overlap: image x 1910 2066 and amp range 2016 4031 and image y 441 597 and amp range 0 2047 Range in image: x 106 157 , y 0 157 (with image size 157 x 157) Found overlap: image x 1904 2061 and amp range 2016 4031 and image y 455 612 and amp range 0 2047 dzp -3.875605e-05 -> scaling image by 0.9999643049955675 Range in image: x 112 158 , y 0 158 (with image size 158 x 158) dzp 0.0057031885 -> scaling image by 1.00526665101287 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 78460209 Faked up EXPNUM 78460209 Faked up EXPNUM 78460209 90prime get_fwhm: self.fwhm = 3.19 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1620 1779 and amp range 0 2015 and image y 896 1057 and amp range 0 2047 Range in image: x 0 160 , y 0 162 (with image size 160 x 162) dzp 0.0003063402 -> scaling image by 1.0002821895558327 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 81940137 Faked up EXPNUM 81940137 Faked up EXPNUM 81940137 90prime get_fwhm: self.fwhm = 3.43 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3904 4016 and amp range 2016 4031 and image y 2406 2562 and amp range 2048 4095 Range in image: x 0 113 , y 0 157 (with image size 113 x 157) dzp -0.01723723 -> scaling image by 0.9842492857213068 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.67 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3769 4039 and amp range 2006 4053 and image y 3648 3919 and amp range 2040 4078 Range in image: x 0 271 , y 0 272 (with image size 271 x 272) dzp 0.012593497 -> scaling image by 1.0116665687010755 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 4.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1051 1325 and amp range 0 2005 and image y 2678 2951 and amp range 2040 4078 Range in image: x 0 275 , y 0 274 (with image size 275 x 274) dzp 0.032569118 -> scaling image by 1.0304517167935139 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.99 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2180 2453 and amp range 2048 4095 and image y 2705 2976 and amp range 2040 4078 Range in image: x 0 274 , y 0 272 (with image size 274 x 272) dzp -0.032976374 -> scaling image by 0.9700842429001929 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 7.61 s, CPU: 2.79 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 174 MB, VmData: 207 MB, maxrss: 0.17822 MB Stage tims finished: 2020-08-03T06:40:54.206704 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 4.544 s worker CPU, 21.870 s worker Wall, Wall: 7.610 s, Cores in use: 0.96, Total efficiency (on 32 cores): 3.0 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-03T06:40:54.228968 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00592.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00662.fits Matched 7 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/reference-custom-253902p64019.fits Stage refs : Wall: 4.41 s, CPU: 4.06 s, VmPeak: 981 MB, VmSize: 974 MB, VmRSS: 198 MB, VmData: 230 MB, maxrss: 0.203116 MB Stage refs finished: 2020-08-03T06:40:58.636188 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 4.409 s, Cores in use: 0.92, Total efficiency (on 32 cores): 2.9 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-03T06:40:58.748977 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outlier-mask-custom-253902p64019.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-pre-custom-253902p64019.jpg Image k4m_170616_094923_ooi_zd_ls9-CCD1 z : masking 504 positive outlier pixels and 0 negative outlier pixels Image k4m_170417_114757_ooi_zd_ls9-CCD2 z : masking 518 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outlier-mask-custom-253902p64019.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-post-custom-253902p64019.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-masked-pos-custom-253902p64019.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-masked-neg-custom-253902p64019.jpg Stage outliers : Wall: 2.83 s, CPU: 2.48 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 214 MB, VmData: 243 MB, maxrss: 0.219252 MB Stage outliers finished: 2020-08-03T06:41:01.583648 Resources for stage outliers : 0.779 s worker CPU, 0.962 s worker Wall, Wall: 2.838 s, Cores in use: 1.15, Total efficiency (on 32 cores): 3.6 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-03T06:41:01.612601 Stage halos : Wall: 2.25 s, CPU: 2.20 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 214 MB, VmData: 243 MB, maxrss: 0.219436 MB Stage halos finished: 2020-08-03T06:41:03.858490 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 2.248 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-03T06:41:03.879724 PSF sigmas in coadd pixels: ['2.35', '2.58', '2.53'] Using average PSF sigma 2.4855290467111053 Gaussian PSF norm 0.11349486828454121 vs pixelized 0.09004545 average tim pixel scale / coadd scale: 1.7298274639752187 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-copsf-g.fits.fz PSF sigmas in coadd pixels: ['3.31', '3.17', '2.28'] Using average PSF sigma 2.9185513181477085 Gaussian PSF norm 0.09665575863607319 vs pixelized 0.07617333 average tim pixel scale / coadd scale: 1.7304864324600868 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.55', '1.68', '1.78'] Using average PSF sigma 1.668413903004053 Gaussian PSF norm 0.16907962182882438 vs pixelized 0.15161294 average tim pixel scale / coadd scale: 0.9909274748825105 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.15 s, CPU: 0.14 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 216 MB, VmData: 243 MB, maxrss: 0.2212 MB Stage fit_on_coadds finished: 2020-08-03T06:41:04.029606 Resources for stage fit_on_coadds : 0.133 s worker CPU, 0.134 s worker Wall, Wall: 0.153 s, Cores in use: 1.80, Total efficiency (on 32 cores): 5.6 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-03T06:41:04.044081 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 23 potential peaks Of 23 potential peaks: 6 in veto map, 5 cut by saddle test, 0 cut by aper test, 12 kept SED z : 12 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 28 potential peaks Of 28 potential peaks: 22 in veto map, 6 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 32 potential peaks Of 32 potential peaks: 25 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 57 potential peaks Of 57 potential peaks: 50 in veto map, 7 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 14 new sources Median coadd sky for g : 0.00014913297 Median coadd sky for r : 0.0003203117 Median coadd sky for z : -8.8657034e-05 Stage srcs : Wall: 2.44 s, CPU: 2.37 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 216 MB, VmData: 243 MB, maxrss: 0.22158 MB Stage srcs finished: 2020-08-03T06:41:06.483736 Resources for stage srcs : 0.094 s worker CPU, 0.094 s worker Wall, Wall: 2.442 s, Cores in use: 1.01, Total efficiency (on 32 cores): 3.2 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-03T06:41:06.500568 Blob 1 of 9, id: 1, sources: 8, size: 193x212, npix 21585, brick X: 0,193, Y: 14,226, one pixel: 160 14 Blob 2 of 9, id: 6, sources: 1, size: 30x44, npix 963, brick X: 0,30, Y: 217,261, one pixel: 7 217 Blob 3 of 9, id: 7, sources: 2, size: 49x30, npix 956, brick X: 69,118, Y: 227,257, one pixel: 79 227 Blob 4 of 9, id: 5, sources: 1, size: 35x36, npix 886, brick X: 202,237, Y: 83,119, one pixel: 216 83 Blob 5 of 9, id: 8, sources: 1, size: 27x25, npix 409, brick X: 210,237, Y: 228,253, one pixel: 220 228 Blob 6 of 9, id: 0, sources: 1, size: 29x17, npix 364, brick X: 123,152, Y: 0,17, one pixel: 127 0 Blob 7 of 9, id: 3, sources: 1, size: 22x26, npix 352, brick X: 220,242, Y: 61,87, one pixel: 230 61 Blob 8 of 9, id: 2, sources: 1, size: 23x22, npix 286, brick X: 244,267, Y: 40,62, one pixel: 254 40 Blob 9 of 9, id: 4, sources: 1, size: 20x22, npix 265, brick X: 249,269, Y: 66,88, one pixel: 260 66 Blob 5 finished, total: Wall: 0.16 s, CPU: 0.16 s, VmPeak: 664 MB, VmSize: 664 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.09744 MB Blob 8 finished, total: Wall: 0.16 s, CPU: 0.16 s, VmPeak: 660 MB, VmSize: 660 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092476 MB Blob 7 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 661 MB, VmSize: 661 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093228 MB Blob 4 finished, total: Wall: 0.42 s, CPU: 0.42 s, VmPeak: 663 MB, VmSize: 663 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096496 MB Blob 6 finished, total: Wall: 0.43 s, CPU: 0.43 s, VmPeak: 796 MB, VmSize: 724 MB, VmRSS: 146 MB, VmData: 182 MB, maxrss: 0.14992 MB Blob 9 finished, total: Wall: 0.44 s, CPU: 0.44 s, VmPeak: 803 MB, VmSize: 724 MB, VmRSS: 145 MB, VmData: 182 MB, maxrss: 0.148708 MB Blob 2 finished, total: Wall: 0.52 s, CPU: 0.52 s, VmPeak: 661 MB, VmSize: 661 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093624 MB Blob 3 finished, total: Wall: 0.98 s, CPU: 0.98 s, VmPeak: 661 MB, VmSize: 661 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.093784 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=2.12862, ee1=0.165556, ee2=0.028248 Measuring 8 iterative sources Blob 1 finished, total: Wall: 16.82 s, CPU: 16.82 s, VmPeak: 796 MB, VmSize: 735 MB, VmRSS: 164 MB, VmData: 193 MB, maxrss: 0.167868 MB Old catalog: 21 New catalog: 24 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/blobs-custom-253902p64019.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/all-models-custom-253902p64019.fits Stage fitblobs : Wall: 19.31 s, CPU: 2.13 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 217 MB, VmData: 243 MB, maxrss: 0.222016 MB Stage fitblobs finished: 2020-08-03T06:41:25.807176 Resources for stage fitblobs : 20.387 s worker CPU, 22.222 s worker Wall, Wall: 19.309 s, Cores in use: 1.17, Total efficiency (on 32 cores): 3.6 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-03T06:41:25.826029 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-resid.jpg Stage coadds : Wall: 3.65 s, CPU: 0.53 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 217 MB, VmData: 243 MB, maxrss: 0.222592 MB Stage coadds finished: 2020-08-03T06:41:29.476417 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 23.635 s worker CPU, 46.595 s worker Wall, Wall: 3.653 s, Cores in use: 6.62, Total efficiency (on 32 cores): 20.7 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-03T06:41:29.501084 Cut to 1 unWISE tiles Reading WISE tile 2543p636 band 1 Reading WISE tile 2543p636 band 2 Reading WISE tile 2543p636 band 3 Reading WISE tile 2543p636 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 796 MB, VmSize: 796 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.189588 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 801 MB, VmSize: 793 MB, VmRSS: 142 MB, VmData: 173 MB, maxrss: 0.20486 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 809 MB, VmSize: 804 MB, VmRSS: 159 MB, VmData: 180 MB, maxrss: 0.223468 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 801 MB, VmSize: 793 MB, VmRSS: 142 MB, VmData: 173 MB, maxrss: 0.204808 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 1.74 s, CPU: 0.39 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 218 MB, VmData: 243 MB, maxrss: 0.2233 MB Stage wise_forced finished: 2020-08-03T06:41:31.239306 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.078 s worker CPU, 4.035 s worker Wall, Wall: 1.741 s, Cores in use: 2.00, Total efficiency (on 32 cores): 6.2 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-wise_forced.p Running stage writecat WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage writecat at 2020-08-03T06:41:31.258180 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/tractor-i/cus/tractor-custom-253902p64019.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/tractor/cus/tractor-custom-253902p64019.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/tractor/cus/brick-custom-253902p64019.sha256sum Stage writecat : Wall: 0.61 s, CPU: 0.46 s, VmPeak: 1116 MB, VmSize: 988 MB, VmRSS: 220 MB, VmData: 243 MB, maxrss: 0.356176 MB Stage writecat finished: 2020-08-03T06:41:31.867871 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.612 s, Cores in use: 0.76, Total efficiency (on 32 cores): 2.4 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-writecat.p All done: 52.657 s worker CPU, 95.920 s worker Wall, Wall: 45.292 s, Cores in use: 1.55, Total efficiency (on 32 cores): 4.9 % Finished galaxy PGC2663521 in 0.827 minutes. Started working on galaxy PGC2663521 at Thu Aug 6 07:35:51 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 253.90257150000002 64.0196294 --width 269 --height 269 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-06T07:35:52.361936 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '253.90257150000002', '64.0196294', '--width', '269', '--height', '269', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 253.90257150000002 64.0196294 --width 269 --height 269 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 253.90257150000002 64.0196294 Starting process 22954 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 22955 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 22956 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 22957 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 22958 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 22959 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 22960 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 22961 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 22962 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 22963 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 22964 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 22965 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 22966 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 22967 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 22968 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 22969 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 22970 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 22971 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 22972 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 22973 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 22974 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 22975 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 22976 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 22977 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 22978 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 22980 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 22981 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 22979 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 22982 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 22983 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 22984 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 22985 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-06T07:35:52.846826 Keeping 9 CCDs: k4m_170616_094923_ooi_zd_ls9-CCD1 z expnum 133722 exptime 101.102 propid 2016A-0453 seeing 0.95 MJD 57920.409 object MzLS_698762_z mosaic/CP/V4.3/CP20170615/k4m_170616_094923_ooi_zd_ls9.fits.fz k4m_170417_114757_ooi_zd_ls9-CCD2 z expnum 122694 exptime 80.105 propid 2016A-0453 seeing 1.04 MJD 57860.492 object MzLS_698206_z mosaic/CP/V4.3/CP20170416/k4m_170417_114757_ooi_zd_ls9.fits.fz k4m_170521_100103_ooi_zd_ls9-CCD1 z expnum 129255 exptime 99.104 propid 2016A-0453 seeing 1.09 MJD 57894.417 object MzLS_699314_z mosaic/CP/V4.3/CP20170520/k4m_170521_100103_ooi_zd_ls9.fits.fz ksb_180421_112921_ooi_r_ls9-CCD2 r expnum 82290183 exptime 157.818 propid BASS seeing 2.04 MJD 58229.479 object 41382 90prime/CP/V2.3/CP20180420/ksb_180421_112921_ooi_r_ls9.fits.fz ksb_170403_112240_ooi_g_ls9-CCD2 g expnum 78460209 exptime 118.444 propid BASS seeing 1.45 MJD 57846.474 object 41382 90prime/CP/V2.3/CP20170402/ksb_170403_112240_ooi_g_ls9.fits.fz ksb_180520_080206_ooi_r_ls9-CCD3 r expnum 82580074 exptime 140.88 propid NOTBASS seeing 1.95 MJD 58258.335 object 41393 90prime/CP/V2.3/CP20180519/ksb_180520_080206_ooi_r_ls9.fits.fz ksb_170619_044002_ooi_g_ls9-CCD3 g expnum 79230061 exptime 166.371 propid BASS seeing 1.59 MJD 57923.194 object 41393 90prime/CP/V2.3/CP20170618/ksb_170619_044002_ooi_g_ls9.fits.fz ksb_180317_115658_ooi_g_ls9-CCD3 g expnum 81940137 exptime 109.519 propid BASS seeing 1.56 MJD 58194.498 object 41391 90prime/CP/V2.3/CP20180316/ksb_180317_115658_ooi_g_ls9.fits.fz ksb_180423_105032_ooi_r_ls9-CCD3 r expnum 82310158 exptime 93.947 propid BASS seeing 1.40 MJD 58231.452 object 41391 90prime/CP/V2.3/CP20180422/ksb_180423_105032_ooi_r_ls9.fits.fz Faked up EXPNUM 78460209 Faked up EXPNUM 78460209 Faked up EXPNUM 78460209 90prime get_fwhm: self.fwhm = 3.19 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1620 1779 and amp range 0 2015 and image y 896 1057 and amp range 0 2047 Range in image: x 0 160 , y 0 162 (with image size 160 x 162) dzp 0.0003063402 -> scaling image by 1.0002821895558327 Faked up EXPNUM 79230061 Faked up EXPNUM 79230061 Faked up EXPNUM 79230061 90prime get_fwhm: self.fwhm = 3.51 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1910 2066 and amp range 0 2015 and image y 441 597 and amp range 0 2047 Range in image: x 0 106 , y 0 157 (with image size 157 x 157) dzp -0.036130503 -> scaling image by 0.9672701794860779 Found overlap: image x 1910 2066 and amp range 2016 4031 and image y 441 597 and amp range 0 2047 Range in image: x 106 157 , y 0 157 (with image size 157 x 157) dzp -3.875605e-05 -> scaling image by 0.9999643049955675 mosaic get_fwhm: self.fwhm = 3.67 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3769 4039 and amp range 2006 4053 and image y 3648 3919 and amp range 2040 4078 Range in image: x 0 271 , y 0 272 (with image size 271 x 272) dzp 0.012593497 -> scaling image by 1.0116665687010755 Faked up EXPNUM 82580074 Faked up EXPNUM 82580074 Faked up EXPNUM 82580074 90prime get_fwhm: self.fwhm = 4.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1904 2061 and amp range 0 2015 and image y 455 612 and amp range 0 2047 Range in image: x 0 112 , y 0 158 (with image size 158 x 158) dzp -0.039144866 -> scaling image by 0.9645884414840357 Found overlap: image x 1904 2061 and amp range 2016 4031 and image y 455 612 and amp range 0 2047 Range in image: x 112 158 , y 0 158 (with image size 158 x 158) dzp 0.0057031885 -> scaling image by 1.00526665101287 mosaic get_fwhm: self.fwhm = 4.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1051 1325 and amp range 0 2005 and image y 2678 2951 and amp range 2040 4078 Range in image: x 0 275 , y 0 274 (with image size 275 x 274) dzp 0.032569118 -> scaling image by 1.0304517167935139 mosaic get_fwhm: self.fwhm = 3.99 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2180 2453 and amp range 2048 4095 and image y 2705 2976 and amp range 2040 4078 Range in image: x 0 274 , y 0 272 (with image size 274 x 272) dzp -0.032976374 -> scaling image by 0.9700842429001929 Faked up EXPNUM 81940137 Faked up EXPNUM 81940137 Faked up EXPNUM 81940137 90prime get_fwhm: self.fwhm = 3.43 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3904 4016 and amp range 2016 4031 and image y 2406 2562 and amp range 2048 4095 Range in image: x 0 113 , y 0 157 (with image size 113 x 157) dzp -0.01723723 -> scaling image by 0.9842492857213068 Faked up EXPNUM 82290183 Faked up EXPNUM 82290183 Faked up EXPNUM 82290183 90prime get_fwhm: self.fwhm = 4.49 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1606 1766 and amp range 0 2015 and image y 920 1081 and amp range 0 2047 Range in image: x 0 161 , y 0 162 (with image size 161 x 162) dzp -0.003875085 -> scaling image by 0.9964372765054066 Faked up EXPNUM 82310158 Faked up EXPNUM 82310158 Faked up EXPNUM 82310158 90prime get_fwhm: self.fwhm = 3.09 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3946 4016 and amp range 2016 4031 and image y 2373 2529 and amp range 2048 4095 Range in image: x 0 71 , y 0 157 (with image size 71 x 157) dzp -0.0072602304 -> scaling image by 0.9933353884052984 Stage tims : Wall: 2.22 s, CPU: 0.79 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 173 MB, VmData: 206 MB, maxrss: 0.176672 MB Stage tims finished: 2020-08-06T07:35:55.070448 Resources for stage tims : 4.533 s worker CPU, 9.558 s worker Wall, Wall: 2.226 s, Cores in use: 2.39, Total efficiency (on 32 cores): 7.5 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-06T07:35:55.256865 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00592.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00662.fits Matched 7 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/reference-custom-253902p64019.fits Stage refs : Wall: 2.87 s, CPU: 2.09 s, VmPeak: 981 MB, VmSize: 974 MB, VmRSS: 197 MB, VmData: 230 MB, maxrss: 0.201576 MB Stage refs finished: 2020-08-06T07:35:58.125725 Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 2.872 s, Cores in use: 0.73, Total efficiency (on 32 cores): 2.3 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-06T07:35:58.159504 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outlier-mask-custom-253902p64019.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-pre-custom-253902p64019.jpg Image k4m_170616_094923_ooi_zd_ls9-CCD1 z : masking 504 positive outlier pixels and 0 negative outlier pixels Image k4m_170417_114757_ooi_zd_ls9-CCD2 z : masking 518 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outlier-mask-custom-253902p64019.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-post-custom-253902p64019.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-masked-pos-custom-253902p64019.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/outliers-masked-neg-custom-253902p64019.jpg Stage outliers : Wall: 0.89 s, CPU: 0.52 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 213 MB, VmData: 242 MB, maxrss: 0.217724 MB Stage outliers finished: 2020-08-06T07:35:59.046084 Resources for stage outliers : 0.814 s worker CPU, 1.038 s worker Wall, Wall: 0.889 s, Cores in use: 1.51, Total efficiency (on 32 cores): 4.7 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-06T07:35:59.074484 Stage halos : Wall: 0.17 s, CPU: 0.16 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 213 MB, VmData: 242 MB, maxrss: 0.217908 MB Stage halos finished: 2020-08-06T07:35:59.240956 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.169 s, Cores in use: 0.96, Total efficiency (on 32 cores): 3.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-06T07:35:59.342452 PSF sigmas in coadd pixels: ['2.35', '2.58', '2.53'] Using average PSF sigma 2.4855290467111053 Gaussian PSF norm 0.11349486828454121 vs pixelized 0.09004545 average tim pixel scale / coadd scale: 1.7298274639752187 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-copsf-g.fits.fz PSF sigmas in coadd pixels: ['3.31', '3.17', '2.28'] Using average PSF sigma 2.9185513181477085 Gaussian PSF norm 0.09665575863607319 vs pixelized 0.07617333 average tim pixel scale / coadd scale: 1.7304864324600868 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.55', '1.68', '1.78'] Using average PSF sigma 1.668413903004053 Gaussian PSF norm 0.16907962182882438 vs pixelized 0.15161294 average tim pixel scale / coadd scale: 0.9909274748825105 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.16 s, CPU: 0.14 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 215 MB, VmData: 242 MB, maxrss: 0.219668 MB Stage fit_on_coadds finished: 2020-08-06T07:35:59.506504 Resources for stage fit_on_coadds : 0.145 s worker CPU, 0.145 s worker Wall, Wall: 0.166 s, Cores in use: 1.74, Total efficiency (on 32 cores): 5.4 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-06T07:35:59.518167 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 23 potential peaks Of 23 potential peaks: 6 in veto map, 5 cut by saddle test, 0 cut by aper test, 12 kept SED z : 12 new peaks Found 22 potential peaks Of 22 potential peaks: 18 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 28 potential peaks Of 28 potential peaks: 22 in veto map, 6 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 32 potential peaks Of 32 potential peaks: 25 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 57 potential peaks Of 57 potential peaks: 50 in veto map, 7 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 14 new sources Median coadd sky for g : 0.00014913297 Median coadd sky for r : 0.0003203117 Median coadd sky for z : -8.8657034e-05 Stage srcs : Wall: 0.45 s, CPU: 0.42 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 215 MB, VmData: 242 MB, maxrss: 0.220028 MB Stage srcs finished: 2020-08-06T07:35:59.972405 Resources for stage srcs : 0.093 s worker CPU, 0.093 s worker Wall, Wall: 0.457 s, Cores in use: 1.12, Total efficiency (on 32 cores): 3.5 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-06T07:35:59.986801 Blob 1 of 9, id: 1, sources: 8, size: 193x212, npix 21585, brick X: 0,193, Y: 14,226, one pixel: 160 14 Blob 2 of 9, id: 6, sources: 1, size: 30x44, npix 963, brick X: 0,30, Y: 217,261, one pixel: 7 217 Blob 3 of 9, id: 7, sources: 2, size: 49x30, npix 956, brick X: 69,118, Y: 227,257, one pixel: 79 227 Blob 4 of 9, id: 5, sources: 1, size: 35x36, npix 886, brick X: 202,237, Y: 83,119, one pixel: 216 83 Blob 5 of 9, id: 8, sources: 1, size: 27x25, npix 409, brick X: 210,237, Y: 228,253, one pixel: 220 228 Blob 6 of 9, id: 0, sources: 1, size: 29x17, npix 364, brick X: 123,152, Y: 0,17, one pixel: 127 0 Blob 7 of 9, id: 3, sources: 1, size: 22x26, npix 352, brick X: 220,242, Y: 61,87, one pixel: 230 61 Blob 8 of 9, id: 2, sources: 1, size: 23x22, npix 286, brick X: 244,267, Y: 40,62, one pixel: 254 40 Blob 9 of 9, id: 4, sources: 1, size: 20x22, npix 265, brick X: 249,269, Y: 66,88, one pixel: 260 66 Blob 5 finished, total: Wall: 0.17 s, CPU: 0.17 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093684 MB Blob 8 finished, total: Wall: 0.16 s, CPU: 0.16 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.096776 MB Blob 4 finished, total: Wall: 0.41 s, CPU: 0.41 s, VmPeak: 803 MB, VmSize: 724 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.146208 MB Blob 7 finished, total: Wall: 0.41 s, CPU: 0.41 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094612 MB Blob 9 finished, total: Wall: 0.43 s, CPU: 0.43 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.096416 MB Blob 2 finished, total: Wall: 0.53 s, CPU: 0.53 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145452 MB Blob 6 finished, total: Wall: 0.52 s, CPU: 0.52 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09254 MB Blob 3 finished, total: Wall: 1.03 s, CPU: 1.03 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092464 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=2.07511, ee1=0.180833, ee2=0.251364 Measuring 5 iterative sources Blob 1 finished, total: Wall: 12.60 s, CPU: 12.60 s, VmPeak: 701 MB, VmSize: 690 MB, VmRSS: 123 MB, VmData: 161 MB, maxrss: 0.135928 MB Old catalog: 21 New catalog: 20 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/blobs-custom-253902p64019.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/metrics/cus/all-models-custom-253902p64019.fits Stage fitblobs : Wall: 13.80 s, CPU: 0.27 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 215 MB, VmData: 242 MB, maxrss: 0.220448 MB Stage fitblobs finished: 2020-08-06T07:36:13.781920 Resources for stage fitblobs : 16.368 s worker CPU, 16.395 s worker Wall, Wall: 13.797 s, Cores in use: 1.21, Total efficiency (on 32 cores): 3.8 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-06T07:36:13.805740 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-resid.jpg Stage coadds : Wall: 2.84 s, CPU: 0.99 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 216 MB, VmData: 242 MB, maxrss: 0.2209 MB Stage coadds finished: 2020-08-06T07:36:16.650508 Resources for stage coadds : 26.314 s worker CPU, 31.140 s worker Wall, Wall: 2.847 s, Cores in use: 9.59, Total efficiency (on 32 cores): 30.0 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-06T07:36:16.681231 Cut to 1 unWISE tiles Reading WISE tile 2543p636 band 3 Reading WISE tile 2543p636 band 4 Reading WISE tile 2543p636 band 1 Reading WISE tile 2543p636 band 2 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 811 MB, VmSize: 797 MB, VmRSS: 142 MB, VmData: 175 MB, maxrss: 0.1834 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 804 MB, VmSize: 792 MB, VmRSS: 137 MB, VmData: 170 MB, maxrss: 0.198788 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 808 MB, VmSize: 792 MB, VmRSS: 136 MB, VmData: 170 MB, maxrss: 0.19858 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/254/2543p636/unwise-2543p636-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 810 MB, VmSize: 806 MB, VmRSS: 149 MB, VmData: 179 MB, maxrss: 0.205968 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/tmp-legacysurvey-custom-253902p64019-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 3.29 s, CPU: 1.39 s, VmPeak: 984 MB, VmSize: 984 MB, VmRSS: 216 MB, VmData: 242 MB, maxrss: 0.221604 MB Stage wise_forced finished: 2020-08-06T07:36:19.969781 Resources for stage wise_forced : 3.134 s worker CPU, 4.736 s worker Wall, Wall: 3.291 s, Cores in use: 1.37, Total efficiency (on 32 cores): 4.3 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-06T07:36:20.010351 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/coadd/cus/custom-253902p64019/legacysurvey-custom-253902p64019-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/tractor-i/cus/tractor-custom-253902p64019.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/tractor/cus/tractor-custom-253902p64019.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/tractor/cus/brick-custom-253902p64019.sha256sum Stage writecat : Wall: 3.87 s, CPU: 2.99 s, VmPeak: 1116 MB, VmSize: 988 MB, VmRSS: 218 MB, VmData: 242 MB, maxrss: 0.354616 MB Stage writecat finished: 2020-08-06T07:36:23.876075 Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 3.868 s, Cores in use: 0.77, Total efficiency (on 32 cores): 2.4 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/253/PGC2663521/PGC2663521-largegalaxy-writecat.p All done: 51.407 s worker CPU, 63.112 s worker Wall, Wall: 31.073 s, Cores in use: 1.97, Total efficiency (on 32 cores): 6.2 % Finished galaxy PGC2663521 in 0.546 minutes.