Started working on galaxy PGC2043420 at Wed Aug 5 21:22:09 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.6544125 34.07305529999999 --width 263 --height 263 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T21:22:10.571714 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '273.6544125', '34.07305529999999', '--width', '263', '--height', '263', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.6544125 34.07305529999999 --width 263 --height 263 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 273.6544125 34.07305529999999 Starting process 7392 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 7393 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 7394 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 7395 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 7396 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059404 MB Starting process 7397 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059352 MB Starting process 7398 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059352 MB Starting process 7399 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 7400 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 7401 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 7402 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 7403 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 7404 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 7405 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 7406 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 7407 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 7408 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 7409 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 7410 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 7411 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 7412 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 7413 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 7414 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 7415 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 7416 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 7417 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 7418 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 7419 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 7420 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 7421 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 7422 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 7423 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T21:22:12.503898 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 9 CCDs: k4m_160413_114805_ooi_zd_ls9-CCD3 z expnum 82023 exptime 80.104 propid 2016A-0453 seeing 0.77 MJD 57491.492 object MzLS_575289_z mosaic/CP/V4.3/CP20160412/k4m_160413_114805_ooi_zd_ls9.fits.fz k4m_160409_120318_ooi_zd_ls9-CCD1 z expnum 81543 exptime 91.104 propid 2016A-0453 seeing 1.24 MJD 57487.502 object MzLS_574253_z mosaic/CP/V4.3/CP20160408/k4m_160409_120318_ooi_zd_ls9.fits.fz k4m_170605_112601_ooi_zd_ls9-CCD4 z expnum 131901 exptime 171.106 propid 2016A-0453 seeing 0.89 MJD 57909.476 object MzLS_573214_z mosaic/CP/V4.3/CP20170604/k4m_170605_112601_ooi_zd_ls9.fits.fz ksb_160510_105412_ooi_r_ls9-CCD4 r expnum 75180251 exptime 63.5 propid BASS seeing 1.04 MJD 57518.454 object 06902 90prime/CP/V2.3/CP20160509/ksb_160510_105412_ooi_r_ls9.fits.fz ksb_160602_083605_ooi_g_ls9-CCD4 g expnum 75410133 exptime 126.6 propid BASS seeing 1.19 MJD 57541.358 object 06902 90prime/CP/V2.3/CP20160601/ksb_160602_083605_ooi_g_ls9.fits.fz ksb_160628_064054_ooi_g_ls9-CCD1 g expnum 75670092 exptime 110.8 propid BASS seeing 1.50 MJD 57567.278 object 06903 90prime/CP/V2.3/CP20160627/ksb_160628_064054_ooi_g_ls9.fits.fz ksb_180509_103258_ooi_r_ls9-CCD1 r expnum 82470183 exptime 101.24 propid NOTBASS seeing 1.16 MJD 58247.440 object 06903 90prime/CP/V2.3/CP20180508/ksb_180509_103258_ooi_r_ls9.fits.fz ksb_160414_114005_ooi_r_ls9-CCD1 r expnum 74920206 exptime 54.7 propid BASS seeing 1.10 MJD 57492.486 object 06901 90prime/CP/V2.3/CP20160413/ksb_160414_114005_ooi_r_ls9.fits.fz ksb_160504_102220_ooi_g_ls9-CCD1 g expnum 75120197 exptime 107.4 propid BASS seeing 1.44 MJD 57512.432 object 06901 90prime/CP/V2.3/CP20160503/ksb_160504_102220_ooi_g_ls9.fits.fz Faked up EXPNUM 75670092 Faked up EXPNUM 75670092 Faked up EXPNUM 75670092 90prime get_fwhm: self.fwhm = 3.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1416 1571 and amp range 0 2015 and image y 1899 2053 and amp range 0 2047 Range in image: x 0 156 , y 0 149 (with image size 156 x 155) dzp -0.011813398 -> scaling image by 0.989178436928226 Found overlap: image x 1416 1571 and amp range 0 2015 and image y 1899 2053 and amp range 2048 4095 Range in image: x 0 156 , y 149 155 (with image size 156 x 155) dzp -0.0051191514 -> scaling image by 0.9952961850385206 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 2.95 Found 4 relevant rows in amp-corrections file. Found overlap: image x 543 815 and amp range 0 2005 and image y 3059 3334 and amp range 2040 4078 Range in image: x 0 273 , y 0 276 (with image size 273 x 276) dzp 0.0022078273 -> scaling image by 1.0020355530120029 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 3.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1675 1943 and amp range 0 2047 and image y 1295 1561 and amp range 0 2021 Range in image: x 0 269 , y 0 267 (with image size 269 x 267) dzp -0.018417627 -> scaling image by 0.9831798050134541 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 82470183 Faked up EXPNUM 82470183 Faked up EXPNUM 82470183 90prime get_fwhm: self.fwhm = 2.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1423 1578 and amp range 0 2015 and image y 1887 2042 and amp range 0 2047 Range in image: x 0 156 , y 0 156 (with image size 156 x 156) dzp -0.04811793 -> scaling image by 0.9566494556405901 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75180251 Faked up EXPNUM 75180251 Faked up EXPNUM 75180251 90prime get_fwhm: self.fwhm = 2.29 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1277 1430 and amp range 0 2015 and image y 1411 1564 and amp range 0 2047 Range in image: x 0 154 , y 0 154 (with image size 154 x 154) dzp 0.009781551 -> scaling image by 1.0090498455107675 Faked up EXPNUM 74920206 Faked up EXPNUM 74920206 Faked up EXPNUM 74920206 90prime get_fwhm: self.fwhm = 2.43 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3448 3601 and amp range 2016 4031 and image y 3895 4048 and amp range 2048 4095 Range in image: x 0 154 , y 0 154 (with image size 154 x 154) dzp 0.01301769 -> scaling image by 1.0120619004021345 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' mosaic get_fwhm: self.fwhm = 4.79 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3235 3500 and amp range 2006 4053 and image y 3437 3700 and amp range 2040 4078 Range in image: x 0 266 , y 0 264 (with image size 266 x 264) dzp -0.006092005 -> scaling image by 0.9944047680378836 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75410133 Faked up EXPNUM 75410133 Faked up EXPNUM 75410133 90prime get_fwhm: self.fwhm = 2.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1278 1432 and amp range 0 2015 and image y 1405 1558 and amp range 0 2047 Range in image: x 0 155 , y 0 154 (with image size 155 x 154) dzp 0.009539392 -> scaling image by 1.0088248156909383 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Faked up EXPNUM 75120197 Faked up EXPNUM 75120197 Faked up EXPNUM 75120197 90prime get_fwhm: self.fwhm = 3.17 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3452 3605 and amp range 2016 4031 and image y 3900 4053 and amp range 2048 4095 Range in image: x 0 154 , y 0 154 (with image size 154 x 154) dzp 0.009677861 -> scaling image by 1.0089534840651213 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 20.27 s, CPU: 2.50 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 207 MB, maxrss: 0.178624 MB Stage tims finished: 2020-08-05T21:22:32.772246 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 4.517 s worker CPU, 50.296 s worker Wall, Wall: 20.271 s, Cores in use: 0.35, Total efficiency (on 32 cores): 1.1 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-tims.p Running stage refs WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T21:22:32.792546 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02849.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02721.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits Matched 4 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/reference-custom-273654p34073.fits Stage refs : Wall: 19.00 s, CPU: 7.50 s, VmPeak: 1048 MB, VmSize: 1042 MB, VmRSS: 261 MB, VmData: 291 MB, maxrss: 0.26772 MB Stage refs finished: 2020-08-05T21:22:51.795783 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 19.006 s, Cores in use: 0.40, Total efficiency (on 32 cores): 1.2 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T21:22:51.874088 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/outlier-mask-custom-273654p34073.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/outliers-pre-custom-273654p34073.jpg Image ksb_160504_102220_ooi_g_ls9-CCD1 g : masking 49 positive outlier pixels and 0 negative outlier pixels Image ksb_160510_105412_ooi_r_ls9-CCD4 r : masking 164 positive outlier pixels and 0 negative outlier pixels Image k4m_170605_112601_ooi_zd_ls9-CCD4 z : masking 148 positive outlier pixels and 580 negative outlier pixels Image k4m_160413_114805_ooi_zd_ls9-CCD3 z : masking 322 positive outlier pixels and 0 negative outlier pixels Image k4m_160409_120318_ooi_zd_ls9-CCD1 z : masking 2379 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/outlier-mask-custom-273654p34073.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/outliers-post-custom-273654p34073.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/outliers-masked-pos-custom-273654p34073.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/outliers-masked-neg-custom-273654p34073.jpg Stage outliers : Wall: 2.31 s, CPU: 1.47 s, VmPeak: 1051 MB, VmSize: 1051 MB, VmRSS: 276 MB, VmData: 303 MB, maxrss: 0.283 MB Stage outliers finished: 2020-08-05T21:22:54.183840 Resources for stage outliers : 0.890 s worker CPU, 1.104 s worker Wall, Wall: 2.312 s, Cores in use: 1.02, Total efficiency (on 32 cores): 3.2 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T21:22:54.266169 Stage halos : Wall: 1.23 s, CPU: 1.22 s, VmPeak: 1052 MB, VmSize: 1052 MB, VmRSS: 277 MB, VmData: 303 MB, maxrss: 0.283224 MB Stage halos finished: 2020-08-05T21:22:55.494121 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.230 s, Cores in use: 0.99, Total efficiency (on 32 cores): 3.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-05T21:22:57.073086 PSF sigmas in coadd pixels: ['1.94', '2.43', '2.33'] Using average PSF sigma 2.234235380036828 Gaussian PSF norm 0.12626010414767858 vs pixelized 0.10432548 average tim pixel scale / coadd scale: 1.7290835833987435 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.69', '1.89', '1.79'] Using average PSF sigma 1.7885690468222954 Gaussian PSF norm 0.1577209402539246 vs pixelized 0.13326894 average tim pixel scale / coadd scale: 1.7296977630880384 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.24', '2.02', '1.44'] Using average PSF sigma 1.56730128026393 Gaussian PSF norm 0.17998759735994987 vs pixelized 0.14254975 average tim pixel scale / coadd scale: 0.9909781955274973 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-copsf-z.fits.fz Stage fit_on_coadds : Wall: 1.34 s, CPU: 0.38 s, VmPeak: 1052 MB, VmSize: 986 MB, VmRSS: 212 MB, VmData: 237 MB, maxrss: 0.284688 MB Stage fit_on_coadds finished: 2020-08-05T21:22:58.408209 Resources for stage fit_on_coadds : 0.190 s worker CPU, 0.190 s worker Wall, Wall: 1.337 s, Cores in use: 0.43, Total efficiency (on 32 cores): 1.3 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-fit_on_coadds.p Running stage srcs Warning: Stellar halo subtraction is only implemented for DECam Running stage srcs at 2020-08-05T21:22:58.421419 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 35 potential peaks Of 35 potential peaks: 5 in veto map, 4 cut by saddle test, 0 cut by aper test, 26 kept SED z : 26 new peaks Found 34 potential peaks Of 34 potential peaks: 31 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept SED r : 0 new peaks Found 20 potential peaks Of 20 potential peaks: 19 in veto map, 1 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 31 potential peaks Of 31 potential peaks: 27 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 35 potential peaks Of 35 potential peaks: 32 in veto map, 2 cut by saddle test, 0 cut by aper test, 1 kept SED Red : 1 new peaks Adding 28 new sources Median coadd sky for g : 3.7663453e-05 Median coadd sky for r : 3.7630372e-05 Median coadd sky for z : -0.00023865455 Stage srcs : Wall: 1.57 s, CPU: 1.39 s, VmPeak: 1052 MB, VmSize: 986 MB, VmRSS: 213 MB, VmData: 237 MB, maxrss: 0.284688 MB Stage srcs finished: 2020-08-05T21:22:59.988272 Resources for stage srcs : 0.095 s worker CPU, 0.095 s worker Wall, Wall: 1.569 s, Cores in use: 0.95, Total efficiency (on 32 cores): 3.0 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T21:23:00.003093 Blob 1 of 15, id: 5, sources: 10, size: 174x153, npix 13857, brick X: 64,238, Y: 39,192, one pixel: 168 39 Blob 2 of 15, id: 2, sources: 5, size: 53x65, npix 1801, brick X: 17,70, Y: 17,82, one pixel: 30 17 Blob 3 of 15, id: 6, sources: 3, size: 24x70, npix 1170, brick X: 239,263, Y: 68,138, one pixel: 254 68 Blob 4 of 15, id: 0, sources: 1, size: 46x28, npix 929, brick X: 129,175, Y: 0,28, one pixel: 133 0 Blob 5 of 15, id: 8, sources: 2, size: 48x30, npix 872, brick X: 24,72, Y: 159,189, one pixel: 38 159 Blob 6 of 15, id: 4, sources: 2, size: 34x35, npix 757, brick X: 210,244, Y: 32,67, one pixel: 218 32 Blob 7 of 15, id: 3, sources: 2, size: 42x26, npix 711, brick X: 51,93, Y: 21,47, one pixel: 75 21 Blob 8 of 15, id: 12, sources: 2, size: 31x38, npix 686, brick X: 48,79, Y: 206,244, one pixel: 59 206 Blob 9 of 15, id: 14, sources: 1, size: 33x30, npix 619, brick X: 108,141, Y: 216,246, one pixel: 123 216 Blob 10 of 15, id: 1, sources: 1, size: 31x25, npix 558, brick X: 185,216, Y: 0,25, one pixel: 193 0 Blob 11 of 15, id: 11, sources: 1, size: 24x23, npix 330, brick X: 97,121, Y: 201,224, one pixel: 108 201 Blob 12 of 15, id: 13, sources: 1, size: 22x20, npix 287, brick X: 10,32, Y: 211,231, one pixel: 19 211 Blob 13 of 15, id: 9, sources: 1, size: 19x21, npix 260, brick X: 0,19, Y: 185,206, one pixel: 7 185 Blob 14 of 15, id: 7, sources: 1, size: 19x18, npix 198, brick X: 7,26, Y: 149,167, one pixel: 15 149 Blob 15 of 15, id: 10, sources: 1, size: 14x21, npix 197, brick X: 249,263, Y: 197,218, one pixel: 258 197 Blob 13 finished, total: Wall: 0.18 s, CPU: 0.17 s, VmPeak: 810 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150352 MB Blob 14 finished, total: Wall: 0.21 s, CPU: 0.18 s, VmPeak: 812 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149876 MB Blob 4 finished, total: Wall: 0.22 s, CPU: 0.17 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 89 MB, VmData: 139 MB, maxrss: 0.090856 MB Blob 12 finished, total: Wall: 0.44 s, CPU: 0.44 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.0965 MB Blob 11 finished, total: Wall: 0.51 s, CPU: 0.49 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.09332 MB Blob 10 finished, total: Wall: 0.72 s, CPU: 0.72 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 94 MB, VmData: 139 MB, maxrss: 0.095904 MB Blob 6 finished, total: Wall: 0.75 s, CPU: 0.75 s, VmPeak: 806 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.1503 MB Blob 8 finished, total: Wall: 0.77 s, CPU: 0.76 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.09676 MB Blob 9 finished, total: Wall: 0.79 s, CPU: 0.77 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 95 MB, VmData: 140 MB, maxrss: 0.096828 MB Blob 15 finished, total: Wall: 0.86 s, CPU: 0.74 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.092968 MB Blob 7 finished, total: Wall: 0.99 s, CPU: 0.96 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.095776 MB Blob 5 finished, total: Wall: 1.10 s, CPU: 1.08 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096188 MB Blob 2 finished, total: Wall: 1.80 s, CPU: 1.78 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.150008 MB Blob 3 finished, total: Wall: 1.87 s, CPU: 1.77 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.09448 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.55934, ee1=-0.00633562, ee2=0.37459 Measuring 1 iterative sources Blob 1 finished, total: Wall: 8.72 s, CPU: 8.68 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 150 MB, VmData: 184 MB, maxrss: 0.153164 MB Old catalog: 45 New catalog: 35 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/blobs-custom-273654p34073.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/metrics/cus/all-models-custom-273654p34073.fits Stage fitblobs : Wall: 9.98 s, CPU: 1.26 s, VmPeak: 1052 MB, VmSize: 986 MB, VmRSS: 213 MB, VmData: 237 MB, maxrss: 0.284688 MB Stage fitblobs finished: 2020-08-05T21:23:09.980418 Resources for stage fitblobs : 19.669 s worker CPU, 20.213 s worker Wall, Wall: 9.979 s, Cores in use: 2.10, Total efficiency (on 32 cores): 6.6 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T21:23:09.995546 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/tmp-legacysurvey-custom-273654p34073-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/tmp-legacysurvey-custom-273654p34073-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/tmp-legacysurvey-custom-273654p34073-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/tmp-legacysurvey-custom-273654p34073-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-resid.jpg Stage coadds : Wall: 3.64 s, CPU: 0.62 s, VmPeak: 1052 MB, VmSize: 986 MB, VmRSS: 213 MB, VmData: 237 MB, maxrss: 0.284688 MB Stage coadds finished: 2020-08-05T21:23:13.633261 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 25.410 s worker CPU, 59.644 s worker Wall, Wall: 3.640 s, Cores in use: 7.15, Total efficiency (on 32 cores): 22.3 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T21:23:13.651254 Cut to 2 unWISE tiles Reading WISE tile 2735p333 band 1 Reading WISE tile 2735p333 band 2 Reading WISE tile 2735p333 band 4 Reading WISE tile 2735p333 band 3 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 4 Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 2 Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 1 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 3 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.10 s, CPU: 0.10 s, VmPeak: 814 MB, VmSize: 814 MB, VmRSS: 156 MB, VmData: 181 MB, maxrss: 0.19724 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 813 MB, VmSize: 799 MB, VmRSS: 143 MB, VmData: 171 MB, maxrss: 0.202404 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 807 MB, VmSize: 807 MB, VmRSS: 148 MB, VmData: 179 MB, maxrss: 0.200812 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 813 MB, VmSize: 804 MB, VmRSS: 145 MB, VmData: 172 MB, maxrss: 0.207156 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/tmp-legacysurvey-custom-273654p34073-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/tmp-legacysurvey-custom-273654p34073-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 5.26 s, CPU: 1.08 s, VmPeak: 1052 MB, VmSize: 986 MB, VmRSS: 213 MB, VmData: 237 MB, maxrss: 0.284688 MB Stage wise_forced finished: 2020-08-05T21:23:18.907027 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 6.118 s worker CPU, 11.018 s worker Wall, Wall: 5.258 s, Cores in use: 1.37, Total efficiency (on 32 cores): 4.3 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T21:23:18.928485 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/coadd/cus/custom-273654p34073/legacysurvey-custom-273654p34073-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/tractor-i/cus/tractor-custom-273654p34073.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/tractor/cus/tractor-custom-273654p34073.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/tractor/cus/brick-custom-273654p34073.sha256sum Stage writecat : Wall: 2.79 s, CPU: 2.32 s, VmPeak: 1118 MB, VmSize: 990 MB, VmRSS: 215 MB, VmData: 237 MB, maxrss: 0.351484 MB Stage writecat finished: 2020-08-05T21:23:21.719525 Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 2.794 s, Cores in use: 0.83, Total efficiency (on 32 cores): 2.6 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/273/PGC2043420/PGC2043420-largegalaxy-writecat.p All done: 56.895 s worker CPU, 142.569 s worker Wall, Wall: 69.432 s, Cores in use: 1.11, Total efficiency (on 32 cores): 3.5 % Finished galaxy PGC2043420 in 1.221 minutes. Started working on galaxy PGC2043420 at Fri Aug 7 09:28:18 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.6538414951415 34.07305941906307 --width 263 --height 263 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T09:28:19.280321 legacypipe git version: DR9.5.11-watershed-3-gd509eaaf Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '273.6538414951415', '34.07305941906307', '--width', '263', '--height', '263', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 273.6538414951415 34.07305941906307 --width 263 --height 263 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 273.6538414951415 34.07305941906307 Starting process 30704 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 30705 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 30706 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 30707 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 30708 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05948 MB Starting process 30709 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB Starting process 30710 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB Starting process 30711 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 30712 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 30713 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 30714 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 30715 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 30716 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 30717 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 30718 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 30719 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T09:28:19.748508 Keeping 9 CCDs: k4m_160413_114805_ooi_zd_ls9-CCD3 z expnum 82023 exptime 80.104 propid 2016A-0453 seeing 0.77 MJD 57491.492 object MzLS_575289_z mosaic/CP/V4.3/CP20160412/k4m_160413_114805_ooi_zd_ls9.fits.fz k4m_160409_120318_ooi_zd_ls9-CCD1 z expnum 81543 exptime 91.104 propid 2016A-0453 seeing 1.24 MJD 57487.502 object MzLS_574253_z mosaic/CP/V4.3/CP20160408/k4m_160409_120318_ooi_zd_ls9.fits.fz k4m_170605_112601_ooi_zd_ls9-CCD4 z expnum 131901 exptime 171.106 propid 2016A-0453 seeing 0.89 MJD 57909.476 object MzLS_573214_z mosaic/CP/V4.3/CP20170604/k4m_170605_112601_ooi_zd_ls9.fits.fz ksb_160510_105412_ooi_r_ls9-CCD4 r expnum 75180251 exptime 63.5 propid BASS seeing 1.04 MJD 57518.454 object 06902 90prime/CP/V2.3/CP20160509/ksb_160510_105412_ooi_r_ls9.fits.fz ksb_160602_083605_ooi_g_ls9-CCD4 g expnum 75410133 exptime 126.6 propid BASS seeing 1.19 MJD 57541.358 object 06902 90prime/CP/V2.3/CP20160601/ksb_160602_083605_ooi_g_ls9.fits.fz ksb_160628_064054_ooi_g_ls9-CCD1 g expnum 75670092 exptime 110.8 propid BASS seeing 1.50 MJD 57567.278 object 06903 90prime/CP/V2.3/CP20160627/ksb_160628_064054_ooi_g_ls9.fits.fz ksb_180509_103258_ooi_r_ls9-CCD1 r expnum 82470183 exptime 101.24 propid NOTBASS seeing 1.16 MJD 58247.440 object 06903 90prime/CP/V2.3/CP20180508/ksb_180509_103258_ooi_r_ls9.fits.fz ksb_160414_114005_ooi_r_ls9-CCD1 r expnum 74920206 exptime 54.7 propid BASS seeing 1.10 MJD 57492.486 object 06901 90prime/CP/V2.3/CP20160413/ksb_160414_114005_ooi_r_ls9.fits.fz ksb_160504_102220_ooi_g_ls9-CCD1 g expnum 75120197 exptime 107.4 propid BASS seeing 1.44 MJD 57512.432 object 06901 90prime/CP/V2.3/CP20160503/ksb_160504_102220_ooi_g_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.79 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3236 3500 and amp range 2006 4053 and image y 3430 3694 and amp range 2040 4078 Range in image: x 0 265 , y 0 265 (with image size 265 x 265) dzp -0.006092005 -> scaling image by 0.9944047680378836 mosaic get_fwhm: self.fwhm = 3.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1675 1943 and amp range 0 2047 and image y 1289 1554 and amp range 0 2021 Range in image: x 0 269 , y 0 266 (with image size 269 x 266) dzp -0.018417627 -> scaling image by 0.9831798050134541 mosaic get_fwhm: self.fwhm = 2.95 Found 4 relevant rows in amp-corrections file. Found overlap: image x 543 816 and amp range 0 2005 and image y 3052 3327 and amp range 2040 4078 Range in image: x 0 274 , y 0 276 (with image size 274 x 276) dzp 0.0022078273 -> scaling image by 1.0020355530120029 Faked up EXPNUM 75180251 Faked up EXPNUM 75180251 Faked up EXPNUM 75180251 90prime get_fwhm: self.fwhm = 2.29 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1277 1430 and amp range 0 2015 and image y 1415 1568 and amp range 0 2047 Range in image: x 0 154 , y 0 154 (with image size 154 x 154) dzp 0.009781551 -> scaling image by 1.0090498455107675 Faked up EXPNUM 75410133 Faked up EXPNUM 75410133 Faked up EXPNUM 75410133 90prime get_fwhm: self.fwhm = 2.63 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1278 1432 and amp range 0 2015 and image y 1409 1562 and amp range 0 2047 Range in image: x 0 155 , y 0 154 (with image size 155 x 154) dzp 0.009539392 -> scaling image by 1.0088248156909383 Faked up EXPNUM 75120197 Faked up EXPNUM 75120197 Faked up EXPNUM 75120197 90prime get_fwhm: self.fwhm = 3.17 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3452 3605 and amp range 2016 4031 and image y 3904 4057 and amp range 2048 4095 Range in image: x 0 154 , y 0 154 (with image size 154 x 154) dzp 0.009677861 -> scaling image by 1.0089534840651213 Faked up EXPNUM 75670092 Faked up EXPNUM 75670092 Faked up EXPNUM 75670092 90prime get_fwhm: self.fwhm = 3.31 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1416 1571 and amp range 0 2015 and image y 1903 2057 and amp range 0 2047 Range in image: x 0 156 , y 0 145 (with image size 156 x 155) dzp -0.011813398 -> scaling image by 0.989178436928226 Found overlap: image x 1416 1571 and amp range 0 2015 and image y 1903 2057 and amp range 2048 4095 Range in image: x 0 156 , y 145 155 (with image size 156 x 155) dzp -0.0051191514 -> scaling image by 0.9952961850385206 Faked up EXPNUM 74920206 Faked up EXPNUM 74920206 Faked up EXPNUM 74920206 90prime get_fwhm: self.fwhm = 2.43 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3448 3601 and amp range 2016 4031 and image y 3899 4052 and amp range 2048 4095 Range in image: x 0 154 , y 0 154 (with image size 154 x 154) dzp 0.01301769 -> scaling image by 1.0120619004021345 Faked up EXPNUM 82470183 Faked up EXPNUM 82470183 Faked up EXPNUM 82470183 90prime get_fwhm: self.fwhm = 2.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1423 1578 and amp range 0 2015 and image y 1891 2045 and amp range 0 2047 Range in image: x 0 156 , y 0 155 (with image size 156 x 155) dzp -0.04811793 -> scaling image by 0.9566494556405901 Stage tims : Wall: 77.27 s, CPU: 0.91 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 172 MB, VmData: 206 MB, maxrss: 0.176632 MB Stage tims finished: 2020-08-07T09:29:37.014827 Resources for stage tims : 4.386 s worker CPU, 347.268 s worker Wall, Wall: 77.268 s, Cores in use: 0.07, Total efficiency (on 16 cores): 0.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T09:29:41.823295 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02849.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02721.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits Matched 4 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/reference-custom-273653p34073.fits Stage refs : Wall: 7.85 s, CPU: 7.05 s, VmPeak: 1041 MB, VmSize: 1035 MB, VmRSS: 260 MB, VmData: 291 MB, maxrss: 0.265832 MB Stage refs finished: 2020-08-07T09:29:49.670362 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 7.849 s, Cores in use: 0.90, Total efficiency (on 16 cores): 5.6 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T09:29:49.784613 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/outlier-mask-custom-273653p34073.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/outliers-pre-custom-273653p34073.jpg Image ksb_160504_102220_ooi_g_ls9-CCD1 g : masking 74 positive outlier pixels and 0 negative outlier pixels Image ksb_160510_105412_ooi_r_ls9-CCD4 r : masking 163 positive outlier pixels and 0 negative outlier pixels Image k4m_160409_120318_ooi_zd_ls9-CCD1 z : masking 2690 positive outlier pixels and 0 negative outlier pixels Image k4m_170605_112601_ooi_zd_ls9-CCD4 z : masking 0 positive outlier pixels and 777 negative outlier pixels Image k4m_160413_114805_ooi_zd_ls9-CCD3 z : masking 278 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/outlier-mask-custom-273653p34073.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/outliers-post-custom-273653p34073.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/outliers-masked-pos-custom-273653p34073.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/outliers-masked-neg-custom-273653p34073.jpg Stage outliers : Wall: 0.88 s, CPU: 0.57 s, VmPeak: 1045 MB, VmSize: 1045 MB, VmRSS: 274 MB, VmData: 303 MB, maxrss: 0.28108 MB Stage outliers finished: 2020-08-07T09:29:50.668782 Resources for stage outliers : 0.842 s worker CPU, 0.943 s worker Wall, Wall: 0.885 s, Cores in use: 1.60, Total efficiency (on 16 cores): 10.0 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T09:29:50.754840 Stage halos : Wall: 0.26 s, CPU: 0.24 s, VmPeak: 1045 MB, VmSize: 1045 MB, VmRSS: 275 MB, VmData: 303 MB, maxrss: 0.28132 MB Stage halos finished: 2020-08-07T09:29:51.011064 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.257 s, Cores in use: 0.94, Total efficiency (on 16 cores): 5.9 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-halos.p Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-08-07T09:29:51.106636 PSF sigmas in coadd pixels: ['1.94', '2.43', '2.33'] Using average PSF sigma 2.234235380036828 Gaussian PSF norm 0.12626010414767858 vs pixelized 0.104322575 average tim pixel scale / coadd scale: 1.7290835833987435 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.69', '1.89', '1.79'] Using average PSF sigma 1.7885690468222954 Gaussian PSF norm 0.1577209402539246 vs pixelized 0.13326356 average tim pixel scale / coadd scale: 1.7296977630880384 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.24', '2.02', '1.44'] Using average PSF sigma 1.56730128026393 Gaussian PSF norm 0.17998759735994987 vs pixelized 0.14255613 average tim pixel scale / coadd scale: 0.9909781955274973 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.32 s, CPU: 0.15 s, VmPeak: 1045 MB, VmSize: 1045 MB, VmRSS: 276 MB, VmData: 303 MB, maxrss: 0.282868 MB Stage fit_on_coadds finished: 2020-08-07T09:29:51.428902 Resources for stage fit_on_coadds : 0.196 s worker CPU, 0.196 s worker Wall, Wall: 0.324 s, Cores in use: 1.07, Total efficiency (on 16 cores): 6.7 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-07T09:29:51.485034 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 35 potential peaks Of 35 potential peaks: 7 in veto map, 5 cut by saddle test, 0 cut by aper test, 23 kept SED z : 23 new peaks Found 40 potential peaks Of 40 potential peaks: 35 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 23 potential peaks Of 23 potential peaks: 21 in veto map, 2 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 33 potential peaks Of 33 potential peaks: 28 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 36 potential peaks Of 36 potential peaks: 32 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED Red : 1 new peaks Adding 26 new sources Median coadd sky for g : 4.682158e-05 Median coadd sky for r : 1.0025003e-05 Median coadd sky for z : -0.00023661678 Stage srcs : Wall: 0.51 s, CPU: 0.47 s, VmPeak: 1045 MB, VmSize: 1045 MB, VmRSS: 277 MB, VmData: 303 MB, maxrss: 0.283352 MB Stage srcs finished: 2020-08-07T09:29:51.995086 Resources for stage srcs : 0.095 s worker CPU, 0.095 s worker Wall, Wall: 0.511 s, Cores in use: 1.10, Total efficiency (on 16 cores): 6.9 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T09:29:52.051823 Blob 1 of 15, id: 6, sources: 8, size: 175x153, npix 13843, brick X: 57,232, Y: 39,192, one pixel: 162 39 Blob 2 of 15, id: 5, sources: 5, size: 30x100, npix 1805, brick X: 233,263, Y: 38,138, one pixel: 258 38 Blob 3 of 15, id: 2, sources: 5, size: 52x65, npix 1798, brick X: 11,63, Y: 17,82, one pixel: 23 17 Blob 4 of 15, id: 0, sources: 1, size: 45x28, npix 915, brick X: 123,168, Y: 0,28, one pixel: 127 0 Blob 5 of 15, id: 8, sources: 2, size: 47x30, npix 864, brick X: 18,65, Y: 159,189, one pixel: 31 159 Blob 6 of 15, id: 4, sources: 2, size: 35x35, npix 750, brick X: 203,238, Y: 32,67, one pixel: 212 32 Blob 7 of 15, id: 3, sources: 2, size: 42x27, npix 710, brick X: 45,87, Y: 20,47, one pixel: 73 20 Blob 8 of 15, id: 11, sources: 2, size: 32x38, npix 683, brick X: 41,73, Y: 206,244, one pixel: 53 206 Blob 9 of 15, id: 13, sources: 1, size: 32x30, npix 630, brick X: 102,134, Y: 216,246, one pixel: 116 216 Blob 10 of 15, id: 1, sources: 1, size: 31x25, npix 546, brick X: 178,209, Y: 0,25, one pixel: 187 0 Blob 4 finished, total: Wall: 0.17 s, CPU: 0.17 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 88 MB, VmData: 139 MB, maxrss: 0.09038 MB Blob 11 of 15, id: 9, sources: 1, size: 21x28, npix 455, brick X: 242,263, Y: 193,221, one pixel: 254 193 Blob 12 of 15, id: 10, sources: 1, size: 23x23, npix 337, brick X: 91,114, Y: 201,224, one pixel: 101 201 Blob 13 of 15, id: 14, sources: 1, size: 16x27, npix 323, brick X: 247,263, Y: 235,262, one pixel: 258 235 Blob 14 of 15, id: 12, sources: 1, size: 23x20, npix 270, brick X: 3,26, Y: 211,231, one pixel: 13 211 Blob 15 of 15, id: 7, sources: 1, size: 18x18, npix 180, brick X: 1,19, Y: 149,167, one pixel: 9 149 Blob 8 finished, total: Wall: 0.70 s, CPU: 0.70 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.14738 MB Blob 15 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.09446 MB Blob 9 finished, total: Wall: 0.82 s, CPU: 0.82 s, VmPeak: 806 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.14758 MB Blob 6 finished, total: Wall: 0.90 s, CPU: 0.89 s, VmPeak: 810 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 182 MB, maxrss: 0.147024 MB Blob 14 finished, total: Wall: 0.61 s, CPU: 0.48 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147364 MB Blob 12 finished, total: Wall: 0.77 s, CPU: 0.67 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094464 MB Blob 7 finished, total: Wall: 1.22 s, CPU: 1.22 s, VmPeak: 805 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 182 MB, maxrss: 0.147172 MB Blob 13 finished, total: Wall: 0.86 s, CPU: 0.77 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146768 MB Blob 10 finished, total: Wall: 0.88 s, CPU: 0.87 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 94 MB, VmData: 139 MB, maxrss: 0.095968 MB Blob 5 finished, total: Wall: 1.35 s, CPU: 1.35 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.14696 MB Blob 11 finished, total: Wall: 1.06 s, CPU: 1.06 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.095032 MB Blob 3 finished, total: Wall: 1.88 s, CPU: 1.87 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147832 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (273.64185, 34.06740), pm (-7.9, -9.8), parallax 0.262 with NanoMaggies: g=18.6, r=17.8, z=17.9 Reset brightness to NanoMaggies: g=18.7, r=18.2, z=18 Blob 2 finished, total: Wall: 2.16 s, CPU: 2.16 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 96 MB, VmData: 141 MB, maxrss: 0.098004 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.19301, ee1=0.0661748, ee2=0.376459 Blob 1 finished, total: Wall: 5.91 s, CPU: 5.91 s, VmPeak: 692 MB, VmSize: 688 MB, VmRSS: 114 MB, VmData: 156 MB, maxrss: 0.120728 MB Old catalog: 43 New catalog: 33 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/blobs-custom-273653p34073.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/metrics/cus/all-models-custom-273653p34073.fits Stage fitblobs : Wall: 6.33 s, CPU: 0.68 s, VmPeak: 1045 MB, VmSize: 981 MB, VmRSS: 213 MB, VmData: 239 MB, maxrss: 0.283496 MB Stage fitblobs finished: 2020-08-07T09:29:58.377280 Resources for stage fitblobs : 19.480 s worker CPU, 19.933 s worker Wall, Wall: 6.327 s, Cores in use: 3.19, Total efficiency (on 16 cores): 19.9 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-07T09:29:58.440433 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/tmp-legacysurvey-custom-273653p34073-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/tmp-legacysurvey-custom-273653p34073-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/tmp-legacysurvey-custom-273653p34073-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/tmp-legacysurvey-custom-273653p34073-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-resid.jpg Stage coadds : Wall: 2.49 s, CPU: 0.66 s, VmPeak: 1045 MB, VmSize: 981 MB, VmRSS: 213 MB, VmData: 239 MB, maxrss: 0.283496 MB Stage coadds finished: 2020-08-07T09:30:00.933707 Resources for stage coadds : 9.104 s worker CPU, 9.629 s worker Wall, Wall: 2.495 s, Cores in use: 3.91, Total efficiency (on 16 cores): 24.5 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-07T09:30:00.968493 Cut to 2 unWISE tiles Reading WISE tile 2735p333 band 1 Reading WISE tile 2735p333 band 2 Reading WISE tile 2735p333 band 3 Reading WISE tile 2735p333 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 3 Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 2 Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/273/2735p333/unwise-2735p333-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits Reading WISE tile 2741p348 band 1 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 811 MB, VmSize: 801 MB, VmRSS: 145 MB, VmData: 175 MB, maxrss: 0.207768 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 811 MB, VmSize: 793 MB, VmRSS: 138 MB, VmData: 171 MB, maxrss: 0.198968 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 800 MB, VmSize: 800 MB, VmRSS: 144 MB, VmData: 174 MB, maxrss: 0.204412 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/274/2741p348/unwise-2741p348-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 805 MB, VmSize: 805 MB, VmRSS: 148 MB, VmData: 178 MB, maxrss: 0.186192 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/tmp-legacysurvey-custom-273653p34073-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/tmp-legacysurvey-custom-273653p34073-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 33.62 s, CPU: 0.50 s, VmPeak: 1045 MB, VmSize: 981 MB, VmRSS: 214 MB, VmData: 239 MB, maxrss: 0.283496 MB Stage wise_forced finished: 2020-08-07T09:30:34.589009 Resources for stage wise_forced : 5.937 s worker CPU, 47.119 s worker Wall, Wall: 33.622 s, Cores in use: 0.19, Total efficiency (on 16 cores): 1.2 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-07T09:30:34.606614 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/coadd/cus/custom-273653p34073/legacysurvey-custom-273653p34073-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/tractor-i/cus/tractor-custom-273653p34073.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/tractor/cus/tractor-custom-273653p34073.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/tractor/cus/brick-custom-273653p34073.sha256sum Stage writecat : Wall: 2.04 s, CPU: 0.53 s, VmPeak: 1114 MB, VmSize: 986 MB, VmRSS: 216 MB, VmData: 239 MB, maxrss: 0.351952 MB Stage writecat finished: 2020-08-07T09:30:36.649224 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 2.044 s, Cores in use: 0.26, Total efficiency (on 16 cores): 1.6 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/273/PGC2043420/PGC2043420-largegalaxy-writecat.p All done: 40.046 s worker CPU, 425.188 s worker Wall, Wall: 136.942 s, Cores in use: 0.38, Total efficiency (on 16 cores): 2.4 % Finished galaxy PGC2043420 in 2.319 minutes.