Started working on galaxy PGC3129718 at Sun Jun  7 18:30:32 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 116.3883825 40.190126 --width 311 --height 311 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-07T18:30:34.014808
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '116.3883825', '40.190126', '--width', '311', '--height', '311', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 116.3883825 40.190126 --width 311 --height 311 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 116.3883825 40.190126
Starting process 16674 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057316 MB
Starting process 16675 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057316 MB
Starting process 16676 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.05732 MB
Starting process 16677 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.05732 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-07T18:30:34.496739
Keeping 10 CCDs:
   k4m_160228_044653_ooi_zd_ls9.fits.fz k4m_160228_044653_ooi_zd_ls9-CCD3 z exptime 85.103 propid 2016A-0453 seeing 0.92 MJD 57446.199 object MzLS_605017_z
   k4m_170130_061116_ooi_zd_ls9.fits.fz k4m_170130_061116_ooi_zd_ls9-CCD2 z exptime 80.105 propid 2016A-0453 seeing 1.20 MJD 57783.258 object MzLS_604057_z
   k4m_160226_045719_ooi_zd_ls9.fits.fz k4m_160226_045719_ooi_zd_ls9-CCD1 z exptime 94.104 propid 2016A-0453 seeing 1.49 MJD 57444.206 object MzLS_605975_z
   k4m_160228_044653_ooi_zd_ls9.fits.fz k4m_160228_044653_ooi_zd_ls9-CCD4 z exptime 85.103 propid 2016A-0453 seeing 0.87 MJD 57446.199 object MzLS_605017_z
   ksb_170320_033742_ooi_r_ls9.fits.fz ksb_170320_033742_ooi_r_ls9-CCD1 r exptime 93.363 propid BASS seeing 1.18 MJD 57832.151 object 13701
   ksb_160211_044755_ooi_g_ls9.fits.fz ksb_160211_044755_ooi_g_ls9-CCD1 g exptime 108.2 propid 2015A-0801 seeing 1.27 MJD 57429.200 object 13701
   ksb_160312_034452_ooi_r_ls9.fits.fz ksb_160312_034452_ooi_r_ls9-CCD4 r exptime 94.6 propid 2015A-0801 seeing 1.52 MJD 57459.156 object 13702
   ksb_180115_080043_ooi_g_ls9.fits.fz ksb_180115_080043_ooi_g_ls9-CCD4 g exptime 89.194 propid BASS seeing 1.21 MJD 58133.334 object 13702
   ksb_170214_053536_ooi_r_ls9.fits.fz ksb_170214_053536_ooi_r_ls9-CCD1 r exptime 250.0 propid BASS seeing 1.41 MJD 57798.233 object 13703
   ksb_180115_075632_ooi_g_ls9.fits.fz ksb_180115_075632_ooi_g_ls9-CCD1 g exptime 89.194 propid BASS seeing 1.26 MJD 58133.331 object 13703
mosaic get_fwhm: self.fwhm = 3.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 7 45 and amp range 0 2047 and image y 272 583 and amp range 0 2021
Range in image: x 0 39 , y 0 312 (with image size 39 x 312)
dzp 0.005139143 -> scaling image by 1.0047445456970268
mosaic get_fwhm: self.fwhm = 5.73
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1299 1614 and amp range 0 2005 and image y 3912 4072 and amp range 2040 4078
Range in image: x 0 316 , y 0 161 (with image size 316 x 161)
dzp -0.005990375 -> scaling image by 0.994497853444703
mosaic get_fwhm: self.fwhm = 3.53
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 4005 4046 and amp range 2006 4053 and image y 297 609 and amp range 0 2039
Range in image: x 0 42 , y 0 313 (with image size 42 x 313)
dzp -0.00064781494 -> scaling image by 0.9994035183523652
mosaic get_fwhm: self.fwhm = 4.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2443 2759 and amp range 2048 4095 and image y 3634 3947 and amp range 2040 4078
Range in image: x 0 317 , y 0 314 (with image size 317 x 314)
dzp -0.032976374 -> scaling image by 0.9700842429001929
Faked up EXPNUM 78320060
Faked up EXPNUM 78320060
Faked up EXPNUM 78320060
90prime get_fwhm: self.fwhm = 2.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2491 2672 and amp range 2016 4031 and image y 3611 3792 and amp range 2048 4095
Range in image: x 0 182 , y 0 182 (with image size 182 x 182)
dzp 0.02308583 -> scaling image by 1.021490500713915
Faked up EXPNUM 74290100
Faked up EXPNUM 74290100
Faked up EXPNUM 74290100
90prime get_fwhm: self.fwhm = 2.81
Found 4 relevant rows in amp-corrections file.
Faked up EXPNUM 81330146
Faked up EXPNUM 81330146
Faked up EXPNUM 81330146
90prime get_fwhm: self.fwhm = 2.67
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2495 2676 and amp range 2016 4031 and image y 3613 3793 and amp range 2048 4095
Range in image: x 0 182 , y 0 181 (with image size 182 x 181)
Found overlap: image x 346 526 and amp range 0 2015 and image y 1121 1301 and amp range 0 2047
dzp 0.009677861 -> scaling image by 1.0089534840651213
Range in image: x 0 181 , y 0 181 (with image size 181 x 181)
dzp 0.0059804036 -> scaling image by 1.0055233530762397
Faked up EXPNUM 74590035
Faked up EXPNUM 74590035
Faked up EXPNUM 74590035
90prime get_fwhm: self.fwhm = 3.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 302 482 and amp range 0 2015 and image y 1128 1309 and amp range 0 2047
Range in image: x 0 181 , y 0 182 (with image size 181 x 182)
dzp 0.009781551 -> scaling image by 1.0090498455107675
Faked up EXPNUM 77980090
Faked up EXPNUM 77980090
Faked up EXPNUM 77980090
90prime get_fwhm: self.fwhm = 3.11
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 467 650 and amp range 0 2015 and image y 1577 1758 and amp range 0 2047
Range in image: x 0 184 , y 0 182 (with image size 184 x 182)
dzp -0.024106435 -> scaling image by 0.9780418215011345
Faked up EXPNUM 81330144
Faked up EXPNUM 81330144
Faked up EXPNUM 81330144
90prime get_fwhm: self.fwhm = 2.79
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 498 682 and amp range 0 2015 and image y 1607 1789 and amp range 0 2047
Range in image: x 0 185 , y 0 183 (with image size 185 x 183)
dzp -0.030770088 -> scaling image by 0.9720575212609548
Stage tims : Wall: 7.71 s, CPU: 0.99 s, VmPeak: 942 MB, VmSize: 942 MB, VmRSS: 172 MB, VmData: 205 MB, maxrss: 0.566508 MB
Stage tims finished: 2020-06-07T18:30:42.209075
Resources for stage tims : 3.722 s worker CPU, 18.192 s worker Wall, Wall: 7.714 s, Cores in use: 0.61, Total efficiency (on 4 cores): 15.3 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-06-07T18:30:43.340942
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02153.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02281.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02154.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02025.fits
Matched 5 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/reference-custom-116388p40190.fits
Stage refs : Wall: 4.93 s, CPU: 2.74 s, VmPeak: 975 MB, VmSize: 969 MB, VmRSS: 200 MB, VmData: 232 MB, maxrss: 0.566508 MB
Stage refs finished: 2020-06-07T18:30:48.267819
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 4.928 s, Cores in use: 0.56, Total efficiency (on 4 cores): 13.9 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-07T18:30:48.269358
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/outlier-mask-custom-116388p40190.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/outliers-pre-custom-116388p40190.jpg
Image ksb_160211_044755_ooi_g_ls9-CCD1 g : masking 79 positive outlier pixels and 0 negative outlier pixels
Image ksb_170320_033742_ooi_r_ls9-CCD1 r : masking 119 positive outlier pixels and 0 negative outlier pixels
Image ksb_160312_034452_ooi_r_ls9-CCD4 r : masking 85 positive outlier pixels and 0 negative outlier pixels
Image k4m_160228_044653_ooi_zd_ls9-CCD4 z : masking 0 positive outlier pixels and 555 negative outlier pixels
Image k4m_160226_045719_ooi_zd_ls9-CCD1 z : masking 70 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/outlier-mask-custom-116388p40190.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/outliers-post-custom-116388p40190.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/outliers-masked-pos-custom-116388p40190.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/outliers-masked-neg-custom-116388p40190.jpg
Stage outliers : Wall: 1.04 s, CPU: 0.68 s, VmPeak: 978 MB, VmSize: 978 MB, VmRSS: 216 MB, VmData: 245 MB, maxrss: 0.566508 MB
Stage outliers finished: 2020-06-07T18:30:49.314171
Resources for stage outliers : 1.206 s worker CPU, 1.228 s worker Wall, Wall: 1.046 s, Cores in use: 1.80, Total efficiency (on 4 cores): 45.0 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-07T18:30:49.316205
Stage halos : Wall: 0.28 s, CPU: 0.27 s, VmPeak: 978 MB, VmSize: 978 MB, VmRSS: 216 MB, VmData: 245 MB, maxrss: 0.566508 MB
Stage halos finished: 2020-06-07T18:30:49.594664
Resources for stage halos : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.280 s, Cores in use: 0.99, Total efficiency (on 4 cores): 24.6 %
Stage halos finished
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-06-07T18:30:49.596727
PSF sigmas in coadd pixels: ['2.07', '1.97', '2.05']
Using average PSF sigma 2.0293545796947368
Gaussian PSF norm 0.13900714768944517 vs pixelized 0.12153648
average tim pixel scale / coadd scale: 1.730000753958091
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.92', '2.47', '2.29']
Using average PSF sigma 2.2248540181076
Gaussian PSF norm 0.12679249491336078 vs pixelized 0.10584643
average tim pixel scale / coadd scale: 1.729289931778348
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.49', '1.94', '2.42', '1.41']
Using average PSF sigma 1.8150820328893247
Gaussian PSF norm 0.15541710328366135 vs pixelized 0.12134662
average tim pixel scale / coadd scale: 0.9908114622377271
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.27 s, CPU: 0.22 s, VmPeak: 978 MB, VmSize: 978 MB, VmRSS: 219 MB, VmData: 246 MB, maxrss: 0.566508 MB
Stage fit_on_coadds finished: 2020-06-07T18:30:49.864481
Resources for stage fit_on_coadds : 0.271 s worker CPU, 0.281 s worker Wall, Wall: 0.269 s, Cores in use: 1.83, Total efficiency (on 4 cores): 45.7 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-07T18:30:49.867339
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 42 potential peaks
Of 42 potential peaks: 12 in veto map, 4 cut by saddle test, 0 cut by aper test, 26 kept
SED z : 26 new peaks
Found 78 potential peaks
Of 78 potential peaks: 70 in veto map, 7 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 90 potential peaks
Of 90 potential peaks: 79 in veto map, 7 cut by saddle test, 0 cut by aper test, 4 kept
SED g : 4 new peaks
Found 103 potential peaks
Of 103 potential peaks: 88 in veto map, 10 cut by saddle test, 0 cut by aper test, 5 kept
SED Flat : 5 new peaks
Found 80 potential peaks
Of 80 potential peaks: 72 in veto map, 8 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 36 new sources
Median coadd sky for g : 0.0003963225
Median coadd sky for r : 0.0008381215
Median coadd sky for z : -0.00013092095
Stage srcs : Wall: 0.76 s, CPU: 0.70 s, VmPeak: 978 MB, VmSize: 978 MB, VmRSS: 219 MB, VmData: 246 MB, maxrss: 0.566508 MB
Stage srcs finished: 2020-06-07T18:30:50.624214
Resources for stage srcs : 0.172 s worker CPU, 0.182 s worker Wall, Wall: 0.758 s, Cores in use: 1.15, Total efficiency (on 4 cores): 28.7 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/PGC3129718-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-07T18:30:50.642187
Blob 1 of 14, id: 3, sources: 15, size: 227x231, npix 28638, brick X: 50,277, Y: 22,253, one pixel: 234 22
Blob 2 of 14, id: 12, sources: 5, size: 88x47, npix 2560, brick X: 136,224, Y: 264,311, one pixel: 205 264
Blob 3 of 14, id: 1, sources: 2, size: 67x63, npix 2301, brick X: 32,99, Y: 0,63, one pixel: 68 0
Blob 4 of 14, id: 8, sources: 2, size: 30x60, npix 1256, brick X: 0,30, Y: 216,276, one pixel: 10 216
Blob 5 of 14, id: 2, sources: 1, size: 44x39, npix 1153, brick X: 113,157, Y: 19,58, one pixel: 137 19
Blob 4 finished, total: Wall: 1.12 s, CPU: 1.12 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.14268 MB
Blob 6 of 14, id: 9, sources: 2, size: 37x44, npix 881, brick X: 216,253, Y: 221,265, one pixel: 226 221
Blob 3 finished, total: Wall: 1.47 s, CPU: 1.47 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.140924 MB
Blob 7 of 14, id: 0, sources: 2, size: 48x25, npix 813, brick X: 9,57, Y: 0,25, one pixel: 19 0
Measuring 2 iterative sources
Blob 6 finished, total: Wall: 1.12 s, CPU: 1.12 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.14304 MB
Blob 8 of 14, id: 5, sources: 2, size: 31x41, npix 802, brick X: 252,283, Y: 82,123, one pixel: 267 82
Blob 5 finished, total: Wall: 1.47 s, CPU: 1.47 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143404 MB
Blob 9 of 14, id: 11, sources: 1, size: 33x37, npix 678, brick X: 112,145, Y: 253,290, one pixel: 120 253
Blob 7 finished, total: Wall: 1.37 s, CPU: 1.37 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 179 MB, maxrss: 0.14358 MB
Blob 10 of 14, id: 10, sources: 2, size: 46x23, npix 611, brick X: 147,193, Y: 236,259, one pixel: 161 236
Blob 8 finished, total: Wall: 1.48 s, CPU: 1.48 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143404 MB
Blob 11 of 14, id: 13, sources: 1, size: 24x24, npix 361, brick X: 276,300, Y: 282,306, one pixel: 286 282
Blob 9 finished, total: Wall: 0.60 s, CPU: 0.60 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 179 MB, maxrss: 0.14358 MB
Blob 12 of 14, id: 7, sources: 1, size: 21x21, npix 269, brick X: 34,55, Y: 157,178, one pixel: 43 157
Blob 13 of 14, id: 4, sources: 1, size: 19x20, npix 218, brick X: 19,38, Y: 50,70, one pixel: 28 50
Blob 2 finished, total: Wall: 4.94 s, CPU: 4.93 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143864 MB
Blob 10 finished, total: Wall: 0.80 s, CPU: 0.80 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143732 MB
Blob 14 of 14, id: 6, sources: 1, size: 15x19, npix 183, brick X: 296,311, Y: 90,109, one pixel: 304 90
Blob 12 finished, total: Wall: 0.06 s, CPU: 0.06 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143732 MB
Blob 11 finished, total: Wall: 0.56 s, CPU: 0.56 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 179 MB, maxrss: 0.14358 MB
Blob 13 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143864 MB
Blob 14 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143732 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=0.654701, ee1=-0.136147, ee2=0.0412209
Measuring 4 iterative sources
Blob 1 finished, total: Wall: 21.32 s, CPU: 21.32 s, VmPeak: 805 MB, VmSize: 729 MB, VmRSS: 162 MB, VmData: 195 MB, maxrss: 0.166072 MB
Old catalog: 38
New catalog: 41
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/blobs-custom-116388p40190.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/metrics/cus/all-models-custom-116388p40190.fits
Stage fitblobs : Wall: 21.79 s, CPU: 0.46 s, VmPeak: 978 MB, VmSize: 978 MB, VmRSS: 219 MB, VmData: 246 MB, maxrss: 0.566508 MB
Stage fitblobs finished: 2020-06-07T18:31:12.428264
Resources for stage fitblobs : 37.017 s worker CPU, 37.052 s worker Wall, Wall: 21.788 s, Cores in use: 1.72, Total efficiency (on 4 cores): 43.0 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-07T18:31:12.430628
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/tmp-legacysurvey-custom-116388p40190-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/tmp-legacysurvey-custom-116388p40190-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/tmp-legacysurvey-custom-116388p40190-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/tmp-legacysurvey-custom-116388p40190-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-resid.jpg
Stage coadds : Wall: 1.59 s, CPU: 0.73 s, VmPeak: 978 MB, VmSize: 978 MB, VmRSS: 220 MB, VmData: 247 MB, maxrss: 0.566508 MB
Stage coadds finished: 2020-06-07T18:31:14.025514
Resources for stage coadds : 2.394 s worker CPU, 2.493 s worker Wall, Wall: 1.596 s, Cores in use: 1.96, Total efficiency (on 4 cores): 48.9 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-07T18:31:14.027101
Cut to 1 unWISE tiles
Reading WISE tile 1167p408 band 1
Reading WISE tile 1167p408 band 2
Reading WISE tile 1167p408 band 3
Reading WISE tile 1167p408 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.16 s, CPU: 0.16 s, VmPeak: 805 MB, VmSize: 797 MB, VmRSS: 149 MB, VmData: 178 MB, maxrss: 0.193288 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 805 MB, VmSize: 796 MB, VmRSS: 153 MB, VmData: 178 MB, maxrss: 0.217504 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 805 MB, VmSize: 788 MB, VmRSS: 141 MB, VmData: 170 MB, maxrss: 0.205164 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/116/1167p408/unwise-1167p408-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 805 MB, VmSize: 792 MB, VmRSS: 146 MB, VmData: 173 MB, maxrss: 0.208904 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/tmp-legacysurvey-custom-116388p40190-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/tmp-legacysurvey-custom-116388p40190-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.40 s, CPU: 0.50 s, VmPeak: 978 MB, VmSize: 978 MB, VmRSS: 221 MB, VmData: 247 MB, maxrss: 0.566508 MB
Stage wise_forced finished: 2020-06-07T18:31:16.426425
Resources for stage wise_forced : 5.045 s worker CPU, 6.189 s worker Wall, Wall: 2.401 s, Cores in use: 2.31, Total efficiency (on 4 cores): 57.8 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-07T18:31:16.428295
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/coadd/cus/custom-116388p40190/legacysurvey-custom-116388p40190-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/tractor-i/cus/tractor-custom-116388p40190.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/tractor/cus/tractor-custom-116388p40190.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/116/PGC3129718/tractor/cus/brick-custom-116388p40190.sha256sum
Stage writecat : Wall: 0.80 s, CPU: 0.51 s, VmPeak: 1111 MB, VmSize: 983 MB, VmRSS: 223 MB, VmData: 248 MB, maxrss: 0.566508 MB
Stage writecat finished: 2020-06-07T18:31:17.230979
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.804 s, Cores in use: 0.64, Total efficiency (on 4 cores): 16.0 %
Stage writecat finished
All done: 49.829 s worker CPU, 65.620 s worker Wall, Wall: 42.740 s, Cores in use: 1.35, Total efficiency (on 4 cores): 33.7 %
Finished galaxy PGC3129718 in 0.752 minutes.