Started working on galaxy PGC2778248 at Sun Jun  7 18:31:37 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 166.2556665 78.98225 --width 263 --height 263 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-07T18:31:38.404661
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '166.2556665', '78.98225', '--width', '263', '--height', '263', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 166.2556665 78.98225 --width 263 --height 263 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 166.2556665 78.98225
Starting process 7985 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057376 MB
Starting process 7986 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057472 MB
Starting process 7987 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057536 MB
Starting process 7988 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057476 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-07T18:31:38.900586
Keeping 12 CCDs:
   k4m_171229_062500_ooi_zd_ls9.fits.fz k4m_171229_062500_ooi_zd_ls9-CCD3 z exptime 153.103 propid 2016A-0453 seeing 1.49 MJD 58116.267 object MzLS_729806_z
   k4m_171229_062811_ooi_zd_ls9.fits.fz k4m_171229_062811_ooi_zd_ls9-CCD2 z exptime 169.104 propid 2016A-0453 seeing 1.51 MJD 58116.270 object MzLS_729068_z
   k4m_171127_083019_ooi_zd_ls9.fits.fz k4m_171127_083019_ooi_zd_ls9-CCD4 z exptime 147.106 propid 2016A-0453 seeing 0.80 MJD 58084.354 object MzLS_729317_z
   k4m_171114_095052_ooi_zd_ls9.fits.fz k4m_171114_095052_ooi_zd_ls9-CCD1 z exptime 113.103 propid 2016A-0453 seeing 0.97 MJD 58071.410 object MzLS_729564_z
   k4m_180113_072952_ooi_zd_ls9.fits.fz k4m_180113_072952_ooi_zd_ls9-CCD1 z exptime 119.104 propid 2016A-0453 seeing 1.26 MJD 58131.312 object MzLS_729564_z
   k4m_171229_061355_ooi_zd_ls9.fits.fz k4m_171229_061355_ooi_zd_ls9-CCD4 z exptime 128.105 propid 2016A-0453 seeing 1.30 MJD 58116.260 object MzLS_729067_z
   ksb_180612_040805_ooi_r_ls9.fits.fz ksb_180612_040805_ooi_r_ls9-CCD2 r exptime 127.4 propid NOTBASS seeing 1.37 MJD 58281.172 object 51671
   ksb_180609_042013_ooi_g_ls9.fits.fz ksb_180609_042013_ooi_g_ls9-CCD2 g exptime 223.72 propid NOTBASS seeing 1.69 MJD 58278.181 object 51671
   ksb_190211_111439_ooi_r_ls9.fits.fz ksb_190211_111439_ooi_r_ls9-CCD1 r exptime 141.621 propid NOTBASS seeing 1.73 MJD 58525.469 object 
   ksb_180606_051458_ooi_g_ls9.fits.fz ksb_180606_051458_ooi_g_ls9-CCD1 g exptime 250.0 propid NOTBASS seeing 2.15 MJD 58275.219 object 51673
   ksb_190131_125517_ooi_r_ls9.fits.fz ksb_190131_125517_ooi_r_ls9-CCD4 r exptime 127.4 propid NOTBASS seeing 1.12 MJD 58514.538 object 
   ksb_180607_045103_ooi_g_ls9.fits.fz ksb_180607_045103_ooi_g_ls9-CCD4 g exptime 210.56 propid NOTBASS seeing 1.58 MJD 58276.202 object 51672
mosaic get_fwhm: self.fwhm = 5.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3738 4021 and amp range 2048 4095 and image y 653 926 and amp range 0 2039
Range in image: x 0 284 , y 0 274 (with image size 284 x 274)
dzp -0.012875672 -> scaling image by 0.9882111074411305
mosaic get_fwhm: self.fwhm = 3.73
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2582 2849 and amp range 2006 4053 and image y 2556 2825 and amp range 2040 4078
Range in image: x 0 268 , y 0 270 (with image size 268 x 270)
dzp 0.0036595785 -> scaling image by 1.003376283213225
mosaic get_fwhm: self.fwhm = 5.73
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 0 89 and amp range 0 2005 and image y 222 487 and amp range 0 2039
Range in image: x 0 90 , y 0 266 (with image size 90 x 266)
dzp 0.0014252509 -> scaling image by 1.0013135665633692
mosaic get_fwhm: self.fwhm = 3.09
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 985 1255 and amp range 0 2047 and image y 2936 3207 and amp range 2022 4060
Range in image: x 0 271 , y 0 272 (with image size 271 x 272)
dzp -0.016192675 -> scaling image by 0.9851966584456431
mosaic get_fwhm: self.fwhm = 4.99
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3753 4015 and amp range 2048 4095 and image y 3975 4054 and amp range 2022 4060
Range in image: x 0 263 , y 0 80 (with image size 263 x 80)
dzp -0.019603321 -> scaling image by 0.9821066946375949
mosaic get_fwhm: self.fwhm = 4.87
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2552 2819 and amp range 2006 4053 and image y 2623 2892 and amp range 2040 4078
Range in image: x 0 268 , y 0 270 (with image size 268 x 270)
dzp 0.0036595785 -> scaling image by 1.003376283213225
Faked up EXPNUM 82810044
Faked up EXPNUM 82810044
Faked up EXPNUM 82810044
90prime get_fwhm: self.fwhm = 3.03
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 138 and amp range 0 2015 and image y 3190 3344 and amp range 2048 4095
Range in image: x 0 124 , y 0 155 (with image size 124 x 155)
dzp 0.0503617 -> scaling image by 1.047477444828492
Faked up EXPNUM 82780032
Faked up EXPNUM 82780032
Faked up EXPNUM 82780032
90prime get_fwhm: self.fwhm = 3.73
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 20 175 and amp range 0 2015 and image y 3217 3372 and amp range 2048 4095
Range in image: x 0 156 , y 0 156 (with image size 156 x 156)
dzp 0.0459271 -> scaling image by 1.0432078344950917
Faked up EXPNUM 82750056
Faked up EXPNUM 82750056
Faked up EXPNUM 82750056
90prime get_fwhm: self.fwhm = 4.75
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2125 2284 and amp range 2016 4031 and image y 1291 1450 and amp range 0 2047
Range in image: x 0 160 , y 0 160 (with image size 160 x 160)
dzp -0.019046262 -> scaling image by 0.9826107140833663
Faked up EXPNUM 85250242
Faked up EXPNUM 85250242
Faked up EXPNUM 85250242
90prime get_fwhm: self.fwhm = 3.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2121 2279 and amp range 2016 4031 and image y 1284 1442 and amp range 0 2047
Range in image: x 0 159 , y 0 159 (with image size 159 x 159)
dzp -0.023422003 -> scaling image by 0.9786585595476589
Faked up EXPNUM 85140243
Faked up EXPNUM 85140243
Faked up EXPNUM 85140243
90prime get_fwhm: self.fwhm = 2.47
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2001 2154 and amp range 0 2015 and image y 617 769 and amp range 0 2047
Range in image: x 0 15 , y 0 153 (with image size 154 x 153)
dzp 0.013109513 -> scaling image by 1.0121474960606016
Found overlap: image x 2001 2154 and amp range 2016 4031 and image y 617 769 and amp range 0 2047
Range in image: x 15 154 , y 0 153 (with image size 154 x 153)
dzp 0.033414163 -> scaling image by 1.0312540450862397
Faked up EXPNUM 82760048
Faked up EXPNUM 82760048
Faked up EXPNUM 82760048
90prime get_fwhm: self.fwhm = 3.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2024 2176 and amp range 2016 4031 and image y 637 789 and amp range 0 2047
Range in image: x 0 153 , y 0 153 (with image size 153 x 153)
dzp 0.023266561 -> scaling image by 1.0216605511853067
Stage tims : Wall: 7.27 s, CPU: 0.98 s, VmPeak: 942 MB, VmSize: 942 MB, VmRSS: 173 MB, VmData: 207 MB, maxrss: 0.5682 MB
Stage tims finished: 2020-06-07T18:31:46.166363
Resources for stage tims : 4.385 s worker CPU, 19.634 s worker Wall, Wall: 7.267 s, Cores in use: 0.74, Total efficiency (on 4 cores): 18.5 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-06-07T18:31:46.195192
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00096.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00097.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00126.fits
Matched 6 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/reference-custom-166255p78982.fits
Stage refs : Wall: 2.59 s, CPU: 1.99 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 193 MB, VmData: 226 MB, maxrss: 0.5682 MB
Stage refs finished: 2020-06-07T18:31:48.783191
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 2.589 s, Cores in use: 0.77, Total efficiency (on 4 cores): 19.3 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-07T18:31:48.784849
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/outlier-mask-custom-166255p78982.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/outliers-pre-custom-166255p78982.jpg
Image ksb_180609_042013_ooi_g_ls9-CCD2 g : masking 357 positive outlier pixels and 0 negative outlier pixels
Image k4m_171229_062811_ooi_zd_ls9-CCD2 z : masking 167 positive outlier pixels and 0 negative outlier pixels
Image k4m_171127_083019_ooi_zd_ls9-CCD4 z : masking 1577 positive outlier pixels and 0 negative outlier pixels
Image k4m_171229_061355_ooi_zd_ls9-CCD4 z : masking 0 positive outlier pixels and 129 negative outlier pixels
Image k4m_180113_072952_ooi_zd_ls9-CCD1 z : masking 98 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/outlier-mask-custom-166255p78982.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/outliers-post-custom-166255p78982.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/outliers-masked-pos-custom-166255p78982.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/outliers-masked-neg-custom-166255p78982.jpg
Stage outliers : Wall: 1.15 s, CPU: 0.85 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 199 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage outliers finished: 2020-06-07T18:31:49.939901
Resources for stage outliers : 1.350 s worker CPU, 1.402 s worker Wall, Wall: 1.156 s, Cores in use: 1.90, Total efficiency (on 4 cores): 47.6 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-07T18:31:49.941755
Stage halos : Wall: 0.25 s, CPU: 0.24 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 199 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage halos finished: 2020-06-07T18:31:50.191688
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.251 s, Cores in use: 0.97, Total efficiency (on 4 cores): 24.2 %
Stage halos finished
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-06-07T18:31:50.193555
PSF sigmas in coadd pixels: ['2.75', '3.50', '2.57']
Using average PSF sigma 2.9375475284928405
Gaussian PSF norm 0.09603071577146934 vs pixelized 0.078082725
average tim pixel scale / coadd scale: 1.73017984249039
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.23', '2.81', '1.82']
Using average PSF sigma 2.2856666035655753
Gaussian PSF norm 0.12341904603839347 vs pixelized 0.09896145
average tim pixel scale / coadd scale: 1.7308858658706072
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.42', '2.45', '1.30', '1.57', '2.05', '2.10']
Using average PSF sigma 1.983696263723311
Gaussian PSF norm 0.14220664571117284 vs pixelized 0.10990544
average tim pixel scale / coadd scale: 0.991144387286071
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.26 s, CPU: 0.23 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 201 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage fit_on_coadds finished: 2020-06-07T18:31:50.458334
Resources for stage fit_on_coadds : 0.306 s worker CPU, 0.329 s worker Wall, Wall: 0.266 s, Cores in use: 2.02, Total efficiency (on 4 cores): 50.5 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-07T18:31:50.461472
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 15 potential peaks
Of 15 potential peaks: 3 in veto map, 3 cut by saddle test, 0 cut by aper test, 9 kept
SED z : 9 new peaks
Found 33 potential peaks
Of 33 potential peaks: 26 in veto map, 5 cut by saddle test, 0 cut by aper test, 2 kept
SED r : 2 new peaks
Found 28 potential peaks
Of 28 potential peaks: 22 in veto map, 5 cut by saddle test, 1 cut by aper test, 0 kept
SED g : 0 new peaks
Found 41 potential peaks
Of 41 potential peaks: 33 in veto map, 6 cut by saddle test, 1 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 35 potential peaks
Of 35 potential peaks: 27 in veto map, 6 cut by saddle test, 0 cut by aper test, 2 kept
SED Red : 2 new peaks
Adding 14 new sources
Median coadd sky for g : 0.00035981313
Median coadd sky for r : 0.0007183538
Median coadd sky for z : -0.0004294605
Stage srcs : Wall: 0.68 s, CPU: 0.62 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 201 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage srcs finished: 2020-06-07T18:31:51.145605
Resources for stage srcs : 0.120 s worker CPU, 0.121 s worker Wall, Wall: 0.685 s, Cores in use: 1.09, Total efficiency (on 4 cores): 27.2 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/PGC2778248-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-07T18:31:51.158561
Blob 1 of 3, id: 0, sources: 16, size: 250x245, npix 31074, brick X: 13,263, Y: 0,245, one pixel: 140 0
Blob 2 of 3, id: 2, sources: 2, size: 39x32, npix 764, brick X: 114,153, Y: 223,255, one pixel: 141 223
Blob 3 of 3, id: 1, sources: 1, size: 27x27, npix 473, brick X: 121,148, Y: 51,78, one pixel: 131 51
Blob 3 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 179 MB, maxrss: 0.143288 MB
Blob 2 finished, total: Wall: 0.73 s, CPU: 0.73 s, VmPeak: 803 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141176 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (165.38037, 78.88394), pm (40.8, -11.4), parallax 3.291 with NanoMaggies: g=7.85, r=7.1, z=6.77
Reset brightness to NanoMaggies: g=7.85, r=7.1, z=6.77
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (166.21599, 78.97134), pm (-12.8, 7.4), parallax -1.428 with NanoMaggies: g=12, r=11.4, z=13
Reset brightness to NanoMaggies: g=11.8, r=11.4, z=11.3
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (166.22297, 78.96813), pm (2.4, -0.1), parallax 0.324 with NanoMaggies: g=18, r=17.4, z=17.2
Reset brightness to NanoMaggies: g=18, r=17.4, z=17.2
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.37192, ee1=-0.796438, ee2=-0.339642
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (166.19337, 78.97029), pm (-7.2, -6.6), parallax 2.465 with NanoMaggies: g=21.1, r=19.6, z=17.9
Reset brightness to NanoMaggies: g=21.1, r=19.6, z=17.9
Measuring 5 iterative sources
Blob 1 finished, total: Wall: 23.78 s, CPU: 23.77 s, VmPeak: 805 MB, VmSize: 725 MB, VmRSS: 158 MB, VmData: 190 MB, maxrss: 0.169736 MB
Old catalog: 21
New catalog: 21
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/blobs-custom-166255p78982.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/metrics/cus/all-models-custom-166255p78982.fits
Stage fitblobs : Wall: 24.15 s, CPU: 0.32 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 202 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage fitblobs finished: 2020-06-07T18:32:15.303831
Resources for stage fitblobs : 24.862 s worker CPU, 24.914 s worker Wall, Wall: 24.146 s, Cores in use: 1.04, Total efficiency (on 4 cores): 26.1 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-07T18:32:15.305732
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/tmp-legacysurvey-custom-166255p78982-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/tmp-legacysurvey-custom-166255p78982-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/tmp-legacysurvey-custom-166255p78982-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/tmp-legacysurvey-custom-166255p78982-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-resid.jpg
Stage coadds : Wall: 1.38 s, CPU: 0.55 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 202 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage coadds finished: 2020-06-07T18:32:16.689004
Resources for stage coadds : 2.274 s worker CPU, 2.364 s worker Wall, Wall: 1.384 s, Cores in use: 2.04, Total efficiency (on 4 cores): 51.0 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-07T18:32:16.690495
Cut to 1 unWISE tiles
Reading WISE tile 1656p787 band 1
Reading WISE tile 1656p787 band 2
Reading WISE tile 1656p787 band 4
Reading WISE tile 1656p787 band 3
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 804 MB, VmSize: 790 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.188276 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 805 MB, VmSize: 797 MB, VmRSS: 154 MB, VmData: 178 MB, maxrss: 0.218236 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 803 MB, VmSize: 792 MB, VmRSS: 147 MB, VmData: 173 MB, maxrss: 0.212004 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/165/1656p787/unwise-1656p787-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 804 MB, VmSize: 792 MB, VmRSS: 148 MB, VmData: 173 MB, maxrss: 0.212424 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/tmp-legacysurvey-custom-166255p78982-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/tmp-legacysurvey-custom-166255p78982-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.44 s, CPU: 0.39 s, VmPeak: 973 MB, VmSize: 961 MB, VmRSS: 203 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage wise_forced finished: 2020-06-07T18:32:19.131688
Resources for stage wise_forced : 4.150 s worker CPU, 6.061 s worker Wall, Wall: 2.442 s, Cores in use: 1.86, Total efficiency (on 4 cores): 46.5 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-07T18:32:19.133420
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/coadd/cus/custom-166255p78982/legacysurvey-custom-166255p78982-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/tractor-i/cus/tractor-custom-166255p78982.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/tractor/cus/tractor-custom-166255p78982.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/166/PGC2778248/tractor/cus/brick-custom-166255p78982.sha256sum
Stage writecat : Wall: 1.26 s, CPU: 0.56 s, VmPeak: 1094 MB, VmSize: 966 MB, VmRSS: 205 MB, VmData: 229 MB, maxrss: 0.5682 MB
Stage writecat finished: 2020-06-07T18:32:20.390066
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.258 s, Cores in use: 0.44, Total efficiency (on 4 cores): 11.1 %
Stage writecat finished
All done: 37.449 s worker CPU, 54.827 s worker Wall, Wall: 41.494 s, Cores in use: 1.07, Total efficiency (on 4 cores): 26.6 %
Finished galaxy PGC2778248 in 0.729 minutes.