Started working on galaxy PGC2121668 at Sun Jun  7 16:36:10 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 222.7291395 38.240599100000004 --width 231 --height 231 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-07T16:36:12.953668
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '222.7291395', '38.240599100000004', '--width', '231', '--height', '231', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 222.7291395 38.240599100000004 --width 231 --height 231 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 222.7291395 38.240599100000004
Starting process 16456 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057304 MB
Starting process 16457 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057304 MB
Starting process 16458 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057304 MB
Starting process 16459 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-07T16:36:13.458056
Keeping 11 CCDs:
   k4m_160309_105356_ooi_zd_ls9.fits.fz k4m_160309_105356_ooi_zd_ls9-CCD2 z exptime 94.105 propid 2016A-0453 seeing 1.30 MJD 57456.454 object MzLS_594448_z
   k4m_160501_075108_ooi_zd_ls9.fits.fz k4m_160501_075108_ooi_zd_ls9-CCD2 z exptime 250.1 propid 2016A-0453 seeing 2.15 MJD 57509.327 object MzLS_595435_z
   k4m_160315_105041_ooi_zd_ls9.fits.fz k4m_160315_105041_ooi_zd_ls9-CCD3 z exptime 80.102 propid 2016A-0453 seeing 1.03 MJD 57462.452 object MzLS_596419_z
   ksb_160409_095421_ooi_g_ls9.fits.fz ksb_160409_095421_ooi_g_ls9-CCD2 g exptime 78.3 propid BASS seeing 1.55 MJD 57487.413 object 11872
   ksb_160214_125315_ooi_r_ls9.fits.fz ksb_160214_125315_ooi_r_ls9-CCD2 r exptime 133.5 propid 2015A-0801 seeing 1.74 MJD 57432.537 object 11872
   ksb_160508_083936_ooi_g_ls9.fits.fz ksb_160508_083936_ooi_g_ls9-CCD3 g exptime 109.6 propid BASS seeing 1.68 MJD 57516.361 object 11883
   ksb_170307_120916_ooi_r_ls9.fits.fz ksb_170307_120916_ooi_r_ls9-CCD3 r exptime 66.89 propid BASS seeing 1.13 MJD 57819.506 object 11883
   ksb_160216_121810_ooi_r_ls9.fits.fz ksb_160216_121810_ooi_r_ls9-CCD1 r exptime 69.2 propid 2015A-0801 seeing 1.28 MJD 57434.513 object 11871
   ksb_170522_081345_ooi_g_ls9.fits.fz ksb_170522_081345_ooi_g_ls9-CCD1 g exptime 91.789 propid 2015A-0801 seeing 1.54 MJD 57895.343 object 11871
   ksb_170321_110645_ooi_r_ls9.fits.fz ksb_170321_110645_ooi_r_ls9-CCD4 r exptime 126.4 propid BASS seeing 1.75 MJD 57833.463 object 13203
   ksb_160601_080356_ooi_g_ls9.fits.fz ksb_160601_080356_ooi_g_ls9-CCD4 g exptime 90.4 propid BASS seeing 1.36 MJD 57540.336 object 13203
mosaic get_fwhm: self.fwhm = 4.99
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2533 2767 and amp range 2048 4095 and image y 3386 3619 and amp range 2040 4078
Range in image: x 0 235 , y 0 234 (with image size 235 x 234)
dzp -0.009154885 -> scaling image by 0.9916034885139223
mosaic get_fwhm: self.fwhm = 3.97
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1444 1679 and amp range 0 2005 and image y 1897 2133 and amp range 0 2039
Range in image: x 0 236 , y 0 143 (with image size 236 x 237)
dzp -0.005856539 -> scaling image by 0.9946204500110131
Found overlap: image x 1444 1679 and amp range 0 2005 and image y 1897 2133 and amp range 2040 4078
Range in image: x 0 236 , y 143 237 (with image size 236 x 237)
dzp 0.0022078273 -> scaling image by 1.0020355530120029
Faked up EXPNUM 74870148
Faked up EXPNUM 74870148
Faked up EXPNUM 74870148
90prime get_fwhm: self.fwhm = 3.41
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 54 and amp range 0 2015 and image y 3157 3291 and amp range 2048 4095
Range in image: x 0 40 , y 0 135 (with image size 40 x 135)
dzp 0.01693853 -> scaling image by 1.015723292931903
mosaic get_fwhm: self.fwhm = 8.29
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 7 77 and amp range 0 2047 and image y 3651 3883 and amp range 2040 4078
Range in image: x 0 71 , y 0 233 (with image size 71 x 233)
dzp 0.008550294 -> scaling image by 1.0079062018164673
Faked up EXPNUM 74320181
Faked up EXPNUM 74320181
Faked up EXPNUM 74320181
90prime get_fwhm: self.fwhm = 3.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 63 and amp range 0 2015 and image y 3163 3298 and amp range 2048 4095
Range in image: x 0 49 , y 0 136 (with image size 49 x 136)
dzp 0.016961161 -> scaling image by 1.0157444649385616
Faked up EXPNUM 75160116
Faked up EXPNUM 75160116
Faked up EXPNUM 75160116
90prime get_fwhm: self.fwhm = 3.71
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 156 292 and amp range 0 2015 and image y 2748 2883 and amp range 2048 4095
Range in image: x 0 137 , y 0 136 (with image size 137 x 136)
dzp -0.0045482004 -> scaling image by 0.9958197144381092
Faked up EXPNUM 74340217
Faked up EXPNUM 74340217
Faked up EXPNUM 74340217
90prime get_fwhm: self.fwhm = 2.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2070 2207 and amp range 2016 4031 and image y 1182 1318 and amp range 0 2047
Range in image: x 0 138 , y 0 137 (with image size 138 x 137)
dzp -0.0041296566 -> scaling image by 0.9962036700684744
Faked up EXPNUM 78190185
Faked up EXPNUM 78190185
Faked up EXPNUM 78190185
90prime get_fwhm: self.fwhm = 2.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 177 312 and amp range 0 2015 and image y 2740 2875 and amp range 2048 4095
Range in image: x 0 136 , y 0 136 (with image size 136 x 136)
dzp -0.07725406 -> scaling image by 0.9313188157188347
Faked up EXPNUM 78330215
Faked up EXPNUM 78330215
Faked up EXPNUM 78330215
90prime get_fwhm: self.fwhm = 3.85
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3899 4016 and amp range 2016 4031 and image y 2368 2503 and amp range 2048 4095
Range in image: x 0 118 , y 0 136 (with image size 118 x 136)
dzp 0.0004463038 -> scaling image by 1.0004111454932232
Faked up EXPNUM 78950121
Faked up EXPNUM 78950121
Faked up EXPNUM 78950121
90prime get_fwhm: self.fwhm = 3.41
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2082 2217 and amp range 2016 4031 and image y 1180 1316 and amp range 0 2047
Range in image: x 0 136 , y 0 137 (with image size 136 x 137)
dzp -0.011406615 -> scaling image by 0.9895491130016149
Faked up EXPNUM 75400084
Faked up EXPNUM 75400084
Faked up EXPNUM 75400084
90prime get_fwhm: self.fwhm = 3.01
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3937 4016 and amp range 2016 4031 and image y 2378 2513 and amp range 2048 4095
Range in image: x 0 80 , y 0 136 (with image size 80 x 136)
dzp 0.0013913503 -> scaling image by 1.001282302423891
Stage tims : Wall: 8.50 s, CPU: 0.95 s, VmPeak: 942 MB, VmSize: 942 MB, VmRSS: 170 MB, VmData: 204 MB, maxrss: 0.56656 MB
Stage tims finished: 2020-06-07T16:36:21.962932
Resources for stage tims : 3.360 s worker CPU, 20.907 s worker Wall, Wall: 8.506 s, Cores in use: 0.51, Total efficiency (on 4 cores): 12.7 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-06-07T16:36:21.982111
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02447.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02191.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02318.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02319.fits
Matched 5 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/reference-custom-222729p38240.fits
Stage refs : Wall: 3.05 s, CPU: 1.59 s, VmPeak: 972 MB, VmSize: 956 MB, VmRSS: 185 MB, VmData: 218 MB, maxrss: 0.56656 MB
Stage refs finished: 2020-06-07T16:36:25.028329
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 3.047 s, Cores in use: 0.52, Total efficiency (on 4 cores): 13.0 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-07T16:36:25.029673
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/outlier-mask-custom-222729p38240.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/outliers-pre-custom-222729p38240.jpg
Image ksb_160409_095421_ooi_g_ls9-CCD2 g : masking 284 positive outlier pixels and 0 negative outlier pixels
Image ksb_160214_125315_ooi_r_ls9-CCD2 r : masking 144 positive outlier pixels and 0 negative outlier pixels
Image k4m_160501_075108_ooi_zd_ls9-CCD2 z : masking 132 positive outlier pixels and 0 negative outlier pixels
Image k4m_160309_105356_ooi_zd_ls9-CCD2 z : masking 1971 positive outlier pixels and 0 negative outlier pixels
Image k4m_160315_105041_ooi_zd_ls9-CCD3 z : masking 343 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/outlier-mask-custom-222729p38240.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/outliers-post-custom-222729p38240.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/outliers-masked-pos-custom-222729p38240.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/outliers-masked-neg-custom-222729p38240.jpg
Stage outliers : Wall: 0.83 s, CPU: 0.63 s, VmPeak: 972 MB, VmSize: 966 MB, VmRSS: 196 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage outliers finished: 2020-06-07T16:36:25.855865
Resources for stage outliers : 0.684 s worker CPU, 0.686 s worker Wall, Wall: 0.827 s, Cores in use: 1.58, Total efficiency (on 4 cores): 39.6 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-07T16:36:25.857648
Stage halos : Wall: 0.44 s, CPU: 0.40 s, VmPeak: 972 MB, VmSize: 966 MB, VmRSS: 196 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage halos finished: 2020-06-07T16:36:26.296854
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.440 s, Cores in use: 0.91, Total efficiency (on 4 cores): 22.8 %
Stage halos finished
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-06-07T16:36:26.298605
PSF sigmas in coadd pixels: ['2.51', '2.73', '2.51', '2.22']
Using average PSF sigma 2.491626381995284
Gaussian PSF norm 0.11321713151390612 vs pixelized 0.09290476
average tim pixel scale / coadd scale: 1.729788923616528
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.82', '1.83', '2.08', '2.83']
Using average PSF sigma 2.3922250927233835
Gaussian PSF norm 0.11792150857038819 vs pixelized 0.09314218
average tim pixel scale / coadd scale: 1.7297165417236822
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.10', '3.50', '1.67']
Using average PSF sigma 2.424606778112204
Gaussian PSF norm 0.11634661517919076 vs pixelized 0.09095141
average tim pixel scale / coadd scale: 0.9909212431081265
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.17 s, CPU: 0.16 s, VmPeak: 972 MB, VmSize: 966 MB, VmRSS: 198 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage fit_on_coadds finished: 2020-06-07T16:36:26.469353
Resources for stage fit_on_coadds : 0.164 s worker CPU, 0.164 s worker Wall, Wall: 0.172 s, Cores in use: 1.88, Total efficiency (on 4 cores): 46.9 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-07T16:36:26.472340
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 9 potential peaks
Of 9 potential peaks: 4 in veto map, 1 cut by saddle test, 0 cut by aper test, 4 kept
SED z : 4 new peaks
Found 12 potential peaks
Of 12 potential peaks: 8 in veto map, 1 cut by saddle test, 0 cut by aper test, 3 kept
SED r : 3 new peaks
Found 11 potential peaks
Of 11 potential peaks: 10 in veto map, 1 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 13 potential peaks
Of 13 potential peaks: 10 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED Flat : 0 new peaks
Found 12 potential peaks
Of 12 potential peaks: 10 in veto map, 2 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 7 new sources
Median coadd sky for g : 0.00016011816
Median coadd sky for r : 0.00011337834
Median coadd sky for z : -1.3038783e-05
Stage srcs : Wall: 0.49 s, CPU: 0.45 s, VmPeak: 972 MB, VmSize: 966 MB, VmRSS: 198 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage srcs finished: 2020-06-07T16:36:26.963768
Resources for stage srcs : 0.091 s worker CPU, 0.099 s worker Wall, Wall: 0.492 s, Cores in use: 1.11, Total efficiency (on 4 cores): 27.7 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/PGC2121668-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-07T16:36:26.973660
Blob 1 of 7, id: 1, sources: 4, size: 134x132, npix 9274, brick X: 32,166, Y: 30,162, one pixel: 111 30
Blob 2 of 7, id: 2, sources: 1, size: 57x59, npix 2391, brick X: 146,203, Y: 43,102, one pixel: 177 43
Blob 3 of 7, id: 0, sources: 1, size: 47x64, npix 2305, brick X: 0,47, Y: 27,91, one pixel: 13 27
Blob 4 of 7, id: 3, sources: 2, size: 53x30, npix 973, brick X: 173,226, Y: 130,160, one pixel: 213 130
Blob 5 of 7, id: 6, sources: 1, size: 30x29, npix 575, brick X: 194,224, Y: 182,211, one pixel: 206 182
Blob 2 finished, total: Wall: 0.14 s, CPU: 0.14 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 136 MB, VmData: 178 MB, maxrss: 0.13952 MB
Blob 6 of 7, id: 5, sources: 1, size: 26x25, npix 414, brick X: 143,169, Y: 171,196, one pixel: 153 171
Blob 5 finished, total: Wall: 0.47 s, CPU: 0.47 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 136 MB, VmData: 178 MB, maxrss: 0.13952 MB
Blob 7 of 7, id: 4, sources: 1, size: 19x20, npix 227, brick X: 58,77, Y: 155,175, one pixel: 66 155
Blob 4 finished, total: Wall: 0.92 s, CPU: 0.92 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 136 MB, VmData: 178 MB, maxrss: 0.139356 MB
Blob 7 finished, total: Wall: 0.29 s, CPU: 0.29 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.1413 MB
Blob 6 finished, total: Wall: 0.72 s, CPU: 0.72 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 136 MB, VmData: 178 MB, maxrss: 0.13952 MB
Measuring 2 iterative sources
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.49674, ee1=-0.181628, ee2=-0.428324
Blob 1 finished, total: Wall: 3.66 s, CPU: 3.66 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 141 MB, VmData: 178 MB, maxrss: 0.144204 MB
Blob 3 finished, total: Wall: 4.98 s, CPU: 4.98 s, VmPeak: 803 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.14288 MB
Old catalog: 13
New catalog: 13
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/blobs-custom-222729p38240.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/metrics/cus/all-models-custom-222729p38240.fits
Stage fitblobs : Wall: 5.44 s, CPU: 0.44 s, VmPeak: 972 MB, VmSize: 966 MB, VmRSS: 198 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage fitblobs finished: 2020-06-07T16:36:32.418031
Resources for stage fitblobs : 11.270 s worker CPU, 11.312 s worker Wall, Wall: 5.446 s, Cores in use: 2.15, Total efficiency (on 4 cores): 53.7 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-07T16:36:32.419874
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/tmp-legacysurvey-custom-222729p38240-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/tmp-legacysurvey-custom-222729p38240-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/tmp-legacysurvey-custom-222729p38240-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/tmp-legacysurvey-custom-222729p38240-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-resid.jpg
Stage coadds : Wall: 1.99 s, CPU: 0.64 s, VmPeak: 972 MB, VmSize: 966 MB, VmRSS: 199 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage coadds finished: 2020-06-07T16:36:34.408843
Resources for stage coadds : 1.714 s worker CPU, 1.789 s worker Wall, Wall: 1.990 s, Cores in use: 1.18, Total efficiency (on 4 cores): 29.6 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-07T16:36:34.410367
Cut to 1 unWISE tiles
Reading WISE tile 2235p378 band 1
Reading WISE tile 2235p378 band 2
Reading WISE tile 2235p378 band 3
Reading WISE tile 2235p378 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 804 MB, VmSize: 794 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.1892 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 804 MB, VmSize: 788 MB, VmRSS: 140 MB, VmData: 170 MB, maxrss: 0.204884 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 804 MB, VmSize: 792 MB, VmRSS: 144 MB, VmData: 173 MB, maxrss: 0.208128 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/223/2235p378/unwise-2235p378-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 803 MB, VmSize: 792 MB, VmRSS: 144 MB, VmData: 173 MB, maxrss: 0.208696 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/tmp-legacysurvey-custom-222729p38240-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/tmp-legacysurvey-custom-222729p38240-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.58 s, CPU: 0.58 s, VmPeak: 972 MB, VmSize: 966 MB, VmRSS: 200 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage wise_forced finished: 2020-06-07T16:36:36.991313
Resources for stage wise_forced : 4.690 s worker CPU, 6.433 s worker Wall, Wall: 2.582 s, Cores in use: 2.04, Total efficiency (on 4 cores): 51.1 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-07T16:36:36.992970
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/coadd/cus/custom-222729p38240/legacysurvey-custom-222729p38240-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/tractor-i/cus/tractor-custom-222729p38240.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/tractor/cus/tractor-custom-222729p38240.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/222/PGC2121668/tractor/cus/brick-custom-222729p38240.sha256sum
Stage writecat : Wall: 0.66 s, CPU: 0.52 s, VmPeak: 1098 MB, VmSize: 970 MB, VmRSS: 202 MB, VmData: 227 MB, maxrss: 0.56656 MB
Stage writecat finished: 2020-06-07T16:36:37.648275
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.656 s, Cores in use: 0.80, Total efficiency (on 4 cores): 19.9 %
Stage writecat finished
All done: 21.975 s worker CPU, 41.393 s worker Wall, Wall: 24.195 s, Cores in use: 1.17, Total efficiency (on 4 cores): 29.3 %
Finished galaxy PGC2121668 in 0.463 minutes.