Started working on galaxy PGC2048812 at Sun Jun  7 19:35:30 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 167.15347350000002 34.4136787 --width 239 --height 239 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-07T19:35:31.485201
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '167.15347350000002', '34.4136787', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 167.15347350000002 34.4136787 --width 239 --height 239 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 167.15347350000002 34.4136787
Starting process 63934 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057372 MB
Starting process 63939 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB
Starting process 63943 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057376 MB
Starting process 63947 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057312 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-07T19:35:31.990631
Keeping 10 CCDs:
   k4m_160229_083624_ooi_zd_ls9.fits.fz k4m_160229_083624_ooi_zd_ls9-CCD1 z exptime 80.107 propid 2016A-0453 seeing 0.76 MJD 57447.359 object MzLS_577203_z
   k4m_160210_094538_ooi_zd_ls9.fits.fz k4m_160210_094538_ooi_zd_ls9-CCD4 z exptime 80.104 propid 2016A-0453 seeing 0.86 MJD 57428.407 object MzLS_575136_z
   k4m_160211_094458_ooi_zd_ls9.fits.fz k4m_160211_094458_ooi_zd_ls9-CCD1 z exptime 80.103 propid 2016A-0453 seeing 0.82 MJD 57429.406 object MzLS_576171_z
   k4m_160229_083429_ooi_zd_ls9.fits.fz k4m_160229_083429_ooi_zd_ls9-CCD4 z exptime 80.104 propid 2016A-0453 seeing 0.79 MJD 57447.357 object MzLS_574099_z
   ksb_170519_051524_ooi_g_ls9.fits.fz ksb_170519_051524_ooi_g_ls9-CCD3 g exptime 122.699 propid 2015A-0801 seeing 1.60 MJD 57892.219 object 06031
   ksb_160203_091134_ooi_r_ls9.fits.fz ksb_160203_091134_ooi_r_ls9-CCD3 r exptime 58.2 propid 2015A-0801 seeing 1.83 MJD 57421.383 object 06031
   ksb_160215_093142_ooi_r_ls9.fits.fz ksb_160215_093142_ooi_r_ls9-CCD4 r exptime 101.9 propid 2015A-0801 seeing 2.15 MJD 57433.397 object 07403
   ksb_160309_084133_ooi_g_ls9.fits.fz ksb_160309_084133_ooi_g_ls9-CCD4 g exptime 149.2 propid 2015A-0801 seeing 2.03 MJD 57456.362 object 07403
   ksb_160116_112256_ooi_r_ls9.fits.fz ksb_160116_112256_ooi_r_ls9-CCD3 r exptime 74.3 propid 2015A-0801 seeing 1.44 MJD 57403.474 object 06032
   ksb_160206_095953_ooi_g_ls9.fits.fz ksb_160206_095953_ooi_g_ls9-CCD3 g exptime 126.7 propid 2015A-0801 seeing 1.82 MJD 57424.417 object 06032
mosaic get_fwhm: self.fwhm = 3.15
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2556 2798 and amp range 2006 4053 and image y 1245 1489 and amp range 0 2039
Range in image: x 0 243 , y 0 245 (with image size 243 x 245)
dzp 0.001388291 -> scaling image by 1.0012794811153327
mosaic get_fwhm: self.fwhm = 2.91
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 0 66 and amp range 0 2005 and image y 2323 2566 and amp range 2040 4078
Range in image: x 0 67 , y 0 244 (with image size 67 x 244)
dzp -0.005990375 -> scaling image by 0.994497853444703
mosaic get_fwhm: self.fwhm = 3.31
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 977 1217 and amp range 0 2047 and image y 1082 1322 and amp range 0 2021
Range in image: x 0 241 , y 0 241 (with image size 241 x 241)
dzp 0.005139143 -> scaling image by 1.0047445456970268
mosaic get_fwhm: self.fwhm = 3.03
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3717 3968 and amp range 2048 4095 and image y 2798 3046 and amp range 2022 4060
Range in image: x 0 252 , y 0 249 (with image size 252 x 249)
dzp -0.0028835079 -> scaling image by 0.9973477146455455
Faked up EXPNUM 78920101
Faked up EXPNUM 78920101
Faked up EXPNUM 78920101
90prime get_fwhm: self.fwhm = 3.53
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 768 907 and amp range 0 2015 and image y 225 363 and amp range 0 2047
Range in image: x 0 140 , y 0 139 (with image size 140 x 139)
dzp -0.036130503 -> scaling image by 0.9672701794860779
Faked up EXPNUM 74210166
Faked up EXPNUM 74210166
Faked up EXPNUM 74210166
90prime get_fwhm: self.fwhm = 4.03
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 778 918 and amp range 0 2015 and image y 219 358 and amp range 0 2047
Range in image: x 0 141 , y 0 140 (with image size 141 x 140)
dzp -0.024507914 -> scaling image by 0.9776802328037623
Faked up EXPNUM 74560105
Faked up EXPNUM 74560105
Faked up EXPNUM 74560105
90prime get_fwhm: self.fwhm = 4.47
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2529 2668 and amp range 2016 4031 and image y 180 319 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp 0.010226944 -> scaling image by 1.0094638657706572
Faked up EXPNUM 74330145
Faked up EXPNUM 74330145
Faked up EXPNUM 74330145
90prime get_fwhm: self.fwhm = 4.75
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2518 2658 and amp range 2016 4031 and image y 172 311 and amp range 0 2047
Range in image: x 0 141 , y 0 140 (with image size 141 x 140)
dzp 0.0047082007 -> scaling image by 1.0043458289369953
Faked up EXPNUM 74030073
Faked up EXPNUM 74030073
Faked up EXPNUM 74030073
90prime get_fwhm: self.fwhm = 3.17
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2741 2880 and amp range 2016 4031 and image y 2225 2363 and amp range 2048 4095
Range in image: x 0 140 , y 0 139 (with image size 140 x 139)
dzp -0.0005558773 -> scaling image by 0.99948814914245
Faked up EXPNUM 74240142
Faked up EXPNUM 74240142
Faked up EXPNUM 74240142
90prime get_fwhm: self.fwhm = 4.01
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2790 2928 and amp range 2016 4031 and image y 2207 2346 and amp range 2048 4095
Range in image: x 0 139 , y 0 140 (with image size 139 x 140)
dzp -0.0012536051 -> scaling image by 0.9988460533227724
Stage tims : Wall: 5.79 s, CPU: 0.97 s, VmPeak: 941 MB, VmSize: 941 MB, VmRSS: 171 MB, VmData: 204 MB, maxrss: 0.56658 MB
Stage tims finished: 2020-06-07T19:35:37.781115
Resources for stage tims : 3.336 s worker CPU, 14.130 s worker Wall, Wall: 5.792 s, Cores in use: 0.74, Total efficiency (on 4 cores): 18.6 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-06-07T19:35:37.876543
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02683.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02811.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02684.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02555.fits
Matched 4 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/reference-custom-167153p34413.fits
Stage refs : Wall: 4.35 s, CPU: 1.36 s, VmPeak: 972 MB, VmSize: 951 MB, VmRSS: 180 MB, VmData: 213 MB, maxrss: 0.56658 MB
Stage refs finished: 2020-06-07T19:35:42.223279
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 4.348 s, Cores in use: 0.31, Total efficiency (on 4 cores): 7.9 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-07T19:35:42.225040
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/outlier-mask-custom-167153p34413.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/outliers-pre-custom-167153p34413.jpg
Image k4m_160229_083624_ooi_zd_ls9-CCD1 z : masking 99 positive outlier pixels and 0 negative outlier pixels
Image k4m_160210_094538_ooi_zd_ls9-CCD4 z : masking 373 positive outlier pixels and 0 negative outlier pixels
Image k4m_160229_083429_ooi_zd_ls9-CCD4 z : masking 140 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/outlier-mask-custom-167153p34413.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/outliers-post-custom-167153p34413.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/outliers-masked-pos-custom-167153p34413.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/outliers-masked-neg-custom-167153p34413.jpg
Stage outliers : Wall: 1.58 s, CPU: 0.83 s, VmPeak: 972 MB, VmSize: 960 MB, VmRSS: 193 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage outliers finished: 2020-06-07T19:35:43.805462
Resources for stage outliers : 0.757 s worker CPU, 0.757 s worker Wall, Wall: 1.582 s, Cores in use: 1.01, Total efficiency (on 4 cores): 25.1 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-07T19:35:43.807233
Stage halos : Wall: 0.25 s, CPU: 0.24 s, VmPeak: 972 MB, VmSize: 960 MB, VmRSS: 193 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage halos finished: 2020-06-07T19:35:44.055559
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 0.249 s, Cores in use: 0.97, Total efficiency (on 4 cores): 24.2 %
Stage halos finished
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-06-07T19:35:44.057474
PSF sigmas in coadd pixels: ['2.60', '3.29', '2.95']
Using average PSF sigma 2.947392639491444
Gaussian PSF norm 0.09570994647749138 vs pixelized 0.080070116
average tim pixel scale / coadd scale: 1.730089665049303
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.97', '3.50', '2.33']
Using average PSF sigma 2.9326854326887815
Gaussian PSF norm 0.09618992498463924 vs pixelized 0.08502391
average tim pixel scale / coadd scale: 1.7301489168934467
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.23', '1.40', '1.33', '1.28']
Using average PSF sigma 1.3071745467444182
Gaussian PSF norm 0.2158049913658807 vs pixelized 0.1828119
average tim pixel scale / coadd scale: 0.9909220095830507
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 1.50 s, CPU: 0.17 s, VmPeak: 972 MB, VmSize: 960 MB, VmRSS: 195 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage fit_on_coadds finished: 2020-06-07T19:35:45.554039
Resources for stage fit_on_coadds : 0.192 s worker CPU, 0.192 s worker Wall, Wall: 1.498 s, Cores in use: 0.24, Total efficiency (on 4 cores): 6.0 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-07T19:35:45.556720
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 27 potential peaks
Of 27 potential peaks: 11 in veto map, 4 cut by saddle test, 0 cut by aper test, 12 kept
SED z : 12 new peaks
Found 9 potential peaks
Of 9 potential peaks: 9 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED r : 0 new peaks
Found 8 potential peaks
Of 8 potential peaks: 8 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 20 potential peaks
Of 20 potential peaks: 15 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 30 potential peaks
Of 30 potential peaks: 27 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 13 new sources
Median coadd sky for g : 4.4069202e-05
Median coadd sky for r : 0.000155777
Median coadd sky for z : 0.00015643162
Stage srcs : Wall: 0.50 s, CPU: 0.46 s, VmPeak: 972 MB, VmSize: 960 MB, VmRSS: 195 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage srcs finished: 2020-06-07T19:35:46.059594
Resources for stage srcs : 0.096 s worker CPU, 0.105 s worker Wall, Wall: 0.504 s, Cores in use: 1.11, Total efficiency (on 4 cores): 27.7 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/PGC2048812-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-07T19:35:46.080279
Blob 1 of 10, id: 5, sources: 1, size: 128x116, npix 9906, brick X: 55,183, Y: 62,178, one pixel: 98 62
Blob 2 of 10, id: 8, sources: 3, size: 50x51, npix 1516, brick X: 179,229, Y: 145,196, one pixel: 189 145
Blob 3 of 10, id: 1, sources: 2, size: 39x42, npix 1226, brick X: 149,188, Y: 0,42, one pixel: 154 0
Blob 4 of 10, id: 0, sources: 2, size: 50x32, npix 1049, brick X: 66,116, Y: 0,32, one pixel: 95 0
Blob 5 of 10, id: 4, sources: 2, size: 18x33, npix 443, brick X: 221,239, Y: 50,83, one pixel: 231 50
Blob 4 finished, total: Wall: 1.23 s, CPU: 1.23 s, VmPeak: 803 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.14178 MB
Blob 6 of 10, id: 9, sources: 1, size: 35x21, npix 423, brick X: 63,98, Y: 170,191, one pixel: 71 170
Measuring 1 iterative sources
Blob 5 finished, total: Wall: 0.76 s, CPU: 0.76 s, VmPeak: 803 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.14178 MB
Blob 7 of 10, id: 3, sources: 1, size: 26x25, npix 415, brick X: 28,54, Y: 46,71, one pixel: 41 46
Blob 3 finished, total: Wall: 2.06 s, CPU: 2.06 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141716 MB
Blob 8 of 10, id: 6, sources: 1, size: 25x22, npix 336, brick X: 210,235, Y: 104,126, one pixel: 219 104
Blob 9 of 10, id: 7, sources: 1, size: 23x22, npix 313, brick X: 1,24, Y: 122,144, one pixel: 11 122
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.48251, ee1=-0.0854359, ee2=-0.0846611
Measuring 2 iterative sources
Blob 6 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 803 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.14178 MB
Blob 10 of 10, id: 2, sources: 1, size: 20x20, npix 196, brick X: 3,23, Y: 2,22, one pixel: 11 2
Blob 7 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142352 MB
Blob 9 finished, total: Wall: 0.51 s, CPU: 0.51 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142352 MB
Blob 10 finished, total: Wall: 0.06 s, CPU: 0.06 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142352 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 3.41 s, CPU: 3.40 s, VmPeak: 804 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141624 MB
Blob 8 finished, total: Wall: 1.01 s, CPU: 1.01 s, VmPeak: 803 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.141892 MB
Blob 1 finished, total: Wall: 4.11 s, CPU: 4.11 s, VmPeak: 803 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.143788 MB
Old catalog: 17
New catalog: 17
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/blobs-custom-167153p34413.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/metrics/cus/all-models-custom-167153p34413.fits
Stage fitblobs : Wall: 4.49 s, CPU: 0.35 s, VmPeak: 972 MB, VmSize: 960 MB, VmRSS: 195 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage fitblobs finished: 2020-06-07T19:35:50.572390
Resources for stage fitblobs : 14.315 s worker CPU, 14.460 s worker Wall, Wall: 4.493 s, Cores in use: 3.26, Total efficiency (on 4 cores): 81.6 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-07T19:35:50.574373
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/tmp-legacysurvey-custom-167153p34413-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/tmp-legacysurvey-custom-167153p34413-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/tmp-legacysurvey-custom-167153p34413-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/tmp-legacysurvey-custom-167153p34413-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-resid.jpg
Stage coadds : Wall: 1.29 s, CPU: 0.55 s, VmPeak: 972 MB, VmSize: 960 MB, VmRSS: 196 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage coadds finished: 2020-06-07T19:35:51.868454
Resources for stage coadds : 1.907 s worker CPU, 1.983 s worker Wall, Wall: 1.295 s, Cores in use: 1.90, Total efficiency (on 4 cores): 47.4 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-07T19:35:51.870099
Cut to 1 unWISE tiles
Reading WISE tile 1662p348 band 1
Reading WISE tile 1662p348 band 4
Reading WISE tile 1662p348 band 2
Reading WISE tile 1662p348 band 3
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 804 MB, VmSize: 794 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.18914 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 803 MB, VmSize: 792 MB, VmRSS: 144 MB, VmData: 173 MB, maxrss: 0.20882 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 803 MB, VmSize: 792 MB, VmRSS: 144 MB, VmData: 173 MB, maxrss: 0.208888 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/166/1662p348/unwise-1662p348-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 853 MB, VmSize: 853 MB, VmRSS: 204 MB, VmData: 234 MB, maxrss: 0.209336 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/tmp-legacysurvey-custom-167153p34413-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/tmp-legacysurvey-custom-167153p34413-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.25 s, CPU: 0.44 s, VmPeak: 972 MB, VmSize: 960 MB, VmRSS: 197 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage wise_forced finished: 2020-06-07T19:35:54.116352
Resources for stage wise_forced : 4.780 s worker CPU, 6.024 s worker Wall, Wall: 2.247 s, Cores in use: 2.32, Total efficiency (on 4 cores): 58.1 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-07T19:35:54.118227
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/coadd/cus/custom-167153p34413/legacysurvey-custom-167153p34413-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/tractor-i/cus/tractor-custom-167153p34413.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/tractor/cus/tractor-custom-167153p34413.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/PGC2048812/tractor/cus/brick-custom-167153p34413.sha256sum
Stage writecat : Wall: 0.67 s, CPU: 0.40 s, VmPeak: 1092 MB, VmSize: 964 MB, VmRSS: 199 MB, VmData: 224 MB, maxrss: 0.56658 MB
Stage writecat finished: 2020-06-07T19:35:54.792103
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.675 s, Cores in use: 0.60, Total efficiency (on 4 cores): 15.0 %
Stage writecat finished
All done: 25.385 s worker CPU, 37.654 s worker Wall, Wall: 22.810 s, Cores in use: 1.37, Total efficiency (on 4 cores): 34.2 %
Finished galaxy PGC2048812 in 0.413 minutes.