Started working on galaxy PGC2113214 at Sun Jun  7 16:48:42 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 100.921584 37.926694399999995 --width 307 --height 307 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-07T16:48:43.248265
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '100.921584', '37.926694399999995', '--width', '307', '--height', '307', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 100.921584 37.926694399999995 --width 307 --height 307 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 100.921584 37.926694399999995
Starting process 60373 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB
Starting process 60375 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB
Starting process 60378 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB
Starting process 60379 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057312 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-07T16:48:43.940392
Keeping 13 CCDs:
   k4m_170222_040839_ooi_zd_ls9.fits.fz k4m_170222_040839_ooi_zd_ls9-CCD1 z exptime 80.106 propid 2016A-0453 seeing 0.94 MJD 57806.173 object MzLS_594281_z
   k4m_170127_043522_ooi_zd_ls9.fits.fz k4m_170127_043522_ooi_zd_ls9-CCD4 z exptime 233.105 propid 2016A-0453 seeing 1.50 MJD 57780.191 object MzLS_592298_z
   k4m_170319_031543_ooi_zd_ls9.fits.fz k4m_170319_031543_ooi_zd_ls9-CCD1 z exptime 80.105 propid 2016A-0453 seeing 0.70 MJD 57831.136 object MzLS_594281_z
   k4m_160210_034141_ooi_zd_ls9.fits.fz k4m_160210_034141_ooi_zd_ls9-CCD4 z exptime 80.102 propid 2016A-0453 seeing 0.84 MJD 57428.154 object MzLS_593291_z
   k4m_170302_043403_ooi_zd_ls9.fits.fz k4m_170302_043403_ooi_zd_ls9-CCD3 z exptime 80.105 propid 2016A-0453 seeing 0.94 MJD 57814.190 object MzLS_595268_z
   k4m_170226_040724_ooi_zd_ls9.fits.fz k4m_170226_040724_ooi_zd_ls9-CCD3 z exptime 131.105 propid 2016A-0453 seeing 0.96 MJD 57810.172 object MzLS_595268_z
   ksb_151113_093225_ooi_g_ls9.fits.fz ksb_151113_093225_ooi_g_ls9-CCD4 g exptime 225.2 propid 2015A-0801 seeing 1.28 MJD 57339.398 object 61321
   ksb_160112_062546_ooi_g_ls9.fits.fz ksb_160112_062546_ooi_g_ls9-CCD4 g exptime 203.5 propid 2015A-0801 seeing 1.21 MJD 57399.268 object 61321
   ksb_170324_025609_ooi_r_ls9.fits.fz ksb_170324_025609_ooi_r_ls9-CCD4 r exptime 212.2 propid BASS seeing 2.00 MJD 57836.122 object 61321
   ksb_160115_061459_ooi_r_ls9.fits.fz ksb_160115_061459_ooi_r_ls9-CCD2 r exptime 166.0 propid 2015A-0801 seeing 1.31 MJD 57402.260 object 61312
   ksb_151112_111331_ooi_g_ls9.fits.fz ksb_151112_111331_ooi_g_ls9-CCD2 g exptime 250.0 propid 2015A-0801 seeing 1.74 MJD 57338.468 object 61312
   ksb_170329_032023_ooi_g_ls9.fits.fz ksb_170329_032023_ooi_g_ls9-CCD3 g exptime 250.0 propid BASS seeing 1.49 MJD 57841.139 object 61323
   ksb_180319_025108_ooi_r_ls9.fits.fz ksb_180319_025108_ooi_r_ls9-CCD3 r exptime 133.66 propid BASS seeing 1.23 MJD 58196.119 object 61323
mosaic get_fwhm: self.fwhm = 3.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2471 2781 and amp range 2006 4053 and image y 2125 2437 and amp range 2040 4078
Range in image: x 0 311 , y 0 313 (with image size 311 x 313)
dzp 0.012593497 -> scaling image by 1.0116665687010755
mosaic get_fwhm: self.fwhm = 2.71
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2454 2763 and amp range 2006 4053 and image y 2158 2470 and amp range 2040 4078
Range in image: x 0 310 , y 0 313 (with image size 310 x 313)
dzp 0.012593497 -> scaling image by 1.0116665687010755
mosaic get_fwhm: self.fwhm = 5.77
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3575 3896 and amp range 2048 4095 and image y 2777 3096 and amp range 2022 4060
Range in image: x 0 322 , y 0 320 (with image size 322 x 320)
dzp -0.02499645 -> scaling image by 0.9772404160512723
mosaic get_fwhm: self.fwhm = 3.25
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 849 1157 and amp range 0 2047 and image y 916 1225 and amp range 0 2021
Range in image: x 0 309 , y 0 310 (with image size 309 x 310)
dzp 0.005139143 -> scaling image by 1.0047445456970268
mosaic get_fwhm: self.fwhm = 3.69
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 0 39 and amp range 0 2005 and image y 822 1133 and amp range 0 2039
Range in image: x 0 40 , y 0 312 (with image size 40 x 312)
dzp 0.0013971253 -> scaling image by 1.0012876282438632
mosaic get_fwhm: self.fwhm = 3.63
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 0 51 and amp range 0 2005 and image y 844 1155 and amp range 0 2039
Range in image: x 0 52 , y 0 312 (with image size 52 x 312)
dzp 0.0013971253 -> scaling image by 1.0012876282438632
Faked up EXPNUM 73390081
Faked up EXPNUM 73390081
Faked up EXPNUM 73390081
90prime get_fwhm: self.fwhm = 2.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 453 631 and amp range 0 2015 and image y 3089 3268 and amp range 2048 4095
Range in image: x 0 179 , y 0 180 (with image size 179 x 180)
dzp 0.009255477 -> scaling image by 1.0085610475070432
Faked up EXPNUM 73990070
Faked up EXPNUM 73990070
Faked up EXPNUM 73990070
90prime get_fwhm: self.fwhm = 2.67
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 436 613 and amp range 0 2015 and image y 3068 3247 and amp range 2048 4095
Range in image: x 0 178 , y 0 180 (with image size 178 x 180)
dzp 0.009255477 -> scaling image by 1.0085610475070432
Faked up EXPNUM 74020116
Faked up EXPNUM 74020116
Faked up EXPNUM 74020116
90prime get_fwhm: self.fwhm = 2.89
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2365 2545 and amp range 2016 4031 and image y 1501 1681 and amp range 0 2047
Range in image: x 0 181 , y 0 181 (with image size 181 x 181)
dzp -0.012503507 -> scaling image by 0.9885499019198392
Faked up EXPNUM 78360039
Faked up EXPNUM 78360039
Faked up EXPNUM 78360039
90prime get_fwhm: self.fwhm = 4.41
WARNING: image median 0.017425312 is more than 1 sigma away from zero!
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 475 653 and amp range 0 2015 and image y 2863 3042 and amp range 2048 4095
Range in image: x 0 179 , y 0 180 (with image size 179 x 180)
dzp 0.01103811 -> scaling image by 1.0102183288830229
Faked up EXPNUM 73380138
Faked up EXPNUM 73380138
Faked up EXPNUM 73380138
90prime get_fwhm: self.fwhm = 3.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2394 2573 and amp range 2016 4031 and image y 1495 1676 and amp range 0 2047
Range in image: x 0 180 , y 0 182 (with image size 180 x 182)
dzp -0.008960029 -> scaling image by 0.9917814665476246
Faked up EXPNUM 78410030
Faked up EXPNUM 78410030
Faked up EXPNUM 78410030
90prime get_fwhm: self.fwhm = 3.29
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2622 2799 and amp range 2016 4031 and image y 1088 1266 and amp range 0 2047
Range in image: x 0 178 , y 0 179 (with image size 178 x 179)
dzp -3.875605e-05 -> scaling image by 0.9999643049955675
Faked up EXPNUM 81960042
Faked up EXPNUM 81960042
Faked up EXPNUM 81960042
90prime get_fwhm: self.fwhm = 2.71
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2631 2809 and amp range 2016 4031 and image y 1086 1264 and amp range 0 2047
Range in image: x 0 179 , y 0 179 (with image size 179 x 179)
dzp 0.0057031885 -> scaling image by 1.00526665101287
Stage tims : Wall: 6.89 s, CPU: 1.21 s, VmPeak: 942 MB, VmSize: 942 MB, VmRSS: 175 MB, VmData: 209 MB, maxrss: 0.566564 MB
Stage tims finished: 2020-06-07T16:48:50.832800
Resources for stage tims : 4.329 s worker CPU, 18.559 s worker Wall, Wall: 6.894 s, Cores in use: 0.80, Total efficiency (on 4 cores): 20.1 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-06-07T16:48:50.918281
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02275.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02404.fits
Matched 4 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/reference-custom-100921p37926.fits
Stage refs : Wall: 6.86 s, CPU: 6.45 s, VmPeak: 1013 MB, VmSize: 1007 MB, VmRSS: 241 MB, VmData: 273 MB, maxrss: 0.566564 MB
Stage refs finished: 2020-06-07T16:48:57.777866
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 6.861 s, Cores in use: 0.94, Total efficiency (on 4 cores): 23.5 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-07T16:48:57.779535
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/outlier-mask-custom-100921p37926.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/outliers-pre-custom-100921p37926.jpg
Image ksb_160112_062546_ooi_g_ls9-CCD4 g : masking 93 positive outlier pixels and 0 negative outlier pixels
Image ksb_151112_111331_ooi_g_ls9-CCD2 g : masking 106 positive outlier pixels and 0 negative outlier pixels
Image ksb_170324_025609_ooi_r_ls9-CCD4 r : masking 2819 positive outlier pixels and 0 negative outlier pixels
Image k4m_170127_043522_ooi_zd_ls9-CCD4 z : masking 5886 positive outlier pixels and 0 negative outlier pixels
Image k4m_170222_040839_ooi_zd_ls9-CCD1 z : masking 1236 positive outlier pixels and 0 negative outlier pixels
Image k4m_170302_043403_ooi_zd_ls9-CCD3 z : masking 0 positive outlier pixels and 168 negative outlier pixels
Image k4m_160210_034141_ooi_zd_ls9-CCD4 z : masking 528 positive outlier pixels and 0 negative outlier pixels
Image k4m_170226_040724_ooi_zd_ls9-CCD3 z : masking 0 positive outlier pixels and 461 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/outlier-mask-custom-100921p37926.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/outliers-post-custom-100921p37926.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/outliers-masked-pos-custom-100921p37926.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/outliers-masked-neg-custom-100921p37926.jpg
Stage outliers : Wall: 1.60 s, CPU: 1.12 s, VmPeak: 1016 MB, VmSize: 1016 MB, VmRSS: 261 MB, VmData: 290 MB, maxrss: 0.566564 MB
Stage outliers finished: 2020-06-07T16:48:59.378531
Resources for stage outliers : 2.030 s worker CPU, 2.193 s worker Wall, Wall: 1.600 s, Cores in use: 1.97, Total efficiency (on 4 cores): 49.1 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-07T16:48:59.380389
Stage halos : Wall: 0.38 s, CPU: 0.36 s, VmPeak: 1016 MB, VmSize: 1016 MB, VmRSS: 261 MB, VmData: 290 MB, maxrss: 0.566564 MB
Stage halos finished: 2020-06-07T16:48:59.755739
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.376 s, Cores in use: 0.96, Total efficiency (on 4 cores): 23.9 %
Stage halos finished
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-06-07T16:48:59.757650
PSF sigmas in coadd pixels: ['2.08', '1.97', '2.82', '2.42']
Using average PSF sigma 2.3230848575040626
Gaussian PSF norm 0.12143111813701142 vs pixelized 0.10348032
average tim pixel scale / coadd scale: 1.7303822251953866
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['3.25', '2.13', '2.00']
Using average PSF sigma 2.4569509914903302
Gaussian PSF norm 0.11481498521985818 vs pixelized 0.089020774
average tim pixel scale / coadd scale: 1.7304126509173794
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.52', '2.43', '1.14', '1.37', '1.53', '1.56']
Using average PSF sigma 1.592334917956963
Gaussian PSF norm 0.1771579512530055 vs pixelized 0.1381189
average tim pixel scale / coadd scale: 0.9908118704689368
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.36 s, CPU: 0.29 s, VmPeak: 1016 MB, VmSize: 1016 MB, VmRSS: 263 MB, VmData: 290 MB, maxrss: 0.566564 MB
Stage fit_on_coadds finished: 2020-06-07T16:49:00.114550
Resources for stage fit_on_coadds : 0.514 s worker CPU, 0.555 s worker Wall, Wall: 0.358 s, Cores in use: 2.24, Total efficiency (on 4 cores): 56.0 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-07T16:49:00.117743
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 73 potential peaks
Of 73 potential peaks: 31 in veto map, 9 cut by saddle test, 0 cut by aper test, 33 kept
SED z : 33 new peaks
Found 125 potential peaks
Of 125 potential peaks: 115 in veto map, 9 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 130 potential peaks
Of 130 potential peaks: 122 in veto map, 8 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 157 potential peaks
Of 157 potential peaks: 143 in veto map, 11 cut by saddle test, 0 cut by aper test, 3 kept
SED Flat : 3 new peaks
Found 154 potential peaks
Of 154 potential peaks: 140 in veto map, 12 cut by saddle test, 0 cut by aper test, 2 kept
SED Red : 2 new peaks
Adding 39 new sources
Median coadd sky for g : 0.0006207062
Median coadd sky for r : 0.0015644052
Median coadd sky for z : -0.00021374409
Stage srcs : Wall: 1.15 s, CPU: 1.07 s, VmPeak: 1016 MB, VmSize: 1016 MB, VmRSS: 263 MB, VmData: 290 MB, maxrss: 0.566564 MB
Stage srcs finished: 2020-06-07T16:49:01.263844
Resources for stage srcs : 0.183 s worker CPU, 0.183 s worker Wall, Wall: 1.147 s, Cores in use: 1.09, Total efficiency (on 4 cores): 27.2 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/PGC2113214-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-07T16:49:01.282357
Blob 1 of 2, id: 0, sources: 50, size: 299x307, npix 64074, brick X: 0,299, Y: 0,307, one pixel: 109 0
Blob 2 of 2, id: 1, sources: 2, size: 30x35, npix 581, brick X: 277,307, Y: 125,160, one pixel: 286 125
Blob 2 finished, total: Wall: 0.59 s, CPU: 0.59 s, VmPeak: 809 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141128 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (100.94740, 37.91755), pm (2.3, 0.6), parallax 0.808 with NanoMaggies: g=15.5, r=15.1, z=14.9
Reset brightness to NanoMaggies: g=15.5, r=15.1, z=14.9
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (100.92544, 37.91353), pm (0.3, -2.3), parallax 0.254 with NanoMaggies: g=16.5, r=16, z=(flux -823)
Reset brightness to NanoMaggies: g=16.5, r=16, z=15.9
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=0.980349, ee1=0.103742, ee2=-0.0483927
Measuring 8 iterative sources
Blob 1 finished, total: Wall: 36.25 s, CPU: 36.25 s, VmPeak: 810 MB, VmSize: 724 MB, VmRSS: 157 MB, VmData: 189 MB, maxrss: 0.187292 MB
Old catalog: 65
New catalog: 55
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/blobs-custom-100921p37926.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/metrics/cus/all-models-custom-100921p37926.fits
Stage fitblobs : Wall: 36.90 s, CPU: 0.56 s, VmPeak: 1016 MB, VmSize: 1016 MB, VmRSS: 266 MB, VmData: 293 MB, maxrss: 0.566564 MB
Stage fitblobs finished: 2020-06-07T16:49:38.182690
Resources for stage fitblobs : 36.920 s worker CPU, 36.944 s worker Wall, Wall: 36.902 s, Cores in use: 1.02, Total efficiency (on 4 cores): 25.4 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-07T16:49:38.185246
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/tmp-legacysurvey-custom-100921p37926-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/tmp-legacysurvey-custom-100921p37926-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/tmp-legacysurvey-custom-100921p37926-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/tmp-legacysurvey-custom-100921p37926-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-resid.jpg
Stage coadds : Wall: 1.59 s, CPU: 0.67 s, VmPeak: 1016 MB, VmSize: 1016 MB, VmRSS: 266 MB, VmData: 293 MB, maxrss: 0.566564 MB
Stage coadds finished: 2020-06-07T16:49:39.770899
Resources for stage coadds : 2.666 s worker CPU, 2.805 s worker Wall, Wall: 1.587 s, Cores in use: 2.11, Total efficiency (on 4 cores): 52.6 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-07T16:49:39.772654
Cut to 1 unWISE tiles
Reading WISE tile 1004p378 band 1
Reading WISE tile 1004p378 band 2
Reading WISE tile 1004p378 band 3
Reading WISE tile 1004p378 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.18 s, CPU: 0.18 s, VmPeak: 810 MB, VmSize: 792 MB, VmRSS: 145 MB, VmData: 177 MB, maxrss: 0.184048 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.15 s, CPU: 0.15 s, VmPeak: 810 MB, VmSize: 800 MB, VmRSS: 157 MB, VmData: 182 MB, maxrss: 0.221824 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 813 MB, VmSize: 784 MB, VmRSS: 137 MB, VmData: 170 MB, maxrss: 0.200492 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/100/1004p378/unwise-1004p378-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 809 MB, VmSize: 792 MB, VmRSS: 144 MB, VmData: 174 MB, maxrss: 0.209008 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/tmp-legacysurvey-custom-100921p37926-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/tmp-legacysurvey-custom-100921p37926-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 3.91 s, CPU: 0.61 s, VmPeak: 1016 MB, VmSize: 1016 MB, VmRSS: 267 MB, VmData: 293 MB, maxrss: 0.566564 MB
Stage wise_forced finished: 2020-06-07T16:49:43.677998
Resources for stage wise_forced : 5.318 s worker CPU, 7.059 s worker Wall, Wall: 3.907 s, Cores in use: 1.52, Total efficiency (on 4 cores): 38.0 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-07T16:49:43.680052
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/coadd/cus/custom-100921p37926/legacysurvey-custom-100921p37926-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/tractor-i/cus/tractor-custom-100921p37926.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/tractor/cus/tractor-custom-100921p37926.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/100/PGC2113214/tractor/cus/brick-custom-100921p37926.sha256sum
Stage writecat : Wall: 0.91 s, CPU: 0.69 s, VmPeak: 1149 MB, VmSize: 1021 MB, VmRSS: 269 MB, VmData: 293 MB, maxrss: 0.566564 MB
Stage writecat finished: 2020-06-07T16:49:44.590288
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.911 s, Cores in use: 0.75, Total efficiency (on 4 cores): 18.9 %
Stage writecat finished
All done: 51.961 s worker CPU, 68.301 s worker Wall, Wall: 60.658 s, Cores in use: 1.07, Total efficiency (on 4 cores): 26.8 %
Finished galaxy PGC2113214 in 1.050 minutes.