Started working on galaxy PGC104351 at Thu Jun  4 21:13:45 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 176.28771 -6.9588852 --width 275 --height 275 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-04T21:13:46.527989
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '176.28771', '-6.9588852', '--width', '275', '--height', '275', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 176.28771 -6.9588852 --width 275 --height 275 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 176.28771 -6.9588852
Starting process 33359 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057304 MB
Starting process 33360 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057304 MB
Starting process 33362 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB
Starting process 33361 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057304 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-04T21:13:50.179945
Keeping 11 CCDs:
   c4d_190102_081436_ooi_z_ls9.fits.fz c4d_190102_081436_ooi_z_ls9-S2 z exptime 105.0 propid 2014B-0404 seeing 0.71 MJD 58485.343 object DECaLS_38730_z
   c4d_180516_000058_ooi_r_ls9.fits.fz c4d_180516_000058_ooi_r_ls9-S2 r exptime 121.0 propid 2014B-0404 seeing 0.98 MJD 58254.001 object DECaLS_38730_r
   c4d_150415_030117_ooi_r_ls9.fits.fz c4d_150415_030117_ooi_r_ls9-S4 r exptime 60.0 propid 2014B-0404 seeing 0.78 MJD 57127.125 object DECaLS_6986_r
   c4d_150415_030252_ooi_g_ls9.fits.fz c4d_150415_030252_ooi_g_ls9-S4 g exptime 65.0 propid 2014B-0404 seeing 0.80 MJD 57127.126 object DECaLS_6986_g
   c4d_151227_074546_ooi_z_ls9.fits.fz c4d_151227_074546_ooi_z_ls9-S4 z exptime 117.0 propid 2014B-0404 seeing 0.91 MJD 57383.322 object DECaLS_6986_z
   c4d_170304_052539_ooi_r_ls9.fits.fz c4d_170304_052539_ooi_r_ls9-S3 r exptime 76.0 propid 2014B-0404 seeing 1.17 MJD 57816.225 object DECaLS_22858_r
   c4d_190307_051502_ooi_r_ls9.fits.fz c4d_190307_051502_ooi_r_ls9-S3 r exptime 40.0 propid 2014B-0404 seeing 1.15 MJD 58549.219 object DECaLS_22858_r
   c4d_170110_053323_ooi_z_ls9.fits.fz c4d_170110_053323_ooi_z_ls9-S3 z exptime 92.0 propid 2014B-0404 seeing 0.92 MJD 57763.230 object DECaLS_22858_z
   c4d_160410_032142_ooi_g_ls9.fits.fz c4d_160410_032142_ooi_g_ls9-S3 g exptime 90.0 propid 2014B-0404 seeing 1.40 MJD 57488.138 object DECaLS_22858_g
   c4d_181216_072852_ooi_g_ls9.fits.fz c4d_181216_072852_ooi_g_ls9-S2 g exptime 161.0 propid 2014B-0404 seeing 1.32 MJD 58468.312 object DECaLS_38730_g
   c4d_170718_230111_ooi_z_ls9.fits.fz c4d_170718_230111_ooi_z_ls9-S2 z exptime 80.0 propid 2014B-0404 seeing 1.07 MJD 57952.958 object DECaLS_38730_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95340ea-z-20190102T081436_oi[S2,noinherit][2046,4094][real]: DECaLS_3
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9555e83-r-20150415T025927_oi[S4,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9559045-g-20150415T030055_oi[S4,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950f1c8-r-20170304T052335_oi[S3,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95527d4-z-20151227T074258_oi[S4,noinherit][2046,4094][real]: DECaLS_6
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539ab3-r-20190307T051502_oi[S3,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953d2b6-z-20170110T053100_oi[S3,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9568910-g-20160410T031922_oi[S3,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9560003-r-20180516T000058_oi[S2,noinherit][2046,4094][real]: DECaLS_3
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9518ea9-z-20170718T225857_oi[S2,noinherit][2046,4094][real]: DECaLS_3
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539c09-g-20181216T072852_oi[S2,noinherit][2046,4094][real]: DECaLS_3
Stage tims : Wall: 81.29 s, CPU: 4.27 s, VmPeak: 995 MB, VmSize: 995 MB, VmRSS: 175 MB, VmData: 208 MB, maxrss: 0.456824 MB
Stage tims finished: 2020-06-04T21:15:11.469867
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95340ea-z-20190102T081436_oi[S2,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95527d4-z-20151227T074258_oi[S4,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9568910-g-20160410T031922_oi[S3,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9560003-r-20180516T000058_oi[S2,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9555e83-r-20150415T025927_oi[S4,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950f1c8-r-20170304T052335_oi[S3,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953d2b6-z-20170110T053100_oi[S3,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9518ea9-z-20170718T225857_oi[S2,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
Resources for stage tims : 4.384 s worker CPU, 249.000 s worker Wall, Wall: 81.291 s, Cores in use: 0.11, Total efficiency (on 4 cores): 2.7 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-tims.p
Running stage refs
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9559045-g-20150415T030055_oi[S4,noinherit][2046,4094][real]: DECaLS_6 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539ab3-r-20190307T051502_oi[S3,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539c09-g-20181216T072852_oi[S2,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
Running stage refs at 2020-06-04T21:15:11.493710
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06910.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06783.fits
Matched 7 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/reference-custom-176287m06958.fits
Stage refs : Wall: 7.42 s, CPU: 6.91 s, VmPeak: 1026 MB, VmSize: 1012 MB, VmRSS: 193 MB, VmData: 225 MB, maxrss: 0.456824 MB
Stage refs finished: 2020-06-04T21:15:18.915177
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 7.422 s, Cores in use: 0.93, Total efficiency (on 4 cores): 23.3 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-04T21:15:18.916489
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/outlier-mask-custom-176287m06958.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/outliers-pre-custom-176287m06958.jpg
Image c4d_150415_030252_ooi_g_ls9-S4 g : masking 162 positive outlier pixels and 0 negative outlier pixels
Image c4d_170304_052539_ooi_r_ls9-S3 r : masking 427 positive outlier pixels and 0 negative outlier pixels
Image c4d_170718_230111_ooi_z_ls9-S2 z : masking 902 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/outlier-mask-custom-176287m06958.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/outliers-post-custom-176287m06958.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/outliers-masked-pos-custom-176287m06958.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/outliers-masked-neg-custom-176287m06958.jpg
Stage outliers : Wall: 4.33 s, CPU: 4.06 s, VmPeak: 1026 MB, VmSize: 1021 MB, VmRSS: 206 MB, VmData: 236 MB, maxrss: 0.456824 MB
Stage outliers finished: 2020-06-04T21:15:23.249193
Resources for stage outliers : 0.880 s worker CPU, 0.889 s worker Wall, Wall: 4.334 s, Cores in use: 1.14, Total efficiency (on 4 cores): 28.5 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-04T21:15:23.250924
Stage halos : Wall: 5.69 s, CPU: 5.69 s, VmPeak: 1026 MB, VmSize: 1021 MB, VmRSS: 206 MB, VmData: 236 MB, maxrss: 0.456824 MB
Stage halos finished: 2020-06-04T21:15:28.937896
Resources for stage halos : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 5.688 s, Cores in use: 1.00, Total efficiency (on 4 cores): 25.0 %
Stage halos finished
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-06-04T21:15:28.939643
PSF sigmas in coadd pixels: ['1.30', '2.27', '2.15']
Using average PSF sigma 1.9057887045722908
Gaussian PSF norm 0.14801997256940805 vs pixelized 0.121570945
average tim pixel scale / coadd scale: 1.0011317221739309
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.58', '1.26', '1.91', '1.86']
Using average PSF sigma 1.6534990917034422
Gaussian PSF norm 0.17060474552983446 vs pixelized 0.12531282
average tim pixel scale / coadd scale: 1.0010706957737125
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.16', '1.48', '1.49', '1.74']
Using average PSF sigma 1.466780345921707
Gaussian PSF norm 0.19232245138696155 vs pixelized 0.16907983
average tim pixel scale / coadd scale: 1.0011455520475638
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.20 s, CPU: 0.18 s, VmPeak: 1026 MB, VmSize: 1021 MB, VmRSS: 209 MB, VmData: 237 MB, maxrss: 0.456824 MB
Stage fit_on_coadds finished: 2020-06-04T21:15:29.143959
Resources for stage fit_on_coadds : 0.211 s worker CPU, 0.224 s worker Wall, Wall: 0.205 s, Cores in use: 1.89, Total efficiency (on 4 cores): 47.4 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-04T21:15:29.146793
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 21 potential peaks
Of 21 potential peaks: 7 in veto map, 1 cut by saddle test, 0 cut by aper test, 13 kept
SED z : 13 new peaks
Found 22 potential peaks
Of 22 potential peaks: 17 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept
SED r : 2 new peaks
Found 17 potential peaks
Of 17 potential peaks: 14 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 26 potential peaks
Of 26 potential peaks: 19 in veto map, 3 cut by saddle test, 0 cut by aper test, 4 kept
SED Flat : 4 new peaks
Found 29 potential peaks
Of 29 potential peaks: 25 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 19 new sources
Median coadd sky for g : -1.736188e-05
Median coadd sky for r : -8.138092e-05
Median coadd sky for z : 0.000110129426
Stage srcs : Wall: 5.57 s, CPU: 5.52 s, VmPeak: 1026 MB, VmSize: 1021 MB, VmRSS: 210 MB, VmData: 237 MB, maxrss: 0.456824 MB
Stage srcs finished: 2020-06-04T21:15:34.718826
Resources for stage srcs : 0.126 s worker CPU, 0.135 s worker Wall, Wall: 5.573 s, Cores in use: 1.01, Total efficiency (on 4 cores): 25.3 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/PGC104351-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-04T21:15:34.889937
Blob 1 of 15, id: 5, sources: 4, size: 133x84, npix 7549, brick X: 68,201, Y: 97,181, one pixel: 81 97
Blob 2 of 15, id: 1, sources: 3, size: 61x54, npix 1728, brick X: 27,88, Y: 27,81, one pixel: 37 27
Blob 3 of 15, id: 7, sources: 2, size: 54x34, npix 1053, brick X: 47,101, Y: 157,191, one pixel: 82 157
Blob 4 of 15, id: 0, sources: 1, size: 44x37, npix 986, brick X: 189,233, Y: 9,46, one pixel: 215 9
Blob 5 of 15, id: 11, sources: 1, size: 31x31, npix 578, brick X: 109,140, Y: 235,266, one pixel: 125 235
Blob 3 finished, total: Wall: 1.62 s, CPU: 1.62 s, VmPeak: 957 MB, VmSize: 719 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145628 MB
Blob 6 of 15, id: 10, sources: 1, size: 27x32, npix 564, brick X: 224,251, Y: 208,240, one pixel: 237 208
Blob 4 finished, total: Wall: 1.96 s, CPU: 1.96 s, VmPeak: 947 MB, VmSize: 714 MB, VmRSS: 139 MB, VmData: 177 MB, maxrss: 0.141864 MB
Blob 7 of 15, id: 9, sources: 1, size: 26x26, npix 439, brick X: 131,157, Y: 197,223, one pixel: 142 197
Blob 5 finished, total: Wall: 1.01 s, CPU: 1.01 s, VmPeak: 957 MB, VmSize: 719 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145628 MB
Blob 8 of 15, id: 4, sources: 1, size: 24x25, npix 406, brick X: 35,59, Y: 79,104, one pixel: 45 79
Measuring 1 iterative sources
Blob 6 finished, total: Wall: 1.62 s, CPU: 1.62 s, VmPeak: 947 MB, VmSize: 714 MB, VmRSS: 139 MB, VmData: 177 MB, maxrss: 0.142432 MB
Blob 9 of 15, id: 3, sources: 1, size: 21x28, npix 367, brick X: 215,236, Y: 53,81, one pixel: 225 53
Blob 7 finished, total: Wall: 1.04 s, CPU: 1.04 s, VmPeak: 957 MB, VmSize: 719 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145628 MB
Blob 10 of 15, id: 2, sources: 1, size: 24x24, npix 360, brick X: 149,173, Y: 36,60, one pixel: 160 36
Blob 8 finished, total: Wall: 0.79 s, CPU: 0.79 s, VmPeak: 947 MB, VmSize: 714 MB, VmRSS: 139 MB, VmData: 177 MB, maxrss: 0.142432 MB
Blob 11 of 15, id: 12, sources: 1, size: 22x21, npix 281, brick X: 80,102, Y: 252,273, one pixel: 89 252
Measuring 1 iterative sources
Blob 9 finished, total: Wall: 0.84 s, CPU: 0.84 s, VmPeak: 957 MB, VmSize: 719 MB, VmRSS: 144 MB, VmData: 182 MB, maxrss: 0.14778 MB
Blob 12 of 15, id: 13, sources: 1, size: 20x19, npix 244, brick X: 230,250, Y: 256,275, one pixel: 238 256
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.58522, ee1=-0.416025, ee2=-0.0579385
Measuring 2 iterative sources
Blob 11 finished, total: Wall: 0.73 s, CPU: 0.73 s, VmPeak: 957 MB, VmSize: 719 MB, VmRSS: 144 MB, VmData: 182 MB, maxrss: 0.14778 MB
Blob 10 finished, total: Wall: 0.87 s, CPU: 0.87 s, VmPeak: 947 MB, VmSize: 714 MB, VmRSS: 139 MB, VmData: 177 MB, maxrss: 0.142432 MB
Blob 13 of 15, id: 8, sources: 1, size: 19x19, npix 208, brick X: 101,120, Y: 195,214, one pixel: 109 195
Blob 14 of 15, id: 6, sources: 1, size: 19x20, npix 200, brick X: 1,20, Y: 114,134, one pixel: 10 114
Blob 13 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 947 MB, VmSize: 714 MB, VmRSS: 139 MB, VmData: 177 MB, maxrss: 0.142432 MB
Blob 15 of 15, id: 14, sources: 1, size: 20x15, npix 185, brick X: 48,68, Y: 260,275, one pixel: 57 260
Blob 14 finished, total: Wall: 0.35 s, CPU: 0.35 s, VmPeak: 947 MB, VmSize: 714 MB, VmRSS: 139 MB, VmData: 177 MB, maxrss: 0.142432 MB
Blob 12 finished, total: Wall: 0.83 s, CPU: 0.83 s, VmPeak: 957 MB, VmSize: 719 MB, VmRSS: 144 MB, VmData: 182 MB, maxrss: 0.14778 MB
Blob 2 finished, total: Wall: 6.13 s, CPU: 6.13 s, VmPeak: 962 MB, VmSize: 720 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148736 MB
Blob 1 finished, total: Wall: 6.47 s, CPU: 6.46 s, VmPeak: 967 MB, VmSize: 720 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150232 MB
Blob 15 finished, total: Wall: 0.53 s, CPU: 0.53 s, VmPeak: 947 MB, VmSize: 714 MB, VmRSS: 139 MB, VmData: 177 MB, maxrss: 0.142432 MB
Old catalog: 23
New catalog: 24
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/blobs-custom-176287m06958.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/metrics/cus/all-models-custom-176287m06958.fits
Stage fitblobs : Wall: 13.42 s, CPU: 5.52 s, VmPeak: 1026 MB, VmSize: 1021 MB, VmRSS: 210 MB, VmData: 237 MB, maxrss: 0.456824 MB
Stage fitblobs finished: 2020-06-04T21:15:48.306939
Resources for stage fitblobs : 25.170 s worker CPU, 25.198 s worker Wall, Wall: 13.418 s, Cores in use: 2.29, Total efficiency (on 4 cores): 57.2 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-04T21:15:48.308726
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/tmp-legacysurvey-custom-176287m06958-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/tmp-legacysurvey-custom-176287m06958-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/tmp-legacysurvey-custom-176287m06958-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/tmp-legacysurvey-custom-176287m06958-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-resid.jpg
Stage coadds : Wall: 8.40 s, CPU: 7.00 s, VmPeak: 1026 MB, VmSize: 1021 MB, VmRSS: 210 MB, VmData: 237 MB, maxrss: 0.456824 MB
Stage coadds finished: 2020-06-04T21:15:56.710603
Resources for stage coadds : 2.386 s worker CPU, 2.560 s worker Wall, Wall: 8.403 s, Cores in use: 1.12, Total efficiency (on 4 cores): 27.9 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-04T21:15:56.712296
Cut to 1 unWISE tiles
Reading WISE tile 1769m076 band 2
Reading WISE tile 1769m076 band 1
Reading WISE tile 1769m076 band 3
Reading WISE tile 1769m076 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 962 MB, VmSize: 798 MB, VmRSS: 148 MB, VmData: 177 MB, maxrss: 0.189444 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 967 MB, VmSize: 795 MB, VmRSS: 146 MB, VmData: 174 MB, maxrss: 0.210312 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 957 MB, VmSize: 795 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.209728 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/176/1769m076/unwise-1769m076-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 947 MB, VmSize: 795 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.210004 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/tmp-legacysurvey-custom-176287m06958-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/tmp-legacysurvey-custom-176287m06958-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 13.87 s, CPU: 6.04 s, VmPeak: 1026 MB, VmSize: 1021 MB, VmRSS: 211 MB, VmData: 237 MB, maxrss: 0.456824 MB
Stage wise_forced finished: 2020-06-04T21:16:10.584154
Resources for stage wise_forced : 3.947 s worker CPU, 7.480 s worker Wall, Wall: 13.873 s, Cores in use: 0.72, Total efficiency (on 4 cores): 18.0 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-04T21:16:10.585844
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/coadd/cus/custom-176287m06958/legacysurvey-custom-176287m06958-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/tractor-i/cus/tractor-custom-176287m06958.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/tractor/cus/tractor-custom-176287m06958.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/176/PGC104351/tractor/cus/brick-custom-176287m06958.sha256sum
Stage writecat : Wall: 8.12 s, CPU: 7.99 s, VmPeak: 1154 MB, VmSize: 1026 MB, VmRSS: 213 MB, VmData: 237 MB, maxrss: 0.456824 MB
Stage writecat finished: 2020-06-04T21:16:18.707940
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 8.123 s, Cores in use: 0.98, Total efficiency (on 4 cores): 24.6 %
Stage writecat finished
All done: 37.105 s worker CPU, 285.488 s worker Wall, Wall: 148.533 s, Cores in use: 0.61, Total efficiency (on 4 cores): 15.2 %
Finished galaxy PGC104351 in 2.555 minutes.