Started working on galaxy IC2628_GROUP at Fri Jun  5 05:14:52 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 167.91222034031892 12.12344773664682 --width 705 --height 705 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-05T05:14:54.131639
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '167.91222034031892', '12.12344773664682', '--width', '705', '--height', '705', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 167.91222034031892 12.12344773664682 --width 705 --height 705 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 167.91222034031892 12.12344773664682
Starting process 25309 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057376 MB
Starting process 25310 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.0574 MB
Starting process 25311 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057468 MB
Starting process 25312 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.05738 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-05T05:14:56.663298
Keeping 19 CCDs:
   c4d_170514_004925_ooi_z_ls9.fits.fz c4d_170514_004925_ooi_z_ls9-S20 z exptime 250.0 propid 2014B-0404 seeing 1.49 MJD 57887.031 object DECaLS_35118_z
   c4d_170514_004925_ooi_z_ls9.fits.fz c4d_170514_004925_ooi_z_ls9-S21 z exptime 250.0 propid 2014B-0404 seeing 1.48 MJD 57887.031 object DECaLS_35118_z
   c4d_180318_043628_ooi_r_ls9.fits.fz c4d_180318_043628_ooi_r_ls9-S20 r exptime 51.0 propid 2014B-0404 seeing 1.13 MJD 58195.192 object DECaLS_35118_r
   c4d_180318_043343_ooi_g_ls9.fits.fz c4d_180318_043343_ooi_g_ls9-S20 g exptime 135.0 propid 2014B-0404 seeing 1.32 MJD 58195.190 object DECaLS_35118_g
   c4d_180318_043628_ooi_r_ls9.fits.fz c4d_180318_043628_ooi_r_ls9-S21 r exptime 51.0 propid 2014B-0404 seeing 1.09 MJD 58195.192 object DECaLS_35118_r
   c4d_180318_043343_ooi_g_ls9.fits.fz c4d_180318_043343_ooi_g_ls9-S21 g exptime 135.0 propid 2014B-0404 seeing 1.26 MJD 58195.190 object DECaLS_35118_g
   c4d_180317_044325_ooi_r_ls9.fits.fz c4d_180317_044325_ooi_r_ls9-N31 r exptime 52.0 propid 2014B-0404 seeing 1.24 MJD 58194.197 object DECaLS_35096_r
   c4d_180317_044035_ooi_g_ls9.fits.fz c4d_180317_044035_ooi_g_ls9-N31 g exptime 142.0 propid 2014B-0404 seeing 1.38 MJD 58194.195 object DECaLS_35096_g
   c4d_170317_040816_ooi_z_ls9.fits.fz c4d_170317_040816_ooi_z_ls9-N31 z exptime 105.0 propid 2014B-0404 seeing 1.07 MJD 57829.171 object DECaLS_35096_z
   c4d_170111_062352_ooi_z_ls9.fits.fz c4d_170111_062352_ooi_z_ls9-S21 z exptime 87.0 propid 2014B-0404 seeing 1.16 MJD 57764.265 object DECaLS_19246_z
   c4d_170422_003227_ooi_g_ls9.fits.fz c4d_170422_003227_ooi_g_ls9-S21 g exptime 163.0 propid 2014B-0404 seeing 1.21 MJD 57865.020 object DECaLS_19246_g
   c4d_170422_000843_ooi_r_ls9.fits.fz c4d_170422_000843_ooi_r_ls9-S21 r exptime 78.0 propid 2014B-0404 seeing 1.19 MJD 57865.005 object DECaLS_19246_r
   c4d_160117_081507_ooi_g_ls9.fits.fz c4d_160117_081507_ooi_g_ls9-S22 g exptime 65.0 propid 2014B-0404 seeing 1.32 MJD 57404.342 object DECaLS_3374_g
   c4d_160117_081329_ooi_r_ls9.fits.fz c4d_160117_081329_ooi_r_ls9-S22 r exptime 40.0 propid 2014B-0404 seeing 1.08 MJD 57404.342 object DECaLS_3374_r
   c4d_190306_051857_ooi_g_ls9.fits.fz c4d_190306_051857_ooi_g_ls9-S22 g exptime 81.0 propid 2014B-0404 seeing 1.20 MJD 58548.221 object DECaLS_3374_g
   c4d_160110_083003_ooi_z_ls9.fits.fz c4d_160110_083003_ooi_z_ls9-S22 z exptime 87.0 propid 2014B-0404 seeing 1.03 MJD 57397.353 object DECaLS_3374_z
   c4d_170514_004925_ooi_z_ls9.fits.fz c4d_170514_004925_ooi_z_ls9-S25 z exptime 250.0 propid 2014B-0404 seeing 1.48 MJD 57887.031 object DECaLS_35118_z
   c4d_180318_043628_ooi_r_ls9.fits.fz c4d_180318_043628_ooi_r_ls9-S25 r exptime 51.0 propid 2014B-0404 seeing 1.16 MJD 58195.192 object DECaLS_35118_r
   c4d_180318_043343_ooi_g_ls9.fits.fz c4d_180318_043343_ooi_g_ls9-S25 g exptime 135.0 propid 2014B-0404 seeing 1.34 MJD 58195.190 object DECaLS_35118_g
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95164f7-z-20170514T004423_oi[S20,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95164f7-z-20170514T004423_oi[S21,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952df04-r-20180318T043628_oi[S20,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952e886-g-20180318T043343_oi[S20,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952e886-g-20180318T043343_oi[S21,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952df04-r-20180318T043628_oi[S21,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952da1d-r-20180317T044325_oi[N31,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952dda8-g-20180317T044035_oi[N31,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953c314-z-20170111T062137_oi[S21,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95176d0-r-20170422T000637_oi[S21,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9517d11-g-20170422T002854_oi[S21,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956e86d-g-20160117T081308_oi[S22,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953a3c6-g-20190306T051857_oi[S22,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950fef3-z-20170317T040541_oi[N31,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9553d83-z-20160110T082744_oi[S22,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95164f7-z-20170514T004423_oi[S25,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952e886-g-20180318T043343_oi[S25,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952df04-r-20180318T043628_oi[S25,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9571eaa-r-20160117T081200_oi[S22,noinherit][2046,4094][real]: DECaLS_
Stage tims : Wall: 137.30 s, CPU: 3.65 s, VmPeak: 997 MB, VmSize: 997 MB, VmRSS: 214 MB, VmData: 248 MB, maxrss: 0.454636 MB
Stage tims finished: 2020-06-05T05:17:13.963209
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952df04-r-20180318T043628_oi[S20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952dda8-g-20180317T044035_oi[N31,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9517d11-g-20170422T002854_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953a3c6-g-20190306T051857_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95164f7-z-20170514T004423_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95164f7-z-20170514T004423_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952df04-r-20180318T043628_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953c314-z-20170111T062137_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956e86d-g-20160117T081308_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9553d83-z-20160110T082744_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952e886-g-20180318T043343_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95164f7-z-20170514T004423_oi[S20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952e886-g-20180318T043343_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950fef3-z-20170317T040541_oi[N31,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952df04-r-20180318T043628_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Resources for stage tims : 6.368 s worker CPU, 430.789 s worker Wall, Wall: 137.301 s, Cores in use: 0.07, Total efficiency (on 4 cores): 1.8 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-tims.p
Running stage refs
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952e886-g-20180318T043343_oi[S20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952da1d-r-20180317T044325_oi[N31,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95176d0-r-20170422T000637_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9571eaa-r-20160117T081200_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Running stage refs at 2020-06-05T05:17:14.036173
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-04732.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-04859.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-04988.fits
Matched 12 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/reference-custom-167912p12123.fits
Stage refs : Wall: 22.74 s, CPU: 9.32 s, VmPeak: 1042 MB, VmSize: 1023 MB, VmRSS: 241 MB, VmData: 274 MB, maxrss: 0.454636 MB
Stage refs finished: 2020-06-05T05:17:36.778160
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 22.743 s, Cores in use: 0.41, Total efficiency (on 4 cores): 10.2 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-05T05:17:36.779811
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/outlier-mask-custom-167912p12123.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/outliers-pre-custom-167912p12123.jpg
Image c4d_180318_043343_ooi_g_ls9-S20 g : masking 668 positive outlier pixels and 125 negative outlier pixels
Image c4d_180318_043343_ooi_g_ls9-S25 g : masking 166 positive outlier pixels and 0 negative outlier pixels
Image c4d_190306_051857_ooi_g_ls9-S22 g : masking 2338 positive outlier pixels and 73 negative outlier pixels
Image c4d_170422_003227_ooi_g_ls9-S21 g : masking 886 positive outlier pixels and 0 negative outlier pixels
Image c4d_160117_081507_ooi_g_ls9-S22 g : masking 3412 positive outlier pixels and 293 negative outlier pixels
Image c4d_180318_043628_ooi_r_ls9-S20 r : masking 147 positive outlier pixels and 0 negative outlier pixels
Image c4d_180317_044325_ooi_r_ls9-N31 r : masking 167 positive outlier pixels and 0 negative outlier pixels
Image c4d_170422_000843_ooi_r_ls9-S21 r : masking 146 positive outlier pixels and 0 negative outlier pixels
Image c4d_180318_043628_ooi_r_ls9-S25 r : masking 101 positive outlier pixels and 259 negative outlier pixels
Image c4d_160117_081329_ooi_r_ls9-S22 r : masking 2434 positive outlier pixels and 107 negative outlier pixels
Image c4d_170514_004925_ooi_z_ls9-S21 z : masking 210 positive outlier pixels and 0 negative outlier pixels
Image c4d_170514_004925_ooi_z_ls9-S25 z : masking 217 positive outlier pixels and 0 negative outlier pixels
Image c4d_170111_062352_ooi_z_ls9-S21 z : masking 751 positive outlier pixels and 257 negative outlier pixels
Image c4d_160110_083003_ooi_z_ls9-S22 z : masking 29830 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/outlier-mask-custom-167912p12123.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/outliers-post-custom-167912p12123.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/outliers-masked-pos-custom-167912p12123.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/outliers-masked-neg-custom-167912p12123.jpg
Stage outliers : Wall: 10.19 s, CPU: 8.23 s, VmPeak: 1161 MB, VmSize: 1075 MB, VmRSS: 277 MB, VmData: 306 MB, maxrss: 0.454636 MB
Stage outliers finished: 2020-06-05T05:17:46.965896
Resources for stage outliers : 7.848 s worker CPU, 8.105 s worker Wall, Wall: 10.188 s, Cores in use: 1.58, Total efficiency (on 4 cores): 39.5 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-05T05:17:46.968365
Stage halos : Wall: 12.80 s, CPU: 4.56 s, VmPeak: 1161 MB, VmSize: 1011 MB, VmRSS: 263 MB, VmData: 293 MB, maxrss: 0.454636 MB
Stage halos finished: 2020-06-05T05:17:59.767661
Resources for stage halos : 0.908 s worker CPU, 23.710 s worker Wall, Wall: 13.160 s, Cores in use: 0.42, Total efficiency (on 4 cores): 10.4 %
Stage halos finished
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-06-05T05:18:00.128826
PSF sigmas in coadd pixels: ['2.15', '2.05', '2.24', '1.96', '2.14', '1.94', '2.18']
Using average PSF sigma 2.095321037882318
Gaussian PSF norm 0.134630821088392 vs pixelized 0.11896698
average tim pixel scale / coadd scale: 1.0011262425923295
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.84', '1.76', '2.01', '1.93', '1.75', '1.89']
Using average PSF sigma 1.864016447485328
Gaussian PSF norm 0.15133707224227622 vs pixelized 0.13229951
average tim pixel scale / coadd scale: 1.0011447022601478
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.42', '2.41', '1.74', '1.88', '1.68', '2.41']
Using average PSF sigma 2.0893180880257067
Gaussian PSF norm 0.13501763728109134 vs pixelized 0.11269541
average tim pixel scale / coadd scale: 1.0010758861419495
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 16.38 s, CPU: 1.01 s, VmPeak: 1161 MB, VmSize: 1031 MB, VmRSS: 281 MB, VmData: 313 MB, maxrss: 0.454636 MB
Stage fit_on_coadds finished: 2020-06-05T05:18:16.509858
Resources for stage fit_on_coadds : 1.500 s worker CPU, 1.666 s worker Wall, Wall: 16.382 s, Cores in use: 0.15, Total efficiency (on 4 cores): 3.8 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-05T05:18:16.513628
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 209 potential peaks
Of 209 potential peaks: 85 in veto map, 25 cut by saddle test, 1 cut by aper test, 98 kept
SED z : 98 new peaks
Found 184 potential peaks
Of 184 potential peaks: 148 in veto map, 19 cut by saddle test, 0 cut by aper test, 17 kept
SED r : 17 new peaks
Found 223 potential peaks
Of 223 potential peaks: 158 in veto map, 32 cut by saddle test, 0 cut by aper test, 33 kept
SED g : 33 new peaks
Found 294 potential peaks
Of 294 potential peaks: 247 in veto map, 34 cut by saddle test, 0 cut by aper test, 13 kept
SED Flat : 13 new peaks
Found 309 potential peaks
Of 309 potential peaks: 264 in veto map, 36 cut by saddle test, 0 cut by aper test, 9 kept
SED Red : 9 new peaks
Adding 170 new sources
Median coadd sky for g : -6.0868544e-05
Median coadd sky for r : -4.5771685e-05
Median coadd sky for z : 2.1754688e-05
Stage srcs : Wall: 9.49 s, CPU: 9.19 s, VmPeak: 1161 MB, VmSize: 1038 MB, VmRSS: 292 MB, VmData: 320 MB, maxrss: 0.454636 MB
Stage srcs finished: 2020-06-05T05:18:26.003254
Resources for stage srcs : 0.775 s worker CPU, 0.775 s worker Wall, Wall: 9.492 s, Cores in use: 1.05, Total efficiency (on 4 cores): 26.2 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/IC2628_GROUP-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-05T05:18:26.046892
Blob 1 of 119, id: 40, sources: 15, size: 348x281, npix 51469, brick X: 186,534, Y: 205,486, one pixel: 469 205
Blob 2 of 119, id: 43, sources: 2, size: 100x86, npix 5290, brick X: 175,275, Y: 232,318, one pixel: 208 232
Blob 3 of 119, id: 0, sources: 5, size: 114x61, npix 4325, brick X: 20,134, Y: 0,61, one pixel: 48 0
Blob 4 of 119, id: 2, sources: 4, size: 84x65, npix 2782, brick X: 240,324, Y: 0,65, one pixel: 265 0
Blob 5 of 119, id: 52, sources: 1, size: 86x45, npix 2572, brick X: 0,86, Y: 298,343, one pixel: 15 298
Blob 2 finished, total: Wall: 1.23 s, CPU: 1.23 s, VmPeak: 963 MB, VmSize: 729 MB, VmRSS: 154 MB, VmData: 192 MB, maxrss: 0.157504 MB
Blob 6 of 119, id: 32, sources: 2, size: 53x78, npix 2558, brick X: 652,705, Y: 158,236, one pixel: 682 158
Measuring 3 iterative sources
Blob 5 finished, total: Wall: 4.44 s, CPU: 3.06 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 156 MB, VmData: 192 MB, maxrss: 0.159984 MB
Measuring 3 iterative sources
Measuring 6 iterative sources
Blob 7 of 119, id: 79, sources: 2, size: 65x59, npix 2286, brick X: 440,505, Y: 507,566, one pixel: 473 507
Blob 6 finished, total: Wall: 3.69 s, CPU: 3.69 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 156 MB, VmData: 192 MB, maxrss: 0.159984 MB
Blob 4 finished, total: Wall: 14.59 s, CPU: 10.15 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 3 finished, total: Wall: 14.79 s, CPU: 10.37 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 193 MB, maxrss: 0.161484 MB
Blob 8 of 119, id: 35, sources: 1, size: 65x88, npix 2180, brick X: 302,367, Y: 178,266, one pixel: 353 178
Blob 9 of 119, id: 99, sources: 3, size: 68x50, npix 1929, brick X: 15,83, Y: 591,641, one pixel: 43 591
Blob 10 of 119, id: 39, sources: 1, size: 43x55, npix 1821, brick X: 0,43, Y: 201,256, one pixel: 19 201
Measuring 2 iterative sources
Measuring 4 iterative sources
Blob 7 finished, total: Wall: 2.51 s, CPU: 2.35 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160692 MB
Blob 11 of 119, id: 85, sources: 3, size: 37x70, npix 1549, brick X: 668,705, Y: 525,595, one pixel: 682 525
Measuring 2 iterative sources
Blob 9 finished, total: Wall: 2.82 s, CPU: 2.82 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 193 MB, maxrss: 0.161484 MB
Blob 12 of 119, id: 92, sources: 2, size: 64x45, npix 1544, brick X: 229,293, Y: 564,609, one pixel: 245 564
Blob 10 finished, total: Wall: 3.52 s, CPU: 3.52 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160692 MB
Blob 13 of 119, id: 31, sources: 3, size: 41x68, npix 1532, brick X: 560,601, Y: 155,223, one pixel: 589 155
Blob 8 finished, total: Wall: 6.22 s, CPU: 6.21 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 14 of 119, id: 71, sources: 3, size: 50x60, npix 1516, brick X: 397,447, Y: 475,535, one pixel: 420 475
Blob 11 finished, total: Wall: 4.64 s, CPU: 4.64 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 193 MB, maxrss: 0.161484 MB
Blob 15 of 119, id: 57, sources: 4, size: 47x49, npix 1423, brick X: 6,53, Y: 361,410, one pixel: 22 361
Blob 13 finished, total: Wall: 1.89 s, CPU: 1.89 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 16 of 119, id: 22, sources: 2, size: 58x36, npix 1402, brick X: 229,287, Y: 108,144, one pixel: 247 108
Blob 12 finished, total: Wall: 2.16 s, CPU: 2.16 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160692 MB
Blob 17 of 119, id: 59, sources: 2, size: 58x39, npix 1186, brick X: 514,572, Y: 390,429, one pixel: 560 390
Measuring 2 iterative sources
Measuring 1 iterative sources
Measuring 2 iterative sources
Blob 14 finished, total: Wall: 2.63 s, CPU: 2.62 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 193 MB, maxrss: 0.161484 MB
Blob 18 of 119, id: 111, sources: 1, size: 40x43, npix 1117, brick X: 569,609, Y: 648,691, one pixel: 592 648
Blob 15 finished, total: Wall: 2.00 s, CPU: 2.00 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 19 of 119, id: 6, sources: 2, size: 46x34, npix 1094, brick X: 144,190, Y: 7,41, one pixel: 156 7
Blob 16 finished, total: Wall: 3.03 s, CPU: 3.03 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160692 MB
Blob 20 of 119, id: 33, sources: 2, size: 30x52, npix 977, brick X: 63,93, Y: 171,223, one pixel: 75 171
Blob 18 finished, total: Wall: 1.53 s, CPU: 1.53 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 21 of 119, id: 5, sources: 1, size: 34x39, npix 914, brick X: 483,517, Y: 5,44, one pixel: 497 5
Blob 20 finished, total: Wall: 1.27 s, CPU: 1.27 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 22 of 119, id: 102, sources: 2, size: 48x34, npix 883, brick X: 159,207, Y: 600,634, one pixel: 195 600
Blob 19 finished, total: Wall: 2.16 s, CPU: 2.16 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160692 MB
Blob 23 of 119, id: 24, sources: 2, size: 42x31, npix 857, brick X: 608,650, Y: 130,161, one pixel: 636 130
Blob 17 finished, total: Wall: 3.52 s, CPU: 3.51 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 193 MB, maxrss: 0.161484 MB
Blob 24 of 119, id: 115, sources: 2, size: 50x29, npix 812, brick X: 234,284, Y: 668,697, one pixel: 245 668
Measuring 1 iterative sources
Blob 21 finished, total: Wall: 0.90 s, CPU: 0.90 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 25 of 119, id: 28, sources: 2, size: 38x40, npix 791, brick X: 14,52, Y: 145,185, one pixel: 37 145
Measuring 1 iterative sources
Blob 23 finished, total: Wall: 1.51 s, CPU: 1.51 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.16174 MB
Blob 26 of 119, id: 42, sources: 3, size: 50x28, npix 790, brick X: 46,96, Y: 227,255, one pixel: 70 227
Blob 24 finished, total: Wall: 1.94 s, CPU: 1.94 s, VmPeak: 977 MB, VmSize: 699 MB, VmRSS: 126 MB, VmData: 162 MB, maxrss: 0.157044 MB
Blob 27 of 119, id: 1, sources: 3, size: 39x34, npix 785, brick X: 219,258, Y: 0,34, one pixel: 224 0
Blob 22 finished, total: Wall: 2.40 s, CPU: 2.40 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 28 of 119, id: 68, sources: 2, size: 39x38, npix 768, brick X: 0,39, Y: 460,498, one pixel: 27 460
Blob 25 finished, total: Wall: 1.47 s, CPU: 1.47 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.16174 MB
Blob 29 of 119, id: 103, sources: 1, size: 32x33, npix 753, brick X: 431,463, Y: 606,639, one pixel: 441 606
Blob 26 finished, total: Wall: 2.00 s, CPU: 2.00 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 30 of 119, id: 76, sources: 2, size: 30x44, npix 718, brick X: 222,252, Y: 497,541, one pixel: 232 497
Blob 28 finished, total: Wall: 1.17 s, CPU: 1.17 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.16174 MB
Blob 31 of 119, id: 38, sources: 1, size: 33x32, npix 699, brick X: 119,152, Y: 192,224, one pixel: 134 192
Blob 27 finished, total: Wall: 2.13 s, CPU: 2.13 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 32 of 119, id: 114, sources: 1, size: 32x32, npix 690, brick X: 624,656, Y: 663,695, one pixel: 636 663
Blob 31 finished, total: Wall: 0.83 s, CPU: 0.83 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 33 of 119, id: 3, sources: 2, size: 42x30, npix 671, brick X: 358,400, Y: 0,30, one pixel: 372 0
Blob 30 finished, total: Wall: 1.00 s, CPU: 1.00 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.16174 MB
Blob 34 of 119, id: 41, sources: 2, size: 31x37, npix 662, brick X: 643,674, Y: 219,256, one pixel: 654 219
Blob 33 finished, total: Wall: 0.89 s, CPU: 0.89 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 35 of 119, id: 98, sources: 1, size: 31x31, npix 637, brick X: 625,656, Y: 584,615, one pixel: 640 584
Blob 29 finished, total: Wall: 2.45 s, CPU: 2.45 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 36 of 119, id: 116, sources: 2, size: 37x24, npix 613, brick X: 514,551, Y: 669,693, one pixel: 524 669
Blob 32 finished, total: Wall: 1.80 s, CPU: 1.80 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 37 of 119, id: 21, sources: 1, size: 29x33, npix 596, brick X: 645,674, Y: 104,137, one pixel: 656 104
Blob 35 finished, total: Wall: 0.82 s, CPU: 0.82 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 38 of 119, id: 10, sources: 1, size: 28x30, npix 568, brick X: 568,596, Y: 42,72, one pixel: 581 42
Blob 34 finished, total: Wall: 1.88 s, CPU: 1.88 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 39 of 119, id: 117, sources: 1, size: 29x27, npix 550, brick X: 91,120, Y: 678,705, one pixel: 105 678
Blob 36 finished, total: Wall: 1.63 s, CPU: 1.63 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 40 of 119, id: 108, sources: 1, size: 28x28, npix 548, brick X: 653,681, Y: 638,666, one pixel: 662 638
Blob 38 finished, total: Wall: 1.25 s, CPU: 1.25 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 41 of 119, id: 107, sources: 1, size: 33x31, npix 546, brick X: 477,510, Y: 634,665, one pixel: 497 634
Blob 39 finished, total: Wall: 0.77 s, CPU: 0.77 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 42 of 119, id: 113, sources: 1, size: 32x27, npix 543, brick X: 352,384, Y: 651,678, one pixel: 373 651
Blob 37 finished, total: Wall: 2.17 s, CPU: 2.17 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 43 of 119, id: 12, sources: 1, size: 26x30, npix 515, brick X: 259,285, Y: 66,96, one pixel: 272 66
Measuring 1 iterative sources
Measuring 1 iterative sources
Blob 40 finished, total: Wall: 1.32 s, CPU: 1.32 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 44 of 119, id: 15, sources: 1, size: 28x29, npix 502, brick X: 406,434, Y: 78,107, one pixel: 418 78
Blob 42 finished, total: Wall: 1.09 s, CPU: 1.09 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 45 of 119, id: 19, sources: 1, size: 29x28, npix 500, brick X: 61,90, Y: 95,123, one pixel: 75 95
Blob 41 finished, total: Wall: 1.38 s, CPU: 1.38 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 46 of 119, id: 9, sources: 2, size: 38x24, npix 498, brick X: 132,170, Y: 41,65, one pixel: 141 41
Blob 45 finished, total: Wall: 0.79 s, CPU: 0.79 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 47 of 119, id: 55, sources: 1, size: 21x41, npix 486, brick X: 83,104, Y: 336,377, one pixel: 92 336
Blob 43 finished, total: Wall: 1.06 s, CPU: 1.06 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 48 of 119, id: 67, sources: 2, size: 38x24, npix 484, brick X: 478,516, Y: 454,478, one pixel: 505 454
Measuring 1 iterative sources
Blob 44 finished, total: Wall: 1.35 s, CPU: 1.35 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 49 of 119, id: 51, sources: 1, size: 28x31, npix 482, brick X: 148,176, Y: 286,317, one pixel: 166 286
Blob 47 finished, total: Wall: 0.45 s, CPU: 0.45 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 50 of 119, id: 73, sources: 1, size: 29x29, npix 481, brick X: 528,557, Y: 481,510, one pixel: 540 481
Blob 46 finished, total: Wall: 0.52 s, CPU: 0.52 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 51 of 119, id: 64, sources: 1, size: 26x28, npix 476, brick X: 280,306, Y: 437,465, one pixel: 291 437
Measuring 1 iterative sources
Blob 49 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 52 of 119, id: 95, sources: 1, size: 27x27, npix 470, brick X: 117,144, Y: 571,598, one pixel: 128 571
Blob 48 finished, total: Wall: 0.60 s, CPU: 0.60 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 50 finished, total: Wall: 0.57 s, CPU: 0.57 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 53 of 119, id: 58, sources: 1, size: 31x26, npix 467, brick X: 65,96, Y: 379,405, one pixel: 87 379
Blob 54 of 119, id: 78, sources: 2, size: 22x35, npix 464, brick X: 573,595, Y: 506,541, one pixel: 583 506
Measuring 1 iterative sources
Blob 52 finished, total: Wall: 0.66 s, CPU: 0.66 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 53 finished, total: Wall: 0.67 s, CPU: 0.66 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 51 finished, total: Wall: 1.96 s, CPU: 1.96 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.25435, ee1=-0.0985785, ee2=0.09349
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.25164, ee1=-0.0278281, ee2=0.147521
Measuring 2 iterative sources
Blob 55 of 119, id: 44, sources: 1, size: 28x26, npix 454, brick X: 113,141, Y: 235,261, one pixel: 125 235
Blob 56 of 119, id: 82, sources: 1, size: 27x26, npix 449, brick X: 334,361, Y: 511,537, one pixel: 343 511
Blob 57 of 119, id: 36, sources: 1, size: 26x26, npix 429, brick X: 415,441, Y: 181,207, one pixel: 426 181
Blob 55 finished, total: Wall: 0.83 s, CPU: 0.83 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 58 of 119, id: 86, sources: 1, size: 26x24, npix 422, brick X: 244,270, Y: 529,553, one pixel: 254 529
Blob 54 finished, total: Wall: 1.09 s, CPU: 1.09 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 59 of 119, id: 62, sources: 1, size: 26x25, npix 416, brick X: 155,181, Y: 423,448, one pixel: 166 423
Blob 57 finished, total: Wall: 0.67 s, CPU: 0.67 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 56 finished, total: Wall: 1.60 s, CPU: 1.60 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 58 finished, total: Wall: 0.55 s, CPU: 0.55 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 60 of 119, id: 83, sources: 1, size: 28x25, npix 414, brick X: 508,536, Y: 514,539, one pixel: 524 514
Blob 61 of 119, id: 109, sources: 1, size: 24x26, npix 412, brick X: 123,147, Y: 648,674, one pixel: 134 648
Blob 62 of 119, id: 70, sources: 1, size: 27x25, npix 409, brick X: 95,122, Y: 462,487, one pixel: 104 462
Measuring 1 iterative sources
Blob 59 finished, total: Wall: 0.82 s, CPU: 0.82 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 63 of 119, id: 66, sources: 1, size: 25x26, npix 399, brick X: 69,94, Y: 447,473, one pixel: 81 447
Blob 60 finished, total: Wall: 1.01 s, CPU: 1.01 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 64 of 119, id: 4, sources: 1, size: 33x16, npix 397, brick X: 591,624, Y: 0,16, one pixel: 593 0
Blob 62 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 65 of 119, id: 14, sources: 1, size: 24x26, npix 375, brick X: 330,354, Y: 72,98, one pixel: 339 72
Blob 61 finished, total: Wall: 1.31 s, CPU: 1.31 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 63 finished, total: Wall: 0.31 s, CPU: 0.31 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 1 finished, total: Wall: 51.77 s, CPU: 48.64 s, VmPeak: 967 MB, VmSize: 759 MB, VmRSS: 195 MB, VmData: 222 MB, maxrss: 0.201428 MB
Blob 64 finished, total: Wall: 1.04 s, CPU: 1.04 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 66 of 119, id: 27, sources: 1, size: 24x25, npix 362, brick X: 325,349, Y: 142,167, one pixel: 335 142
Blob 67 of 119, id: 46, sources: 1, size: 22x25, npix 359, brick X: 102,124, Y: 261,286, one pixel: 113 261
Blob 68 of 119, id: 84, sources: 1, size: 24x24, npix 342, brick X: 34,58, Y: 520,544, one pixel: 46 520
Blob 69 of 119, id: 69, sources: 1, size: 24x23, npix 341, brick X: 235,259, Y: 461,484, one pixel: 246 461
Blob 68 finished, total: Wall: 0.28 s, CPU: 0.27 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 70 of 119, id: 97, sources: 1, size: 24x23, npix 338, brick X: 410,434, Y: 576,599, one pixel: 420 576
Blob 69 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 71 of 119, id: 54, sources: 1, size: 22x25, npix 335, brick X: 672,694, Y: 301,326, one pixel: 682 301
Blob 66 finished, total: Wall: 0.79 s, CPU: 0.71 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 72 of 119, id: 49, sources: 1, size: 22x24, npix 322, brick X: 643,665, Y: 263,287, one pixel: 653 263
Blob 65 finished, total: Wall: 0.90 s, CPU: 0.89 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 73 of 119, id: 77, sources: 1, size: 22x22, npix 321, brick X: 179,201, Y: 504,526, one pixel: 188 504
Blob 67 finished, total: Wall: 1.05 s, CPU: 1.05 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 74 of 119, id: 74, sources: 1, size: 19x26, npix 317, brick X: 686,705, Y: 490,516, one pixel: 697 490
Blob 71 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 72 finished, total: Wall: 0.25 s, CPU: 0.25 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 73 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 70 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 75 of 119, id: 88, sources: 1, size: 22x23, npix 314, brick X: 611,633, Y: 536,559, one pixel: 621 536
Blob 76 of 119, id: 112, sources: 1, size: 22x22, npix 303, brick X: 160,182, Y: 649,671, one pixel: 169 649
Blob 77 of 119, id: 105, sources: 1, size: 22x23, npix 294, brick X: 554,576, Y: 627,650, one pixel: 564 627
Blob 78 of 119, id: 37, sources: 1, size: 21x22, npix 291, brick X: 263,284, Y: 189,211, one pixel: 272 189
Blob 75 finished, total: Wall: 0.54 s, CPU: 0.34 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 79 of 119, id: 61, sources: 1, size: 22x21, npix 291, brick X: 84,106, Y: 415,436, one pixel: 92 415
Blob 77 finished, total: Wall: 0.53 s, CPU: 0.33 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 80 of 119, id: 118, sources: 1, size: 23x22, npix 291, brick X: 174,197, Y: 678,700, one pixel: 184 678
Blob 74 finished, total: Wall: 0.64 s, CPU: 0.64 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 81 of 119, id: 34, sources: 1, size: 22x22, npix 283, brick X: 528,550, Y: 173,195, one pixel: 538 173
Blob 76 finished, total: Wall: 0.85 s, CPU: 0.71 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 82 of 119, id: 75, sources: 1, size: 22x22, npix 282, brick X: 259,281, Y: 493,515, one pixel: 268 493
Blob 78 finished, total: Wall: 0.34 s, CPU: 0.34 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 83 of 119, id: 18, sources: 1, size: 21x21, npix 270, brick X: 523,544, Y: 92,113, one pixel: 532 92
Blob 80 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 84 of 119, id: 63, sources: 1, size: 21x21, npix 270, brick X: 672,693, Y: 436,457, one pixel: 681 436
Blob 79 finished, total: Wall: 0.39 s, CPU: 0.39 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 85 of 119, id: 80, sources: 1, size: 21x21, npix 270, brick X: 629,650, Y: 507,528, one pixel: 638 507
Blob 81 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 83 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 86 of 119, id: 45, sources: 1, size: 21x22, npix 262, brick X: 134,155, Y: 258,280, one pixel: 144 258
Blob 87 of 119, id: 8, sources: 1, size: 21x21, npix 260, brick X: 301,322, Y: 38,59, one pixel: 310 38
Blob 84 finished, total: Wall: 0.35 s, CPU: 0.35 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 88 of 119, id: 89, sources: 1, size: 21x21, npix 259, brick X: 3,24, Y: 540,561, one pixel: 13 540
Blob 82 finished, total: Wall: 0.60 s, CPU: 0.60 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 89 of 119, id: 81, sources: 1, size: 20x22, npix 250, brick X: 113,133, Y: 510,532, one pixel: 122 510
Blob 85 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 90 of 119, id: 87, sources: 1, size: 21x21, npix 250, brick X: 56,77, Y: 536,557, one pixel: 65 536
Blob 87 finished, total: Wall: 0.31 s, CPU: 0.31 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 88 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 89 finished, total: Wall: 0.29 s, CPU: 0.29 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 86 finished, total: Wall: 0.99 s, CPU: 0.99 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 91 of 119, id: 13, sources: 1, size: 21x21, npix 249, brick X: 430,451, Y: 71,92, one pixel: 439 71
Blob 92 of 119, id: 65, sources: 1, size: 20x21, npix 240, brick X: 128,148, Y: 442,463, one pixel: 137 442
Blob 93 of 119, id: 90, sources: 1, size: 20x21, npix 239, brick X: 412,432, Y: 544,565, one pixel: 421 544
Blob 94 of 119, id: 47, sources: 1, size: 20x20, npix 238, brick X: 23,43, Y: 263,283, one pixel: 31 263
Blob 92 finished, total: Wall: 0.31 s, CPU: 0.31 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 95 of 119, id: 96, sources: 1, size: 20x21, npix 229, brick X: 362,382, Y: 574,595, one pixel: 371 574
Blob 91 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 96 of 119, id: 30, sources: 1, size: 20x19, npix 227, brick X: 306,326, Y: 151,170, one pixel: 315 151
Blob 93 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 97 of 119, id: 106, sources: 1, size: 19x20, npix 227, brick X: 9,28, Y: 633,653, one pixel: 17 633
Blob 94 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 98 of 119, id: 91, sources: 1, size: 20x20, npix 220, brick X: 41,61, Y: 557,577, one pixel: 50 557
Blob 96 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 99 of 119, id: 50, sources: 1, size: 20x20, npix 210, brick X: 677,697, Y: 275,295, one pixel: 687 275
Blob 98 finished, total: Wall: 0.06 s, CPU: 0.06 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 100 of 119, id: 20, sources: 1, size: 20x19, npix 209, brick X: 312,332, Y: 95,114, one pixel: 320 95
Blob 90 finished, total: Wall: 0.82 s, CPU: 0.82 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 101 of 119, id: 53, sources: 1, size: 20x19, npix 209, brick X: 520,540, Y: 301,320, one pixel: 529 301
Blob 97 finished, total: Wall: 0.22 s, CPU: 0.22 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 102 of 119, id: 93, sources: 1, size: 19x20, npix 200, brick X: 606,625, Y: 566,586, one pixel: 615 566
Blob 100 finished, total: Wall: 0.25 s, CPU: 0.25 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 103 of 119, id: 110, sources: 1, size: 20x19, npix 199, brick X: 327,347, Y: 648,667, one pixel: 337 648
Blob 101 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 104 of 119, id: 56, sources: 1, size: 19x18, npix 198, brick X: 676,695, Y: 340,358, one pixel: 684 340
Blob 95 finished, total: Wall: 0.77 s, CPU: 0.77 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 105 of 119, id: 100, sources: 1, size: 18x19, npix 198, brick X: 376,394, Y: 591,610, one pixel: 384 591
Blob 104 finished, total: Wall: 0.06 s, CPU: 0.06 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 106 of 119, id: 26, sources: 1, size: 19x19, npix 190, brick X: 398,417, Y: 134,153, one pixel: 407 134
Blob 99 finished, total: Wall: 0.62 s, CPU: 0.62 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 107 of 119, id: 72, sources: 1, size: 19x19, npix 190, brick X: 503,522, Y: 478,497, one pixel: 512 478
Blob 102 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 108 of 119, id: 104, sources: 1, size: 18x19, npix 189, brick X: 518,536, Y: 608,627, one pixel: 526 608
Blob 105 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 106 finished, total: Wall: 0.29 s, CPU: 0.29 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Blob 109 of 119, id: 16, sources: 1, size: 18x19, npix 180, brick X: 236,254, Y: 79,98, one pixel: 245 79
Blob 110 of 119, id: 17, sources: 1, size: 18x18, npix 180, brick X: 209,227, Y: 89,107, one pixel: 217 89
Blob 108 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 111 of 119, id: 48, sources: 1, size: 18x18, npix 180, brick X: 515,533, Y: 263,281, one pixel: 523 263
Blob 107 finished, total: Wall: 0.31 s, CPU: 0.31 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 112 of 119, id: 11, sources: 1, size: 18x18, npix 171, brick X: 173,191, Y: 49,67, one pixel: 181 49
Blob 111 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 113 of 119, id: 23, sources: 1, size: 18x18, npix 171, brick X: 110,128, Y: 120,138, one pixel: 118 120
Blob 103 finished, total: Wall: 0.79 s, CPU: 0.79 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 114 of 119, id: 60, sources: 1, size: 18x18, npix 171, brick X: 13,31, Y: 407,425, one pixel: 21 407
Blob 113 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 115 of 119, id: 25, sources: 1, size: 16x18, npix 167, brick X: 0,16, Y: 133,151, one pixel: 6 133
Blob 110 finished, total: Wall: 0.35 s, CPU: 0.35 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 116 of 119, id: 29, sources: 1, size: 17x18, npix 162, brick X: 424,441, Y: 150,168, one pixel: 432 150
Blob 112 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 117 of 119, id: 94, sources: 1, size: 18x17, npix 162, brick X: 196,214, Y: 570,587, one pixel: 204 570
Blob 114 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 118 of 119, id: 7, sources: 1, size: 17x17, npix 145, brick X: 679,696, Y: 17,34, one pixel: 687 17
Blob 116 finished, total: Wall: 0.07 s, CPU: 0.07 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 119 of 119, id: 101, sources: 1, size: 17x17, npix 145, brick X: 458,475, Y: 594,611, one pixel: 466 594
Blob 117 finished, total: Wall: 0.06 s, CPU: 0.06 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 119 finished, total: Wall: 0.05 s, CPU: 0.05 s, VmPeak: 969 MB, VmSize: 731 MB, VmRSS: 158 MB, VmData: 194 MB, maxrss: 0.162004 MB
Blob 115 finished, total: Wall: 0.39 s, CPU: 0.39 s, VmPeak: 977 MB, VmSize: 700 MB, VmRSS: 127 MB, VmData: 163 MB, maxrss: 0.157044 MB
Blob 118 finished, total: Wall: 0.29 s, CPU: 0.29 s, VmPeak: 967 MB, VmSize: 718 MB, VmRSS: 154 MB, VmData: 181 MB, maxrss: 0.201428 MB
Blob 109 finished, total: Wall: 1.36 s, CPU: 1.36 s, VmPeak: 963 MB, VmSize: 730 MB, VmRSS: 157 MB, VmData: 193 MB, maxrss: 0.160952 MB
Old catalog: 179
New catalog: 166
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/blobs-custom-167912p12123.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/metrics/cus/all-models-custom-167912p12123.fits
Stage fitblobs : Wall: 71.28 s, CPU: 10.06 s, VmPeak: 1161 MB, VmSize: 1038 MB, VmRSS: 292 MB, VmData: 320 MB, maxrss: 0.454636 MB
Stage fitblobs finished: 2020-06-05T05:19:37.322568
Resources for stage fitblobs : 190.724 s worker CPU, 243.540 s worker Wall, Wall: 71.277 s, Cores in use: 2.82, Total efficiency (on 4 cores): 70.4 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-05T05:19:37.324535
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/tmp-legacysurvey-custom-167912p12123-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/tmp-legacysurvey-custom-167912p12123-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/tmp-legacysurvey-custom-167912p12123-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/tmp-legacysurvey-custom-167912p12123-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-resid.jpg
Stage coadds : Wall: 7.28 s, CPU: 4.94 s, VmPeak: 1161 MB, VmSize: 1107 MB, VmRSS: 365 MB, VmData: 395 MB, maxrss: 0.454636 MB
Stage coadds finished: 2020-06-05T05:19:44.606168
Resources for stage coadds : 6.551 s worker CPU, 6.844 s worker Wall, Wall: 7.285 s, Cores in use: 1.58, Total efficiency (on 4 cores): 39.4 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-05T05:19:44.610327
Cut to 1 unWISE tiles
Reading WISE tile 1684p121 band 1
Reading WISE tile 1684p121 band 2
Reading WISE tile 1684p121 band 3
Reading WISE tile 1684p121 band 4
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.66 s, CPU: 0.75 s, VmPeak: 1009 MB, VmSize: 1009 MB, VmRSS: 161 MB, VmData: 196 MB, maxrss: 0.223172 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.65 s, CPU: 0.71 s, VmPeak: 1066 MB, VmSize: 1002 MB, VmRSS: 155 MB, VmData: 189 MB, maxrss: 0.216756 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 2.02 s, CPU: 3.03 s, VmPeak: 1008 MB, VmSize: 1008 MB, VmRSS: 163 MB, VmData: 195 MB, maxrss: 0.20222 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/168/1684p121/unwise-1684p121-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.35 s, CPU: 0.96 s, VmPeak: 1004 MB, VmSize: 1004 MB, VmRSS: 157 MB, VmData: 191 MB, maxrss: 0.218776 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/tmp-legacysurvey-custom-167912p12123-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/tmp-legacysurvey-custom-167912p12123-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 7.68 s, CPU: 4.06 s, VmPeak: 1161 MB, VmSize: 1032 MB, VmRSS: 291 MB, VmData: 321 MB, maxrss: 0.454636 MB
Stage wise_forced finished: 2020-06-05T05:19:52.286506
Resources for stage wise_forced : 9.793 s worker CPU, 11.821 s worker Wall, Wall: 7.678 s, Cores in use: 1.81, Total efficiency (on 4 cores): 45.1 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-05T05:19:52.288882
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/coadd/cus/custom-167912p12123/legacysurvey-custom-167912p12123-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/tractor-i/cus/tractor-custom-167912p12123.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/tractor/cus/tractor-custom-167912p12123.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/167/IC2628_GROUP/tractor/cus/brick-custom-167912p12123.sha256sum
Stage writecat : Wall: 21.41 s, CPU: 4.59 s, VmPeak: 1180 MB, VmSize: 1052 MB, VmRSS: 309 MB, VmData: 337 MB, maxrss: 0.454636 MB
Stage writecat finished: 2020-06-05T05:20:13.695034
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 21.407 s, Cores in use: 0.21, Total efficiency (on 4 cores): 5.4 %
Stage writecat finished
All done: 224.467 s worker CPU, 727.253 s worker Wall, Wall: 317.037 s, Cores in use: 0.90, Total efficiency (on 4 cores): 22.4 %
Finished galaxy IC2628_GROUP in 5.366 minutes.