Started working on galaxy PGC1563680 at Thu Jun  4 20:39:32 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 163.6677885 18.5777867 --width 247 --height 247 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-06-04T20:39:40.736839
legacypipe git version: DR9.5.2
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '163.6677885', '18.5777867', '--width', '247', '--height', '247', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 163.6677885 18.5777867 --width 247 --height 247 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 163.6677885 18.5777867
Starting process 45131 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 698 MB, VmSize: 698 MB, VmRSS: 56 MB, VmData: 129 MB, maxrss: 0.05738 MB
Starting process 45135 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 698 MB, VmSize: 698 MB, VmRSS: 56 MB, VmData: 129 MB, maxrss: 0.05738 MB
Starting process 45138 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 698 MB, VmSize: 698 MB, VmRSS: 56 MB, VmData: 129 MB, maxrss: 0.057384 MB
Starting process 45143 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 698 MB, VmSize: 698 MB, VmRSS: 56 MB, VmData: 129 MB, maxrss: 0.057384 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-06-04T20:39:41.137426
Keeping 12 CCDs:
   c4d_150410_022740_ooi_z_ls9.fits.fz c4d_150410_022740_ooi_z_ls9-S29 z exptime 112.0 propid 2014B-0404 seeing 1.14 MJD 57122.101 object DECaLS_3294_z
   c4d_170303_035101_ooi_r_ls9.fits.fz c4d_170303_035101_ooi_r_ls9-S29 r exptime 82.0 propid 2014B-0404 seeing 1.10 MJD 57815.159 object DECaLS_3294_r
   c4d_170303_035244_ooi_g_ls9.fits.fz c4d_170303_035244_ooi_g_ls9-S29 g exptime 71.0 propid 2014B-0404 seeing 1.24 MJD 57815.160 object DECaLS_3294_g
   c4d_180219_055508_ooi_g_ls9.fits.fz c4d_180219_055508_ooi_g_ls9-N28 g exptime 115.0 propid 2014B-0404 seeing 0.88 MJD 58168.247 object DECaLS_19149_g
   c4d_180221_062855_ooi_r_ls9.fits.fz c4d_180221_062855_ooi_r_ls9-N28 r exptime 48.0 propid 2014B-0404 seeing 1.04 MJD 58170.270 object DECaLS_19149_r
   c4d_170111_063416_ooi_z_ls9.fits.fz c4d_170111_063416_ooi_z_ls9-N28 z exptime 143.0 propid 2014B-0404 seeing 1.16 MJD 57764.272 object DECaLS_19149_z
   c4d_180219_062814_ooi_g_ls9.fits.fz c4d_180219_062814_ooi_g_ls9-S29 g exptime 108.0 propid 2014B-0404 seeing 1.32 MJD 58168.270 object DECaLS_19166_g
   c4d_160228_051501_ooi_z_ls9.fits.fz c4d_160228_051501_ooi_z_ls9-S29 z exptime 80.0 propid 2014B-0404 seeing 1.04 MJD 57446.217 object DECaLS_19166_z
   c4d_180315_032519_ooi_r_ls9.fits.fz c4d_180315_032519_ooi_r_ls9-S29 r exptime 84.0 propid 2014B-0404 seeing 1.36 MJD 58192.143 object DECaLS_19166_r
   c4d_180316_035613_ooi_r_ls9.fits.fz c4d_180316_035613_ooi_r_ls9-N23 r exptime 53.0 propid 2014B-0404 seeing 1.09 MJD 58193.164 object DECaLS_35021_r
   c4d_180316_034212_ooi_g_ls9.fits.fz c4d_180316_034212_ooi_g_ls9-N23 g exptime 104.0 propid 2014B-0404 seeing 1.08 MJD 58193.154 object DECaLS_35021_g
   c4d_170306_024336_ooi_z_ls9.fits.fz c4d_170306_024336_ooi_z_ls9-N23 z exptime 90.0 propid 2014B-0404 seeing 1.00 MJD 57818.112 object DECaLS_35021_z
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950e32f-g-20170303T035043_oi[S29,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9570153-z-20150410T022457_oi[S29,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952b7a4-g-20180219T055508_oi[N28,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950e697-r-20170303T034850_oi[S29,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952ba12-r-20180221T062855_oi[N28,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953c314-z-20170111T063102_oi[N28,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952b7a4-g-20180219T062814_oi[S29,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952cf52-r-20180315T032519_oi[S29,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952d43a-r-20180316T035613_oi[N23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952d735-g-20180316T034212_oi[N23,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95503fa-z-20160228T051249_oi[S29,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950dd28-z-20170306T024118_oi[N23,noinherit][2046,4094][real]: DECaLS_
Stage tims : Wall: 310.70 s, CPU: 1.18 s, VmPeak: 996 MB, VmSize: 996 MB, VmRSS: 174 MB, VmData: 208 MB, maxrss: 0.454584 MB
Stage tims finished: 2020-06-04T20:44:51.842057
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952b7a4-g-20180219T055508_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952b7a4-g-20180219T062814_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952d735-g-20180316T034212_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950e32f-g-20170303T035043_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952ba12-r-20180221T062855_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952cf52-r-20180315T032519_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950e697-r-20170303T034850_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95503fa-z-20160228T051249_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9570153-z-20150410T022457_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953c314-z-20170111T063102_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_952d43a-r-20180316T035613_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950dd28-z-20170306T024118_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Resources for stage tims : 4.898 s worker CPU, 1015.357 s worker Wall, Wall: 310.706 s, Cores in use: 0.02, Total efficiency (on 4 cores): 0.5 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-06-04T20:44:51.887233
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-04218.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-04090.fits
Matched 10 large galaxies to Gaia stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/reference-custom-163667p18577.fits
Stage refs : Wall: 9.98 s, CPU: 9.31 s, VmPeak: 1026 MB, VmSize: 1006 MB, VmRSS: 186 MB, VmData: 218 MB, maxrss: 0.454584 MB
Stage refs finished: 2020-06-04T20:45:01.871474
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 9.985 s, Cores in use: 0.93, Total efficiency (on 4 cores): 23.3 %
Stage refs finished
Running stage outliers
Running stage outliers at 2020-06-04T20:45:01.872854
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/outlier-mask-custom-163667p18577.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/outliers-pre-custom-163667p18577.jpg
Image c4d_180219_062814_ooi_g_ls9-S29 g : masking 374 positive outlier pixels and 0 negative outlier pixels
Image c4d_160228_051501_ooi_z_ls9-S29 z : masking 255 positive outlier pixels and 0 negative outlier pixels
Image c4d_150410_022740_ooi_z_ls9-S29 z : masking 563 positive outlier pixels and 0 negative outlier pixels
Image c4d_170306_024336_ooi_z_ls9-N23 z : masking 280 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/outlier-mask-custom-163667p18577.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/outliers-post-custom-163667p18577.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/outliers-masked-pos-custom-163667p18577.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/outliers-masked-neg-custom-163667p18577.jpg
Stage outliers : Wall: 12.31 s, CPU: 10.70 s, VmPeak: 1026 MB, VmSize: 1016 MB, VmRSS: 196 MB, VmData: 227 MB, maxrss: 0.454584 MB
Stage outliers finished: 2020-06-04T20:45:14.186042
Resources for stage outliers : 0.711 s worker CPU, 0.712 s worker Wall, Wall: 12.314 s, Cores in use: 0.93, Total efficiency (on 4 cores): 23.2 %
Stage outliers finished
Running stage halos
Running stage halos at 2020-06-04T20:45:14.187612
Stage halos : Wall: 14.21 s, CPU: 8.26 s, VmPeak: 1026 MB, VmSize: 1016 MB, VmRSS: 197 MB, VmData: 227 MB, maxrss: 0.454584 MB
Stage halos finished: 2020-06-04T20:45:28.393349
Resources for stage halos : 0.211 s worker CPU, 13.669 s worker Wall, Wall: 14.207 s, Cores in use: 0.60, Total efficiency (on 4 cores): 14.9 %
Stage halos finished
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-06-04T20:45:28.395343
PSF sigmas in coadd pixels: ['2.02', '1.42', '2.14', '1.76']
Using average PSF sigma 1.8351516416484315
Gaussian PSF norm 0.15371742877906566 vs pixelized 0.15313965
average tim pixel scale / coadd scale: 1.0011825927814026
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.78', '1.69', '2.20', '1.77']
Using average PSF sigma 1.8584686260823933
Gaussian PSF norm 0.15178883722590847 vs pixelized 0.14288752
average tim pixel scale / coadd scale: 1.0011941398927615
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.85', '1.89', '1.69', '1.63']
Using average PSF sigma 1.7649918347143847
Gaussian PSF norm 0.15982781689158773 vs pixelized 0.1489067
average tim pixel scale / coadd scale: 1.0011704958075984
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.16 s, VmPeak: 1026 MB, VmSize: 1016 MB, VmRSS: 199 MB, VmData: 228 MB, maxrss: 0.454584 MB
Stage fit_on_coadds finished: 2020-06-04T20:45:28.579988
Resources for stage fit_on_coadds : 0.175 s worker CPU, 0.175 s worker Wall, Wall: 0.186 s, Cores in use: 1.81, Total efficiency (on 4 cores): 45.3 %
Stage fit_on_coadds finished
Running stage srcs
Running stage srcs at 2020-06-04T20:45:28.582475
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 64 potential peaks
Of 64 potential peaks: 32 in veto map, 10 cut by saddle test, 0 cut by aper test, 22 kept
SED z : 22 new peaks
Found 93 potential peaks
Of 93 potential peaks: 78 in veto map, 8 cut by saddle test, 0 cut by aper test, 7 kept
SED r : 7 new peaks
Found 69 potential peaks
Of 69 potential peaks: 59 in veto map, 6 cut by saddle test, 0 cut by aper test, 4 kept
SED g : 4 new peaks
Found 102 potential peaks
Of 102 potential peaks: 91 in veto map, 7 cut by saddle test, 0 cut by aper test, 4 kept
SED Flat : 4 new peaks
Found 118 potential peaks
Of 118 potential peaks: 110 in veto map, 8 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 37 new sources
Median coadd sky for g : 0.0002872416
Median coadd sky for r : 0.0008084943
Median coadd sky for z : 0.0014356363
Stage srcs : Wall: 6.93 s, CPU: 6.89 s, VmPeak: 1026 MB, VmSize: 1016 MB, VmRSS: 199 MB, VmData: 228 MB, maxrss: 0.454584 MB
Stage srcs finished: 2020-06-04T20:45:35.511002
Resources for stage srcs : 0.099 s worker CPU, 0.099 s worker Wall, Wall: 6.929 s, Cores in use: 1.01, Total efficiency (on 4 cores): 25.2 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/PGC1563680-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-06-04T20:45:35.537532
Blob 1 of 6, id: 5, sources: 31, size: 247x206, npix 30434, brick X: 0,247, Y: 41,247, one pixel: 62 41
Blob 2 of 6, id: 0, sources: 6, size: 100x87, npix 3890, brick X: 0,100, Y: 0,87, one pixel: 35 0
Blob 3 of 6, id: 1, sources: 1, size: 35x22, npix 556, brick X: 121,156, Y: 0,22, one pixel: 125 0
Blob 4 of 6, id: 2, sources: 1, size: 22x18, npix 268, brick X: 225,247, Y: 0,18, one pixel: 231 0
Blob 5 of 6, id: 3, sources: 1, size: 21x20, npix 247, brick X: 201,222, Y: 8,28, one pixel: 210 8
Blob 4 finished, total: Wall: 0.63 s, CPU: 0.63 s, VmPeak: 1027 MB, VmSize: 786 MB, VmRSS: 145 MB, VmData: 185 MB, maxrss: 0.148856 MB
Blob 6 of 6, id: 4, sources: 1, size: 20x20, npix 219, brick X: 148,168, Y: 13,33, one pixel: 157 13
Blob 5 finished, total: Wall: 0.40 s, CPU: 0.40 s, VmPeak: 1027 MB, VmSize: 786 MB, VmRSS: 145 MB, VmData: 185 MB, maxrss: 0.148856 MB
Blob 6 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 1027 MB, VmSize: 786 MB, VmRSS: 145 MB, VmData: 185 MB, maxrss: 0.148856 MB
Blob 3 finished, total: Wall: 2.32 s, CPU: 2.32 s, VmPeak: 1021 MB, VmSize: 785 MB, VmRSS: 145 MB, VmData: 184 MB, maxrss: 0.148328 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (163.67865, 18.57170), pm (0.0, 0.0), parallax 0.000 with NanoMaggies: g=16.7, r=18.6, z=14.2
Reset brightness to NanoMaggies: g=21.6, r=20.6, z=20.1
Measuring 4 iterative sources
Blob 2 finished, total: Wall: 10.09 s, CPU: 10.09 s, VmPeak: 1031 MB, VmSize: 786 MB, VmRSS: 146 MB, VmData: 185 MB, maxrss: 0.149428 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (163.67803, 18.57944), pm (2.3, -4.7), parallax 1.086 with NanoMaggies: g=18.1, r=16.9, z=16.7
Reset brightness to NanoMaggies: g=15.4, r=14.9, z=14.6
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=0.781011, ee1=-0.127712, ee2=-0.146131
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (163.67939, 18.57887), pm (-26.8, -19.2), parallax 0.549 with NanoMaggies: g=11.8, r=10.6, z=11.9
Reset brightness to NanoMaggies: g=21.5, r=20.2, z=19.4
Measuring 10 iterative sources
Blob 1 finished, total: Wall: 37.97 s, CPU: 37.97 s, VmPeak: 1028 MB, VmSize: 790 MB, VmRSS: 156 MB, VmData: 190 MB, maxrss: 0.159752 MB
Old catalog: 42
New catalog: 49
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/blobs-custom-163667p18577.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/metrics/cus/all-models-custom-163667p18577.fits
Stage fitblobs : Wall: 45.49 s, CPU: 5.37 s, VmPeak: 1026 MB, VmSize: 1016 MB, VmRSS: 200 MB, VmData: 228 MB, maxrss: 0.454584 MB
Stage fitblobs finished: 2020-06-04T20:46:21.028826
Resources for stage fitblobs : 51.603 s worker CPU, 51.634 s worker Wall, Wall: 45.493 s, Cores in use: 1.25, Total efficiency (on 4 cores): 31.3 %
Stage fitblobs finished
Running stage coadds
Running stage coadds at 2020-06-04T20:46:21.031192
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/tmp-legacysurvey-custom-163667p18577-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/tmp-legacysurvey-custom-163667p18577-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/tmp-legacysurvey-custom-163667p18577-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/tmp-legacysurvey-custom-163667p18577-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-resid.jpg
Stage coadds : Wall: 11.55 s, CPU: 6.60 s, VmPeak: 1026 MB, VmSize: 1016 MB, VmRSS: 201 MB, VmData: 228 MB, maxrss: 0.454584 MB
Stage coadds finished: 2020-06-04T20:46:32.576959
Resources for stage coadds : 3.130 s worker CPU, 3.243 s worker Wall, Wall: 11.547 s, Cores in use: 0.84, Total efficiency (on 4 cores): 21.1 %
Stage coadds finished
Running stage wise_forced
Running stage wise_forced at 2020-06-04T20:46:32.578471
Cut to 1 unWISE tiles
Reading WISE tile 1633p181 band 1
Reading WISE tile 1633p181 band 2
Reading WISE tile 1633p181 band 3
Reading WISE tile 1633p181 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.18 s, CPU: 0.18 s, VmPeak: 1031 MB, VmSize: 864 MB, VmRSS: 149 MB, VmData: 179 MB, maxrss: 0.191876 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 1027 MB, VmSize: 857 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.206868 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 1021 MB, VmSize: 861 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.210852 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/163/1633p181/unwise-1633p181-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 1028 MB, VmSize: 865 MB, VmRSS: 154 MB, VmData: 180 MB, maxrss: 0.218684 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/tmp-legacysurvey-custom-163667p18577-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/tmp-legacysurvey-custom-163667p18577-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 8.73 s, CPU: 6.76 s, VmPeak: 1026 MB, VmSize: 1016 MB, VmRSS: 202 MB, VmData: 228 MB, maxrss: 0.454584 MB
Stage wise_forced finished: 2020-06-04T20:46:41.310028
Resources for stage wise_forced : 5.006 s worker CPU, 6.497 s worker Wall, Wall: 8.733 s, Cores in use: 1.35, Total efficiency (on 4 cores): 33.7 %
Stage wise_forced finished
Running stage writecat
Running stage writecat at 2020-06-04T20:46:41.311907
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/coadd/cus/custom-163667p18577/legacysurvey-custom-163667p18577-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/tractor-i/cus/tractor-custom-163667p18577.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/tractor/cus/tractor-custom-163667p18577.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/163/PGC1563680/tractor/cus/brick-custom-163667p18577.sha256sum
Stage writecat : Wall: 5.82 s, CPU: 5.66 s, VmPeak: 1148 MB, VmSize: 1020 MB, VmRSS: 203 MB, VmData: 228 MB, maxrss: 0.454584 MB
Stage writecat finished: 2020-06-04T20:46:47.134093
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 5.823 s, Cores in use: 0.97, Total efficiency (on 4 cores): 24.3 %
Stage writecat finished
All done: 65.834 s worker CPU, 1091.388 s worker Wall, Wall: 426.002 s, Cores in use: 0.30, Total efficiency (on 4 cores): 7.4 %
Finished galaxy PGC1563680 in 7.256 minutes.