Started working on galaxy PGC1968748 at Wed Aug  5 19:57:10 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.31688099999997 31.871704100000002 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T19:57:13.129997
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.31688099999997', '31.871704100000002', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.31688099999997 31.871704100000002 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.31688099999997 31.871704100000002
Starting process 46787 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 46788 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 46789 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 46790 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 46791 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059472 MB
Starting process 46792 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059484 MB
Starting process 46793 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059484 MB
Starting process 46794 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 46795 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 46796 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 46798 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 46799 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 46797 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 46800 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 46801 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 46802 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T19:57:14.041288
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 3 CCDs:
  c4d_160408_091648_ooi_r_ls9-N5 r expnum 535320 exptime 197.0 propid 2014B-0404 seeing 1.71 MJD 57486.384 object DECaLS_18777_r 
    decam/CP/V4.8.2a/CP20160407/c4d_160408_091648_ooi_r_ls9.fits.fz
  c4d_160803_023048_ooi_g_ls9-N5 g expnum 558496 exptime 130.0 propid 2014B-0404 seeing 1.50 MJD 57603.103 object DECaLS_18777_g 
    decam/CP/V4.8.2a/CP20160802/c4d_160803_023048_ooi_g_ls9.fits.fz
  c4d_160408_100155_ooi_z_ls9-N5 z expnum 535341 exptime 138.0 propid 2014B-0404 seeing 1.51 MJD 57486.416 object DECaLS_34649_z 
    decam/CP/V4.8.2a/CP20160407/c4d_160408_100155_ooi_z_ls9.fits.fz
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95b90ac-g-20160803T022744_oi[N5,noinherit][2046,4094][real]: DECaLS_1
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954d4d0-z-20160408T095847_oi[N5,noinherit][2046,4094][real]: DECaLS_3
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956f863-r-20160408T091240_oi[N5,noinherit][2046,4094][real]: DECaLS_1
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 34.79 s, CPU: 1.83 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 172 MB, VmData: 204 MB, maxrss: 0.175616 MB
Stage tims finished: 2020-08-05T19:57:48.834820
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95b90ac-g-20160803T022744_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956f863-r-20160408T091240_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 1.747 s worker CPU, 85.101 s worker Wall, Wall: 34.795 s, Cores in use: 0.10, Total efficiency (on 16 cores): 0.6 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-tims.p
Running stage refs
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954d4d0-z-20160408T095847_oi[N5,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T19:57:48.847376
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02978.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02850.fits
Matched 9 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/reference-custom-276316p31871.fits
Stage refs : Wall: 20.45 s, CPU: 8.94 s, VmPeak: 1137 MB, VmSize: 1132 MB, VmRSS: 296 MB, VmData: 326 MB, maxrss: 0.303036 MB
Stage refs finished: 2020-08-05T19:58:09.300406
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 20.455 s, Cores in use: 0.44, Total efficiency (on 16 cores): 2.7 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T19:58:09.313533
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outlier-mask-custom-276316p31871.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-pre-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outlier-mask-custom-276316p31871.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-post-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-masked-pos-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-masked-neg-custom-276316p31871.jpg
Stage outliers : Wall: 0.99 s, CPU: 0.52 s, VmPeak: 1141 MB, VmSize: 1141 MB, VmRSS: 305 MB, VmData: 332 MB, maxrss: 0.312424 MB
Stage outliers finished: 2020-08-05T19:58:10.304041
Resources for stage outliers : 0.253 s worker CPU, 0.376 s worker Wall, Wall: 0.992 s, Cores in use: 0.78, Total efficiency (on 16 cores): 4.9 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T19:58:10.317924
Stage halos : Wall: 1.26 s, CPU: 1.04 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 305 MB, VmData: 332 MB, maxrss: 0.31266 MB
Stage halos finished: 2020-08-05T19:58:11.577393
Resources for stage halos : 0.228 s worker CPU, 0.514 s worker Wall, Wall: 1.261 s, Cores in use: 1.00, Total efficiency (on 16 cores): 6.3 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-05T19:58:11.598511
PSF sigmas in coadd pixels: ['2.44']
Using average PSF sigma 2.438958240362723
Gaussian PSF norm 0.1156620015486304 vs pixelized 0.106097795
average tim pixel scale / coadd scale: 1.0014111855963088
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.77']
Using average PSF sigma 2.772490147231376
Gaussian PSF norm 0.10174780677059558 vs pixelized 0.087725334
average tim pixel scale / coadd scale: 1.0015308556594806
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.46']
Using average PSF sigma 2.4562310448803744
Gaussian PSF norm 0.11484863867422415 vs pixelized 0.09799772
average tim pixel scale / coadd scale: 1.001407186596704
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.12 s, CPU: 0.07 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 307 MB, VmData: 332 MB, maxrss: 0.314124 MB
Stage fit_on_coadds finished: 2020-08-05T19:58:11.716325
Resources for stage fit_on_coadds : 0.037 s worker CPU, 0.037 s worker Wall, Wall: 0.119 s, Cores in use: 0.89, Total efficiency (on 16 cores): 5.6 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T19:58:11.738670
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 37 potential peaks
Of 37 potential peaks: 14 in veto map, 5 cut by saddle test, 0 cut by aper test, 18 kept
SED z : 18 new peaks
Found 35 potential peaks
Of 35 potential peaks: 29 in veto map, 4 cut by saddle test, 0 cut by aper test, 2 kept
SED r : 2 new peaks
Found 28 potential peaks
Of 28 potential peaks: 22 in veto map, 6 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 42 potential peaks
Of 42 potential peaks: 33 in veto map, 9 cut by saddle test, 0 cut by aper test, 0 kept
SED Flat : 0 new peaks
Found 46 potential peaks
Of 46 potential peaks: 35 in veto map, 8 cut by saddle test, 0 cut by aper test, 3 kept
SED Red : 3 new peaks
Adding 23 new sources
Median coadd sky for g : 0.00021451892
Median coadd sky for r : 0.00029209926
Median coadd sky for z : 0.0009049034
Stage srcs : Wall: 1.86 s, CPU: 1.71 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 307 MB, VmData: 332 MB, maxrss: 0.314656 MB
Stage srcs finished: 2020-08-05T19:58:13.602533
Resources for stage srcs : 0.099 s worker CPU, 0.099 s worker Wall, Wall: 1.865 s, Cores in use: 0.97, Total efficiency (on 16 cores): 6.1 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T19:58:13.672834
Blob 1 of 13, id: 3, sources: 17, size: 220x218, npix 19627, brick X: 37,257, Y: 11,229, one pixel: 108 11
Blob 2 of 13, id: 10, sources: 6, size: 70x70, npix 3545, brick X: 187,257, Y: 187,257, one pixel: 204 187
Blob 3 of 13, id: 2, sources: 5, size: 78x61, npix 2410, brick X: 179,257, Y: 0,61, one pixel: 183 0
Blob 4 of 13, id: 11, sources: 3, size: 75x57, npix 2300, brick X: 8,83, Y: 200,257, one pixel: 23 200
Blob 5 of 13, id: 0, sources: 5, size: 78x40, npix 2135, brick X: 0,78, Y: 0,40, one pixel: 5 0
Blob 6 of 13, id: 12, sources: 1, size: 51x48, npix 1851, brick X: 94,145, Y: 209,257, one pixel: 122 209
Blob 7 of 13, id: 7, sources: 2, size: 42x54, npix 1485, brick X: 13,55, Y: 119,173, one pixel: 25 119
Blob 8 of 13, id: 6, sources: 1, size: 26x26, npix 417, brick X: 5,31, Y: 68,94, one pixel: 16 68
Blob 9 of 13, id: 5, sources: 1, size: 22x23, npix 333, brick X: 227,249, Y: 67,90, one pixel: 236 67
Blob 10 of 13, id: 1, sources: 1, size: 25x16, npix 284, brick X: 78,103, Y: 0,16, one pixel: 81 0
Blob 11 of 13, id: 4, sources: 1, size: 21x21, npix 261, brick X: 198,219, Y: 33,54, one pixel: 207 33
Blob 12 of 13, id: 9, sources: 1, size: 18x22, npix 251, brick X: 239,257, Y: 168,190, one pixel: 247 168
Blob 13 of 13, id: 8, sources: 1, size: 20x20, npix 229, brick X: 167,187, Y: 167,187, one pixel: 176 167
Blob 11 finished, total: Wall: 0.07 s, CPU: 0.07 s, VmPeak: 959 MB, VmSize: 732 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.145944 MB
Blob 8 finished, total: Wall: 0.22 s, CPU: 0.21 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.097056 MB
Measuring 2 iterative sources
Blob 13 finished, total: Wall: 0.27 s, CPU: 0.25 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.096792 MB
Blob 12 finished, total: Wall: 0.42 s, CPU: 0.41 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.145964 MB
Blob 9 finished, total: Wall: 0.76 s, CPU: 0.69 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 88 MB, VmData: 137 MB, maxrss: 0.090404 MB
Blob 10 finished, total: Wall: 1.27 s, CPU: 1.26 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093608 MB
Blob 6 finished, total: Wall: 1.67 s, CPU: 1.67 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 112 MB, VmData: 166 MB, maxrss: 0.115156 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.32575, 31.86080), pm (0.2, -4.8), parallax 0.247 with NanoMaggies: g=13.4, r=12.8, z=12.2
Reset brightness to NanoMaggies: g=13.8, r=13, z=12.6
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.33117, 31.86343), pm (-1.2, -11.3), parallax 0.298 with NanoMaggies: g=18, r=17.5, z=17.2
Reset brightness to NanoMaggies: g=18, r=17.5, z=17.2
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.32453, 31.85826), pm (12.8, 3.8), parallax 2.857 with NanoMaggies: g=20.1, r=18.6, z=16.9
Reset brightness to NanoMaggies: g=20.1, r=18.6, z=16.9
Blob 5 finished, total: Wall: 1.76 s, CPU: 1.75 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096388 MB
Measuring 1 iterative sources
Blob 7 finished, total: Wall: 2.22 s, CPU: 2.21 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 113 MB, VmData: 166 MB, maxrss: 0.115912 MB
Blob 4 finished, total: Wall: 2.24 s, CPU: 2.24 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 95 MB, VmData: 140 MB, maxrss: 0.097136 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.30989, 31.86174), pm (-9.2, 6.6), parallax 1.345 with NanoMaggies: g=15.3, r=14.4, z=14.2
Reset brightness to NanoMaggies: g=15, r=14.4, z=14.1
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.31058, 31.86126), pm (3.8, -4.7), parallax 0.120 with NanoMaggies: g=15.4, r=16.1, z=15.1
Reset brightness to NanoMaggies: g=20.5, r=19.6, z=19.2
Blob 3 finished, total: Wall: 2.44 s, CPU: 2.43 s, VmPeak: 671 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.097896 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.30499, 31.87913), pm (-2.8, -3.8), parallax -0.106 with NanoMaggies: g=23.3, r=19.3, z=25.4
Reset brightness to NanoMaggies: g=19.7, r=19.3, z=19.1
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 3.42 s, CPU: 3.42 s, VmPeak: 951 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 181 MB, maxrss: 0.151804 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.32919, ee1=0.0648916, ee2=0.40389
Measuring 10 iterative sources
Blob 1 finished, total: Wall: 20.19 s, CPU: 20.19 s, VmPeak: 702 MB, VmSize: 691 MB, VmRSS: 123 MB, VmData: 162 MB, maxrss: 0.137024 MB
Old catalog: 49
New catalog: 55
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/blobs-custom-276316p31871.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/all-models-custom-276316p31871.fits
Stage fitblobs : Wall: 22.28 s, CPU: 1.39 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 308 MB, VmData: 332 MB, maxrss: 0.315096 MB
Stage fitblobs finished: 2020-08-05T19:58:35.955801
Resources for stage fitblobs : 37.032 s worker CPU, 37.565 s worker Wall, Wall: 22.285 s, Cores in use: 1.72, Total efficiency (on 16 cores): 10.8 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T19:58:35.995055
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-resid.jpg
Stage coadds : Wall: 5.92 s, CPU: 2.74 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 309 MB, VmData: 334 MB, maxrss: 0.316904 MB
Stage coadds finished: 2020-08-05T19:58:41.915474
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 16.556 s worker CPU, 35.181 s worker Wall, Wall: 5.922 s, Cores in use: 3.26, Total efficiency (on 16 cores): 20.4 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T19:58:41.930701
Cut to 1 unWISE tiles
Reading WISE tile 2770p318 band 1
Reading WISE tile 2770p318 band 3
Reading WISE tile 2770p318 band 2
Reading WISE tile 2770p318 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 805 MB, VmSize: 803 MB, VmRSS: 143 MB, VmData: 171 MB, maxrss: 0.206236 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 804 MB, VmSize: 804 MB, VmRSS: 144 MB, VmData: 176 MB, maxrss: 0.186284 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 959 MB, VmSize: 809 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.210924 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 951 MB, VmSize: 810 MB, VmRSS: 149 MB, VmData: 175 MB, maxrss: 0.213084 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.03 s, CPU: 0.61 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 310 MB, VmData: 334 MB, maxrss: 0.317416 MB
Stage wise_forced finished: 2020-08-05T19:58:43.962000
Resources for stage wise_forced : 3.345 s worker CPU, 4.604 s worker Wall, Wall: 2.033 s, Cores in use: 1.95, Total efficiency (on 16 cores): 12.2 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T19:58:43.982072
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/tractor-i/cus/tractor-custom-276316p31871.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/tractor/cus/tractor-custom-276316p31871.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/tractor/cus/brick-custom-276316p31871.sha256sum
Stage writecat : Wall: 1.85 s, CPU: 0.81 s, VmPeak: 1274 MB, VmSize: 1146 MB, VmRSS: 312 MB, VmData: 334 MB, maxrss: 0.450292 MB
Stage writecat finished: 2020-08-05T19:58:45.830255
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.850 s, Cores in use: 0.44, Total efficiency (on 16 cores): 2.7 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-writecat.p
All done: 59.299 s worker CPU, 163.480 s worker Wall, Wall: 91.839 s, Cores in use: 0.86, Total efficiency (on 16 cores): 5.4 %
Finished galaxy PGC1968748 in 1.614 minutes.
Started working on galaxy PGC1968748 at Wed Aug  5 21:05:03 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.31688099999997 31.871704100000002 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T21:05:04.142443
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.31688099999997', '31.871704100000002', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.31688099999997 31.871704100000002 --width 257 --height 257 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.31688099999997 31.871704100000002
Starting process 60190 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 60191 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 60192 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 60193 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 60194 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 60195 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 60196 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 60197 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 60198 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 60199 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 60200 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 60201 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 60202 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 60203 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 60204 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 60205 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 60206 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 60207 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 60208 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 60209 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 60210 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 60211 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB
Starting process 60212 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 60213 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 60214 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 60215 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 60216 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 60217 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 60218 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 60219 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 60220 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 60221 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T21:05:05.283678
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 3 CCDs:
  c4d_160408_091648_ooi_r_ls9-N5 r expnum 535320 exptime 197.0 propid 2014B-0404 seeing 1.71 MJD 57486.384 object DECaLS_18777_r 
    decam/CP/V4.8.2a/CP20160407/c4d_160408_091648_ooi_r_ls9.fits.fz
  c4d_160803_023048_ooi_g_ls9-N5 g expnum 558496 exptime 130.0 propid 2014B-0404 seeing 1.50 MJD 57603.103 object DECaLS_18777_g 
    decam/CP/V4.8.2a/CP20160802/c4d_160803_023048_ooi_g_ls9.fits.fz
  c4d_160408_100155_ooi_z_ls9-N5 z expnum 535341 exptime 138.0 propid 2014B-0404 seeing 1.51 MJD 57486.416 object DECaLS_34649_z 
    decam/CP/V4.8.2a/CP20160407/c4d_160408_100155_ooi_z_ls9.fits.fz
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954d4d0-z-20160408T095847_oi[N5,noinherit][2046,4094][real]: DECaLS_3
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95b90ac-g-20160803T022744_oi[N5,noinherit][2046,4094][real]: DECaLS_1
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956f863-r-20160408T091240_oi[N5,noinherit][2046,4094][real]: DECaLS_1
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 16.56 s, CPU: 1.68 s, VmPeak: 1010 MB, VmSize: 1010 MB, VmRSS: 172 MB, VmData: 204 MB, maxrss: 0.17564 MB
Stage tims finished: 2020-08-05T21:05:21.841826
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95b90ac-g-20160803T022744_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 1.622 s worker CPU, 36.098 s worker Wall, Wall: 16.561 s, Cores in use: 0.20, Total efficiency (on 32 cores): 0.6 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-tims.p
Running stage refs
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954d4d0-z-20160408T095847_oi[N5,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956f863-r-20160408T091240_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T21:05:21.868363
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02978.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02850.fits
Matched 9 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/reference-custom-276316p31871.fits
Stage refs : Wall: 27.50 s, CPU: 9.04 s, VmPeak: 1137 MB, VmSize: 1132 MB, VmRSS: 296 MB, VmData: 326 MB, maxrss: 0.303464 MB
Stage refs finished: 2020-08-05T21:05:49.367087
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 27.501 s, Cores in use: 0.33, Total efficiency (on 32 cores): 1.0 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T21:05:49.385569
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outlier-mask-custom-276316p31871.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-pre-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outlier-mask-custom-276316p31871.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-post-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-masked-pos-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/outliers-masked-neg-custom-276316p31871.jpg
Stage outliers : Wall: 1.13 s, CPU: 0.67 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 305 MB, VmData: 332 MB, maxrss: 0.312428 MB
Stage outliers finished: 2020-08-05T21:05:50.518503
Resources for stage outliers : 0.284 s worker CPU, 0.495 s worker Wall, Wall: 1.135 s, Cores in use: 0.84, Total efficiency (on 32 cores): 2.6 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T21:05:50.533354
Stage halos : Wall: 0.55 s, CPU: 0.31 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 305 MB, VmData: 332 MB, maxrss: 0.312684 MB
Stage halos finished: 2020-08-05T21:05:51.083144
Resources for stage halos : 0.318 s worker CPU, 0.615 s worker Wall, Wall: 0.552 s, Cores in use: 1.15, Total efficiency (on 32 cores): 3.6 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-05T21:05:51.100889
PSF sigmas in coadd pixels: ['2.44']
Using average PSF sigma 2.438958240362723
Gaussian PSF norm 0.1156620015486304 vs pixelized 0.106097795
average tim pixel scale / coadd scale: 1.0014111855963088
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.77']
Using average PSF sigma 2.772490147231376
Gaussian PSF norm 0.10174780677059558 vs pixelized 0.087725334
average tim pixel scale / coadd scale: 1.0015308556594806
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.46']
Using average PSF sigma 2.4562310448803744
Gaussian PSF norm 0.11484863867422415 vs pixelized 0.09799772
average tim pixel scale / coadd scale: 1.001407186596704
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.16 s, CPU: 0.08 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 307 MB, VmData: 332 MB, maxrss: 0.314148 MB
Stage fit_on_coadds finished: 2020-08-05T21:05:51.256387
Resources for stage fit_on_coadds : 0.058 s worker CPU, 0.073 s worker Wall, Wall: 0.158 s, Cores in use: 0.88, Total efficiency (on 32 cores): 2.7 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T21:05:51.282831
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 37 potential peaks
Of 37 potential peaks: 14 in veto map, 5 cut by saddle test, 0 cut by aper test, 18 kept
SED z : 18 new peaks
Found 35 potential peaks
Of 35 potential peaks: 29 in veto map, 4 cut by saddle test, 0 cut by aper test, 2 kept
SED r : 2 new peaks
Found 28 potential peaks
Of 28 potential peaks: 22 in veto map, 6 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 42 potential peaks
Of 42 potential peaks: 33 in veto map, 9 cut by saddle test, 0 cut by aper test, 0 kept
SED Flat : 0 new peaks
Found 46 potential peaks
Of 46 potential peaks: 35 in veto map, 8 cut by saddle test, 0 cut by aper test, 3 kept
SED Red : 3 new peaks
Adding 23 new sources
Median coadd sky for g : 0.00021451892
Median coadd sky for r : 0.00029209926
Median coadd sky for z : 0.0009049034
Stage srcs : Wall: 2.73 s, CPU: 2.56 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 307 MB, VmData: 332 MB, maxrss: 0.31468 MB
Stage srcs finished: 2020-08-05T21:05:54.011669
Resources for stage srcs : 0.127 s worker CPU, 0.166 s worker Wall, Wall: 2.731 s, Cores in use: 0.99, Total efficiency (on 32 cores): 3.1 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T21:05:54.041437
Blob 1 of 13, id: 3, sources: 17, size: 220x218, npix 19627, brick X: 37,257, Y: 11,229, one pixel: 108 11
Blob 2 of 13, id: 10, sources: 6, size: 70x70, npix 3545, brick X: 187,257, Y: 187,257, one pixel: 204 187
Blob 3 of 13, id: 2, sources: 5, size: 78x61, npix 2410, brick X: 179,257, Y: 0,61, one pixel: 183 0
Blob 4 of 13, id: 11, sources: 3, size: 75x57, npix 2300, brick X: 8,83, Y: 200,257, one pixel: 23 200
Blob 5 of 13, id: 0, sources: 5, size: 78x40, npix 2135, brick X: 0,78, Y: 0,40, one pixel: 5 0
Blob 6 of 13, id: 12, sources: 1, size: 51x48, npix 1851, brick X: 94,145, Y: 209,257, one pixel: 122 209
Blob 7 of 13, id: 7, sources: 2, size: 42x54, npix 1485, brick X: 13,55, Y: 119,173, one pixel: 25 119
Blob 8 of 13, id: 6, sources: 1, size: 26x26, npix 417, brick X: 5,31, Y: 68,94, one pixel: 16 68
Blob 9 of 13, id: 5, sources: 1, size: 22x23, npix 333, brick X: 227,249, Y: 67,90, one pixel: 236 67
Blob 10 of 13, id: 1, sources: 1, size: 25x16, npix 284, brick X: 78,103, Y: 0,16, one pixel: 81 0
Blob 11 of 13, id: 4, sources: 1, size: 21x21, npix 261, brick X: 198,219, Y: 33,54, one pixel: 207 33
Blob 12 of 13, id: 9, sources: 1, size: 18x22, npix 251, brick X: 239,257, Y: 168,190, one pixel: 247 168
Blob 13 of 13, id: 8, sources: 1, size: 20x20, npix 229, brick X: 167,187, Y: 167,187, one pixel: 176 167
Blob 11 finished, total: Wall: 0.15 s, CPU: 0.07 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 82 MB, VmData: 136 MB, maxrss: 0.083992 MB
Measuring 2 iterative sources
Blob 8 finished, total: Wall: 0.24 s, CPU: 0.22 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094084 MB
Blob 13 finished, total: Wall: 0.30 s, CPU: 0.27 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094184 MB
Blob 12 finished, total: Wall: 0.41 s, CPU: 0.39 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094316 MB
Blob 9 finished, total: Wall: 0.79 s, CPU: 0.78 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.097416 MB
Blob 10 finished, total: Wall: 1.02 s, CPU: 0.99 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.094752 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.32575, 31.86080), pm (0.2, -4.8), parallax 0.247 with NanoMaggies: g=13.4, r=12.8, z=12.2
Reset brightness to NanoMaggies: g=13.8, r=13, z=12.6
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.33117, 31.86343), pm (-1.2, -11.3), parallax 0.298 with NanoMaggies: g=18, r=17.5, z=17.2
Reset brightness to NanoMaggies: g=18, r=17.5, z=17.2
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.32453, 31.85826), pm (12.8, 3.8), parallax 2.857 with NanoMaggies: g=20.1, r=18.6, z=16.9
Reset brightness to NanoMaggies: g=20.1, r=18.6, z=16.9
Blob 5 finished, total: Wall: 1.82 s, CPU: 1.78 s, VmPeak: 669 MB, VmSize: 667 MB, VmRSS: 89 MB, VmData: 138 MB, maxrss: 0.092672 MB
Blob 6 finished, total: Wall: 2.01 s, CPU: 2.00 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094068 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.30989, 31.86174), pm (-9.2, 6.6), parallax 1.345 with NanoMaggies: g=15.3, r=14.4, z=14.2
Reset brightness to NanoMaggies: g=15, r=14.4, z=14.1
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.31058, 31.86126), pm (3.8, -4.7), parallax 0.120 with NanoMaggies: g=15.4, r=16.1, z=15.1
Reset brightness to NanoMaggies: g=20.5, r=19.6, z=19.2
Blob 3 finished, total: Wall: 2.31 s, CPU: 2.29 s, VmPeak: 671 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.098152 MB
Measuring 1 iterative sources
Blob 7 finished, total: Wall: 2.36 s, CPU: 2.29 s, VmPeak: 668 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 139 MB, maxrss: 0.092716 MB
Blob 4 finished, total: Wall: 2.43 s, CPU: 2.42 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148368 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.30499, 31.87913), pm (-2.8, -3.8), parallax -0.106 with NanoMaggies: g=23.3, r=19.3, z=25.4
Reset brightness to NanoMaggies: g=19.7, r=19.3, z=19.1
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 3.65 s, CPU: 3.57 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 142 MB, maxrss: 0.096524 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.32919, ee1=0.0648916, ee2=0.40389
Measuring 10 iterative sources
Blob 1 finished, total: Wall: 22.25 s, CPU: 22.20 s, VmPeak: 703 MB, VmSize: 692 MB, VmRSS: 124 MB, VmData: 163 MB, maxrss: 0.1373 MB
Old catalog: 49
New catalog: 55
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/blobs-custom-276316p31871.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/metrics/cus/all-models-custom-276316p31871.fits
Stage fitblobs : Wall: 25.31 s, CPU: 2.79 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 308 MB, VmData: 332 MB, maxrss: 0.31512 MB
Stage fitblobs finished: 2020-08-05T21:06:19.347466
Resources for stage fitblobs : 39.517 s worker CPU, 42.361 s worker Wall, Wall: 25.308 s, Cores in use: 1.67, Total efficiency (on 32 cores): 5.2 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T21:06:19.430957
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-resid.jpg
Stage coadds : Wall: 4.01 s, CPU: 0.97 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 310 MB, VmData: 334 MB, maxrss: 0.316928 MB
Stage coadds finished: 2020-08-05T21:06:23.439345
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 34.278 s worker CPU, 67.676 s worker Wall, Wall: 4.010 s, Cores in use: 8.79, Total efficiency (on 32 cores): 27.5 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T21:06:23.457136
Cut to 1 unWISE tiles
Reading WISE tile 2770p318 band 1
Reading WISE tile 2770p318 band 2
Reading WISE tile 2770p318 band 3
Reading WISE tile 2770p318 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 956 MB, VmSize: 809 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.211472 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 959 MB, VmSize: 805 MB, VmRSS: 144 MB, VmData: 174 MB, maxrss: 0.206724 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 807 MB, VmSize: 805 MB, VmRSS: 143 MB, VmData: 173 MB, maxrss: 0.206304 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 803 MB, VmSize: 803 MB, VmRSS: 142 MB, VmData: 174 MB, maxrss: 0.181312 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.07 s, CPU: 0.57 s, VmPeak: 1142 MB, VmSize: 1142 MB, VmRSS: 310 MB, VmData: 334 MB, maxrss: 0.31744 MB
Stage wise_forced finished: 2020-08-05T21:06:25.526696
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage wise_forced : 3.370 s worker CPU, 4.904 s worker Wall, Wall: 2.072 s, Cores in use: 1.90, Total efficiency (on 32 cores): 5.9 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T21:06:25.544893
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/tractor-i/cus/tractor-custom-276316p31871.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/tractor/cus/tractor-custom-276316p31871.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/tractor/cus/brick-custom-276316p31871.sha256sum
Stage writecat : Wall: 1.36 s, CPU: 1.00 s, VmPeak: 1274 MB, VmSize: 1146 MB, VmRSS: 312 MB, VmData: 334 MB, maxrss: 0.450372 MB
Stage writecat finished: 2020-08-05T21:06:26.907526
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.365 s, Cores in use: 0.74, Total efficiency (on 32 cores): 2.3 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC1968748/PGC1968748-largegalaxy-writecat.p
All done: 79.579 s worker CPU, 152.395 s worker Wall, Wall: 81.653 s, Cores in use: 1.22, Total efficiency (on 32 cores): 3.8 %
Finished galaxy PGC1968748 in 1.404 minutes.
Started working on galaxy PGC1968748 at Fri Aug  7 09:25:26 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.3167010687475 31.87186591385624 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-07T09:25:27.222834
legacypipe git version: DR9.5.11-watershed-3-gd509eaaf
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.3167010687475', '31.87186591385624', '--width', '257', '--height', '257', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.3167010687475 31.87186591385624 --width 257 --height 257 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.3167010687475 31.87186591385624
Starting process 6753 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 6754 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 6755 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 6756 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 6757 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB
Starting process 6758 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059556 MB
Starting process 6759 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059556 MB
Starting process 6760 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 6761 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 6762 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 6763 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 6764 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 6765 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 6766 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 6767 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 6768 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-07T09:25:27.664354
Keeping 3 CCDs:
  c4d_160408_091648_ooi_r_ls9-N5 r expnum 535320 exptime 197.0 propid 2014B-0404 seeing 1.71 MJD 57486.384 object DECaLS_18777_r 
    decam/CP/V4.8.2a/CP20160407/c4d_160408_091648_ooi_r_ls9.fits.fz
  c4d_160803_023048_ooi_g_ls9-N5 g expnum 558496 exptime 130.0 propid 2014B-0404 seeing 1.50 MJD 57603.103 object DECaLS_18777_g 
    decam/CP/V4.8.2a/CP20160802/c4d_160803_023048_ooi_g_ls9.fits.fz
  c4d_160408_100155_ooi_z_ls9-N5 z expnum 535341 exptime 138.0 propid 2014B-0404 seeing 1.51 MJD 57486.416 object DECaLS_34649_z 
    decam/CP/V4.8.2a/CP20160407/c4d_160408_100155_ooi_z_ls9.fits.fz
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954d4d0-z-20160408T095847_oi[N5,noinherit][2046,4094][real]: DECaLS_3
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956f863-r-20160408T091240_oi[N5,noinherit][2046,4094][real]: DECaLS_1
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95b90ac-g-20160803T022744_oi[N5,noinherit][2046,4094][real]: DECaLS_1
Stage tims : Wall: 5.24 s, CPU: 0.78 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 170 MB, VmData: 203 MB, maxrss: 0.173696 MB
Stage tims finished: 2020-08-07T09:25:32.908703
Warning: splinesky does not have sky-template version/run/scale values
Warning: splinesky does not have sky-template version/run/scale values
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954d4d0-z-20160408T095847_oi[N5,noinherit][2046,4094][real]: DECaLS_3 [astropy.io.fits.card]
Warning: splinesky does not have sky-template version/run/scale values
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_956f863-r-20160408T091240_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95b90ac-g-20160803T022744_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card]
Resources for stage tims : 1.462 s worker CPU, 11.111 s worker Wall, Wall: 5.246 s, Cores in use: 0.43, Total efficiency (on 16 cores): 2.7 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-08-07T09:25:32.927952
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02978.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02850.fits
Matched 8 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/reference-custom-276316p31871.fits
Stage refs : Wall: 9.64 s, CPU: 8.84 s, VmPeak: 1130 MB, VmSize: 1125 MB, VmRSS: 294 MB, VmData: 325 MB, maxrss: 0.30116 MB
Stage refs finished: 2020-08-07T09:25:42.572238
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 9.646 s, Cores in use: 0.92, Total efficiency (on 16 cores): 5.7 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-07T09:25:42.588061
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/outlier-mask-custom-276316p31871.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/outliers-pre-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/outlier-mask-custom-276316p31871.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/outliers-post-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/outliers-masked-pos-custom-276316p31871.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/outliers-masked-neg-custom-276316p31871.jpg
Stage outliers : Wall: 10.25 s, CPU: 0.42 s, VmPeak: 1135 MB, VmSize: 1135 MB, VmRSS: 303 MB, VmData: 332 MB, maxrss: 0.310588 MB
Stage outliers finished: 2020-08-07T09:25:52.836364
Resources for stage outliers : 0.260 s worker CPU, 0.315 s worker Wall, Wall: 10.250 s, Cores in use: 0.07, Total efficiency (on 16 cores): 0.4 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-07T09:25:52.888509
Stage halos : Wall: 0.49 s, CPU: 0.33 s, VmPeak: 1135 MB, VmSize: 1135 MB, VmRSS: 304 MB, VmData: 332 MB, maxrss: 0.310868 MB
Stage halos finished: 2020-08-07T09:25:53.380840
Resources for stage halos : 0.202 s worker CPU, 0.411 s worker Wall, Wall: 0.494 s, Cores in use: 1.07, Total efficiency (on 16 cores): 6.7 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-07T09:25:53.410404
PSF sigmas in coadd pixels: ['2.44']
Using average PSF sigma 2.438958240362723
Gaussian PSF norm 0.1156620015486304 vs pixelized 0.106101625
average tim pixel scale / coadd scale: 1.0014111855963088
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.77']
Using average PSF sigma 2.772490147231376
Gaussian PSF norm 0.10174780677059558 vs pixelized 0.087725915
average tim pixel scale / coadd scale: 1.0015308556594806
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.46']
Using average PSF sigma 2.4562310448803744
Gaussian PSF norm 0.11484863867422415 vs pixelized 0.09799733
average tim pixel scale / coadd scale: 1.001407186596704
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.11 s, CPU: 0.07 s, VmPeak: 1135 MB, VmSize: 1135 MB, VmRSS: 305 MB, VmData: 332 MB, maxrss: 0.31234 MB
Stage fit_on_coadds finished: 2020-08-07T09:25:53.516316
Resources for stage fit_on_coadds : 0.037 s worker CPU, 0.037 s worker Wall, Wall: 0.107 s, Cores in use: 1.00, Total efficiency (on 16 cores): 6.2 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-07T09:25:53.528011
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 39 potential peaks
Of 39 potential peaks: 15 in veto map, 6 cut by saddle test, 0 cut by aper test, 18 kept
SED z : 18 new peaks
Found 36 potential peaks
Of 36 potential peaks: 30 in veto map, 5 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 28 potential peaks
Of 28 potential peaks: 23 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 41 potential peaks
Of 41 potential peaks: 33 in veto map, 8 cut by saddle test, 0 cut by aper test, 0 kept
SED Flat : 0 new peaks
Found 49 potential peaks
Of 49 potential peaks: 36 in veto map, 9 cut by saddle test, 0 cut by aper test, 4 kept
SED Red : 4 new peaks
Adding 23 new sources
Median coadd sky for g : 0.00022552127
Median coadd sky for r : 0.0003016798
Median coadd sky for z : 0.00069541443
Stage srcs : Wall: 0.55 s, CPU: 0.49 s, VmPeak: 1135 MB, VmSize: 1135 MB, VmRSS: 306 MB, VmData: 332 MB, maxrss: 0.312884 MB
Stage srcs finished: 2020-08-07T09:25:54.079105
Resources for stage srcs : 0.137 s worker CPU, 0.137 s worker Wall, Wall: 0.553 s, Cores in use: 1.13, Total efficiency (on 16 cores): 7.1 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-07T09:25:54.105000
Blob 1 of 13, id: 3, sources: 16, size: 222x219, npix 19754, brick X: 35,257, Y: 8,227, one pixel: 106 8
Blob 2 of 13, id: 10, sources: 6, size: 72x72, npix 3653, brick X: 185,257, Y: 185,257, one pixel: 202 185
Blob 3 of 13, id: 2, sources: 5, size: 79x66, npix 2518, brick X: 178,257, Y: 0,66, one pixel: 183 0
Blob 4 of 13, id: 11, sources: 4, size: 76x58, npix 2325, brick X: 5,81, Y: 198,256, one pixel: 21 198
Blob 5 of 13, id: 12, sources: 1, size: 50x50, npix 1941, brick X: 92,142, Y: 207,257, one pixel: 119 207
Blob 6 of 13, id: 0, sources: 5, size: 76x38, npix 1903, brick X: 0,76, Y: 0,38, one pixel: 9 0
Blob 7 of 13, id: 7, sources: 2, size: 42x54, npix 1506, brick X: 11,53, Y: 116,170, one pixel: 24 116
Blob 8 of 13, id: 9, sources: 2, size: 21x33, npix 436, brick X: 236,257, Y: 166,199, one pixel: 245 166
Blob 9 of 13, id: 6, sources: 1, size: 25x26, npix 415, brick X: 3,28, Y: 66,92, one pixel: 13 66
Blob 10 of 13, id: 5, sources: 1, size: 22x24, npix 325, brick X: 225,247, Y: 64,88, one pixel: 236 64
Blob 11 of 13, id: 4, sources: 1, size: 21x21, npix 261, brick X: 196,217, Y: 31,52, one pixel: 205 31
Blob 12 of 13, id: 8, sources: 1, size: 20x20, npix 228, brick X: 165,185, Y: 165,185, one pixel: 174 165
Blob 13 of 13, id: 1, sources: 1, size: 20x14, npix 170, brick X: 78,98, Y: 0,14, one pixel: 82 0
Blob 11 finished, total: Wall: 0.07 s, CPU: 0.07 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 108 MB, VmData: 166 MB, maxrss: 0.11008 MB
Measuring 2 iterative sources
Blob 9 finished, total: Wall: 0.31 s, CPU: 0.31 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 93 MB, VmData: 141 MB, maxrss: 0.09492 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.30560, 31.87594), pm (0.0, 0.0), parallax 0.000 with NanoMaggies: g=22.8, r=21.2, z=19.4
Reset brightness to NanoMaggies: g=22.1, r=21.1, z=20.6
Measuring 1 iterative sources
Blob 12 finished, total: Wall: 0.42 s, CPU: 0.42 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.095736 MB
Blob 10 finished, total: Wall: 0.54 s, CPU: 0.53 s, VmPeak: 956 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146416 MB
Blob 8 finished, total: Wall: 0.64 s, CPU: 0.63 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096116 MB
Blob 13 finished, total: Wall: 0.98 s, CPU: 0.94 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093248 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.32575, 31.86080), pm (0.2, -4.8), parallax 0.247 with NanoMaggies: g=13.2, r=12.9, z=12.1
Reset brightness to NanoMaggies: g=13.8, r=13, z=12.6
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.33117, 31.86343), pm (-1.2, -11.3), parallax 0.298 with NanoMaggies: g=18, r=17.5, z=17.2
Reset brightness to NanoMaggies: g=18, r=17.5, z=17.2
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.32453, 31.85826), pm (12.8, 3.8), parallax 2.857 with NanoMaggies: g=20.1, r=18.6, z=16.9
Reset brightness to NanoMaggies: g=20.1, r=18.6, z=16.9
Blob 6 finished, total: Wall: 1.54 s, CPU: 1.54 s, VmPeak: 951 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146796 MB
Blob 5 finished, total: Wall: 1.72 s, CPU: 1.72 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.097464 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.30989, 31.86174), pm (-9.2, 6.6), parallax 1.345 with NanoMaggies: g=15.4, r=14.5, z=14.3
Reset brightness to NanoMaggies: g=15, r=14.4, z=14.1
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.31058, 31.86126), pm (3.8, -4.7), parallax 0.120 with NanoMaggies: g=15, r=15.2, z=14.8
Reset brightness to NanoMaggies: g=20.5, r=19.6, z=19.2
Blob 3 finished, total: Wall: 2.09 s, CPU: 2.08 s, VmPeak: 959 MB, VmSize: 726 MB, VmRSS: 141 MB, VmData: 181 MB, maxrss: 0.144792 MB
Blob 7 finished, total: Wall: 2.24 s, CPU: 2.21 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.094884 MB
Measuring 1 iterative sources
Blob 4 finished, total: Wall: 2.59 s, CPU: 2.59 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 96 MB, VmData: 141 MB, maxrss: 0.0982 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.30499, 31.87913), pm (-2.8, -3.8), parallax -0.106 with NanoMaggies: g=20, r=18.9, z=18.2
Reset brightness to NanoMaggies: g=19.7, r=19.3, z=19.1
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 4.07 s, CPU: 4.07 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 95 MB, VmData: 142 MB, maxrss: 0.097768 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=0.80234, ee1=0.0642735, ee2=0.387876
Measuring 8 iterative sources
Blob 1 finished, total: Wall: 15.74 s, CPU: 15.74 s, VmPeak: 699 MB, VmSize: 698 MB, VmRSS: 128 MB, VmData: 166 MB, maxrss: 0.135956 MB
Old catalog: 49
New catalog: 53
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/blobs-custom-276316p31871.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/metrics/cus/all-models-custom-276316p31871.fits
Stage fitblobs : Wall: 16.15 s, CPU: 0.34 s, VmPeak: 1135 MB, VmSize: 1135 MB, VmRSS: 306 MB, VmData: 332 MB, maxrss: 0.313404 MB
Stage fitblobs finished: 2020-08-07T09:26:10.257519
Resources for stage fitblobs : 33.081 s worker CPU, 33.225 s worker Wall, Wall: 16.154 s, Cores in use: 2.07, Total efficiency (on 16 cores): 12.9 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-07T09:26:10.279383
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-resid.jpg
Stage coadds : Wall: 11.72 s, CPU: 0.63 s, VmPeak: 1135 MB, VmSize: 1135 MB, VmRSS: 308 MB, VmData: 333 MB, maxrss: 0.315216 MB
Stage coadds finished: 2020-08-07T09:26:22.004218
Resources for stage coadds : 16.945 s worker CPU, 17.552 s worker Wall, Wall: 11.727 s, Cores in use: 1.50, Total efficiency (on 16 cores): 9.4 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-07T09:26:22.021512
Cut to 1 unWISE tiles
Reading WISE tile 2770p318 band 1
Reading WISE tile 2770p318 band 3
Reading WISE tile 2770p318 band 4
Reading WISE tile 2770p318 band 2
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 959 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.208732 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 802 MB, VmSize: 802 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.187532 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 801 MB, VmSize: 801 MB, VmRSS: 145 MB, VmData: 179 MB, maxrss: 0.200744 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/277/2770p318/unwise-2770p318-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 956 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.208072 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/tmp-legacysurvey-custom-276316p31871-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 14.23 s, CPU: 0.65 s, VmPeak: 1135 MB, VmSize: 1135 MB, VmRSS: 308 MB, VmData: 333 MB, maxrss: 0.315728 MB
Stage wise_forced finished: 2020-08-07T09:26:36.254275
Resources for stage wise_forced : 3.118 s worker CPU, 22.781 s worker Wall, Wall: 14.234 s, Cores in use: 0.27, Total efficiency (on 16 cores): 1.7 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-07T09:26:36.300836
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/coadd/cus/custom-276316p31871/legacysurvey-custom-276316p31871-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/tractor-i/cus/tractor-custom-276316p31871.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/tractor/cus/tractor-custom-276316p31871.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/tractor/cus/brick-custom-276316p31871.sha256sum
Stage writecat : Wall: 1.01 s, CPU: 0.57 s, VmPeak: 1267 MB, VmSize: 1139 MB, VmRSS: 310 MB, VmData: 333 MB, maxrss: 0.448588 MB
Stage writecat finished: 2020-08-07T09:26:37.315341
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.016 s, Cores in use: 0.56, Total efficiency (on 16 cores): 3.5 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC1968748/PGC1968748-largegalaxy-writecat.p
All done: 55.245 s worker CPU, 85.573 s worker Wall, Wall: 69.686 s, Cores in use: 0.98, Total efficiency (on 16 cores): 6.1 %
Finished galaxy PGC1968748 in 1.205 minutes.