Started working on galaxy PGC1041779 at Wed Aug  5 18:45:13 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 163.5982785 -5.6726454 --width 233 --height 233 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T18:45:14.553082
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '163.5982785', '-5.6726454', '--width', '233', '--height', '233', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 163.5982785 -5.6726454 --width 233 --height 233 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 163.5982785 -5.6726454
Starting process 3038 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 3039 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 3040 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 3041 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 3042 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059324 MB
Starting process 3043 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059332 MB
Starting process 3044 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 3045 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 3046 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 3047 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 3048 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 3049 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 3050 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 3051 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 3052 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 3053 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T18:45:15.088914
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 10 CCDs:
  c4d_170327_005523_ooi_r_ls9-S19 r expnum 633964 exptime 64.0 propid 2014B-0404 seeing 1.11 MJD 57839.037 object DECaLS_23011_r 
    decam/CP/V4.8.2/CP20170326/c4d_170327_005523_ooi_r_ls9.fits.fz
  c4d_170328_023211_ooi_g_ls9-S19 g expnum 634385 exptime 86.0 propid 2014B-0404 seeing 1.44 MJD 57840.104 object DECaLS_23011_g 
    decam/CP/V4.8.2/CP20170327/c4d_170328_023211_ooi_g_ls9.fits.fz
  c4d_170304_010835_ooi_z_ls9-S19 z expnum 625641 exptime 203.0 propid 2014B-0404 seeing 1.53 MJD 57816.045 object DECaLS_23011_z 
    decam/CP/V4.8.2/CP20170303/c4d_170304_010835_ooi_z_ls9.fits.fz
  c4d_181215_064155_ooi_r_ls9-S18 r expnum 803328 exptime 47.0 propid 2014B-0404 seeing 1.03 MJD 58467.279 object DECaLS_38883_r 
    decam/CP/V4.8.2a/CP20181214/c4d_181215_064155_ooi_r_ls9.fits.fz
  c4d_181215_064312_ooi_g_ls9-S18 g expnum 803329 exptime 95.0 propid 2014B-0404 seeing 1.20 MJD 58467.280 object DECaLS_38883_g 
    decam/CP/V4.8.2a/CP20181214/c4d_181215_064312_ooi_g_ls9.fits.fz
  c4d_170306_010031_ooi_z_ls9-S18 z expnum 626329 exptime 162.0 propid 2014B-0404 seeing 1.07 MJD 57818.040 object DECaLS_38883_z 
    decam/CP/V4.8.2/CP20170305/c4d_170306_010031_ooi_z_ls9.fits.fz
  c4d_170301_045711_ooi_g_ls9-S8 g expnum 624537 exptime 123.0 propid 2014B-0404 seeing 1.37 MJD 57813.204 object DECaLS_7125_g 
    decam/CP/V4.8.2/CP20170228/c4d_170301_045711_ooi_g_ls9.fits.fz
  c4d_170301_045436_ooi_r_ls9-S8 r expnum 624536 exptime 48.0 propid 2014B-0404 seeing 1.25 MJD 57813.203 object DECaLS_7125_r 
    decam/CP/V4.8.2/CP20170228/c4d_170301_045436_ooi_r_ls9.fits.fz
  c4d_190206_060626_ooi_z_ls9-S8 z expnum 820178 exptime 147.0 propid 2014B-0404 seeing 1.26 MJD 58520.254 object DECaLS_7125_z 
    decam/CP/V4.8.2a/CP20190205/c4d_190206_060626_ooi_z_ls9.fits.fz
  c4d_151227_061850_ooi_z_ls9-S8 z expnum 506085 exptime 89.0 propid 2014B-0404 seeing 1.15 MJD 57383.261 object DECaLS_7125_z 
    decam/CP/V4.8.2a/CP20151226/c4d_151227_061850_ooi_z_ls9.fits.fz
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95127dc-g-20170328T022958_oi[S19,noinherit][2046,4094][real]: DECaLS_
Clipping 7 pixels with anomalously large oow values: max 0.0553754 vs median 0.0377271
Masking 0 additional saturated pixels in DECam expnum 634385 S19 CCD, decam/CP/V4.8.2/CP20170327/c4d_170328_023211_ooi_g_ls9.fits.fz slice (slice(1604, 1838, None), slice(1497, 1731, None))
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c76c-r-20170301T045259_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950bfa6-z-20170304T010422_oi[S19,noinherit][2046,4094][real]: DECaLS_
Masking 0 additional saturated pixels in DECam expnum 625641 S19 CCD, decam/CP/V4.8.2/CP20170303/c4d_170304_010835_ooi_z_ls9.fits.fz slice (slice(1617, 1851, None), slice(1493, 1726, None))
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c43f-g-20170301T045416_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_951060a-r-20170327T005328_oi[S19,noinherit][2046,4094][real]: DECaLS_
Masking 0 additional saturated pixels in DECam expnum 633964 S19 CCD, decam/CP/V4.8.2/CP20170326/c4d_170327_005523_ooi_r_ls9.fits.fz slice (slice(1601, 1835, None), slice(1505, 1738, None))
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539067-r-20181215T064155_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95527d4-z-20151227T061631_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9534704-z-20190206T060626_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950dd28-z-20170306T005702_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953988f-g-20181215T064312_oi[S18,noinherit][2046,4094][real]: DECaLS_
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 103.50 s, CPU: 1.03 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 177 MB, VmData: 209 MB, maxrss: 0.181724 MB
Stage tims finished: 2020-08-05T18:46:58.586383
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c76c-r-20170301T045259_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95127dc-g-20170328T022958_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950bfa6-z-20170304T010422_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95527d4-z-20151227T061631_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539067-r-20181215T064155_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950dd28-z-20170306T005702_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953988f-g-20181215T064312_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_951060a-r-20170327T005328_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9534704-z-20190206T060626_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 5.786 s worker CPU, 940.668 s worker Wall, Wall: 103.499 s, Cores in use: 0.07, Total efficiency (on 16 cores): 0.4 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-tims.p
Running stage refs
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c43f-g-20170301T045416_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T18:46:58.607730
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06650.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06778.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/reference-custom-163598m05672.fits
Stage refs : Wall: 26.01 s, CPU: 1.71 s, VmPeak: 1042 MB, VmSize: 1028 MB, VmRSS: 195 MB, VmData: 226 MB, maxrss: 0.199244 MB
Stage refs finished: 2020-08-05T18:47:24.613942
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 26.008 s, Cores in use: 0.07, Total efficiency (on 16 cores): 0.4 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T18:47:24.739115
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outlier-mask-custom-163598m05672.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-pre-custom-163598m05672.jpg
Image c4d_170306_010031_ooi_z_ls9-S18 z : masking 277 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outlier-mask-custom-163598m05672.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-post-custom-163598m05672.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-masked-pos-custom-163598m05672.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-masked-neg-custom-163598m05672.jpg
Stage outliers : Wall: 0.75 s, CPU: 0.55 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 208 MB, VmData: 236 MB, maxrss: 0.213184 MB
Stage outliers finished: 2020-08-05T18:47:25.488980
Resources for stage outliers : 0.638 s worker CPU, 0.761 s worker Wall, Wall: 0.751 s, Cores in use: 1.58, Total efficiency (on 16 cores): 9.9 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T18:47:25.511228
Stage halos : Wall: 0.21 s, CPU: 0.20 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 208 MB, VmData: 236 MB, maxrss: 0.213384 MB
Stage halos finished: 2020-08-05T18:47:25.717231
Resources for stage halos : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.207 s, Cores in use: 0.99, Total efficiency (on 16 cores): 6.2 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-05T18:47:25.739593
PSF sigmas in coadd pixels: ['2.35', '1.95', '2.22']
Using average PSF sigma 2.1737905356269915
Gaussian PSF norm 0.129770917275851 vs pixelized 0.11235141
average tim pixel scale / coadd scale: 1.0011520170969248
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.80', '1.68', '2.04']
Using average PSF sigma 1.836969071120912
Gaussian PSF norm 0.15356534642237876 vs pixelized 0.1351989
average tim pixel scale / coadd scale: 1.0011852754436377
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.48', '1.73', '2.05', '1.87']
Using average PSF sigma 2.0318648714128495
Gaussian PSF norm 0.13883540964893232 vs pixelized 0.11096806
average tim pixel scale / coadd scale: 1.0011938899552861
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.14 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 210 MB, VmData: 236 MB, maxrss: 0.215132 MB
Stage fit_on_coadds finished: 2020-08-05T18:47:25.915163
Resources for stage fit_on_coadds : 0.141 s worker CPU, 0.141 s worker Wall, Wall: 0.177 s, Cores in use: 1.59, Total efficiency (on 16 cores): 10.0 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T18:47:25.924589
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 23 potential peaks
Of 23 potential peaks: 7 in veto map, 5 cut by saddle test, 0 cut by aper test, 11 kept
SED z : 11 new peaks
Found 24 potential peaks
Of 24 potential peaks: 14 in veto map, 5 cut by saddle test, 0 cut by aper test, 5 kept
SED r : 5 new peaks
Found 17 potential peaks
Of 17 potential peaks: 13 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept
SED g : 1 new peaks
Found 23 potential peaks
Of 23 potential peaks: 17 in veto map, 3 cut by saddle test, 0 cut by aper test, 3 kept
SED Flat : 3 new peaks
Found 28 potential peaks
Of 28 potential peaks: 24 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 20 new sources
Median coadd sky for g : -1.3269377e-05
Median coadd sky for r : 1.536508e-05
Median coadd sky for z : 0.00022239046
Stage srcs : Wall: 0.50 s, CPU: 0.47 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 211 MB, VmData: 236 MB, maxrss: 0.21556 MB
Stage srcs finished: 2020-08-05T18:47:26.420987
Resources for stage srcs : 0.061 s worker CPU, 0.062 s worker Wall, Wall: 0.498 s, Cores in use: 1.07, Total efficiency (on 16 cores): 6.7 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T18:47:26.452194
Blob 1 of 11, id: 5, sources: 7, size: 190x117, npix 13116, brick X: 43,233, Y: 74,191, one pixel: 96 74
Blob 2 of 11, id: 7, sources: 2, size: 58x62, npix 2469, brick X: 71,129, Y: 171,233, one pixel: 114 171
Blob 3 of 11, id: 3, sources: 5, size: 64x57, npix 1730, brick X: 0,64, Y: 41,98, one pixel: 31 41
Blob 4 of 11, id: 6, sources: 1, size: 27x29, npix 555, brick X: 0,27, Y: 132,161, one pixel: 9 132
Blob 5 of 11, id: 1, sources: 1, size: 28x28, npix 489, brick X: 12,40, Y: 8,36, one pixel: 24 8
Blob 6 of 11, id: 0, sources: 1, size: 29x22, npix 467, brick X: 59,88, Y: 0,22, one pixel: 65 0
Blob 7 of 11, id: 9, sources: 1, size: 25x23, npix 381, brick X: 35,60, Y: 189,212, one pixel: 44 189
Blob 8 of 11, id: 4, sources: 1, size: 22x23, npix 293, brick X: 144,166, Y: 64,87, one pixel: 155 64
Blob 9 of 11, id: 8, sources: 1, size: 21x20, npix 229, brick X: 186,207, Y: 179,199, one pixel: 196 179
Blob 10 of 11, id: 10, sources: 1, size: 19x19, npix 208, brick X: 141,160, Y: 196,215, one pixel: 149 196
Blob 11 of 11, id: 2, sources: 1, size: 17x18, npix 162, brick X: 49,66, Y: 32,50, one pixel: 57 32
Blob 11 finished, total: Wall: 0.09 s, CPU: 0.09 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 87 MB, VmData: 140 MB, maxrss: 0.088992 MB
Blob 10 finished, total: Wall: 0.38 s, CPU: 0.38 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092176 MB
Blob 7 finished, total: Wall: 0.53 s, CPU: 0.53 s, VmPeak: 941 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148036 MB
Blob 4 finished, total: Wall: 0.65 s, CPU: 0.65 s, VmPeak: 939 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148232 MB
Blob 6 finished, total: Wall: 0.80 s, CPU: 0.80 s, VmPeak: 939 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148864 MB
Blob 9 finished, total: Wall: 0.80 s, CPU: 0.80 s, VmPeak: 940 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148804 MB
Blob 8 finished, total: Wall: 0.84 s, CPU: 0.84 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093232 MB
Blob 5 finished, total: Wall: 0.90 s, CPU: 0.90 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092772 MB
Blob 3 finished, total: Wall: 1.83 s, CPU: 1.83 s, VmPeak: 946 MB, VmSize: 733 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148324 MB
Measuring 3 iterative sources
Blob 2 finished, total: Wall: 3.94 s, CPU: 3.94 s, VmPeak: 946 MB, VmSize: 733 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149828 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.71693, ee1=-0.00242661, ee2=-0.24398
Measuring 2 iterative sources
Blob 1 finished, total: Wall: 11.13 s, CPU: 11.12 s, VmPeak: 687 MB, VmSize: 687 MB, VmRSS: 113 MB, VmData: 158 MB, maxrss: 0.116224 MB
Old catalog: 23
New catalog: 24
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/blobs-custom-163598m05672.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/all-models-custom-163598m05672.fits
Stage fitblobs : Wall: 11.75 s, CPU: 0.59 s, VmPeak: 1042 MB, VmSize: 1036 MB, VmRSS: 210 MB, VmData: 235 MB, maxrss: 0.215756 MB
Stage fitblobs finished: 2020-08-05T18:47:38.205299
Resources for stage fitblobs : 22.022 s worker CPU, 22.221 s worker Wall, Wall: 11.755 s, Cores in use: 1.92, Total efficiency (on 16 cores): 12.0 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T18:47:38.216194
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-resid.jpg
Stage coadds : Wall: 2.29 s, CPU: 0.48 s, VmPeak: 1042 MB, VmSize: 1036 MB, VmRSS: 210 MB, VmData: 235 MB, maxrss: 0.215756 MB
Stage coadds finished: 2020-08-05T18:47:40.507260
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 10.667 s worker CPU, 14.888 s worker Wall, Wall: 2.293 s, Cores in use: 4.86, Total efficiency (on 16 cores): 30.4 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T18:47:40.517791
Cut to 1 unWISE tiles
Reading WISE tile 1640m061 band 1
Reading WISE tile 1640m061 band 2
Reading WISE tile 1640m061 band 4
Reading WISE tile 1640m061 band 3
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 943 MB, VmSize: 807 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.187336 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 939 MB, VmSize: 802 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.203604 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 939 MB, VmSize: 807 MB, VmRSS: 146 MB, VmData: 172 MB, maxrss: 0.209972 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 941 MB, VmSize: 870 MB, VmRSS: 209 MB, VmData: 235 MB, maxrss: 0.21364 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 17.20 s, CPU: 0.38 s, VmPeak: 1042 MB, VmSize: 1036 MB, VmRSS: 211 MB, VmData: 235 MB, maxrss: 0.215952 MB
Stage wise_forced finished: 2020-08-05T18:47:57.713422
Resources for stage wise_forced : 2.987 s worker CPU, 20.079 s worker Wall, Wall: 17.197 s, Cores in use: 0.20, Total efficiency (on 16 cores): 1.2 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T18:47:57.727053
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/tractor-i/cus/tractor-custom-163598m05672.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/tractor/cus/tractor-custom-163598m05672.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/tractor/cus/brick-custom-163598m05672.sha256sum
Stage writecat : Wall: 0.77 s, CPU: 0.56 s, VmPeak: 1169 MB, VmSize: 1041 MB, VmRSS: 213 MB, VmData: 235 MB, maxrss: 0.348956 MB
Stage writecat finished: 2020-08-05T18:47:58.493540
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.768 s, Cores in use: 0.73, Total efficiency (on 16 cores): 4.6 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-writecat.p
All done: 42.306 s worker CPU, 998.823 s worker Wall, Wall: 163.421 s, Cores in use: 0.30, Total efficiency (on 16 cores): 1.9 %
Finished galaxy PGC1041779 in 2.751 minutes.
Started working on galaxy PGC1041779 at Wed Aug  5 18:58:41 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 163.5982785 -5.6726454 --width 233 --height 233 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T18:58:43.210769
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '163.5982785', '-5.6726454', '--width', '233', '--height', '233', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 163.5982785 -5.6726454 --width 233 --height 233 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 163.5982785 -5.6726454
Starting process 5043 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 5045 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 5047 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 5050 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 5051 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05948 MB
Starting process 5054 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB
Starting process 5055 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB
Starting process 5057 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 5059 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 5061 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 5063 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 5065 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 5067 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 5069 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 5071 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 5074 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 5077 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 5078 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 5079 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 5080 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 5081 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 5083 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 5084 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 5086 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 5087 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 5088 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 5089 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 5090 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB
Starting process 5091 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 5092 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 5093 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 5094 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T18:58:44.400680
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 10 CCDs:
  c4d_170327_005523_ooi_r_ls9-S19 r expnum 633964 exptime 64.0 propid 2014B-0404 seeing 1.11 MJD 57839.037 object DECaLS_23011_r 
    decam/CP/V4.8.2/CP20170326/c4d_170327_005523_ooi_r_ls9.fits.fz
  c4d_170328_023211_ooi_g_ls9-S19 g expnum 634385 exptime 86.0 propid 2014B-0404 seeing 1.44 MJD 57840.104 object DECaLS_23011_g 
    decam/CP/V4.8.2/CP20170327/c4d_170328_023211_ooi_g_ls9.fits.fz
  c4d_170304_010835_ooi_z_ls9-S19 z expnum 625641 exptime 203.0 propid 2014B-0404 seeing 1.53 MJD 57816.045 object DECaLS_23011_z 
    decam/CP/V4.8.2/CP20170303/c4d_170304_010835_ooi_z_ls9.fits.fz
  c4d_181215_064155_ooi_r_ls9-S18 r expnum 803328 exptime 47.0 propid 2014B-0404 seeing 1.03 MJD 58467.279 object DECaLS_38883_r 
    decam/CP/V4.8.2a/CP20181214/c4d_181215_064155_ooi_r_ls9.fits.fz
  c4d_181215_064312_ooi_g_ls9-S18 g expnum 803329 exptime 95.0 propid 2014B-0404 seeing 1.20 MJD 58467.280 object DECaLS_38883_g 
    decam/CP/V4.8.2a/CP20181214/c4d_181215_064312_ooi_g_ls9.fits.fz
  c4d_170306_010031_ooi_z_ls9-S18 z expnum 626329 exptime 162.0 propid 2014B-0404 seeing 1.07 MJD 57818.040 object DECaLS_38883_z 
    decam/CP/V4.8.2/CP20170305/c4d_170306_010031_ooi_z_ls9.fits.fz
  c4d_170301_045711_ooi_g_ls9-S8 g expnum 624537 exptime 123.0 propid 2014B-0404 seeing 1.37 MJD 57813.204 object DECaLS_7125_g 
    decam/CP/V4.8.2/CP20170228/c4d_170301_045711_ooi_g_ls9.fits.fz
  c4d_170301_045436_ooi_r_ls9-S8 r expnum 624536 exptime 48.0 propid 2014B-0404 seeing 1.25 MJD 57813.203 object DECaLS_7125_r 
    decam/CP/V4.8.2/CP20170228/c4d_170301_045436_ooi_r_ls9.fits.fz
  c4d_190206_060626_ooi_z_ls9-S8 z expnum 820178 exptime 147.0 propid 2014B-0404 seeing 1.26 MJD 58520.254 object DECaLS_7125_z 
    decam/CP/V4.8.2a/CP20190205/c4d_190206_060626_ooi_z_ls9.fits.fz
  c4d_151227_061850_ooi_z_ls9-S8 z expnum 506085 exptime 89.0 propid 2014B-0404 seeing 1.15 MJD 57383.261 object DECaLS_7125_z 
    decam/CP/V4.8.2a/CP20151226/c4d_151227_061850_ooi_z_ls9.fits.fz
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950bfa6-z-20170304T010422_oi[S19,noinherit][2046,4094][real]: DECaLS_
Masking 0 additional saturated pixels in DECam expnum 625641 S19 CCD, decam/CP/V4.8.2/CP20170303/c4d_170304_010835_ooi_z_ls9.fits.fz slice (slice(1617, 1851, None), slice(1493, 1726, None))
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95127dc-g-20170328T022958_oi[S19,noinherit][2046,4094][real]: DECaLS_
Clipping 7 pixels with anomalously large oow values: max 0.0553754 vs median 0.0377271
Masking 0 additional saturated pixels in DECam expnum 634385 S19 CCD, decam/CP/V4.8.2/CP20170327/c4d_170328_023211_ooi_g_ls9.fits.fz slice (slice(1604, 1838, None), slice(1497, 1731, None))
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953988f-g-20181215T064312_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_951060a-r-20170327T005328_oi[S19,noinherit][2046,4094][real]: DECaLS_
Masking 0 additional saturated pixels in DECam expnum 633964 S19 CCD, decam/CP/V4.8.2/CP20170326/c4d_170327_005523_ooi_r_ls9.fits.fz slice (slice(1601, 1835, None), slice(1505, 1738, None))
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c76c-r-20170301T045259_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539067-r-20181215T064155_oi[S18,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c43f-g-20170301T045416_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95527d4-z-20151227T061631_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9534704-z-20190206T060626_oi[S8,noinherit][2046,4094][real]: DECaLS_7
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950dd28-z-20170306T005702_oi[S18,noinherit][2046,4094][real]: DECaLS_
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 41.14 s, CPU: 1.14 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 177 MB, VmData: 209 MB, maxrss: 0.181496 MB
Stage tims finished: 2020-08-05T18:59:25.541684
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95127dc-g-20170328T022958_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_953988f-g-20181215T064312_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c76c-r-20170301T045259_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95527d4-z-20151227T061631_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9539067-r-20181215T064155_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 5.503 s worker CPU, 341.775 s worker Wall, Wall: 41.144 s, Cores in use: 0.16, Total efficiency (on 32 cores): 0.5 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-tims.p
Running stage refs
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950bfa6-z-20170304T010422_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9534704-z-20190206T060626_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950dd28-z-20170306T005702_oi[S18,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_951060a-r-20170327T005328_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T18:59:33.691521
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06650.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06778.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/reference-custom-163598m05672.fits
Stage refs : Wall: 10.68 s, CPU: 1.81 s, VmPeak: 1042 MB, VmSize: 1028 MB, VmRSS: 195 MB, VmData: 226 MB, maxrss: 0.199236 MB
Stage refs finished: 2020-08-05T18:59:44.372729
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_950c43f-g-20170301T045416_oi[S8,noinherit][2046,4094][real]: DECaLS_7 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 10.684 s, Cores in use: 0.17, Total efficiency (on 32 cores): 0.5 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T18:59:44.461187
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outlier-mask-custom-163598m05672.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-pre-custom-163598m05672.jpg
Image c4d_170306_010031_ooi_z_ls9-S18 z : masking 277 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outlier-mask-custom-163598m05672.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-post-custom-163598m05672.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-masked-pos-custom-163598m05672.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/outliers-masked-neg-custom-163598m05672.jpg
Stage outliers : Wall: 0.99 s, CPU: 0.69 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 208 MB, VmData: 235 MB, maxrss: 0.212496 MB
Stage outliers finished: 2020-08-05T18:59:45.452177
Resources for stage outliers : 0.732 s worker CPU, 0.937 s worker Wall, Wall: 0.993 s, Cores in use: 1.43, Total efficiency (on 32 cores): 4.5 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T18:59:45.479358
Stage halos : Wall: 0.45 s, CPU: 0.45 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 208 MB, VmData: 236 MB, maxrss: 0.212592 MB
Stage halos finished: 2020-08-05T18:59:45.928653
Resources for stage halos : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 0.452 s, Cores in use: 1.00, Total efficiency (on 32 cores): 3.1 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-05T18:59:45.969953
PSF sigmas in coadd pixels: ['2.35', '1.95', '2.22']
Using average PSF sigma 2.1737905356269915
Gaussian PSF norm 0.129770917275851 vs pixelized 0.11235142
average tim pixel scale / coadd scale: 1.0011520170969248
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.80', '1.68', '2.04']
Using average PSF sigma 1.836969071120912
Gaussian PSF norm 0.15356534642237876 vs pixelized 0.1351989
average tim pixel scale / coadd scale: 1.0011852754436377
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.48', '1.73', '2.05', '1.87']
Using average PSF sigma 2.0318648714128495
Gaussian PSF norm 0.13883540964893232 vs pixelized 0.110968046
average tim pixel scale / coadd scale: 1.0011938899552861
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.16 s, CPU: 0.15 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 209 MB, VmData: 236 MB, maxrss: 0.214308 MB
Stage fit_on_coadds finished: 2020-08-05T18:59:46.126165
Resources for stage fit_on_coadds : 0.147 s worker CPU, 0.148 s worker Wall, Wall: 0.159 s, Cores in use: 1.88, Total efficiency (on 32 cores): 5.9 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T18:59:46.138018
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 23 potential peaks
Of 23 potential peaks: 7 in veto map, 5 cut by saddle test, 0 cut by aper test, 11 kept
SED z : 11 new peaks
Found 24 potential peaks
Of 24 potential peaks: 14 in veto map, 5 cut by saddle test, 0 cut by aper test, 5 kept
SED r : 5 new peaks
Found 17 potential peaks
Of 17 potential peaks: 13 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept
SED g : 1 new peaks
Found 23 potential peaks
Of 23 potential peaks: 17 in veto map, 3 cut by saddle test, 0 cut by aper test, 3 kept
SED Flat : 3 new peaks
Found 28 potential peaks
Of 28 potential peaks: 24 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 20 new sources
Median coadd sky for g : -1.3269377e-05
Median coadd sky for r : 1.536508e-05
Median coadd sky for z : 0.00022239053
Stage srcs : Wall: 0.50 s, CPU: 0.47 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 210 MB, VmData: 236 MB, maxrss: 0.214736 MB
Stage srcs finished: 2020-08-05T18:59:46.639122
Resources for stage srcs : 0.061 s worker CPU, 0.061 s worker Wall, Wall: 0.503 s, Cores in use: 1.07, Total efficiency (on 32 cores): 3.3 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T18:59:46.651695
Blob 1 of 11, id: 5, sources: 7, size: 190x117, npix 13116, brick X: 43,233, Y: 74,191, one pixel: 96 74
Blob 2 of 11, id: 7, sources: 2, size: 58x62, npix 2469, brick X: 71,129, Y: 171,233, one pixel: 114 171
Blob 3 of 11, id: 3, sources: 5, size: 64x57, npix 1730, brick X: 0,64, Y: 41,98, one pixel: 31 41
Blob 4 of 11, id: 6, sources: 1, size: 27x29, npix 555, brick X: 0,27, Y: 132,161, one pixel: 9 132
Blob 5 of 11, id: 1, sources: 1, size: 28x28, npix 489, brick X: 12,40, Y: 8,36, one pixel: 24 8
Blob 6 of 11, id: 0, sources: 1, size: 29x22, npix 467, brick X: 59,88, Y: 0,22, one pixel: 65 0
Blob 7 of 11, id: 9, sources: 1, size: 25x23, npix 381, brick X: 35,60, Y: 189,212, one pixel: 44 189
Blob 8 of 11, id: 4, sources: 1, size: 22x23, npix 293, brick X: 144,166, Y: 64,87, one pixel: 155 64
Blob 9 of 11, id: 8, sources: 1, size: 21x20, npix 229, brick X: 186,207, Y: 179,199, one pixel: 196 179
Blob 10 of 11, id: 10, sources: 1, size: 19x19, npix 208, brick X: 141,160, Y: 196,215, one pixel: 149 196
Blob 11 of 11, id: 2, sources: 1, size: 17x18, npix 162, brick X: 49,66, Y: 32,50, one pixel: 57 32
Blob 11 finished, total: Wall: 0.10 s, CPU: 0.10 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 87 MB, VmData: 140 MB, maxrss: 0.088924 MB
Blob 10 finished, total: Wall: 0.29 s, CPU: 0.28 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092588 MB
Blob 7 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 939 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148256 MB
Blob 9 finished, total: Wall: 0.59 s, CPU: 0.58 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092856 MB
Blob 4 finished, total: Wall: 0.67 s, CPU: 0.66 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.09344 MB
Blob 8 finished, total: Wall: 0.66 s, CPU: 0.66 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094452 MB
Blob 5 finished, total: Wall: 0.77 s, CPU: 0.77 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094164 MB
Blob 6 finished, total: Wall: 0.82 s, CPU: 0.82 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093324 MB
Blob 3 finished, total: Wall: 1.83 s, CPU: 1.83 s, VmPeak: 939 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148636 MB
Measuring 3 iterative sources
Blob 2 finished, total: Wall: 3.64 s, CPU: 3.64 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.09654 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.71693, ee1=-0.00242668, ee2=-0.24398
Measuring 2 iterative sources
Blob 1 finished, total: Wall: 11.44 s, CPU: 11.43 s, VmPeak: 687 MB, VmSize: 687 MB, VmRSS: 113 MB, VmData: 158 MB, maxrss: 0.115912 MB
Old catalog: 23
New catalog: 24
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/blobs-custom-163598m05672.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/metrics/cus/all-models-custom-163598m05672.fits
Stage fitblobs : Wall: 11.90 s, CPU: 0.45 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 210 MB, VmData: 236 MB, maxrss: 0.214932 MB
Stage fitblobs finished: 2020-08-05T18:59:58.554665
Resources for stage fitblobs : 21.472 s worker CPU, 21.534 s worker Wall, Wall: 11.905 s, Cores in use: 1.84, Total efficiency (on 32 cores): 5.8 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T18:59:58.571643
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-resid.jpg
Stage coadds : Wall: 3.63 s, CPU: 0.89 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 210 MB, VmData: 236 MB, maxrss: 0.215448 MB
Stage coadds finished: 2020-08-05T19:00:02.204664
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 27.507 s worker CPU, 48.746 s worker Wall, Wall: 3.635 s, Cores in use: 7.81, Total efficiency (on 32 cores): 24.4 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T19:00:02.221520
Cut to 1 unWISE tiles
Reading WISE tile 1640m061 band 1
Reading WISE tile 1640m061 band 2
Reading WISE tile 1640m061 band 3
Reading WISE tile 1640m061 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 939 MB, VmSize: 807 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.187608 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 939 MB, VmSize: 806 MB, VmRSS: 146 MB, VmData: 171 MB, maxrss: 0.20976 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 940 MB, VmSize: 810 MB, VmRSS: 150 MB, VmData: 179 MB, maxrss: 0.205472 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/164/1640m061/unwise-1640m061-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 940 MB, VmSize: 801 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.202896 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/tmp-legacysurvey-custom-163598m05672-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 5.58 s, CPU: 0.79 s, VmPeak: 1042 MB, VmSize: 1037 MB, VmRSS: 211 MB, VmData: 236 MB, maxrss: 0.216152 MB
Stage wise_forced finished: 2020-08-05T19:00:07.800755
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage wise_forced : 2.977 s worker CPU, 7.283 s worker Wall, Wall: 5.582 s, Cores in use: 0.68, Total efficiency (on 32 cores): 2.1 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T19:00:07.820192
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/coadd/cus/custom-163598m05672/legacysurvey-custom-163598m05672-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/tractor-i/cus/tractor-custom-163598m05672.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/tractor/cus/tractor-custom-163598m05672.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/tractor/cus/brick-custom-163598m05672.sha256sum
Stage writecat : Wall: 1.92 s, CPU: 0.87 s, VmPeak: 1170 MB, VmSize: 1042 MB, VmRSS: 213 MB, VmData: 236 MB, maxrss: 0.349156 MB
Stage writecat finished: 2020-08-05T19:00:09.735568
Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 1.918 s, Cores in use: 0.46, Total efficiency (on 32 cores): 1.4 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/163/PGC1041779/PGC1041779-largegalaxy-writecat.p
All done: 58.407 s worker CPU, 420.493 s worker Wall, Wall: 85.373 s, Cores in use: 0.78, Total efficiency (on 32 cores): 2.4 %
Finished galaxy PGC1041779 in 1.640 minutes.