Started working on galaxy PGC2668065 at Wed Aug  5 19:17:59 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 243.293466 64.4350052 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T19:18:03.584650
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '243.293466', '64.4350052', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 243.293466 64.4350052 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 243.293466 64.4350052
Starting process 18703 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 18704 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 18705 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 18706 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 18707 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 18708 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 18709 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 18710 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 18711 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 18712 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 18713 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 18714 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 18715 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 18716 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 18717 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 18718 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T19:18:05.200874
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 7 CCDs:
  k4m_170521_091733_ooi_zd_ls9-CCD2 z expnum 129236 exptime 84.104 propid 2016A-0453 seeing 1.03 MJD 57894.387 object MzLS_699306_z 
    mosaic/CP/V4.3/CP20170520/k4m_170521_091733_ooi_zd_ls9.fits.fz
  k4m_170805_060522_ooi_zd_ls9-CCD1 z expnum 3135769 exptime 124.106 propid 2016A-0453 seeing 1.10 MJD 57970.254 object MzLS_700397_z 
    mosaic/CP/V4.3/CP20170804/k4m_170805_060522_ooi_zd_ls9.fits.fz
  k4m_170417_111220_ooi_zd_ls9-CCD2 z expnum 122676 exptime 80.103 propid 2016A-0453 seeing 1.09 MJD 57860.467 object MzLS_699853_z 
    mosaic/CP/V4.3/CP20170416/k4m_170417_111220_ooi_zd_ls9.fits.fz
  ksb_180416_081709_ooi_g_ls9-CCD2 g expnum 82240105 exptime 123.66 propid BASS seeing 1.51 MJD 58224.345 object 42213 
    90prime/CP/V2.3/CP20180415/ksb_180416_081709_ooi_g_ls9.fits.fz
  ksb_180420_101554_ooi_r_ls9-CCD2 r expnum 82280082 exptime 250.0 propid BASS seeing 2.70 MJD 58228.428 object 42213 
    90prime/CP/V2.3/CP20180419/ksb_180420_101554_ooi_r_ls9.fits.fz
  ksb_170403_104948_ooi_g_ls9-CCD3 g expnum 78460195 exptime 103.065 propid BASS seeing 1.45 MJD 57846.451 object 41342 
    90prime/CP/V2.3/CP20170402/ksb_170403_104948_ooi_g_ls9.fits.fz
  ksb_170531_092633_ooi_r_ls9-CCD3 r expnum 79040107 exptime 250.0 propid 2015A-0801 seeing 1.60 MJD 57904.393 object 41342 
    90prime/CP/V2.3/CP20170530/ksb_170531_092633_ooi_r_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 3.95
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2539 2784 and amp range 2048 4095 and image y 1867 2110 and amp range 0 2039
Range in image: x 0 246 , y 0 173 (with image size 246 x 244)
dzp 0.0044767857 -> scaling image by 1.004131784351374
Found overlap: image x 2539 2784 and amp range 2048 4095 and image y 1867 2110 and amp range 2040 4078
Range in image: x 0 246 , y 173 244 (with image size 246 x 244)
dzp -0.032976374 -> scaling image by 0.9700842429001929
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 78460195
Faked up EXPNUM 78460195
Faked up EXPNUM 78460195
90prime get_fwhm: self.fwhm = 3.21
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2596 2735 and amp range 2016 4031 and image y 1636 1775 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp -3.875605e-05 -> scaling image by 0.9999643049955675
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 4.19
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 7 86 and amp range 0 2047 and image y 1319 1561 and amp range 0 2039
Range in image: x 0 80 , y 0 243 (with image size 80 x 243)
dzp -0.0096683735 -> scaling image by 0.9911346301237048
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82240105
Faked up EXPNUM 82240105
Faked up EXPNUM 82240105
90prime get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2333 2473 and amp range 2016 4031 and image y 2836 2977 and amp range 2048 4095
Range in image: x 0 141 , y 0 142 (with image size 141 x 142)
dzp 0.013237011 -> scaling image by 1.012266359574015
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 4.25
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1416 1659 and amp range 0 2005 and image y 2161 2404 and amp range 2040 4078
Range in image: x 0 244 , y 0 244 (with image size 244 x 244)
dzp 0.014419514 -> scaling image by 1.0133694453441033
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82280082
Faked up EXPNUM 82280082
Faked up EXPNUM 82280082
90prime get_fwhm: self.fwhm = 5.95
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2339 2480 and amp range 2016 4031 and image y 2842 2983 and amp range 2048 4095
Range in image: x 0 142 , y 0 142 (with image size 142 x 142)
dzp 0.019008487 -> scaling image by 1.0176616178968958
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79040107
Faked up EXPNUM 79040107
Faked up EXPNUM 79040107
90prime get_fwhm: self.fwhm = 3.53
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2599 2738 and amp range 2016 4031 and image y 1635 1773 and amp range 0 2047
Range in image: x 0 140 , y 0 139 (with image size 140 x 139)
dzp -0.0023383768 -> scaling image by 0.9978485929508539
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 24.35 s, CPU: 1.02 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 172 MB, VmData: 205 MB, maxrss: 0.176532 MB
Stage tims finished: 2020-08-05T19:18:29.548047
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 3.389 s worker CPU, 44.796 s worker Wall, Wall: 24.349 s, Cores in use: 0.18, Total efficiency (on 16 cores): 1.1 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-tims.p
Running stage refs
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T19:18:29.564111
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00660.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00589.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00590.fits
Matched 16 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/reference-custom-243293p64435.fits
Stage refs : Wall: 13.31 s, CPU: 3.96 s, VmPeak: 987 MB, VmSize: 976 MB, VmRSS: 193 MB, VmData: 224 MB, maxrss: 0.197424 MB
Stage refs finished: 2020-08-05T19:18:42.875683
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 13.313 s, Cores in use: 0.30, Total efficiency (on 16 cores): 1.9 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T19:18:42.913823
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outlier-mask-custom-243293p64435.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-pre-custom-243293p64435.jpg
Image k4m_170521_091733_ooi_zd_ls9-CCD2 z : masking 215 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outlier-mask-custom-243293p64435.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-post-custom-243293p64435.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-masked-pos-custom-243293p64435.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-masked-neg-custom-243293p64435.jpg
Stage outliers : Wall: 1.66 s, CPU: 1.26 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 204 MB, VmData: 232 MB, maxrss: 0.20912 MB
Stage outliers finished: 2020-08-05T19:18:44.571160
Resources for stage outliers : 0.444 s worker CPU, 0.523 s worker Wall, Wall: 1.659 s, Cores in use: 1.03, Total efficiency (on 16 cores): 6.4 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T19:18:44.589788
Stage halos : Wall: 0.99 s, CPU: 0.93 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 205 MB, VmData: 232 MB, maxrss: 0.209428 MB
Stage halos finished: 2020-08-05T19:18:45.583466
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.995 s, Cores in use: 0.94, Total efficiency (on 16 cores): 5.9 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T19:18:45.606315
PSF sigmas in coadd pixels: ['2.45', '2.36']
Using average PSF sigma 2.4076392722627125
Gaussian PSF norm 0.11716655190989801 vs pixelized 0.09443834
average tim pixel scale / coadd scale: 1.7302535573831042
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['4.38', '2.60']
Using average PSF sigma 3.4902754164500287
Gaussian PSF norm 0.08082307500558158 vs pixelized 0.05897451
average tim pixel scale / coadd scale: 1.7301686786164934
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.67', '1.79', '1.77']
Using average PSF sigma 1.7415192061007227
Gaussian PSF norm 0.161982015923609 vs pixelized 0.13587123
average tim pixel scale / coadd scale: 0.9909378389564861
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.14 s, CPU: 0.10 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 206 MB, VmData: 232 MB, maxrss: 0.21102 MB
Stage fit_on_coadds finished: 2020-08-05T19:18:45.744022
Resources for stage fit_on_coadds : 0.110 s worker CPU, 0.110 s worker Wall, Wall: 0.139 s, Cores in use: 1.51, Total efficiency (on 16 cores): 9.4 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T19:18:45.757338
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 16 potential peaks
Of 16 potential peaks: 3 in veto map, 1 cut by saddle test, 0 cut by aper test, 12 kept
SED z : 12 new peaks
Found 10 potential peaks
Of 10 potential peaks: 10 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED r : 0 new peaks
Found 11 potential peaks
Of 11 potential peaks: 11 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 16 potential peaks
Of 16 potential peaks: 13 in veto map, 2 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 17 potential peaks
Of 17 potential peaks: 15 in veto map, 2 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 13 new sources
Median coadd sky for g : 0.000114623486
Median coadd sky for r : 0.00044442946
Median coadd sky for z : -0.00010727634
Stage srcs : Wall: 1.23 s, CPU: 1.18 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 206 MB, VmData: 232 MB, maxrss: 0.211404 MB
Stage srcs finished: 2020-08-05T19:18:46.987713
Resources for stage srcs : 0.083 s worker CPU, 0.083 s worker Wall, Wall: 1.232 s, Cores in use: 1.03, Total efficiency (on 16 cores): 6.4 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T19:18:47.001274
Blob 1 of 7, id: 3, sources: 3, size: 115x106, npix 9134, brick X: 66,181, Y: 64,170, one pixel: 113 64
Blob 2 of 7, id: 4, sources: 4, size: 83x83, npix 3404, brick X: 38,121, Y: 156,239, one pixel: 52 156
Blob 3 of 7, id: 5, sources: 2, size: 66x61, npix 2521, brick X: 122,188, Y: 167,228, one pixel: 156 167
Blob 4 of 7, id: 1, sources: 3, size: 44x61, npix 1595, brick X: 195,239, Y: 32,93, one pixel: 205 32
Blob 5 of 7, id: 0, sources: 2, size: 52x43, npix 1240, brick X: 60,112, Y: 19,62, one pixel: 71 19
Blob 6 of 7, id: 6, sources: 1, size: 25x24, npix 384, brick X: 191,216, Y: 191,215, one pixel: 201 191
Blob 7 of 7, id: 2, sources: 1, size: 21x23, npix 279, brick X: 139,160, Y: 45,68, one pixel: 148 45
Blob 7 finished, total: Wall: 0.05 s, CPU: 0.05 s, VmPeak: 808 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149492 MB
Blob 6 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 787 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 170 MB, maxrss: 0.136724 MB
Blob 5 finished, total: Wall: 0.82 s, CPU: 0.82 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094696 MB
Blob 3 finished, total: Wall: 1.29 s, CPU: 1.28 s, VmPeak: 669 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.094628 MB
Measuring 1 iterative sources
Blob 4 finished, total: Wall: 1.43 s, CPU: 1.42 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094164 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 2.13 s, CPU: 2.12 s, VmPeak: 803 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150964 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.48958, ee1=-0.0413716, ee2=-0.104729
Blob 1 finished, total: Wall: 2.90 s, CPU: 2.89 s, VmPeak: 803 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.1508 MB
Old catalog: 21
New catalog: 15
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/blobs-custom-243293p64435.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/all-models-custom-243293p64435.fits
Stage fitblobs : Wall: 3.48 s, CPU: 0.53 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 207 MB, VmData: 232 MB, maxrss: 0.211568 MB
Stage fitblobs finished: 2020-08-05T19:18:50.478495
Resources for stage fitblobs : 8.909 s worker CPU, 8.969 s worker Wall, Wall: 3.479 s, Cores in use: 2.71, Total efficiency (on 16 cores): 17.0 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T19:18:50.943289
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-resid.jpg
Stage coadds : Wall: 3.54 s, CPU: 1.00 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 207 MB, VmData: 232 MB, maxrss: 0.212084 MB
Stage coadds finished: 2020-08-05T19:18:54.486011
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 10.553 s worker CPU, 18.664 s worker Wall, Wall: 3.544 s, Cores in use: 3.26, Total efficiency (on 16 cores): 20.4 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T19:18:54.507902
Cut to 1 unWISE tiles
Reading WISE tile 2446p651 band 1
Reading WISE tile 2446p651 band 4
Reading WISE tile 2446p651 band 2
Reading WISE tile 2446p651 band 3
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 808 MB, VmSize: 804 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.18638 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 808 MB, VmSize: 806 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.211632 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 813 MB, VmSize: 798 MB, VmRSS: 140 MB, VmData: 170 MB, maxrss: 0.203564 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.02 s, CPU: 0.02 s, VmPeak: 809 MB, VmSize: 799 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.203436 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 4.70 s, CPU: 1.72 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 208 MB, VmData: 232 MB, maxrss: 0.212788 MB
Stage wise_forced finished: 2020-08-05T19:18:59.212709
Resources for stage wise_forced : 2.984 s worker CPU, 6.932 s worker Wall, Wall: 4.706 s, Cores in use: 1.00, Total efficiency (on 16 cores): 6.2 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T19:18:59.226309
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/tractor-i/cus/tractor-custom-243293p64435.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/tractor/cus/tractor-custom-243293p64435.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/tractor/cus/brick-custom-243293p64435.sha256sum
Stage writecat : Wall: 0.81 s, CPU: 0.54 s, VmPeak: 1118 MB, VmSize: 990 MB, VmRSS: 209 MB, VmData: 232 MB, maxrss: 0.345584 MB
Stage writecat finished: 2020-08-05T19:19:00.041288
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.817 s, Cores in use: 0.66, Total efficiency (on 16 cores): 4.1 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-writecat.p
All done: 26.474 s worker CPU, 80.081 s worker Wall, Wall: 54.857 s, Cores in use: 0.71, Total efficiency (on 16 cores): 4.4 %
Finished galaxy PGC2668065 in 1.018 minutes.
Started working on galaxy PGC2668065 at Wed Aug  5 19:27:11 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 243.293466 64.4350052 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T19:27:14.033771
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '243.293466', '64.4350052', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 243.293466 64.4350052 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 243.293466 64.4350052
Starting process 20743 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 20744 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 20745 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 20746 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 20747 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05948 MB
Starting process 20748 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB
Starting process 20749 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB
Starting process 20750 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 20751 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 20752 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 20753 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 20754 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 20755 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 20756 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 20757 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 20758 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 20759 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 20760 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 20761 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB
Starting process 20762 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 20763 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 20764 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 20765 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 20766 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 20767 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 20768 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 20769 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 20770 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB
Starting process 20771 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 20772 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 20773 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 20774 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T19:27:15.334022
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 7 CCDs:
  k4m_170521_091733_ooi_zd_ls9-CCD2 z expnum 129236 exptime 84.104 propid 2016A-0453 seeing 1.03 MJD 57894.387 object MzLS_699306_z 
    mosaic/CP/V4.3/CP20170520/k4m_170521_091733_ooi_zd_ls9.fits.fz
  k4m_170805_060522_ooi_zd_ls9-CCD1 z expnum 3135769 exptime 124.106 propid 2016A-0453 seeing 1.10 MJD 57970.254 object MzLS_700397_z 
    mosaic/CP/V4.3/CP20170804/k4m_170805_060522_ooi_zd_ls9.fits.fz
  k4m_170417_111220_ooi_zd_ls9-CCD2 z expnum 122676 exptime 80.103 propid 2016A-0453 seeing 1.09 MJD 57860.467 object MzLS_699853_z 
    mosaic/CP/V4.3/CP20170416/k4m_170417_111220_ooi_zd_ls9.fits.fz
  ksb_180416_081709_ooi_g_ls9-CCD2 g expnum 82240105 exptime 123.66 propid BASS seeing 1.51 MJD 58224.345 object 42213 
    90prime/CP/V2.3/CP20180415/ksb_180416_081709_ooi_g_ls9.fits.fz
  ksb_180420_101554_ooi_r_ls9-CCD2 r expnum 82280082 exptime 250.0 propid BASS seeing 2.70 MJD 58228.428 object 42213 
    90prime/CP/V2.3/CP20180419/ksb_180420_101554_ooi_r_ls9.fits.fz
  ksb_170403_104948_ooi_g_ls9-CCD3 g expnum 78460195 exptime 103.065 propid BASS seeing 1.45 MJD 57846.451 object 41342 
    90prime/CP/V2.3/CP20170402/ksb_170403_104948_ooi_g_ls9.fits.fz
  ksb_170531_092633_ooi_r_ls9-CCD3 r expnum 79040107 exptime 250.0 propid 2015A-0801 seeing 1.60 MJD 57904.393 object 41342 
    90prime/CP/V2.3/CP20170530/ksb_170531_092633_ooi_r_ls9.fits.fz
Faked up EXPNUM 82280082
Faked up EXPNUM 82280082
Faked up EXPNUM 82280082
90prime get_fwhm: self.fwhm = 5.95
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2339 2480 and amp range 2016 4031 and image y 2842 2983 and amp range 2048 4095
Range in image: x 0 142 , y 0 142 (with image size 142 x 142)
dzp 0.019008487 -> scaling image by 1.0176616178968958
Faked up EXPNUM 78460195
Faked up EXPNUM 78460195
Faked up EXPNUM 78460195
90prime get_fwhm: self.fwhm = 3.21
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2596 2735 and amp range 2016 4031 and image y 1636 1775 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp -3.875605e-05 -> scaling image by 0.9999643049955675
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.95
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2539 2784 and amp range 2048 4095 and image y 1867 2110 and amp range 0 2039
Range in image: x 0 246 , y 0 173 (with image size 246 x 244)
dzp 0.0044767857 -> scaling image by 1.004131784351374
Found overlap: image x 2539 2784 and amp range 2048 4095 and image y 1867 2110 and amp range 2040 4078
Range in image: x 0 246 , y 173 244 (with image size 246 x 244)
dzp -0.032976374 -> scaling image by 0.9700842429001929
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 4.25
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1416 1659 and amp range 0 2005 and image y 2161 2404 and amp range 2040 4078
Range in image: x 0 244 , y 0 244 (with image size 244 x 244)
dzp 0.014419514 -> scaling image by 1.0133694453441033
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82240105
Faked up EXPNUM 82240105
Faked up EXPNUM 82240105
90prime get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2333 2473 and amp range 2016 4031 and image y 2836 2977 and amp range 2048 4095
Range in image: x 0 141 , y 0 142 (with image size 141 x 142)
dzp 0.013237011 -> scaling image by 1.012266359574015
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 4.19
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 7 86 and amp range 0 2047 and image y 1319 1561 and amp range 0 2039
Range in image: x 0 80 , y 0 243 (with image size 80 x 243)
dzp -0.0096683735 -> scaling image by 0.9911346301237048
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79040107
Faked up EXPNUM 79040107
Faked up EXPNUM 79040107
90prime get_fwhm: self.fwhm = 3.53
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2599 2738 and amp range 2016 4031 and image y 1635 1773 and amp range 0 2047
Range in image: x 0 140 , y 0 139 (with image size 140 x 139)
dzp -0.0023383768 -> scaling image by 0.9978485929508539
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 20.78 s, CPU: 1.03 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 172 MB, VmData: 205 MB, maxrss: 0.17646 MB
Stage tims finished: 2020-08-05T19:27:36.116838
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 3.529 s worker CPU, 41.177 s worker Wall, Wall: 20.786 s, Cores in use: 0.22, Total efficiency (on 32 cores): 0.7 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-tims.p
Running stage refs
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T19:27:36.142447
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00660.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00589.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00590.fits
Matched 16 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/reference-custom-243293p64435.fits
Stage refs : Wall: 5.28 s, CPU: 2.17 s, VmPeak: 987 MB, VmSize: 976 MB, VmRSS: 193 MB, VmData: 224 MB, maxrss: 0.197556 MB
Stage refs finished: 2020-08-05T19:27:41.419664
Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 5.280 s, Cores in use: 0.41, Total efficiency (on 32 cores): 1.3 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T19:27:41.448167
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outlier-mask-custom-243293p64435.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-pre-custom-243293p64435.jpg
Image k4m_170521_091733_ooi_zd_ls9-CCD2 z : masking 215 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outlier-mask-custom-243293p64435.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-post-custom-243293p64435.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-masked-pos-custom-243293p64435.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/outliers-masked-neg-custom-243293p64435.jpg
Stage outliers : Wall: 1.44 s, CPU: 1.00 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 204 MB, VmData: 232 MB, maxrss: 0.209132 MB
Stage outliers finished: 2020-08-05T19:27:42.883244
Resources for stage outliers : 0.531 s worker CPU, 1.055 s worker Wall, Wall: 1.437 s, Cores in use: 1.07, Total efficiency (on 32 cores): 3.3 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T19:27:42.908981
Stage halos : Wall: 0.75 s, CPU: 0.73 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 204 MB, VmData: 232 MB, maxrss: 0.20938 MB
Stage halos finished: 2020-08-05T19:27:43.656294
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.749 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T19:27:43.674120
PSF sigmas in coadd pixels: ['2.45', '2.36']
Using average PSF sigma 2.4076392722627125
Gaussian PSF norm 0.11716655190989801 vs pixelized 0.09443834
average tim pixel scale / coadd scale: 1.7302535573831042
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['4.38', '2.60']
Using average PSF sigma 3.4902754164500287
Gaussian PSF norm 0.08082307500558158 vs pixelized 0.05897451
average tim pixel scale / coadd scale: 1.7301686786164934
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.67', '1.79', '1.77']
Using average PSF sigma 1.7415192061007227
Gaussian PSF norm 0.161982015923609 vs pixelized 0.13587123
average tim pixel scale / coadd scale: 0.9909378389564861
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.13 s, CPU: 0.10 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 206 MB, VmData: 232 MB, maxrss: 0.21102 MB
Stage fit_on_coadds finished: 2020-08-05T19:27:43.803283
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage fit_on_coadds : 0.087 s worker CPU, 0.087 s worker Wall, Wall: 0.131 s, Cores in use: 1.45, Total efficiency (on 32 cores): 4.5 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fit_on_coadds.p
Running stage srcs
Warning: Stellar halo subtraction is only implemented for DECam
Running stage srcs at 2020-08-05T19:27:43.819984
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 16 potential peaks
Of 16 potential peaks: 3 in veto map, 1 cut by saddle test, 0 cut by aper test, 12 kept
SED z : 12 new peaks
Found 10 potential peaks
Of 10 potential peaks: 10 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED r : 0 new peaks
Found 11 potential peaks
Of 11 potential peaks: 11 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 16 potential peaks
Of 16 potential peaks: 13 in veto map, 2 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 17 potential peaks
Of 17 potential peaks: 15 in veto map, 2 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 13 new sources
Median coadd sky for g : 0.000114623486
Median coadd sky for r : 0.00044442946
Median coadd sky for z : -0.00010727634
Stage srcs : Wall: 1.44 s, CPU: 1.31 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 206 MB, VmData: 232 MB, maxrss: 0.211392 MB
Stage srcs finished: 2020-08-05T19:27:45.263554
Resources for stage srcs : 0.084 s worker CPU, 0.084 s worker Wall, Wall: 1.445 s, Cores in use: 0.97, Total efficiency (on 32 cores): 3.0 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T19:27:45.292341
Blob 1 of 7, id: 3, sources: 3, size: 115x106, npix 9134, brick X: 66,181, Y: 64,170, one pixel: 113 64
Blob 2 of 7, id: 4, sources: 4, size: 83x83, npix 3404, brick X: 38,121, Y: 156,239, one pixel: 52 156
Blob 3 of 7, id: 5, sources: 2, size: 66x61, npix 2521, brick X: 122,188, Y: 167,228, one pixel: 156 167
Blob 4 of 7, id: 1, sources: 3, size: 44x61, npix 1595, brick X: 195,239, Y: 32,93, one pixel: 205 32
Blob 5 of 7, id: 0, sources: 2, size: 52x43, npix 1240, brick X: 60,112, Y: 19,62, one pixel: 71 19
Blob 6 of 7, id: 6, sources: 1, size: 25x24, npix 384, brick X: 191,216, Y: 191,215, one pixel: 201 191
Blob 7 of 7, id: 2, sources: 1, size: 21x23, npix 279, brick X: 139,160, Y: 45,68, one pixel: 148 45
Blob 7 finished, total: Wall: 0.14 s, CPU: 0.06 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 85 MB, VmData: 138 MB, maxrss: 0.08746 MB
Blob 6 finished, total: Wall: 0.29 s, CPU: 0.22 s, VmPeak: 809 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149792 MB
Blob 5 finished, total: Wall: 0.79 s, CPU: 0.72 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.09296 MB
Blob 3 finished, total: Wall: 1.39 s, CPU: 1.32 s, VmPeak: 808 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149968 MB
Measuring 1 iterative sources
Blob 4 finished, total: Wall: 1.57 s, CPU: 1.49 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.0938 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 2.31 s, CPU: 2.28 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 184 MB, maxrss: 0.15146 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.48958, ee1=-0.0413716, ee2=-0.104729
Blob 1 finished, total: Wall: 3.15 s, CPU: 3.15 s, VmPeak: 676 MB, VmSize: 672 MB, VmRSS: 98 MB, VmData: 143 MB, maxrss: 0.103444 MB
Old catalog: 21
New catalog: 15
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/blobs-custom-243293p64435.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/metrics/cus/all-models-custom-243293p64435.fits
Stage fitblobs : Wall: 7.28 s, CPU: 3.94 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 207 MB, VmData: 232 MB, maxrss: 0.211556 MB
Stage fitblobs finished: 2020-08-05T19:27:52.569364
Resources for stage fitblobs : 9.344 s worker CPU, 10.338 s worker Wall, Wall: 7.279 s, Cores in use: 1.82, Total efficiency (on 32 cores): 5.7 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T19:27:52.586806
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-resid.jpg
Stage coadds : Wall: 4.77 s, CPU: 1.50 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 207 MB, VmData: 232 MB, maxrss: 0.212072 MB
Stage coadds finished: 2020-08-05T19:27:57.353117
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 27.203 s worker CPU, 63.004 s worker Wall, Wall: 4.768 s, Cores in use: 6.02, Total efficiency (on 32 cores): 18.8 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T19:27:57.384817
Cut to 1 unWISE tiles
Reading WISE tile 2446p651 band 2
Reading WISE tile 2446p651 band 1
Reading WISE tile 2446p651 band 4
Reading WISE tile 2446p651 band 3
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 804 MB, VmSize: 804 MB, VmRSS: 145 MB, VmData: 176 MB, maxrss: 0.189508 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 803 MB, VmSize: 802 MB, VmRSS: 142 MB, VmData: 174 MB, maxrss: 0.205188 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 807 MB, VmSize: 807 MB, VmRSS: 145 MB, VmData: 179 MB, maxrss: 0.19978 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 805 MB, VmSize: 803 MB, VmRSS: 141 MB, VmData: 170 MB, maxrss: 0.204408 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/tmp-legacysurvey-custom-243293p64435-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 5.38 s, CPU: 2.82 s, VmPeak: 987 MB, VmSize: 986 MB, VmRSS: 208 MB, VmData: 232 MB, maxrss: 0.212776 MB
Stage wise_forced finished: 2020-08-05T19:28:02.764523
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage wise_forced : 3.256 s worker CPU, 7.835 s worker Wall, Wall: 5.382 s, Cores in use: 1.13, Total efficiency (on 32 cores): 3.5 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-wise_forced.p
Running stage writecat
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage writecat at 2020-08-05T19:28:02.778359
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/coadd/cus/custom-243293p64435/legacysurvey-custom-243293p64435-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/tractor-i/cus/tractor-custom-243293p64435.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/tractor/cus/tractor-custom-243293p64435.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/tractor/cus/brick-custom-243293p64435.sha256sum
Stage writecat : Wall: 0.83 s, CPU: 0.62 s, VmPeak: 1118 MB, VmSize: 990 MB, VmRSS: 209 MB, VmData: 232 MB, maxrss: 0.345588 MB
Stage writecat finished: 2020-08-05T19:28:03.603574
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.827 s, Cores in use: 0.75, Total efficiency (on 32 cores): 2.3 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/243/PGC2668065/PGC2668065-largegalaxy-writecat.p
All done: 44.041 s worker CPU, 123.588 s worker Wall, Wall: 48.290 s, Cores in use: 1.23, Total efficiency (on 32 cores): 3.8 %
Finished galaxy PGC2668065 in 0.869 minutes.
Started working on galaxy PGC2668065 at Fri Aug  7 09:23:08 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 243.29323240361424 64.43473135025542 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-07T09:23:09.256639
legacypipe git version: DR9.5.11-watershed-3-gd509eaaf
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '243.29323240361424', '64.43473135025542', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 243.29323240361424 64.43473135025542 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 243.29323240361424 64.43473135025542
Starting process 10084 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 10085 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 10086 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 10087 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 10088 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05934 MB
Starting process 10089 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059328 MB
Starting process 10090 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059328 MB
Starting process 10091 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 10092 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 10093 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 10094 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 10095 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 10096 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 10097 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 10098 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 10099 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-07T09:23:09.769317
Keeping 7 CCDs:
  k4m_170521_091733_ooi_zd_ls9-CCD2 z expnum 129236 exptime 84.104 propid 2016A-0453 seeing 1.03 MJD 57894.387 object MzLS_699306_z 
    mosaic/CP/V4.3/CP20170520/k4m_170521_091733_ooi_zd_ls9.fits.fz
  k4m_170805_060522_ooi_zd_ls9-CCD1 z expnum 3135769 exptime 124.106 propid 2016A-0453 seeing 1.10 MJD 57970.254 object MzLS_700397_z 
    mosaic/CP/V4.3/CP20170804/k4m_170805_060522_ooi_zd_ls9.fits.fz
  k4m_170417_111220_ooi_zd_ls9-CCD2 z expnum 122676 exptime 80.103 propid 2016A-0453 seeing 1.09 MJD 57860.467 object MzLS_699853_z 
    mosaic/CP/V4.3/CP20170416/k4m_170417_111220_ooi_zd_ls9.fits.fz
  ksb_180416_081709_ooi_g_ls9-CCD2 g expnum 82240105 exptime 123.66 propid BASS seeing 1.51 MJD 58224.345 object 42213 
    90prime/CP/V2.3/CP20180415/ksb_180416_081709_ooi_g_ls9.fits.fz
  ksb_180420_101554_ooi_r_ls9-CCD2 r expnum 82280082 exptime 250.0 propid BASS seeing 2.70 MJD 58228.428 object 42213 
    90prime/CP/V2.3/CP20180419/ksb_180420_101554_ooi_r_ls9.fits.fz
  ksb_170403_104948_ooi_g_ls9-CCD3 g expnum 78460195 exptime 103.065 propid BASS seeing 1.45 MJD 57846.451 object 41342 
    90prime/CP/V2.3/CP20170402/ksb_170403_104948_ooi_g_ls9.fits.fz
  ksb_170531_092633_ooi_r_ls9-CCD3 r expnum 79040107 exptime 250.0 propid 2015A-0801 seeing 1.60 MJD 57904.393 object 41342 
    90prime/CP/V2.3/CP20170530/ksb_170531_092633_ooi_r_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 3.95
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2536 2780 and amp range 2048 4095 and image y 1865 2109 and amp range 0 2039
Range in image: x 0 245 , y 0 175 (with image size 245 x 245)
dzp 0.0044767857 -> scaling image by 1.004131784351374
Found overlap: image x 2536 2780 and amp range 2048 4095 and image y 1865 2109 and amp range 2040 4078
Range in image: x 0 245 , y 175 245 (with image size 245 x 245)
dzp -0.032976374 -> scaling image by 0.9700842429001929
mosaic get_fwhm: self.fwhm = 4.25
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1412 1655 and amp range 0 2005 and image y 2160 2403 and amp range 2040 4078
Range in image: x 0 244 , y 0 244 (with image size 244 x 244)
dzp 0.014419514 -> scaling image by 1.0133694453441033
mosaic get_fwhm: self.fwhm = 4.19
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 7 82 and amp range 0 2047 and image y 1317 1560 and amp range 0 2039
Range in image: x 0 76 , y 0 244 (with image size 76 x 244)
dzp -0.0096683735 -> scaling image by 0.9911346301237048
Faked up EXPNUM 82280082
Faked up EXPNUM 82280082
Faked up EXPNUM 82280082
90prime get_fwhm: self.fwhm = 5.95
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2341 2482 and amp range 2016 4031 and image y 2843 2984 and amp range 2048 4095
Range in image: x 0 142 , y 0 142 (with image size 142 x 142)
dzp 0.019008487 -> scaling image by 1.0176616178968958
Faked up EXPNUM 78460195
Faked up EXPNUM 78460195
Faked up EXPNUM 78460195
90prime get_fwhm: self.fwhm = 3.21
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2598 2737 and amp range 2016 4031 and image y 1637 1776 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp -3.875605e-05 -> scaling image by 0.9999643049955675
Faked up EXPNUM 79040107
Faked up EXPNUM 79040107
Faked up EXPNUM 79040107
90prime get_fwhm: self.fwhm = 3.53
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2601 2740 and amp range 2016 4031 and image y 1635 1774 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp -0.0023383768 -> scaling image by 0.9978485929508539
Faked up EXPNUM 82240105
Faked up EXPNUM 82240105
Faked up EXPNUM 82240105
90prime get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2335 2476 and amp range 2016 4031 and image y 2837 2978 and amp range 2048 4095
Range in image: x 0 142 , y 0 142 (with image size 142 x 142)
dzp 0.013237011 -> scaling image by 1.012266359574015
Stage tims : Wall: 47.78 s, CPU: 0.88 s, VmPeak: 950 MB, VmSize: 950 MB, VmRSS: 171 MB, VmData: 204 MB, maxrss: 0.174824 MB
Stage tims finished: 2020-08-07T09:23:57.547486
Resources for stage tims : 3.191 s worker CPU, 129.453 s worker Wall, Wall: 47.780 s, Cores in use: 0.09, Total efficiency (on 16 cores): 0.5 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-08-07T09:23:57.563744
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00660.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00589.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00590.fits
Matched 16 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/reference-custom-243293p64434.fits
Stage refs : Wall: 4.39 s, CPU: 2.03 s, VmPeak: 981 MB, VmSize: 970 MB, VmRSS: 191 MB, VmData: 224 MB, maxrss: 0.195328 MB
Stage refs finished: 2020-08-07T09:24:01.957755
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 4.396 s, Cores in use: 0.46, Total efficiency (on 16 cores): 2.9 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-07T09:24:02.010847
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/outlier-mask-custom-243293p64434.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/outliers-pre-custom-243293p64434.jpg
Image k4m_170521_091733_ooi_zd_ls9-CCD2 z : masking 140 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/outlier-mask-custom-243293p64434.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/outliers-post-custom-243293p64434.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/outliers-masked-pos-custom-243293p64434.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/outliers-masked-neg-custom-243293p64434.jpg
Stage outliers : Wall: 4.82 s, CPU: 0.53 s, VmPeak: 981 MB, VmSize: 979 MB, VmRSS: 202 MB, VmData: 232 MB, maxrss: 0.206952 MB
Stage outliers finished: 2020-08-07T09:24:06.835676
Resources for stage outliers : 0.448 s worker CPU, 0.504 s worker Wall, Wall: 4.826 s, Cores in use: 0.20, Total efficiency (on 16 cores): 1.3 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-07T09:24:07.826263
Stage halos : Wall: 0.39 s, CPU: 0.32 s, VmPeak: 981 MB, VmSize: 979 MB, VmRSS: 202 MB, VmData: 232 MB, maxrss: 0.207312 MB
Stage halos finished: 2020-08-07T09:24:08.220256
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.395 s, Cores in use: 0.82, Total efficiency (on 16 cores): 5.1 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-07T09:24:08.924161
PSF sigmas in coadd pixels: ['2.45', '2.36']
Using average PSF sigma 2.4076392722627125
Gaussian PSF norm 0.11716655190989801 vs pixelized 0.09443848
average tim pixel scale / coadd scale: 1.7302535573831042
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['4.38', '2.60']
Using average PSF sigma 3.4902754164500287
Gaussian PSF norm 0.08082307500558158 vs pixelized 0.05897428
average tim pixel scale / coadd scale: 1.7301686786164934
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.67', '1.79', '1.77']
Using average PSF sigma 1.7415192061007227
Gaussian PSF norm 0.161982015923609 vs pixelized 0.13586712
average tim pixel scale / coadd scale: 0.9909378389564861
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.69 s, CPU: 0.11 s, VmPeak: 981 MB, VmSize: 979 MB, VmRSS: 204 MB, VmData: 232 MB, maxrss: 0.209152 MB
Stage fit_on_coadds finished: 2020-08-07T09:24:09.612197
Resources for stage fit_on_coadds : 0.108 s worker CPU, 0.108 s worker Wall, Wall: 0.690 s, Cores in use: 0.31, Total efficiency (on 16 cores): 2.0 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-fit_on_coadds.p
Running stage srcs
Warning: Stellar halo subtraction is only implemented for DECam
Running stage srcs at 2020-08-07T09:24:09.669312
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 15 potential peaks
Of 15 potential peaks: 3 in veto map, 1 cut by saddle test, 0 cut by aper test, 11 kept
SED z : 11 new peaks
Found 10 potential peaks
Of 10 potential peaks: 10 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED r : 0 new peaks
Found 11 potential peaks
Of 11 potential peaks: 11 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 15 potential peaks
Of 15 potential peaks: 13 in veto map, 1 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 18 potential peaks
Of 18 potential peaks: 15 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 12 new sources
Median coadd sky for g : 0.00011360908
Median coadd sky for r : 0.00042833848
Median coadd sky for z : -0.00010391435
Stage srcs : Wall: 0.51 s, CPU: 0.47 s, VmPeak: 981 MB, VmSize: 979 MB, VmRSS: 205 MB, VmData: 232 MB, maxrss: 0.209508 MB
Stage srcs finished: 2020-08-07T09:24:10.177444
Resources for stage srcs : 0.082 s worker CPU, 0.082 s worker Wall, Wall: 0.510 s, Cores in use: 1.08, Total efficiency (on 16 cores): 6.8 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-07T09:24:10.215531
Blob 1 of 7, id: 3, sources: 2, size: 114x107, npix 9123, brick X: 65,179, Y: 67,174, one pixel: 124 67
Blob 2 of 7, id: 4, sources: 4, size: 82x79, npix 3188, brick X: 36,118, Y: 160,239, one pixel: 50 160
Blob 3 of 7, id: 5, sources: 2, size: 66x60, npix 2502, brick X: 121,187, Y: 172,232, one pixel: 153 172
Blob 4 of 7, id: 1, sources: 3, size: 45x61, npix 1634, brick X: 194,239, Y: 36,97, one pixel: 203 36
Blob 5 of 7, id: 0, sources: 2, size: 52x43, npix 1245, brick X: 59,111, Y: 23,66, one pixel: 70 23
Blob 6 of 7, id: 6, sources: 1, size: 25x24, npix 385, brick X: 190,215, Y: 195,219, one pixel: 199 195
Blob 7 of 7, id: 2, sources: 1, size: 21x23, npix 298, brick X: 138,159, Y: 49,72, one pixel: 146 49
Blob 7 finished, total: Wall: 0.07 s, CPU: 0.07 s, VmPeak: 808 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145424 MB
Blob 6 finished, total: Wall: 0.21 s, CPU: 0.21 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 89 MB, VmData: 137 MB, maxrss: 0.09156 MB
Blob 5 finished, total: Wall: 0.75 s, CPU: 0.75 s, VmPeak: 808 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145784 MB
Blob 3 finished, total: Wall: 1.33 s, CPU: 1.33 s, VmPeak: 669 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093876 MB
Measuring 1 iterative sources
Blob 4 finished, total: Wall: 1.50 s, CPU: 1.50 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094712 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.22796, ee1=-0.0366058, ee2=-0.0949378
Blob 1 finished, total: Wall: 1.92 s, CPU: 1.92 s, VmPeak: 681 MB, VmSize: 675 MB, VmRSS: 99 MB, VmData: 144 MB, maxrss: 0.106728 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 2.25 s, CPU: 2.25 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 96 MB, VmData: 142 MB, maxrss: 0.0987 MB
Old catalog: 20
New catalog: 14
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/blobs-custom-243293p64434.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/metrics/cus/all-models-custom-243293p64434.fits
Stage fitblobs : Wall: 2.70 s, CPU: 0.37 s, VmPeak: 981 MB, VmSize: 979 MB, VmRSS: 205 MB, VmData: 232 MB, maxrss: 0.209744 MB
Stage fitblobs finished: 2020-08-07T09:24:12.910914
Resources for stage fitblobs : 8.123 s worker CPU, 8.209 s worker Wall, Wall: 2.697 s, Cores in use: 3.15, Total efficiency (on 16 cores): 19.7 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-07T09:24:12.921554
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/tmp-legacysurvey-custom-243293p64434-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/tmp-legacysurvey-custom-243293p64434-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/tmp-legacysurvey-custom-243293p64434-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/tmp-legacysurvey-custom-243293p64434-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-resid.jpg
Stage coadds : Wall: 2.30 s, CPU: 0.58 s, VmPeak: 981 MB, VmSize: 979 MB, VmRSS: 205 MB, VmData: 232 MB, maxrss: 0.210268 MB
Stage coadds finished: 2020-08-07T09:24:15.223682
Resources for stage coadds : 10.254 s worker CPU, 10.726 s worker Wall, Wall: 2.304 s, Cores in use: 4.70, Total efficiency (on 16 cores): 29.4 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-07T09:24:15.277077
Cut to 1 unWISE tiles
Reading WISE tile 2446p651 band 1
Reading WISE tile 2446p651 band 2
Reading WISE tile 2446p651 band 3
Reading WISE tile 2446p651 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 809 MB, VmSize: 792 MB, VmRSS: 136 MB, VmData: 170 MB, maxrss: 0.198468 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 805 MB, VmSize: 805 MB, VmRSS: 148 MB, VmData: 176 MB, maxrss: 0.187688 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 808 MB, VmSize: 797 MB, VmRSS: 140 MB, VmData: 171 MB, maxrss: 0.203384 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/244/2446p651/unwise-2446p651-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 803 MB, VmSize: 795 MB, VmRSS: 139 MB, VmData: 173 MB, maxrss: 0.201892 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/tmp-legacysurvey-custom-243293p64434-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/tmp-legacysurvey-custom-243293p64434-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 5.83 s, CPU: 0.40 s, VmPeak: 981 MB, VmSize: 979 MB, VmRSS: 206 MB, VmData: 232 MB, maxrss: 0.210972 MB
Stage wise_forced finished: 2020-08-07T09:24:21.104176
Resources for stage wise_forced : 2.842 s worker CPU, 13.627 s worker Wall, Wall: 5.828 s, Cores in use: 0.56, Total efficiency (on 16 cores): 3.5 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-07T09:24:21.114673
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/coadd/cus/custom-243293p64434/legacysurvey-custom-243293p64434-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/tractor-i/cus/tractor-custom-243293p64434.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/tractor/cus/tractor-custom-243293p64434.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/tractor/cus/brick-custom-243293p64434.sha256sum
Stage writecat : Wall: 0.92 s, CPU: 0.51 s, VmPeak: 1112 MB, VmSize: 984 MB, VmRSS: 208 MB, VmData: 232 MB, maxrss: 0.343988 MB
Stage writecat finished: 2020-08-07T09:24:22.032606
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.920 s, Cores in use: 0.56, Total efficiency (on 16 cores): 3.5 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/243/PGC2668065/PGC2668065-largegalaxy-writecat.p
All done: 25.049 s worker CPU, 162.713 s worker Wall, Wall: 72.301 s, Cores in use: 0.43, Total efficiency (on 16 cores): 2.7 %
Finished galaxy PGC2668065 in 1.242 minutes.