Started working on galaxy PGC2061294 at Wed Aug  5 20:15:06 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.04010550000004 35.22572329999999 --width 263 --height 263 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T20:15:08.791759
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.04010550000004', '35.22572329999999', '--width', '263', '--height', '263', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.04010550000004 35.22572329999999 --width 263 --height 263 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.04010550000004 35.22572329999999
Starting process 37540 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059224 MB
Starting process 37541 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 37542 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 37543 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 37544 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0594 MB
Starting process 37545 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059408 MB
Starting process 37546 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059408 MB
Starting process 37547 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 37548 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 37549 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 37550 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 37551 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 37552 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 37553 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 37554 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 37555 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T20:15:10.109742
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 9 CCDs:
  k4m_160512_103813_ooi_zd_ls9-CCD1 z expnum 88582 exptime 233.104 propid 2016A-0453 seeing 1.68 MJD 57520.443 object MzLS_580439_z 
    mosaic/CP/V4.3/CP20160511/k4m_160512_103813_ooi_zd_ls9.fits.fz
  k4m_160414_115956_ooi_zd_ls9-CCD3 z expnum 82277 exptime 80.104 propid 2016A-0453 seeing 0.82 MJD 57492.500 object MzLS_581461_z 
    mosaic/CP/V4.3/CP20160413/k4m_160414_115956_ooi_zd_ls9.fits.fz
  k4m_160415_115838_ooi_zd_ls9-CCD2 z expnum 82536 exptime 91.102 propid 2016A-0453 seeing 1.11 MJD 57493.499 object MzLS_579415_z 
    mosaic/CP/V4.3/CP20160414/k4m_160415_115838_ooi_zd_ls9.fits.fz
  ksb_180509_110248_ooi_r_ls9-CCD3 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.07 MJD 58247.460 object 08283 
    90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz
  ksb_160702_081612_ooi_g_ls9-CCD3 g expnum 75710100 exptime 134.2 propid BASS seeing 2.02 MJD 57571.345 object 08283 
    90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz
  ksb_160602_085049_ooi_g_ls9-CCD2 g expnum 75410138 exptime 112.9 propid BASS seeing 1.18 MJD 57541.369 object 08272 
    90prime/CP/V2.3/CP20160601/ksb_160602_085049_ooi_g_ls9.fits.fz
  ksb_160511_110116_ooi_r_ls9-CCD2 r expnum 75190251 exptime 59.6 propid BASS seeing 1.12 MJD 57519.459 object 08272 
    90prime/CP/V2.3/CP20160510/ksb_160511_110116_ooi_r_ls9.fits.fz
  ksb_160430_105427_ooi_r_ls9-CCD3 r expnum 75080189 exptime 135.9 propid BASS seeing 1.07 MJD 57508.454 object 08281 
    90prime/CP/V2.3/CP20160429/ksb_160430_105427_ooi_r_ls9.fits.fz
  ksb_160504_104522_ooi_g_ls9-CCD3 g expnum 75120206 exptime 97.1 propid BASS seeing 1.54 MJD 57512.448 object 08281 
    90prime/CP/V2.3/CP20160503/ksb_160504_104522_ooi_g_ls9.fits.fz
Faked up EXPNUM 75190251
Faked up EXPNUM 75190251
Faked up EXPNUM 75190251
90prime get_fwhm: self.fwhm = 2.47
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 154 and amp range 0 2015 and image y 582 737 and amp range 0 2047
Range in image: x 0 140 , y 0 156 (with image size 140 x 156)
dzp -0.006045583 -> scaling image by 0.9944472857505693
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75120206
Faked up EXPNUM 75120206
Faked up EXPNUM 75120206
90prime get_fwhm: self.fwhm = 3.39
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2265 2417 and amp range 2016 4031 and image y 2146 2299 and amp range 2048 4095
Range in image: x 0 153 , y 0 154 (with image size 153 x 154)
dzp -0.0012536051 -> scaling image by 0.9988460533227724
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
90prime get_fwhm: self.fwhm = 4.45
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 252 407 and amp range 0 2015 and image y 164 318 and amp range 0 2047
Range in image: x 0 156 , y 0 155 (with image size 156 x 155)
dzp -0.03470582 -> scaling image by 0.9685402452207784
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
90prime get_fwhm: self.fwhm = 2.37
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 259 413 and amp range 0 2015 and image y 143 297 and amp range 0 2047
Range in image: x 0 155 , y 0 155 (with image size 155 x 155)
dzp -0.039144866 -> scaling image by 0.9645884414840357
Faked up EXPNUM 75080189
Faked up EXPNUM 75080189
Faked up EXPNUM 75080189
90prime get_fwhm: self.fwhm = 2.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2275 2428 and amp range 2016 4031 and image y 2145 2298 and amp range 2048 4095
Range in image: x 0 154 , y 0 154 (with image size 154 x 154)
dzp -0.0005558773 -> scaling image by 0.99948814914245
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 6.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2959 3223 and amp range 2006 4053 and image y 3181 3445 and amp range 2040 4078
Range in image: x 0 265 , y 0 265 (with image size 265 x 265)
dzp -0.006092005 -> scaling image by 0.9944047680378836
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.15
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 257 530 and amp range 0 2005 and image y 2416 2689 and amp range 2040 4078
Range in image: x 0 274 , y 0 274 (with image size 274 x 274)
dzp 0.0022078273 -> scaling image by 1.0020355530120029
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75410138
Faked up EXPNUM 75410138
Faked up EXPNUM 75410138
90prime get_fwhm: self.fwhm = 2.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 154 and amp range 0 2015 and image y 554 709 and amp range 0 2047
Range in image: x 0 140 , y 0 156 (with image size 140 x 156)
dzp -0.008523628 -> scaling image by 0.992180183483648
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 4.27
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1380 1646 and amp range 0 2047 and image y 1870 2137 and amp range 0 2039
Range in image: x 0 267 , y 0 170 (with image size 267 x 268)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 1380 1646 and amp range 0 2047 and image y 1870 2137 and amp range 2040 4078
Range in image: x 0 267 , y 170 268 (with image size 267 x 268)
dzp 0.008550294 -> scaling image by 1.0079062018164673
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 6.71 s, CPU: 1.03 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 175 MB, VmData: 207 MB, maxrss: 0.179096 MB
Stage tims finished: 2020-08-05T20:15:16.822998
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 4.744 s worker CPU, 29.782 s worker Wall, Wall: 6.715 s, Cores in use: 0.86, Total efficiency (on 16 cores): 5.4 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-tims.p
Running stage refs
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T20:15:16.865047
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/reference-custom-276040p35225.fits
Stage refs : Wall: 17.75 s, CPU: 8.87 s, VmPeak: 1057 MB, VmSize: 1052 MB, VmRSS: 272 MB, VmData: 301 MB, maxrss: 0.278208 MB
Stage refs finished: 2020-08-05T20:15:34.612380
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 17.749 s, Cores in use: 0.50, Total efficiency (on 16 cores): 3.1 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T20:15:34.670679
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outlier-mask-custom-276040p35225.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-pre-custom-276040p35225.jpg
Image ksb_160702_081612_ooi_g_ls9-CCD3 g : masking 211 positive outlier pixels and 0 negative outlier pixels
Image ksb_180509_110248_ooi_r_ls9-CCD3 r : masking 217 positive outlier pixels and 0 negative outlier pixels
Image k4m_160414_115956_ooi_zd_ls9-CCD3 z : masking 626 positive outlier pixels and 176 negative outlier pixels
Image k4m_160415_115838_ooi_zd_ls9-CCD2 z : masking 144 positive outlier pixels and 0 negative outlier pixels
Image k4m_160512_103813_ooi_zd_ls9-CCD1 z : masking 3063 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outlier-mask-custom-276040p35225.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-post-custom-276040p35225.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-masked-pos-custom-276040p35225.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-masked-neg-custom-276040p35225.jpg
Stage outliers : Wall: 2.54 s, CPU: 1.88 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 286 MB, VmData: 312 MB, maxrss: 0.292732 MB
Stage outliers finished: 2020-08-05T20:15:37.209781
Resources for stage outliers : 0.718 s worker CPU, 1.011 s worker Wall, Wall: 2.541 s, Cores in use: 1.02, Total efficiency (on 16 cores): 6.4 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T20:15:37.265140
Stage halos : Wall: 1.31 s, CPU: 1.28 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 286 MB, VmData: 313 MB, maxrss: 0.292956 MB
Stage halos finished: 2020-08-05T20:15:38.574185
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.310 s, Cores in use: 0.98, Total efficiency (on 16 cores): 6.1 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T20:15:38.722304
PSF sigmas in coadd pixels: ['3.27', '1.92', '2.49']
Using average PSF sigma 2.5633488526757144
Gaussian PSF norm 0.1100493175087806 vs pixelized 0.08826158
average tim pixel scale / coadd scale: 1.7293613805712862
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.74', '1.82', '1.73']
Using average PSF sigma 1.7642925029244916
Gaussian PSF norm 0.15989116958003152 vs pixelized 0.14083272
average tim pixel scale / coadd scale: 1.729941635413934
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.74', '1.33', '1.80']
Using average PSF sigma 1.9551686318582966
Gaussian PSF norm 0.1442815658850665 vs pixelized 0.11465468
average tim pixel scale / coadd scale: 0.9909404716312259
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.19 s, CPU: 0.16 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 288 MB, VmData: 313 MB, maxrss: 0.294424 MB
Stage fit_on_coadds finished: 2020-08-05T20:15:38.917110
Resources for stage fit_on_coadds : 0.182 s worker CPU, 0.182 s worker Wall, Wall: 0.196 s, Cores in use: 1.77, Total efficiency (on 16 cores): 11.0 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T20:15:38.933128
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 45 potential peaks
Of 45 potential peaks: 8 in veto map, 7 cut by saddle test, 0 cut by aper test, 30 kept
SED z : 30 new peaks
Found 59 potential peaks
Of 59 potential peaks: 51 in veto map, 5 cut by saddle test, 0 cut by aper test, 3 kept
SED r : 3 new peaks
Found 28 potential peaks
Of 28 potential peaks: 26 in veto map, 2 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 47 potential peaks
Of 47 potential peaks: 43 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept
SED Flat : 2 new peaks
Found 48 potential peaks
Of 48 potential peaks: 44 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept
SED Red : 1 new peaks
Adding 36 new sources
Median coadd sky for g : 0.00020142595
Median coadd sky for r : 0.0008423793
Median coadd sky for z : -0.00035325304
Stage srcs : Wall: 1.08 s, CPU: 1.00 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 288 MB, VmData: 313 MB, maxrss: 0.29506 MB
Stage srcs finished: 2020-08-05T20:15:40.014194
Resources for stage srcs : 0.091 s worker CPU, 0.091 s worker Wall, Wall: 1.083 s, Cores in use: 1.01, Total efficiency (on 16 cores): 6.3 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T20:15:40.031701
Blob 1 of 12, id: 4, sources: 13, size: 163x170, npix 14050, brick X: 53,216, Y: 44,214, one pixel: 121 44
Blob 2 of 12, id: 6, sources: 7, size: 88x146, npix 5384, brick X: 0,88, Y: 102,248, one pixel: 8 102
Blob 3 of 12, id: 9, sources: 7, size: 112x69, npix 2786, brick X: 149,261, Y: 189,258, one pixel: 223 189
Blob 4 of 12, id: 5, sources: 5, size: 74x81, npix 2762, brick X: 187,261, Y: 49,130, one pixel: 212 49
Blob 5 of 12, id: 3, sources: 3, size: 59x67, npix 2029, brick X: 18,77, Y: 11,78, one pixel: 36 11
Blob 6 of 12, id: 1, sources: 2, size: 54x38, npix 1306, brick X: 186,240, Y: 0,38, one pixel: 217 0
Blob 7 of 12, id: 2, sources: 2, size: 41x41, npix 856, brick X: 139,180, Y: 4,45, one pixel: 153 4
Blob 8 of 12, id: 10, sources: 1, size: 22x35, npix 555, brick X: 0,22, Y: 190,225, one pixel: 5 190
Blob 9 of 12, id: 7, sources: 1, size: 25x24, npix 396, brick X: 211,236, Y: 111,135, one pixel: 221 111
Blob 10 of 12, id: 11, sources: 1, size: 24x17, npix 291, brick X: 6,30, Y: 246,263, one pixel: 16 246
Blob 11 of 12, id: 8, sources: 1, size: 22x20, npix 260, brick X: 238,260, Y: 145,165, one pixel: 247 145
Blob 12 of 12, id: 0, sources: 1, size: 22x17, npix 247, brick X: 167,189, Y: 0,17, one pixel: 172 0
Blob 10 finished, total: Wall: 0.19 s, CPU: 0.18 s, VmPeak: 807 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.151036 MB
Blob 11 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 811 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.15102 MB
Blob 9 finished, total: Wall: 0.49 s, CPU: 0.48 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093516 MB
Blob 12 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 808 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150956 MB
Blob 8 finished, total: Wall: 0.76 s, CPU: 0.76 s, VmPeak: 814 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151484 MB
Blob 6 finished, total: Wall: 0.78 s, CPU: 0.77 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094664 MB
Blob 7 finished, total: Wall: 0.96 s, CPU: 0.95 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 138 MB, maxrss: 0.0943 MB
Blob 5 finished, total: Wall: 1.00 s, CPU: 0.99 s, VmPeak: 816 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149936 MB
Measuring 1 iterative sources
Measuring 2 iterative sources
Measuring 1 iterative sources
Blob 4 finished, total: Wall: 2.47 s, CPU: 2.47 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 184 MB, maxrss: 0.151648 MB
Blob 3 finished, total: Wall: 2.70 s, CPU: 2.70 s, VmPeak: 814 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 184 MB, maxrss: 0.151996 MB
Blob 2 finished, total: Wall: 2.86 s, CPU: 2.86 s, VmPeak: 672 MB, VmSize: 672 MB, VmRSS: 99 MB, VmData: 143 MB, maxrss: 0.100976 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.32208, ee1=0.0862986, ee2=-0.220817
Measuring 3 iterative sources
Blob 1 finished, total: Wall: 9.61 s, CPU: 9.61 s, VmPeak: 691 MB, VmSize: 686 MB, VmRSS: 114 MB, VmData: 157 MB, maxrss: 0.122004 MB
Old catalog: 51
New catalog: 47
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/blobs-custom-276040p35225.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/all-models-custom-276040p35225.fits
Stage fitblobs : Wall: 10.36 s, CPU: 0.73 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 288 MB, VmData: 313 MB, maxrss: 0.295224 MB
Stage fitblobs finished: 2020-08-05T20:15:50.387922
Resources for stage fitblobs : 22.765 s worker CPU, 22.902 s worker Wall, Wall: 10.358 s, Cores in use: 2.27, Total efficiency (on 16 cores): 14.2 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T20:15:50.402911
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-resid.jpg
Stage coadds : Wall: 3.03 s, CPU: 0.70 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 289 MB, VmData: 313 MB, maxrss: 0.295548 MB
Stage coadds finished: 2020-08-05T20:15:53.433111
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 9.786 s worker CPU, 17.786 s worker Wall, Wall: 3.032 s, Cores in use: 3.46, Total efficiency (on 16 cores): 21.6 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T20:15:53.447673
Cut to 1 unWISE tiles
Reading WISE tile 2759p348 band 4
Reading WISE tile 2759p348 band 3
Reading WISE tile 2759p348 band 2
Reading WISE tile 2759p348 band 1
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 810 MB, VmSize: 805 MB, VmRSS: 147 MB, VmData: 177 MB, maxrss: 0.189116 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 811 MB, VmSize: 804 MB, VmRSS: 144 MB, VmData: 172 MB, maxrss: 0.207588 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 813 MB, VmSize: 799 MB, VmRSS: 142 MB, VmData: 171 MB, maxrss: 0.204928 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 814 MB, VmSize: 805 MB, VmRSS: 146 MB, VmData: 173 MB, maxrss: 0.209668 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 3.26 s, CPU: 1.30 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 289 MB, VmData: 313 MB, maxrss: 0.29606 MB
Stage wise_forced finished: 2020-08-05T20:15:56.704126
Resources for stage wise_forced : 3.295 s worker CPU, 5.072 s worker Wall, Wall: 3.258 s, Cores in use: 1.41, Total efficiency (on 16 cores): 8.8 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T20:15:56.779016
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/tractor-i/cus/tractor-custom-276040p35225.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/tractor/cus/tractor-custom-276040p35225.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/tractor/cus/brick-custom-276040p35225.sha256sum
Stage writecat : Wall: 2.76 s, CPU: 1.68 s, VmPeak: 1194 MB, VmSize: 1066 MB, VmRSS: 291 MB, VmData: 313 MB, maxrss: 0.42894 MB
Stage writecat finished: 2020-08-05T20:15:59.539747
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 2.763 s, Cores in use: 0.61, Total efficiency (on 16 cores): 3.8 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-writecat.p
All done: 41.587 s worker CPU, 76.833 s worker Wall, Wall: 49.461 s, Cores in use: 1.22, Total efficiency (on 16 cores): 7.6 %
Finished galaxy PGC2061294 in 0.900 minutes.
Started working on galaxy PGC2061294 at Wed Aug  5 22:12:19 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.04010550000004 35.22572329999999 --width 263 --height 263 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T22:12:20.283228
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.04010550000004', '35.22572329999999', '--width', '263', '--height', '263', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.04010550000004 35.22572329999999 --width 263 --height 263 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.04010550000004 35.22572329999999
Starting process 59003 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059228 MB
Starting process 59004 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059228 MB
Starting process 59005 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 59006 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 59007 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059532 MB
Starting process 59008 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05954 MB
Starting process 59009 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05954 MB
Starting process 59010 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 59011 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 59012 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 59013 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 59014 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 59015 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 59016 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 59017 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 59018 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 59019 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 59020 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 59021 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 59022 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 59023 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 59024 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 59025 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 59026 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB
Starting process 59027 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB
Starting process 59028 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB
Starting process 59029 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 59030 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 59031 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 59032 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 59033 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 59034 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T22:12:20.837740
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 9 CCDs:
  k4m_160512_103813_ooi_zd_ls9-CCD1 z expnum 88582 exptime 233.104 propid 2016A-0453 seeing 1.68 MJD 57520.443 object MzLS_580439_z 
    mosaic/CP/V4.3/CP20160511/k4m_160512_103813_ooi_zd_ls9.fits.fz
  k4m_160414_115956_ooi_zd_ls9-CCD3 z expnum 82277 exptime 80.104 propid 2016A-0453 seeing 0.82 MJD 57492.500 object MzLS_581461_z 
    mosaic/CP/V4.3/CP20160413/k4m_160414_115956_ooi_zd_ls9.fits.fz
  k4m_160415_115838_ooi_zd_ls9-CCD2 z expnum 82536 exptime 91.102 propid 2016A-0453 seeing 1.11 MJD 57493.499 object MzLS_579415_z 
    mosaic/CP/V4.3/CP20160414/k4m_160415_115838_ooi_zd_ls9.fits.fz
  ksb_180509_110248_ooi_r_ls9-CCD3 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.07 MJD 58247.460 object 08283 
    90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz
  ksb_160702_081612_ooi_g_ls9-CCD3 g expnum 75710100 exptime 134.2 propid BASS seeing 2.02 MJD 57571.345 object 08283 
    90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz
  ksb_160602_085049_ooi_g_ls9-CCD2 g expnum 75410138 exptime 112.9 propid BASS seeing 1.18 MJD 57541.369 object 08272 
    90prime/CP/V2.3/CP20160601/ksb_160602_085049_ooi_g_ls9.fits.fz
  ksb_160511_110116_ooi_r_ls9-CCD2 r expnum 75190251 exptime 59.6 propid BASS seeing 1.12 MJD 57519.459 object 08272 
    90prime/CP/V2.3/CP20160510/ksb_160511_110116_ooi_r_ls9.fits.fz
  ksb_160430_105427_ooi_r_ls9-CCD3 r expnum 75080189 exptime 135.9 propid BASS seeing 1.07 MJD 57508.454 object 08281 
    90prime/CP/V2.3/CP20160429/ksb_160430_105427_ooi_r_ls9.fits.fz
  ksb_160504_104522_ooi_g_ls9-CCD3 g expnum 75120206 exptime 97.1 propid BASS seeing 1.54 MJD 57512.448 object 08281 
    90prime/CP/V2.3/CP20160503/ksb_160504_104522_ooi_g_ls9.fits.fz
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
90prime get_fwhm: self.fwhm = 4.45
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 252 407 and amp range 0 2015 and image y 164 318 and amp range 0 2047
Range in image: x 0 156 , y 0 155 (with image size 156 x 155)
dzp -0.03470582 -> scaling image by 0.9685402452207784
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75080189
Faked up EXPNUM 75080189
Faked up EXPNUM 75080189
90prime get_fwhm: self.fwhm = 2.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2275 2428 and amp range 2016 4031 and image y 2145 2298 and amp range 2048 4095
Range in image: x 0 154 , y 0 154 (with image size 154 x 154)
dzp -0.0005558773 -> scaling image by 0.99948814914245
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75190251
Faked up EXPNUM 75190251
Faked up EXPNUM 75190251
90prime get_fwhm: self.fwhm = 2.47
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 154 and amp range 0 2015 and image y 582 737 and amp range 0 2047
Range in image: x 0 140 , y 0 156 (with image size 140 x 156)
dzp -0.006045583 -> scaling image by 0.9944472857505693
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
90prime get_fwhm: self.fwhm = 2.37
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 259 413 and amp range 0 2015 and image y 143 297 and amp range 0 2047
Range in image: x 0 155 , y 0 155 (with image size 155 x 155)
dzp -0.039144866 -> scaling image by 0.9645884414840357
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 6.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2959 3223 and amp range 2006 4053 and image y 3181 3445 and amp range 2040 4078
Range in image: x 0 265 , y 0 265 (with image size 265 x 265)
dzp -0.006092005 -> scaling image by 0.9944047680378836
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.15
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 257 530 and amp range 0 2005 and image y 2416 2689 and amp range 2040 4078
Range in image: x 0 274 , y 0 274 (with image size 274 x 274)
dzp 0.0022078273 -> scaling image by 1.0020355530120029
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 4.27
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1380 1646 and amp range 0 2047 and image y 1870 2137 and amp range 0 2039
Range in image: x 0 267 , y 0 170 (with image size 267 x 268)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 1380 1646 and amp range 0 2047 and image y 1870 2137 and amp range 2040 4078
Range in image: x 0 267 , y 170 268 (with image size 267 x 268)
dzp 0.008550294 -> scaling image by 1.0079062018164673
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75120206
Faked up EXPNUM 75120206
Faked up EXPNUM 75120206
90prime get_fwhm: self.fwhm = 3.39
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2265 2417 and amp range 2016 4031 and image y 2146 2299 and amp range 2048 4095
Range in image: x 0 153 , y 0 154 (with image size 153 x 154)
dzp -0.0012536051 -> scaling image by 0.9988460533227724
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75410138
Faked up EXPNUM 75410138
Faked up EXPNUM 75410138
90prime get_fwhm: self.fwhm = 2.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 154 and amp range 0 2015 and image y 554 709 and amp range 0 2047
Range in image: x 0 140 , y 0 156 (with image size 140 x 156)
dzp -0.008523628 -> scaling image by 0.992180183483648
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 5.94 s, CPU: 1.04 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 175 MB, VmData: 207 MB, maxrss: 0.17876 MB
Stage tims finished: 2020-08-05T22:12:26.780279
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 5.028 s worker CPU, 24.696 s worker Wall, Wall: 5.945 s, Cores in use: 1.02, Total efficiency (on 32 cores): 3.2 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-tims.p
Running stage refs
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T22:12:26.871107
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/reference-custom-276040p35225.fits
Stage refs : Wall: 18.76 s, CPU: 9.58 s, VmPeak: 1057 MB, VmSize: 1052 MB, VmRSS: 271 MB, VmData: 301 MB, maxrss: 0.277548 MB
Stage refs finished: 2020-08-05T22:12:45.629107
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 18.760 s, Cores in use: 0.51, Total efficiency (on 32 cores): 1.6 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T22:12:45.758285
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outlier-mask-custom-276040p35225.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-pre-custom-276040p35225.jpg
Image ksb_160702_081612_ooi_g_ls9-CCD3 g : masking 211 positive outlier pixels and 0 negative outlier pixels
Image ksb_180509_110248_ooi_r_ls9-CCD3 r : masking 217 positive outlier pixels and 0 negative outlier pixels
Image k4m_160512_103813_ooi_zd_ls9-CCD1 z : masking 3063 positive outlier pixels and 0 negative outlier pixels
Image k4m_160414_115956_ooi_zd_ls9-CCD3 z : masking 626 positive outlier pixels and 176 negative outlier pixels
Image k4m_160415_115838_ooi_zd_ls9-CCD2 z : masking 144 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outlier-mask-custom-276040p35225.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-post-custom-276040p35225.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-masked-pos-custom-276040p35225.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/outliers-masked-neg-custom-276040p35225.jpg
Stage outliers : Wall: 2.36 s, CPU: 1.63 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 286 MB, VmData: 313 MB, maxrss: 0.292648 MB
Stage outliers finished: 2020-08-05T22:12:48.114727
Resources for stage outliers : 0.960 s worker CPU, 1.591 s worker Wall, Wall: 2.359 s, Cores in use: 1.10, Total efficiency (on 32 cores): 3.4 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T22:12:48.169612
Stage halos : Wall: 0.63 s, CPU: 0.61 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 286 MB, VmData: 313 MB, maxrss: 0.29308 MB
Stage halos finished: 2020-08-05T22:12:48.795739
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.628 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T22:12:48.886633
PSF sigmas in coadd pixels: ['3.27', '1.92', '2.49']
Using average PSF sigma 2.5633488526757144
Gaussian PSF norm 0.1100493175087806 vs pixelized 0.08826158
average tim pixel scale / coadd scale: 1.7293613805712862
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.74', '1.82', '1.73']
Using average PSF sigma 1.7642925029244916
Gaussian PSF norm 0.15989116958003152 vs pixelized 0.14083272
average tim pixel scale / coadd scale: 1.729941635413934
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.74', '1.33', '1.80']
Using average PSF sigma 1.9551686318582966
Gaussian PSF norm 0.1442815658850665 vs pixelized 0.11465468
average tim pixel scale / coadd scale: 0.9909404716312259
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.17 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 288 MB, VmData: 313 MB, maxrss: 0.294544 MB
Stage fit_on_coadds finished: 2020-08-05T22:12:49.062520
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage fit_on_coadds : 0.200 s worker CPU, 0.200 s worker Wall, Wall: 0.178 s, Cores in use: 2.07, Total efficiency (on 32 cores): 6.5 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T22:12:49.074847
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 45 potential peaks
Of 45 potential peaks: 8 in veto map, 7 cut by saddle test, 0 cut by aper test, 30 kept
SED z : 30 new peaks
Found 59 potential peaks
Of 59 potential peaks: 51 in veto map, 5 cut by saddle test, 0 cut by aper test, 3 kept
SED r : 3 new peaks
Found 28 potential peaks
Of 28 potential peaks: 26 in veto map, 2 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 47 potential peaks
Of 47 potential peaks: 43 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept
SED Flat : 2 new peaks
Found 48 potential peaks
Of 48 potential peaks: 44 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept
SED Red : 1 new peaks
Adding 36 new sources
Median coadd sky for g : 0.00020142595
Median coadd sky for r : 0.0008423793
Median coadd sky for z : -0.00035325304
Stage srcs : Wall: 0.55 s, CPU: 0.51 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 288 MB, VmData: 313 MB, maxrss: 0.295052 MB
Stage srcs finished: 2020-08-05T22:12:49.621480
Resources for stage srcs : 0.089 s worker CPU, 0.089 s worker Wall, Wall: 0.549 s, Cores in use: 1.09, Total efficiency (on 32 cores): 3.4 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T22:12:49.637832
Blob 1 of 12, id: 4, sources: 13, size: 163x170, npix 14050, brick X: 53,216, Y: 44,214, one pixel: 121 44
Blob 2 of 12, id: 6, sources: 7, size: 88x146, npix 5384, brick X: 0,88, Y: 102,248, one pixel: 8 102
Blob 3 of 12, id: 9, sources: 7, size: 112x69, npix 2786, brick X: 149,261, Y: 189,258, one pixel: 223 189
Blob 4 of 12, id: 5, sources: 5, size: 74x81, npix 2762, brick X: 187,261, Y: 49,130, one pixel: 212 49
Blob 5 of 12, id: 3, sources: 3, size: 59x67, npix 2029, brick X: 18,77, Y: 11,78, one pixel: 36 11
Blob 6 of 12, id: 1, sources: 2, size: 54x38, npix 1306, brick X: 186,240, Y: 0,38, one pixel: 217 0
Blob 7 of 12, id: 2, sources: 2, size: 41x41, npix 856, brick X: 139,180, Y: 4,45, one pixel: 153 4
Blob 8 of 12, id: 10, sources: 1, size: 22x35, npix 555, brick X: 0,22, Y: 190,225, one pixel: 5 190
Blob 9 of 12, id: 7, sources: 1, size: 25x24, npix 396, brick X: 211,236, Y: 111,135, one pixel: 221 111
Blob 10 of 12, id: 11, sources: 1, size: 24x17, npix 291, brick X: 6,30, Y: 246,263, one pixel: 16 246
Blob 11 of 12, id: 8, sources: 1, size: 22x20, npix 260, brick X: 238,260, Y: 145,165, one pixel: 247 145
Blob 12 of 12, id: 0, sources: 1, size: 22x17, npix 247, brick X: 167,189, Y: 0,17, one pixel: 172 0
Blob 10 finished, total: Wall: 0.21 s, CPU: 0.21 s, VmPeak: 813 MB, VmSize: 731 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150548 MB
Blob 11 finished, total: Wall: 0.21 s, CPU: 0.21 s, VmPeak: 814 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149676 MB
Blob 12 finished, total: Wall: 0.36 s, CPU: 0.36 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.09536 MB
Blob 9 finished, total: Wall: 0.40 s, CPU: 0.39 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092728 MB
Blob 8 finished, total: Wall: 0.60 s, CPU: 0.59 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093328 MB
Blob 6 finished, total: Wall: 0.68 s, CPU: 0.68 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.092936 MB
Blob 5 finished, total: Wall: 0.82 s, CPU: 0.81 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.095936 MB
Blob 7 finished, total: Wall: 1.00 s, CPU: 1.00 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094512 MB
Measuring 1 iterative sources
Measuring 2 iterative sources
Blob 4 finished, total: Wall: 2.34 s, CPU: 2.33 s, VmPeak: 671 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.0981 MB
Measuring 1 iterative sources
Blob 3 finished, total: Wall: 2.79 s, CPU: 2.79 s, VmPeak: 816 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150468 MB
Blob 2 finished, total: Wall: 2.92 s, CPU: 2.91 s, VmPeak: 810 MB, VmSize: 732 MB, VmRSS: 149 MB, VmData: 184 MB, maxrss: 0.15298 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.32208, ee1=0.0862988, ee2=-0.220817
Measuring 3 iterative sources
Blob 1 finished, total: Wall: 8.87 s, CPU: 8.87 s, VmPeak: 691 MB, VmSize: 685 MB, VmRSS: 113 MB, VmData: 157 MB, maxrss: 0.121556 MB
Old catalog: 51
New catalog: 47
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/blobs-custom-276040p35225.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/metrics/cus/all-models-custom-276040p35225.fits
Stage fitblobs : Wall: 9.33 s, CPU: 0.47 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 288 MB, VmData: 313 MB, maxrss: 0.295216 MB
Stage fitblobs finished: 2020-08-05T22:12:58.972684
Resources for stage fitblobs : 21.326 s worker CPU, 21.409 s worker Wall, Wall: 9.337 s, Cores in use: 2.33, Total efficiency (on 32 cores): 7.3 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T22:12:58.987533
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-resid.jpg
Stage coadds : Wall: 7.00 s, CPU: 0.54 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 289 MB, VmData: 313 MB, maxrss: 0.29554 MB
Stage coadds finished: 2020-08-05T22:13:05.992369
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 26.206 s worker CPU, 60.204 s worker Wall, Wall: 7.007 s, Cores in use: 3.82, Total efficiency (on 32 cores): 11.9 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T22:13:06.065161
Cut to 1 unWISE tiles
Reading WISE tile 2759p348 band 1
Reading WISE tile 2759p348 band 3
Reading WISE tile 2759p348 band 2
Reading WISE tile 2759p348 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 807 MB, VmSize: 807 MB, VmRSS: 146 MB, VmData: 179 MB, maxrss: 0.185204 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 814 MB, VmSize: 799 MB, VmRSS: 143 MB, VmData: 171 MB, maxrss: 0.20268 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 864 MB, VmSize: 864 MB, VmRSS: 206 MB, VmData: 236 MB, maxrss: 0.211136 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 800 MB, VmSize: 800 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.19962 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/tmp-legacysurvey-custom-276040p35225-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 7.31 s, CPU: 0.98 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 289 MB, VmData: 313 MB, maxrss: 0.296052 MB
Stage wise_forced finished: 2020-08-05T22:13:13.377384
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage wise_forced : 3.374 s worker CPU, 5.203 s worker Wall, Wall: 7.314 s, Cores in use: 0.60, Total efficiency (on 32 cores): 1.9 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-wise_forced.p
Running stage writecat
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage writecat at 2020-08-05T22:13:13.397971
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/coadd/cus/custom-276040p35225/legacysurvey-custom-276040p35225-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/tractor-i/cus/tractor-custom-276040p35225.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/tractor/cus/tractor-custom-276040p35225.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/tractor/cus/brick-custom-276040p35225.sha256sum
Stage writecat : Wall: 2.61 s, CPU: 1.25 s, VmPeak: 1193 MB, VmSize: 1065 MB, VmRSS: 291 MB, VmData: 313 MB, maxrss: 0.428916 MB
Stage writecat finished: 2020-08-05T22:13:16.003146
Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 2.608 s, Cores in use: 0.48, Total efficiency (on 32 cores): 1.5 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2061294/PGC2061294-largegalaxy-writecat.p
All done: 57.190 s worker CPU, 113.402 s worker Wall, Wall: 55.240 s, Cores in use: 1.34, Total efficiency (on 32 cores): 4.2 %
Finished galaxy PGC2061294 in 0.954 minutes.
Started working on galaxy PGC2061294 at Fri Aug  7 09:43:21 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.03954759984316 35.225787904104685 --width 263 --height 263 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-07T09:43:23.552248
legacypipe git version: DR9.5.11-watershed-3-gd509eaaf
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.03954759984316', '35.225787904104685', '--width', '263', '--height', '263', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.03954759984316 35.225787904104685 --width 263 --height 263 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.03954759984316 35.225787904104685
Starting process 32595 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059228 MB
Starting process 32596 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 32597 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 32598 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 32599 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059536 MB
Starting process 32600 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB
Starting process 32601 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB
Starting process 32602 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 32603 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 32604 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 32605 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 32606 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 32607 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 32608 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 32609 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 32610 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-07T09:43:24.019965
Keeping 9 CCDs:
  k4m_160512_103813_ooi_zd_ls9-CCD1 z expnum 88582 exptime 233.104 propid 2016A-0453 seeing 1.68 MJD 57520.443 object MzLS_580439_z 
    mosaic/CP/V4.3/CP20160511/k4m_160512_103813_ooi_zd_ls9.fits.fz
  k4m_160414_115956_ooi_zd_ls9-CCD3 z expnum 82277 exptime 80.104 propid 2016A-0453 seeing 0.82 MJD 57492.500 object MzLS_581461_z 
    mosaic/CP/V4.3/CP20160413/k4m_160414_115956_ooi_zd_ls9.fits.fz
  k4m_160415_115838_ooi_zd_ls9-CCD2 z expnum 82536 exptime 91.102 propid 2016A-0453 seeing 1.11 MJD 57493.499 object MzLS_579415_z 
    mosaic/CP/V4.3/CP20160414/k4m_160415_115838_ooi_zd_ls9.fits.fz
  ksb_180509_110248_ooi_r_ls9-CCD3 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.07 MJD 58247.460 object 08283 
    90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz
  ksb_160702_081612_ooi_g_ls9-CCD3 g expnum 75710100 exptime 134.2 propid BASS seeing 2.02 MJD 57571.345 object 08283 
    90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz
  ksb_160602_085049_ooi_g_ls9-CCD2 g expnum 75410138 exptime 112.9 propid BASS seeing 1.18 MJD 57541.369 object 08272 
    90prime/CP/V2.3/CP20160601/ksb_160602_085049_ooi_g_ls9.fits.fz
  ksb_160511_110116_ooi_r_ls9-CCD2 r expnum 75190251 exptime 59.6 propid BASS seeing 1.12 MJD 57519.459 object 08272 
    90prime/CP/V2.3/CP20160510/ksb_160511_110116_ooi_r_ls9.fits.fz
  ksb_160430_105427_ooi_r_ls9-CCD3 r expnum 75080189 exptime 135.9 propid BASS seeing 1.07 MJD 57508.454 object 08281 
    90prime/CP/V2.3/CP20160429/ksb_160430_105427_ooi_r_ls9.fits.fz
  ksb_160504_104522_ooi_g_ls9-CCD3 g expnum 75120206 exptime 97.1 propid BASS seeing 1.54 MJD 57512.448 object 08281 
    90prime/CP/V2.3/CP20160503/ksb_160504_104522_ooi_g_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 4.27
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1381 1647 and amp range 0 2047 and image y 1864 2131 and amp range 0 2039
Range in image: x 0 267 , y 0 176 (with image size 267 x 268)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 1381 1647 and amp range 0 2047 and image y 1864 2131 and amp range 2040 4078
Range in image: x 0 267 , y 176 268 (with image size 267 x 268)
dzp 0.008550294 -> scaling image by 1.0079062018164673
mosaic get_fwhm: self.fwhm = 6.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2960 3224 and amp range 2006 4053 and image y 3175 3439 and amp range 2040 4078
Range in image: x 0 265 , y 0 265 (with image size 265 x 265)
dzp -0.006092005 -> scaling image by 0.9944047680378836
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
90prime get_fwhm: self.fwhm = 2.37
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 258 413 and amp range 0 2015 and image y 147 300 and amp range 0 2047
Range in image: x 0 156 , y 0 154 (with image size 156 x 154)
dzp -0.039144866 -> scaling image by 0.9645884414840357
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
90prime get_fwhm: self.fwhm = 4.45
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 252 406 and amp range 0 2015 and image y 168 321 and amp range 0 2047
Range in image: x 0 155 , y 0 154 (with image size 155 x 154)
dzp -0.03470582 -> scaling image by 0.9685402452207784
Faked up EXPNUM 75080189
Faked up EXPNUM 75080189
Faked up EXPNUM 75080189
90prime get_fwhm: self.fwhm = 2.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2275 2427 and amp range 2016 4031 and image y 2148 2302 and amp range 2048 4095
Range in image: x 0 153 , y 0 155 (with image size 153 x 155)
dzp -0.0005558773 -> scaling image by 0.99948814914245
Faked up EXPNUM 75190251
Faked up EXPNUM 75190251
Faked up EXPNUM 75190251
90prime get_fwhm: self.fwhm = 2.47
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 153 and amp range 0 2015 and image y 585 741 and amp range 0 2047
Range in image: x 0 139 , y 0 157 (with image size 139 x 157)
dzp -0.006045583 -> scaling image by 0.9944472857505693
Faked up EXPNUM 75410138
Faked up EXPNUM 75410138
Faked up EXPNUM 75410138
90prime get_fwhm: self.fwhm = 2.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 153 and amp range 0 2015 and image y 557 712 and amp range 0 2047
Range in image: x 0 139 , y 0 156 (with image size 139 x 156)
dzp -0.008523628 -> scaling image by 0.992180183483648
mosaic get_fwhm: self.fwhm = 3.15
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 258 531 and amp range 0 2005 and image y 2410 2683 and amp range 2040 4078
Range in image: x 0 274 , y 0 274 (with image size 274 x 274)
dzp 0.0022078273 -> scaling image by 1.0020355530120029
Faked up EXPNUM 75120206
Faked up EXPNUM 75120206
Faked up EXPNUM 75120206
90prime get_fwhm: self.fwhm = 3.39
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2264 2417 and amp range 2016 4031 and image y 2149 2303 and amp range 2048 4095
Range in image: x 0 154 , y 0 155 (with image size 154 x 155)
dzp -0.0012536051 -> scaling image by 0.9988460533227724
Stage tims : Wall: 100.06 s, CPU: 0.90 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 173 MB, VmData: 206 MB, maxrss: 0.176716 MB
Stage tims finished: 2020-08-07T09:45:04.083439
Resources for stage tims : 4.391 s worker CPU, 465.704 s worker Wall, Wall: 100.065 s, Cores in use: 0.05, Total efficiency (on 16 cores): 0.3 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-08-07T09:45:04.145925
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02593.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/reference-custom-276039p35225.fits
Stage refs : Wall: 17.99 s, CPU: 8.27 s, VmPeak: 1050 MB, VmSize: 1045 MB, VmRSS: 269 MB, VmData: 301 MB, maxrss: 0.27568 MB
Stage refs finished: 2020-08-07T09:45:22.131692
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 17.987 s, Cores in use: 0.46, Total efficiency (on 16 cores): 2.9 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-07T09:45:22.173497
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/outlier-mask-custom-276039p35225.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/outliers-pre-custom-276039p35225.jpg
Image ksb_160702_081612_ooi_g_ls9-CCD3 g : masking 213 positive outlier pixels and 0 negative outlier pixels
Image ksb_180509_110248_ooi_r_ls9-CCD3 r : masking 218 positive outlier pixels and 0 negative outlier pixels
Image k4m_160414_115956_ooi_zd_ls9-CCD3 z : masking 622 positive outlier pixels and 0 negative outlier pixels
Image k4m_160512_103813_ooi_zd_ls9-CCD1 z : masking 2563 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/outlier-mask-custom-276039p35225.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/outliers-post-custom-276039p35225.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/outliers-masked-pos-custom-276039p35225.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/outliers-masked-neg-custom-276039p35225.jpg
Stage outliers : Wall: 1.03 s, CPU: 0.63 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 285 MB, VmData: 313 MB, maxrss: 0.29162 MB
Stage outliers finished: 2020-08-07T09:45:23.206577
Resources for stage outliers : 0.733 s worker CPU, 0.817 s worker Wall, Wall: 1.035 s, Cores in use: 1.32, Total efficiency (on 16 cores): 8.3 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-07T09:45:23.286398
Stage halos : Wall: 0.27 s, CPU: 0.26 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 285 MB, VmData: 313 MB, maxrss: 0.291932 MB
Stage halos finished: 2020-08-07T09:45:23.560241
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.275 s, Cores in use: 0.96, Total efficiency (on 16 cores): 6.0 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-07T09:45:23.589677
PSF sigmas in coadd pixels: ['3.27', '1.92', '2.49']
Using average PSF sigma 2.5633488526757144
Gaussian PSF norm 0.1100493175087806 vs pixelized 0.08825999
average tim pixel scale / coadd scale: 1.7293613805712862
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.74', '1.82', '1.73']
Using average PSF sigma 1.7642925029244916
Gaussian PSF norm 0.15989116958003152 vs pixelized 0.14083008
average tim pixel scale / coadd scale: 1.729941635413934
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.74', '1.33', '1.80']
Using average PSF sigma 1.9551686318582966
Gaussian PSF norm 0.1442815658850665 vs pixelized 0.11465388
average tim pixel scale / coadd scale: 0.9909404716312259
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.15 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 287 MB, VmData: 313 MB, maxrss: 0.2934 MB
Stage fit_on_coadds finished: 2020-08-07T09:45:23.771019
Resources for stage fit_on_coadds : 0.183 s worker CPU, 0.183 s worker Wall, Wall: 0.183 s, Cores in use: 1.81, Total efficiency (on 16 cores): 11.3 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-07T09:45:23.797234
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 44 potential peaks
Of 44 potential peaks: 9 in veto map, 7 cut by saddle test, 0 cut by aper test, 28 kept
SED z : 28 new peaks
Found 32 potential peaks
Of 32 potential peaks: 25 in veto map, 4 cut by saddle test, 0 cut by aper test, 3 kept
SED r : 3 new peaks
Found 28 potential peaks
Of 28 potential peaks: 26 in veto map, 1 cut by saddle test, 0 cut by aper test, 1 kept
SED g : 1 new peaks
Found 47 potential peaks
Of 47 potential peaks: 43 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept
SED Flat : 2 new peaks
Found 49 potential peaks
Of 49 potential peaks: 45 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept
SED Red : 1 new peaks
Adding 35 new sources
Median coadd sky for g : 0.000193214
Median coadd sky for r : 0.0008503359
Median coadd sky for z : -0.00027025212
Stage srcs : Wall: 0.51 s, CPU: 0.46 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 287 MB, VmData: 313 MB, maxrss: 0.293888 MB
Stage srcs finished: 2020-08-07T09:45:24.305101
Resources for stage srcs : 0.110 s worker CPU, 0.111 s worker Wall, Wall: 0.509 s, Cores in use: 1.12, Total efficiency (on 16 cores): 7.0 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-07T09:45:24.317203
Blob 1 of 12, id: 4, sources: 12, size: 163x170, npix 14046, brick X: 47,210, Y: 43,213, one pixel: 115 43
Blob 2 of 12, id: 6, sources: 7, size: 81x145, npix 4941, brick X: 0,81, Y: 102,247, one pixel: 4 102
Blob 3 of 12, id: 5, sources: 5, size: 74x81, npix 2750, brick X: 181,255, Y: 48,129, one pixel: 206 48
Blob 4 of 12, id: 9, sources: 7, size: 112x68, npix 2750, brick X: 143,255, Y: 189,257, one pixel: 217 189
Blob 5 of 12, id: 3, sources: 3, size: 59x67, npix 2029, brick X: 12,71, Y: 10,77, one pixel: 30 10
Blob 6 of 12, id: 1, sources: 2, size: 53x37, npix 1293, brick X: 180,233, Y: 0,37, one pixel: 207 0
Blob 7 of 12, id: 2, sources: 2, size: 41x41, npix 869, brick X: 133,174, Y: 3,44, one pixel: 147 3
Blob 8 of 12, id: 7, sources: 1, size: 26x24, npix 389, brick X: 204,230, Y: 110,134, one pixel: 215 110
Blob 9 of 12, id: 11, sources: 1, size: 24x18, npix 305, brick X: 0,24, Y: 245,263, one pixel: 10 245
Blob 10 of 12, id: 10, sources: 1, size: 16x25, npix 275, brick X: 0,16, Y: 195,220, one pixel: 3 195
Blob 11 of 12, id: 8, sources: 1, size: 22x20, npix 250, brick X: 232,254, Y: 144,164, one pixel: 241 144
Blob 12 of 12, id: 0, sources: 1, size: 21x16, npix 225, brick X: 161,182, Y: 0,16, one pixel: 166 0
Blob 10 finished, total: Wall: 0.23 s, CPU: 0.22 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.09588 MB
Blob 9 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.146044 MB
Blob 11 finished, total: Wall: 0.28 s, CPU: 0.27 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093504 MB
Blob 8 finished, total: Wall: 0.48 s, CPU: 0.48 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.09514 MB
Blob 12 finished, total: Wall: 0.51 s, CPU: 0.51 s, VmPeak: 816 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147224 MB
Blob 6 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 814 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147604 MB
Blob 5 finished, total: Wall: 1.03 s, CPU: 1.03 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.1479 MB
Blob 7 finished, total: Wall: 1.04 s, CPU: 1.04 s, VmPeak: 814 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147176 MB
Measuring 1 iterative sources
Measuring 2 iterative sources
Blob 3 finished, total: Wall: 2.27 s, CPU: 2.27 s, VmPeak: 671 MB, VmSize: 668 MB, VmRSS: 94 MB, VmData: 139 MB, maxrss: 0.098672 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 2.54 s, CPU: 2.53 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 96 MB, VmData: 141 MB, maxrss: 0.09818 MB
Blob 4 finished, total: Wall: 2.67 s, CPU: 2.67 s, VmPeak: 807 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147492 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=0.966596, ee1=0.0791069, ee2=-0.20084
Blob 1 finished, total: Wall: 6.96 s, CPU: 6.96 s, VmPeak: 810 MB, VmSize: 728 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150352 MB
Old catalog: 50
New catalog: 42
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/blobs-custom-276039p35225.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/metrics/cus/all-models-custom-276039p35225.fits
Stage fitblobs : Wall: 7.37 s, CPU: 0.35 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 287 MB, VmData: 313 MB, maxrss: 0.294052 MB
Stage fitblobs finished: 2020-08-07T09:45:31.686690
Resources for stage fitblobs : 19.255 s worker CPU, 19.418 s worker Wall, Wall: 7.371 s, Cores in use: 2.66, Total efficiency (on 16 cores): 16.6 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-07T09:45:31.700605
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/tmp-legacysurvey-custom-276039p35225-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/tmp-legacysurvey-custom-276039p35225-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/tmp-legacysurvey-custom-276039p35225-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/tmp-legacysurvey-custom-276039p35225-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-resid.jpg
Stage coadds : Wall: 2.46 s, CPU: 0.58 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 287 MB, VmData: 313 MB, maxrss: 0.294376 MB
Stage coadds finished: 2020-08-07T09:45:34.159959
Resources for stage coadds : 9.387 s worker CPU, 9.939 s worker Wall, Wall: 2.461 s, Cores in use: 4.05, Total efficiency (on 16 cores): 25.3 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-07T09:45:34.186692
Cut to 1 unWISE tiles
Reading WISE tile 2759p348 band 1
Reading WISE tile 2759p348 band 2
Reading WISE tile 2759p348 band 3
Reading WISE tile 2759p348 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 816 MB, VmSize: 802 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.188088 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 814 MB, VmSize: 798 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.204652 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 807 MB, VmSize: 799 MB, VmRSS: 143 MB, VmData: 173 MB, maxrss: 0.205824 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 814 MB, VmSize: 798 MB, VmRSS: 142 MB, VmData: 171 MB, maxrss: 0.204728 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/tmp-legacysurvey-custom-276039p35225-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/tmp-legacysurvey-custom-276039p35225-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 4.40 s, CPU: 0.51 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 288 MB, VmData: 313 MB, maxrss: 0.294888 MB
Stage wise_forced finished: 2020-08-07T09:45:38.582293
Resources for stage wise_forced : 3.040 s worker CPU, 8.072 s worker Wall, Wall: 4.397 s, Cores in use: 0.81, Total efficiency (on 16 cores): 5.0 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-07T09:45:38.612576
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/coadd/cus/custom-276039p35225/legacysurvey-custom-276039p35225-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/tractor-i/cus/tractor-custom-276039p35225.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/tractor/cus/tractor-custom-276039p35225.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/tractor/cus/brick-custom-276039p35225.sha256sum
Stage writecat : Wall: 0.72 s, CPU: 0.57 s, VmPeak: 1187 MB, VmSize: 1059 MB, VmRSS: 290 MB, VmData: 313 MB, maxrss: 0.427772 MB
Stage writecat finished: 2020-08-07T09:45:39.329177
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.718 s, Cores in use: 0.79, Total efficiency (on 16 cores): 5.0 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2061294/PGC2061294-largegalaxy-writecat.p
All done: 37.103 s worker CPU, 504.249 s worker Wall, Wall: 135.352 s, Cores in use: 0.37, Total efficiency (on 16 cores): 2.3 %
Finished galaxy PGC2061294 in 2.317 minutes.