Started working on galaxy PGC2046768 at Wed Aug  5 19:12:22 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.26172 34.2825775 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T19:12:24.761852
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.26172', '34.2825775', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.26172 34.2825775 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.26172 34.2825775
Starting process 36997 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 36998 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 36999 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 37000 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 37001 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 37002 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 37003 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 37004 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 37005 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 37006 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 37007 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 37008 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 37009 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 37010 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 37011 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 37012 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T19:12:26.195490
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 9 CCDs:
  k4m_160414_120201_ooi_zd_ls9-CCD3 z expnum 82278 exptime 80.104 propid 2016A-0453 seeing 0.85 MJD 57492.501 object MzLS_575293_z 
    mosaic/CP/V4.3/CP20160413/k4m_160414_120201_ooi_zd_ls9.fits.fz
  k4m_160415_115629_ooi_zd_ls9-CCD3 z expnum 82535 exptime 85.104 propid 2016A-0453 seeing 0.99 MJD 57493.498 object MzLS_576327_z 
    mosaic/CP/V4.3/CP20160414/k4m_160415_115629_ooi_zd_ls9.fits.fz
  k4m_160429_110817_ooi_zd_ls9-CCD2 z expnum 85812 exptime 84.103 propid 2016A-0453 seeing 1.43 MJD 57507.464 object MzLS_574257_z 
    mosaic/CP/V4.3/CP20160428/k4m_160429_110817_ooi_zd_ls9.fits.fz
  ksb_160602_084734_ooi_g_ls9-CCD3 g expnum 75410137 exptime 139.4 propid BASS seeing 1.24 MJD 57541.366 object 06922 
    90prime/CP/V2.3/CP20160601/ksb_160602_084734_ooi_g_ls9.fits.fz
  ksb_160510_110001_ooi_r_ls9-CCD3 r expnum 75180254 exptime 67.9 propid BASS seeing 0.96 MJD 57518.458 object 06922 
    90prime/CP/V2.3/CP20160509/ksb_160510_110001_ooi_r_ls9.fits.fz
  ksb_160504_104228_ooi_g_ls9-CCD1 g expnum 75120205 exptime 118.3 propid BASS seeing 1.50 MJD 57512.446 object 06921 
    90prime/CP/V2.3/CP20160503/ksb_160504_104228_ooi_g_ls9.fits.fz
  ksb_160430_104359_ooi_r_ls9-CCD1 r expnum 75080186 exptime 155.6 propid BASS seeing 1.21 MJD 57508.447 object 06921 
    90prime/CP/V2.3/CP20160429/ksb_160430_104359_ooi_r_ls9.fits.fz
  ksb_180509_110248_ooi_r_ls9-CCD2 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.18 MJD 58247.460 object 08283 
    90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz
  ksb_160702_081612_ooi_g_ls9-CCD2 g expnum 75710100 exptime 134.2 propid BASS seeing 1.97 MJD 57571.345 object 08283 
    90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz
Faked up EXPNUM 75410137
Faked up EXPNUM 75410137
Faked up EXPNUM 75410137
90prime get_fwhm: self.fwhm = 2.73
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3791 3930 and amp range 2016 4031 and image y 374 513 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp -0.0022715968 -> scaling image by 0.9979099691834846
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 5.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1848 2090 and amp range 0 2047 and image y 1507 1751 and amp range 0 2039
Range in image: x 0 200 , y 0 245 (with image size 243 x 245)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 1848 2090 and amp range 2048 4095 and image y 1507 1751 and amp range 0 2039
Range in image: x 200 243 , y 0 245 (with image size 243 x 245)
dzp 0.0072022504 -> scaling image by 1.0066555682883527
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.27
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3471 3711 and amp range 2006 4053 and image y 1068 1309 and amp range 0 2039
Range in image: x 0 241 , y 0 242 (with image size 241 x 242)
dzp -0.00064781494 -> scaling image by 0.9994035183523652
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
90prime get_fwhm: self.fwhm = 2.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3545 3686 and amp range 2016 4031 and image y 3080 3220 and amp range 2048 4095
Range in image: x 0 142 , y 0 141 (with image size 142 x 141)
dzp 0.019008487 -> scaling image by 1.0176616178968958
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
90prime get_fwhm: self.fwhm = 4.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3546 3686 and amp range 2016 4031 and image y 3122 3262 and amp range 2048 4095
Range in image: x 0 141 , y 0 141 (with image size 141 x 141)
dzp 0.0026774607 -> scaling image by 1.002469075566243
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75180254
Faked up EXPNUM 75180254
Faked up EXPNUM 75180254
90prime get_fwhm: self.fwhm = 2.11
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3790 3929 and amp range 2016 4031 and image y 385 524 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp 0.004245165 -> scaling image by 1.0039175950705865
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 734 979 and amp range 0 2005 and image y 2438 2685 and amp range 2040 4078
Range in image: x 0 246 , y 0 248 (with image size 246 x 248)
dzp 0.0022078273 -> scaling image by 1.0020355530120029
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75120205
Faked up EXPNUM 75120205
Faked up EXPNUM 75120205
90prime get_fwhm: self.fwhm = 3.31
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1786 1926 and amp range 0 2015 and image y 2855 2995 and amp range 2048 4095
Range in image: x 0 141 , y 0 141 (with image size 141 x 141)
dzp -0.0051191514 -> scaling image by 0.9952961850385206
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75080186
Faked up EXPNUM 75080186
Faked up EXPNUM 75080186
90prime get_fwhm: self.fwhm = 2.67
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1789 1929 and amp range 0 2015 and image y 2859 2999 and amp range 2048 4095
Range in image: x 0 141 , y 0 141 (with image size 141 x 141)
dzp -0.00015519658 -> scaling image by 0.9998570688798908
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 11.86 s, CPU: 1.15 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 206 MB, maxrss: 0.177976 MB
Stage tims finished: 2020-08-05T19:12:38.060025
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 4.804 s worker CPU, 43.611 s worker Wall, Wall: 11.867 s, Cores in use: 0.50, Total efficiency (on 16 cores): 3.1 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-tims.p
Running stage refs
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T19:12:38.082227
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02850.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/reference-custom-276261p34282.fits
Stage refs : Wall: 10.59 s, CPU: 8.44 s, VmPeak: 1061 MB, VmSize: 1056 MB, VmRSS: 274 MB, VmData: 304 MB, maxrss: 0.28106 MB
Stage refs finished: 2020-08-05T19:12:48.670388
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 10.590 s, Cores in use: 0.80, Total efficiency (on 16 cores): 5.0 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T19:12:48.695378
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outlier-mask-custom-276261p34282.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-pre-custom-276261p34282.jpg
Image ksb_160702_081612_ooi_g_ls9-CCD2 g : masking 235 positive outlier pixels and 0 negative outlier pixels
Image ksb_180509_110248_ooi_r_ls9-CCD2 r : masking 0 positive outlier pixels and 154 negative outlier pixels
Image k4m_160429_110817_ooi_zd_ls9-CCD2 z : masking 286 positive outlier pixels and 0 negative outlier pixels
Image k4m_160414_120201_ooi_zd_ls9-CCD3 z : masking 357 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outlier-mask-custom-276261p34282.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-post-custom-276261p34282.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-masked-pos-custom-276261p34282.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-masked-neg-custom-276261p34282.jpg
Stage outliers : Wall: 2.85 s, CPU: 1.18 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 287 MB, VmData: 314 MB, maxrss: 0.29432 MB
Stage outliers finished: 2020-08-05T19:12:51.542605
Resources for stage outliers : 0.706 s worker CPU, 0.772 s worker Wall, Wall: 2.849 s, Cores in use: 0.66, Total efficiency (on 16 cores): 4.1 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T19:12:51.616600
Stage halos : Wall: 1.30 s, CPU: 1.25 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 287 MB, VmData: 314 MB, maxrss: 0.294324 MB
Stage halos finished: 2020-08-05T19:12:52.914598
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.299 s, Cores in use: 0.96, Total efficiency (on 16 cores): 6.0 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T19:12:52.939459
PSF sigmas in coadd pixels: ['2.01', '2.44', '3.20']
Using average PSF sigma 2.5485454615782905
Gaussian PSF norm 0.11068854608509886 vs pixelized 0.090572864
average tim pixel scale / coadd scale: 1.729256140231688
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.55', '1.97', '1.92']
Using average PSF sigma 1.8133085508575446
Gaussian PSF norm 0.15556910689053483 vs pixelized 0.12600622
average tim pixel scale / coadd scale: 1.7298810505699223
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.38', '1.61', '2.32']
Using average PSF sigma 1.7668533620224107
Gaussian PSF norm 0.15965942496268123 vs pixelized 0.1396107
average tim pixel scale / coadd scale: 0.9909501692052675
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.13 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 290 MB, VmData: 315 MB, maxrss: 0.296504 MB
Stage fit_on_coadds finished: 2020-08-05T19:12:53.119132
Resources for stage fit_on_coadds : 0.149 s worker CPU, 0.149 s worker Wall, Wall: 0.181 s, Cores in use: 1.55, Total efficiency (on 16 cores): 9.7 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fit_on_coadds.p
Running stage srcs
Warning: Stellar halo subtraction is only implemented for DECam
Running stage srcs at 2020-08-05T19:12:53.135759
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 37 potential peaks
Of 37 potential peaks: 14 in veto map, 4 cut by saddle test, 0 cut by aper test, 19 kept
SED z : 19 new peaks
Found 42 potential peaks
Of 42 potential peaks: 37 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 34 potential peaks
Of 34 potential peaks: 29 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept
SED g : 1 new peaks
Found 47 potential peaks
Of 47 potential peaks: 40 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 51 potential peaks
Of 51 potential peaks: 43 in veto map, 8 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 22 new sources
Median coadd sky for g : 0.000392333
Median coadd sky for r : 0.0009858158
Median coadd sky for z : -0.00035302492
Stage srcs : Wall: 1.39 s, CPU: 1.33 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 290 MB, VmData: 315 MB, maxrss: 0.297168 MB
Stage srcs finished: 2020-08-05T19:12:54.526810
Resources for stage srcs : 0.072 s worker CPU, 0.072 s worker Wall, Wall: 1.392 s, Cores in use: 1.01, Total efficiency (on 16 cores): 6.3 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T19:12:54.541278
Blob 1 of 11, id: 0, sources: 19, size: 210x182, npix 19441, brick X: 0,210, Y: 0,182, one pixel: 1 0
Blob 2 of 11, id: 6, sources: 5, size: 86x90, npix 4545, brick X: 152,238, Y: 136,226, one pixel: 192 136
Blob 3 of 11, id: 8, sources: 3, size: 103x51, npix 3355, brick X: 65,168, Y: 188,239, one pixel: 120 188
Blob 4 of 11, id: 2, sources: 2, size: 35x63, npix 1364, brick X: 204,239, Y: 51,114, one pixel: 220 51
Blob 5 of 11, id: 4, sources: 1, size: 35x35, npix 837, brick X: 0,35, Y: 112,147, one pixel: 8 112
Blob 6 of 11, id: 9, sources: 1, size: 35x29, npix 725, brick X: 28,63, Y: 210,239, one pixel: 40 210
Blob 7 of 11, id: 3, sources: 1, size: 31x31, npix 622, brick X: 2,33, Y: 77,108, one pixel: 19 77
Blob 8 of 11, id: 10, sources: 4, size: 30x25, npix 577, brick X: 209,239, Y: 214,239, one pixel: 230 214
Blob 9 of 11, id: 1, sources: 1, size: 30x18, npix 365, brick X: 191,221, Y: 0,18, one pixel: 195 0
Blob 10 of 11, id: 5, sources: 1, size: 23x22, npix 324, brick X: 40,63, Y: 114,136, one pixel: 49 114
Blob 11 of 11, id: 7, sources: 1, size: 19x19, npix 199, brick X: 27,46, Y: 140,159, one pixel: 36 140
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.21633, 34.31537), pm (21.8, -13.6), parallax 1.170 with NanoMaggies: g=12.3, r=11.5, z=11.1
Reset brightness to NanoMaggies: g=12.3, r=11.5, z=11.1
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.22931, 34.29391), pm (-6.1, -37.2), parallax 1.150 with NanoMaggies: g=14.1, r=13.7, z=13.5
Reset brightness to NanoMaggies: g=14.1, r=13.7, z=13.5
Blob 8 finished, total: Wall: 0.18 s, CPU: 0.18 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150164 MB
Blob 11 finished, total: Wall: 0.24 s, CPU: 0.23 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149848 MB
Blob 10 finished, total: Wall: 0.26 s, CPU: 0.25 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 92 MB, VmData: 141 MB, maxrss: 0.0945 MB
Measuring 1 iterative sources
Blob 9 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.094896 MB
Blob 5 finished, total: Wall: 0.59 s, CPU: 0.58 s, VmPeak: 808 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149568 MB
Blob 7 finished, total: Wall: 0.59 s, CPU: 0.59 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093008 MB
Blob 6 finished, total: Wall: 0.86 s, CPU: 0.85 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149784 MB
Measuring 1 iterative sources
Blob 4 finished, total: Wall: 1.53 s, CPU: 1.52 s, VmPeak: 808 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149548 MB
Measuring 1 iterative sources
Measuring 2 iterative sources
Blob 3 finished, total: Wall: 2.74 s, CPU: 2.74 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 96 MB, VmData: 142 MB, maxrss: 0.098128 MB
Blob 2 finished, total: Wall: 3.65 s, CPU: 3.65 s, VmPeak: 673 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.099828 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.27722, 34.27539), pm (-0.9, 3.0), parallax 0.958 with NanoMaggies: g=16, r=15.4, z=15.2
Reset brightness to NanoMaggies: g=16, r=15.4, z=15.2
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.4054, ee1=0.0649449, ee2=0.335952
Measuring 2 iterative sources
Blob 1 finished, total: Wall: 12.01 s, CPU: 12.01 s, VmPeak: 814 MB, VmSize: 733 MB, VmRSS: 155 MB, VmData: 184 MB, maxrss: 0.159192 MB
Old catalog: 51
New catalog: 43
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/blobs-custom-276261p34282.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/all-models-custom-276261p34282.fits
Stage fitblobs : Wall: 13.26 s, CPU: 1.08 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 292 MB, VmData: 316 MB, maxrss: 0.298612 MB
Stage fitblobs finished: 2020-08-05T19:13:07.798296
Resources for stage fitblobs : 23.087 s worker CPU, 23.593 s worker Wall, Wall: 13.258 s, Cores in use: 1.82, Total efficiency (on 16 cores): 11.4 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T19:13:07.820419
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-resid.jpg
Stage coadds : Wall: 3.32 s, CPU: 1.32 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 292 MB, VmData: 316 MB, maxrss: 0.298932 MB
Stage coadds finished: 2020-08-05T19:13:11.136548
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 9.642 s worker CPU, 15.292 s worker Wall, Wall: 3.318 s, Cores in use: 3.30, Total efficiency (on 16 cores): 20.6 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T19:13:11.153560
Cut to 1 unWISE tiles
Reading WISE tile 2759p348 band 1
Reading WISE tile 2759p348 band 2
Reading WISE tile 2759p348 band 3
Reading WISE tile 2759p348 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 807 MB, VmSize: 805 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.192204 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 812 MB, VmSize: 799 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.204776 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 808 MB, VmSize: 806 MB, VmRSS: 147 MB, VmData: 174 MB, maxrss: 0.210548 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 813 MB, VmSize: 804 MB, VmRSS: 145 MB, VmData: 172 MB, maxrss: 0.208404 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.58 s, CPU: 0.85 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 292 MB, VmData: 316 MB, maxrss: 0.299448 MB
Stage wise_forced finished: 2020-08-05T19:13:13.737546
Resources for stage wise_forced : 3.057 s worker CPU, 5.152 s worker Wall, Wall: 2.585 s, Cores in use: 1.51, Total efficiency (on 16 cores): 9.4 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T19:13:13.752128
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/tractor-i/cus/tractor-custom-276261p34282.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/tractor/cus/tractor-custom-276261p34282.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/tractor/cus/brick-custom-276261p34282.sha256sum
Stage writecat : Wall: 0.94 s, CPU: 0.66 s, VmPeak: 1197 MB, VmSize: 1069 MB, VmRSS: 294 MB, VmData: 316 MB, maxrss: 0.432328 MB
Stage writecat finished: 2020-08-05T19:13:14.688087
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.938 s, Cores in use: 0.71, Total efficiency (on 16 cores): 4.4 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-writecat.p
All done: 41.521 s worker CPU, 88.646 s worker Wall, Wall: 48.514 s, Cores in use: 1.22, Total efficiency (on 16 cores): 7.6 %
Finished galaxy PGC2046768 in 0.958 minutes.
Started working on galaxy PGC2046768 at Wed Aug  5 19:34:10 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.26172 34.2825775 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T19:34:11.556132
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.26172', '34.2825775', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.26172 34.2825775 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.26172 34.2825775
Starting process 41869 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 41870 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 41871 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 41872 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 41873 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059352 MB
Starting process 41874 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05934 MB
Starting process 41875 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05934 MB
Starting process 41876 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 41877 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 41878 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 41879 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 41880 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 41881 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 41882 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 41883 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 41884 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 41885 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 41886 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB
Starting process 41887 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 41888 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 41889 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 41890 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 41891 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 41892 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 41893 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 41894 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 41895 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB
Starting process 41896 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 41897 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB
Starting process 41898 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB
Starting process 41899 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB
Starting process 41900 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05932 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T19:34:12.574265
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 9 CCDs:
  k4m_160414_120201_ooi_zd_ls9-CCD3 z expnum 82278 exptime 80.104 propid 2016A-0453 seeing 0.85 MJD 57492.501 object MzLS_575293_z 
    mosaic/CP/V4.3/CP20160413/k4m_160414_120201_ooi_zd_ls9.fits.fz
  k4m_160415_115629_ooi_zd_ls9-CCD3 z expnum 82535 exptime 85.104 propid 2016A-0453 seeing 0.99 MJD 57493.498 object MzLS_576327_z 
    mosaic/CP/V4.3/CP20160414/k4m_160415_115629_ooi_zd_ls9.fits.fz
  k4m_160429_110817_ooi_zd_ls9-CCD2 z expnum 85812 exptime 84.103 propid 2016A-0453 seeing 1.43 MJD 57507.464 object MzLS_574257_z 
    mosaic/CP/V4.3/CP20160428/k4m_160429_110817_ooi_zd_ls9.fits.fz
  ksb_160602_084734_ooi_g_ls9-CCD3 g expnum 75410137 exptime 139.4 propid BASS seeing 1.24 MJD 57541.366 object 06922 
    90prime/CP/V2.3/CP20160601/ksb_160602_084734_ooi_g_ls9.fits.fz
  ksb_160510_110001_ooi_r_ls9-CCD3 r expnum 75180254 exptime 67.9 propid BASS seeing 0.96 MJD 57518.458 object 06922 
    90prime/CP/V2.3/CP20160509/ksb_160510_110001_ooi_r_ls9.fits.fz
  ksb_160504_104228_ooi_g_ls9-CCD1 g expnum 75120205 exptime 118.3 propid BASS seeing 1.50 MJD 57512.446 object 06921 
    90prime/CP/V2.3/CP20160503/ksb_160504_104228_ooi_g_ls9.fits.fz
  ksb_160430_104359_ooi_r_ls9-CCD1 r expnum 75080186 exptime 155.6 propid BASS seeing 1.21 MJD 57508.447 object 06921 
    90prime/CP/V2.3/CP20160429/ksb_160430_104359_ooi_r_ls9.fits.fz
  ksb_180509_110248_ooi_r_ls9-CCD2 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.18 MJD 58247.460 object 08283 
    90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz
  ksb_160702_081612_ooi_g_ls9-CCD2 g expnum 75710100 exptime 134.2 propid BASS seeing 1.97 MJD 57571.345 object 08283 
    90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 5.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1848 2090 and amp range 0 2047 and image y 1507 1751 and amp range 0 2039
Range in image: x 0 200 , y 0 245 (with image size 243 x 245)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 1848 2090 and amp range 2048 4095 and image y 1507 1751 and amp range 0 2039
Range in image: x 200 243 , y 0 245 (with image size 243 x 245)
dzp 0.0072022504 -> scaling image by 1.0066555682883527
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75410137
Faked up EXPNUM 75410137
Faked up EXPNUM 75410137
90prime get_fwhm: self.fwhm = 2.73
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3791 3930 and amp range 2016 4031 and image y 374 513 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp -0.0022715968 -> scaling image by 0.9979099691834846
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
90prime get_fwhm: self.fwhm = 2.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3545 3686 and amp range 2016 4031 and image y 3080 3220 and amp range 2048 4095
Range in image: x 0 142 , y 0 141 (with image size 142 x 141)
dzp 0.019008487 -> scaling image by 1.0176616178968958
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
90prime get_fwhm: self.fwhm = 4.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3546 3686 and amp range 2016 4031 and image y 3122 3262 and amp range 2048 4095
Range in image: x 0 141 , y 0 141 (with image size 141 x 141)
dzp 0.0026774607 -> scaling image by 1.002469075566243
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.27
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3471 3711 and amp range 2006 4053 and image y 1068 1309 and amp range 0 2039
Range in image: x 0 241 , y 0 242 (with image size 241 x 242)
dzp -0.00064781494 -> scaling image by 0.9994035183523652
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75180254
Faked up EXPNUM 75180254
Faked up EXPNUM 75180254
90prime get_fwhm: self.fwhm = 2.11
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3790 3929 and amp range 2016 4031 and image y 385 524 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp 0.004245165 -> scaling image by 1.0039175950705865
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 734 979 and amp range 0 2005 and image y 2438 2685 and amp range 2040 4078
Range in image: x 0 246 , y 0 248 (with image size 246 x 248)
dzp 0.0022078273 -> scaling image by 1.0020355530120029
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75120205
Faked up EXPNUM 75120205
Faked up EXPNUM 75120205
90prime get_fwhm: self.fwhm = 3.31
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1786 1926 and amp range 0 2015 and image y 2855 2995 and amp range 2048 4095
Range in image: x 0 141 , y 0 141 (with image size 141 x 141)
dzp -0.0051191514 -> scaling image by 0.9952961850385206
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75080186
Faked up EXPNUM 75080186
Faked up EXPNUM 75080186
90prime get_fwhm: self.fwhm = 2.67
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1789 1929 and amp range 0 2015 and image y 2859 2999 and amp range 2048 4095
Range in image: x 0 141 , y 0 141 (with image size 141 x 141)
dzp -0.00015519658 -> scaling image by 0.9998570688798908
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 20.48 s, CPU: 1.46 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 206 MB, maxrss: 0.178112 MB
Stage tims finished: 2020-08-05T19:34:33.049930
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 4.774 s worker CPU, 61.327 s worker Wall, Wall: 20.478 s, Cores in use: 0.30, Total efficiency (on 32 cores): 1.0 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-08-05T19:34:33.080858
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02850.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/reference-custom-276261p34282.fits
Stage refs : Wall: 12.37 s, CPU: 8.46 s, VmPeak: 1061 MB, VmSize: 1056 MB, VmRSS: 275 MB, VmData: 304 MB, maxrss: 0.28138 MB
Stage refs finished: 2020-08-05T19:34:45.447582
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 12.369 s, Cores in use: 0.68, Total efficiency (on 32 cores): 2.1 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T19:34:45.475344
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outlier-mask-custom-276261p34282.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-pre-custom-276261p34282.jpg
Image ksb_160702_081612_ooi_g_ls9-CCD2 g : masking 235 positive outlier pixels and 0 negative outlier pixels
Image ksb_180509_110248_ooi_r_ls9-CCD2 r : masking 0 positive outlier pixels and 154 negative outlier pixels
Image k4m_160414_120201_ooi_zd_ls9-CCD3 z : masking 357 positive outlier pixels and 0 negative outlier pixels
Image k4m_160429_110817_ooi_zd_ls9-CCD2 z : masking 286 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outlier-mask-custom-276261p34282.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-post-custom-276261p34282.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-masked-pos-custom-276261p34282.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/outliers-masked-neg-custom-276261p34282.jpg
Stage outliers : Wall: 2.09 s, CPU: 1.36 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 288 MB, VmData: 315 MB, maxrss: 0.295084 MB
Stage outliers finished: 2020-08-05T19:34:47.569205
Resources for stage outliers : 0.735 s worker CPU, 0.928 s worker Wall, Wall: 2.096 s, Cores in use: 1.00, Total efficiency (on 32 cores): 3.1 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-outliers.p
Running stage halos
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage halos at 2020-08-05T19:34:47.600859
Stage halos : Wall: 1.92 s, CPU: 1.86 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 288 MB, VmData: 315 MB, maxrss: 0.29514 MB
Stage halos finished: 2020-08-05T19:34:49.522344
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.924 s, Cores in use: 0.97, Total efficiency (on 32 cores): 3.0 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-05T19:34:49.587591
PSF sigmas in coadd pixels: ['2.01', '2.44', '3.20']
Using average PSF sigma 2.5485454615782905
Gaussian PSF norm 0.11068854608509886 vs pixelized 0.090572864
average tim pixel scale / coadd scale: 1.729256140231688
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.55', '1.97', '1.92']
Using average PSF sigma 1.8133085508575446
Gaussian PSF norm 0.15556910689053483 vs pixelized 0.12600622
average tim pixel scale / coadd scale: 1.7298810505699223
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.38', '1.61', '2.32']
Using average PSF sigma 1.7668533620224107
Gaussian PSF norm 0.15965942496268123 vs pixelized 0.1396107
average tim pixel scale / coadd scale: 0.9909501692052675
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.13 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 290 MB, VmData: 315 MB, maxrss: 0.29666 MB
Stage fit_on_coadds finished: 2020-08-05T19:34:49.766077
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage fit_on_coadds : 0.157 s worker CPU, 0.158 s worker Wall, Wall: 0.180 s, Cores in use: 1.63, Total efficiency (on 32 cores): 5.1 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fit_on_coadds.p
Running stage srcs
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage srcs at 2020-08-05T19:34:49.786968
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 37 potential peaks
Of 37 potential peaks: 14 in veto map, 4 cut by saddle test, 0 cut by aper test, 19 kept
SED z : 19 new peaks
Found 42 potential peaks
Of 42 potential peaks: 37 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 34 potential peaks
Of 34 potential peaks: 29 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept
SED g : 1 new peaks
Found 47 potential peaks
Of 47 potential peaks: 40 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 51 potential peaks
Of 51 potential peaks: 43 in veto map, 8 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 22 new sources
Median coadd sky for g : 0.000392333
Median coadd sky for r : 0.0009858158
Median coadd sky for z : -0.00035302492
Stage srcs : Wall: 2.40 s, CPU: 2.33 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 290 MB, VmData: 315 MB, maxrss: 0.297164 MB
Stage srcs finished: 2020-08-05T19:34:52.190100
Resources for stage srcs : 0.072 s worker CPU, 0.072 s worker Wall, Wall: 2.405 s, Cores in use: 1.00, Total efficiency (on 32 cores): 3.1 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T19:34:52.214978
Blob 1 of 11, id: 0, sources: 19, size: 210x182, npix 19441, brick X: 0,210, Y: 0,182, one pixel: 1 0
Blob 2 of 11, id: 6, sources: 5, size: 86x90, npix 4545, brick X: 152,238, Y: 136,226, one pixel: 192 136
Blob 3 of 11, id: 8, sources: 3, size: 103x51, npix 3355, brick X: 65,168, Y: 188,239, one pixel: 120 188
Blob 4 of 11, id: 2, sources: 2, size: 35x63, npix 1364, brick X: 204,239, Y: 51,114, one pixel: 220 51
Blob 5 of 11, id: 4, sources: 1, size: 35x35, npix 837, brick X: 0,35, Y: 112,147, one pixel: 8 112
Blob 6 of 11, id: 9, sources: 1, size: 35x29, npix 725, brick X: 28,63, Y: 210,239, one pixel: 40 210
Blob 7 of 11, id: 3, sources: 1, size: 31x31, npix 622, brick X: 2,33, Y: 77,108, one pixel: 19 77
Blob 8 of 11, id: 10, sources: 4, size: 30x25, npix 577, brick X: 209,239, Y: 214,239, one pixel: 230 214
Blob 9 of 11, id: 1, sources: 1, size: 30x18, npix 365, brick X: 191,221, Y: 0,18, one pixel: 195 0
Blob 10 of 11, id: 5, sources: 1, size: 23x22, npix 324, brick X: 40,63, Y: 114,136, one pixel: 49 114
Blob 11 of 11, id: 7, sources: 1, size: 19x19, npix 199, brick X: 27,46, Y: 140,159, one pixel: 36 140
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.21633, 34.31537), pm (21.8, -13.6), parallax 1.170 with NanoMaggies: g=12.3, r=11.5, z=11.1
Reset brightness to NanoMaggies: g=12.3, r=11.5, z=11.1
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.22931, 34.29391), pm (-6.1, -37.2), parallax 1.150 with NanoMaggies: g=14.1, r=13.7, z=13.5
Reset brightness to NanoMaggies: g=14.1, r=13.7, z=13.5
Blob 8 finished, total: Wall: 0.16 s, CPU: 0.15 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 88 MB, VmData: 140 MB, maxrss: 0.089876 MB
Blob 11 finished, total: Wall: 0.16 s, CPU: 0.16 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.09334 MB
Blob 10 finished, total: Wall: 0.17 s, CPU: 0.17 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092452 MB
Measuring 1 iterative sources
Blob 9 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09248 MB
Blob 5 finished, total: Wall: 0.53 s, CPU: 0.52 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.09232 MB
Blob 7 finished, total: Wall: 0.61 s, CPU: 0.61 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093264 MB
Blob 6 finished, total: Wall: 0.76 s, CPU: 0.76 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.092688 MB
Measuring 1 iterative sources
Blob 4 finished, total: Wall: 1.42 s, CPU: 1.41 s, VmPeak: 807 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149508 MB
Measuring 1 iterative sources
Measuring 2 iterative sources
Blob 3 finished, total: Wall: 2.93 s, CPU: 2.92 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 96 MB, VmData: 142 MB, maxrss: 0.097892 MB
Blob 2 finished, total: Wall: 3.79 s, CPU: 3.78 s, VmPeak: 673 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.099172 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.27722, 34.27539), pm (-0.9, 3.0), parallax 0.958 with NanoMaggies: g=16, r=15.4, z=15.2
Reset brightness to NanoMaggies: g=16, r=15.4, z=15.2
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.4054, ee1=0.0649451, ee2=0.335953
Measuring 2 iterative sources
Blob 1 finished, total: Wall: 12.04 s, CPU: 12.03 s, VmPeak: 691 MB, VmSize: 691 MB, VmRSS: 124 MB, VmData: 162 MB, maxrss: 0.126664 MB
Old catalog: 51
New catalog: 43
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/blobs-custom-276261p34282.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/metrics/cus/all-models-custom-276261p34282.fits
Stage fitblobs : Wall: 13.48 s, CPU: 1.35 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 292 MB, VmData: 316 MB, maxrss: 0.298608 MB
Stage fitblobs finished: 2020-08-05T19:35:05.697887
Resources for stage fitblobs : 22.911 s worker CPU, 23.019 s worker Wall, Wall: 13.485 s, Cores in use: 1.80, Total efficiency (on 32 cores): 5.6 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T19:35:05.712939
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-resid.jpg
Stage coadds : Wall: 3.25 s, CPU: 0.65 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 292 MB, VmData: 316 MB, maxrss: 0.298932 MB
Stage coadds finished: 2020-08-05T19:35:08.961216
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 26.699 s worker CPU, 48.328 s worker Wall, Wall: 3.251 s, Cores in use: 8.41, Total efficiency (on 32 cores): 26.3 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T19:35:08.987863
Cut to 1 unWISE tiles
Reading WISE tile 2759p348 band 1
Reading WISE tile 2759p348 band 3
Reading WISE tile 2759p348 band 4
Reading WISE tile 2759p348 band 2
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 809 MB, VmSize: 809 MB, VmRSS: 150 MB, VmData: 176 MB, maxrss: 0.19484 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 798 MB, VmSize: 798 MB, VmRSS: 137 MB, VmData: 170 MB, maxrss: 0.1971 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 816 MB, VmSize: 799 MB, VmRSS: 140 MB, VmData: 171 MB, maxrss: 0.20228 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 806 MB, VmSize: 804 MB, VmRSS: 143 MB, VmData: 172 MB, maxrss: 0.205924 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 3.51 s, CPU: 1.48 s, VmPeak: 1065 MB, VmSize: 1065 MB, VmRSS: 292 MB, VmData: 316 MB, maxrss: 0.299444 MB
Stage wise_forced finished: 2020-08-05T19:35:12.497280
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage wise_forced : 3.243 s worker CPU, 5.360 s worker Wall, Wall: 3.512 s, Cores in use: 1.35, Total efficiency (on 32 cores): 4.2 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-wise_forced.p
Running stage writecat
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage writecat at 2020-08-05T19:35:12.515805
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/tractor-i/cus/tractor-custom-276261p34282.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/tractor/cus/tractor-custom-276261p34282.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/tractor/cus/brick-custom-276261p34282.sha256sum
Stage writecat : Wall: 3.15 s, CPU: 0.79 s, VmPeak: 1197 MB, VmSize: 1069 MB, VmRSS: 294 MB, VmData: 316 MB, maxrss: 0.432308 MB
Stage writecat finished: 2020-08-05T19:35:15.661245
Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 3.148 s, Cores in use: 0.25, Total efficiency (on 32 cores): 0.8 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/276/PGC2046768/PGC2046768-largegalaxy-writecat.p
All done: 58.597 s worker CPU, 139.199 s worker Wall, Wall: 63.120 s, Cores in use: 1.25, Total efficiency (on 32 cores): 3.9 %
Finished galaxy PGC2046768 in 1.107 minutes.
Started working on galaxy PGC2046768 at Fri Aug  7 09:17:19 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.26150503365807 34.28264743273207 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-07T09:17:21.986197
legacypipe git version: DR9.5.11-watershed-3-gd509eaaf
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '276.26150503365807', '34.28264743273207', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 276.26150503365807 34.28264743273207 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 276.26150503365807 34.28264743273207
Starting process 6211 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 6213 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 6215 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 6217 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 6219 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059536 MB
Starting process 6221 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB
Starting process 6223 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB
Starting process 6225 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 6227 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 6229 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 6231 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 6233 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 6235 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 6239 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 6237 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 6241 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-07T09:17:33.098987
Keeping 9 CCDs:
  k4m_160414_120201_ooi_zd_ls9-CCD3 z expnum 82278 exptime 80.104 propid 2016A-0453 seeing 0.85 MJD 57492.501 object MzLS_575293_z 
    mosaic/CP/V4.3/CP20160413/k4m_160414_120201_ooi_zd_ls9.fits.fz
  k4m_160415_115629_ooi_zd_ls9-CCD3 z expnum 82535 exptime 85.104 propid 2016A-0453 seeing 0.99 MJD 57493.498 object MzLS_576327_z 
    mosaic/CP/V4.3/CP20160414/k4m_160415_115629_ooi_zd_ls9.fits.fz
  k4m_160429_110817_ooi_zd_ls9-CCD2 z expnum 85812 exptime 84.103 propid 2016A-0453 seeing 1.43 MJD 57507.464 object MzLS_574257_z 
    mosaic/CP/V4.3/CP20160428/k4m_160429_110817_ooi_zd_ls9.fits.fz
  ksb_160602_084734_ooi_g_ls9-CCD3 g expnum 75410137 exptime 139.4 propid BASS seeing 1.24 MJD 57541.366 object 06922 
    90prime/CP/V2.3/CP20160601/ksb_160602_084734_ooi_g_ls9.fits.fz
  ksb_160510_110001_ooi_r_ls9-CCD3 r expnum 75180254 exptime 67.9 propid BASS seeing 0.96 MJD 57518.458 object 06922 
    90prime/CP/V2.3/CP20160509/ksb_160510_110001_ooi_r_ls9.fits.fz
  ksb_160504_104228_ooi_g_ls9-CCD1 g expnum 75120205 exptime 118.3 propid BASS seeing 1.50 MJD 57512.446 object 06921 
    90prime/CP/V2.3/CP20160503/ksb_160504_104228_ooi_g_ls9.fits.fz
  ksb_160430_104359_ooi_r_ls9-CCD1 r expnum 75080186 exptime 155.6 propid BASS seeing 1.21 MJD 57508.447 object 06921 
    90prime/CP/V2.3/CP20160429/ksb_160430_104359_ooi_r_ls9.fits.fz
  ksb_180509_110248_ooi_r_ls9-CCD2 r expnum 82470196 exptime 101.925 propid NOTBASS seeing 1.18 MJD 58247.460 object 08283 
    90prime/CP/V2.3/CP20180508/ksb_180509_110248_ooi_r_ls9.fits.fz
  ksb_160702_081612_ooi_g_ls9-CCD2 g expnum 75710100 exptime 134.2 propid BASS seeing 1.97 MJD 57571.345 object 08283 
    90prime/CP/V2.3/CP20160701/ksb_160702_081612_ooi_g_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 3.27
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3472 3712 and amp range 2006 4053 and image y 1066 1307 and amp range 0 2039
Range in image: x 0 241 , y 0 242 (with image size 241 x 242)
dzp -0.00064781494 -> scaling image by 0.9994035183523652
mosaic get_fwhm: self.fwhm = 5.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1849 2091 and amp range 0 2047 and image y 1505 1749 and amp range 0 2039
Range in image: x 0 199 , y 0 245 (with image size 243 x 245)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 1849 2091 and amp range 2048 4095 and image y 1505 1749 and amp range 0 2039
Range in image: x 199 243 , y 0 245 (with image size 243 x 245)
dzp 0.0072022504 -> scaling image by 1.0066555682883527
mosaic get_fwhm: self.fwhm = 3.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 735 980 and amp range 0 2005 and image y 2436 2683 and amp range 2040 4078
Range in image: x 0 246 , y 0 248 (with image size 246 x 248)
dzp 0.0022078273 -> scaling image by 1.0020355530120029
Faked up EXPNUM 75120205
Faked up EXPNUM 75120205
Faked up EXPNUM 75120205
90prime get_fwhm: self.fwhm = 3.31
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1786 1926 and amp range 0 2015 and image y 2857 2996 and amp range 2048 4095
Range in image: x 0 141 , y 0 140 (with image size 141 x 140)
dzp -0.0051191514 -> scaling image by 0.9952961850385206
Faked up EXPNUM 75410137
Faked up EXPNUM 75410137
Faked up EXPNUM 75410137
90prime get_fwhm: self.fwhm = 2.73
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3791 3930 and amp range 2016 4031 and image y 376 515 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp -0.0022715968 -> scaling image by 0.9979099691834846
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
Faked up EXPNUM 82470196
90prime get_fwhm: self.fwhm = 2.61
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3544 3685 and amp range 2016 4031 and image y 3082 3222 and amp range 2048 4095
Range in image: x 0 142 , y 0 141 (with image size 142 x 141)
dzp 0.019008487 -> scaling image by 1.0176616178968958
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
Faked up EXPNUM 75710100
90prime get_fwhm: self.fwhm = 4.35
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3545 3686 and amp range 2016 4031 and image y 3123 3263 and amp range 2048 4095
Range in image: x 0 142 , y 0 141 (with image size 142 x 141)
dzp 0.0026774607 -> scaling image by 1.002469075566243
Faked up EXPNUM 75080186
Faked up EXPNUM 75080186
Faked up EXPNUM 75080186
90prime get_fwhm: self.fwhm = 2.67
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1788 1928 and amp range 0 2015 and image y 2860 3000 and amp range 2048 4095
Range in image: x 0 141 , y 0 141 (with image size 141 x 141)
dzp -0.00015519658 -> scaling image by 0.9998570688798908
Faked up EXPNUM 75180254
Faked up EXPNUM 75180254
Faked up EXPNUM 75180254
90prime get_fwhm: self.fwhm = 2.11
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3789 3928 and amp range 2016 4031 and image y 386 525 and amp range 0 2047
Range in image: x 0 140 , y 0 140 (with image size 140 x 140)
dzp 0.004245165 -> scaling image by 1.0039175950705865
Stage tims : Wall: 98.85 s, CPU: 5.65 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 172 MB, VmData: 206 MB, maxrss: 0.175752 MB
Stage tims finished: 2020-08-07T09:19:11.950182
Resources for stage tims : 4.261 s worker CPU, 410.924 s worker Wall, Wall: 98.853 s, Cores in use: 0.10, Total efficiency (on 16 cores): 0.6 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-08-07T09:19:12.026429
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02722.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02850.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02594.fits
Matched 7 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/reference-custom-276261p34282.fits
Stage refs : Wall: 8.53 s, CPU: 8.08 s, VmPeak: 1055 MB, VmSize: 1049 MB, VmRSS: 273 MB, VmData: 304 MB, maxrss: 0.279028 MB
Stage refs finished: 2020-08-07T09:19:20.551817
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 8.527 s, Cores in use: 0.95, Total efficiency (on 16 cores): 5.9 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-07T09:19:20.570926
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/outlier-mask-custom-276261p34282.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/outliers-pre-custom-276261p34282.jpg
Image ksb_160702_081612_ooi_g_ls9-CCD2 g : masking 231 positive outlier pixels and 0 negative outlier pixels
Image ksb_180509_110248_ooi_r_ls9-CCD2 r : masking 0 positive outlier pixels and 158 negative outlier pixels
Image k4m_160429_110817_ooi_zd_ls9-CCD2 z : masking 287 positive outlier pixels and 0 negative outlier pixels
Image k4m_160414_120201_ooi_zd_ls9-CCD3 z : masking 360 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/outlier-mask-custom-276261p34282.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/outliers-post-custom-276261p34282.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/outliers-masked-pos-custom-276261p34282.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/outliers-masked-neg-custom-276261p34282.jpg
Stage outliers : Wall: 11.58 s, CPU: 0.62 s, VmPeak: 1059 MB, VmSize: 1059 MB, VmRSS: 286 MB, VmData: 315 MB, maxrss: 0.293008 MB
Stage outliers finished: 2020-08-07T09:19:32.147845
Resources for stage outliers : 0.716 s worker CPU, 0.837 s worker Wall, Wall: 11.578 s, Cores in use: 0.12, Total efficiency (on 16 cores): 0.7 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-07T09:19:32.174398
Stage halos : Wall: 0.41 s, CPU: 0.40 s, VmPeak: 1059 MB, VmSize: 1059 MB, VmRSS: 286 MB, VmData: 315 MB, maxrss: 0.293048 MB
Stage halos finished: 2020-08-07T09:19:32.589304
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.416 s, Cores in use: 0.97, Total efficiency (on 16 cores): 6.1 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-07T09:19:33.667597
PSF sigmas in coadd pixels: ['2.01', '2.44', '3.20']
Using average PSF sigma 2.5485454615782905
Gaussian PSF norm 0.11068854608509886 vs pixelized 0.09057238
average tim pixel scale / coadd scale: 1.729256140231688
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.55', '1.97', '1.92']
Using average PSF sigma 1.8133085508575446
Gaussian PSF norm 0.15556910689053483 vs pixelized 0.12600923
average tim pixel scale / coadd scale: 1.7298810505699223
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.38', '1.61', '2.32']
Using average PSF sigma 1.7668533620224107
Gaussian PSF norm 0.15965942496268123 vs pixelized 0.13960701
average tim pixel scale / coadd scale: 0.9909501692052675
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 5.39 s, CPU: 0.13 s, VmPeak: 1059 MB, VmSize: 1059 MB, VmRSS: 288 MB, VmData: 315 MB, maxrss: 0.294512 MB
Stage fit_on_coadds finished: 2020-08-07T09:19:39.056552
Resources for stage fit_on_coadds : 0.141 s worker CPU, 0.141 s worker Wall, Wall: 5.391 s, Cores in use: 0.05, Total efficiency (on 16 cores): 0.3 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-07T09:19:39.653721
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 37 potential peaks
Of 37 potential peaks: 15 in veto map, 4 cut by saddle test, 0 cut by aper test, 18 kept
SED z : 18 new peaks
Found 40 potential peaks
Of 40 potential peaks: 35 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 33 potential peaks
Of 33 potential peaks: 29 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 51 potential peaks
Of 51 potential peaks: 44 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 54 potential peaks
Of 54 potential peaks: 45 in veto map, 8 cut by saddle test, 0 cut by aper test, 1 kept
SED Red : 1 new peaks
Adding 21 new sources
Median coadd sky for g : 0.0003996479
Median coadd sky for r : 0.0009953516
Median coadd sky for z : -0.00043167357
Stage srcs : Wall: 0.55 s, CPU: 0.51 s, VmPeak: 1059 MB, VmSize: 1059 MB, VmRSS: 288 MB, VmData: 315 MB, maxrss: 0.294996 MB
Stage srcs finished: 2020-08-07T09:19:40.203448
Resources for stage srcs : 0.077 s worker CPU, 0.077 s worker Wall, Wall: 0.551 s, Cores in use: 1.07, Total efficiency (on 16 cores): 6.7 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-07T09:19:40.263006
Blob 1 of 11, id: 0, sources: 20, size: 207x181, npix 19206, brick X: 0,207, Y: 0,181, one pixel: 0 0
Blob 2 of 11, id: 6, sources: 5, size: 88x89, npix 4583, brick X: 150,238, Y: 135,224, one pixel: 190 135
Blob 3 of 11, id: 8, sources: 3, size: 102x52, npix 3398, brick X: 63,165, Y: 187,239, one pixel: 118 187
Blob 4 of 11, id: 2, sources: 3, size: 35x63, npix 1345, brick X: 202,237, Y: 50,113, one pixel: 218 50
Blob 5 of 11, id: 4, sources: 1, size: 32x35, npix 807, brick X: 0,32, Y: 111,146, one pixel: 6 111
Blob 6 of 11, id: 9, sources: 1, size: 34x30, npix 723, brick X: 26,60, Y: 209,239, one pixel: 37 209
Blob 7 of 11, id: 10, sources: 3, size: 33x26, npix 671, brick X: 206,239, Y: 213,239, one pixel: 227 213
Blob 8 of 11, id: 3, sources: 1, size: 31x31, npix 610, brick X: 0,31, Y: 76,107, one pixel: 17 76
Blob 9 of 11, id: 1, sources: 1, size: 29x17, npix 324, brick X: 189,218, Y: 0,17, one pixel: 193 0
Blob 10 of 11, id: 5, sources: 1, size: 23x22, npix 323, brick X: 38,61, Y: 113,135, one pixel: 47 113
Blob 11 of 11, id: 7, sources: 1, size: 19x19, npix 190, brick X: 25,44, Y: 139,158, one pixel: 34 139
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.21633, 34.31537), pm (21.8, -13.6), parallax 1.170 with NanoMaggies: g=12.3, r=11.5, z=11.1
Reset brightness to NanoMaggies: g=12.3, r=11.5, z=11.1
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.22931, 34.29391), pm (-6.1, -37.2), parallax 1.150 with NanoMaggies: g=14.1, r=13.7, z=13.5
Reset brightness to NanoMaggies: g=14.1, r=13.7, z=13.5
Blob 7 finished, total: Wall: 0.11 s, CPU: 0.11 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.14586 MB
Measuring 1 iterative sources
Blob 11 finished, total: Wall: 0.21 s, CPU: 0.21 s, VmPeak: 814 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145748 MB
Blob 10 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 816 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.14602 MB
Blob 5 finished, total: Wall: 0.58 s, CPU: 0.58 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 92 MB, VmData: 138 MB, maxrss: 0.09376 MB
Blob 9 finished, total: Wall: 0.63 s, CPU: 0.63 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146024 MB
Blob 8 finished, total: Wall: 0.72 s, CPU: 0.72 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094464 MB
Blob 6 finished, total: Wall: 0.74 s, CPU: 0.74 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145756 MB
Blob 4 finished, total: Wall: 1.57 s, CPU: 1.56 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093484 MB
Measuring 2 iterative sources
Blob 3 finished, total: Wall: 2.61 s, CPU: 2.61 s, VmPeak: 671 MB, VmSize: 670 MB, VmRSS: 96 MB, VmData: 142 MB, maxrss: 0.098376 MB
Measuring 2 iterative sources
Blob 2 finished, total: Wall: 3.56 s, CPU: 3.56 s, VmPeak: 807 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147224 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.34956, 34.23261), pm (12.3, -51.9), parallax 9.097 with NanoMaggies: g=9.14, r=8.83, z=8.77
Reset brightness to NanoMaggies: g=9.14, r=8.83, z=8.77
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.27722, 34.27539), pm (-0.9, 3.0), parallax 0.958 with NanoMaggies: g=16, r=15.4, z=15.2
Reset brightness to NanoMaggies: g=16, r=15.4, z=15.2
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=0.876948, ee1=0.0712075, ee2=0.287708
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (276.27616, 34.26954), pm (6.4, 6.4), parallax 0.524 with NanoMaggies: g=18.9, r=17.7, z=16.9
Reset brightness to NanoMaggies: g=18.9, r=17.7, z=16.9
Blob 1 finished, total: Wall: 10.42 s, CPU: 10.41 s, VmPeak: 813 MB, VmSize: 729 MB, VmRSS: 152 MB, VmData: 184 MB, maxrss: 0.155928 MB
Old catalog: 50
New catalog: 44
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/blobs-custom-276261p34282.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/metrics/cus/all-models-custom-276261p34282.fits
Stage fitblobs : Wall: 10.83 s, CPU: 0.38 s, VmPeak: 1059 MB, VmSize: 1059 MB, VmRSS: 289 MB, VmData: 316 MB, maxrss: 0.296348 MB
Stage fitblobs finished: 2020-08-07T09:19:51.092180
Resources for stage fitblobs : 21.531 s worker CPU, 21.584 s worker Wall, Wall: 10.831 s, Cores in use: 2.02, Total efficiency (on 16 cores): 12.6 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-07T09:19:51.105656
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-resid.jpg
Stage coadds : Wall: 1.82 s, CPU: 0.63 s, VmPeak: 1059 MB, VmSize: 1059 MB, VmRSS: 290 MB, VmData: 316 MB, maxrss: 0.296672 MB
Stage coadds finished: 2020-08-07T09:19:52.929708
Resources for stage coadds : 9.439 s worker CPU, 10.293 s worker Wall, Wall: 1.825 s, Cores in use: 5.51, Total efficiency (on 16 cores): 34.5 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-07T09:19:52.943544
Cut to 1 unWISE tiles
Reading WISE tile 2759p348 band 1
Reading WISE tile 2759p348 band 4
Reading WISE tile 2759p348 band 3
Reading WISE tile 2759p348 band 2
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 813 MB, VmSize: 802 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.190168 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 808 MB, VmSize: 796 MB, VmRSS: 141 MB, VmData: 174 MB, maxrss: 0.204088 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 863 MB, VmSize: 863 MB, VmRSS: 206 MB, VmData: 237 MB, maxrss: 0.211452 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/275/2759p348/unwise-2759p348-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 816 MB, VmSize: 797 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.20366 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/tmp-legacysurvey-custom-276261p34282-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 22.92 s, CPU: 0.62 s, VmPeak: 1059 MB, VmSize: 1059 MB, VmRSS: 290 MB, VmData: 316 MB, maxrss: 0.297184 MB
Stage wise_forced finished: 2020-08-07T09:20:15.864846
Resources for stage wise_forced : 2.906 s worker CPU, 24.688 s worker Wall, Wall: 22.923 s, Cores in use: 0.15, Total efficiency (on 16 cores): 1.0 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-07T09:20:15.897558
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/coadd/cus/custom-276261p34282/legacysurvey-custom-276261p34282-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/tractor-i/cus/tractor-custom-276261p34282.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/tractor/cus/tractor-custom-276261p34282.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/tractor/cus/brick-custom-276261p34282.sha256sum
Stage writecat : Wall: 0.95 s, CPU: 0.62 s, VmPeak: 1191 MB, VmSize: 1063 MB, VmRSS: 292 MB, VmData: 316 MB, maxrss: 0.430108 MB
Stage writecat finished: 2020-08-07T09:20:16.851852
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.956 s, Cores in use: 0.65, Total efficiency (on 16 cores): 4.0 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/276/PGC2046768/PGC2046768-largegalaxy-writecat.p
All done: 39.073 s worker CPU, 468.547 s worker Wall, Wall: 163.806 s, Cores in use: 0.35, Total efficiency (on 16 cores): 2.2 %
Finished galaxy PGC2046768 in 2.968 minutes.