Started working on galaxy PGC2330639 at Wed Aug  5 21:29:51 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.454682 48.991344500000004 --width 323 --height 323 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T21:29:53.236628
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.454682', '48.991344500000004', '--width', '323', '--height', '323', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.454682 48.991344500000004 --width 323 --height 323 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 277.454682 48.991344500000004
Starting process 58386 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 58387 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 58388 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 58389 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 58390 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059536 MB
Starting process 58391 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB
Starting process 58392 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB
Starting process 58393 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 58394 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 58395 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 58396 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 58397 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 58398 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 58400 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 58401 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 58399 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T21:29:55.219386
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 13 CCDs:
  k4m_170809_091309_ooi_zd_ls9-CCD4 z expnum 136595 exptime 144.105 propid 2016A-0453 seeing 0.90 MJD 57974.384 object MzLS_644610_z 
    mosaic/CP/V4.3/CP20170808/k4m_170809_091309_ooi_zd_ls9.fits.fz
  k4m_160523_095059_ooi_zd_ls9-CCD3 z expnum 90940 exptime 85.104 propid 2016A-0453 seeing 0.91 MJD 57531.410 object MzLS_646260_z 
    mosaic/CP/V4.3/CP20160522/k4m_160523_095059_ooi_zd_ls9.fits.fz
  k4m_160516_103009_ooi_zd_ls9-CCD2 z expnum 89308 exptime 80.105 propid 2016A-0453 seeing 1.07 MJD 57524.438 object MzLS_645437_z 
    mosaic/CP/V4.3/CP20160515/k4m_160516_103009_ooi_zd_ls9.fits.fz
  ksb_160611_105048_ooi_r_ls9-CCD1 r expnum 75500183 exptime 65.6 propid BASS seeing 1.16 MJD 57550.452 object 26051 
    90prime/CP/V2.3/CP20160610/ksb_160611_105048_ooi_r_ls9.fits.fz
  ksb_170604_105818_ooi_g_ls9-CCD1 g expnum 79080200 exptime 101.086 propid BASS seeing 1.28 MJD 57908.457 object 26051 
    90prime/CP/V2.3/CP20170603/ksb_170604_105818_ooi_g_ls9.fits.fz
  ksb_180515_104926_ooi_g_ls9-CCD4 g expnum 82530170 exptime 110.218 propid NOTBASS seeing 1.26 MJD 58253.451 object 26061 
    90prime/CP/V2.3/CP20180514/ksb_180515_104926_ooi_g_ls9.fits.fz
  ksb_160611_105655_ooi_r_ls9-CCD4 r expnum 75500186 exptime 69.7 propid BASS seeing 1.21 MJD 57550.456 object 26061 
    90prime/CP/V2.3/CP20160610/ksb_160611_105655_ooi_r_ls9.fits.fz
  ksb_160611_105048_ooi_r_ls9-CCD2 r expnum 75500183 exptime 65.6 propid BASS seeing 1.09 MJD 57550.452 object 26051 
    90prime/CP/V2.3/CP20160610/ksb_160611_105048_ooi_r_ls9.fits.fz
  ksb_170604_105818_ooi_g_ls9-CCD2 g expnum 79080200 exptime 101.086 propid BASS seeing 1.22 MJD 57908.457 object 26051 
    90prime/CP/V2.3/CP20170603/ksb_170604_105818_ooi_g_ls9.fits.fz
  ksb_160709_095905_ooi_g_ls9-CCD3 g expnum 75780118 exptime 181.7 propid BASS seeing 1.93 MJD 57578.416 object 26063 
    90prime/CP/V2.3/CP20160708/ksb_160709_095905_ooi_g_ls9.fits.fz
  ksb_180518_105428_ooi_r_ls9-CCD3 r expnum 82560212 exptime 76.885 propid NOTBASS seeing 1.28 MJD 58256.454 object 26063 
    90prime/CP/V2.3/CP20180517/ksb_180518_105428_ooi_r_ls9.fits.fz
  ksb_160612_105317_ooi_r_ls9-CCD2 r expnum 75510134 exptime 86.0 propid BASS seeing 1.26 MJD 57551.454 object 26052 
    90prime/CP/V2.3/CP20160611/ksb_160612_105317_ooi_r_ls9.fits.fz
  ksb_160706_094923_ooi_g_ls9-CCD2 g expnum 75750116 exptime 182.7 propid BASS seeing 1.77 MJD 57575.409 object 26052 
    90prime/CP/V2.3/CP20160705/ksb_160706_094923_ooi_g_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 4.13
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 7 110 and amp range 0 2047 and image y 1804 2130 and amp range 0 2039
Range in image: x 0 104 , y 0 236 (with image size 104 x 327)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 7 110 and amp range 0 2047 and image y 1804 2130 and amp range 2040 4078
Range in image: x 0 104 , y 236 327 (with image size 104 x 327)
dzp 0.008550294 -> scaling image by 1.0079062018164673
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82560212
Faked up EXPNUM 82560212
Faked up EXPNUM 82560212
90prime get_fwhm: self.fwhm = 2.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2136 2324 and amp range 2016 4031 and image y 1574 1763 and amp range 0 2047
Range in image: x 0 189 , y 0 190 (with image size 189 x 190)
dzp 0.0057031885 -> scaling image by 1.00526665101287
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75780118
Faked up EXPNUM 75780118
Faked up EXPNUM 75780118
90prime get_fwhm: self.fwhm = 4.25
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2119 2306 and amp range 2016 4031 and image y 1564 1752 and amp range 0 2047
Range in image: x 0 188 , y 0 189 (with image size 188 x 189)
dzp -0.0022715968 -> scaling image by 0.9979099691834846
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1367 1695 and amp range 0 2005 and image y 262 588 and amp range 0 2039
Range in image: x 0 329 , y 0 327 (with image size 329 x 327)
dzp -0.005856539 -> scaling image by 0.9946204500110131
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
90prime get_fwhm: self.fwhm = 2.41
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 34 and amp range 0 2015 and image y 15 183 and amp range 0 2047
Range in image: x 0 20 , y 0 169 (with image size 20 x 169)
dzp -0.006045583 -> scaling image by 0.9944472857505693
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75500186
Faked up EXPNUM 75500186
Faked up EXPNUM 75500186
90prime get_fwhm: self.fwhm = 2.67
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 114 and amp range 0 2015 and image y 3550 3738 and amp range 2048 4095
Range in image: x 0 100 , y 0 189 (with image size 100 x 189)
dzp 0.004205519 -> scaling image by 1.0038809374310762
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
90prime get_fwhm: self.fwhm = 2.69
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 43 and amp range 0 2015 and image y 15 195 and amp range 0 2047
Range in image: x 0 29 , y 0 181 (with image size 29 x 181)
dzp 0.0003063402 -> scaling image by 1.0002821895558327
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82530170
Faked up EXPNUM 82530170
Faked up EXPNUM 82530170
90prime get_fwhm: self.fwhm = 2.77
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 136 and amp range 0 2015 and image y 3538 3727 and amp range 2048 4095
Range in image: x 0 122 , y 0 190 (with image size 122 x 190)
dzp 0.027550658 -> scaling image by 1.0256997821704772
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75510134
Faked up EXPNUM 75510134
Faked up EXPNUM 75510134
90prime get_fwhm: self.fwhm = 2.77
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1861 2051 and amp range 0 2015 and image y 1993 2183 and amp range 0 2047
Range in image: x 0 155 , y 0 55 (with image size 191 x 191)
dzp -0.006045583 -> scaling image by 0.9944472857505693
Found overlap: image x 1861 2051 and amp range 2016 4031 and image y 1993 2183 and amp range 0 2047
Range in image: x 155 191 , y 0 55 (with image size 191 x 191)
dzp -0.012503507 -> scaling image by 0.9885499019198392
Found overlap: image x 1861 2051 and amp range 0 2015 and image y 1993 2183 and amp range 2048 4095
Range in image: x 0 155 , y 55 191 (with image size 191 x 191)
dzp 0.016961161 -> scaling image by 1.0157444649385616
Found overlap: image x 1861 2051 and amp range 2016 4031 and image y 1993 2183 and amp range 2048 4095
Range in image: x 155 191 , y 55 191 (with image size 191 x 191)
dzp 0.0002007609 -> scaling image by 1.0001849247187462
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 75750116
Faked up EXPNUM 75750116
Faked up EXPNUM 75750116
90prime get_fwhm: self.fwhm = 3.91
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1868 2057 and amp range 0 2015 and image y 2005 2195 and amp range 0 2047
Range in image: x 0 148 , y 0 43 (with image size 190 x 191)
dzp -0.008523628 -> scaling image by 0.992180183483648
Found overlap: image x 1868 2057 and amp range 2016 4031 and image y 2005 2195 and amp range 0 2047
Range in image: x 148 190 , y 0 43 (with image size 190 x 191)
dzp -0.008960029 -> scaling image by 0.9917814665476246
Found overlap: image x 1868 2057 and amp range 0 2015 and image y 2005 2195 and amp range 2048 4095
Range in image: x 0 148 , y 43 191 (with image size 190 x 191)
dzp 0.01693853 -> scaling image by 1.015723292931903
Found overlap: image x 1868 2057 and amp range 2016 4031 and image y 2005 2195 and amp range 2048 4095
Range in image: x 148 190 , y 43 191 (with image size 190 x 191)
dzp 0.0026774607 -> scaling image by 1.002469075566243
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.45
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2462 2793 and amp range 2048 4095 and image y 645 971 and amp range 0 2021
Range in image: x 0 332 , y 0 327 (with image size 332 x 327)
dzp -0.029506588 -> scaling image by 0.9731893881648719
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 7.92 s, CPU: 2.09 s, VmPeak: 958 MB, VmSize: 958 MB, VmRSS: 177 MB, VmData: 209 MB, maxrss: 0.18078 MB
Stage tims finished: 2020-08-05T21:30:03.134910
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
90prime get_fwhm: self.fwhm = 2.55
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 7.975 s worker CPU, 35.022 s worker Wall, Wall: 7.917 s, Cores in use: 1.27, Total efficiency (on 16 cores): 7.9 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-tims.p
Running stage refs
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T21:30:03.242466
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits
Matched 5 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/reference-custom-277454p48991.fits
Stage refs : Wall: 8.92 s, CPU: 5.50 s, VmPeak: 1020 MB, VmSize: 1015 MB, VmRSS: 237 MB, VmData: 266 MB, maxrss: 0.242404 MB
Stage refs finished: 2020-08-05T21:30:12.165392
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 8.925 s, Cores in use: 0.62, Total efficiency (on 16 cores): 3.9 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T21:30:12.188425
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/outlier-mask-custom-277454p48991.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/outliers-pre-custom-277454p48991.jpg
Image ksb_160706_094923_ooi_g_ls9-CCD2 g : masking 588 positive outlier pixels and 0 negative outlier pixels
Image ksb_170604_105818_ooi_g_ls9-CCD2 g : masking 267 positive outlier pixels and 0 negative outlier pixels
Image ksb_180518_105428_ooi_r_ls9-CCD3 r : masking 100 positive outlier pixels and 0 negative outlier pixels
Image k4m_160516_103009_ooi_zd_ls9-CCD2 z : masking 128 positive outlier pixels and 0 negative outlier pixels
Image k4m_170809_091309_ooi_zd_ls9-CCD4 z : masking 673 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/outlier-mask-custom-277454p48991.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/outliers-post-custom-277454p48991.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/outliers-masked-pos-custom-277454p48991.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/outliers-masked-neg-custom-277454p48991.jpg
Stage outliers : Wall: 1.77 s, CPU: 1.17 s, VmPeak: 1025 MB, VmSize: 1025 MB, VmRSS: 254 MB, VmData: 281 MB, maxrss: 0.260376 MB
Stage outliers finished: 2020-08-05T21:30:13.956564
Resources for stage outliers : 0.987 s worker CPU, 1.139 s worker Wall, Wall: 1.770 s, Cores in use: 1.22, Total efficiency (on 16 cores): 7.6 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T21:30:14.049375
Stage halos : Wall: 1.47 s, CPU: 1.47 s, VmPeak: 1025 MB, VmSize: 1025 MB, VmRSS: 255 MB, VmData: 281 MB, maxrss: 0.260664 MB
Stage halos finished: 2020-08-05T21:30:15.523564
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.475 s, Cores in use: 1.00, Total efficiency (on 16 cores): 6.2 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T21:30:15.592782
PSF sigmas in coadd pixels: ['2.04', '1.98', '3.13', '2.88']
Using average PSF sigma 2.506041701579642
Gaussian PSF norm 0.11256588092531117 vs pixelized 0.09030683
average tim pixel scale / coadd scale: 1.7296869991141022
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.97', '1.77', '2.08', '2.04']
Using average PSF sigma 1.9653485860538331
Gaussian PSF norm 0.14353422785943953 vs pixelized 0.12039162
average tim pixel scale / coadd scale: 1.729788973604023
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.45', '1.47', '1.74']
Using average PSF sigma 1.556005553909068
Gaussian PSF norm 0.181294206222585 vs pixelized 0.14513844
average tim pixel scale / coadd scale: 0.9909504691302378
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.20 s, CPU: 0.19 s, VmPeak: 1025 MB, VmSize: 1025 MB, VmRSS: 257 MB, VmData: 283 MB, maxrss: 0.263472 MB
Stage fit_on_coadds finished: 2020-08-05T21:30:15.790155
Resources for stage fit_on_coadds : 0.221 s worker CPU, 0.222 s worker Wall, Wall: 0.199 s, Cores in use: 2.06, Total efficiency (on 16 cores): 12.8 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T21:30:15.802637
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 39 potential peaks
Of 39 potential peaks: 7 in veto map, 3 cut by saddle test, 0 cut by aper test, 29 kept
SED z : 29 new peaks
Found 31 potential peaks
Of 31 potential peaks: 26 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 28 potential peaks
Of 28 potential peaks: 24 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept
SED g : 2 new peaks
Found 36 potential peaks
Of 36 potential peaks: 31 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept
SED Flat : 3 new peaks
Found 49 potential peaks
Of 49 potential peaks: 42 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept
SED Red : 1 new peaks
Adding 36 new sources
Median coadd sky for g : -1.6855967e-05
Median coadd sky for r : -0.00012658232
Median coadd sky for z : 4.6927365e-05
Stage srcs : Wall: 1.36 s, CPU: 1.29 s, VmPeak: 1025 MB, VmSize: 1025 MB, VmRSS: 258 MB, VmData: 283 MB, maxrss: 0.264076 MB
Stage srcs finished: 2020-08-05T21:30:17.166326
Resources for stage srcs : 0.110 s worker CPU, 0.110 s worker Wall, Wall: 1.365 s, Cores in use: 1.03, Total efficiency (on 16 cores): 6.4 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T21:30:17.181657
Blob 1 of 25, id: 10, sources: 6, size: 183x109, npix 11716, brick X: 76,259, Y: 102,211, one pixel: 118 102
Blob 2 of 25, id: 2, sources: 8, size: 103x63, npix 3221, brick X: 63,166, Y: 36,99, one pixel: 124 36
Blob 3 of 25, id: 21, sources: 4, size: 72x48, npix 2114, brick X: 98,170, Y: 257,305, one pixel: 122 257
Blob 4 of 25, id: 0, sources: 1, size: 55x37, npix 1383, brick X: 238,293, Y: 0,37, one pixel: 243 0
Blob 5 of 25, id: 23, sources: 2, size: 60x36, npix 1276, brick X: 203,263, Y: 265,301, one pixel: 245 265
Blob 6 of 25, id: 8, sources: 1, size: 37x34, npix 835, brick X: 131,168, Y: 77,111, one pixel: 145 77
Blob 7 of 25, id: 18, sources: 1, size: 35x34, npix 817, brick X: 168,203, Y: 217,251, one pixel: 181 217
Blob 8 of 25, id: 15, sources: 1, size: 33x33, npix 759, brick X: 40,73, Y: 180,213, one pixel: 54 180
Blob 9 of 25, id: 17, sources: 2, size: 40x34, npix 714, brick X: 57,97, Y: 203,237, one pixel: 85 203
Blob 10 of 25, id: 19, sources: 1, size: 28x27, npix 494, brick X: 4,32, Y: 236,263, one pixel: 16 236
Blob 11 of 25, id: 7, sources: 1, size: 27x27, npix 481, brick X: 175,202, Y: 71,98, one pixel: 186 71
Blob 12 of 25, id: 12, sources: 1, size: 27x27, npix 457, brick X: 19,46, Y: 113,140, one pixel: 31 113
Blob 13 of 25, id: 1, sources: 1, size: 26x26, npix 426, brick X: 12,38, Y: 33,59, one pixel: 22 33
Blob 14 of 25, id: 5, sources: 1, size: 26x25, npix 424, brick X: 271,297, Y: 66,91, one pixel: 282 66
Blob 15 of 25, id: 3, sources: 1, size: 28x23, npix 419, brick X: 293,321, Y: 42,65, one pixel: 305 42
Blob 16 of 25, id: 13, sources: 1, size: 26x25, npix 417, brick X: 47,73, Y: 134,159, one pixel: 59 134
Blob 17 of 25, id: 9, sources: 1, size: 25x25, npix 378, brick X: 237,262, Y: 93,118, one pixel: 249 93
Measuring 1 iterative sources
Blob 14 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096296 MB
Blob 18 of 25, id: 20, sources: 1, size: 23x23, npix 338, brick X: 221,244, Y: 236,259, one pixel: 230 236
Blob 19 of 25, id: 4, sources: 1, size: 23x23, npix 319, brick X: 201,224, Y: 46,69, one pixel: 211 46
Blob 12 finished, total: Wall: 0.58 s, CPU: 0.57 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.15032 MB
Blob 20 of 25, id: 14, sources: 1, size: 22x22, npix 284, brick X: 286,308, Y: 141,163, one pixel: 296 141
Blob 11 finished, total: Wall: 0.71 s, CPU: 0.71 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 118 MB, VmData: 161 MB, maxrss: 0.121272 MB
Blob 21 of 25, id: 6, sources: 1, size: 21x21, npix 250, brick X: 248,269, Y: 70,91, one pixel: 257 70
Blob 13 finished, total: Wall: 0.68 s, CPU: 0.68 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 93 MB, VmData: 138 MB, maxrss: 0.095004 MB
Blob 22 of 25, id: 16, sources: 1, size: 20x21, npix 239, brick X: 17,37, Y: 190,211, one pixel: 26 190
Blob 17 finished, total: Wall: 0.53 s, CPU: 0.53 s, VmPeak: 669 MB, VmSize: 668 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096428 MB
Blob 23 of 25, id: 24, sources: 1, size: 21x15, npix 214, brick X: 260,281, Y: 308,323, one pixel: 268 308
Blob 6 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 184 MB, maxrss: 0.150084 MB
Blob 16 finished, total: Wall: 0.84 s, CPU: 0.83 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.09644 MB
Blob 24 of 25, id: 11, sources: 1, size: 18x18, npix 171, brick X: 64,82, Y: 104,122, one pixel: 72 104
Blob 25 of 25, id: 22, sources: 1, size: 18x17, npix 162, brick X: 267,285, Y: 259,276, one pixel: 275 259
Blob 18 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.15032 MB
Blob 7 finished, total: Wall: 1.02 s, CPU: 1.01 s, VmPeak: 805 MB, VmSize: 731 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151956 MB
Blob 20 finished, total: Wall: 0.21 s, CPU: 0.21 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 93 MB, VmData: 138 MB, maxrss: 0.095216 MB
Blob 8 finished, total: Wall: 0.93 s, CPU: 0.91 s, VmPeak: 814 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 184 MB, maxrss: 0.151392 MB
Blob 24 finished, total: Wall: 0.07 s, CPU: 0.07 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.15032 MB
Blob 19 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 118 MB, VmData: 161 MB, maxrss: 0.121272 MB
Blob 25 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 805 MB, VmSize: 731 MB, VmRSS: 149 MB, VmData: 183 MB, maxrss: 0.152556 MB
Blob 15 finished, total: Wall: 1.03 s, CPU: 1.02 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 141 MB, VmData: 182 MB, maxrss: 0.14486 MB
Blob 21 finished, total: Wall: 0.24 s, CPU: 0.24 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096428 MB
Blob 22 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 184 MB, maxrss: 0.150084 MB
Blob 10 finished, total: Wall: 1.15 s, CPU: 1.14 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.151032 MB
Blob 5 finished, total: Wall: 1.27 s, CPU: 1.27 s, VmPeak: 790 MB, VmSize: 719 MB, VmRSS: 136 MB, VmData: 171 MB, maxrss: 0.139164 MB
Blob 9 finished, total: Wall: 1.37 s, CPU: 1.35 s, VmPeak: 806 MB, VmSize: 731 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151544 MB
Blob 23 finished, total: Wall: 0.56 s, CPU: 0.56 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096624 MB
Blob 4 finished, total: Wall: 2.23 s, CPU: 2.22 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 149 MB, VmData: 184 MB, maxrss: 0.152764 MB
Measuring 1 iterative sources
Blob 3 finished, total: Wall: 2.84 s, CPU: 2.83 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 150 MB, VmData: 184 MB, maxrss: 0.153124 MB
Measuring 2 iterative sources
Blob 2 finished, total: Wall: 3.97 s, CPU: 3.96 s, VmPeak: 807 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 184 MB, maxrss: 0.1517 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.87086, ee1=-0.680881, ee2=-0.0473334
Measuring 1 iterative sources
Blob 1 finished, total: Wall: 9.51 s, CPU: 9.48 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 151 MB, VmData: 184 MB, maxrss: 0.154864 MB
Old catalog: 51
New catalog: 43
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/blobs-custom-277454p48991.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/metrics/cus/all-models-custom-277454p48991.fits
Stage fitblobs : Wall: 12.97 s, CPU: 1.79 s, VmPeak: 1025 MB, VmSize: 1025 MB, VmRSS: 258 MB, VmData: 283 MB, maxrss: 0.264432 MB
Stage fitblobs finished: 2020-08-05T21:30:30.155669
Resources for stage fitblobs : 31.865 s worker CPU, 40.206 s worker Wall, Wall: 12.976 s, Cores in use: 2.59, Total efficiency (on 16 cores): 16.2 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T21:30:30.174889
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-resid.jpg
Stage coadds : Wall: 4.62 s, CPU: 2.04 s, VmPeak: 1025 MB, VmSize: 1025 MB, VmRSS: 259 MB, VmData: 283 MB, maxrss: 0.26476 MB
Stage coadds finished: 2020-08-05T21:30:34.794205
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 7.671 s worker CPU, 13.827 s worker Wall, Wall: 4.621 s, Cores in use: 2.10, Total efficiency (on 16 cores): 13.1 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-05T21:30:34.853945
Cut to 1 unWISE tiles
Reading WISE tile 2767p484 band 1
Reading WISE tile 2767p484 band 4
Reading WISE tile 2767p484 band 3
Reading WISE tile 2767p484 band 2
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 812 MB, VmSize: 809 MB, VmRSS: 151 MB, VmData: 177 MB, maxrss: 0.193624 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 806 MB, VmSize: 804 MB, VmRSS: 147 MB, VmData: 171 MB, maxrss: 0.208756 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 811 MB, VmSize: 804 MB, VmRSS: 144 MB, VmData: 171 MB, maxrss: 0.2074 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 814 MB, VmSize: 812 MB, VmRSS: 151 MB, VmData: 180 MB, maxrss: 0.21484 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 4.18 s, CPU: 1.99 s, VmPeak: 1025 MB, VmSize: 1025 MB, VmRSS: 259 MB, VmData: 283 MB, maxrss: 0.265288 MB
Stage wise_forced finished: 2020-08-05T21:30:39.036313
Resources for stage wise_forced : 3.563 s worker CPU, 6.125 s worker Wall, Wall: 4.184 s, Cores in use: 1.33, Total efficiency (on 16 cores): 8.3 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T21:30:39.056034
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/tractor-i/cus/tractor-custom-277454p48991.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/tractor/cus/tractor-custom-277454p48991.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/tractor/cus/brick-custom-277454p48991.sha256sum
Stage writecat : Wall: 2.12 s, CPU: 1.95 s, VmPeak: 1157 MB, VmSize: 1029 MB, VmRSS: 261 MB, VmData: 283 MB, maxrss: 0.398168 MB
Stage writecat finished: 2020-08-05T21:30:41.173770
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 2.119 s, Cores in use: 0.92, Total efficiency (on 16 cores): 5.7 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/277/PGC2330639/PGC2330639-largegalaxy-writecat.p
All done: 52.397 s worker CPU, 96.656 s worker Wall, Wall: 46.788 s, Cores in use: 1.54, Total efficiency (on 16 cores): 9.6 %
Finished galaxy PGC2330639 in 0.838 minutes.
Started working on galaxy PGC2330639 at Fri Aug  7 09:46:02 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.45452941736556 48.991527629614986 --width 323 --height 323 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-07T09:46:03.774065
legacypipe git version: DR9.5.11-watershed-3-gd509eaaf
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '277.45452941736556', '48.991527629614986', '--width', '323', '--height', '323', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 277.45452941736556 48.991527629614986 --width 323 --height 323 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 277.45452941736556 48.991527629614986
Starting process 33195 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 33196 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 33197 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 33198 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 33199 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 33200 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 33201 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 33202 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 33203 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 33204 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 33205 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 33206 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 33207 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 33208 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 33209 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 33210 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-07T09:46:04.273948
Keeping 13 CCDs:
  k4m_170809_091309_ooi_zd_ls9-CCD4 z expnum 136595 exptime 144.105 propid 2016A-0453 seeing 0.90 MJD 57974.384 object MzLS_644610_z 
    mosaic/CP/V4.3/CP20170808/k4m_170809_091309_ooi_zd_ls9.fits.fz
  k4m_160523_095059_ooi_zd_ls9-CCD3 z expnum 90940 exptime 85.104 propid 2016A-0453 seeing 0.91 MJD 57531.410 object MzLS_646260_z 
    mosaic/CP/V4.3/CP20160522/k4m_160523_095059_ooi_zd_ls9.fits.fz
  k4m_160516_103009_ooi_zd_ls9-CCD2 z expnum 89308 exptime 80.105 propid 2016A-0453 seeing 1.07 MJD 57524.438 object MzLS_645437_z 
    mosaic/CP/V4.3/CP20160515/k4m_160516_103009_ooi_zd_ls9.fits.fz
  ksb_160611_105048_ooi_r_ls9-CCD1 r expnum 75500183 exptime 65.6 propid BASS seeing 1.16 MJD 57550.452 object 26051 
    90prime/CP/V2.3/CP20160610/ksb_160611_105048_ooi_r_ls9.fits.fz
  ksb_170604_105818_ooi_g_ls9-CCD1 g expnum 79080200 exptime 101.086 propid BASS seeing 1.28 MJD 57908.457 object 26051 
    90prime/CP/V2.3/CP20170603/ksb_170604_105818_ooi_g_ls9.fits.fz
  ksb_180515_104926_ooi_g_ls9-CCD4 g expnum 82530170 exptime 110.218 propid NOTBASS seeing 1.26 MJD 58253.451 object 26061 
    90prime/CP/V2.3/CP20180514/ksb_180515_104926_ooi_g_ls9.fits.fz
  ksb_160611_105655_ooi_r_ls9-CCD4 r expnum 75500186 exptime 69.7 propid BASS seeing 1.21 MJD 57550.456 object 26061 
    90prime/CP/V2.3/CP20160610/ksb_160611_105655_ooi_r_ls9.fits.fz
  ksb_160611_105048_ooi_r_ls9-CCD2 r expnum 75500183 exptime 65.6 propid BASS seeing 1.09 MJD 57550.452 object 26051 
    90prime/CP/V2.3/CP20160610/ksb_160611_105048_ooi_r_ls9.fits.fz
  ksb_170604_105818_ooi_g_ls9-CCD2 g expnum 79080200 exptime 101.086 propid BASS seeing 1.22 MJD 57908.457 object 26051 
    90prime/CP/V2.3/CP20170603/ksb_170604_105818_ooi_g_ls9.fits.fz
  ksb_160709_095905_ooi_g_ls9-CCD3 g expnum 75780118 exptime 181.7 propid BASS seeing 1.93 MJD 57578.416 object 26063 
    90prime/CP/V2.3/CP20160708/ksb_160709_095905_ooi_g_ls9.fits.fz
  ksb_180518_105428_ooi_r_ls9-CCD3 r expnum 82560212 exptime 76.885 propid NOTBASS seeing 1.28 MJD 58256.454 object 26063 
    90prime/CP/V2.3/CP20180517/ksb_180518_105428_ooi_r_ls9.fits.fz
  ksb_160612_105317_ooi_r_ls9-CCD2 r expnum 75510134 exptime 86.0 propid BASS seeing 1.26 MJD 57551.454 object 26052 
    90prime/CP/V2.3/CP20160611/ksb_160612_105317_ooi_r_ls9.fits.fz
  ksb_160706_094923_ooi_g_ls9-CCD2 g expnum 75750116 exptime 182.7 propid BASS seeing 1.77 MJD 57575.409 object 26052 
    90prime/CP/V2.3/CP20160705/ksb_160706_094923_ooi_g_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 3.49
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1370 1697 and amp range 0 2005 and image y 260 586 and amp range 0 2039
Range in image: x 0 328 , y 0 327 (with image size 328 x 327)
dzp -0.005856539 -> scaling image by 0.9946204500110131
Faked up EXPNUM 82530170
Faked up EXPNUM 82530170
Faked up EXPNUM 82530170
90prime get_fwhm: self.fwhm = 2.77
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 134 and amp range 0 2015 and image y 3539 3727 and amp range 2048 4095
Range in image: x 0 120 , y 0 189 (with image size 120 x 189)
dzp 0.027550658 -> scaling image by 1.0256997821704772
Faked up EXPNUM 82560212
Faked up EXPNUM 82560212
Faked up EXPNUM 82560212
90prime get_fwhm: self.fwhm = 2.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2135 2323 and amp range 2016 4031 and image y 1575 1763 and amp range 0 2047
Range in image: x 0 189 , y 0 189 (with image size 189 x 189)
dzp 0.0057031885 -> scaling image by 1.00526665101287
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
90prime get_fwhm: self.fwhm = 2.69
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 41 and amp range 0 2015 and image y 15 195 and amp range 0 2047
Range in image: x 0 27 , y 0 181 (with image size 27 x 181)
dzp 0.0003063402 -> scaling image by 1.0002821895558327
Faked up EXPNUM 75750116
Faked up EXPNUM 75750116
Faked up EXPNUM 75750116
90prime get_fwhm: self.fwhm = 3.91
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1867 2056 and amp range 0 2015 and image y 2005 2195 and amp range 0 2047
Range in image: x 0 149 , y 0 43 (with image size 190 x 191)
dzp -0.008523628 -> scaling image by 0.992180183483648
Found overlap: image x 1867 2056 and amp range 2016 4031 and image y 2005 2195 and amp range 0 2047
Range in image: x 149 190 , y 0 43 (with image size 190 x 191)
dzp -0.008960029 -> scaling image by 0.9917814665476246
Found overlap: image x 1867 2056 and amp range 0 2015 and image y 2005 2195 and amp range 2048 4095
Range in image: x 0 149 , y 43 191 (with image size 190 x 191)
dzp 0.01693853 -> scaling image by 1.015723292931903
Found overlap: image x 1867 2056 and amp range 2016 4031 and image y 2005 2195 and amp range 2048 4095
Range in image: x 149 190 , y 43 191 (with image size 190 x 191)
dzp 0.0026774607 -> scaling image by 1.002469075566243
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
90prime get_fwhm: self.fwhm = 2.41
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 33 and amp range 0 2015 and image y 15 184 and amp range 0 2047
Range in image: x 0 19 , y 0 170 (with image size 19 x 170)
dzp -0.006045583 -> scaling image by 0.9944472857505693
Faked up EXPNUM 75780118
Faked up EXPNUM 75780118
Faked up EXPNUM 75780118
90prime get_fwhm: self.fwhm = 4.25
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2117 2304 and amp range 2016 4031 and image y 1565 1752 and amp range 0 2047
Range in image: x 0 188 , y 0 188 (with image size 188 x 188)
dzp -0.0022715968 -> scaling image by 0.9979099691834846
Faked up EXPNUM 75500186
Faked up EXPNUM 75500186
Faked up EXPNUM 75500186
90prime get_fwhm: self.fwhm = 2.67
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 15 112 and amp range 0 2015 and image y 3550 3739 and amp range 2048 4095
Range in image: x 0 98 , y 0 190 (with image size 98 x 190)
dzp 0.004205519 -> scaling image by 1.0038809374310762
mosaic get_fwhm: self.fwhm = 4.13
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 7 113 and amp range 0 2047 and image y 1802 2128 and amp range 0 2039
Range in image: x 0 107 , y 0 238 (with image size 107 x 327)
dzp -0.006320339 -> scaling image by 0.9941956631368829
Found overlap: image x 7 113 and amp range 0 2047 and image y 1802 2128 and amp range 2040 4078
Range in image: x 0 107 , y 238 327 (with image size 107 x 327)
dzp 0.008550294 -> scaling image by 1.0079062018164673
Faked up EXPNUM 75510134
Faked up EXPNUM 75510134
Faked up EXPNUM 75510134
90prime get_fwhm: self.fwhm = 2.77
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1860 2049 and amp range 0 2015 and image y 1994 2184 and amp range 0 2047
Range in image: x 0 156 , y 0 54 (with image size 190 x 191)
dzp -0.006045583 -> scaling image by 0.9944472857505693
Found overlap: image x 1860 2049 and amp range 2016 4031 and image y 1994 2184 and amp range 0 2047
Range in image: x 156 190 , y 0 54 (with image size 190 x 191)
dzp -0.012503507 -> scaling image by 0.9885499019198392
Found overlap: image x 1860 2049 and amp range 0 2015 and image y 1994 2184 and amp range 2048 4095
Range in image: x 0 156 , y 54 191 (with image size 190 x 191)
dzp 0.016961161 -> scaling image by 1.0157444649385616
Found overlap: image x 1860 2049 and amp range 2016 4031 and image y 1994 2184 and amp range 2048 4095
Range in image: x 156 190 , y 54 191 (with image size 190 x 191)
dzp 0.0002007609 -> scaling image by 1.0001849247187462
mosaic get_fwhm: self.fwhm = 3.45
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2465 2795 and amp range 2048 4095 and image y 644 970 and amp range 0 2021
Range in image: x 0 331 , y 0 327 (with image size 331 x 327)
dzp -0.029506588 -> scaling image by 0.9731893881648719
Stage tims : Wall: 9.36 s, CPU: 0.83 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 175 MB, VmData: 209 MB, maxrss: 0.179172 MB
Stage tims finished: 2020-08-07T09:46:13.638046
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
Faked up EXPNUM 75500183
90prime get_fwhm: self.fwhm = 2.55
Resources for stage tims : 6.745 s worker CPU, 53.720 s worker Wall, Wall: 9.366 s, Cores in use: 0.81, Total efficiency (on 16 cores): 5.1 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-tims.p
Running stage refs
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
Faked up EXPNUM 79080200
Running stage refs at 2020-08-07T09:46:22.736260
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01598.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01487.fits
Matched 5 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/reference-custom-277454p48991.fits
Stage refs : Wall: 32.00 s, CPU: 4.91 s, VmPeak: 1014 MB, VmSize: 1009 MB, VmRSS: 235 MB, VmData: 266 MB, maxrss: 0.240336 MB
Stage refs finished: 2020-08-07T09:46:54.734184
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 32.000 s, Cores in use: 0.15, Total efficiency (on 16 cores): 1.0 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-07T09:46:54.926463
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/outlier-mask-custom-277454p48991.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/outliers-pre-custom-277454p48991.jpg
Image ksb_160706_094923_ooi_g_ls9-CCD2 g : masking 532 positive outlier pixels and 0 negative outlier pixels
Image ksb_170604_105818_ooi_g_ls9-CCD2 g : masking 268 positive outlier pixels and 0 negative outlier pixels
Image ksb_180518_105428_ooi_r_ls9-CCD3 r : masking 95 positive outlier pixels and 0 negative outlier pixels
Image k4m_160516_103009_ooi_zd_ls9-CCD2 z : masking 126 positive outlier pixels and 0 negative outlier pixels
Image k4m_170809_091309_ooi_zd_ls9-CCD4 z : masking 694 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/outlier-mask-custom-277454p48991.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/outliers-post-custom-277454p48991.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/outliers-masked-pos-custom-277454p48991.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/outliers-masked-neg-custom-277454p48991.jpg
Stage outliers : Wall: 1.30 s, CPU: 0.87 s, VmPeak: 1018 MB, VmSize: 1018 MB, VmRSS: 253 MB, VmData: 281 MB, maxrss: 0.259068 MB
Stage outliers finished: 2020-08-07T09:46:56.225454
Resources for stage outliers : 0.872 s worker CPU, 1.011 s worker Wall, Wall: 1.300 s, Cores in use: 1.34, Total efficiency (on 16 cores): 8.4 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-07T09:46:56.260606
Stage halos : Wall: 0.30 s, CPU: 0.29 s, VmPeak: 1019 MB, VmSize: 1019 MB, VmRSS: 253 MB, VmData: 282 MB, maxrss: 0.259416 MB
Stage halos finished: 2020-08-07T09:46:56.563999
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.305 s, Cores in use: 0.97, Total efficiency (on 16 cores): 6.1 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-07T09:46:56.663693
PSF sigmas in coadd pixels: ['2.04', '1.98', '3.13', '2.88']
Using average PSF sigma 2.506041701579642
Gaussian PSF norm 0.11256588092531117 vs pixelized 0.090307064
average tim pixel scale / coadd scale: 1.7296869991141022
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.97', '1.77', '2.08', '2.04']
Using average PSF sigma 1.9653485860538331
Gaussian PSF norm 0.14353422785943953 vs pixelized 0.12039237
average tim pixel scale / coadd scale: 1.729788973604023
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['1.45', '1.47', '1.74']
Using average PSF sigma 1.556005553909068
Gaussian PSF norm 0.181294206222585 vs pixelized 0.14513753
average tim pixel scale / coadd scale: 0.9909504691302378
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.27 s, CPU: 0.18 s, VmPeak: 1019 MB, VmSize: 1019 MB, VmRSS: 255 MB, VmData: 282 MB, maxrss: 0.260936 MB
Stage fit_on_coadds finished: 2020-08-07T09:46:56.933761
Resources for stage fit_on_coadds : 0.234 s worker CPU, 0.234 s worker Wall, Wall: 0.272 s, Cores in use: 1.54, Total efficiency (on 16 cores): 9.6 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-07T09:46:56.975305
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 39 potential peaks
Of 39 potential peaks: 8 in veto map, 3 cut by saddle test, 0 cut by aper test, 28 kept
SED z : 28 new peaks
Found 32 potential peaks
Of 32 potential peaks: 26 in veto map, 4 cut by saddle test, 0 cut by aper test, 2 kept
SED r : 2 new peaks
Found 31 potential peaks
Of 31 potential peaks: 27 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept
SED g : 2 new peaks
Found 37 potential peaks
Of 37 potential peaks: 32 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept
SED Flat : 3 new peaks
Found 49 potential peaks
Of 49 potential peaks: 43 in veto map, 5 cut by saddle test, 0 cut by aper test, 1 kept
SED Red : 1 new peaks
Adding 36 new sources
Median coadd sky for g : -1.4012098e-05
Median coadd sky for r : -0.00011672353
Median coadd sky for z : 6.7364954e-07
Stage srcs : Wall: 0.53 s, CPU: 0.49 s, VmPeak: 1019 MB, VmSize: 1019 MB, VmRSS: 255 MB, VmData: 282 MB, maxrss: 0.26142 MB
Stage srcs finished: 2020-08-07T09:46:57.506555
Resources for stage srcs : 0.108 s worker CPU, 0.108 s worker Wall, Wall: 0.533 s, Cores in use: 1.12, Total efficiency (on 16 cores): 7.0 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-07T09:46:57.614746
Blob 1 of 26, id: 10, sources: 5, size: 183x109, npix 11674, brick X: 75,258, Y: 99,208, one pixel: 118 99
Blob 2 of 26, id: 2, sources: 8, size: 103x62, npix 3233, brick X: 61,164, Y: 34,96, one pixel: 121 34
Blob 3 of 26, id: 21, sources: 4, size: 72x49, npix 2097, brick X: 97,169, Y: 254,303, one pixel: 122 254
Blob 4 of 26, id: 22, sources: 2, size: 61x35, npix 1258, brick X: 201,262, Y: 263,298, one pixel: 241 263
Blob 5 of 26, id: 0, sources: 1, size: 53x35, npix 1251, brick X: 238,291, Y: 0,35, one pixel: 241 0
Blob 6 of 26, id: 8, sources: 1, size: 36x34, npix 825, brick X: 130,166, Y: 74,108, one pixel: 145 74
Blob 7 of 26, id: 18, sources: 1, size: 35x34, npix 819, brick X: 167,202, Y: 214,248, one pixel: 181 214
Blob 8 of 26, id: 15, sources: 1, size: 33x33, npix 748, brick X: 39,72, Y: 178,211, one pixel: 51 178
Blob 9 of 26, id: 17, sources: 2, size: 40x33, npix 719, brick X: 56,96, Y: 201,234, one pixel: 83 201
Blob 10 of 26, id: 19, sources: 1, size: 28x27, npix 486, brick X: 3,31, Y: 233,260, one pixel: 16 233
Blob 11 of 26, id: 6, sources: 1, size: 27x27, npix 477, brick X: 174,201, Y: 68,95, one pixel: 186 68
Blob 12 of 26, id: 12, sources: 1, size: 27x26, npix 455, brick X: 18,45, Y: 111,137, one pixel: 29 111
Blob 13 of 26, id: 1, sources: 1, size: 25x25, npix 424, brick X: 11,36, Y: 31,56, one pixel: 20 31
Blob 14 of 26, id: 3, sources: 1, size: 28x24, npix 424, brick X: 292,320, Y: 39,63, one pixel: 305 39
Blob 15 of 26, id: 13, sources: 1, size: 26x25, npix 421, brick X: 46,72, Y: 132,157, one pixel: 57 132
Blob 16 of 26, id: 5, sources: 1, size: 26x26, npix 416, brick X: 270,296, Y: 63,89, one pixel: 282 63
Blob 17 of 26, id: 9, sources: 1, size: 24x25, npix 379, brick X: 236,260, Y: 91,116, one pixel: 246 91
Blob 5 finished, total: Wall: 0.19 s, CPU: 0.18 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 113 MB, VmData: 161 MB, maxrss: 0.115516 MB
Blob 18 of 26, id: 20, sources: 1, size: 24x23, npix 329, brick X: 219,243, Y: 234,257, one pixel: 228 234
Blob 16 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147908 MB
Blob 19 of 26, id: 4, sources: 1, size: 22x23, npix 314, brick X: 200,222, Y: 44,67, one pixel: 209 44
Blob 20 of 26, id: 23, sources: 1, size: 24x18, npix 282, brick X: 257,281, Y: 305,323, one pixel: 267 305
Blob 12 finished, total: Wall: 0.58 s, CPU: 0.58 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147412 MB
Blob 21 of 26, id: 14, sources: 1, size: 21x21, npix 279, brick X: 285,306, Y: 139,160, one pixel: 294 139
Blob 13 finished, total: Wall: 0.62 s, CPU: 0.61 s, VmPeak: 814 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146228 MB
Blob 22 of 26, id: 24, sources: 1, size: 25x16, npix 278, brick X: 126,151, Y: 307,323, one pixel: 137 307
Blob 11 finished, total: Wall: 0.71 s, CPU: 0.71 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147916 MB
Blob 23 of 26, id: 7, sources: 1, size: 21x20, npix 249, brick X: 246,267, Y: 68,88, one pixel: 255 68
Blob 6 finished, total: Wall: 0.76 s, CPU: 0.76 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145404 MB
Blob 24 of 26, id: 16, sources: 1, size: 20x21, npix 248, brick X: 16,36, Y: 187,208, one pixel: 25 187
Blob 17 finished, total: Wall: 0.58 s, CPU: 0.58 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 114 MB, VmData: 161 MB, maxrss: 0.116692 MB
Blob 25 of 26, id: 11, sources: 1, size: 17x17, npix 145, brick X: 63,80, Y: 102,119, one pixel: 71 102
Blob 15 finished, total: Wall: 0.81 s, CPU: 0.81 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.1471 MB
Blob 8 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 805 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147532 MB
Blob 18 finished, total: Wall: 0.56 s, CPU: 0.56 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147908 MB
Blob 26 of 26, id: 25, sources: 1, size: 18x13, npix 142, brick X: 33,51, Y: 310,323, one pixel: 41 310
Blob 19 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148232 MB
Blob 25 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147908 MB
Blob 14 finished, total: Wall: 0.97 s, CPU: 0.97 s, VmPeak: 790 MB, VmSize: 712 MB, VmRSS: 130 MB, VmData: 169 MB, maxrss: 0.132916 MB
Blob 7 finished, total: Wall: 1.06 s, CPU: 1.06 s, VmPeak: 807 MB, VmSize: 725 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148216 MB
Blob 10 finished, total: Wall: 1.13 s, CPU: 1.13 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148016 MB
Blob 24 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.1471 MB
Blob 21 finished, total: Wall: 0.43 s, CPU: 0.43 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147916 MB
Blob 23 finished, total: Wall: 0.42 s, CPU: 0.42 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 117 MB, VmData: 161 MB, maxrss: 0.119732 MB
Blob 4 finished, total: Wall: 1.25 s, CPU: 1.25 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148296 MB
Blob 26 finished, total: Wall: 0.38 s, CPU: 0.38 s, VmPeak: 805 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147532 MB
Blob 20 finished, total: Wall: 0.62 s, CPU: 0.62 s, VmPeak: 814 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146228 MB
Blob 9 finished, total: Wall: 1.37 s, CPU: 1.36 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093272 MB
Blob 22 finished, total: Wall: 0.66 s, CPU: 0.66 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145404 MB
Measuring 1 iterative sources
Blob 3 finished, total: Wall: 2.91 s, CPU: 2.91 s, VmPeak: 806 MB, VmSize: 725 MB, VmRSS: 145 MB, VmData: 183 MB, maxrss: 0.148384 MB
Measuring 2 iterative sources
Blob 2 finished, total: Wall: 4.03 s, CPU: 4.03 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 97 MB, VmData: 142 MB, maxrss: 0.09882 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.6238, ee1=-0.632805, ee2=-0.0431245
Measuring 1 iterative sources
Blob 1 finished, total: Wall: 8.23 s, CPU: 8.23 s, VmPeak: 691 MB, VmSize: 689 MB, VmRSS: 115 MB, VmData: 157 MB, maxrss: 0.119852 MB
Old catalog: 50
New catalog: 42
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/blobs-custom-277454p48991.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/metrics/cus/all-models-custom-277454p48991.fits
Stage fitblobs : Wall: 8.74 s, CPU: 0.40 s, VmPeak: 1019 MB, VmSize: 1019 MB, VmRSS: 256 MB, VmData: 282 MB, maxrss: 0.261776 MB
Stage fitblobs finished: 2020-08-07T09:47:06.357071
Resources for stage fitblobs : 30.302 s worker CPU, 30.654 s worker Wall, Wall: 8.744 s, Cores in use: 3.51, Total efficiency (on 16 cores): 21.9 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-07T09:47:06.524337
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-resid.jpg
Stage coadds : Wall: 2.23 s, CPU: 0.69 s, VmPeak: 1019 MB, VmSize: 1019 MB, VmRSS: 256 MB, VmData: 282 MB, maxrss: 0.262112 MB
Stage coadds finished: 2020-08-07T09:47:08.750934
Resources for stage coadds : 7.773 s worker CPU, 8.339 s worker Wall, Wall: 2.228 s, Cores in use: 3.80, Total efficiency (on 16 cores): 23.8 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-07T09:47:08.870994
Cut to 1 unWISE tiles
Reading WISE tile 2767p484 band 1
Reading WISE tile 2767p484 band 2
Reading WISE tile 2767p484 band 4
Reading WISE tile 2767p484 band 3
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 814 MB, VmSize: 797 MB, VmRSS: 143 MB, VmData: 171 MB, maxrss: 0.204308 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 806 MB, VmSize: 801 MB, VmRSS: 146 MB, VmData: 175 MB, maxrss: 0.1891 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 812 MB, VmSize: 797 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.204724 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/276/2767p484/unwise-2767p484-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 805 MB, VmSize: 803 MB, VmRSS: 151 MB, VmData: 177 MB, maxrss: 0.215068 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/tmp-legacysurvey-custom-277454p48991-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 2.24 s, CPU: 0.55 s, VmPeak: 1019 MB, VmSize: 1019 MB, VmRSS: 256 MB, VmData: 282 MB, maxrss: 0.26264 MB
Stage wise_forced finished: 2020-08-07T09:47:11.112744
Resources for stage wise_forced : 3.264 s worker CPU, 4.710 s worker Wall, Wall: 2.243 s, Cores in use: 1.70, Total efficiency (on 16 cores): 10.6 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-07T09:47:11.140099
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/coadd/cus/custom-277454p48991/legacysurvey-custom-277454p48991-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/tractor-i/cus/tractor-custom-277454p48991.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/tractor/cus/tractor-custom-277454p48991.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/tractor/cus/brick-custom-277454p48991.sha256sum
Stage writecat : Wall: 0.79 s, CPU: 0.55 s, VmPeak: 1151 MB, VmSize: 1023 MB, VmRSS: 258 MB, VmData: 282 MB, maxrss: 0.395524 MB
Stage writecat finished: 2020-08-07T09:47:11.928236
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.790 s, Cores in use: 0.70, Total efficiency (on 16 cores): 4.4 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/277/PGC2330639/PGC2330639-largegalaxy-writecat.p
All done: 49.302 s worker CPU, 98.782 s worker Wall, Wall: 67.725 s, Cores in use: 0.87, Total efficiency (on 16 cores): 5.5 %
Finished galaxy PGC2330639 in 1.172 minutes.