Started working on galaxy PGC2561020 at Wed Aug  5 19:42:14 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 281.935416 57.2457504 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T19:42:15.150703
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '281.935416', '57.2457504', '--width', '251', '--height', '251', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 281.935416 57.2457504 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 281.935416 57.2457504
Starting process 41982 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 41983 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 41984 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 41985 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 41986 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB
Starting process 41987 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059556 MB
Starting process 41988 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059556 MB
Starting process 41989 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 41990 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 41991 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 41992 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 41993 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 41994 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 41995 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 41996 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 41997 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T19:42:15.788802
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 11 CCDs:
  k4m_170928_052544_ooi_zd_ls9-CCD2 z expnum 141159 exptime 232.101 propid 2016A-0453 seeing 1.73 MJD 58024.226 object MzLS_676460_z 
    mosaic/CP/V4.3/CP20170927/k4m_170928_052544_ooi_zd_ls9.fits.fz
  k4m_160531_101113_ooi_zd_ls9-CCD4 z expnum 92513 exptime 80.104 propid 2016A-0453 seeing 0.82 MJD 57539.424 object MzLS_677143_z 
    mosaic/CP/V4.3/CP20160530/k4m_160531_101113_ooi_zd_ls9.fits.fz
  k4m_160605_094121_ooi_zd_ls9-CCD1 z expnum 93542 exptime 80.103 propid 2016A-0453 seeing 0.86 MJD 57544.404 object MzLS_677824_z 
    mosaic/CP/V4.3/CP20160604/k4m_160605_094121_ooi_zd_ls9.fits.fz
  ksb_160713_092300_ooi_r_ls9-CCD1 r expnum 75820111 exptime 118.1 propid BASS seeing 1.72 MJD 57582.391 object 34872 
    90prime/CP/V2.3/CP20160712/ksb_160713_092300_ooi_r_ls9.fits.fz
  ksb_160708_093212_ooi_g_ls9-CCD1 g expnum 75770127 exptime 237.8 propid BASS seeing 1.92 MJD 57577.397 object 34872 
    90prime/CP/V2.3/CP20160707/ksb_160708_093212_ooi_g_ls9.fits.fz
  ksb_180519_075539_ooi_r_ls9-CCD2 r expnum 82570133 exptime 82.105 propid NOTBASS seeing 1.28 MJD 58257.330 object 35893 
    90prime/CP/V2.3/CP20180518/ksb_180519_075539_ooi_r_ls9.fits.fz
  ksb_170621_091210_ooi_g_ls9-CCD2 g expnum 79250071 exptime 167.401 propid BASS seeing 1.54 MJD 57925.383 object 35893 
    90prime/CP/V2.3/CP20170620/ksb_170621_091210_ooi_g_ls9.fits.fz
  ksb_170619_093122_ooi_r_ls9-CCD3 r expnum 79230133 exptime 125.8 propid BASS seeing 1.51 MJD 57923.397 object 34883 
    90prime/CP/V2.3/CP20170618/ksb_170619_093122_ooi_r_ls9.fits.fz
  ksb_170621_092717_ooi_g_ls9-CCD3 g expnum 79250074 exptime 126.3 propid BASS seeing 1.54 MJD 57925.394 object 34883 
    90prime/CP/V2.3/CP20170620/ksb_170621_092717_ooi_g_ls9.fits.fz
  ksb_170617_100817_ooi_r_ls9-CCD3 r expnum 79210156 exptime 128.316 propid BASS seeing 1.27 MJD 57921.422 object 34881 
    90prime/CP/V2.3/CP20170616/ksb_170617_100817_ooi_r_ls9.fits.fz
  ksb_180516_110301_ooi_g_ls9-CCD3 g expnum 82540191 exptime 108.621 propid NOTBASS seeing 1.17 MJD 58254.460 object 34881 
    90prime/CP/V2.3/CP20180515/ksb_180516_110301_ooi_g_ls9.fits.fz
Faked up EXPNUM 82570133
Faked up EXPNUM 82570133
Faked up EXPNUM 82570133
90prime get_fwhm: self.fwhm = 2.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3810 3959 and amp range 2016 4031 and image y 1228 1378 and amp range 0 2047
Range in image: x 0 150 , y 0 151 (with image size 150 x 151)
dzp 0.014483032 -> scaling image by 1.0134287315188357
mosaic get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2002 2257 and amp range 0 2005 and image y 1031 1289 and amp range 0 2039
Range in image: x 0 4 , y 0 259 (with image size 256 x 259)
dzp -0.011066288 -> scaling image by 0.989859339076806
Found overlap: image x 2002 2257 and amp range 2006 4053 and image y 1031 1289 and amp range 0 2039
Range in image: x 4 256 , y 0 259 (with image size 256 x 259)
dzp 0.001388291 -> scaling image by 1.0012794811153327
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.17
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 452 707 and amp range 0 2047 and image y 2907 3164 and amp range 2022 4060
Range in image: x 0 256 , y 0 258 (with image size 256 x 258)
dzp 0.0033053984 -> scaling image by 1.0030490232713904
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79230133
Faked up EXPNUM 79230133
Faked up EXPNUM 79230133
90prime get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 116 263 and amp range 0 2015 and image y 399 545 and amp range 0 2047
Range in image: x 0 148 , y 0 147 (with image size 148 x 147)
dzp -0.043123975 -> scaling image by 0.961059796099719
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79210156
Faked up EXPNUM 79210156
Faked up EXPNUM 79210156
90prime get_fwhm: self.fwhm = 2.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2108 2254 and amp range 2016 4031 and image y 2403 2548 and amp range 2048 4095
Range in image: x 0 147 , y 0 146 (with image size 147 x 146)
dzp -0.035697103 -> scaling image by 0.9676563679910417
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 6.65
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3138 3395 and amp range 2048 4095 and image y 3908 4072 and amp range 2040 4078
Range in image: x 0 258 , y 0 165 (with image size 258 x 165)
dzp -0.036442947 -> scaling image by 0.9669918668439786
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82540191
Faked up EXPNUM 82540191
Faked up EXPNUM 82540191
90prime get_fwhm: self.fwhm = 2.59
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2115 2261 and amp range 2016 4031 and image y 2409 2555 and amp range 2048 4095
Range in image: x 0 147 , y 0 147 (with image size 147 x 147)
dzp -0.01723723 -> scaling image by 0.9842492857213068
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79250074
Faked up EXPNUM 79250074
Faked up EXPNUM 79250074
90prime get_fwhm: self.fwhm = 3.39
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 129 276 and amp range 0 2015 and image y 448 594 and amp range 0 2047
Range in image: x 0 148 , y 0 147 (with image size 148 x 147)
dzp -0.036130503 -> scaling image by 0.9672701794860779
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79250071
Faked up EXPNUM 79250071
Faked up EXPNUM 79250071
90prime get_fwhm: self.fwhm = 3.41
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3845 3993 and amp range 2016 4031 and image y 1243 1393 and amp range 0 2047
Range in image: x 0 149 , y 0 151 (with image size 149 x 151)
dzp 0.01918938 -> scaling image by 1.017831182878158
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 10.03 s, CPU: 1.30 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 206 MB, maxrss: 0.178164 MB
Stage tims finished: 2020-08-05T19:42:25.819778
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Faked up EXPNUM 75770127
Faked up EXPNUM 75770127
Faked up EXPNUM 75770127
90prime get_fwhm: self.fwhm = 4.25
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 5.951 s worker CPU, 41.750 s worker Wall, Wall: 10.033 s, Cores in use: 0.72, Total efficiency (on 16 cores): 4.5 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-tims.p
Running stage refs
Faked up EXPNUM 75820111
Faked up EXPNUM 75820111
Faked up EXPNUM 75820111
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T19:42:25.847960
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00992.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00993.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00905.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01084.fits
Matched 1 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/reference-custom-281935p57245.fits
Stage refs : Wall: 9.60 s, CPU: 5.95 s, VmPeak: 1031 MB, VmSize: 1025 MB, VmRSS: 244 MB, VmData: 273 MB, maxrss: 0.249608 MB
Stage refs finished: 2020-08-05T19:42:35.444596
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 9.598 s, Cores in use: 0.62, Total efficiency (on 16 cores): 3.9 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-05T19:42:35.518874
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outlier-mask-custom-281935p57245.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-pre-custom-281935p57245.jpg
Image ksb_180516_110301_ooi_g_ls9-CCD3 g : masking 130 positive outlier pixels and 0 negative outlier pixels
Image ksb_180519_075539_ooi_r_ls9-CCD2 r : masking 75 positive outlier pixels and 0 negative outlier pixels
Image k4m_170928_052544_ooi_zd_ls9-CCD2 z : masking 796 positive outlier pixels and 0 negative outlier pixels
Image k4m_160531_101113_ooi_zd_ls9-CCD4 z : masking 205 positive outlier pixels and 0 negative outlier pixels
Image k4m_160605_094121_ooi_zd_ls9-CCD1 z : masking 872 positive outlier pixels and 205 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outlier-mask-custom-281935p57245.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-post-custom-281935p57245.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-masked-pos-custom-281935p57245.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-masked-neg-custom-281935p57245.jpg
Stage outliers : Wall: 0.87 s, CPU: 0.62 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 257 MB, VmData: 285 MB, maxrss: 0.262868 MB
Stage outliers finished: 2020-08-05T19:42:36.387527
Resources for stage outliers : 0.732 s worker CPU, 0.919 s worker Wall, Wall: 0.870 s, Cores in use: 1.56, Total efficiency (on 16 cores): 9.8 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T19:42:36.405705
Stage halos : Wall: 1.27 s, CPU: 1.18 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 257 MB, VmData: 285 MB, maxrss: 0.263048 MB
Stage halos finished: 2020-08-05T19:42:37.672903
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.269 s, Cores in use: 0.93, Total efficiency (on 16 cores): 5.8 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T19:42:37.765424
PSF sigmas in coadd pixels: ['2.51', '2.49', '1.91']
Using average PSF sigma 2.303189566421691
Gaussian PSF norm 0.12248005804061958 vs pixelized 0.10394893
average tim pixel scale / coadd scale: 1.729316258525746
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.08', '2.45', '2.07']
Using average PSF sigma 2.2004279633013373
Gaussian PSF norm 0.12819996676948553 vs pixelized 0.113201894
average tim pixel scale / coadd scale: 1.7294686204106866
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.80', '1.34', '1.40']
Using average PSF sigma 1.8484345064832688
Gaussian PSF norm 0.15261281413241762 vs pixelized 0.14565936
average tim pixel scale / coadd scale: 0.9909642656788742
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.21 s, CPU: 0.14 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 260 MB, VmData: 286 MB, maxrss: 0.266188 MB
Stage fit_on_coadds finished: 2020-08-05T19:42:37.970460
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage fit_on_coadds : 0.164 s worker CPU, 0.164 s worker Wall, Wall: 0.206 s, Cores in use: 1.50, Total efficiency (on 16 cores): 9.4 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T19:42:37.995153
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 35 potential peaks
Of 35 potential peaks: 9 in veto map, 5 cut by saddle test, 0 cut by aper test, 21 kept
SED z : 21 new peaks
Found 28 potential peaks
Of 28 potential peaks: 24 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept
SED r : 2 new peaks
Found 26 potential peaks
Of 26 potential peaks: 25 in veto map, 1 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 34 potential peaks
Of 34 potential peaks: 31 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED Flat : 0 new peaks
Found 33 potential peaks
Of 33 potential peaks: 30 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 23 new sources
Median coadd sky for g : 6.173941e-05
Median coadd sky for r : 7.974107e-05
Median coadd sky for z : -0.00022157143
Stage srcs : Wall: 2.71 s, CPU: 2.56 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 261 MB, VmData: 286 MB, maxrss: 0.266876 MB
Stage srcs finished: 2020-08-05T19:42:40.706574
Resources for stage srcs : 0.088 s worker CPU, 0.088 s worker Wall, Wall: 2.713 s, Cores in use: 0.98, Total efficiency (on 16 cores): 6.1 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T19:42:40.726311
Blob 1 of 9, id: 2, sources: 23, size: 232x224, npix 22323, brick X: 0,232, Y: 22,246, one pixel: 140 22
Blob 2 of 9, id: 6, sources: 1, size: 35x54, npix 1464, brick X: 0,35, Y: 146,200, one pixel: 7 146
Blob 3 of 9, id: 7, sources: 1, size: 35x35, npix 866, brick X: 208,243, Y: 179,214, one pixel: 220 179
Blob 4 of 9, id: 4, sources: 3, size: 25x41, npix 802, brick X: 226,251, Y: 63,104, one pixel: 238 63
Blob 5 of 9, id: 0, sources: 1, size: 32x30, npix 637, brick X: 77,109, Y: 5,35, one pixel: 92 5
Blob 6 of 9, id: 3, sources: 2, size: 20x38, npix 539, brick X: 0,20, Y: 38,76, one pixel: 6 38
Blob 7 of 9, id: 5, sources: 1, size: 26x28, npix 469, brick X: 44,70, Y: 124,152, one pixel: 56 124
Blob 8 of 9, id: 1, sources: 1, size: 22x22, npix 313, brick X: 225,247, Y: 9,31, one pixel: 233 9
Blob 9 of 9, id: 8, sources: 1, size: 23x16, npix 247, brick X: 3,26, Y: 235,251, one pixel: 12 235
Blob 8 finished, total: Wall: 0.22 s, CPU: 0.20 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150092 MB
Blob 9 finished, total: Wall: 0.20 s, CPU: 0.16 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 93 MB, VmData: 141 MB, maxrss: 0.095716 MB
Blob 7 finished, total: Wall: 0.43 s, CPU: 0.40 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.091808 MB
Blob 5 finished, total: Wall: 0.68 s, CPU: 0.68 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151176 MB
Blob 3 finished, total: Wall: 0.74 s, CPU: 0.72 s, VmPeak: 790 MB, VmSize: 719 MB, VmRSS: 134 MB, VmData: 171 MB, maxrss: 0.137612 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (281.90880, 57.24275), pm (-3.7, -6.2), parallax 0.998 with NanoMaggies: g=15.4, r=14.8, z=14.6
Reset brightness to NanoMaggies: g=15.4, r=14.8, z=14.6
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (281.91021, 57.24161), pm (1.0, -1.4), parallax 0.441 with NanoMaggies: g=16.2, r=15.7, z=15.5
Reset brightness to NanoMaggies: g=16.2, r=15.7, z=15.5
Blob 4 finished, total: Wall: 0.74 s, CPU: 0.70 s, VmPeak: 811 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149968 MB
Blob 6 finished, total: Wall: 0.92 s, CPU: 0.89 s, VmPeak: 812 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149924 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 1.19 s, CPU: 1.17 s, VmPeak: 805 MB, VmSize: 731 MB, VmRSS: 148 MB, VmData: 183 MB, maxrss: 0.151412 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.67202, ee1=0.0560786, ee2=-0.488505
Measuring 12 iterative sources
Blob 1 finished, total: Wall: 20.30 s, CPU: 20.29 s, VmPeak: 814 MB, VmSize: 741 MB, VmRSS: 166 MB, VmData: 199 MB, maxrss: 0.169856 MB
Old catalog: 40
New catalog: 47
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/blobs-custom-281935p57245.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/all-models-custom-281935p57245.fits
Stage fitblobs : Wall: 22.32 s, CPU: 1.76 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 261 MB, VmData: 286 MB, maxrss: 0.267432 MB
Stage fitblobs finished: 2020-08-05T19:43:03.044368
Resources for stage fitblobs : 25.338 s worker CPU, 26.190 s worker Wall, Wall: 22.319 s, Cores in use: 1.21, Total efficiency (on 16 cores): 7.6 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T19:43:03.059889
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-resid.jpg
Stage coadds : Wall: 3.22 s, CPU: 1.21 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 261 MB, VmData: 286 MB, maxrss: 0.267756 MB
Stage coadds finished: 2020-08-05T19:43:06.284165
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 8.163 s worker CPU, 13.619 s worker Wall, Wall: 3.226 s, Cores in use: 2.91, Total efficiency (on 16 cores): 18.2 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T19:43:06.301407
Cut to 1 unWISE tiles
Reading WISE tile 2824p575 band 1
Reading WISE tile 2824p575 band 2
Reading WISE tile 2824p575 band 3
Reading WISE tile 2824p575 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 805 MB, VmSize: 804 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.190528 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 812 MB, VmSize: 799 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.20488 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 815 MB, VmSize: 813 MB, VmRSS: 159 MB, VmData: 181 MB, maxrss: 0.223484 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 812 MB, VmSize: 812 MB, VmRSS: 151 MB, VmData: 180 MB, maxrss: 0.207756 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 3.79 s, CPU: 0.93 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 262 MB, VmData: 286 MB, maxrss: 0.268268 MB
Stage wise_forced finished: 2020-08-05T19:43:10.089229
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage wise_forced : 3.135 s worker CPU, 7.712 s worker Wall, Wall: 3.789 s, Cores in use: 1.07, Total efficiency (on 16 cores): 6.7 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T19:43:10.129472
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/tractor-i/cus/tractor-custom-281935p57245.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/tractor/cus/tractor-custom-281935p57245.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/tractor/cus/brick-custom-281935p57245.sha256sum
Stage writecat : Wall: 1.00 s, CPU: 0.80 s, VmPeak: 1167 MB, VmSize: 1039 MB, VmRSS: 264 MB, VmData: 286 MB, maxrss: 0.401132 MB
Stage writecat finished: 2020-08-05T19:43:11.127739
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.000 s, Cores in use: 0.80, Total efficiency (on 16 cores): 5.0 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-writecat.p
All done: 43.574 s worker CPU, 90.446 s worker Wall, Wall: 55.368 s, Cores in use: 1.09, Total efficiency (on 16 cores): 6.8 %
Finished galaxy PGC2561020 in 0.953 minutes.
Started working on galaxy PGC2561020 at Wed Aug  5 20:27:03 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 281.935416 57.2457504 --width 251 --height 251 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-05T20:27:06.637242
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '281.935416', '57.2457504', '--width', '251', '--height', '251', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 281.935416 57.2457504 --width 251 --height 251 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 281.935416 57.2457504
Starting process 51796 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 51797 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB
Starting process 51798 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 51799 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 51800 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05948 MB
Starting process 51801 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB
Starting process 51802 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059428 MB
Starting process 51803 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 51804 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 51805 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 51806 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 51807 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 51808 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 51809 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 51810 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 51811 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 51812 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB
Starting process 51813 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 51814 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB
Starting process 51815 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 51816 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB
Starting process 51817 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 51818 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 51819 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 51820 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 51821 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 51822 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 51823 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB
Starting process 51824 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 51825 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 51826 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB
Starting process 51827 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-05T20:27:08.513132
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 11 CCDs:
  k4m_170928_052544_ooi_zd_ls9-CCD2 z expnum 141159 exptime 232.101 propid 2016A-0453 seeing 1.73 MJD 58024.226 object MzLS_676460_z 
    mosaic/CP/V4.3/CP20170927/k4m_170928_052544_ooi_zd_ls9.fits.fz
  k4m_160531_101113_ooi_zd_ls9-CCD4 z expnum 92513 exptime 80.104 propid 2016A-0453 seeing 0.82 MJD 57539.424 object MzLS_677143_z 
    mosaic/CP/V4.3/CP20160530/k4m_160531_101113_ooi_zd_ls9.fits.fz
  k4m_160605_094121_ooi_zd_ls9-CCD1 z expnum 93542 exptime 80.103 propid 2016A-0453 seeing 0.86 MJD 57544.404 object MzLS_677824_z 
    mosaic/CP/V4.3/CP20160604/k4m_160605_094121_ooi_zd_ls9.fits.fz
  ksb_160713_092300_ooi_r_ls9-CCD1 r expnum 75820111 exptime 118.1 propid BASS seeing 1.72 MJD 57582.391 object 34872 
    90prime/CP/V2.3/CP20160712/ksb_160713_092300_ooi_r_ls9.fits.fz
  ksb_160708_093212_ooi_g_ls9-CCD1 g expnum 75770127 exptime 237.8 propid BASS seeing 1.92 MJD 57577.397 object 34872 
    90prime/CP/V2.3/CP20160707/ksb_160708_093212_ooi_g_ls9.fits.fz
  ksb_180519_075539_ooi_r_ls9-CCD2 r expnum 82570133 exptime 82.105 propid NOTBASS seeing 1.28 MJD 58257.330 object 35893 
    90prime/CP/V2.3/CP20180518/ksb_180519_075539_ooi_r_ls9.fits.fz
  ksb_170621_091210_ooi_g_ls9-CCD2 g expnum 79250071 exptime 167.401 propid BASS seeing 1.54 MJD 57925.383 object 35893 
    90prime/CP/V2.3/CP20170620/ksb_170621_091210_ooi_g_ls9.fits.fz
  ksb_170619_093122_ooi_r_ls9-CCD3 r expnum 79230133 exptime 125.8 propid BASS seeing 1.51 MJD 57923.397 object 34883 
    90prime/CP/V2.3/CP20170618/ksb_170619_093122_ooi_r_ls9.fits.fz
  ksb_170621_092717_ooi_g_ls9-CCD3 g expnum 79250074 exptime 126.3 propid BASS seeing 1.54 MJD 57925.394 object 34883 
    90prime/CP/V2.3/CP20170620/ksb_170621_092717_ooi_g_ls9.fits.fz
  ksb_170617_100817_ooi_r_ls9-CCD3 r expnum 79210156 exptime 128.316 propid BASS seeing 1.27 MJD 57921.422 object 34881 
    90prime/CP/V2.3/CP20170616/ksb_170617_100817_ooi_r_ls9.fits.fz
  ksb_180516_110301_ooi_g_ls9-CCD3 g expnum 82540191 exptime 108.621 propid NOTBASS seeing 1.17 MJD 58254.460 object 34881 
    90prime/CP/V2.3/CP20180515/ksb_180516_110301_ooi_g_ls9.fits.fz
Faked up EXPNUM 79230133
Faked up EXPNUM 79230133
Faked up EXPNUM 79230133
90prime get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 116 263 and amp range 0 2015 and image y 399 545 and amp range 0 2047
Range in image: x 0 148 , y 0 147 (with image size 148 x 147)
dzp -0.043123975 -> scaling image by 0.961059796099719
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82570133
Faked up EXPNUM 82570133
Faked up EXPNUM 82570133
90prime get_fwhm: self.fwhm = 2.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3810 3959 and amp range 2016 4031 and image y 1228 1378 and amp range 0 2047
Range in image: x 0 150 , y 0 151 (with image size 150 x 151)
dzp 0.014483032 -> scaling image by 1.0134287315188357
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2002 2257 and amp range 0 2005 and image y 1031 1289 and amp range 0 2039
Range in image: x 0 4 , y 0 259 (with image size 256 x 259)
dzp -0.011066288 -> scaling image by 0.989859339076806
Found overlap: image x 2002 2257 and amp range 2006 4053 and image y 1031 1289 and amp range 0 2039
Range in image: x 4 256 , y 0 259 (with image size 256 x 259)
dzp 0.001388291 -> scaling image by 1.0012794811153327
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79210156
Faked up EXPNUM 79210156
Faked up EXPNUM 79210156
90prime get_fwhm: self.fwhm = 2.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2108 2254 and amp range 2016 4031 and image y 2403 2548 and amp range 2048 4095
Range in image: x 0 147 , y 0 146 (with image size 147 x 146)
dzp -0.035697103 -> scaling image by 0.9676563679910417
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 82540191
Faked up EXPNUM 82540191
Faked up EXPNUM 82540191
90prime get_fwhm: self.fwhm = 2.59
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2115 2261 and amp range 2016 4031 and image y 2409 2555 and amp range 2048 4095
Range in image: x 0 147 , y 0 147 (with image size 147 x 147)
dzp -0.01723723 -> scaling image by 0.9842492857213068
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 3.17
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 452 707 and amp range 0 2047 and image y 2907 3164 and amp range 2022 4060
Range in image: x 0 256 , y 0 258 (with image size 256 x 258)
dzp 0.0033053984 -> scaling image by 1.0030490232713904
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
mosaic get_fwhm: self.fwhm = 6.65
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3138 3395 and amp range 2048 4095 and image y 3908 4072 and amp range 2040 4078
Range in image: x 0 258 , y 0 165 (with image size 258 x 165)
dzp -0.036442947 -> scaling image by 0.9669918668439786
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79250074
Faked up EXPNUM 79250074
Faked up EXPNUM 79250074
90prime get_fwhm: self.fwhm = 3.39
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 129 276 and amp range 0 2015 and image y 448 594 and amp range 0 2047
Range in image: x 0 148 , y 0 147 (with image size 148 x 147)
dzp -0.036130503 -> scaling image by 0.9672701794860779
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Faked up EXPNUM 79250071
Faked up EXPNUM 79250071
Faked up EXPNUM 79250071
90prime get_fwhm: self.fwhm = 3.41
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3845 3993 and amp range 2016 4031 and image y 1243 1393 and amp range 0 2047
Range in image: x 0 149 , y 0 151 (with image size 149 x 151)
dzp 0.01918938 -> scaling image by 1.017831182878158
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 11.85 s, CPU: 1.61 s, VmPeak: 957 MB, VmSize: 957 MB, VmRSS: 174 MB, VmData: 206 MB, maxrss: 0.178088 MB
Stage tims finished: 2020-08-05T20:27:20.360567
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 5.772 s worker CPU, 50.635 s worker Wall, Wall: 11.850 s, Cores in use: 0.62, Total efficiency (on 32 cores): 1.9 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-tims.p
Running stage refs
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage refs at 2020-08-05T20:27:20.385991
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00992.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00993.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00905.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01084.fits
Matched 1 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/reference-custom-281935p57245.fits
Stage refs : Wall: 16.20 s, CPU: 5.55 s, VmPeak: 1031 MB, VmSize: 1025 MB, VmRSS: 244 MB, VmData: 273 MB, maxrss: 0.249916 MB
Stage refs finished: 2020-08-05T20:27:36.587043
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 16.203 s, Cores in use: 0.34, Total efficiency (on 32 cores): 1.1 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-refs.p
Running stage outliers
Faked up EXPNUM 75820111
Faked up EXPNUM 75820111
Faked up EXPNUM 75820111
Running stage outliers at 2020-08-05T20:27:36.745652
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outlier-mask-custom-281935p57245.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-pre-custom-281935p57245.jpg
Image ksb_180516_110301_ooi_g_ls9-CCD3 g : masking 130 positive outlier pixels and 0 negative outlier pixels
Image ksb_180519_075539_ooi_r_ls9-CCD2 r : masking 75 positive outlier pixels and 0 negative outlier pixels
Image k4m_160531_101113_ooi_zd_ls9-CCD4 z : masking 205 positive outlier pixels and 0 negative outlier pixels
Image k4m_170928_052544_ooi_zd_ls9-CCD2 z : masking 796 positive outlier pixels and 0 negative outlier pixels
Image k4m_160605_094121_ooi_zd_ls9-CCD1 z : masking 872 positive outlier pixels and 205 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outlier-mask-custom-281935p57245.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-post-custom-281935p57245.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-masked-pos-custom-281935p57245.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/outliers-masked-neg-custom-281935p57245.jpg
Stage outliers : Wall: 1.88 s, CPU: 1.43 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 258 MB, VmData: 285 MB, maxrss: 0.26378 MB
Stage outliers finished: 2020-08-05T20:27:38.629954
Faked up EXPNUM 75770127
Faked up EXPNUM 75770127
Faked up EXPNUM 75770127
90prime get_fwhm: self.fwhm = 4.25
Resources for stage outliers : 0.865 s worker CPU, 1.086 s worker Wall, Wall: 1.887 s, Cores in use: 1.22, Total efficiency (on 32 cores): 3.8 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-05T20:27:38.654129
Stage halos : Wall: 0.98 s, CPU: 0.95 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 258 MB, VmData: 285 MB, maxrss: 0.26386 MB
Stage halos finished: 2020-08-05T20:27:39.631310
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.979 s, Cores in use: 0.97, Total efficiency (on 32 cores): 3.0 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-05T20:27:40.386306
PSF sigmas in coadd pixels: ['2.51', '2.49', '1.91']
Using average PSF sigma 2.303189566421691
Gaussian PSF norm 0.12248005804061958 vs pixelized 0.10394893
average tim pixel scale / coadd scale: 1.729316258525746
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.08', '2.45', '2.07']
Using average PSF sigma 2.2004279633013373
Gaussian PSF norm 0.12819996676948553 vs pixelized 0.113201894
average tim pixel scale / coadd scale: 1.7294686204106866
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.80', '1.34', '1.40']
Using average PSF sigma 1.8484345064832688
Gaussian PSF norm 0.15261281413241762 vs pixelized 0.14565936
average tim pixel scale / coadd scale: 0.9909642656788742
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.17 s, CPU: 0.14 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 259 MB, VmData: 285 MB, maxrss: 0.265368 MB
Stage fit_on_coadds finished: 2020-08-05T20:27:40.553598
Resources for stage fit_on_coadds : 0.147 s worker CPU, 0.147 s worker Wall, Wall: 0.170 s, Cores in use: 1.72, Total efficiency (on 32 cores): 5.4 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-05T20:27:40.567005
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 35 potential peaks
Of 35 potential peaks: 9 in veto map, 5 cut by saddle test, 0 cut by aper test, 21 kept
SED z : 21 new peaks
Found 28 potential peaks
Of 28 potential peaks: 24 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept
SED r : 2 new peaks
Found 26 potential peaks
Of 26 potential peaks: 25 in veto map, 1 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 34 potential peaks
Of 34 potential peaks: 31 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED Flat : 0 new peaks
Found 33 potential peaks
Of 33 potential peaks: 30 in veto map, 3 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 23 new sources
Median coadd sky for g : 6.1739454e-05
Median coadd sky for r : 7.9740996e-05
Median coadd sky for z : -0.00022157143
Stage srcs : Wall: 1.27 s, CPU: 1.16 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 260 MB, VmData: 285 MB, maxrss: 0.265872 MB
Stage srcs finished: 2020-08-05T20:27:41.834993
Resources for stage srcs : 0.080 s worker CPU, 0.080 s worker Wall, Wall: 1.270 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-05T20:27:41.859243
Blob 1 of 9, id: 2, sources: 23, size: 232x224, npix 22323, brick X: 0,232, Y: 22,246, one pixel: 140 22
Blob 2 of 9, id: 6, sources: 1, size: 35x54, npix 1464, brick X: 0,35, Y: 146,200, one pixel: 7 146
Blob 3 of 9, id: 7, sources: 1, size: 35x35, npix 866, brick X: 208,243, Y: 179,214, one pixel: 220 179
Blob 4 of 9, id: 4, sources: 3, size: 25x41, npix 802, brick X: 226,251, Y: 63,104, one pixel: 238 63
Blob 5 of 9, id: 0, sources: 1, size: 32x30, npix 637, brick X: 77,109, Y: 5,35, one pixel: 92 5
Blob 6 of 9, id: 3, sources: 2, size: 20x38, npix 539, brick X: 0,20, Y: 38,76, one pixel: 6 38
Blob 7 of 9, id: 5, sources: 1, size: 26x28, npix 469, brick X: 44,70, Y: 124,152, one pixel: 56 124
Blob 8 of 9, id: 1, sources: 1, size: 22x22, npix 313, brick X: 225,247, Y: 9,31, one pixel: 233 9
Blob 9 of 9, id: 8, sources: 1, size: 23x16, npix 247, brick X: 3,26, Y: 235,251, one pixel: 12 235
Blob 9 finished, total: Wall: 0.18 s, CPU: 0.17 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092252 MB
Blob 8 finished, total: Wall: 0.19 s, CPU: 0.18 s, VmPeak: 811 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149372 MB
Blob 7 finished, total: Wall: 0.43 s, CPU: 0.43 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092236 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (281.90880, 57.24275), pm (-3.7, -6.2), parallax 0.998 with NanoMaggies: g=15.4, r=14.8, z=14.6
Reset brightness to NanoMaggies: g=15.4, r=14.8, z=14.6
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (281.91021, 57.24161), pm (1.0, -1.4), parallax 0.441 with NanoMaggies: g=16.2, r=15.7, z=15.5
Reset brightness to NanoMaggies: g=16.2, r=15.7, z=15.5
Blob 4 finished, total: Wall: 0.67 s, CPU: 0.66 s, VmPeak: 812 MB, VmSize: 731 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149336 MB
Blob 5 finished, total: Wall: 0.69 s, CPU: 0.68 s, VmPeak: 813 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 183 MB, maxrss: 0.150188 MB
Blob 3 finished, total: Wall: 0.73 s, CPU: 0.73 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093464 MB
Blob 6 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093404 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 1.06 s, CPU: 1.06 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.095504 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.67202, ee1=0.0560786, ee2=-0.488505
Measuring 12 iterative sources
Blob 1 finished, total: Wall: 20.62 s, CPU: 20.62 s, VmPeak: 712 MB, VmSize: 692 MB, VmRSS: 125 MB, VmData: 163 MB, maxrss: 0.146932 MB
Old catalog: 40
New catalog: 47
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/blobs-custom-281935p57245.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/metrics/cus/all-models-custom-281935p57245.fits
Stage fitblobs : Wall: 21.77 s, CPU: 1.08 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 260 MB, VmData: 285 MB, maxrss: 0.266428 MB
Stage fitblobs finished: 2020-08-05T20:28:03.626536
Resources for stage fitblobs : 25.507 s worker CPU, 25.629 s worker Wall, Wall: 21.770 s, Cores in use: 1.22, Total efficiency (on 32 cores): 3.8 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-05T20:28:03.643792
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-resid.jpg
Stage coadds : Wall: 3.51 s, CPU: 0.75 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 260 MB, VmData: 285 MB, maxrss: 0.266752 MB
Stage coadds finished: 2020-08-05T20:28:07.154877
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 24.563 s worker CPU, 51.807 s worker Wall, Wall: 3.514 s, Cores in use: 7.20, Total efficiency (on 32 cores): 22.5 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-05T20:28:07.183747
Cut to 1 unWISE tiles
Reading WISE tile 2824p575 band 2
Reading WISE tile 2824p575 band 1
Reading WISE tile 2824p575 band 3
Reading WISE tile 2824p575 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.09 s, CPU: 0.09 s, VmPeak: 805 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 175 MB, maxrss: 0.190144 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 812 MB, VmSize: 803 MB, VmRSS: 144 MB, VmData: 171 MB, maxrss: 0.20722 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 807 MB, VmSize: 807 MB, VmRSS: 149 MB, VmData: 179 MB, maxrss: 0.200104 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 813 MB, VmSize: 804 MB, VmRSS: 145 MB, VmData: 171 MB, maxrss: 0.208428 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/tmp-legacysurvey-custom-281935p57245-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 4.62 s, CPU: 2.11 s, VmPeak: 1034 MB, VmSize: 1034 MB, VmRSS: 261 MB, VmData: 285 MB, maxrss: 0.267264 MB
Stage wise_forced finished: 2020-08-05T20:28:11.805659
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage wise_forced : 3.563 s worker CPU, 7.534 s worker Wall, Wall: 4.624 s, Cores in use: 1.23, Total efficiency (on 32 cores): 3.8 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-05T20:28:11.829108
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/coadd/cus/custom-281935p57245/legacysurvey-custom-281935p57245-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/tractor-i/cus/tractor-custom-281935p57245.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/tractor/cus/tractor-custom-281935p57245.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/tractor/cus/brick-custom-281935p57245.sha256sum
Stage writecat : Wall: 1.95 s, CPU: 1.24 s, VmPeak: 1166 MB, VmSize: 1038 MB, VmRSS: 263 MB, VmData: 285 MB, maxrss: 0.400144 MB
Stage writecat finished: 2020-08-05T20:28:13.783866
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.957 s, Cores in use: 0.63, Total efficiency (on 32 cores): 2.0 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/281/PGC2561020/PGC2561020-largegalaxy-writecat.p
All done: 60.504 s worker CPU, 136.924 s worker Wall, Wall: 65.302 s, Cores in use: 1.17, Total efficiency (on 32 cores): 3.7 %
Finished galaxy PGC2561020 in 1.186 minutes.
Started working on galaxy PGC2561020 at Fri Aug  7 09:27:30 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 281.93476425184394 57.245533256454415 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-07T09:27:31.315276
legacypipe git version: DR9.5.11-watershed-3-gd509eaaf
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '281.93476425184394', '57.245533256454415', '--width', '251', '--height', '251', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 281.93476425184394 57.245533256454415 --width 251 --height 251 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 281.93476425184394 57.245533256454415
Starting process 47750 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059228 MB
Starting process 47751 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB
Starting process 47752 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 47753 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 47754 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB
Starting process 47755 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059324 MB
Starting process 47756 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059324 MB
Starting process 47757 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 47758 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB
Starting process 47759 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 47760 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 47761 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 47762 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 47763 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 47764 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB
Starting process 47765 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-07T09:27:31.779561
Keeping 11 CCDs:
  k4m_170928_052544_ooi_zd_ls9-CCD2 z expnum 141159 exptime 232.101 propid 2016A-0453 seeing 1.73 MJD 58024.226 object MzLS_676460_z 
    mosaic/CP/V4.3/CP20170927/k4m_170928_052544_ooi_zd_ls9.fits.fz
  k4m_160531_101113_ooi_zd_ls9-CCD4 z expnum 92513 exptime 80.104 propid 2016A-0453 seeing 0.82 MJD 57539.424 object MzLS_677143_z 
    mosaic/CP/V4.3/CP20160530/k4m_160531_101113_ooi_zd_ls9.fits.fz
  k4m_160605_094121_ooi_zd_ls9-CCD1 z expnum 93542 exptime 80.103 propid 2016A-0453 seeing 0.86 MJD 57544.404 object MzLS_677824_z 
    mosaic/CP/V4.3/CP20160604/k4m_160605_094121_ooi_zd_ls9.fits.fz
  ksb_160713_092300_ooi_r_ls9-CCD1 r expnum 75820111 exptime 118.1 propid BASS seeing 1.72 MJD 57582.391 object 34872 
    90prime/CP/V2.3/CP20160712/ksb_160713_092300_ooi_r_ls9.fits.fz
  ksb_160708_093212_ooi_g_ls9-CCD1 g expnum 75770127 exptime 237.8 propid BASS seeing 1.92 MJD 57577.397 object 34872 
    90prime/CP/V2.3/CP20160707/ksb_160708_093212_ooi_g_ls9.fits.fz
  ksb_180519_075539_ooi_r_ls9-CCD2 r expnum 82570133 exptime 82.105 propid NOTBASS seeing 1.28 MJD 58257.330 object 35893 
    90prime/CP/V2.3/CP20180518/ksb_180519_075539_ooi_r_ls9.fits.fz
  ksb_170621_091210_ooi_g_ls9-CCD2 g expnum 79250071 exptime 167.401 propid BASS seeing 1.54 MJD 57925.383 object 35893 
    90prime/CP/V2.3/CP20170620/ksb_170621_091210_ooi_g_ls9.fits.fz
  ksb_170619_093122_ooi_r_ls9-CCD3 r expnum 79230133 exptime 125.8 propid BASS seeing 1.51 MJD 57923.397 object 34883 
    90prime/CP/V2.3/CP20170618/ksb_170619_093122_ooi_r_ls9.fits.fz
  ksb_170621_092717_ooi_g_ls9-CCD3 g expnum 79250074 exptime 126.3 propid BASS seeing 1.54 MJD 57925.394 object 34883 
    90prime/CP/V2.3/CP20170620/ksb_170621_092717_ooi_g_ls9.fits.fz
  ksb_170617_100817_ooi_r_ls9-CCD3 r expnum 79210156 exptime 128.316 propid BASS seeing 1.27 MJD 57921.422 object 34881 
    90prime/CP/V2.3/CP20170616/ksb_170617_100817_ooi_r_ls9.fits.fz
  ksb_180516_110301_ooi_g_ls9-CCD3 g expnum 82540191 exptime 108.621 propid NOTBASS seeing 1.17 MJD 58254.460 object 34881 
    90prime/CP/V2.3/CP20180515/ksb_180516_110301_ooi_g_ls9.fits.fz
mosaic get_fwhm: self.fwhm = 6.65
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3135 3392 and amp range 2048 4095 and image y 3903 4072 and amp range 2040 4078
Range in image: x 0 258 , y 0 170 (with image size 258 x 170)
dzp -0.036442947 -> scaling image by 0.9669918668439786
mosaic get_fwhm: self.fwhm = 3.17
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 449 704 and amp range 0 2047 and image y 2903 3159 and amp range 2022 4060
Range in image: x 0 256 , y 0 257 (with image size 256 x 257)
dzp 0.0033053984 -> scaling image by 1.0030490232713904
mosaic get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 1999 2254 and amp range 0 2005 and image y 1026 1284 and amp range 0 2039
Range in image: x 0 7 , y 0 259 (with image size 256 x 259)
dzp -0.011066288 -> scaling image by 0.989859339076806
Found overlap: image x 1999 2254 and amp range 2006 4053 and image y 1026 1284 and amp range 0 2039
Range in image: x 7 256 , y 0 259 (with image size 256 x 259)
dzp 0.001388291 -> scaling image by 1.0012794811153327
Faked up EXPNUM 82570133
Faked up EXPNUM 82570133
Faked up EXPNUM 82570133
90prime get_fwhm: self.fwhm = 2.83
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3812 3961 and amp range 2016 4031 and image y 1231 1381 and amp range 0 2047
Range in image: x 0 150 , y 0 151 (with image size 150 x 151)
dzp 0.014483032 -> scaling image by 1.0134287315188357
Faked up EXPNUM 79230133
Faked up EXPNUM 79230133
Faked up EXPNUM 79230133
90prime get_fwhm: self.fwhm = 3.33
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 118 265 and amp range 0 2015 and image y 402 548 and amp range 0 2047
Range in image: x 0 148 , y 0 147 (with image size 148 x 147)
dzp -0.043123975 -> scaling image by 0.961059796099719
Faked up EXPNUM 79250071
Faked up EXPNUM 79250071
Faked up EXPNUM 79250071
90prime get_fwhm: self.fwhm = 3.41
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 3846 3995 and amp range 2016 4031 and image y 1246 1395 and amp range 0 2047
Range in image: x 0 150 , y 0 150 (with image size 150 x 150)
dzp 0.01918938 -> scaling image by 1.017831182878158
Faked up EXPNUM 82540191
Faked up EXPNUM 82540191
Faked up EXPNUM 82540191
90prime get_fwhm: self.fwhm = 2.59
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2117 2263 and amp range 2016 4031 and image y 2412 2558 and amp range 2048 4095
Range in image: x 0 147 , y 0 147 (with image size 147 x 147)
dzp -0.01723723 -> scaling image by 0.9842492857213068
Faked up EXPNUM 79250074
Faked up EXPNUM 79250074
Faked up EXPNUM 79250074
90prime get_fwhm: self.fwhm = 3.39
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 131 278 and amp range 0 2015 and image y 451 597 and amp range 0 2047
Range in image: x 0 148 , y 0 147 (with image size 148 x 147)
dzp -0.036130503 -> scaling image by 0.9672701794860779
Faked up EXPNUM 79210156
Faked up EXPNUM 79210156
Faked up EXPNUM 79210156
90prime get_fwhm: self.fwhm = 2.81
Found 4 relevant rows in amp-corrections file.
Found overlap: image x 2110 2256 and amp range 2016 4031 and image y 2405 2551 and amp range 2048 4095
Range in image: x 0 147 , y 0 147 (with image size 147 x 147)
dzp -0.035697103 -> scaling image by 0.9676563679910417
Stage tims : Wall: 87.25 s, CPU: 0.89 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 172 MB, VmData: 206 MB, maxrss: 0.175984 MB
Stage tims finished: 2020-08-07T09:28:59.034481
Faked up EXPNUM 75820111
Faked up EXPNUM 75820111
Faked up EXPNUM 75820111
Faked up EXPNUM 75770127
Faked up EXPNUM 75770127
Faked up EXPNUM 75770127
90prime get_fwhm: self.fwhm = 4.25
Resources for stage tims : 5.471 s worker CPU, 408.485 s worker Wall, Wall: 87.257 s, Cores in use: 0.07, Total efficiency (on 16 cores): 0.5 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-08-07T09:29:01.526371
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00992.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00993.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00905.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01084.fits
Matched 1 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/reference-custom-281934p57245.fits
Stage refs : Wall: 7.11 s, CPU: 4.57 s, VmPeak: 1024 MB, VmSize: 1018 MB, VmRSS: 242 MB, VmData: 273 MB, maxrss: 0.247744 MB
Stage refs finished: 2020-08-07T09:29:08.637331
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 7.113 s, Cores in use: 0.64, Total efficiency (on 16 cores): 4.0 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-07T09:29:11.596055
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/outlier-mask-custom-281934p57245.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/outliers-pre-custom-281934p57245.jpg
Image ksb_180516_110301_ooi_g_ls9-CCD3 g : masking 129 positive outlier pixels and 0 negative outlier pixels
Image ksb_180519_075539_ooi_r_ls9-CCD2 r : masking 73 positive outlier pixels and 0 negative outlier pixels
Image k4m_170928_052544_ooi_zd_ls9-CCD2 z : masking 662 positive outlier pixels and 0 negative outlier pixels
Image k4m_160531_101113_ooi_zd_ls9-CCD4 z : masking 0 positive outlier pixels and 177 negative outlier pixels
Image k4m_160605_094121_ooi_zd_ls9-CCD1 z : masking 1161 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/outlier-mask-custom-281934p57245.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/outliers-post-custom-281934p57245.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/outliers-masked-pos-custom-281934p57245.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/outliers-masked-neg-custom-281934p57245.jpg
Stage outliers : Wall: 2.46 s, CPU: 0.53 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 256 MB, VmData: 285 MB, maxrss: 0.262124 MB
Stage outliers finished: 2020-08-07T09:29:14.056681
Resources for stage outliers : 0.688 s worker CPU, 0.765 s worker Wall, Wall: 2.462 s, Cores in use: 0.50, Total efficiency (on 16 cores): 3.1 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-07T09:29:14.165311
Stage halos : Wall: 0.32 s, CPU: 0.31 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 256 MB, VmData: 285 MB, maxrss: 0.262216 MB
Stage halos finished: 2020-08-07T09:29:14.488618
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Warning: Stellar halo subtraction is only implemented for DECam
Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.325 s, Cores in use: 0.96, Total efficiency (on 16 cores): 6.0 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-halos.p
Running stage fit_on_coadds
Warning: Stellar halo subtraction is only implemented for DECam
Running stage fit_on_coadds at 2020-08-07T09:29:14.510084
PSF sigmas in coadd pixels: ['2.51', '2.49', '1.91']
Using average PSF sigma 2.303189566421691
Gaussian PSF norm 0.12248005804061958 vs pixelized 0.10394329
average tim pixel scale / coadd scale: 1.729316258525746
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['2.08', '2.45', '2.07']
Using average PSF sigma 2.2004279633013373
Gaussian PSF norm 0.12819996676948553 vs pixelized 0.11320545
average tim pixel scale / coadd scale: 1.7294686204106866
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.80', '1.34', '1.40']
Using average PSF sigma 1.8484345064832688
Gaussian PSF norm 0.15261281413241762 vs pixelized 0.14565487
average tim pixel scale / coadd scale: 0.9909642656788742
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.23 s, CPU: 0.13 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 258 MB, VmData: 285 MB, maxrss: 0.263712 MB
Stage fit_on_coadds finished: 2020-08-07T09:29:14.739075
Resources for stage fit_on_coadds : 0.155 s worker CPU, 0.155 s worker Wall, Wall: 0.230 s, Cores in use: 1.26, Total efficiency (on 16 cores): 7.9 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-07T09:29:14.825820
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 34 potential peaks
Of 34 potential peaks: 9 in veto map, 4 cut by saddle test, 0 cut by aper test, 21 kept
SED z : 21 new peaks
Found 28 potential peaks
Of 28 potential peaks: 26 in veto map, 1 cut by saddle test, 0 cut by aper test, 1 kept
SED r : 1 new peaks
Found 25 potential peaks
Of 25 potential peaks: 25 in veto map, 0 cut by saddle test, 0 cut by aper test, 0 kept
SED g : 0 new peaks
Found 32 potential peaks
Of 32 potential peaks: 29 in veto map, 2 cut by saddle test, 0 cut by aper test, 1 kept
SED Flat : 1 new peaks
Found 32 potential peaks
Of 32 potential peaks: 30 in veto map, 2 cut by saddle test, 0 cut by aper test, 0 kept
SED Red : 0 new peaks
Adding 23 new sources
Median coadd sky for g : 7.534737e-05
Median coadd sky for r : 9.496947e-05
Median coadd sky for z : -0.00023483715
Stage srcs : Wall: 0.46 s, CPU: 0.42 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 258 MB, VmData: 285 MB, maxrss: 0.2642 MB
Stage srcs finished: 2020-08-07T09:29:15.283364
Resources for stage srcs : 0.073 s worker CPU, 0.073 s worker Wall, Wall: 0.459 s, Cores in use: 1.09, Total efficiency (on 16 cores): 6.8 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-07T09:29:15.312509
Blob 1 of 10, id: 2, sources: 23, size: 227x224, npix 22153, brick X: 0,227, Y: 25,249, one pixel: 135 25
Blob 2 of 10, id: 7, sources: 1, size: 30x53, npix 1222, brick X: 0,30, Y: 149,202, one pixel: 6 149
Blob 3 of 10, id: 8, sources: 3, size: 48x43, npix 1203, brick X: 203,251, Y: 174,217, one pixel: 246 174
Blob 4 of 10, id: 5, sources: 2, size: 30x42, npix 985, brick X: 221,251, Y: 66,108, one pixel: 233 66
Blob 5 of 10, id: 0, sources: 1, size: 32x30, npix 638, brick X: 72,104, Y: 8,38, one pixel: 87 8
Blob 6 of 10, id: 6, sources: 1, size: 27x28, npix 479, brick X: 39,66, Y: 127,155, one pixel: 51 127
Blob 7 of 10, id: 1, sources: 1, size: 22x22, npix 303, brick X: 220,242, Y: 12,34, one pixel: 228 12
Blob 8 of 10, id: 4, sources: 1, size: 14x20, npix 196, brick X: 0,14, Y: 42,62, one pixel: 3 42
Blob 9 of 10, id: 3, sources: 1, size: 13x19, npix 151, brick X: 238,251, Y: 36,55, one pixel: 247 36
Blob 10 of 10, id: 9, sources: 1, size: 17x12, npix 120, brick X: 1,18, Y: 239,251, one pixel: 9 239
Blob 10 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.14594 MB
Blob 7 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145852 MB
Blob 6 finished, total: Wall: 0.40 s, CPU: 0.39 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 139 MB, VmData: 182 MB, maxrss: 0.14238 MB
Blob 9 finished, total: Wall: 0.42 s, CPU: 0.42 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092488 MB
Blob 8 finished, total: Wall: 0.49 s, CPU: 0.49 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146064 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (281.90880, 57.24275), pm (-3.7, -6.2), parallax 0.998 with NanoMaggies: g=15.4, r=14.8, z=14.6
Reset brightness to NanoMaggies: g=15.4, r=14.8, z=14.6
Blob 4 finished, total: Wall: 0.64 s, CPU: 0.64 s, VmPeak: 805 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 182 MB, maxrss: 0.146544 MB
Blob 5 finished, total: Wall: 0.65 s, CPU: 0.65 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146796 MB
Measuring 3 iterative sources
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (281.91781, 57.25019), pm (-1.3, 1.4), parallax 0.008 with NanoMaggies: g=20.9, r=20.6, z=20.5
Reset brightness to NanoMaggies: g=20.7, r=20.6, z=20.8
Blob 3 finished, total: Wall: 1.18 s, CPU: 1.17 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.095412 MB
Blob 2 finished, total: Wall: 1.99 s, CPU: 1.98 s, VmPeak: 811 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 183 MB, maxrss: 0.147176 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.2915, ee1=0.0359941, ee2=-0.467317
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (281.95477, 57.24286), pm (6.6, -2.5), parallax 1.122 with NanoMaggies: g=15.7, r=(flux -705), z=15.7
Reset brightness to NanoMaggies: g=21.2, r=19.8, z=18.8
Measuring 10 iterative sources
Blob 1 finished, total: Wall: 17.18 s, CPU: 17.17 s, VmPeak: 705 MB, VmSize: 704 MB, VmRSS: 134 MB, VmData: 172 MB, maxrss: 0.137092 MB
Old catalog: 40
New catalog: 45
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/blobs-custom-281934p57245.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/metrics/cus/all-models-custom-281934p57245.fits
Stage fitblobs : Wall: 17.64 s, CPU: 0.37 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 258 MB, VmData: 285 MB, maxrss: 0.264592 MB
Stage fitblobs finished: 2020-08-07T09:29:32.949847
Resources for stage fitblobs : 23.624 s worker CPU, 23.727 s worker Wall, Wall: 17.639 s, Cores in use: 1.36, Total efficiency (on 16 cores): 8.5 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-07T09:29:32.997152
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/tmp-legacysurvey-custom-281934p57245-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/tmp-legacysurvey-custom-281934p57245-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/tmp-legacysurvey-custom-281934p57245-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/tmp-legacysurvey-custom-281934p57245-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-resid.jpg
Stage coadds : Wall: 16.24 s, CPU: 0.59 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 259 MB, VmData: 285 MB, maxrss: 0.264916 MB
Stage coadds finished: 2020-08-07T09:29:49.241311
Resources for stage coadds : 8.386 s worker CPU, 8.780 s worker Wall, Wall: 16.245 s, Cores in use: 0.55, Total efficiency (on 16 cores): 3.5 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-07T09:29:49.276572
Cut to 1 unWISE tiles
Reading WISE tile 2824p575 band 1
Reading WISE tile 2824p575 band 2
Reading WISE tile 2824p575 band 4
Reading WISE tile 2824p575 band 3
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.07 s, VmPeak: 814 MB, VmSize: 793 MB, VmRSS: 138 MB, VmData: 171 MB, maxrss: 0.196032 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 811 MB, VmSize: 799 MB, VmRSS: 142 MB, VmData: 173 MB, maxrss: 0.205632 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 805 MB, VmSize: 796 MB, VmRSS: 140 MB, VmData: 174 MB, maxrss: 0.202444 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/282/2824p575/unwise-2824p575-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 811 MB, VmSize: 801 MB, VmRSS: 146 MB, VmData: 175 MB, maxrss: 0.188828 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/tmp-legacysurvey-custom-281934p57245-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/tmp-legacysurvey-custom-281934p57245-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 22.69 s, CPU: 0.50 s, VmPeak: 1028 MB, VmSize: 1028 MB, VmRSS: 259 MB, VmData: 285 MB, maxrss: 0.265428 MB
Stage wise_forced finished: 2020-08-07T09:30:11.970353
Resources for stage wise_forced : 3.136 s worker CPU, 32.318 s worker Wall, Wall: 22.695 s, Cores in use: 0.16, Total efficiency (on 16 cores): 1.0 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-07T09:30:11.998831
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/coadd/cus/custom-281934p57245/legacysurvey-custom-281934p57245-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/tractor-i/cus/tractor-custom-281934p57245.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/tractor/cus/tractor-custom-281934p57245.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/tractor/cus/brick-custom-281934p57245.sha256sum
Stage writecat : Wall: 14.21 s, CPU: 0.70 s, VmPeak: 1160 MB, VmSize: 1032 MB, VmRSS: 261 MB, VmData: 285 MB, maxrss: 0.398312 MB
Stage writecat finished: 2020-08-07T09:30:26.211317
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 14.214 s, Cores in use: 0.05, Total efficiency (on 16 cores): 0.3 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/281/PGC2561020/PGC2561020-largegalaxy-writecat.p
All done: 41.537 s worker CPU, 474.308 s worker Wall, Wall: 174.471 s, Cores in use: 0.29, Total efficiency (on 16 cores): 1.8 %
Finished galaxy PGC2561020 in 2.993 minutes.