Started working on galaxy PGC2693994 at Sun Jun 7 18:33:15 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 122.83329149999999 67.01872220000001 --width 251 --height 251 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-06-07T18:33:16.839553 legacypipe git version: DR9.5.2 Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '122.83329149999999', '67.01872220000001', '--width', '251', '--height', '251', '--pixscale', '0.262', '--threads', '4', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 122.83329149999999 67.01872220000001 --width 251 --height 251 --pixscale 0.262 --threads 4 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 122.83329149999999 67.01872220000001 Starting process 31391 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057528 MB Starting process 31392 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057528 MB Starting process 31393 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057592 MB Starting process 31395 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057532 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-06-07T18:33:17.272091 Keeping 10 CCDs: k4m_170309_050336_ooi_zd_ls9.fits.fz k4m_170309_050336_ooi_zd_ls9-CCD2 z exptime 129.104 propid 2016A-0453 seeing 1.03 MJD 57821.211 object MzLS_706532_z k4m_160218_061055_ooi_zd_ls9.fits.fz k4m_160218_061055_ooi_zd_ls9-CCD1 z exptime 115.101 propid 2016A-0453 seeing 1.09 MJD 57436.258 object MzLS_707027_z k4m_170420_031651_ooi_zd_ls9.fits.fz k4m_170420_031651_ooi_zd_ls9-CCD2 z exptime 164.103 propid 2016A-0453 seeing 1.27 MJD 57863.137 object MzLS_706034_z k4m_180128_062854_ooi_zd_ls9.fits.fz k4m_180128_062854_ooi_zd_ls9-CCD1 z exptime 89.104 propid 2016A-0453 seeing 1.13 MJD 58146.270 object MzLS_707027_z ksb_190117_110131_ooi_g_ls9.fits.fz ksb_190117_110131_ooi_g_ls9-CCD3 g exptime 105.421 propid NOTBASS seeing 1.31 MJD 58500.459 object ksb_190130_084345_ooi_r_ls9.fits.fz ksb_190130_084345_ooi_r_ls9-CCD3 r exptime 73.121 propid NOTBASS seeing 1.45 MJD 58513.364 object ksb_190107_113426_ooi_g_ls9.fits.fz ksb_190107_113426_ooi_g_ls9-CCD3 g exptime 94.208 propid NOTBASS seeing 1.37 MJD 58490.482 object ksb_190208_050709_ooi_r_ls9.fits.fz ksb_190208_050709_ooi_r_ls9-CCD3 r exptime 93.573 propid NOTBASS seeing 1.66 MJD 58522.213 object ksb_180411_045447_ooi_g_ls9.fits.fz ksb_180411_045447_ooi_g_ls9-CCD2 g exptime 209.506 propid BASS seeing 1.65 MJD 58219.205 object 43392 ksb_190211_051720_ooi_r_ls9.fits.fz ksb_190211_051720_ooi_r_ls9-CCD2 r exptime 122.844 propid NOTBASS seeing 1.62 MJD 58525.220 object mosaic get_fwhm: self.fwhm = 4.35 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2010 2265 and amp range 2006 4053 and image y 2118 2375 and amp range 2040 4078 Range in image: x 0 256 , y 0 258 (with image size 256 x 258) dzp 0.0036595785 -> scaling image by 1.003376283213225 mosaic get_fwhm: self.fwhm = 3.95 Found 4 relevant rows in amp-corrections file. Found overlap: image x 430 683 and amp range 0 2047 and image y 2863 3116 and amp range 2040 4078 Range in image: x 0 254 , y 0 254 (with image size 254 x 254) dzp 0.015739892 -> scaling image by 1.0146025667655956 mosaic get_fwhm: self.fwhm = 4.21 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2017 2272 and amp range 2006 4053 and image y 2153 2410 and amp range 2040 4078 Range in image: x 0 256 , y 0 258 (with image size 256 x 258) dzp -0.006092005 -> scaling image by 0.9944047680378836 Faked up EXPNUM 85000191 Faked up EXPNUM 85000191 Faked up EXPNUM 85000191 90prime get_fwhm: self.fwhm = 2.89 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1935 2081 and amp range 0 2015 and image y 698 844 and amp range 0 2047 Range in image: x 0 81 , y 0 147 (with image size 147 x 147) dzp -0.032927223 -> scaling image by 0.9701281597885938 Found overlap: image x 1935 2081 and amp range 2016 4031 and image y 698 844 and amp range 0 2047 Range in image: x 81 147 , y 0 147 (with image size 147 x 147) dzp -0.0041084206 -> scaling image by 0.9962231551010422 Faked up EXPNUM 85130065 Faked up EXPNUM 85130065 Faked up EXPNUM 85130065 90prime get_fwhm: self.fwhm = 3.19 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1937 2083 and amp range 0 2015 and image y 690 836 and amp range 0 2047 Range in image: x 0 79 , y 0 147 (with image size 147 x 147) dzp -0.040645268 -> scaling image by 0.9632563770903896 Found overlap: image x 1937 2083 and amp range 2016 4031 and image y 690 836 and amp range 0 2047 Range in image: x 79 147 , y 0 147 (with image size 147 x 147) dzp 0.0010388457 -> scaling image by 1.0009572701813714 Faked up EXPNUM 84900052 Faked up EXPNUM 84900052 Faked up EXPNUM 84900052 90prime get_fwhm: self.fwhm = 3.01 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3892 4016 and amp range 2016 4031 and image y 2687 2835 and amp range 2048 4095 Range in image: x 0 125 , y 0 149 (with image size 125 x 149) dzp -0.021718625 -> scaling image by 0.9801951516091928 mosaic get_fwhm: self.fwhm = 4.89 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3125 3382 and amp range 2048 4095 and image y 3956 4072 and amp range 2040 4078 Range in image: x 0 258 , y 0 117 (with image size 258 x 117) dzp -0.032976374 -> scaling image by 0.9700842429001929 Faked up EXPNUM 82190061 Faked up EXPNUM 82190061 Faked up EXPNUM 82190061 90prime get_fwhm: self.fwhm = 3.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1659 1807 and amp range 0 2015 and image y 1181 1330 and amp range 0 2047 Range in image: x 0 149 , y 0 150 (with image size 149 x 150) dzp -0.0036920775 -> scaling image by 0.9966052462302274 Faked up EXPNUM 85220109 Faked up EXPNUM 85220109 Faked up EXPNUM 85220109 90prime get_fwhm: self.fwhm = 3.65 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3881 4016 and amp range 2016 4031 and image y 2648 2795 and amp range 2048 4095 Range in image: x 0 136 , y 0 148 (with image size 136 x 148) dzp -0.016491456 -> scaling image by 0.984925581678532 Faked up EXPNUM 85250111 Faked up EXPNUM 85250111 Faked up EXPNUM 85250111 90prime get_fwhm: self.fwhm = 3.57 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1663 1810 and amp range 0 2015 and image y 1168 1317 and amp range 0 2047 Range in image: x 0 148 , y 0 150 (with image size 148 x 150) dzp 0.0012875209 -> scaling image by 1.001186553937296 Stage tims : Wall: 7.87 s, CPU: 0.96 s, VmPeak: 942 MB, VmSize: 942 MB, VmRSS: 171 MB, VmData: 205 MB, maxrss: 0.566516 MB Stage tims finished: 2020-06-07T18:33:25.145747 Resources for stage tims : 3.568 s worker CPU, 17.632 s worker Wall, Wall: 7.875 s, Cores in use: 0.57, Total efficiency (on 4 cores): 14.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-tims.p Running stage refs Running stage refs at 2020-06-07T18:33:25.242097 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00440.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00501.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-00502.fits Matched 4 large galaxies to Gaia stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/reference-custom-122833p67018.fits Stage refs : Wall: 3.89 s, CPU: 2.32 s, VmPeak: 972 MB, VmSize: 964 MB, VmRSS: 194 MB, VmData: 227 MB, maxrss: 0.566516 MB Stage refs finished: 2020-06-07T18:33:29.130404 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 3.889 s, Cores in use: 0.60, Total efficiency (on 4 cores): 14.9 % Stage refs finished Running stage outliers Running stage outliers at 2020-06-07T18:33:29.131914 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/outlier-mask-custom-122833p67018.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/outliers-pre-custom-122833p67018.jpg Image ksb_190107_113426_ooi_g_ls9-CCD3 g : masking 83 positive outlier pixels and 0 negative outlier pixels Image k4m_170420_031651_ooi_zd_ls9-CCD2 z : masking 1783 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/outlier-mask-custom-122833p67018.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/outliers-post-custom-122833p67018.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/outliers-masked-pos-custom-122833p67018.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/outliers-masked-neg-custom-122833p67018.jpg Stage outliers : Wall: 0.80 s, CPU: 0.60 s, VmPeak: 973 MB, VmSize: 973 MB, VmRSS: 207 MB, VmData: 238 MB, maxrss: 0.566516 MB Stage outliers finished: 2020-06-07T18:33:29.936273 Resources for stage outliers : 0.761 s worker CPU, 0.762 s worker Wall, Wall: 0.805 s, Cores in use: 1.69, Total efficiency (on 4 cores): 42.1 % Stage outliers finished Running stage halos Running stage halos at 2020-06-07T18:33:29.938012 Stage halos : Wall: 0.31 s, CPU: 0.30 s, VmPeak: 973 MB, VmSize: 973 MB, VmRSS: 207 MB, VmData: 238 MB, maxrss: 0.566516 MB Stage halos finished: 2020-06-07T18:33:30.252929 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 0.316 s, Cores in use: 0.96, Total efficiency (on 4 cores): 24.0 % Stage halos finished Running stage fit_on_coadds Running stage fit_on_coadds at 2020-06-07T18:33:30.254852 PSF sigmas in coadd pixels: ['2.13', '2.22', '2.69'] Using average PSF sigma 2.344592595494156 Gaussian PSF norm 0.12031718956888657 vs pixelized 0.09845405 average tim pixel scale / coadd scale: 1.730852074323947 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.35', '2.69', '2.63'] Using average PSF sigma 2.556069879054853 Gaussian PSF norm 0.11036270725047125 vs pixelized 0.091875374 average tim pixel scale / coadd scale: 1.7310496249044214 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.67', '1.78', '2.06', '1.83'] Using average PSF sigma 1.834233882033383 Gaussian PSF norm 0.15379434135256262 vs pixelized 0.12603669 average tim pixel scale / coadd scale: 0.9909075382032311 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.23 s, CPU: 0.17 s, VmPeak: 973 MB, VmSize: 973 MB, VmRSS: 210 MB, VmData: 240 MB, maxrss: 0.566516 MB Stage fit_on_coadds finished: 2020-06-07T18:33:30.480035 Resources for stage fit_on_coadds : 0.185 s worker CPU, 0.186 s worker Wall, Wall: 0.226 s, Cores in use: 1.57, Total efficiency (on 4 cores): 39.2 % Stage fit_on_coadds finished Running stage srcs Running stage srcs at 2020-06-07T18:33:30.482815 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 30 potential peaks Of 30 potential peaks: 9 in veto map, 3 cut by saddle test, 0 cut by aper test, 18 kept SED z : 18 new peaks Found 38 potential peaks Of 38 potential peaks: 32 in veto map, 5 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 31 potential peaks Of 31 potential peaks: 27 in veto map, 4 cut by saddle test, 0 cut by aper test, 0 kept SED g : 0 new peaks Found 41 potential peaks Of 41 potential peaks: 34 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 44 potential peaks Of 44 potential peaks: 38 in veto map, 6 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 20 new sources Median coadd sky for g : 0.00026538706 Median coadd sky for r : 0.001001622 Median coadd sky for z : -0.0005673188 Stage srcs : Wall: 0.64 s, CPU: 0.59 s, VmPeak: 973 MB, VmSize: 973 MB, VmRSS: 211 MB, VmData: 240 MB, maxrss: 0.566516 MB Stage srcs finished: 2020-06-07T18:33:31.121171 Resources for stage srcs : 0.107 s worker CPU, 0.117 s worker Wall, Wall: 0.639 s, Cores in use: 1.10, Total efficiency (on 4 cores): 27.5 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/PGC2693994-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-06-07T18:33:31.133225 Blob 1 of 10, id: 0, sources: 9, size: 208x221, npix 20769, brick X: 8,216, Y: 0,221, one pixel: 12 0 Blob 2 of 10, id: 2, sources: 3, size: 50x39, npix 1065, brick X: 146,196, Y: 29,68, one pixel: 180 29 Blob 3 of 10, id: 1, sources: 2, size: 38x42, npix 1031, brick X: 213,251, Y: 25,67, one pixel: 240 25 Blob 4 of 10, id: 3, sources: 1, size: 39x39, npix 1012, brick X: 175,214, Y: 52,91, one pixel: 193 52 Blob 5 of 10, id: 8, sources: 1, size: 39x31, npix 892, brick X: 26,65, Y: 220,251, one pixel: 41 220 Blob 4 finished, total: Wall: 0.41 s, CPU: 0.41 s, VmPeak: 793 MB, VmSize: 713 MB, VmRSS: 136 MB, VmData: 178 MB, maxrss: 0.139752 MB Blob 6 of 10, id: 9, sources: 1, size: 34x22, npix 570, brick X: 212,246, Y: 229,251, one pixel: 226 229 Blob 5 finished, total: Wall: 0.44 s, CPU: 0.44 s, VmPeak: 793 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141636 MB Blob 7 of 10, id: 6, sources: 1, size: 18x33, npix 428, brick X: 233,251, Y: 169,202, one pixel: 246 169 Blob 8 of 10, id: 7, sources: 1, size: 23x26, npix 422, brick X: 0,23, Y: 186,212, one pixel: 6 186 Blob 2 finished, total: Wall: 1.84 s, CPU: 1.59 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.142896 MB Blob 3 finished, total: Wall: 2.03 s, CPU: 1.73 s, VmPeak: 806 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141596 MB Blob 9 of 10, id: 5, sources: 1, size: 25x25, npix 367, brick X: 48,73, Y: 155,180, one pixel: 58 155 Blob 6 finished, total: Wall: 1.34 s, CPU: 1.34 s, VmPeak: 793 MB, VmSize: 713 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141636 MB Blob 10 of 10, id: 4, sources: 2, size: 14x25, npix 258, brick X: 237,251, Y: 56,81, one pixel: 246 56 Blob 8 finished, total: Wall: 0.47 s, CPU: 0.47 s, VmPeak: 806 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142216 MB Blob 9 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 793 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142508 MB Blob 7 finished, total: Wall: 0.94 s, CPU: 0.94 s, VmPeak: 805 MB, VmSize: 713 MB, VmRSS: 140 MB, VmData: 178 MB, maxrss: 0.142912 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (122.80981, 67.01458), pm (0.0, 0.0), parallax 0.000 with NanoMaggies: g=23.1, r=21.4, z=20 Reset brightness to NanoMaggies: g=22.2, r=21.2, z=20.7 Blob 10 finished, total: Wall: 0.46 s, CPU: 0.46 s, VmPeak: 806 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142216 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=0.935407, ee1=0.0277229, ee2=-0.18153 Measuring 7 iterative sources Blob 1 finished, total: Wall: 14.53 s, CPU: 14.35 s, VmPeak: 806 MB, VmSize: 721 MB, VmRSS: 154 MB, VmData: 186 MB, maxrss: 0.157528 MB Old catalog: 26 New catalog: 27 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/blobs-custom-122833p67018.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/metrics/cus/all-models-custom-122833p67018.fits Stage fitblobs : Wall: 14.87 s, CPU: 0.32 s, VmPeak: 973 MB, VmSize: 973 MB, VmRSS: 212 MB, VmData: 240 MB, maxrss: 0.566516 MB Stage fitblobs finished: 2020-06-07T18:33:46.005592 Resources for stage fitblobs : 22.399 s worker CPU, 23.157 s worker Wall, Wall: 14.874 s, Cores in use: 1.53, Total efficiency (on 4 cores): 38.2 % Stage fitblobs finished Running stage coadds Running stage coadds at 2020-06-07T18:33:46.007874 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/tmp-legacysurvey-custom-122833p67018-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/tmp-legacysurvey-custom-122833p67018-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/tmp-legacysurvey-custom-122833p67018-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/tmp-legacysurvey-custom-122833p67018-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-resid.jpg Stage coadds : Wall: 1.28 s, CPU: 0.49 s, VmPeak: 973 MB, VmSize: 973 MB, VmRSS: 212 MB, VmData: 240 MB, maxrss: 0.566516 MB Stage coadds finished: 2020-06-07T18:33:47.287720 Resources for stage coadds : 1.974 s worker CPU, 2.114 s worker Wall, Wall: 1.281 s, Cores in use: 1.93, Total efficiency (on 4 cores): 48.2 % Stage coadds finished Running stage wise_forced Running stage wise_forced at 2020-06-07T18:33:47.289172 Cut to 1 unWISE tiles Reading WISE tile 1224p666 band 2 Reading WISE tile 1224p666 band 1 Reading WISE tile 1224p666 band 3 Reading WISE tile 1224p666 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.08 s, CPU: 0.07 s, VmPeak: 806 MB, VmSize: 794 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.189192 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 853 MB, VmSize: 853 MB, VmRSS: 205 MB, VmData: 234 MB, maxrss: 0.209652 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 806 MB, VmSize: 792 MB, VmRSS: 148 MB, VmData: 173 MB, maxrss: 0.213064 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/122/1224p666/unwise-1224p666-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 794 MB, VmSize: 792 MB, VmRSS: 144 MB, VmData: 173 MB, maxrss: 0.208748 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/tmp-legacysurvey-custom-122833p67018-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/tmp-legacysurvey-custom-122833p67018-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.30 s, CPU: 0.44 s, VmPeak: 973 MB, VmSize: 973 MB, VmRSS: 213 MB, VmData: 240 MB, maxrss: 0.566516 MB Stage wise_forced finished: 2020-06-07T18:33:49.592618 Resources for stage wise_forced : 4.456 s worker CPU, 6.069 s worker Wall, Wall: 2.305 s, Cores in use: 2.13, Total efficiency (on 4 cores): 53.2 % Stage wise_forced finished Running stage writecat Running stage writecat at 2020-06-07T18:33:49.594448 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/coadd/cus/custom-122833p67018/legacysurvey-custom-122833p67018-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/tractor-i/cus/tractor-custom-122833p67018.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/tractor/cus/tractor-custom-122833p67018.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/122/PGC2693994/tractor/cus/brick-custom-122833p67018.sha256sum Stage writecat : Wall: 1.10 s, CPU: 0.51 s, VmPeak: 1106 MB, VmSize: 978 MB, VmRSS: 215 MB, VmData: 240 MB, maxrss: 0.566516 MB Stage writecat finished: 2020-06-07T18:33:50.699481 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.106 s, Cores in use: 0.46, Total efficiency (on 4 cores): 11.5 % Stage writecat finished All done: 33.451 s worker CPU, 50.040 s worker Wall, Wall: 33.433 s, Cores in use: 1.20, Total efficiency (on 4 cores): 30.1 % Finished galaxy PGC2693994 in 0.601 minutes.