Started working on galaxy PGC2395898 at Wed Jun 10 18:33:17 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 94.44210000000001 51.548302 --width 245 --height 245 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-06-10T18:33:17.896282 legacypipe git version: DR9.5.2-11-g917b309b Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '94.44210000000001', '51.548302', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 94.44210000000001 51.548302 --width 245 --height 245 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 94.44210000000001 51.548302 Starting process 6004 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB Starting process 6005 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057308 MB Starting process 6006 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057312 MB Starting process 6007 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057312 MB Starting process 6008 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.057316 MB Starting process 6009 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.05732 MB Starting process 6010 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.05732 MB Starting process 6011 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 632 MB, VmSize: 632 MB, VmRSS: 56 MB, VmData: 127 MB, maxrss: 0.05732 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-06-10T18:33:18.205378 Keeping 8 CCDs: k4m_180209_072450_ooi_zd_ls9.fits.fz k4m_180209_072450_ooi_zd_ls9-CCD3 z exptime 100.107 propid 2016A-0453 seeing 0.89 MJD 58158.309 object MzLS_656468_z k4m_180119_035424_ooi_zd_ls9.fits.fz k4m_180119_035424_ooi_zd_ls9-CCD3 z exptime 163.105 propid 2016A-0453 seeing 1.14 MJD 58137.163 object MzLS_656468_z k4m_170128_031236_ooi_zd_ls9.fits.fz k4m_170128_031236_ooi_zd_ls9-CCD2 z exptime 109.104 propid 2016A-0453 seeing 1.37 MJD 57781.134 object MzLS_655686_z k4m_170221_022457_ooi_zd_ls9.fits.fz k4m_170221_022457_ooi_zd_ls9-CCD2 z exptime 89.104 propid 2016A-0453 seeing 0.82 MJD 57805.101 object MzLS_654902_z ksb_171127_054522_ooi_r_ls9.fits.fz ksb_171127_054522_ooi_r_ls9-CCD4 r exptime 195.6 propid BASS seeing 1.22 MJD 58084.240 object 63571 ksb_180113_062516_ooi_g_ls9.fits.fz ksb_180113_062516_ooi_g_ls9-CCD4 g exptime 233.753 propid BASS seeing 1.39 MJD 58131.268 object 63571 ksb_160104_050151_ooi_g_ls9.fits.fz ksb_160104_050151_ooi_g_ls9-CCD2 g exptime 250.0 propid 2015A-0801 seeing 2.40 MJD 57391.210 object 63573 ksb_181212_113230_ooi_r_ls9.fits.fz ksb_181212_113230_ooi_r_ls9-CCD2 r exptime 250.0 propid NOTBASS seeing 1.38 MJD 58464.481 object mosaic get_fwhm: self.fwhm = 5.27 Found 4 relevant rows in amp-corrections file. Found overlap: image x 33 280 and amp range 0 2047 and image y 1690 1939 and amp range 0 2039 Range in image: x 0 248 , y 0 250 (with image size 248 x 250) dzp -0.0096683735 -> scaling image by 0.9911346301237048 mosaic get_fwhm: self.fwhm = 3.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1630 1878 and amp range 0 2005 and image y 385 631 and amp range 0 2039 Range in image: x 0 249 , y 0 247 (with image size 249 x 247) dzp 0.0014252509 -> scaling image by 1.0013135665633692 mosaic get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1627 1875 and amp range 0 2005 and image y 416 662 and amp range 0 2039 Range in image: x 0 249 , y 0 247 (with image size 249 x 247) dzp 0.0014252509 -> scaling image by 1.0013135665633692 Faked up EXPNUM 81310058 Faked up EXPNUM 81310058 Faked up EXPNUM 81310058 90prime get_fwhm: self.fwhm = 3.07 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3468 3611 and amp range 2016 4031 and image y 1454 1597 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp 0.023266561 -> scaling image by 1.0216605511853067 mosaic get_fwhm: self.fwhm = 3.15 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2747 2998 and amp range 2048 4095 and image y 2875 3122 and amp range 2040 4078 Range in image: x 0 252 , y 0 248 (with image size 252 x 248) dzp -0.032976374 -> scaling image by 0.9700842429001929 Faked up EXPNUM 84640044 Faked up EXPNUM 84640044 Faked up EXPNUM 84640044 90prime get_fwhm: self.fwhm = 3.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1448 1590 and amp range 0 2015 and image y 3876 4018 and amp range 2048 4095 Range in image: x 0 143 , y 0 143 (with image size 143 x 143) dzp 0.049663078 -> scaling image by 1.0468036570994166 Faked up EXPNUM 73910077 Faked up EXPNUM 73910077 Faked up EXPNUM 73910077 90prime get_fwhm: self.fwhm = 5.29 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1449 1591 and amp range 0 2015 and image y 3922 4064 and amp range 2048 4095 Range in image: x 0 143 , y 0 143 (with image size 143 x 143) dzp 0.01693853 -> scaling image by 1.015723292931903 Faked up EXPNUM 80850097 Faked up EXPNUM 80850097 Faked up EXPNUM 80850097 90prime get_fwhm: self.fwhm = 2.69 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3472 3615 and amp range 2016 4031 and image y 1452 1595 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp 0.043530058 -> scaling image by 1.0409072255640852 Stage tims : Wall: 2.95 s, CPU: 0.72 s, VmPeak: 941 MB, VmSize: 941 MB, VmRSS: 170 MB, VmData: 204 MB, maxrss: 0.245444 MB Stage tims finished: 2020-06-10T18:33:21.152417 Resources for stage tims : 5.183 s worker CPU, 13.432 s worker Wall, Wall: 2.948 s, Cores in use: 2.00, Total efficiency (on 8 cores): 25.0 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-tims.p Running stage refs Running stage refs at 2020-06-10T18:33:21.167749 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01432.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01327.fits Matched 17 large galaxies to Gaia stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/reference-custom-094442p51548.fits Stage refs : Wall: 5.02 s, CPU: 4.58 s, VmPeak: 1007 MB, VmSize: 1001 MB, VmRSS: 233 MB, VmData: 264 MB, maxrss: 0.245444 MB Stage refs finished: 2020-06-10T18:33:26.191397 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 5.025 s, Cores in use: 0.91, Total efficiency (on 8 cores): 11.4 % Stage refs finished Running stage outliers Running stage outliers at 2020-06-10T18:33:26.193263 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outlier-mask-custom-094442p51548.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-pre-custom-094442p51548.jpg Image ksb_160104_050151_ooi_g_ls9-CCD2 g : masking 151 positive outlier pixels and 0 negative outlier pixels Image ksb_181212_113230_ooi_r_ls9-CCD2 r : masking 105 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outlier-mask-custom-094442p51548.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-post-custom-094442p51548.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-masked-pos-custom-094442p51548.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-masked-neg-custom-094442p51548.jpg Stage outliers : Wall: 0.60 s, CPU: 0.42 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 245 MB, VmData: 275 MB, maxrss: 0.250924 MB Stage outliers finished: 2020-06-10T18:33:26.795621 Resources for stage outliers : 0.530 s worker CPU, 0.532 s worker Wall, Wall: 0.604 s, Cores in use: 1.57, Total efficiency (on 8 cores): 19.6 % Stage outliers finished Running stage halos Running stage halos at 2020-06-10T18:33:26.797924 Stage halos : Wall: 0.11 s, CPU: 0.10 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 245 MB, VmData: 275 MB, maxrss: 0.251052 MB Stage halos finished: 2020-06-10T18:33:26.904488 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.108 s, Cores in use: 0.94, Total efficiency (on 8 cores): 11.7 % Stage halos finished Running stage fit_on_coadds Warning: Stellar halo subtraction is only implemented for DECam Running stage fit_on_coadds at 2020-06-10T18:33:26.906325 PSF sigmas in coadd pixels: ['2.26', '3.89'] Using average PSF sigma 3.07805462586082 Gaussian PSF norm 0.09164710379205389 vs pixelized 0.08706361 average tim pixel scale / coadd scale: 1.7306418602447244 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.98', '2.25'] Using average PSF sigma 2.1140485502069972 Gaussian PSF norm 0.13343818037965913 vs pixelized 0.1053732 average tim pixel scale / coadd scale: 1.7310027699590524 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.44', '1.86', '2.22', '1.33'] Using average PSF sigma 1.7118592919444549 Gaussian PSF norm 0.1647885390471867 vs pixelized 0.12621099 average tim pixel scale / coadd scale: 0.9908473532591797 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.14 s, CPU: 0.13 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 247 MB, VmData: 275 MB, maxrss: 0.25268 MB Stage fit_on_coadds finished: 2020-06-10T18:33:27.042536 Resources for stage fit_on_coadds : 0.114 s worker CPU, 0.116 s worker Wall, Wall: 0.137 s, Cores in use: 1.75, Total efficiency (on 8 cores): 21.8 % Stage fit_on_coadds finished Running stage srcs Running stage srcs at 2020-06-10T18:33:27.045215 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 29 potential peaks Of 29 potential peaks: 5 in veto map, 3 cut by saddle test, 0 cut by aper test, 21 kept SED z : 21 new peaks Found 33 potential peaks Of 33 potential peaks: 27 in veto map, 5 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 26 potential peaks Of 26 potential peaks: 20 in veto map, 3 cut by saddle test, 0 cut by aper test, 3 kept SED g : 3 new peaks Found 41 potential peaks Of 41 potential peaks: 35 in veto map, 6 cut by saddle test, 0 cut by aper test, 0 kept SED Flat : 0 new peaks Found 40 potential peaks Of 40 potential peaks: 32 in veto map, 6 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 27 new sources Median coadd sky for g : 0.00045767945 Median coadd sky for r : 0.0005858192 Median coadd sky for z : -0.0003010308 Stage srcs : Wall: 0.38 s, CPU: 0.35 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 247 MB, VmData: 275 MB, maxrss: 0.25298 MB Stage srcs finished: 2020-06-10T18:33:27.428576 Resources for stage srcs : 0.084 s worker CPU, 0.084 s worker Wall, Wall: 0.384 s, Cores in use: 1.12, Total efficiency (on 8 cores): 14.1 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-06-10T18:33:27.439411 Blob 1 of 15, id: 4, sources: 11, size: 150x181, npix 15141, brick X: 32,182, Y: 30,211, one pixel: 130 30 Blob 2 of 15, id: 11, sources: 4, size: 78x61, npix 2438, brick X: 157,235, Y: 184,245, one pixel: 208 184 Blob 3 of 15, id: 7, sources: 2, size: 49x50, npix 1765, brick X: 166,215, Y: 103,153, one pixel: 190 103 Blob 4 of 15, id: 1, sources: 3, size: 49x37, npix 1087, brick X: 87,136, Y: 0,37, one pixel: 109 0 Blob 5 of 15, id: 0, sources: 3, size: 43x34, npix 1039, brick X: 0,43, Y: 0,34, one pixel: 0 0 Blob 6 of 15, id: 13, sources: 3, size: 44x37, npix 979, brick X: 29,73, Y: 202,239, one pixel: 51 202 Blob 7 of 15, id: 3, sources: 2, size: 39x42, npix 915, brick X: 47,86, Y: 7,49, one pixel: 71 7 Blob 8 of 15, id: 6, sources: 2, size: 28x40, npix 757, brick X: 0,28, Y: 83,123, one pixel: 13 83 Blob 9 of 15, id: 9, sources: 1, size: 24x30, npix 522, brick X: 221,245, Y: 132,162, one pixel: 235 132 Blob 8 finished, total: Wall: 1.11 s, CPU: 1.11 s, VmPeak: 794 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141276 MB Blob 10 of 15, id: 5, sources: 1, size: 27x27, npix 470, brick X: 210,237, Y: 75,102, one pixel: 221 75 Measuring 1 iterative sources Blob 7 finished, total: Wall: 1.62 s, CPU: 1.62 s, VmPeak: 795 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141288 MB Blob 11 of 15, id: 12, sources: 1, size: 25x24, npix 378, brick X: 169,194, Y: 186,210, one pixel: 178 186 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (94.44243, 51.53016), pm (2.1, -2.5), parallax 0.844 with NanoMaggies: g=15, r=14.5, z=14.4 Reset brightness to NanoMaggies: g=15, r=14.5, z=14.4 Blob 4 finished, total: Wall: 1.65 s, CPU: 1.65 s, VmPeak: 801 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141464 MB Blob 12 of 15, id: 8, sources: 1, size: 25x24, npix 366, brick X: 20,45, Y: 129,153, one pixel: 32 129 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (94.45632, 51.53799), pm (-5.5, -3.5), parallax 1.551 with NanoMaggies: g=14.4, r=13.9, z=14.4 Reset brightness to NanoMaggies: g=14.3, r=13.8, z=13.7 Blob 5 finished, total: Wall: 1.82 s, CPU: 1.82 s, VmPeak: 800 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141044 MB Blob 13 of 15, id: 10, sources: 1, size: 18x28, npix 354, brick X: 227,245, Y: 170,198, one pixel: 238 170 Blob 14 of 15, id: 2, sources: 1, size: 23x24, npix 339, brick X: 152,175, Y: 3,27, one pixel: 162 3 Blob 11 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 801 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141464 MB Blob 15 of 15, id: 14, sources: 1, size: 19x19, npix 190, brick X: 126,145, Y: 204,223, one pixel: 135 204 Blob 12 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 800 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141044 MB Blob 9 finished, total: Wall: 1.05 s, CPU: 1.05 s, VmPeak: 794 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141276 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (94.43184, 51.56311), pm (-7.6, 1.7), parallax 1.061 with NanoMaggies: g=11.6, r=10.7, z=10.3 Reset brightness to NanoMaggies: g=11.6, r=10.7, z=10.3 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (94.43239, 51.56102), pm (2.1, -1.0), parallax -0.123 with NanoMaggies: g=19.2, r=18.3, z=18 Reset brightness to NanoMaggies: g=19.2, r=18.3, z=18 Blob 2 finished, total: Wall: 2.22 s, CPU: 2.21 s, VmPeak: 801 MB, VmSize: 712 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142328 MB Blob 14 finished, total: Wall: 0.26 s, CPU: 0.26 s, VmPeak: 800 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141044 MB Blob 15 finished, total: Wall: 0.15 s, CPU: 0.15 s, VmPeak: 794 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141276 MB Blob 3 finished, total: Wall: 2.42 s, CPU: 2.42 s, VmPeak: 800 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141372 MB Blob 10 finished, total: Wall: 0.84 s, CPU: 0.84 s, VmPeak: 795 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141288 MB Blob 13 finished, total: Wall: 0.79 s, CPU: 0.79 s, VmPeak: 801 MB, VmSize: 712 MB, VmRSS: 138 MB, VmData: 178 MB, maxrss: 0.141464 MB Measuring 1 iterative sources Blob 6 finished, total: Wall: 2.87 s, CPU: 2.87 s, VmPeak: 794 MB, VmSize: 713 MB, VmRSS: 139 MB, VmData: 178 MB, maxrss: 0.142328 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=2.14824, ee1=0.393414, ee2=0.88339 Measuring 5 iterative sources Blob 1 finished, total: Wall: 13.71 s, CPU: 13.71 s, VmPeak: 797 MB, VmSize: 713 MB, VmRSS: 141 MB, VmData: 179 MB, maxrss: 0.144704 MB Old catalog: 46 New catalog: 38 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/blobs-custom-094442p51548.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/all-models-custom-094442p51548.fits Stage fitblobs : Wall: 13.96 s, CPU: 0.25 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 247 MB, VmData: 275 MB, maxrss: 0.253232 MB Stage fitblobs finished: 2020-06-10T18:33:41.394562 Resources for stage fitblobs : 31.204 s worker CPU, 31.431 s worker Wall, Wall: 13.956 s, Cores in use: 2.25, Total efficiency (on 8 cores): 28.2 % Stage fitblobs finished Running stage coadds Running stage coadds at 2020-06-10T18:33:41.396995 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-resid.jpg Stage coadds : Wall: 1.04 s, CPU: 0.45 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 248 MB, VmData: 275 MB, maxrss: 0.253748 MB Stage coadds finished: 2020-06-10T18:33:42.438607 Resources for stage coadds : 3.516 s worker CPU, 3.575 s worker Wall, Wall: 1.043 s, Cores in use: 3.81, Total efficiency (on 8 cores): 47.6 % Stage coadds finished Running stage wise_forced Running stage wise_forced at 2020-06-10T18:33:42.440716 Cut to 1 unWISE tiles Reading WISE tile 0953p514 band 1 Reading WISE tile 0953p514 band 2 Reading WISE tile 0953p514 band 3 Reading WISE tile 0953p514 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.08 s, CPU: 0.08 s, VmPeak: 794 MB, VmSize: 794 MB, VmRSS: 146 MB, VmData: 176 MB, maxrss: 0.187924 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.11 s, CPU: 0.11 s, VmPeak: 797 MB, VmSize: 794 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.211692 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 794 MB, VmSize: 788 MB, VmRSS: 140 MB, VmData: 170 MB, maxrss: 0.204852 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 800 MB, VmSize: 792 MB, VmRSS: 144 MB, VmData: 173 MB, maxrss: 0.208376 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.44 s, CPU: 0.35 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 248 MB, VmData: 275 MB, maxrss: 0.254328 MB Stage wise_forced finished: 2020-06-10T18:33:44.880873 Resources for stage wise_forced : 4.057 s worker CPU, 6.217 s worker Wall, Wall: 2.442 s, Cores in use: 1.81, Total efficiency (on 8 cores): 22.6 % Stage wise_forced finished Running stage writecat Running stage writecat at 2020-06-10T18:33:44.883173 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/tractor-i/cus/tractor-custom-094442p51548.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/tractor/cus/tractor-custom-094442p51548.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/tractor/cus/brick-custom-094442p51548.sha256sum Stage writecat : Wall: 0.55 s, CPU: 0.42 s, VmPeak: 1143 MB, VmSize: 1015 MB, VmRSS: 250 MB, VmData: 275 MB, maxrss: 0.387448 MB Stage writecat finished: 2020-06-10T18:33:45.433247 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.551 s, Cores in use: 0.76, Total efficiency (on 8 cores): 9.5 % Stage writecat finished All done: 44.691 s worker CPU, 55.391 s worker Wall, Wall: 27.233 s, Cores in use: 1.93, Total efficiency (on 8 cores): 24.1 % Finished galaxy PGC2395898 in 0.479 minutes. Started working on galaxy PGC2395898 at Fri Aug 7 09:22:48 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 94.44206932829498 51.548778203661406 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T09:22:49.550829 legacypipe git version: DR9.5.11-watershed-3-gd509eaaf Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '94.44206932829498', '51.548778203661406', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'north', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 94.44206932829498 51.548778203661406 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run north --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 94.44206932829498 51.548778203661406 Starting process 17935 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 17936 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 17937 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 17938 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 17939 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB Starting process 17940 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059552 MB Starting process 17941 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059552 MB Starting process 17942 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 17943 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 17944 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 17945 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 17946 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 17947 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 17948 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 17949 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 17950 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T09:22:50.037936 Keeping 8 CCDs: k4m_180209_072450_ooi_zd_ls9-CCD3 z expnum 161369 exptime 100.107 propid 2016A-0453 seeing 0.89 MJD 58158.309 object MzLS_656468_z mosaic/CP/V4.3/CP20180208/k4m_180209_072450_ooi_zd_ls9.fits.fz k4m_180119_035424_ooi_zd_ls9-CCD3 z expnum 156508 exptime 163.105 propid 2016A-0453 seeing 1.14 MJD 58137.163 object MzLS_656468_z mosaic/CP/V4.3/CP20180118/k4m_180119_035424_ooi_zd_ls9.fits.fz k4m_170128_031236_ooi_zd_ls9-CCD2 z expnum 3105043 exptime 109.104 propid 2016A-0453 seeing 1.37 MJD 57781.134 object MzLS_655686_z mosaic/CP/V4.3/CP20170127/k4m_170128_031236_ooi_zd_ls9.fits.fz k4m_170221_022457_ooi_zd_ls9-CCD2 z expnum 109090 exptime 89.104 propid 2016A-0453 seeing 0.82 MJD 57805.101 object MzLS_654902_z mosaic/CP/V4.3/CP20170220/k4m_170221_022457_ooi_zd_ls9.fits.fz ksb_171127_054522_ooi_r_ls9-CCD4 r expnum 80850097 exptime 195.6 propid BASS seeing 1.22 MJD 58084.240 object 63571 90prime/CP/V2.3/CP20171126/ksb_171127_054522_ooi_r_ls9.fits.fz ksb_180113_062516_ooi_g_ls9-CCD4 g expnum 81310058 exptime 233.753 propid BASS seeing 1.39 MJD 58131.268 object 63571 90prime/CP/V2.3/CP20180112/ksb_180113_062516_ooi_g_ls9.fits.fz ksb_160104_050151_ooi_g_ls9-CCD2 g expnum 73910077 exptime 250.0 propid 2015A-0801 seeing 2.40 MJD 57391.210 object 63573 90prime/CP/V2.3/CP20160103/ksb_160104_050151_ooi_g_ls9.fits.fz ksb_181212_113230_ooi_r_ls9-CCD2 r expnum 84640044 exptime 250.0 propid NOTBASS seeing 1.38 MJD 58464.481 object 90prime/CP/V2.3/CP20181211/ksb_181212_113230_ooi_r_ls9.fits.fz mosaic get_fwhm: self.fwhm = 4.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1633 1882 and amp range 0 2005 and image y 416 662 and amp range 0 2039 Range in image: x 0 250 , y 0 247 (with image size 250 x 247) dzp 0.0014252509 -> scaling image by 1.0013135665633692 mosaic get_fwhm: self.fwhm = 3.15 Found 4 relevant rows in amp-corrections file. Found overlap: image x 2754 3005 and amp range 2048 4095 and image y 2875 3122 and amp range 2040 4078 Range in image: x 0 252 , y 0 248 (with image size 252 x 248) dzp -0.032976374 -> scaling image by 0.9700842429001929 Faked up EXPNUM 81310058 Faked up EXPNUM 81310058 Faked up EXPNUM 81310058 90prime get_fwhm: self.fwhm = 3.07 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3464 3607 and amp range 2016 4031 and image y 1454 1597 and amp range 0 2047 Range in image: x 0 144 , y 0 144 (with image size 144 x 144) dzp 0.023266561 -> scaling image by 1.0216605511853067 Faked up EXPNUM 80850097 Faked up EXPNUM 80850097 Faked up EXPNUM 80850097 90prime get_fwhm: self.fwhm = 2.69 Found 4 relevant rows in amp-corrections file. Found overlap: image x 3468 3611 and amp range 2016 4031 and image y 1453 1595 and amp range 0 2047 Range in image: x 0 144 , y 0 143 (with image size 144 x 143) dzp 0.043530058 -> scaling image by 1.0409072255640852 Faked up EXPNUM 84640044 Faked up EXPNUM 84640044 Faked up EXPNUM 84640044 90prime get_fwhm: self.fwhm = 3.05 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1444 1586 and amp range 0 2015 and image y 3876 4018 and amp range 2048 4095 Range in image: x 0 143 , y 0 143 (with image size 143 x 143) dzp 0.049663078 -> scaling image by 1.0468036570994166 Faked up EXPNUM 73910077 Faked up EXPNUM 73910077 Faked up EXPNUM 73910077 90prime get_fwhm: self.fwhm = 5.29 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1445 1587 and amp range 0 2015 and image y 3922 4064 and amp range 2048 4095 Range in image: x 0 143 , y 0 143 (with image size 143 x 143) dzp 0.01693853 -> scaling image by 1.015723292931903 mosaic get_fwhm: self.fwhm = 3.41 Found 4 relevant rows in amp-corrections file. Found overlap: image x 1636 1884 and amp range 0 2005 and image y 385 631 and amp range 0 2039 Range in image: x 0 249 , y 0 247 (with image size 249 x 247) dzp 0.0014252509 -> scaling image by 1.0013135665633692 mosaic get_fwhm: self.fwhm = 5.27 Found 4 relevant rows in amp-corrections file. Found overlap: image x 39 287 and amp range 0 2047 and image y 1690 1939 and amp range 0 2039 Range in image: x 0 249 , y 0 250 (with image size 249 x 250) dzp -0.0096683735 -> scaling image by 0.9911346301237048 Stage tims : Wall: 110.80 s, CPU: 0.94 s, VmPeak: 951 MB, VmSize: 951 MB, VmRSS: 172 MB, VmData: 206 MB, maxrss: 0.176636 MB Stage tims finished: 2020-08-07T09:24:40.833673 Resources for stage tims : 3.571 s worker CPU, 232.560 s worker Wall, Wall: 110.797 s, Cores in use: 0.04, Total efficiency (on 16 cores): 0.3 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-07T09:24:43.579327 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01432.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-01327.fits Matched 17 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/reference-custom-094442p51548.fits Stage refs : Wall: 6.84 s, CPU: 5.10 s, VmPeak: 1016 MB, VmSize: 1011 MB, VmRSS: 236 MB, VmData: 267 MB, maxrss: 0.241224 MB Stage refs finished: 2020-08-07T09:24:50.419216 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 6.841 s, Cores in use: 0.75, Total efficiency (on 16 cores): 4.7 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T09:24:50.455598 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outlier-mask-custom-094442p51548.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-pre-custom-094442p51548.jpg Image ksb_160104_050151_ooi_g_ls9-CCD2 g : masking 213 positive outlier pixels and 0 negative outlier pixels Image ksb_181212_113230_ooi_r_ls9-CCD2 r : masking 105 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outlier-mask-custom-094442p51548.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-post-custom-094442p51548.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-masked-pos-custom-094442p51548.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/outliers-masked-neg-custom-094442p51548.jpg Stage outliers : Wall: 0.83 s, CPU: 0.56 s, VmPeak: 1021 MB, VmSize: 1021 MB, VmRSS: 248 MB, VmData: 277 MB, maxrss: 0.254112 MB Stage outliers finished: 2020-08-07T09:24:51.285984 Resources for stage outliers : 0.579 s worker CPU, 0.642 s worker Wall, Wall: 0.832 s, Cores in use: 1.37, Total efficiency (on 16 cores): 8.5 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T09:24:51.304543 Stage halos : Wall: 0.27 s, CPU: 0.24 s, VmPeak: 1021 MB, VmSize: 1021 MB, VmRSS: 248 MB, VmData: 277 MB, maxrss: 0.254264 MB Stage halos finished: 2020-08-07T09:24:51.569874 Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Warning: Stellar halo subtraction is only implemented for DECam Resources for stage halos : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 0.267 s, Cores in use: 0.91, Total efficiency (on 16 cores): 5.7 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-07T09:24:51.628473 PSF sigmas in coadd pixels: ['2.26', '3.89'] Using average PSF sigma 3.07805462586082 Gaussian PSF norm 0.09164710379205389 vs pixelized 0.087060064 average tim pixel scale / coadd scale: 1.7306418602447244 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.98', '2.25'] Using average PSF sigma 2.1140485502069972 Gaussian PSF norm 0.13343818037965913 vs pixelized 0.10537209 average tim pixel scale / coadd scale: 1.7310027699590524 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.44', '1.86', '2.22', '1.33'] Using average PSF sigma 1.7118592919444549 Gaussian PSF norm 0.1647885390471867 vs pixelized 0.12621716 average tim pixel scale / coadd scale: 0.9908473532591797 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.15 s, CPU: 0.12 s, VmPeak: 1021 MB, VmSize: 1021 MB, VmRSS: 250 MB, VmData: 277 MB, maxrss: 0.25576 MB Stage fit_on_coadds finished: 2020-08-07T09:24:51.780007 Resources for stage fit_on_coadds : 0.127 s worker CPU, 0.127 s worker Wall, Wall: 0.153 s, Cores in use: 1.62, Total efficiency (on 16 cores): 10.1 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-07T09:24:51.800978 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 29 potential peaks Of 29 potential peaks: 6 in veto map, 3 cut by saddle test, 0 cut by aper test, 20 kept SED z : 20 new peaks Found 31 potential peaks Of 31 potential peaks: 24 in veto map, 5 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 28 potential peaks Of 28 potential peaks: 22 in veto map, 3 cut by saddle test, 0 cut by aper test, 3 kept SED g : 3 new peaks Found 42 potential peaks Of 42 potential peaks: 37 in veto map, 5 cut by saddle test, 0 cut by aper test, 0 kept SED Flat : 0 new peaks Found 41 potential peaks Of 41 potential peaks: 34 in veto map, 5 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 27 new sources Median coadd sky for g : 0.00048784562 Median coadd sky for r : 0.0006163961 Median coadd sky for z : -0.00017802727 Stage srcs : Wall: 0.49 s, CPU: 0.46 s, VmPeak: 1021 MB, VmSize: 1021 MB, VmRSS: 250 MB, VmData: 277 MB, maxrss: 0.256248 MB Stage srcs finished: 2020-08-07T09:24:52.288302 Resources for stage srcs : 0.067 s worker CPU, 0.067 s worker Wall, Wall: 0.489 s, Cores in use: 1.07, Total efficiency (on 16 cores): 6.7 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T09:24:52.336385 Blob 1 of 15, id: 4, sources: 10, size: 150x181, npix 15145, brick X: 32,182, Y: 23,204, one pixel: 131 23 Blob 2 of 15, id: 11, sources: 5, size: 88x67, npix 3117, brick X: 157,245, Y: 178,245, one pixel: 208 178 Blob 3 of 15, id: 7, sources: 2, size: 49x51, npix 1768, brick X: 166,215, Y: 96,147, one pixel: 191 96 Blob 4 of 15, id: 13, sources: 4, size: 43x49, npix 1231, brick X: 29,72, Y: 196,245, one pixel: 51 196 Blob 5 of 15, id: 1, sources: 2, size: 39x43, npix 922, brick X: 47,86, Y: 0,43, one pixel: 74 0 Blob 6 of 15, id: 0, sources: 2, size: 42x27, npix 767, brick X: 0,42, Y: 0,27, one pixel: 1 0 Blob 7 of 15, id: 6, sources: 2, size: 27x40, npix 740, brick X: 0,27, Y: 76,116, one pixel: 14 76 Blob 8 of 15, id: 2, sources: 1, size: 33x30, npix 720, brick X: 87,120, Y: 0,30, one pixel: 98 0 Blob 9 of 15, id: 5, sources: 2, size: 36x28, npix 606, brick X: 209,245, Y: 68,96, one pixel: 222 68 Blob 10 of 15, id: 9, sources: 1, size: 24x29, npix 517, brick X: 221,245, Y: 126,155, one pixel: 233 126 Blob 11 of 15, id: 12, sources: 1, size: 25x24, npix 376, brick X: 169,194, Y: 179,203, one pixel: 179 179 Blob 12 of 15, id: 8, sources: 1, size: 25x24, npix 369, brick X: 20,45, Y: 123,147, one pixel: 30 123 Blob 13 of 15, id: 10, sources: 1, size: 18x29, npix 361, brick X: 227,245, Y: 163,192, one pixel: 238 163 Blob 14 of 15, id: 3, sources: 1, size: 23x21, npix 313, brick X: 152,175, Y: 0,21, one pixel: 159 0 Blob 15 of 15, id: 14, sources: 1, size: 19x18, npix 198, brick X: 126,145, Y: 198,216, one pixel: 134 198 Blob 15 finished, total: Wall: 0.25 s, CPU: 0.25 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.09328 MB Blob 12 finished, total: Wall: 0.29 s, CPU: 0.28 s, VmPeak: 814 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 183 MB, maxrss: 0.145828 MB Blob 14 finished, total: Wall: 0.29 s, CPU: 0.29 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 89 MB, VmData: 137 MB, maxrss: 0.091588 MB Blob 11 finished, total: Wall: 0.34 s, CPU: 0.34 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146012 MB Blob 7 finished, total: Wall: 0.92 s, CPU: 0.92 s, VmPeak: 813 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146188 MB Blob 9 finished, total: Wall: 1.04 s, CPU: 1.03 s, VmPeak: 808 MB, VmSize: 724 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145692 MB Blob 13 finished, total: Wall: 1.04 s, CPU: 1.04 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 137 MB, maxrss: 0.09198 MB Blob 8 finished, total: Wall: 1.04 s, CPU: 1.04 s, VmPeak: 812 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 183 MB, maxrss: 0.146828 MB Blob 10 finished, total: Wall: 1.14 s, CPU: 1.14 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093368 MB Blob 6 finished, total: Wall: 1.22 s, CPU: 1.22 s, VmPeak: 806 MB, VmSize: 724 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145252 MB Measuring 1 iterative sources Blob 5 finished, total: Wall: 1.51 s, CPU: 1.51 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.096432 MB Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (94.43184, 51.56311), pm (-7.6, 1.7), parallax 1.061 with NanoMaggies: g=11.6, r=10.7, z=10.3 Reset brightness to NanoMaggies: g=11.6, r=10.7, z=10.3 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (94.43539, 51.57255), pm (-2.2, -16.1), parallax 1.637 with NanoMaggies: g=13.3, r=12.9, z=12.7 Reset brightness to NanoMaggies: g=13.3, r=12.9, z=12.7 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (94.43239, 51.56102), pm (2.1, -1.0), parallax -0.123 with NanoMaggies: g=19.2, r=18.3, z=18 Reset brightness to NanoMaggies: g=19.2, r=18.3, z=18 Blob 2 finished, total: Wall: 1.79 s, CPU: 1.79 s, VmPeak: 672 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.09878 MB Blob 3 finished, total: Wall: 1.83 s, CPU: 1.83 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093232 MB Measuring 1 iterative sources Blob 4 finished, total: Wall: 2.36 s, CPU: 2.36 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.094812 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.99765, ee1=0.374836, ee2=0.860687 Measuring 4 iterative sources Blob 1 finished, total: Wall: 11.32 s, CPU: 11.32 s, VmPeak: 807 MB, VmSize: 728 MB, VmRSS: 146 MB, VmData: 183 MB, maxrss: 0.149776 MB Old catalog: 47 New catalog: 40 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/blobs-custom-094442p51548.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/metrics/cus/all-models-custom-094442p51548.fits Stage fitblobs : Wall: 12.09 s, CPU: 0.61 s, VmPeak: 1021 MB, VmSize: 1021 MB, VmRSS: 250 MB, VmData: 277 MB, maxrss: 0.25644 MB Stage fitblobs finished: 2020-08-07T09:25:04.429789 Resources for stage fitblobs : 26.592 s worker CPU, 27.222 s worker Wall, Wall: 12.095 s, Cores in use: 2.25, Total efficiency (on 16 cores): 14.1 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-07T09:25:04.463317 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-resid.jpg Stage coadds : Wall: 17.12 s, CPU: 0.50 s, VmPeak: 1021 MB, VmSize: 1021 MB, VmRSS: 251 MB, VmData: 277 MB, maxrss: 0.256776 MB Stage coadds finished: 2020-08-07T09:25:21.578337 Resources for stage coadds : 10.067 s worker CPU, 10.575 s worker Wall, Wall: 17.116 s, Cores in use: 0.62, Total efficiency (on 16 cores): 3.9 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-07T09:25:21.660590 Cut to 1 unWISE tiles Reading WISE tile 0953p514 band 1 Reading WISE tile 0953p514 band 2 Reading WISE tile 0953p514 band 3 Reading WISE tile 0953p514 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 808 MB, VmSize: 800 MB, VmRSS: 144 MB, VmData: 174 MB, maxrss: 0.207016 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 812 MB, VmSize: 803 MB, VmRSS: 147 MB, VmData: 177 MB, maxrss: 0.188356 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 814 MB, VmSize: 798 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.205 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/095/0953p514/unwise-0953p514-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 809 MB, VmSize: 807 MB, VmRSS: 150 MB, VmData: 178 MB, maxrss: 0.214252 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/tmp-legacysurvey-custom-094442p51548-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 4.12 s, CPU: 0.49 s, VmPeak: 1021 MB, VmSize: 1021 MB, VmRSS: 251 MB, VmData: 277 MB, maxrss: 0.257288 MB Stage wise_forced finished: 2020-08-07T09:25:25.782149 Resources for stage wise_forced : 3.057 s worker CPU, 11.626 s worker Wall, Wall: 4.123 s, Cores in use: 0.86, Total efficiency (on 16 cores): 5.4 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-07T09:25:25.822381 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/coadd/cus/custom-094442p51548/legacysurvey-custom-094442p51548-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/tractor-i/cus/tractor-custom-094442p51548.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/tractor/cus/tractor-custom-094442p51548.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/tractor/cus/brick-custom-094442p51548.sha256sum Stage writecat : Wall: 1.06 s, CPU: 0.60 s, VmPeak: 1153 MB, VmSize: 1025 MB, VmRSS: 253 MB, VmData: 277 MB, maxrss: 0.390172 MB Stage writecat finished: 2020-08-07T09:25:26.882954 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.062 s, Cores in use: 0.56, Total efficiency (on 16 cores): 3.5 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/094/PGC2395898/PGC2395898-largegalaxy-writecat.p All done: 44.064 s worker CPU, 282.823 s worker Wall, Wall: 156.898 s, Cores in use: 0.34, Total efficiency (on 16 cores): 2.1 % Finished galaxy PGC2395898 in 2.657 minutes.