Started working on galaxy 2MASXJ14253638+0035255 at Mon Aug 3 06:44:21 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.401634 0.5904245 --width 273 --height 273 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-03T06:44:23.363958 legacypipe git version: DR9.5.10-2-ge83842ef Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.401634', '0.5904245', '--width', '273', '--height', '273', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.401634 0.5904245 --width 273 --height 273 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 216.401634 0.5904245 Starting process 22827 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058892 MB Starting process 22828 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058892 MB Starting process 22829 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05894 MB Starting process 22830 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.059196 MB Starting process 22831 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.059196 MB Starting process 22832 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.059196 MB Starting process 22833 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22834 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22835 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.0589 MB Starting process 22836 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058912 MB Starting process 22837 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB Starting process 22838 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB Starting process 22839 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB Starting process 22840 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB Starting process 22841 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB Starting process 22842 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05892 MB Starting process 22843 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058924 MB Starting process 22844 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058924 MB Starting process 22845 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058928 MB Starting process 22846 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058928 MB Starting process 22847 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058932 MB Starting process 22848 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058936 MB Starting process 22849 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058944 MB Starting process 22850 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058944 MB Starting process 22851 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058944 MB Starting process 22852 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058944 MB Starting process 22853 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058944 MB Starting process 22854 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058944 MB Starting process 22855 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058948 MB Starting process 22856 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058952 MB Starting process 22857 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.058956 MB Starting process 22858 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.058956 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-03T06:44:26.656029 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 11 CCDs: c4d_170401_091101_ooi_r_ls9-S26 r expnum 636075 exptime 98.0 propid 2014B-0404 seeing 1.53 MJD 57844.381 object DECaLS_32984_r decam/CP/V4.8.2/CP20170331/c4d_170401_091101_ooi_r_ls9.fits.fz c4d_160724_023544_ooi_z_ls9-S26 z expnum 554132 exptime 250.0 propid 2016A-0190 seeing 1.45 MJD 57593.105 object DECaLS_32984_z decam/CP/V4.8.2a/CP20160723/c4d_160724_023544_ooi_z_ls9.fits.fz c4d_170719_030046_ooi_g_ls9-S26 g expnum 659944 exptime 200.0 propid 2014B-0404 seeing 1.65 MJD 57953.123 object DECaLS_32984_g decam/CP/V4.8.2/CP20170718/c4d_170719_030046_ooi_g_ls9.fits.fz c4d_180211_073607_ooi_z_ls9-S27 z expnum 720129 exptime 80.0 propid 2014B-0404 seeing 0.98 MJD 58160.317 object DECaLS_17112_z decam/CP/V4.8.2a/CP20180210/c4d_180211_073607_ooi_z_ls9.fits.fz c4d_170330_082935_ooi_r_ls9-S27 r expnum 635348 exptime 56.0 propid 2014B-0404 seeing 1.14 MJD 57842.353 object DECaLS_17112_r decam/CP/V4.8.2/CP20170329/c4d_170330_082935_ooi_r_ls9.fits.fz c4d_180211_073607_ooi_z_ls9-S22 z expnum 720129 exptime 80.0 propid 2014B-0404 seeing 0.98 MJD 58160.317 object DECaLS_17112_z decam/CP/V4.8.2a/CP20180210/c4d_180211_073607_ooi_z_ls9.fits.fz c4d_170330_082935_ooi_r_ls9-S22 r expnum 635348 exptime 56.0 propid 2014B-0404 seeing 1.13 MJD 57842.353 object DECaLS_17112_r decam/CP/V4.8.2/CP20170329/c4d_170330_082935_ooi_r_ls9.fits.fz c4d_160607_014141_ooi_g_ls9-S22 g expnum 547221 exptime 62.0 propid 2014B-0404 seeing 1.24 MJD 57546.069 object DECaLS_17112_g decam/CP/V4.8.2a/CP20160606/c4d_160607_014141_ooi_g_ls9.fits.fz c4d_170301_083415_ooi_r_ls9-S23 r expnum 624650 exptime 49.0 propid 2014B-0404 seeing 1.10 MJD 57813.356 object DECaLS_1240_r decam/CP/V4.8.2/CP20170228/c4d_170301_083415_ooi_r_ls9.fits.fz c4d_170302_082352_ooi_g_ls9-S23 g expnum 625055 exptime 94.0 propid 2014B-0404 seeing 0.82 MJD 57814.348 object DECaLS_1240_g decam/CP/V4.8.2/CP20170301/c4d_170302_082352_ooi_g_ls9.fits.fz c4d_150110_082531_ooi_z_ls9-S23 z expnum 396461 exptime 81.0 propid 2014B-0404 seeing 1.12 MJD 57032.350 object DECaLS_1240_z decam/CP/V4.8.2a/CP20150109/c4d_150110_082531_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S27,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95176d0-r-20170401T090831_oi[S26,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951ba61-g-20170719T025633_oi[S26,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S27,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9563f3b-z-20150110T082321_oi[S23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950e32f-g-20170302T082130_oi[S23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950c76c-r-20170301T083236_oi[S23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954b7a0-z-20160724T023045_oi[S26,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95675e1-g-20160607T013949_oi[S22,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0668659 vs median 0.0514287 Warning: splinesky does not have sky-template version/run/scale values Swapping in SKY_JOHN values for 1 splinesky cells; 1275.4062 -> 1274.466 Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 11.18 s, CPU: 2.97 s, VmPeak: 1005 MB, VmSize: 1005 MB, VmRSS: 176 MB, VmData: 208 MB, maxrss: 0.17978 MB Stage tims finished: 2020-08-03T06:44:37.833295 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S27,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951ba61-g-20170719T025633_oi[S26,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9563f3b-z-20150110T082321_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954b7a0-z-20160724T023045_oi[S26,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950e32f-g-20170302T082130_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.774 s worker CPU, 47.046 s worker Wall, Wall: 11.180 s, Cores in use: 0.78, Total efficiency (on 32 cores): 2.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S27,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95176d0-r-20170401T090831_oi[S26,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95675e1-g-20160607T013949_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-03T06:44:37.857889 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06156.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06157.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06029.fits Matched 6 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/reference-custom-216401p00590.fits Stage refs : Wall: 5.11 s, CPU: 4.06 s, VmPeak: 1035 MB, VmSize: 1026 MB, VmRSS: 198 MB, VmData: 230 MB, maxrss: 0.203188 MB Stage refs finished: 2020-08-03T06:44:42.963912 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950c76c-r-20170301T083236_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 5.108 s, Cores in use: 0.80, Total efficiency (on 32 cores): 2.5 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-03T06:44:43.071072 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outlier-mask-custom-216401p00590.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-pre-custom-216401p00590.jpg Image c4d_170719_030046_ooi_g_ls9-S26 g : masking 320 positive outlier pixels and 0 negative outlier pixels Image c4d_170302_082352_ooi_g_ls9-S23 g : masking 436 positive outlier pixels and 0 negative outlier pixels Image c4d_170401_091101_ooi_r_ls9-S26 r : masking 118 positive outlier pixels and 0 negative outlier pixels Image c4d_170301_083415_ooi_r_ls9-S23 r : masking 372 positive outlier pixels and 0 negative outlier pixels Image c4d_160724_023544_ooi_z_ls9-S26 z : masking 1256 positive outlier pixels and 221 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outlier-mask-custom-216401p00590.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-post-custom-216401p00590.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-masked-pos-custom-216401p00590.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-masked-neg-custom-216401p00590.jpg Stage outliers : Wall: 3.01 s, CPU: 2.59 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 213 MB, VmData: 242 MB, maxrss: 0.218236 MB Stage outliers finished: 2020-08-03T06:44:46.083553 Resources for stage outliers : 0.823 s worker CPU, 1.090 s worker Wall, Wall: 3.015 s, Cores in use: 1.13, Total efficiency (on 32 cores): 3.5 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-03T06:44:46.114570 Stage halos : Wall: 2.79 s, CPU: 2.39 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 213 MB, VmData: 242 MB, maxrss: 0.218312 MB Stage halos finished: 2020-08-03T06:44:48.903500 Resources for stage halos : 0.333 s worker CPU, 0.948 s worker Wall, Wall: 2.792 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-03T06:44:49.037450 PSF sigmas in coadd pixels: ['2.69', '2.02', '1.33'] Using average PSF sigma 2.011838567409326 Gaussian PSF norm 0.14021740926118925 vs pixelized 0.11798673 average tim pixel scale / coadd scale: 1.001144418997676 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.49', '1.83', '1.78'] Using average PSF sigma 2.033091829289866 Gaussian PSF norm 0.13875162336981622 vs pixelized 0.11211915 average tim pixel scale / coadd scale: 1.0011934733928274 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-copsf-r.fits.fz PSF sigmas in coadd pixels: ['2.35', '1.60', '1.81'] Using average PSF sigma 1.9213147354638591 Gaussian PSF norm 0.1468238319141255 vs pixelized 0.1109757 average tim pixel scale / coadd scale: 1.0012391619633114 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.15 s, CPU: 0.13 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 215 MB, VmData: 242 MB, maxrss: 0.219984 MB Stage fit_on_coadds finished: 2020-08-03T06:44:49.182540 Resources for stage fit_on_coadds : 0.156 s worker CPU, 0.156 s worker Wall, Wall: 0.147 s, Cores in use: 1.93, Total efficiency (on 32 cores): 6.0 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-03T06:44:49.201671 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 47 potential peaks Of 47 potential peaks: 27 in veto map, 8 cut by saddle test, 0 cut by aper test, 12 kept SED z : 12 new peaks Found 65 potential peaks Of 65 potential peaks: 55 in veto map, 5 cut by saddle test, 0 cut by aper test, 5 kept SED r : 5 new peaks Found 56 potential peaks Of 56 potential peaks: 49 in veto map, 5 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 78 potential peaks Of 78 potential peaks: 69 in veto map, 7 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 77 potential peaks Of 77 potential peaks: 70 in veto map, 7 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 21 new sources Median coadd sky for g : 0.00017350493 Median coadd sky for r : 0.0005249017 Median coadd sky for z : 0.00072207267 Stage srcs : Wall: 2.77 s, CPU: 2.70 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 215 MB, VmData: 242 MB, maxrss: 0.220544 MB Stage srcs finished: 2020-08-03T06:44:51.969694 Resources for stage srcs : 0.083 s worker CPU, 0.083 s worker Wall, Wall: 2.770 s, Cores in use: 1.01, Total efficiency (on 32 cores): 3.1 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-03T06:44:51.984316 Blob 1 of 4, id: 1, sources: 23, size: 223x265, npix 32362, brick X: 37,260, Y: 0,265, one pixel: 225 0 Blob 2 of 4, id: 2, sources: 1, size: 28x52, npix 1131, brick X: 0,28, Y: 123,175, one pixel: 4 123 Blob 3 of 4, id: 3, sources: 1, size: 21x21, npix 259, brick X: 29,50, Y: 141,162, one pixel: 38 141 Blob 4 of 4, id: 0, sources: 1, size: 20x16, npix 206, brick X: 100,120, Y: 0,16, one pixel: 106 0 Blob 3 finished, total: Wall: 0.04 s, CPU: 0.04 s, VmPeak: 660 MB, VmSize: 660 MB, VmRSS: 85 MB, VmData: 138 MB, maxrss: 0.08716 MB Blob 4 finished, total: Wall: 0.73 s, CPU: 0.73 s, VmPeak: 662 MB, VmSize: 662 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094464 MB Blob 2 finished, total: Wall: 1.24 s, CPU: 1.24 s, VmPeak: 951 MB, VmSize: 718 MB, VmRSS: 138 MB, VmData: 179 MB, maxrss: 0.141744 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=0.682043, ee1=0.0270854, ee2=0.227374 Measuring 9 iterative sources Blob 1 finished, total: Wall: 24.46 s, CPU: 24.42 s, VmPeak: 699 MB, VmSize: 699 MB, VmRSS: 141 MB, VmData: 176 MB, maxrss: 0.144012 MB Old catalog: 26 New catalog: 34 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/blobs-custom-216401p00590.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/all-models-custom-216401p00590.fits Stage fitblobs : Wall: 27.26 s, CPU: 2.41 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 216 MB, VmData: 242 MB, maxrss: 0.22146 MB Stage fitblobs finished: 2020-08-03T06:45:19.240642 Resources for stage fitblobs : 26.495 s worker CPU, 27.426 s worker Wall, Wall: 27.259 s, Cores in use: 1.06, Total efficiency (on 32 cores): 3.3 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-03T06:45:19.259441 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-resid.jpg Stage coadds : Wall: 3.10 s, CPU: 0.57 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 217 MB, VmData: 242 MB, maxrss: 0.22206 MB Stage coadds finished: 2020-08-03T06:45:22.361483 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 24.551 s worker CPU, 44.990 s worker Wall, Wall: 3.105 s, Cores in use: 8.09, Total efficiency (on 32 cores): 25.3 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-03T06:45:22.375150 Cut to 1 unWISE tiles Reading WISE tile 2163p000 band 1 Reading WISE tile 2163p000 band 2 Reading WISE tile 2163p000 band 4 Reading WISE tile 2163p000 band 3 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 949 MB, VmSize: 799 MB, VmRSS: 147 MB, VmData: 176 MB, maxrss: 0.192912 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 948 MB, VmSize: 796 MB, VmRSS: 146 MB, VmData: 174 MB, maxrss: 0.206044 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 943 MB, VmSize: 796 MB, VmRSS: 143 MB, VmData: 173 MB, maxrss: 0.206008 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 792 MB, VmSize: 792 MB, VmRSS: 139 MB, VmData: 172 MB, maxrss: 0.1995 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 1.74 s, CPU: 0.44 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 218 MB, VmData: 242 MB, maxrss: 0.222764 MB Stage wise_forced finished: 2020-08-03T06:45:24.117377 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.131 s worker CPU, 4.058 s worker Wall, Wall: 1.745 s, Cores in use: 2.05, Total efficiency (on 32 cores): 6.4 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-wise_forced.p Running stage writecat WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage writecat at 2020-08-03T06:45:24.136881 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/tractor-i/cus/tractor-custom-216401p00590.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/tractor/cus/tractor-custom-216401p00590.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/tractor/cus/brick-custom-216401p00590.sha256sum Stage writecat : Wall: 0.70 s, CPU: 0.50 s, VmPeak: 1168 MB, VmSize: 1040 MB, VmRSS: 219 MB, VmData: 242 MB, maxrss: 0.355532 MB Stage writecat finished: 2020-08-03T06:45:24.840701 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 0.706 s, Cores in use: 0.71, Total efficiency (on 32 cores): 2.2 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-writecat.p All done: 61.353 s worker CPU, 125.804 s worker Wall, Wall: 58.336 s, Cores in use: 1.38, Total efficiency (on 32 cores): 4.3 % Finished galaxy 2MASXJ14253638+0035255 in 1.070 minutes. Started working on galaxy 2MASXJ14253638+0035255 at Thu Aug 6 07:37:33 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.401634 0.5904245 --width 273 --height 273 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-06T07:37:34.565032 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '216.401634', '0.5904245', '--width', '273', '--height', '273', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 216.401634 0.5904245 --width 273 --height 273 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 216.401634 0.5904245 Starting process 5057 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 5058 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 5059 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 5060 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 5061 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059344 MB Starting process 5062 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059348 MB Starting process 5063 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059328 MB Starting process 5064 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 5065 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 5066 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 5067 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 5068 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 5069 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 5070 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 5071 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 5072 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 5073 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 5074 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 5075 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 5076 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 5077 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 5078 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 5079 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 5080 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 5081 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 5082 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 5083 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 5084 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 5085 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 5086 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 5087 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 5088 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-06T07:37:34.919513 Keeping 11 CCDs: c4d_170401_091101_ooi_r_ls9-S26 r expnum 636075 exptime 98.0 propid 2014B-0404 seeing 1.53 MJD 57844.381 object DECaLS_32984_r decam/CP/V4.8.2/CP20170331/c4d_170401_091101_ooi_r_ls9.fits.fz c4d_160724_023544_ooi_z_ls9-S26 z expnum 554132 exptime 250.0 propid 2016A-0190 seeing 1.45 MJD 57593.105 object DECaLS_32984_z decam/CP/V4.8.2a/CP20160723/c4d_160724_023544_ooi_z_ls9.fits.fz c4d_170719_030046_ooi_g_ls9-S26 g expnum 659944 exptime 200.0 propid 2014B-0404 seeing 1.65 MJD 57953.123 object DECaLS_32984_g decam/CP/V4.8.2/CP20170718/c4d_170719_030046_ooi_g_ls9.fits.fz c4d_180211_073607_ooi_z_ls9-S27 z expnum 720129 exptime 80.0 propid 2014B-0404 seeing 0.98 MJD 58160.317 object DECaLS_17112_z decam/CP/V4.8.2a/CP20180210/c4d_180211_073607_ooi_z_ls9.fits.fz c4d_170330_082935_ooi_r_ls9-S27 r expnum 635348 exptime 56.0 propid 2014B-0404 seeing 1.14 MJD 57842.353 object DECaLS_17112_r decam/CP/V4.8.2/CP20170329/c4d_170330_082935_ooi_r_ls9.fits.fz c4d_180211_073607_ooi_z_ls9-S22 z expnum 720129 exptime 80.0 propid 2014B-0404 seeing 0.98 MJD 58160.317 object DECaLS_17112_z decam/CP/V4.8.2a/CP20180210/c4d_180211_073607_ooi_z_ls9.fits.fz c4d_170330_082935_ooi_r_ls9-S22 r expnum 635348 exptime 56.0 propid 2014B-0404 seeing 1.13 MJD 57842.353 object DECaLS_17112_r decam/CP/V4.8.2/CP20170329/c4d_170330_082935_ooi_r_ls9.fits.fz c4d_160607_014141_ooi_g_ls9-S22 g expnum 547221 exptime 62.0 propid 2014B-0404 seeing 1.24 MJD 57546.069 object DECaLS_17112_g decam/CP/V4.8.2a/CP20160606/c4d_160607_014141_ooi_g_ls9.fits.fz c4d_170301_083415_ooi_r_ls9-S23 r expnum 624650 exptime 49.0 propid 2014B-0404 seeing 1.10 MJD 57813.356 object DECaLS_1240_r decam/CP/V4.8.2/CP20170228/c4d_170301_083415_ooi_r_ls9.fits.fz c4d_170302_082352_ooi_g_ls9-S23 g expnum 625055 exptime 94.0 propid 2014B-0404 seeing 0.82 MJD 57814.348 object DECaLS_1240_g decam/CP/V4.8.2/CP20170301/c4d_170302_082352_ooi_g_ls9.fits.fz c4d_150110_082531_ooi_z_ls9-S23 z expnum 396461 exptime 81.0 propid 2014B-0404 seeing 1.12 MJD 57032.350 object DECaLS_1240_z decam/CP/V4.8.2a/CP20150109/c4d_150110_082531_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951ba61-g-20170719T025633_oi[S26,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S27,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S27,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950e32f-g-20170302T082130_oi[S23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95675e1-g-20160607T013949_oi[S22,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0668659 vs median 0.0514287 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9563f3b-z-20150110T082321_oi[S23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950c76c-r-20170301T083236_oi[S23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S22,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95176d0-r-20170401T090831_oi[S26,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954b7a0-z-20160724T023045_oi[S26,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values Swapping in SKY_JOHN values for 1 splinesky cells; 1275.4062 -> 1274.466 Stage tims : Wall: 4.62 s, CPU: 0.81 s, VmPeak: 1005 MB, VmSize: 1005 MB, VmRSS: 175 MB, VmData: 209 MB, maxrss: 0.179184 MB Stage tims finished: 2020-08-06T07:37:39.539696 Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954b7a0-z-20160724T023045_oi[S26,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95176d0-r-20170401T090831_oi[S26,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950c76c-r-20170301T083236_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950e32f-g-20170302T082130_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951ba61-g-20170719T025633_oi[S26,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9563f3b-z-20150110T082321_oi[S23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Resources for stage tims : 5.791 s worker CPU, 35.945 s worker Wall, Wall: 4.623 s, Cores in use: 1.43, Total efficiency (on 32 cores): 4.5 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9515f71-r-20170330T082748_oi[S27,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952764d-z-20180211T073607_oi[S27,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Running stage refs at 2020-08-06T07:37:39.607017 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06156.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06157.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06029.fits Matched 6 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/reference-custom-216401p00590.fits Stage refs : Wall: 2.17 s, CPU: 1.95 s, VmPeak: 1035 MB, VmSize: 1026 MB, VmRSS: 198 MB, VmData: 230 MB, maxrss: 0.202604 MB Stage refs finished: 2020-08-06T07:37:41.778273 Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95675e1-g-20160607T013949_oi[S22,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 2.174 s, Cores in use: 0.90, Total efficiency (on 32 cores): 2.8 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-06T07:37:41.882363 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outlier-mask-custom-216401p00590.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-pre-custom-216401p00590.jpg Image c4d_170302_082352_ooi_g_ls9-S23 g : masking 436 positive outlier pixels and 0 negative outlier pixels Image c4d_170719_030046_ooi_g_ls9-S26 g : masking 320 positive outlier pixels and 0 negative outlier pixels Image c4d_170301_083415_ooi_r_ls9-S23 r : masking 372 positive outlier pixels and 0 negative outlier pixels Image c4d_170401_091101_ooi_r_ls9-S26 r : masking 118 positive outlier pixels and 0 negative outlier pixels Image c4d_160724_023544_ooi_z_ls9-S26 z : masking 1256 positive outlier pixels and 221 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outlier-mask-custom-216401p00590.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-post-custom-216401p00590.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-masked-pos-custom-216401p00590.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/outliers-masked-neg-custom-216401p00590.jpg Stage outliers : Wall: 1.99 s, CPU: 1.43 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 211 MB, VmData: 241 MB, maxrss: 0.216536 MB Stage outliers finished: 2020-08-06T07:37:43.876669 Resources for stage outliers : 0.830 s worker CPU, 1.128 s worker Wall, Wall: 1.996 s, Cores in use: 1.13, Total efficiency (on 32 cores): 3.5 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-06T07:37:43.912558 Stage halos : Wall: 1.28 s, CPU: 1.10 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 212 MB, VmData: 241 MB, maxrss: 0.21676 MB Stage halos finished: 2020-08-06T07:37:45.195839 Resources for stage halos : 0.349 s worker CPU, 1.111 s worker Wall, Wall: 1.286 s, Cores in use: 1.13, Total efficiency (on 32 cores): 3.5 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-06T07:37:45.259399 PSF sigmas in coadd pixels: ['2.69', '2.02', '1.33'] Using average PSF sigma 2.011838567409326 Gaussian PSF norm 0.14021740926118925 vs pixelized 0.11798673 average tim pixel scale / coadd scale: 1.001144418997676 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.49', '1.83', '1.78'] Using average PSF sigma 2.033091829289866 Gaussian PSF norm 0.13875162336981622 vs pixelized 0.11211915 average tim pixel scale / coadd scale: 1.0011934733928274 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-copsf-r.fits.fz PSF sigmas in coadd pixels: ['2.35', '1.60', '1.81'] Using average PSF sigma 1.9213147354638591 Gaussian PSF norm 0.1468238319141255 vs pixelized 0.1109757 average tim pixel scale / coadd scale: 1.0012391619633114 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.19 s, CPU: 0.13 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 213 MB, VmData: 241 MB, maxrss: 0.218356 MB Stage fit_on_coadds finished: 2020-08-06T07:37:45.446080 Resources for stage fit_on_coadds : 0.134 s worker CPU, 0.134 s worker Wall, Wall: 0.189 s, Cores in use: 1.42, Total efficiency (on 32 cores): 4.4 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-06T07:37:45.460600 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 47 potential peaks Of 47 potential peaks: 27 in veto map, 8 cut by saddle test, 0 cut by aper test, 12 kept SED z : 12 new peaks Found 65 potential peaks Of 65 potential peaks: 55 in veto map, 5 cut by saddle test, 0 cut by aper test, 5 kept SED r : 5 new peaks Found 56 potential peaks Of 56 potential peaks: 49 in veto map, 5 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 78 potential peaks Of 78 potential peaks: 69 in veto map, 7 cut by saddle test, 0 cut by aper test, 2 kept SED Flat : 2 new peaks Found 77 potential peaks Of 77 potential peaks: 70 in veto map, 7 cut by saddle test, 0 cut by aper test, 0 kept SED Red : 0 new peaks Adding 21 new sources Median coadd sky for g : 0.00017350493 Median coadd sky for r : 0.0005249017 Median coadd sky for z : 0.00072207267 Stage srcs : Wall: 0.77 s, CPU: 0.72 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 214 MB, VmData: 241 MB, maxrss: 0.218876 MB Stage srcs finished: 2020-08-06T07:37:46.228141 Resources for stage srcs : 0.103 s worker CPU, 0.103 s worker Wall, Wall: 0.770 s, Cores in use: 1.07, Total efficiency (on 32 cores): 3.3 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-06T07:37:46.244490 Blob 1 of 4, id: 1, sources: 23, size: 223x265, npix 32362, brick X: 37,260, Y: 0,265, one pixel: 225 0 Blob 2 of 4, id: 2, sources: 1, size: 28x52, npix 1131, brick X: 0,28, Y: 123,175, one pixel: 4 123 Blob 3 of 4, id: 3, sources: 1, size: 21x21, npix 259, brick X: 29,50, Y: 141,162, one pixel: 38 141 Blob 4 of 4, id: 0, sources: 1, size: 20x16, npix 206, brick X: 100,120, Y: 0,16, one pixel: 106 0 Blob 3 finished, total: Wall: 0.05 s, CPU: 0.04 s, VmPeak: 942 MB, VmSize: 726 MB, VmRSS: 133 MB, VmData: 181 MB, maxrss: 0.136388 MB Blob 4 finished, total: Wall: 0.76 s, CPU: 0.76 s, VmPeak: 697 MB, VmSize: 697 MB, VmRSS: 115 MB, VmData: 168 MB, maxrss: 0.117524 MB Blob 2 finished, total: Wall: 1.30 s, CPU: 1.29 s, VmPeak: 939 MB, VmSize: 722 MB, VmRSS: 138 MB, VmData: 177 MB, maxrss: 0.141584 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=0.680799, ee1=0.0252083, ee2=0.230653 Measuring 8 iterative sources Blob 1 finished, total: Wall: 25.95 s, CPU: 25.64 s, VmPeak: 942 MB, VmSize: 744 MB, VmRSS: 168 MB, VmData: 199 MB, maxrss: 0.171636 MB Old catalog: 26 New catalog: 33 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/blobs-custom-216401p00590.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/metrics/cus/all-models-custom-216401p00590.fits Stage fitblobs : Wall: 27.06 s, CPU: 0.43 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 215 MB, VmData: 243 MB, maxrss: 0.22002 MB Stage fitblobs finished: 2020-08-06T07:38:13.308288 Resources for stage fitblobs : 27.794 s worker CPU, 28.234 s worker Wall, Wall: 27.066 s, Cores in use: 1.04, Total efficiency (on 32 cores): 3.3 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-06T07:38:13.353696 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-resid.jpg Stage coadds : Wall: 3.71 s, CPU: 2.04 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 215 MB, VmData: 243 MB, maxrss: 0.220472 MB Stage coadds finished: 2020-08-06T07:38:17.065459 Resources for stage coadds : 27.053 s worker CPU, 30.928 s worker Wall, Wall: 3.714 s, Cores in use: 7.83, Total efficiency (on 32 cores): 24.5 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-06T07:38:17.104577 Cut to 1 unWISE tiles Reading WISE tile 2163p000 band 2 Reading WISE tile 2163p000 band 4 Reading WISE tile 2163p000 band 1 Reading WISE tile 2163p000 band 3 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 856 MB, VmSize: 856 MB, VmRSS: 201 MB, VmData: 234 MB, maxrss: 0.205684 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 934 MB, VmSize: 801 MB, VmRSS: 144 MB, VmData: 176 MB, maxrss: 0.184324 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 939 MB, VmSize: 804 MB, VmRSS: 146 MB, VmData: 175 MB, maxrss: 0.209344 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/216/2163p000/unwise-2163p000-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 801 MB, VmSize: 801 MB, VmRSS: 144 MB, VmData: 179 MB, maxrss: 0.1965 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/tmp-legacysurvey-custom-216401p00590-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 4.29 s, CPU: 2.14 s, VmPeak: 1036 MB, VmSize: 1036 MB, VmRSS: 216 MB, VmData: 243 MB, maxrss: 0.221176 MB Stage wise_forced finished: 2020-08-06T07:38:21.389615 Resources for stage wise_forced : 3.124 s worker CPU, 5.998 s worker Wall, Wall: 4.287 s, Cores in use: 1.23, Total efficiency (on 32 cores): 3.8 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-06T07:38:21.421308 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/coadd/cus/custom-216401p00590/legacysurvey-custom-216401p00590-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/tractor-i/cus/tractor-custom-216401p00590.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/tractor/cus/tractor-custom-216401p00590.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/tractor/cus/brick-custom-216401p00590.sha256sum Stage writecat : Wall: 1.09 s, CPU: 0.90 s, VmPeak: 1168 MB, VmSize: 1040 MB, VmRSS: 218 MB, VmData: 243 MB, maxrss: 0.35406 MB Stage writecat finished: 2020-08-06T07:38:22.507805 Resources for stage writecat : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 1.089 s, Cores in use: 0.83, Total efficiency (on 32 cores): 2.6 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/216/2MASXJ14253638+0035255/2MASXJ14253638+0035255-largegalaxy-writecat.p All done: 65.184 s worker CPU, 103.587 s worker Wall, Wall: 47.654 s, Cores in use: 1.62, Total efficiency (on 32 cores): 5.0 % Finished galaxy 2MASXJ14253638+0035255 in 0.818 minutes.