Started working on galaxy PGC1906774 at Wed Aug 5 18:53:28 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 128.23275 30.5142 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T18:53:29.630493 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '128.23275', '30.5142', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 128.23275 30.5142 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 128.23275 30.5142 Starting process 65189 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 65190 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 65191 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 65192 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 65193 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059532 MB Starting process 65194 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB Starting process 65195 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB Starting process 65196 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 65197 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 65198 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 65199 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 65200 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 65201 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 65202 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 65204 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 65203 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T18:53:30.114334 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: c4d_141228_065726_ooi_g_ls9-S21 g expnum 392430 exptime 87.0 propid 2014B-0404 seeing 1.55 MJD 57019.288 object DECaLS_10847_g decam/CP/V4.8.2a/CP20141227/c4d_141228_065726_ooi_g_ls9.fits.fz c4d_190303_030009_ooi_g_ls9-S21 g expnum 829482 exptime 171.0 propid 2014B-0404 seeing 1.36 MJD 58545.125 object DECaLS_10847_g decam/CP/V4.8.2a/CP20190302/c4d_190303_030009_ooi_g_ls9.fits.fz c4d_141228_065140_ooi_r_ls9-S21 r expnum 392427 exptime 71.0 propid 2014B-0404 seeing 1.36 MJD 57019.284 object DECaLS_10847_r decam/CP/V4.8.2a/CP20141227/c4d_141228_065140_ooi_r_ls9.fits.fz c4d_141231_052726_ooi_z_ls9-S21 z expnum 393585 exptime 125.0 propid 2014B-0404 seeing 1.00 MJD 57022.225 object DECaLS_10847_z decam/CP/V4.8.2a/CP20141230/c4d_141231_052726_ooi_z_ls9.fits.fz c4d_180213_035946_ooi_r_ls9-N24 r expnum 720791 exptime 138.0 propid 2014B-0404 seeing 1.27 MJD 58162.167 object DECaLS_26712_r decam/CP/V4.8.2a/CP20180212/c4d_180213_035946_ooi_r_ls9.fits.fz c4d_180212_042033_ooi_g_ls9-N24 g expnum 720472 exptime 167.0 propid 2014B-0404 seeing 1.47 MJD 58161.181 object DECaLS_26712_g decam/CP/V4.8.2a/CP20180211/c4d_180212_042033_ooi_g_ls9.fits.fz c4d_170327_001946_ooi_z_ls9-N24 z expnum 633948 exptime 245.0 propid 2014B-0404 seeing 1.35 MJD 57839.010 object DECaLS_26712_z decam/CP/V4.8.2/CP20170326/c4d_170327_001946_ooi_z_ls9.fits.fz c4d_181214_073256_ooi_g_ls9-N23 g expnum 803012 exptime 200.0 propid 2014B-0404 seeing 1.58 MJD 58466.315 object DECaLS_42584_g decam/CP/V4.8.2a/CP20181213/c4d_181214_073256_ooi_g_ls9.fits.fz c4d_181214_073109_ooi_r_ls9-N23 r expnum 803011 exptime 78.0 propid 2014B-0404 seeing 1.19 MJD 58466.313 object DECaLS_42584_r decam/CP/V4.8.2a/CP20181213/c4d_181214_073109_ooi_r_ls9.fits.fz c4d_181215_081002_ooi_z_ls9-N23 z expnum 803372 exptime 91.0 propid 2014B-0404 seeing 0.90 MJD 58467.340 object DECaLS_42584_z decam/CP/V4.8.2a/CP20181214/c4d_181215_081002_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95613b5-z-20141231T052432_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95506eb-r-20141228T064940_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953a046-g-20190303T030009_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9533681-z-20181215T081002_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9539067-r-20181214T073109_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95117e6-z-20170327T001453_oi[N24,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953988f-g-20181214T073256_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9552e01-g-20141228T065503_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a2af-g-20180212T042033_oi[N24,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a3cf-r-20180213T035946_oi[N24,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 133.05 s, CPU: 0.99 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 178 MB, VmData: 210 MB, maxrss: 0.181804 MB Stage tims finished: 2020-08-05T18:55:43.169095 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9552e01-g-20141228T065503_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95506eb-r-20141228T064940_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953988f-g-20181214T073256_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9533681-z-20181215T081002_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95613b5-z-20141231T052432_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 6.005 s worker CPU, 831.662 s worker Wall, Wall: 133.056 s, Cores in use: 0.05, Total efficiency (on 16 cores): 0.3 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9539067-r-20181214T073109_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953a046-g-20190303T030009_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a3cf-r-20180213T035946_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a2af-g-20180212T042033_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T18:55:43.333918 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-03053.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02926.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/reference-custom-128232p30514.fits Stage refs : Wall: 14.85 s, CPU: 2.30 s, VmPeak: 1042 MB, VmSize: 1032 MB, VmRSS: 199 MB, VmData: 230 MB, maxrss: 0.203924 MB Stage refs finished: 2020-08-05T18:55:58.179039 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95117e6-z-20170327T001453_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 14.847 s, Cores in use: 0.15, Total efficiency (on 16 cores): 1.0 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T18:55:58.327904 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outlier-mask-custom-128232p30514.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-pre-custom-128232p30514.jpg Image c4d_141228_065726_ooi_g_ls9-S21 g : masking 300 positive outlier pixels and 0 negative outlier pixels Image c4d_190303_030009_ooi_g_ls9-S21 g : masking 465 positive outlier pixels and 0 negative outlier pixels Image c4d_180213_035946_ooi_r_ls9-N24 r : masking 575 positive outlier pixels and 0 negative outlier pixels Image c4d_141231_052726_ooi_z_ls9-S21 z : masking 207 positive outlier pixels and 0 negative outlier pixels Image c4d_181215_081002_ooi_z_ls9-N23 z : masking 495 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outlier-mask-custom-128232p30514.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-post-custom-128232p30514.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-masked-pos-custom-128232p30514.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-masked-neg-custom-128232p30514.jpg Stage outliers : Wall: 1.05 s, CPU: 0.75 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 212 MB, VmData: 240 MB, maxrss: 0.216828 MB Stage outliers finished: 2020-08-05T18:55:59.382582 Resources for stage outliers : 0.814 s worker CPU, 0.950 s worker Wall, Wall: 1.056 s, Cores in use: 1.48, Total efficiency (on 16 cores): 9.3 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T18:55:59.409741 Stage halos : Wall: 1.35 s, CPU: 0.51 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 212 MB, VmData: 240 MB, maxrss: 0.216948 MB Stage halos finished: 2020-08-05T18:56:00.761758 Resources for stage halos : 0.784 s worker CPU, 2.527 s worker Wall, Wall: 1.354 s, Cores in use: 0.96, Total efficiency (on 16 cores): 6.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T18:56:00.899336 PSF sigmas in coadd pixels: ['2.52', '2.21', '2.39', '2.57'] Using average PSF sigma 2.4217805645929587 Gaussian PSF norm 0.11648239146773866 vs pixelized 0.103182286 average tim pixel scale / coadd scale: 1.0013845672551898 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.21', '2.06', '1.94'] Using average PSF sigma 2.0694272218059897 Gaussian PSF norm 0.13631539626104558 vs pixelized 0.12228018 average tim pixel scale / coadd scale: 1.0014219162452482 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.62', '2.18', '1.46'] Using average PSF sigma 1.755282874074223 Gaussian PSF norm 0.16071186926076603 vs pixelized 0.13221253 average tim pixel scale / coadd scale: 1.0013661635257587 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.17 s, CPU: 0.15 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 213 MB, VmData: 240 MB, maxrss: 0.218548 MB Stage fit_on_coadds finished: 2020-08-05T18:56:01.073917 Resources for stage fit_on_coadds : 0.190 s worker CPU, 0.190 s worker Wall, Wall: 0.176 s, Cores in use: 1.93, Total efficiency (on 16 cores): 12.1 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T18:56:01.087077 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 18 potential peaks Of 18 potential peaks: 7 in veto map, 1 cut by saddle test, 0 cut by aper test, 10 kept SED z : 10 new peaks Found 19 potential peaks Of 19 potential peaks: 14 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 19 potential peaks Of 19 potential peaks: 13 in veto map, 1 cut by saddle test, 0 cut by aper test, 5 kept SED g : 5 new peaks Found 29 potential peaks Of 29 potential peaks: 21 in veto map, 4 cut by saddle test, 0 cut by aper test, 4 kept SED Flat : 4 new peaks Found 32 potential peaks Of 32 potential peaks: 27 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept SED Red : 1 new peaks Adding 22 new sources Median coadd sky for g : -9.2003565e-06 Median coadd sky for r : 4.5100205e-05 Median coadd sky for z : 0.00070433307 Stage srcs : Wall: 0.65 s, CPU: 0.61 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 214 MB, VmData: 240 MB, maxrss: 0.21908 MB Stage srcs finished: 2020-08-05T18:56:01.740314 Resources for stage srcs : 0.082 s worker CPU, 0.082 s worker Wall, Wall: 0.655 s, Cores in use: 1.05, Total efficiency (on 16 cores): 6.6 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T18:56:02.291337 Blob 1 of 15, id: 4, sources: 10, size: 161x174, npix 12083, brick X: 78,239, Y: 43,217, one pixel: 133 43 Blob 2 of 15, id: 7, sources: 1, size: 48x50, npix 1742, brick X: 24,72, Y: 125,175, one pixel: 43 125 Blob 3 of 15, id: 9, sources: 2, size: 35x43, npix 902, brick X: 116,151, Y: 186,229, one pixel: 128 186 Blob 4 of 15, id: 11, sources: 3, size: 56x32, npix 896, brick X: 0,56, Y: 207,239, one pixel: 44 207 Blob 5 of 15, id: 6, sources: 1, size: 34x33, npix 746, brick X: 189,223, Y: 72,105, one pixel: 202 72 Blob 6 of 15, id: 8, sources: 1, size: 28x28, npix 500, brick X: 73,101, Y: 159,187, one pixel: 85 159 Blob 7 of 15, id: 12, sources: 1, size: 29x22, npix 399, brick X: 210,239, Y: 217,239, one pixel: 219 217 Blob 8 of 15, id: 3, sources: 1, size: 22x28, npix 396, brick X: 1,23, Y: 39,67, one pixel: 9 39 Blob 9 of 15, id: 0, sources: 1, size: 23x23, npix 317, brick X: 56,79, Y: 3,26, one pixel: 68 3 Blob 10 of 15, id: 10, sources: 1, size: 22x23, npix 315, brick X: 72,94, Y: 194,217, one pixel: 82 194 Blob 11 of 15, id: 1, sources: 1, size: 20x21, npix 249, brick X: 179,199, Y: 8,29, one pixel: 188 8 Blob 12 of 15, id: 2, sources: 1, size: 20x22, npix 246, brick X: 51,71, Y: 35,57, one pixel: 60 35 Blob 13 of 15, id: 5, sources: 1, size: 20x21, npix 240, brick X: 14,34, Y: 69,90, one pixel: 23 69 Blob 14 of 15, id: 13, sources: 1, size: 21x19, npix 237, brick X: 146,167, Y: 220,239, one pixel: 154 220 Blob 15 of 15, id: 14, sources: 1, size: 21x17, npix 234, brick X: 180,201, Y: 222,239, one pixel: 188 222 Blob 2 finished, total: Wall: 0.23 s, CPU: 0.23 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 108 MB, VmData: 166 MB, maxrss: 0.11094 MB Blob 11 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.095452 MB Blob 10 finished, total: Wall: 0.32 s, CPU: 0.32 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 181 MB, maxrss: 0.150148 MB Blob 13 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 962 MB, VmSize: 732 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146424 MB Blob 15 finished, total: Wall: 0.38 s, CPU: 0.38 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149748 MB Blob 5 finished, total: Wall: 0.64 s, CPU: 0.64 s, VmPeak: 955 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149696 MB Blob 9 finished, total: Wall: 0.64 s, CPU: 0.64 s, VmPeak: 951 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.14824 MB Blob 14 finished, total: Wall: 0.65 s, CPU: 0.65 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.095524 MB Blob 6 finished, total: Wall: 0.74 s, CPU: 0.74 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.15 MB Blob 7 finished, total: Wall: 0.74 s, CPU: 0.74 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 138 MB, maxrss: 0.093032 MB Blob 8 finished, total: Wall: 0.76 s, CPU: 0.76 s, VmPeak: 955 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 181 MB, maxrss: 0.15024 MB Blob 12 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 950 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148048 MB Blob 3 finished, total: Wall: 1.80 s, CPU: 1.80 s, VmPeak: 957 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149888 MB Measuring 1 iterative sources Blob 4 finished, total: Wall: 2.44 s, CPU: 2.44 s, VmPeak: 960 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 181 MB, maxrss: 0.150924 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.38306, ee1=-0.565663, ee2=0.446603 Measuring 1 iterative sources Blob 1 finished, total: Wall: 9.47 s, CPU: 9.46 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 126 MB, VmData: 166 MB, maxrss: 0.12928 MB Old catalog: 32 New catalog: 28 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/blobs-custom-128232p30514.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/all-models-custom-128232p30514.fits Stage fitblobs : Wall: 10.31 s, CPU: 0.81 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 214 MB, VmData: 240 MB, maxrss: 0.219244 MB Stage fitblobs finished: 2020-08-05T18:56:12.599445 Resources for stage fitblobs : 20.518 s worker CPU, 20.620 s worker Wall, Wall: 10.310 s, Cores in use: 2.07, Total efficiency (on 16 cores): 12.9 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T18:56:12.612039 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-resid.jpg Stage coadds : Wall: 2.33 s, CPU: 0.53 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 215 MB, VmData: 240 MB, maxrss: 0.21976 MB Stage coadds finished: 2020-08-05T18:56:14.946426 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 10.548 s worker CPU, 14.996 s worker Wall, Wall: 2.336 s, Cores in use: 4.74, Total efficiency (on 16 cores): 29.6 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T18:56:14.967837 Cut to 1 unWISE tiles Reading WISE tile 1286p302 band 1 Reading WISE tile 1286p302 band 2 Reading WISE tile 1286p302 band 3 Reading WISE tile 1286p302 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 956 MB, VmSize: 812 MB, VmRSS: 151 MB, VmData: 177 MB, maxrss: 0.197816 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 960 MB, VmSize: 809 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.212112 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 956 MB, VmSize: 809 MB, VmRSS: 149 MB, VmData: 174 MB, maxrss: 0.2136 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 806 MB, VmSize: 804 MB, VmRSS: 143 MB, VmData: 172 MB, maxrss: 0.206252 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 1.84 s, CPU: 0.45 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 215 MB, VmData: 240 MB, maxrss: 0.220464 MB Stage wise_forced finished: 2020-08-05T18:56:16.807690 Resources for stage wise_forced : 3.114 s worker CPU, 4.370 s worker Wall, Wall: 1.841 s, Cores in use: 1.94, Total efficiency (on 16 cores): 12.1 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T18:56:16.821979 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/tractor-i/cus/tractor-custom-128232p30514.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/tractor/cus/tractor-custom-128232p30514.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/tractor/cus/brick-custom-128232p30514.sha256sum Stage writecat : Wall: 1.38 s, CPU: 0.50 s, VmPeak: 1173 MB, VmSize: 1045 MB, VmRSS: 217 MB, VmData: 240 MB, maxrss: 0.35334 MB Stage writecat finished: 2020-08-05T18:56:18.203299 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.383 s, Cores in use: 0.37, Total efficiency (on 16 cores): 2.3 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-writecat.p All done: 42.058 s worker CPU, 875.403 s worker Wall, Wall: 168.108 s, Cores in use: 0.30, Total efficiency (on 16 cores): 1.9 % Finished galaxy PGC1906774 in 2.830 minutes. Started working on galaxy PGC1906774 at Wed Aug 5 19:22:04 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 128.23275 30.5142 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:22:08.145813 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '128.23275', '30.5142', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 128.23275 30.5142 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 128.23275 30.5142 Starting process 6276 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 6277 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 6278 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 6279 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 6280 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059532 MB Starting process 6281 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB Starting process 6282 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB Starting process 6283 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 6284 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 6285 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 6286 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 6287 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 6288 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 6289 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 6290 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 6291 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 6292 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 6293 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 6294 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 6295 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 6296 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 6297 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 6298 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 6299 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 6300 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 6301 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 6302 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 6303 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 6305 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 6304 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 6306 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 6307 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:22:09.276237 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: c4d_141228_065726_ooi_g_ls9-S21 g expnum 392430 exptime 87.0 propid 2014B-0404 seeing 1.55 MJD 57019.288 object DECaLS_10847_g decam/CP/V4.8.2a/CP20141227/c4d_141228_065726_ooi_g_ls9.fits.fz c4d_190303_030009_ooi_g_ls9-S21 g expnum 829482 exptime 171.0 propid 2014B-0404 seeing 1.36 MJD 58545.125 object DECaLS_10847_g decam/CP/V4.8.2a/CP20190302/c4d_190303_030009_ooi_g_ls9.fits.fz c4d_141228_065140_ooi_r_ls9-S21 r expnum 392427 exptime 71.0 propid 2014B-0404 seeing 1.36 MJD 57019.284 object DECaLS_10847_r decam/CP/V4.8.2a/CP20141227/c4d_141228_065140_ooi_r_ls9.fits.fz c4d_141231_052726_ooi_z_ls9-S21 z expnum 393585 exptime 125.0 propid 2014B-0404 seeing 1.00 MJD 57022.225 object DECaLS_10847_z decam/CP/V4.8.2a/CP20141230/c4d_141231_052726_ooi_z_ls9.fits.fz c4d_180213_035946_ooi_r_ls9-N24 r expnum 720791 exptime 138.0 propid 2014B-0404 seeing 1.27 MJD 58162.167 object DECaLS_26712_r decam/CP/V4.8.2a/CP20180212/c4d_180213_035946_ooi_r_ls9.fits.fz c4d_180212_042033_ooi_g_ls9-N24 g expnum 720472 exptime 167.0 propid 2014B-0404 seeing 1.47 MJD 58161.181 object DECaLS_26712_g decam/CP/V4.8.2a/CP20180211/c4d_180212_042033_ooi_g_ls9.fits.fz c4d_170327_001946_ooi_z_ls9-N24 z expnum 633948 exptime 245.0 propid 2014B-0404 seeing 1.35 MJD 57839.010 object DECaLS_26712_z decam/CP/V4.8.2/CP20170326/c4d_170327_001946_ooi_z_ls9.fits.fz c4d_181214_073256_ooi_g_ls9-N23 g expnum 803012 exptime 200.0 propid 2014B-0404 seeing 1.58 MJD 58466.315 object DECaLS_42584_g decam/CP/V4.8.2a/CP20181213/c4d_181214_073256_ooi_g_ls9.fits.fz c4d_181214_073109_ooi_r_ls9-N23 r expnum 803011 exptime 78.0 propid 2014B-0404 seeing 1.19 MJD 58466.313 object DECaLS_42584_r decam/CP/V4.8.2a/CP20181213/c4d_181214_073109_ooi_r_ls9.fits.fz c4d_181215_081002_ooi_z_ls9-N23 z expnum 803372 exptime 91.0 propid 2014B-0404 seeing 0.90 MJD 58467.340 object DECaLS_42584_z decam/CP/V4.8.2a/CP20181214/c4d_181215_081002_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95613b5-z-20141231T052432_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953a046-g-20190303T030009_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9552e01-g-20141228T065503_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953988f-g-20181214T073256_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95117e6-z-20170327T001453_oi[N24,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95506eb-r-20141228T064940_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9533681-z-20181215T081002_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a3cf-r-20180213T035946_oi[N24,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a2af-g-20180212T042033_oi[N24,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9539067-r-20181214T073109_oi[N23,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 51.84 s, CPU: 1.28 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 178 MB, VmData: 210 MB, maxrss: 0.181824 MB Stage tims finished: 2020-08-05T19:23:01.120166 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95613b5-z-20141231T052432_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953988f-g-20181214T073256_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9552e01-g-20141228T065503_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953a046-g-20190303T030009_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9539067-r-20181214T073109_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a3cf-r-20180213T035946_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a2af-g-20180212T042033_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.799 s worker CPU, 309.026 s worker Wall, Wall: 51.847 s, Cores in use: 0.14, Total efficiency (on 32 cores): 0.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95117e6-z-20170327T001453_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9533681-z-20181215T081002_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95506eb-r-20141228T064940_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T19:23:04.042800 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-03053.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02926.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/reference-custom-128232p30514.fits Stage refs : Wall: 8.29 s, CPU: 3.73 s, VmPeak: 1042 MB, VmSize: 1032 MB, VmRSS: 199 MB, VmData: 230 MB, maxrss: 0.203964 MB Stage refs finished: 2020-08-05T19:23:12.329614 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 8.289 s, Cores in use: 0.45, Total efficiency (on 32 cores): 1.4 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:23:12.435620 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outlier-mask-custom-128232p30514.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-pre-custom-128232p30514.jpg Image c4d_190303_030009_ooi_g_ls9-S21 g : masking 465 positive outlier pixels and 0 negative outlier pixels Image c4d_141228_065726_ooi_g_ls9-S21 g : masking 300 positive outlier pixels and 0 negative outlier pixels Image c4d_180213_035946_ooi_r_ls9-N24 r : masking 575 positive outlier pixels and 0 negative outlier pixels Image c4d_181215_081002_ooi_z_ls9-N23 z : masking 495 positive outlier pixels and 0 negative outlier pixels Image c4d_141231_052726_ooi_z_ls9-S21 z : masking 207 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outlier-mask-custom-128232p30514.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-post-custom-128232p30514.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-masked-pos-custom-128232p30514.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/outliers-masked-neg-custom-128232p30514.jpg Stage outliers : Wall: 2.93 s, CPU: 2.16 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 212 MB, VmData: 240 MB, maxrss: 0.217444 MB Stage outliers finished: 2020-08-05T19:23:15.364205 Resources for stage outliers : 0.900 s worker CPU, 1.599 s worker Wall, Wall: 2.931 s, Cores in use: 1.05, Total efficiency (on 32 cores): 3.3 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:23:15.445575 Stage halos : Wall: 1.52 s, CPU: 1.24 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 213 MB, VmData: 240 MB, maxrss: 0.217708 MB Stage halos finished: 2020-08-05T19:23:16.962651 Resources for stage halos : 0.926 s worker CPU, 1.920 s worker Wall, Wall: 1.519 s, Cores in use: 1.43, Total efficiency (on 32 cores): 4.5 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T19:23:17.041984 PSF sigmas in coadd pixels: ['2.52', '2.21', '2.39', '2.57'] Using average PSF sigma 2.4217805645929587 Gaussian PSF norm 0.11648239146773866 vs pixelized 0.103182286 average tim pixel scale / coadd scale: 1.0013845672551898 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.21', '2.06', '1.94'] Using average PSF sigma 2.0694272218059897 Gaussian PSF norm 0.13631539626104558 vs pixelized 0.12228018 average tim pixel scale / coadd scale: 1.0014219162452482 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.62', '2.18', '1.46'] Using average PSF sigma 1.755282874074223 Gaussian PSF norm 0.16071186926076603 vs pixelized 0.13221253 average tim pixel scale / coadd scale: 1.0013661635257587 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.19 s, CPU: 0.14 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 214 MB, VmData: 240 MB, maxrss: 0.2193 MB Stage fit_on_coadds finished: 2020-08-05T19:23:17.227137 Resources for stage fit_on_coadds : 0.182 s worker CPU, 0.182 s worker Wall, Wall: 0.187 s, Cores in use: 1.75, Total efficiency (on 32 cores): 5.5 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T19:23:17.239367 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 18 potential peaks Of 18 potential peaks: 7 in veto map, 1 cut by saddle test, 0 cut by aper test, 10 kept SED z : 10 new peaks Found 19 potential peaks Of 19 potential peaks: 14 in veto map, 3 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 19 potential peaks Of 19 potential peaks: 13 in veto map, 1 cut by saddle test, 0 cut by aper test, 5 kept SED g : 5 new peaks Found 29 potential peaks Of 29 potential peaks: 21 in veto map, 4 cut by saddle test, 0 cut by aper test, 4 kept SED Flat : 4 new peaks Found 32 potential peaks Of 32 potential peaks: 27 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept SED Red : 1 new peaks Adding 22 new sources Median coadd sky for g : -9.20033e-06 Median coadd sky for r : 4.5100205e-05 Median coadd sky for z : 0.00070433307 Stage srcs : Wall: 1.18 s, CPU: 1.11 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 215 MB, VmData: 240 MB, maxrss: 0.219832 MB Stage srcs finished: 2020-08-05T19:23:18.421222 Resources for stage srcs : 0.071 s worker CPU, 0.071 s worker Wall, Wall: 1.184 s, Cores in use: 1.00, Total efficiency (on 32 cores): 3.1 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:23:18.442790 Blob 1 of 15, id: 4, sources: 10, size: 161x174, npix 12083, brick X: 78,239, Y: 43,217, one pixel: 133 43 Blob 2 of 15, id: 7, sources: 1, size: 48x50, npix 1742, brick X: 24,72, Y: 125,175, one pixel: 43 125 Blob 3 of 15, id: 9, sources: 2, size: 35x43, npix 902, brick X: 116,151, Y: 186,229, one pixel: 128 186 Blob 4 of 15, id: 11, sources: 3, size: 56x32, npix 896, brick X: 0,56, Y: 207,239, one pixel: 44 207 Blob 5 of 15, id: 6, sources: 1, size: 34x33, npix 746, brick X: 189,223, Y: 72,105, one pixel: 202 72 Blob 6 of 15, id: 8, sources: 1, size: 28x28, npix 500, brick X: 73,101, Y: 159,187, one pixel: 85 159 Blob 7 of 15, id: 12, sources: 1, size: 29x22, npix 399, brick X: 210,239, Y: 217,239, one pixel: 219 217 Blob 8 of 15, id: 3, sources: 1, size: 22x28, npix 396, brick X: 1,23, Y: 39,67, one pixel: 9 39 Blob 9 of 15, id: 0, sources: 1, size: 23x23, npix 317, brick X: 56,79, Y: 3,26, one pixel: 68 3 Blob 10 of 15, id: 10, sources: 1, size: 22x23, npix 315, brick X: 72,94, Y: 194,217, one pixel: 82 194 Blob 11 of 15, id: 1, sources: 1, size: 20x21, npix 249, brick X: 179,199, Y: 8,29, one pixel: 188 8 Blob 12 of 15, id: 2, sources: 1, size: 20x22, npix 246, brick X: 51,71, Y: 35,57, one pixel: 60 35 Blob 13 of 15, id: 5, sources: 1, size: 20x21, npix 240, brick X: 14,34, Y: 69,90, one pixel: 23 69 Blob 14 of 15, id: 13, sources: 1, size: 21x19, npix 237, brick X: 146,167, Y: 220,239, one pixel: 154 220 Blob 15 of 15, id: 14, sources: 1, size: 21x17, npix 234, brick X: 180,201, Y: 222,239, one pixel: 188 222 Blob 11 finished, total: Wall: 0.17 s, CPU: 0.16 s, VmPeak: 960 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149896 MB Blob 2 finished, total: Wall: 0.20 s, CPU: 0.18 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 86 MB, VmData: 138 MB, maxrss: 0.0885 MB Blob 10 finished, total: Wall: 0.27 s, CPU: 0.25 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 112 MB, VmData: 166 MB, maxrss: 0.115168 MB Blob 15 finished, total: Wall: 0.30 s, CPU: 0.29 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 112 MB, VmData: 166 MB, maxrss: 0.114912 MB Blob 13 finished, total: Wall: 0.32 s, CPU: 0.31 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.093768 MB Blob 9 finished, total: Wall: 0.50 s, CPU: 0.48 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092628 MB Blob 14 finished, total: Wall: 0.56 s, CPU: 0.54 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.094404 MB Blob 7 finished, total: Wall: 0.59 s, CPU: 0.58 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 112 MB, VmData: 165 MB, maxrss: 0.115032 MB Blob 5 finished, total: Wall: 0.65 s, CPU: 0.65 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.147832 MB Blob 12 finished, total: Wall: 0.63 s, CPU: 0.62 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.09658 MB Blob 6 finished, total: Wall: 0.68 s, CPU: 0.67 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.0958 MB Blob 8 finished, total: Wall: 0.70 s, CPU: 0.69 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 113 MB, VmData: 166 MB, maxrss: 0.11546 MB Blob 3 finished, total: Wall: 1.72 s, CPU: 1.71 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 113 MB, VmData: 166 MB, maxrss: 0.115356 MB Measuring 1 iterative sources Blob 4 finished, total: Wall: 2.42 s, CPU: 2.41 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.095416 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.38306, ee1=-0.565664, ee2=0.446603 Measuring 1 iterative sources Blob 1 finished, total: Wall: 9.76 s, CPU: 9.75 s, VmPeak: 688 MB, VmSize: 686 MB, VmRSS: 113 MB, VmData: 158 MB, maxrss: 0.115816 MB Old catalog: 32 New catalog: 28 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/blobs-custom-128232p30514.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/metrics/cus/all-models-custom-128232p30514.fits Stage fitblobs : Wall: 16.88 s, CPU: 1.30 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 215 MB, VmData: 240 MB, maxrss: 0.219996 MB Stage fitblobs finished: 2020-08-05T19:23:35.322399 Resources for stage fitblobs : 19.520 s worker CPU, 20.644 s worker Wall, Wall: 16.882 s, Cores in use: 1.23, Total efficiency (on 32 cores): 3.9 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:23:35.356893 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-resid.jpg Stage coadds : Wall: 5.48 s, CPU: 2.24 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 215 MB, VmData: 240 MB, maxrss: 0.220512 MB Stage coadds finished: 2020-08-05T19:23:40.835679 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 26.474 s worker CPU, 55.237 s worker Wall, Wall: 5.481 s, Cores in use: 5.24, Total efficiency (on 32 cores): 16.4 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T19:23:40.852134 Cut to 1 unWISE tiles Reading WISE tile 1286p302 band 1 Reading WISE tile 1286p302 band 2 Reading WISE tile 1286p302 band 3 Reading WISE tile 1286p302 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 802 MB, VmSize: 802 MB, VmRSS: 142 MB, VmData: 174 MB, maxrss: 0.181216 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 955 MB, VmSize: 805 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.208504 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 956 MB, VmSize: 809 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.2109 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 864 MB, VmSize: 864 MB, VmRSS: 204 MB, VmData: 236 MB, maxrss: 0.20874 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/tmp-legacysurvey-custom-128232p30514-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.37 s, CPU: 0.79 s, VmPeak: 1042 MB, VmSize: 1041 MB, VmRSS: 216 MB, VmData: 240 MB, maxrss: 0.221216 MB Stage wise_forced finished: 2020-08-05T19:23:43.226628 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.512 s worker CPU, 4.778 s worker Wall, Wall: 2.376 s, Cores in use: 1.81, Total efficiency (on 32 cores): 5.7 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-wise_forced.p Running stage writecat WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage writecat at 2020-08-05T19:23:43.256920 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/coadd/cus/custom-128232p30514/legacysurvey-custom-128232p30514-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/tractor-i/cus/tractor-custom-128232p30514.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/tractor/cus/tractor-custom-128232p30514.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/tractor/cus/brick-custom-128232p30514.sha256sum Stage writecat : Wall: 1.33 s, CPU: 0.96 s, VmPeak: 1173 MB, VmSize: 1045 MB, VmRSS: 218 MB, VmData: 240 MB, maxrss: 0.354092 MB Stage writecat finished: 2020-08-05T19:23:44.585585 Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 1.331 s, Cores in use: 0.72, Total efficiency (on 32 cores): 2.3 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/128/PGC1906774/PGC1906774-largegalaxy-writecat.p All done: 57.391 s worker CPU, 393.464 s worker Wall, Wall: 95.334 s, Cores in use: 0.76, Total efficiency (on 32 cores): 2.4 % Finished galaxy PGC1906774 in 1.683 minutes. Started working on galaxy PGC1906774 at Fri Aug 7 09:20:30 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 128.23191255153512 30.514379511955113 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-07T09:20:31.982641 legacypipe git version: DR9.5.11-watershed-3-gd509eaaf Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '128.23191255153512', '30.514379511955113', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 128.23191255153512 30.514379511955113 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 128.23191255153512 30.514379511955113 Starting process 5877 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 5878 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 5879 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 5880 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 5881 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059404 MB Starting process 5882 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059412 MB Starting process 5883 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059412 MB Starting process 5884 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 5885 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 5886 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 5887 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 5888 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 5889 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 5890 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 5891 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 5892 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-07T09:20:33.767356 Keeping 10 CCDs: c4d_141228_065726_ooi_g_ls9-S21 g expnum 392430 exptime 87.0 propid 2014B-0404 seeing 1.55 MJD 57019.288 object DECaLS_10847_g decam/CP/V4.8.2a/CP20141227/c4d_141228_065726_ooi_g_ls9.fits.fz c4d_190303_030009_ooi_g_ls9-S21 g expnum 829482 exptime 171.0 propid 2014B-0404 seeing 1.36 MJD 58545.125 object DECaLS_10847_g decam/CP/V4.8.2a/CP20190302/c4d_190303_030009_ooi_g_ls9.fits.fz c4d_141228_065140_ooi_r_ls9-S21 r expnum 392427 exptime 71.0 propid 2014B-0404 seeing 1.36 MJD 57019.284 object DECaLS_10847_r decam/CP/V4.8.2a/CP20141227/c4d_141228_065140_ooi_r_ls9.fits.fz c4d_141231_052726_ooi_z_ls9-S21 z expnum 393585 exptime 125.0 propid 2014B-0404 seeing 1.00 MJD 57022.225 object DECaLS_10847_z decam/CP/V4.8.2a/CP20141230/c4d_141231_052726_ooi_z_ls9.fits.fz c4d_180213_035946_ooi_r_ls9-N24 r expnum 720791 exptime 138.0 propid 2014B-0404 seeing 1.27 MJD 58162.167 object DECaLS_26712_r decam/CP/V4.8.2a/CP20180212/c4d_180213_035946_ooi_r_ls9.fits.fz c4d_180212_042033_ooi_g_ls9-N24 g expnum 720472 exptime 167.0 propid 2014B-0404 seeing 1.47 MJD 58161.181 object DECaLS_26712_g decam/CP/V4.8.2a/CP20180211/c4d_180212_042033_ooi_g_ls9.fits.fz c4d_170327_001946_ooi_z_ls9-N24 z expnum 633948 exptime 245.0 propid 2014B-0404 seeing 1.35 MJD 57839.010 object DECaLS_26712_z decam/CP/V4.8.2/CP20170326/c4d_170327_001946_ooi_z_ls9.fits.fz c4d_181214_073256_ooi_g_ls9-N23 g expnum 803012 exptime 200.0 propid 2014B-0404 seeing 1.58 MJD 58466.315 object DECaLS_42584_g decam/CP/V4.8.2a/CP20181213/c4d_181214_073256_ooi_g_ls9.fits.fz c4d_181214_073109_ooi_r_ls9-N23 r expnum 803011 exptime 78.0 propid 2014B-0404 seeing 1.19 MJD 58466.313 object DECaLS_42584_r decam/CP/V4.8.2a/CP20181213/c4d_181214_073109_ooi_r_ls9.fits.fz c4d_181215_081002_ooi_z_ls9-N23 z expnum 803372 exptime 91.0 propid 2014B-0404 seeing 0.90 MJD 58467.340 object DECaLS_42584_z decam/CP/V4.8.2a/CP20181214/c4d_181215_081002_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953a046-g-20190303T030009_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95613b5-z-20141231T052432_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953988f-g-20181214T073256_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9533681-z-20181215T081002_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95117e6-z-20170327T001453_oi[N24,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9539067-r-20181214T073109_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9552e01-g-20141228T065503_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95506eb-r-20141228T064940_oi[S21,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a2af-g-20180212T042033_oi[N24,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a3cf-r-20180213T035946_oi[N24,noinherit][2046,4094][real]: DECaLS_ Stage tims : Wall: 129.64 s, CPU: 1.71 s, VmPeak: 1005 MB, VmSize: 1005 MB, VmRSS: 176 MB, VmData: 209 MB, maxrss: 0.17996 MB Stage tims finished: 2020-08-07T09:22:43.406918 Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95613b5-z-20141231T052432_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9539067-r-20181214T073109_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953988f-g-20181214T073256_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9552e01-g-20141228T065503_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a3cf-r-20180213T035946_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9533681-z-20181215T081002_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Resources for stage tims : 5.551 s worker CPU, 247.357 s worker Wall, Wall: 129.641 s, Cores in use: 0.06, Total efficiency (on 16 cores): 0.3 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-tims.p Running stage refs Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_953a046-g-20190303T030009_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95117e6-z-20170327T001453_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95506eb-r-20141228T064940_oi[S21,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_952a2af-g-20180212T042033_oi[N24,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Running stage refs at 2020-08-07T09:22:43.566573 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-03053.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-02926.fits Matched 8 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/reference-custom-128231p30514.fits Stage refs : Wall: 2.29 s, CPU: 1.98 s, VmPeak: 1035 MB, VmSize: 1025 MB, VmRSS: 197 MB, VmData: 230 MB, maxrss: 0.201892 MB Stage refs finished: 2020-08-07T09:22:45.858965 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 2.294 s, Cores in use: 0.87, Total efficiency (on 16 cores): 5.4 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-07T09:22:45.882116 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/outlier-mask-custom-128231p30514.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/outliers-pre-custom-128231p30514.jpg Image c4d_141228_065726_ooi_g_ls9-S21 g : masking 291 positive outlier pixels and 0 negative outlier pixels Image c4d_190303_030009_ooi_g_ls9-S21 g : masking 482 positive outlier pixels and 0 negative outlier pixels Image c4d_180212_042033_ooi_g_ls9-N24 g : masking 65 positive outlier pixels and 0 negative outlier pixels Image c4d_141228_065140_ooi_r_ls9-S21 r : masking 542 positive outlier pixels and 54 negative outlier pixels Image c4d_180213_035946_ooi_r_ls9-N24 r : masking 584 positive outlier pixels and 0 negative outlier pixels Image c4d_181214_073109_ooi_r_ls9-N23 r : masking 94 positive outlier pixels and 0 negative outlier pixels Image c4d_141231_052726_ooi_z_ls9-S21 z : masking 150 positive outlier pixels and 0 negative outlier pixels Image c4d_181215_081002_ooi_z_ls9-N23 z : masking 186 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/outlier-mask-custom-128231p30514.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/outliers-post-custom-128231p30514.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/outliers-masked-pos-custom-128231p30514.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/outliers-masked-neg-custom-128231p30514.jpg Stage outliers : Wall: 0.85 s, CPU: 0.63 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 210 MB, VmData: 239 MB, maxrss: 0.215096 MB Stage outliers finished: 2020-08-07T09:22:46.735327 Resources for stage outliers : 0.841 s worker CPU, 0.904 s worker Wall, Wall: 0.855 s, Cores in use: 1.72, Total efficiency (on 16 cores): 10.7 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-07T09:22:46.772816 Stage halos : Wall: 8.54 s, CPU: 0.28 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 210 MB, VmData: 239 MB, maxrss: 0.215228 MB Stage halos finished: 2020-08-07T09:22:55.314185 Resources for stage halos : 0.777 s worker CPU, 9.522 s worker Wall, Wall: 8.543 s, Cores in use: 0.12, Total efficiency (on 16 cores): 0.8 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-07T09:23:01.916466 PSF sigmas in coadd pixels: ['2.52', '2.21', '2.39', '2.57'] Using average PSF sigma 2.4217805645929587 Gaussian PSF norm 0.11648239146773866 vs pixelized 0.10318463 average tim pixel scale / coadd scale: 1.0013845672551898 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-copsf-g.fits.fz PSF sigmas in coadd pixels: ['2.21', '2.06', '1.94'] Using average PSF sigma 2.0694272218059897 Gaussian PSF norm 0.13631539626104558 vs pixelized 0.12229058 average tim pixel scale / coadd scale: 1.0014219162452482 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.62', '2.18', '1.46'] Using average PSF sigma 1.755282874074223 Gaussian PSF norm 0.16071186926076603 vs pixelized 0.13221954 average tim pixel scale / coadd scale: 1.0013661635257587 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-copsf-z.fits.fz Stage fit_on_coadds : Wall: 1.54 s, CPU: 0.16 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 213 MB, VmData: 241 MB, maxrss: 0.218364 MB Stage fit_on_coadds finished: 2020-08-07T09:23:03.458512 Resources for stage fit_on_coadds : 0.191 s worker CPU, 0.191 s worker Wall, Wall: 1.544 s, Cores in use: 0.23, Total efficiency (on 16 cores): 1.4 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-07T09:23:06.519802 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 22 potential peaks Of 22 potential peaks: 12 in veto map, 1 cut by saddle test, 0 cut by aper test, 9 kept SED z : 9 new peaks Found 20 potential peaks Of 20 potential peaks: 14 in veto map, 2 cut by saddle test, 0 cut by aper test, 4 kept SED r : 4 new peaks Found 21 potential peaks Of 21 potential peaks: 14 in veto map, 2 cut by saddle test, 0 cut by aper test, 5 kept SED g : 5 new peaks Found 29 potential peaks Of 29 potential peaks: 23 in veto map, 3 cut by saddle test, 0 cut by aper test, 3 kept SED Flat : 3 new peaks Found 32 potential peaks Of 32 potential peaks: 28 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED Red : 1 new peaks Adding 22 new sources Median coadd sky for g : -1.258786e-05 Median coadd sky for r : 7.232046e-05 Median coadd sky for z : 0.0005937933 Stage srcs : Wall: 0.48 s, CPU: 0.45 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 214 MB, VmData: 241 MB, maxrss: 0.218908 MB Stage srcs finished: 2020-08-07T09:23:07.000583 Resources for stage srcs : 0.068 s worker CPU, 0.068 s worker Wall, Wall: 0.482 s, Cores in use: 1.08, Total efficiency (on 16 cores): 6.7 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-07T09:23:07.013164 Blob 1 of 15, id: 3, sources: 9, size: 170x174, npix 12965, brick X: 69,239, Y: 41,215, one pixel: 122 41 Blob 2 of 15, id: 7, sources: 1, size: 48x50, npix 1741, brick X: 14,62, Y: 122,172, one pixel: 39 122 Blob 3 of 15, id: 9, sources: 3, size: 35x55, npix 1122, brick X: 106,141, Y: 184,239, one pixel: 117 184 Blob 4 of 15, id: 11, sources: 2, size: 45x34, npix 823, brick X: 1,46, Y: 205,239, one pixel: 33 205 Blob 5 of 15, id: 6, sources: 1, size: 34x33, npix 747, brick X: 179,213, Y: 69,102, one pixel: 193 69 Blob 6 of 15, id: 8, sources: 1, size: 28x28, npix 509, brick X: 63,91, Y: 156,184, one pixel: 76 156 Blob 7 of 15, id: 12, sources: 1, size: 33x25, npix 479, brick X: 200,233, Y: 214,239, one pixel: 210 214 Blob 8 of 15, id: 13, sources: 2, size: 34x22, npix 393, brick X: 136,170, Y: 217,239, one pixel: 144 217 Blob 9 of 15, id: 0, sources: 1, size: 23x23, npix 317, brick X: 46,69, Y: 1,24, one pixel: 57 1 Blob 10 of 15, id: 10, sources: 1, size: 22x24, npix 297, brick X: 62,84, Y: 191,215, one pixel: 73 191 Blob 11 of 15, id: 14, sources: 1, size: 22x20, npix 266, brick X: 170,192, Y: 219,239, one pixel: 179 219 Blob 12 of 15, id: 1, sources: 1, size: 20x22, npix 260, brick X: 169,189, Y: 5,27, one pixel: 178 5 Blob 13 of 15, id: 5, sources: 1, size: 20x20, npix 247, brick X: 4,24, Y: 67,87, one pixel: 12 67 Blob 14 of 15, id: 2, sources: 1, size: 20x22, npix 245, brick X: 41,61, Y: 32,54, one pixel: 50 32 Blob 15 of 15, id: 4, sources: 1, size: 15x21, npix 214, brick X: 224,239, Y: 61,82, one pixel: 233 61 Measuring 1 iterative sources Blob 12 finished, total: Wall: 0.26 s, CPU: 0.25 s, VmPeak: 956 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.143024 MB Blob 10 finished, total: Wall: 0.39 s, CPU: 0.39 s, VmPeak: 962 MB, VmSize: 725 MB, VmRSS: 141 MB, VmData: 180 MB, maxrss: 0.14446 MB Blob 13 finished, total: Wall: 0.39 s, CPU: 0.39 s, VmPeak: 950 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 180 MB, maxrss: 0.146124 MB Blob 11 finished, total: Wall: 0.44 s, CPU: 0.44 s, VmPeak: 960 MB, VmSize: 725 MB, VmRSS: 139 MB, VmData: 180 MB, maxrss: 0.142408 MB Blob 9 finished, total: Wall: 0.61 s, CPU: 0.61 s, VmPeak: 955 MB, VmSize: 725 MB, VmRSS: 141 MB, VmData: 180 MB, maxrss: 0.143992 MB Blob 14 finished, total: Wall: 0.61 s, CPU: 0.60 s, VmPeak: 956 MB, VmSize: 725 MB, VmRSS: 141 MB, VmData: 180 MB, maxrss: 0.144836 MB Blob 8 finished, total: Wall: 0.75 s, CPU: 0.74 s, VmPeak: 956 MB, VmSize: 725 MB, VmRSS: 141 MB, VmData: 180 MB, maxrss: 0.144044 MB Blob 7 finished, total: Wall: 0.78 s, CPU: 0.77 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 113 MB, VmData: 166 MB, maxrss: 0.115332 MB Blob 6 finished, total: Wall: 0.83 s, CPU: 0.83 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.096512 MB Blob 15 finished, total: Wall: 0.90 s, CPU: 0.89 s, VmPeak: 957 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 180 MB, maxrss: 0.14508 MB Blob 5 finished, total: Wall: 0.95 s, CPU: 0.94 s, VmPeak: 955 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 180 MB, maxrss: 0.145084 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 1.63 s, CPU: 1.63 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 113 MB, VmData: 166 MB, maxrss: 0.115856 MB Blob 4 finished, total: Wall: 1.91 s, CPU: 1.91 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 114 MB, VmData: 166 MB, maxrss: 0.116324 MB Blob 3 finished, total: Wall: 2.17 s, CPU: 2.17 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 113 MB, VmData: 166 MB, maxrss: 0.115564 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=0.572357, ee1=-0.439831, ee2=0.361498 Blob 1 finished, total: Wall: 6.27 s, CPU: 6.27 s, VmPeak: 951 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146496 MB Old catalog: 32 New catalog: 27 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/blobs-custom-128231p30514.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/metrics/cus/all-models-custom-128231p30514.fits Stage fitblobs : Wall: 6.64 s, CPU: 0.36 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 214 MB, VmData: 241 MB, maxrss: 0.219072 MB Stage fitblobs finished: 2020-08-07T09:23:13.657159 Resources for stage fitblobs : 19.081 s worker CPU, 19.185 s worker Wall, Wall: 6.645 s, Cores in use: 2.93, Total efficiency (on 16 cores): 18.3 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-07T09:23:13.668901 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/tmp-legacysurvey-custom-128231p30514-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/tmp-legacysurvey-custom-128231p30514-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/tmp-legacysurvey-custom-128231p30514-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/tmp-legacysurvey-custom-128231p30514-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-resid.jpg Stage coadds : Wall: 1.71 s, CPU: 0.58 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 214 MB, VmData: 241 MB, maxrss: 0.219584 MB Stage coadds finished: 2020-08-07T09:23:15.379157 Resources for stage coadds : 10.223 s worker CPU, 10.778 s worker Wall, Wall: 1.712 s, Cores in use: 6.31, Total efficiency (on 16 cores): 39.4 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-07T09:23:15.390720 Cut to 1 unWISE tiles Reading WISE tile 1286p302 band 2 Reading WISE tile 1286p302 band 1 Reading WISE tile 1286p302 band 3 Reading WISE tile 1286p302 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 957 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.208204 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 955 MB, VmSize: 805 MB, VmRSS: 148 MB, VmData: 176 MB, maxrss: 0.192636 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 951 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.209052 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/128/1286p302/unwise-1286p302-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 950 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.207876 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/tmp-legacysurvey-custom-128231p30514-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/tmp-legacysurvey-custom-128231p30514-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 3.88 s, CPU: 0.42 s, VmPeak: 1035 MB, VmSize: 1035 MB, VmRSS: 215 MB, VmData: 241 MB, maxrss: 0.220292 MB Stage wise_forced finished: 2020-08-07T09:23:19.267778 Resources for stage wise_forced : 2.949 s worker CPU, 11.692 s worker Wall, Wall: 3.878 s, Cores in use: 0.87, Total efficiency (on 16 cores): 5.4 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-07T09:23:19.312470 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/coadd/cus/custom-128231p30514/legacysurvey-custom-128231p30514-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/tractor-i/cus/tractor-custom-128231p30514.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/tractor/cus/tractor-custom-128231p30514.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/tractor/cus/brick-custom-128231p30514.sha256sum Stage writecat : Wall: 0.68 s, CPU: 0.51 s, VmPeak: 1167 MB, VmSize: 1039 MB, VmRSS: 217 MB, VmData: 241 MB, maxrss: 0.353164 MB Stage writecat finished: 2020-08-07T09:23:19.994851 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.684 s, Cores in use: 0.75, Total efficiency (on 16 cores): 4.7 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/128/PGC1906774/PGC1906774-largegalaxy-writecat.p All done: 39.683 s worker CPU, 299.700 s worker Wall, Wall: 166.256 s, Cores in use: 0.28, Total efficiency (on 16 cores): 1.8 % Finished galaxy PGC1906774 in 2.827 minutes.