Started working on galaxy PGC1318681 at Wed Aug 5 19:26:51 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.897982 7.3067036 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:26:53.572419 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '323.897982', '7.3067036', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.897982 7.3067036 --width 239 --height 239 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 323.897982 7.3067036 Starting process 37644 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 37646 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 37648 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 37650 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 37652 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 37654 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 37656 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 37658 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 37660 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 37662 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 37664 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 37666 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 37668 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 37670 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 37672 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 37674 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:26:54.115003 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: c4d_140814_041842_ooi_z_ls9-S25 z expnum 347705 exptime 238.0 propid 2014B-0404 seeing 0.90 MJD 56883.176 object DECaLS_43591_z decam/CP/V4.8.2a/CP20140813/c4d_140814_041842_ooi_z_ls9.fits.fz c4d_140815_031959_ooi_r_ls9-S25 r expnum 347967 exptime 85.0 propid 2014B-0404 seeing 1.20 MJD 56884.137 object DECaLS_43591_r decam/CP/V4.8.2a/CP20140814/c4d_140815_031959_ooi_r_ls9.fits.fz c4d_151011_010312_ooi_g_ls9-S25 g expnum 483633 exptime 72.0 propid 2014B-0404 seeing 1.38 MJD 57306.042 object DECaLS_43591_g decam/CP/V4.8.2a/CP20151010/c4d_151011_010312_ooi_g_ls9.fits.fz c4d_180516_091924_ooi_g_ls9-S30 g expnum 746811 exptime 200.0 propid 2014B-0404 seeing 1.34 MJD 58254.388 object DECaLS_11847_g decam/CP/V4.8.2a/CP20180515/c4d_180516_091924_ooi_g_ls9.fits.fz c4d_181006_235957_ooi_z_ls9-N28 z expnum 781250 exptime 80.0 propid 2014B-0404 seeing 0.93 MJD 58398.000 object DECaLS_43592_z decam/CP/V4.8.2a/CP20181006/c4d_181006_235957_ooi_z_ls9.fits.fz c4d_140815_032350_ooi_r_ls9-N28 r expnum 347969 exptime 85.0 propid 2014B-0404 seeing 1.08 MJD 56884.140 object DECaLS_43592_r decam/CP/V4.8.2a/CP20140814/c4d_140815_032350_ooi_r_ls9.fits.fz c4d_151011_005943_ooi_g_ls9-N28 g expnum 483631 exptime 71.0 propid 2014B-0404 seeing 1.25 MJD 57306.040 object DECaLS_43592_g decam/CP/V4.8.2a/CP20151010/c4d_151011_005943_ooi_g_ls9.fits.fz c4d_140817_035750_ooi_g_ls9-N20 g expnum 348778 exptime 87.0 propid 2014B-0404 seeing 1.42 MJD 56886.164 object DECaLS_11867_g decam/CP/V4.8.2a/CP20140816/c4d_140817_035750_ooi_g_ls9.fits.fz c4d_140816_034713_ooi_r_ls9-N20 r expnum 348334 exptime 66.0 propid 2014B-0404 seeing 0.96 MJD 56885.156 object DECaLS_11867_r decam/CP/V4.8.2a/CP20140815/c4d_140816_034713_ooi_r_ls9.fits.fz c4d_140816_044100_ooi_z_ls9-N20 z expnum 348356 exptime 115.0 propid 2014B-0404 seeing 0.94 MJD 56885.193 object DECaLS_11867_z decam/CP/V4.8.2a/CP20140815/c4d_140816_044100_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T010110_oi[S25,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0582126 vs median 0.0439778 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140816T034515_oi[N20,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530281-g-20180516T091924_oi[S30,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T005741_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T031743_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c6e6-z-20140814T041350_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T032133_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954a39a-g-20140817T035529_oi[N20,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0506858 vs median 0.0389228 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954d16a-z-20140816T043811_oi[N20,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9531140-z-20181006T235957_oi[N28,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 104.45 s, CPU: 1.01 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 176 MB, VmData: 208 MB, maxrss: 0.179856 MB Stage tims finished: 2020-08-05T19:28:38.561051 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c6e6-z-20140814T041350_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T031743_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T032133_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954a39a-g-20140817T035529_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954d16a-z-20140816T043811_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9531140-z-20181006T235957_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T005741_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 6.084 s worker CPU, 429.018 s worker Wall, Wall: 104.448 s, Cores in use: 0.07, Total efficiency (on 16 cores): 0.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T010110_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140816T034515_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T19:28:38.649780 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05426.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05427.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05555.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05299.fits Matched 2 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/reference-custom-323897p07306.fits Stage refs : Wall: 12.33 s, CPU: 5.60 s, VmPeak: 1064 MB, VmSize: 1058 MB, VmRSS: 224 MB, VmData: 254 MB, maxrss: 0.229652 MB Stage refs finished: 2020-08-05T19:28:50.983704 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530281-g-20180516T091924_oi[S30,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 12.336 s, Cores in use: 0.45, Total efficiency (on 16 cores): 2.8 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:28:51.126892 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outlier-mask-custom-323897p07306.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-pre-custom-323897p07306.jpg Image c4d_151011_010312_ooi_g_ls9-S25 g : masking 0 positive outlier pixels and 101 negative outlier pixels Image c4d_140816_034713_ooi_r_ls9-N20 r : masking 232 positive outlier pixels and 0 negative outlier pixels Image c4d_140815_031959_ooi_r_ls9-S25 r : masking 86 positive outlier pixels and 62 negative outlier pixels Image c4d_140815_032350_ooi_r_ls9-N28 r : masking 55 positive outlier pixels and 0 negative outlier pixels Image c4d_140814_041842_ooi_z_ls9-S25 z : masking 1981 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outlier-mask-custom-323897p07306.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-post-custom-323897p07306.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-masked-pos-custom-323897p07306.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-masked-neg-custom-323897p07306.jpg Stage outliers : Wall: 1.20 s, CPU: 0.96 s, VmPeak: 1067 MB, VmSize: 1067 MB, VmRSS: 235 MB, VmData: 263 MB, maxrss: 0.24094 MB Stage outliers finished: 2020-08-05T19:28:52.328005 Resources for stage outliers : 0.522 s worker CPU, 0.614 s worker Wall, Wall: 1.202 s, Cores in use: 1.23, Total efficiency (on 16 cores): 7.7 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:28:52.412124 Stage halos : Wall: 0.99 s, CPU: 0.48 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 236 MB, VmData: 263 MB, maxrss: 0.241216 MB Stage halos finished: 2020-08-05T19:28:53.404674 Resources for stage halos : 0.211 s worker CPU, 1.293 s worker Wall, Wall: 0.994 s, Cores in use: 0.70, Total efficiency (on 16 cores): 4.4 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T19:28:53.537410 PSF sigmas in coadd pixels: ['2.25', '2.18', '2.02', '2.31'] Using average PSF sigma 2.1892025510017348 Gaussian PSF norm 0.12885732827453417 vs pixelized 0.111106046 average tim pixel scale / coadd scale: 1.0011116605259136 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.95', '1.76', '1.56'] Using average PSF sigma 1.7563636611203257 Gaussian PSF norm 0.16061297441894198 vs pixelized 0.13698794 average tim pixel scale / coadd scale: 1.0011692794452187 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.46', '1.51', '1.53'] Using average PSF sigma 1.4988530491535428 Gaussian PSF norm 0.18820710404744978 vs pixelized 0.14907776 average tim pixel scale / coadd scale: 1.001065572055469 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.17 s, CPU: 0.11 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 237 MB, VmData: 263 MB, maxrss: 0.24268 MB Stage fit_on_coadds finished: 2020-08-05T19:28:53.709268 Resources for stage fit_on_coadds : 0.118 s worker CPU, 0.118 s worker Wall, Wall: 0.173 s, Cores in use: 1.32, Total efficiency (on 16 cores): 8.2 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T19:28:53.738180 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 55 potential peaks Of 55 potential peaks: 34 in veto map, 5 cut by saddle test, 0 cut by aper test, 16 kept SED z : 16 new peaks Found 21 potential peaks Of 21 potential peaks: 13 in veto map, 4 cut by saddle test, 0 cut by aper test, 4 kept SED r : 4 new peaks Found 22 potential peaks Of 22 potential peaks: 17 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 25 potential peaks Of 25 potential peaks: 18 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 37 potential peaks Of 37 potential peaks: 28 in veto map, 6 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 25 new sources Median coadd sky for g : 6.2169085e-05 Median coadd sky for r : 0.00013404334 Median coadd sky for z : 0.00022930698 Stage srcs : Wall: 0.99 s, CPU: 0.94 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 237 MB, VmData: 263 MB, maxrss: 0.243088 MB Stage srcs finished: 2020-08-05T19:28:54.723432 Resources for stage srcs : 0.079 s worker CPU, 0.079 s worker Wall, Wall: 0.987 s, Cores in use: 1.04, Total efficiency (on 16 cores): 6.5 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:28:54.757420 Blob 1 of 12, id: 4, sources: 13, size: 164x145, npix 15120, brick X: 34,198, Y: 40,185, one pixel: 166 40 Blob 2 of 12, id: 2, sources: 1, size: 63x40, npix 1696, brick X: 22,85, Y: 20,60, one pixel: 56 20 Blob 3 of 12, id: 8, sources: 2, size: 49x33, npix 1004, brick X: 81,130, Y: 183,216, one pixel: 99 183 Blob 4 of 12, id: 7, sources: 3, size: 40x53, npix 982, brick X: 199,239, Y: 85,138, one pixel: 224 85 Blob 5 of 12, id: 6, sources: 1, size: 30x31, npix 617, brick X: 0,30, Y: 53,84, one pixel: 13 53 Blob 6 of 12, id: 5, sources: 1, size: 21x26, npix 402, brick X: 218,239, Y: 50,76, one pixel: 228 50 Blob 7 of 12, id: 1, sources: 1, size: 22x26, npix 394, brick X: 0,22, Y: 10,36, one pixel: 10 10 Blob 8 of 12, id: 9, sources: 1, size: 24x24, npix 341, brick X: 186,210, Y: 189,213, one pixel: 198 189 Blob 9 of 12, id: 10, sources: 1, size: 24x23, npix 329, brick X: 174,198, Y: 210,233, one pixel: 185 210 Blob 10 of 12, id: 11, sources: 1, size: 22x20, npix 268, brick X: 55,77, Y: 213,233, one pixel: 64 213 Blob 11 of 12, id: 3, sources: 1, size: 20x21, npix 248, brick X: 174,194, Y: 28,49, one pixel: 182 28 Blob 12 of 12, id: 0, sources: 1, size: 20x16, npix 218, brick X: 112,132, Y: 0,16, one pixel: 117 0 Blob 9 finished, total: Wall: 0.31 s, CPU: 0.30 s, VmPeak: 935 MB, VmSize: 733 MB, VmRSS: 146 MB, VmData: 182 MB, maxrss: 0.149816 MB Blob 11 finished, total: Wall: 0.34 s, CPU: 0.33 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 110 MB, VmData: 164 MB, maxrss: 0.112488 MB Blob 10 finished, total: Wall: 0.41 s, CPU: 0.40 s, VmPeak: 940 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148332 MB Blob 12 finished, total: Wall: 0.45 s, CPU: 0.43 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 110 MB, VmData: 165 MB, maxrss: 0.112672 MB Blob 7 finished, total: Wall: 0.60 s, CPU: 0.58 s, VmPeak: 940 MB, VmSize: 733 MB, VmRSS: 147 MB, VmData: 182 MB, maxrss: 0.150108 MB Blob 6 finished, total: Wall: 0.83 s, CPU: 0.81 s, VmPeak: 934 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149076 MB Blob 8 finished, total: Wall: 0.87 s, CPU: 0.87 s, VmPeak: 937 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.14928 MB Blob 3 finished, total: Wall: 1.25 s, CPU: 1.24 s, VmPeak: 950 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.1488 MB Blob 5 finished, total: Wall: 1.28 s, CPU: 1.27 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 93 MB, VmData: 139 MB, maxrss: 0.09498 MB Blob 2 finished, total: Wall: 1.28 s, CPU: 1.26 s, VmPeak: 953 MB, VmSize: 737 MB, VmRSS: 152 MB, VmData: 186 MB, maxrss: 0.155712 MB Blob 4 finished, total: Wall: 2.00 s, CPU: 1.97 s, VmPeak: 960 MB, VmSize: 732 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.147728 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.23525, ee1=-0.0895783, ee2=-0.0855749 Measuring 9 iterative sources Blob 1 finished, total: Wall: 14.72 s, CPU: 14.71 s, VmPeak: 689 MB, VmSize: 688 MB, VmRSS: 114 MB, VmData: 159 MB, maxrss: 0.116912 MB Old catalog: 34 New catalog: 29 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/blobs-custom-323897p07306.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/all-models-custom-323897p07306.fits Stage fitblobs : Wall: 15.92 s, CPU: 1.10 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 238 MB, VmData: 263 MB, maxrss: 0.243252 MB Stage fitblobs finished: 2020-08-05T19:29:10.679164 Resources for stage fitblobs : 24.348 s worker CPU, 25.105 s worker Wall, Wall: 15.923 s, Cores in use: 1.60, Total efficiency (on 16 cores): 10.0 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:29:10.696372 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-resid.jpg Stage coadds : Wall: 4.21 s, CPU: 1.49 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 238 MB, VmData: 263 MB, maxrss: 0.243572 MB Stage coadds finished: 2020-08-05T19:29:14.901836 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 12.724 s worker CPU, 23.646 s worker Wall, Wall: 4.207 s, Cores in use: 3.38, Total efficiency (on 16 cores): 21.1 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T19:29:14.924166 Cut to 1 unWISE tiles Reading WISE tile 3233p075 band 1 Reading WISE tile 3233p075 band 2 Reading WISE tile 3233p075 band 3 Reading WISE tile 3233p075 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 934 MB, VmSize: 811 MB, VmRSS: 151 MB, VmData: 176 MB, maxrss: 0.192512 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 937 MB, VmSize: 806 MB, VmRSS: 145 MB, VmData: 171 MB, maxrss: 0.207916 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 864 MB, VmSize: 864 MB, VmRSS: 205 MB, VmData: 236 MB, maxrss: 0.209668 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 940 MB, VmSize: 810 MB, VmRSS: 150 MB, VmData: 175 MB, maxrss: 0.213012 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 5.46 s, CPU: 1.83 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 238 MB, VmData: 263 MB, maxrss: 0.244088 MB Stage wise_forced finished: 2020-08-05T19:29:20.383346 Resources for stage wise_forced : 2.979 s worker CPU, 7.897 s worker Wall, Wall: 5.461 s, Cores in use: 0.88, Total efficiency (on 16 cores): 5.5 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:29:20.399119 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/tractor-i/cus/tractor-custom-323897p07306.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/tractor/cus/tractor-custom-323897p07306.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/tractor/cus/brick-custom-323897p07306.sha256sum Stage writecat : Wall: 0.85 s, CPU: 0.66 s, VmPeak: 1200 MB, VmSize: 1072 MB, VmRSS: 240 MB, VmData: 263 MB, maxrss: 0.377092 MB Stage writecat finished: 2020-08-05T19:29:21.246353 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.849 s, Cores in use: 0.78, Total efficiency (on 16 cores): 4.9 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-writecat.p All done: 47.068 s worker CPU, 487.772 s worker Wall, Wall: 147.151 s, Cores in use: 0.42, Total efficiency (on 16 cores): 2.6 % Finished galaxy PGC1318681 in 2.509 minutes. Started working on galaxy PGC1318681 at Wed Aug 5 19:49:18 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.897982 7.3067036 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:49:20.161311 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '323.897982', '7.3067036', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.897982 7.3067036 --width 239 --height 239 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 323.897982 7.3067036 Starting process 42443 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 42444 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 42445 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 42446 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 42447 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB Starting process 42448 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05956 MB Starting process 42449 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05956 MB Starting process 42450 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 42451 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 42452 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 42453 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 42454 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 42455 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 42456 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 42457 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 42458 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 42459 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 42460 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 42461 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 42462 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 42463 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 42464 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 42465 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 42467 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 42468 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 42466 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 42469 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 42470 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Starting process 42471 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB Starting process 42472 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB Starting process 42473 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB Starting process 42474 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05932 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:49:21.849849 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 10 CCDs: c4d_140814_041842_ooi_z_ls9-S25 z expnum 347705 exptime 238.0 propid 2014B-0404 seeing 0.90 MJD 56883.176 object DECaLS_43591_z decam/CP/V4.8.2a/CP20140813/c4d_140814_041842_ooi_z_ls9.fits.fz c4d_140815_031959_ooi_r_ls9-S25 r expnum 347967 exptime 85.0 propid 2014B-0404 seeing 1.20 MJD 56884.137 object DECaLS_43591_r decam/CP/V4.8.2a/CP20140814/c4d_140815_031959_ooi_r_ls9.fits.fz c4d_151011_010312_ooi_g_ls9-S25 g expnum 483633 exptime 72.0 propid 2014B-0404 seeing 1.38 MJD 57306.042 object DECaLS_43591_g decam/CP/V4.8.2a/CP20151010/c4d_151011_010312_ooi_g_ls9.fits.fz c4d_180516_091924_ooi_g_ls9-S30 g expnum 746811 exptime 200.0 propid 2014B-0404 seeing 1.34 MJD 58254.388 object DECaLS_11847_g decam/CP/V4.8.2a/CP20180515/c4d_180516_091924_ooi_g_ls9.fits.fz c4d_181006_235957_ooi_z_ls9-N28 z expnum 781250 exptime 80.0 propid 2014B-0404 seeing 0.93 MJD 58398.000 object DECaLS_43592_z decam/CP/V4.8.2a/CP20181006/c4d_181006_235957_ooi_z_ls9.fits.fz c4d_140815_032350_ooi_r_ls9-N28 r expnum 347969 exptime 85.0 propid 2014B-0404 seeing 1.08 MJD 56884.140 object DECaLS_43592_r decam/CP/V4.8.2a/CP20140814/c4d_140815_032350_ooi_r_ls9.fits.fz c4d_151011_005943_ooi_g_ls9-N28 g expnum 483631 exptime 71.0 propid 2014B-0404 seeing 1.25 MJD 57306.040 object DECaLS_43592_g decam/CP/V4.8.2a/CP20151010/c4d_151011_005943_ooi_g_ls9.fits.fz c4d_140817_035750_ooi_g_ls9-N20 g expnum 348778 exptime 87.0 propid 2014B-0404 seeing 1.42 MJD 56886.164 object DECaLS_11867_g decam/CP/V4.8.2a/CP20140816/c4d_140817_035750_ooi_g_ls9.fits.fz c4d_140816_034713_ooi_r_ls9-N20 r expnum 348334 exptime 66.0 propid 2014B-0404 seeing 0.96 MJD 56885.156 object DECaLS_11867_r decam/CP/V4.8.2a/CP20140815/c4d_140816_034713_ooi_r_ls9.fits.fz c4d_140816_044100_ooi_z_ls9-N20 z expnum 348356 exptime 115.0 propid 2014B-0404 seeing 0.94 MJD 56885.193 object DECaLS_11867_z decam/CP/V4.8.2a/CP20140815/c4d_140816_044100_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T031743_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140816T034515_oi[N20,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T010110_oi[S25,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0582126 vs median 0.0439778 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c6e6-z-20140814T041350_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T032133_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530281-g-20180516T091924_oi[S30,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954d16a-z-20140816T043811_oi[N20,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T005741_oi[N28,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954a39a-g-20140817T035529_oi[N20,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0506858 vs median 0.0389228 Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9531140-z-20181006T235957_oi[N28,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 70.43 s, CPU: 1.67 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 176 MB, VmData: 208 MB, maxrss: 0.179956 MB Stage tims finished: 2020-08-05T19:50:32.278755 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530281-g-20180516T091924_oi[S30,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c6e6-z-20140814T041350_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954d16a-z-20140816T043811_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T031743_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T032133_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T005741_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954a39a-g-20140817T035529_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9531140-z-20181006T235957_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 5.889 s worker CPU, 258.518 s worker Wall, Wall: 70.432 s, Cores in use: 0.11, Total efficiency (on 32 cores): 0.3 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-05T19:50:32.364516 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05426.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05427.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05555.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05299.fits Matched 2 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/reference-custom-323897p07306.fits Stage refs : Wall: 16.80 s, CPU: 6.02 s, VmPeak: 1064 MB, VmSize: 1058 MB, VmRSS: 224 MB, VmData: 254 MB, maxrss: 0.229692 MB Stage refs finished: 2020-08-05T19:50:49.166869 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140816T034515_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage refs : 0.001 s worker CPU, 0.001 s worker Wall, Wall: 16.805 s, Cores in use: 0.36, Total efficiency (on 32 cores): 1.1 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-refs.p Running stage outliers WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T010110_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage outliers at 2020-08-05T19:50:49.261893 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outlier-mask-custom-323897p07306.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-pre-custom-323897p07306.jpg Image c4d_151011_010312_ooi_g_ls9-S25 g : masking 0 positive outlier pixels and 101 negative outlier pixels Image c4d_140815_032350_ooi_r_ls9-N28 r : masking 55 positive outlier pixels and 0 negative outlier pixels Image c4d_140816_034713_ooi_r_ls9-N20 r : masking 232 positive outlier pixels and 0 negative outlier pixels Image c4d_140815_031959_ooi_r_ls9-S25 r : masking 86 positive outlier pixels and 62 negative outlier pixels Image c4d_140814_041842_ooi_z_ls9-S25 z : masking 1981 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outlier-mask-custom-323897p07306.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-post-custom-323897p07306.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-masked-pos-custom-323897p07306.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/outliers-masked-neg-custom-323897p07306.jpg Stage outliers : Wall: 2.74 s, CPU: 2.17 s, VmPeak: 1067 MB, VmSize: 1067 MB, VmRSS: 236 MB, VmData: 263 MB, maxrss: 0.241568 MB Stage outliers finished: 2020-08-05T19:50:52.004247 Resources for stage outliers : 0.648 s worker CPU, 1.157 s worker Wall, Wall: 2.745 s, Cores in use: 1.03, Total efficiency (on 32 cores): 3.2 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:50:52.111845 Stage halos : Wall: 1.50 s, CPU: 1.04 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 236 MB, VmData: 264 MB, maxrss: 0.241936 MB Stage halos finished: 2020-08-05T19:50:53.615264 Resources for stage halos : 0.297 s worker CPU, 1.311 s worker Wall, Wall: 1.506 s, Cores in use: 0.89, Total efficiency (on 32 cores): 2.8 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T19:50:53.640217 PSF sigmas in coadd pixels: ['2.25', '2.18', '2.02', '2.31'] Using average PSF sigma 2.1892025510017348 Gaussian PSF norm 0.12885732827453417 vs pixelized 0.11110604 average tim pixel scale / coadd scale: 1.0011116605259136 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.95', '1.76', '1.56'] Using average PSF sigma 1.7563636611203257 Gaussian PSF norm 0.16061297441894198 vs pixelized 0.13698794 average tim pixel scale / coadd scale: 1.0011692794452187 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.46', '1.51', '1.53'] Using average PSF sigma 1.4988530491535428 Gaussian PSF norm 0.18820710404744978 vs pixelized 0.14907776 average tim pixel scale / coadd scale: 1.001065572055469 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.13 s, CPU: 0.11 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 238 MB, VmData: 264 MB, maxrss: 0.2434 MB Stage fit_on_coadds finished: 2020-08-05T19:50:53.765360 Resources for stage fit_on_coadds : 0.110 s worker CPU, 0.110 s worker Wall, Wall: 0.127 s, Cores in use: 1.75, Total efficiency (on 32 cores): 5.5 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T19:50:53.778666 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 55 potential peaks Of 55 potential peaks: 34 in veto map, 5 cut by saddle test, 0 cut by aper test, 16 kept SED z : 16 new peaks Found 21 potential peaks Of 21 potential peaks: 13 in veto map, 4 cut by saddle test, 0 cut by aper test, 4 kept SED r : 4 new peaks Found 22 potential peaks Of 22 potential peaks: 17 in veto map, 4 cut by saddle test, 0 cut by aper test, 1 kept SED g : 1 new peaks Found 25 potential peaks Of 25 potential peaks: 18 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 37 potential peaks Of 37 potential peaks: 28 in veto map, 6 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 25 new sources Median coadd sky for g : 6.2169085e-05 Median coadd sky for r : 0.00013404334 Median coadd sky for z : 0.00022930698 Stage srcs : Wall: 0.77 s, CPU: 0.72 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 238 MB, VmData: 264 MB, maxrss: 0.243748 MB Stage srcs finished: 2020-08-05T19:50:54.544419 Resources for stage srcs : 0.077 s worker CPU, 0.077 s worker Wall, Wall: 0.768 s, Cores in use: 1.05, Total efficiency (on 32 cores): 3.3 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:50:54.559344 Blob 1 of 12, id: 4, sources: 13, size: 164x145, npix 15120, brick X: 34,198, Y: 40,185, one pixel: 166 40 Blob 2 of 12, id: 2, sources: 1, size: 63x40, npix 1696, brick X: 22,85, Y: 20,60, one pixel: 56 20 Blob 3 of 12, id: 8, sources: 2, size: 49x33, npix 1004, brick X: 81,130, Y: 183,216, one pixel: 99 183 Blob 4 of 12, id: 7, sources: 3, size: 40x53, npix 982, brick X: 199,239, Y: 85,138, one pixel: 224 85 Blob 5 of 12, id: 6, sources: 1, size: 30x31, npix 617, brick X: 0,30, Y: 53,84, one pixel: 13 53 Blob 6 of 12, id: 5, sources: 1, size: 21x26, npix 402, brick X: 218,239, Y: 50,76, one pixel: 228 50 Blob 7 of 12, id: 1, sources: 1, size: 22x26, npix 394, brick X: 0,22, Y: 10,36, one pixel: 10 10 Blob 8 of 12, id: 9, sources: 1, size: 24x24, npix 341, brick X: 186,210, Y: 189,213, one pixel: 198 189 Blob 9 of 12, id: 10, sources: 1, size: 24x23, npix 329, brick X: 174,198, Y: 210,233, one pixel: 185 210 Blob 10 of 12, id: 11, sources: 1, size: 22x20, npix 268, brick X: 55,77, Y: 213,233, one pixel: 64 213 Blob 11 of 12, id: 3, sources: 1, size: 20x21, npix 248, brick X: 174,194, Y: 28,49, one pixel: 182 28 Blob 12 of 12, id: 0, sources: 1, size: 20x16, npix 218, brick X: 112,132, Y: 0,16, one pixel: 117 0 Blob 9 finished, total: Wall: 0.23 s, CPU: 0.22 s, VmPeak: 666 MB, VmSize: 666 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.091792 MB Blob 11 finished, total: Wall: 0.25 s, CPU: 0.25 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 110 MB, VmData: 165 MB, maxrss: 0.112576 MB Blob 10 finished, total: Wall: 0.28 s, CPU: 0.28 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 109 MB, VmData: 164 MB, maxrss: 0.11194 MB Blob 12 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 940 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148036 MB Blob 7 finished, total: Wall: 0.63 s, CPU: 0.63 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 110 MB, VmData: 165 MB, maxrss: 0.1125 MB Blob 6 finished, total: Wall: 0.67 s, CPU: 0.66 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092112 MB Blob 8 finished, total: Wall: 0.75 s, CPU: 0.74 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.092236 MB Blob 5 finished, total: Wall: 1.07 s, CPU: 1.06 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 112 MB, VmData: 166 MB, maxrss: 0.114744 MB Blob 2 finished, total: Wall: 1.15 s, CPU: 1.15 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.095816 MB Blob 3 finished, total: Wall: 1.33 s, CPU: 1.33 s, VmPeak: 935 MB, VmSize: 733 MB, VmRSS: 146 MB, VmData: 182 MB, maxrss: 0.149972 MB Blob 4 finished, total: Wall: 1.85 s, CPU: 1.85 s, VmPeak: 940 MB, VmSize: 733 MB, VmRSS: 146 MB, VmData: 182 MB, maxrss: 0.149304 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.23525, ee1=-0.0895783, ee2=-0.0855749 Measuring 9 iterative sources Blob 1 finished, total: Wall: 15.20 s, CPU: 15.20 s, VmPeak: 689 MB, VmSize: 688 MB, VmRSS: 115 MB, VmData: 159 MB, maxrss: 0.117328 MB Old catalog: 34 New catalog: 29 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/blobs-custom-323897p07306.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/metrics/cus/all-models-custom-323897p07306.fits Stage fitblobs : Wall: 15.64 s, CPU: 0.41 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 238 MB, VmData: 264 MB, maxrss: 0.243912 MB Stage fitblobs finished: 2020-08-05T19:51:10.195322 Resources for stage fitblobs : 23.846 s worker CPU, 23.927 s worker Wall, Wall: 15.638 s, Cores in use: 1.55, Total efficiency (on 32 cores): 4.8 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:51:10.228628 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-resid.jpg Stage coadds : Wall: 6.58 s, CPU: 1.77 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 239 MB, VmData: 264 MB, maxrss: 0.244236 MB Stage coadds finished: 2020-08-05T19:51:16.808468 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 29.386 s worker CPU, 78.792 s worker Wall, Wall: 6.582 s, Cores in use: 4.73, Total efficiency (on 32 cores): 14.8 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T19:51:16.830215 Cut to 1 unWISE tiles Reading WISE tile 3233p075 band 3 Reading WISE tile 3233p075 band 1 Reading WISE tile 3233p075 band 2 Reading WISE tile 3233p075 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 804 MB, VmSize: 804 MB, VmRSS: 143 MB, VmData: 175 MB, maxrss: 0.181 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 940 MB, VmSize: 810 MB, VmRSS: 149 MB, VmData: 175 MB, maxrss: 0.212324 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 934 MB, VmSize: 804 MB, VmRSS: 144 MB, VmData: 173 MB, maxrss: 0.206012 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 942 MB, VmSize: 801 MB, VmRSS: 141 MB, VmData: 171 MB, maxrss: 0.203332 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 4.08 s, CPU: 1.59 s, VmPeak: 1068 MB, VmSize: 1068 MB, VmRSS: 239 MB, VmData: 264 MB, maxrss: 0.244748 MB Stage wise_forced finished: 2020-08-05T19:51:20.909272 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.440 s worker CPU, 6.463 s worker Wall, Wall: 4.081 s, Cores in use: 1.23, Total efficiency (on 32 cores): 3.9 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:51:20.933125 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/tractor-i/cus/tractor-custom-323897p07306.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/tractor/cus/tractor-custom-323897p07306.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/tractor/cus/brick-custom-323897p07306.sha256sum Stage writecat : Wall: 1.99 s, CPU: 1.23 s, VmPeak: 1200 MB, VmSize: 1072 MB, VmRSS: 241 MB, VmData: 264 MB, maxrss: 0.377756 MB Stage writecat finished: 2020-08-05T19:51:22.925205 Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 1.995 s, Cores in use: 0.62, Total efficiency (on 32 cores): 1.9 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1318681/PGC1318681-largegalaxy-writecat.p All done: 63.698 s worker CPU, 370.363 s worker Wall, Wall: 121.359 s, Cores in use: 0.66, Total efficiency (on 32 cores): 2.1 % Finished galaxy PGC1318681 in 2.079 minutes. Started working on galaxy PGC1318681 at Mon Aug 10 08:10:40 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.89763953206494 7.306717345402549 --width 239 --height 239 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-10T08:10:41.659988 legacypipe git version: DR9.5.11-watershed-6-g6a2880fc Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '323.89763953206494', '7.306717345402549', '--width', '239', '--height', '239', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.89763953206494 7.306717345402549 --width 239 --height 239 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 323.89763953206494 7.306717345402549 Starting process 384 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 385 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 386 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 387 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 388 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059544 MB Starting process 389 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059492 MB Starting process 390 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059492 MB Starting process 391 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-10T08:10:42.103417 Keeping 10 CCDs: c4d_140814_041842_ooi_z_ls9-S25 z expnum 347705 exptime 238.0 propid 2014B-0404 seeing 0.90 MJD 56883.176 object DECaLS_43591_z decam/CP/V4.8.2a/CP20140813/c4d_140814_041842_ooi_z_ls9.fits.fz c4d_140815_031959_ooi_r_ls9-S25 r expnum 347967 exptime 85.0 propid 2014B-0404 seeing 1.20 MJD 56884.137 object DECaLS_43591_r decam/CP/V4.8.2a/CP20140814/c4d_140815_031959_ooi_r_ls9.fits.fz c4d_151011_010312_ooi_g_ls9-S25 g expnum 483633 exptime 72.0 propid 2014B-0404 seeing 1.38 MJD 57306.042 object DECaLS_43591_g decam/CP/V4.8.2a/CP20151010/c4d_151011_010312_ooi_g_ls9.fits.fz c4d_180516_091924_ooi_g_ls9-S30 g expnum 746811 exptime 200.0 propid 2014B-0404 seeing 1.34 MJD 58254.388 object DECaLS_11847_g decam/CP/V4.8.2a/CP20180515/c4d_180516_091924_ooi_g_ls9.fits.fz c4d_181006_235957_ooi_z_ls9-N28 z expnum 781250 exptime 80.0 propid 2014B-0404 seeing 0.93 MJD 58398.000 object DECaLS_43592_z decam/CP/V4.8.2a/CP20181006/c4d_181006_235957_ooi_z_ls9.fits.fz c4d_140815_032350_ooi_r_ls9-N28 r expnum 347969 exptime 85.0 propid 2014B-0404 seeing 1.08 MJD 56884.140 object DECaLS_43592_r decam/CP/V4.8.2a/CP20140814/c4d_140815_032350_ooi_r_ls9.fits.fz c4d_151011_005943_ooi_g_ls9-N28 g expnum 483631 exptime 71.0 propid 2014B-0404 seeing 1.25 MJD 57306.040 object DECaLS_43592_g decam/CP/V4.8.2a/CP20151010/c4d_151011_005943_ooi_g_ls9.fits.fz c4d_140817_035750_ooi_g_ls9-N20 g expnum 348778 exptime 87.0 propid 2014B-0404 seeing 1.42 MJD 56886.164 object DECaLS_11867_g decam/CP/V4.8.2a/CP20140816/c4d_140817_035750_ooi_g_ls9.fits.fz c4d_140816_034713_ooi_r_ls9-N20 r expnum 348334 exptime 66.0 propid 2014B-0404 seeing 0.96 MJD 56885.156 object DECaLS_11867_r decam/CP/V4.8.2a/CP20140815/c4d_140816_034713_ooi_r_ls9.fits.fz c4d_140816_044100_ooi_z_ls9-N20 z expnum 348356 exptime 115.0 propid 2014B-0404 seeing 0.94 MJD 56885.193 object DECaLS_11867_z decam/CP/V4.8.2a/CP20140815/c4d_140816_044100_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c6e6-z-20140814T041350_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954a39a-g-20140817T035529_oi[N20,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0506858 vs median 0.0389228 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530281-g-20180516T091924_oi[S30,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T031743_oi[S25,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T010110_oi[S25,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0582126 vs median 0.0439778 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T005741_oi[N28,noinherit][2046,4094][real]: DECaLS_ Clipping 1 pixels with anomalously large oow values: max 0.0546393 vs median 0.0416248 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T032133_oi[N28,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140816T034515_oi[N20,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954d16a-z-20140816T043811_oi[N20,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9531140-z-20181006T235957_oi[N28,noinherit][2046,4094][real]: DECaLS_ Stage tims : Wall: 88.25 s, CPU: 1.01 s, VmPeak: 1005 MB, VmSize: 1005 MB, VmRSS: 174 MB, VmData: 207 MB, maxrss: 0.177888 MB Stage tims finished: 2020-08-10T08:12:10.350014 Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c6e6-z-20140814T041350_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T031743_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T010110_oi[S25,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140815T032133_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95642df-g-20151011T005741_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530281-g-20180516T091924_oi[S30,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954d16a-z-20140816T043811_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954a39a-g-20140817T035529_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9548395-r-20140816T034515_oi[N20,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Resources for stage tims : 5.566 s worker CPU, 211.363 s worker Wall, Wall: 88.248 s, Cores in use: 0.07, Total efficiency (on 8 cores): 0.9 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-tims.p Running stage refs Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9531140-z-20181006T235957_oi[N28,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Running stage refs at 2020-08-10T08:12:10.436492 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05426.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05427.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05555.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-05299.fits Matched 2 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/reference-custom-323897p07306.fits Stage refs : Wall: 19.08 s, CPU: 3.91 s, VmPeak: 1057 MB, VmSize: 1051 MB, VmRSS: 222 MB, VmData: 254 MB, maxrss: 0.227688 MB Stage refs finished: 2020-08-10T08:12:29.519706 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 19.084 s, Cores in use: 0.21, Total efficiency (on 8 cores): 2.6 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-10T08:12:29.603776 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/outlier-mask-custom-323897p07306.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/outliers-pre-custom-323897p07306.jpg Image c4d_151011_010312_ooi_g_ls9-S25 g : masking 0 positive outlier pixels and 171 negative outlier pixels Image c4d_180516_091924_ooi_g_ls9-S30 g : masking 36 positive outlier pixels and 0 negative outlier pixels Image c4d_151011_005943_ooi_g_ls9-N28 g : masking 112 positive outlier pixels and 0 negative outlier pixels Image c4d_140816_034713_ooi_r_ls9-N20 r : masking 240 positive outlier pixels and 0 negative outlier pixels Image c4d_140815_031959_ooi_r_ls9-S25 r : masking 87 positive outlier pixels and 114 negative outlier pixels Image c4d_140815_032350_ooi_r_ls9-N28 r : masking 126 positive outlier pixels and 0 negative outlier pixels Image c4d_140814_041842_ooi_z_ls9-S25 z : masking 2032 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/outlier-mask-custom-323897p07306.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/outliers-post-custom-323897p07306.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/outliers-masked-pos-custom-323897p07306.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/outliers-masked-neg-custom-323897p07306.jpg Stage outliers : Wall: 1.70 s, CPU: 0.62 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 233 MB, VmData: 262 MB, maxrss: 0.238512 MB Stage outliers finished: 2020-08-10T08:12:31.299483 Resources for stage outliers : 0.516 s worker CPU, 0.516 s worker Wall, Wall: 1.697 s, Cores in use: 0.67, Total efficiency (on 8 cores): 8.4 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-10T08:12:31.319090 Stage halos : Wall: 3.98 s, CPU: 0.45 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 233 MB, VmData: 262 MB, maxrss: 0.238984 MB Stage halos finished: 2020-08-10T08:12:35.303521 Resources for stage halos : 0.191 s worker CPU, 4.157 s worker Wall, Wall: 3.986 s, Cores in use: 0.16, Total efficiency (on 8 cores): 2.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-10T08:12:35.383741 PSF sigmas in coadd pixels: ['2.25', '2.18', '2.02', '2.31'] Using average PSF sigma 2.1892025510017348 Gaussian PSF norm 0.12885732827453417 vs pixelized 0.11110651 average tim pixel scale / coadd scale: 1.0011116605259136 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.95', '1.76', '1.56'] Using average PSF sigma 1.7563636611203257 Gaussian PSF norm 0.16061297441894198 vs pixelized 0.13698602 average tim pixel scale / coadd scale: 1.0011692794452187 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.46', '1.51', '1.53'] Using average PSF sigma 1.4988530491535428 Gaussian PSF norm 0.18820710404744978 vs pixelized 0.14908408 average tim pixel scale / coadd scale: 1.001065572055469 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.13 s, CPU: 0.12 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 235 MB, VmData: 262 MB, maxrss: 0.240452 MB Stage fit_on_coadds finished: 2020-08-10T08:12:35.517679 Resources for stage fit_on_coadds : 0.128 s worker CPU, 0.128 s worker Wall, Wall: 0.135 s, Cores in use: 1.82, Total efficiency (on 8 cores): 22.7 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-10T08:12:35.527827 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 56 potential peaks Of 56 potential peaks: 34 in veto map, 5 cut by saddle test, 0 cut by aper test, 17 kept SED z : 17 new peaks Found 21 potential peaks Of 21 potential peaks: 14 in veto map, 4 cut by saddle test, 0 cut by aper test, 3 kept SED r : 3 new peaks Found 25 potential peaks Of 25 potential peaks: 18 in veto map, 5 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 26 potential peaks Of 26 potential peaks: 19 in veto map, 6 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 39 potential peaks Of 39 potential peaks: 30 in veto map, 7 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 25 new sources Median coadd sky for g : 1.8042654e-05 Median coadd sky for r : 0.00015138458 Median coadd sky for z : 0.0003147498 Stage srcs : Wall: 0.91 s, CPU: 0.88 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 235 MB, VmData: 262 MB, maxrss: 0.24084 MB Stage srcs finished: 2020-08-10T08:12:36.437702 Resources for stage srcs : 0.066 s worker CPU, 0.067 s worker Wall, Wall: 0.911 s, Cores in use: 1.04, Total efficiency (on 8 cores): 13.0 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-10T08:12:36.450766 Blob 1 of 12, id: 4, sources: 12, size: 165x145, npix 15196, brick X: 29,194, Y: 40,185, one pixel: 161 40 Blob 2 of 12, id: 2, sources: 1, size: 62x39, npix 1683, brick X: 17,79, Y: 20,59, one pixel: 49 20 Blob 3 of 12, id: 7, sources: 3, size: 42x53, npix 1018, brick X: 194,236, Y: 85,138, one pixel: 219 85 Blob 4 of 12, id: 8, sources: 2, size: 49x33, npix 998, brick X: 76,125, Y: 183,216, one pixel: 93 183 Blob 5 of 12, id: 5, sources: 2, size: 26x49, npix 836, brick X: 213,239, Y: 42,91, one pixel: 230 42 Blob 6 of 12, id: 6, sources: 1, size: 26x31, npix 570, brick X: 0,26, Y: 53,84, one pixel: 8 53 Blob 7 of 12, id: 9, sources: 1, size: 24x24, npix 342, brick X: 182,206, Y: 189,213, one pixel: 192 189 Blob 8 of 12, id: 10, sources: 1, size: 23x23, npix 327, brick X: 170,193, Y: 210,233, one pixel: 180 210 Blob 9 of 12, id: 1, sources: 1, size: 18x24, npix 283, brick X: 0,18, Y: 10,34, one pixel: 7 10 Blob 8 finished, total: Wall: 0.32 s, CPU: 0.32 s, VmPeak: 960 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.14372 MB Blob 10 of 12, id: 11, sources: 1, size: 22x20, npix 260, brick X: 50,72, Y: 213,233, one pixel: 59 213 Blob 7 finished, total: Wall: 0.99 s, CPU: 0.96 s, VmPeak: 950 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.143764 MB Blob 11 of 12, id: 3, sources: 1, size: 20x21, npix 239, brick X: 169,189, Y: 28,49, one pixel: 177 28 Blob 9 finished, total: Wall: 0.72 s, CPU: 0.72 s, VmPeak: 960 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.14372 MB Blob 12 of 12, id: 0, sources: 1, size: 21x16, npix 216, brick X: 107,128, Y: 0,16, one pixel: 112 0 Blob 10 finished, total: Wall: 0.38 s, CPU: 0.38 s, VmPeak: 950 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.143764 MB Blob 6 finished, total: Wall: 1.55 s, CPU: 1.51 s, VmPeak: 937 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.1464 MB Blob 2 finished, total: Wall: 1.63 s, CPU: 1.63 s, VmPeak: 953 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 180 MB, maxrss: 0.145852 MB Measuring 1 iterative sources Blob 12 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 950 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.143764 MB Blob 4 finished, total: Wall: 1.83 s, CPU: 1.83 s, VmPeak: 969 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.144992 MB Blob 11 finished, total: Wall: 0.77 s, CPU: 0.77 s, VmPeak: 960 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.14372 MB Blob 5 finished, total: Wall: 2.00 s, CPU: 2.00 s, VmPeak: 965 MB, VmSize: 727 MB, VmRSS: 142 MB, VmData: 182 MB, maxrss: 0.145524 MB Blob 3 finished, total: Wall: 2.21 s, CPU: 2.21 s, VmPeak: 942 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.145424 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=0.856663, ee1=-0.0901235, ee2=-0.0765066 Measuring 8 iterative sources Blob 1 finished, total: Wall: 18.08 s, CPU: 18.07 s, VmPeak: 934 MB, VmSize: 727 MB, VmRSS: 145 MB, VmData: 182 MB, maxrss: 0.148628 MB Old catalog: 34 New catalog: 29 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/blobs-custom-323897p07306.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/metrics/cus/all-models-custom-323897p07306.fits Stage fitblobs : Wall: 18.39 s, CPU: 0.32 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 235 MB, VmData: 262 MB, maxrss: 0.241004 MB Stage fitblobs finished: 2020-08-10T08:12:54.840822 Resources for stage fitblobs : 30.857 s worker CPU, 31.004 s worker Wall, Wall: 18.391 s, Cores in use: 1.70, Total efficiency (on 8 cores): 21.2 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-10T08:12:54.852420 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-resid.jpg Stage coadds : Wall: 1.49 s, CPU: 0.52 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 236 MB, VmData: 262 MB, maxrss: 0.241328 MB Stage coadds finished: 2020-08-10T08:12:56.345735 Resources for stage coadds : 4.239 s worker CPU, 4.870 s worker Wall, Wall: 1.494 s, Cores in use: 3.19, Total efficiency (on 8 cores): 39.8 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-10T08:12:56.359874 Cut to 1 unWISE tiles Reading WISE tile 3233p075 band 3 Reading WISE tile 3233p075 band 2 Reading WISE tile 3233p075 band 4 Reading WISE tile 3233p075 band 1 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 937 MB, VmSize: 806 MB, VmRSS: 149 MB, VmData: 177 MB, maxrss: 0.19176 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 942 MB, VmSize: 800 MB, VmRSS: 142 MB, VmData: 171 MB, maxrss: 0.204736 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 969 MB, VmSize: 801 MB, VmRSS: 144 MB, VmData: 172 MB, maxrss: 0.20858 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3233p075/unwise-3233p075-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 965 MB, VmSize: 804 MB, VmRSS: 146 MB, VmData: 175 MB, maxrss: 0.20942 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/tmp-legacysurvey-custom-323897p07306-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 3.84 s, CPU: 0.75 s, VmPeak: 1061 MB, VmSize: 1061 MB, VmRSS: 236 MB, VmData: 262 MB, maxrss: 0.24184 MB Stage wise_forced finished: 2020-08-10T08:13:00.204738 Resources for stage wise_forced : 3.252 s worker CPU, 6.658 s worker Wall, Wall: 3.846 s, Cores in use: 1.04, Total efficiency (on 8 cores): 13.0 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-10T08:13:00.218347 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/coadd/cus/custom-323897p07306/legacysurvey-custom-323897p07306-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/tractor-i/cus/tractor-custom-323897p07306.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/tractor/cus/tractor-custom-323897p07306.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/tractor/cus/brick-custom-323897p07306.sha256sum Stage writecat : Wall: 0.74 s, CPU: 0.51 s, VmPeak: 1193 MB, VmSize: 1065 MB, VmRSS: 238 MB, VmData: 262 MB, maxrss: 0.374828 MB Stage writecat finished: 2020-08-10T08:13:00.962394 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.745 s, Cores in use: 0.69, Total efficiency (on 8 cores): 8.6 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1318681/PGC1318681-largegalaxy-writecat.p All done: 44.817 s worker CPU, 258.764 s worker Wall, Wall: 138.875 s, Cores in use: 0.39, Total efficiency (on 8 cores): 4.9 % Finished galaxy PGC1318681 in 2.348 minutes.