Started working on galaxy PGC1025201 at Wed Aug 5 19:01:29 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 212.24746649999997 -7.045903699999999 --width 233 --height 233 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:01:30.661063 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '212.24746649999997', '-7.045903699999999', '--width', '233', '--height', '233', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 212.24746649999997 -7.045903699999999 --width 233 --height 233 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 212.24746649999997 -7.045903699999999 Starting process 1971 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 1972 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 1973 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 1974 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 1975 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05934 MB Starting process 1976 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059328 MB Starting process 1977 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059328 MB Starting process 1978 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 1979 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 1980 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 1981 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 1982 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 1983 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 1984 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 1985 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 1986 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:01:31.280685 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 7 CCDs: c4d_190307_065801_ooi_z_ls9-S29 z expnum 830719 exptime 80.0 propid 2014B-0404 seeing 0.93 MJD 58549.290 object DECaLS_17181_z decam/CP/V4.8.2a/CP20190306/c4d_190307_065801_ooi_z_ls9.fits.fz c4d_170304_075532_ooi_g_ls9-S29 g expnum 625787 exptime 154.0 propid 2014B-0404 seeing 1.30 MJD 57816.328 object DECaLS_17181_g decam/CP/V4.8.2/CP20170303/c4d_170304_075532_ooi_g_ls9.fits.fz c4d_160410_050652_ooi_r_ls9-S29 r expnum 536018 exptime 85.0 propid 2014B-0404 seeing 1.23 MJD 57488.212 object DECaLS_17181_r decam/CP/V4.8.2a/CP20160409/c4d_160410_050652_ooi_r_ls9.fits.fz c4d_160722_025838_ooi_z_ls9-S29 z expnum 553729 exptime 180.0 propid 2016A-0190 seeing 1.67 MJD 57591.121 object DECaLS_17181_z decam/CP/V4.8.2a/CP20160721/c4d_160722_025838_ooi_z_ls9.fits.fz c4d_170401_083230_ooi_g_ls9-N23 g expnum 636060 exptime 175.0 propid 2014B-0404 seeing 1.42 MJD 57844.353 object DECaLS_33067_g decam/CP/V4.8.2/CP20170331/c4d_170401_083230_ooi_g_ls9.fits.fz c4d_170505_013002_ooi_z_ls9-N23 z expnum 647369 exptime 196.0 propid 2014B-0404 seeing 0.71 MJD 57878.060 object DECaLS_33067_z decam/CP/V4.8.2/CP20170504/c4d_170505_013002_ooi_z_ls9.fits.fz c4d_170522_035509_ooi_r_ls9-N23 r expnum 650165 exptime 69.0 propid 2014B-0404 seeing 0.95 MJD 57895.162 object DECaLS_33067_r decam/CP/V4.8.2/CP20170521/c4d_170522_035509_ooi_r_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950f8db-g-20170304T075205_oi[S29,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c242-z-20160722T025448_oi[S29,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95155af-z-20170505T012557_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95173ea-g-20170401T082845_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9535707-z-20190307T065801_oi[S29,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951a62c-r-20170522T035307_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956cfb1-r-20160410T050436_oi[S29,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 10.78 s, CPU: 0.96 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 174 MB, VmData: 207 MB, maxrss: 0.178648 MB Stage tims finished: 2020-08-05T19:01:42.061951 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95173ea-g-20170401T082845_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95155af-z-20170505T012557_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951a62c-r-20170522T035307_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950f8db-g-20170304T075205_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9535707-z-20190307T065801_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956cfb1-r-20160410T050436_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 4.061 s worker CPU, 55.235 s worker Wall, Wall: 10.783 s, Cores in use: 0.47, Total efficiency (on 16 cores): 2.9 % Started working on galaxy PGC1025201 at Wed Aug 5 19:01:42 2020 Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c242-z-20160722T025448_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 212.24746649999997 -7.045903699999999 --width 233 --height 233 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:01:43.796531 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '212.24746649999997', '-7.045903699999999', '--width', '233', '--height', '233', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 212.24746649999997 -7.045903699999999 --width 233 --height 233 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 212.24746649999997 -7.045903699999999 Starting process 2060 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 2061 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 2062 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 2063 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 2064 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059408 MB Starting process 2065 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059416 MB Starting process 2066 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059416 MB Starting process 2067 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 2068 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 2069 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059256 MB Starting process 2070 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 2071 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 2072 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 2073 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 2074 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 2075 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 2076 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 2077 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 2078 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 2079 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 2080 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 2081 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 2082 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059292 MB Starting process 2083 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 2084 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 2085 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 2086 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 2087 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 2088 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 2089 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 2090 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 2091 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Reading pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-tims.p Running stage refs Running stage refs at 2020-08-05T19:01:42.235163 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06923.fits Matched 10 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/reference-custom-212247m07045.fits Stage refs : Wall: 5.14 s, CPU: 2.76 s, VmPeak: 1041 MB, VmSize: 1034 MB, VmRSS: 199 MB, VmData: 230 MB, maxrss: 0.20338 MB Stage refs finished: 2020-08-05T19:01:47.370538 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 5.137 s, Cores in use: 0.54, Total efficiency (on 16 cores): 3.4 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:01:47.416782 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/outlier-mask-custom-212247m07045.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/outliers-pre-custom-212247m07045.jpg Image c4d_170401_083230_ooi_g_ls9-N23 g : masking 341 positive outlier pixels and 0 negative outlier pixels Image c4d_160410_050652_ooi_r_ls9-S29 r : masking 112 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/outlier-mask-custom-212247m07045.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/outliers-post-custom-212247m07045.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/outliers-masked-pos-custom-212247m07045.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/outliers-masked-neg-custom-212247m07045.jpg Stage outliers : Wall: 1.39 s, CPU: 0.99 s, VmPeak: 1043 MB, VmSize: 1043 MB, VmRSS: 210 MB, VmData: 238 MB, maxrss: 0.215324 MB Stage outliers finished: 2020-08-05T19:01:48.807382 Resources for stage outliers : 0.534 s worker CPU, 0.793 s worker Wall, Wall: 1.392 s, Cores in use: 1.10, Total efficiency (on 16 cores): 6.9 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-outliers.p Running stage halos Running stage refs at 2020-08-05T19:01:44.720005 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06923.fits Matched 10 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/reference-custom-212247m07045.fits Stage refs : Wall: 4.70 s, CPU: 2.46 s, VmPeak: 913 MB, VmSize: 908 MB, VmRSS: 170 MB, VmData: 201 MB, maxrss: 0.174324 MB Stage refs finished: 2020-08-05T19:01:49.415683 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 4.698 s, Cores in use: 0.52, Total efficiency (on 32 cores): 1.6 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:01:49.507156 Stage outliers : Wall: 1.68 s, CPU: 1.65 s, VmPeak: 913 MB, VmSize: 908 MB, VmRSS: 171 MB, VmData: 201 MB, maxrss: 0.175308 MB Stage outliers finished: 2020-08-05T19:01:51.192196 Resources for stage outliers : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.688 s, Cores in use: 0.98, Total efficiency (on 32 cores): 3.1 % Running stage halos at 2020-08-05T19:01:48.964914 Stage halos : Wall: 2.28 s, CPU: 1.92 s, VmPeak: 1043 MB, VmSize: 1043 MB, VmRSS: 210 MB, VmData: 238 MB, maxrss: 0.215324 MB Stage halos finished: 2020-08-05T19:01:51.248860 Resources for stage halos : 0.126 s worker CPU, 1.458 s worker Wall, Wall: 2.285 s, Cores in use: 0.90, Total efficiency (on 16 cores): 5.6 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-halos.p Running stage fit_on_coadds Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-outliers.p Running stage halos Running stage fit_on_coadds at 2020-08-05T19:01:51.277327 PSF sigmas in coadd pixels: ['2.10', '2.30'] Using average PSF sigma 2.2019974134807656 Gaussian PSF norm 0.12810859360999982 vs pixelized 0.11319851 average tim pixel scale / coadd scale: 1.0011475515473662 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.99', '1.54'] Using average PSF sigma 1.7656686059034938 Gaussian PSF norm 0.159766555757234 vs pixelized 0.12337613 average tim pixel scale / coadd scale: 1.001086816740869 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.52', '2.72', '1.16'] Using average PSF sigma 1.7969394431748915 Gaussian PSF norm 0.15698625395826574 vs pixelized 0.13191807 average tim pixel scale / coadd scale: 1.0011316555239373 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.19 s, CPU: 0.12 s, VmPeak: 1043 MB, VmSize: 1043 MB, VmRSS: 212 MB, VmData: 238 MB, maxrss: 0.216916 MB Stage fit_on_coadds finished: 2020-08-05T19:01:51.465460 Resources for stage fit_on_coadds : 0.098 s worker CPU, 0.098 s worker Wall, Wall: 0.189 s, Cores in use: 1.15, Total efficiency (on 16 cores): 7.2 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T19:01:51.488129 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 8 potential peaks Of 8 potential peaks: 0 in veto map, 1 cut by saddle test, 0 cut by aper test, 7 kept SED z : 7 new peaks Found 8 potential peaks Of 8 potential peaks: 7 in veto map, 1 cut by saddle test, 0 cut by aper test, 0 kept SED r : 0 new peaks Found 9 potential peaks Of 9 potential peaks: 7 in veto map, 0 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 13 potential peaks Of 13 potential peaks: 9 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 15 potential peaks Of 15 potential peaks: 10 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 13 new sources Median coadd sky for g : 8.994588e-05 Median coadd sky for r : 9.102108e-06 Median coadd sky for z : -0.00018777419 Stage srcs : Wall: 2.13 s, CPU: 2.03 s, VmPeak: 1043 MB, VmSize: 1043 MB, VmRSS: 212 MB, VmData: 238 MB, maxrss: 0.21734 MB Stage srcs finished: 2020-08-05T19:01:53.613642 Resources for stage srcs : 0.069 s worker CPU, 0.069 s worker Wall, Wall: 2.127 s, Cores in use: 0.99, Total efficiency (on 16 cores): 6.2 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-srcs.p Running stage fitblobs Running stage halos at 2020-08-05T19:01:51.304409 Stage halos : Wall: 2.40 s, CPU: 2.09 s, VmPeak: 913 MB, VmSize: 908 MB, VmRSS: 175 MB, VmData: 205 MB, maxrss: 0.179076 MB Stage halos finished: 2020-08-05T19:01:53.701603 Resources for stage halos : 0.228 s worker CPU, 1.357 s worker Wall, Wall: 2.400 s, Cores in use: 0.97, Total efficiency (on 32 cores): 3.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T19:01:53.739648 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Running stage fitblobs at 2020-08-05T19:01:53.629659 Blob 1 of 12, id: 6, sources: 2, size: 102x62, npix 4092, brick X: 65,167, Y: 76,138, one pixel: 116 76 Blob 2 of 12, id: 3, sources: 1, size: 49x40, npix 1235, brick X: 144,193, Y: 25,65, one pixel: 178 25 Blob 3 of 12, id: 10, sources: 1, size: 37x38, npix 889, brick X: 104,141, Y: 181,219, one pixel: 128 181 Blob 4 of 12, id: 0, sources: 2, size: 31x34, npix 724, brick X: 74,105, Y: 0,34, one pixel: 79 0 Blob 5 of 12, id: 7, sources: 1, size: 29x41, npix 695, brick X: 35,64, Y: 135,176, one pixel: 54 135 Blob 6 of 12, id: 8, sources: 1, size: 27x25, npix 417, brick X: 119,146, Y: 156,181, one pixel: 132 156 Blob 7 of 12, id: 11, sources: 1, size: 23x23, npix 315, brick X: 56,79, Y: 185,208, one pixel: 65 185 Blob 8 of 12, id: 1, sources: 1, size: 20x20, npix 228, brick X: 154,174, Y: 3,23, one pixel: 162 3 Blob 9 of 12, id: 5, sources: 1, size: 19x19, npix 199, brick X: 178,197, Y: 52,71, one pixel: 187 52 Blob 10 of 12, id: 9, sources: 1, size: 18x19, npix 189, brick X: 85,103, Y: 170,189, one pixel: 94 170 Blob 11 of 12, id: 4, sources: 1, size: 18x18, npix 180, brick X: 48,66, Y: 52,70, one pixel: 56 52 Blob 12 of 12, id: 2, sources: 1, size: 17x17, npix 145, brick X: 44,61, Y: 16,33, one pixel: 52 16 PSF sigmas in coadd pixels: ['2.10', '2.30'] Using average PSF sigma 2.2019974134807656 Gaussian PSF norm 0.12810859360999982 vs pixelized 0.11319851 average tim pixel scale / coadd scale: 1.0011475515473662 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.99', '1.54'] Using average PSF sigma 1.7656686059034938 Gaussian PSF norm 0.159766555757234 vs pixelized 0.12337613 average tim pixel scale / coadd scale: 1.001086816740869 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.52', '2.72', '1.16'] Using average PSF sigma 1.7969394431748915 Gaussian PSF norm 0.15698625395826574 vs pixelized 0.13191807 average tim pixel scale / coadd scale: 1.0011316555239373 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-copsf-z.fits.fz Stage fit_on_coadds : Wall: 1.33 s, CPU: 0.49 s, VmPeak: 945 MB, VmSize: 945 MB, VmRSS: 189 MB, VmData: 218 MB, maxrss: 0.19306 MB Stage fit_on_coadds finished: 2020-08-05T19:01:55.065840 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage fit_on_coadds : 0.146 s worker CPU, 0.290 s worker Wall, Wall: 1.329 s, Cores in use: 0.48, Total efficiency (on 32 cores): 1.5 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fit_on_coadds.p Running stage srcs Blob 11 finished, total: Wall: 0.10 s, CPU: 0.08 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 87 MB, VmData: 140 MB, maxrss: 0.089508 MB Blob 9 finished, total: Wall: 0.09 s, CPU: 0.09 s, VmPeak: 955 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148352 MB Blob 12 finished, total: Wall: 0.32 s, CPU: 0.30 s, VmPeak: 962 MB, VmSize: 732 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146776 MB Blob 10 finished, total: Wall: 0.41 s, CPU: 0.39 s, VmPeak: 957 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148 MB Blob 8 finished, total: Wall: 0.52 s, CPU: 0.52 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.09514 MB Blob 7 finished, total: Wall: 0.58 s, CPU: 0.55 s, VmPeak: 952 MB, VmSize: 732 MB, VmRSS: 144 MB, VmData: 180 MB, maxrss: 0.147644 MB Blob 6 finished, total: Wall: 0.81 s, CPU: 0.79 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 110 MB, VmData: 165 MB, maxrss: 0.112604 MB Blob 5 finished, total: Wall: 1.11 s, CPU: 1.11 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.093656 MB Blob 4 finished, total: Wall: 1.48 s, CPU: 1.47 s, VmPeak: 942 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148548 MB Measuring 1 iterative sources Blob 3 finished, total: Wall: 1.52 s, CPU: 1.50 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.147896 MB Blob 2 finished, total: Wall: 1.79 s, CPU: 1.76 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.096252 MB Running stage srcs at 2020-08-05T19:01:55.091011 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 8 potential peaks Of 8 potential peaks: 0 in veto map, 1 cut by saddle test, 0 cut by aper test, 7 kept SED z : 7 new peaks Found 8 potential peaks Of 8 potential peaks: 7 in veto map, 1 cut by saddle test, 0 cut by aper test, 0 kept SED r : 0 new peaks Found 9 potential peaks Of 9 potential peaks: 7 in veto map, 0 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 13 potential peaks Of 13 potential peaks: 9 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 15 potential peaks Of 15 potential peaks: 10 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 13 new sources Median coadd sky for g : 8.994588e-05 Median coadd sky for r : 9.102108e-06 Median coadd sky for z : -0.00018777419 Stage srcs : Wall: 3.18 s, CPU: 3.04 s, VmPeak: 945 MB, VmSize: 945 MB, VmRSS: 189 MB, VmData: 218 MB, maxrss: 0.193736 MB Stage srcs finished: 2020-08-05T19:01:58.272347 Resources for stage srcs : 0.102 s worker CPU, 0.177 s worker Wall, Wall: 3.184 s, Cores in use: 0.99, Total efficiency (on 32 cores): 3.1 % Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.65189, ee1=-1.05749, ee2=0.92598 Measuring 3 iterative sources Blob 1 finished, total: Wall: 5.04 s, CPU: 5.02 s, VmPeak: 673 MB, VmSize: 671 MB, VmRSS: 96 MB, VmData: 142 MB, maxrss: 0.099476 MB Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-srcs.p Running stage fitblobs Old catalog: 16 New catalog: 14 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/blobs-custom-212247m07045.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/all-models-custom-212247m07045.fits Stage fitblobs : Wall: 11.24 s, CPU: 1.37 s, VmPeak: 1043 MB, VmSize: 1043 MB, VmRSS: 212 MB, VmData: 238 MB, maxrss: 0.217504 MB Stage fitblobs finished: 2020-08-05T19:02:04.866056 Resources for stage fitblobs : 13.753 s worker CPU, 15.366 s worker Wall, Wall: 11.238 s, Cores in use: 1.35, Total efficiency (on 16 cores): 8.4 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fitblobs.p Running stage coadds Running stage fitblobs at 2020-08-05T19:02:04.642939 Reading /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p Running stage coadds at 2020-08-05T19:02:04.912256 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-ccds.fits Keeping 12 of 12 checkpointed results Skipping blob 6 Skipping blob 3 Skipping blob 10 Skipping blob 0 Skipping blob 7 Skipping blob 8 Skipping blob 11 Skipping blob 1 Skipping blob 5 Skipping blob 9 Skipping blob 4 Skipping blob 2 Old catalog: 16 New catalog: 14 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/blobs-custom-212247m07045.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/metrics/cus/all-models-custom-212247m07045.fits Stage fitblobs : Wall: 2.36 s, CPU: 2.20 s, VmPeak: 945 MB, VmSize: 945 MB, VmRSS: 190 MB, VmData: 219 MB, maxrss: 0.1942 MB Stage fitblobs finished: 2020-08-05T19:02:07.007659 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-nexp-g.fits.fz Resources for stage fitblobs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 2.367 s, Cores in use: 0.93, Total efficiency (on 32 cores): 2.9 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-fitblobs.p Running stage coadds Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Running stage coadds at 2020-08-05T19:02:07.068234 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image.jpg CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model.jpg CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel.jpg CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-resid.jpg CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-resid.jpg CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage coadds : Wall: 5.36 s, CPU: 2.24 s, VmPeak: 1043 MB, VmSize: 1043 MB, VmRSS: 213 MB, VmData: 238 MB, maxrss: 0.218052 MB Stage coadds finished: 2020-08-05T19:02:10.268471 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 12.665 s worker CPU, 22.555 s worker Wall, Wall: 5.358 s, Cores in use: 2.78, Total efficiency (on 16 cores): 17.4 % CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-coadds.p Running stage wise_forced CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Running stage wise_forced at 2020-08-05T19:02:10.299907 Cut to 1 unWISE tiles Reading WISE tile 2120m076 band 3 Reading WISE tile 2120m076 band 1 Reading WISE tile 2120m076 band 2 Reading WISE tile 2120m076 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.04 s, CPU: 0.04 s, VmPeak: 802 MB, VmSize: 802 MB, VmRSS: 141 MB, VmData: 174 MB, maxrss: 0.181088 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 956 MB, VmSize: 805 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.208728 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 957 MB, VmSize: 810 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.211184 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-resid.jpg Stage coadds : Wall: 5.53 s, CPU: 1.99 s, VmPeak: 949 MB, VmSize: 949 MB, VmRSS: 192 MB, VmData: 219 MB, maxrss: 0.196548 MB Stage coadds finished: 2020-08-05T19:02:12.597160 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 36.288 s worker CPU, 77.849 s worker Wall, Wall: 5.531 s, Cores in use: 6.92, Total efficiency (on 32 cores): 21.6 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 812 MB, VmSize: 812 MB, VmRSS: 151 MB, VmData: 180 MB, maxrss: 0.208596 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.88 s, CPU: 0.40 s, VmPeak: 1043 MB, VmSize: 1043 MB, VmRSS: 214 MB, VmData: 238 MB, maxrss: 0.21876 MB Stage wise_forced finished: 2020-08-05T19:02:13.174859 Resources for stage wise_forced : 3.839 s worker CPU, 6.659 s worker Wall, Wall: 2.877 s, Cores in use: 1.48, Total efficiency (on 16 cores): 9.2 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:02:13.201889 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/tractor-i/cus/tractor-custom-212247m07045.fits Reading SFD maps... Running stage wise_forced at 2020-08-05T19:02:12.625855 Cut to 1 unWISE tiles Reading WISE tile 2120m076 band 4 Reading WISE tile 2120m076 band 2 Reading WISE tile 2120m076 band 3 Reading WISE tile 2120m076 band 1 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 802 MB, VmSize: 802 MB, VmRSS: 142 MB, VmData: 174 MB, maxrss: 0.181112 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 862 MB, VmSize: 862 MB, VmRSS: 201 MB, VmData: 234 MB, maxrss: 0.205512 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 799 MB, VmSize: 799 MB, VmRSS: 137 MB, VmData: 171 MB, maxrss: 0.197548 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 863 MB, VmSize: 863 MB, VmRSS: 202 MB, VmData: 235 MB, maxrss: 0.206956 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/tmp-legacysurvey-custom-212247m07045-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 5.96 s, CPU: 2.89 s, VmPeak: 971 MB, VmSize: 971 MB, VmRSS: 195 MB, VmData: 220 MB, maxrss: 0.199688 MB Stage wise_forced finished: 2020-08-05T19:02:18.581510 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.130 s worker CPU, 7.842 s worker Wall, Wall: 5.958 s, Cores in use: 1.01, Total efficiency (on 32 cores): 3.2 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-wise_forced.p Running stage writecat END Rejected (1), keyrecords following the END keyrecord were ignored. Running stage writecat at 2020-08-05T19:02:18.601133 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/coadd/cus/custom-212247m07045/legacysurvey-custom-212247m07045-maskbits.fits.fz END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/tractor/cus/tractor-custom-212247m07045.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/tractor/cus/brick-custom-212247m07045.sha256sum Stage writecat : Wall: 7.14 s, CPU: 2.23 s, VmPeak: 1175 MB, VmSize: 1047 MB, VmRSS: 216 MB, VmData: 238 MB, maxrss: 0.351764 MB Stage writecat finished: 2020-08-05T19:02:20.343263 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 7.143 s, Cores in use: 0.31, Total efficiency (on 16 cores): 2.0 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/PGC1025201-largegalaxy-writecat.p All done: 35.147 s worker CPU, 102.237 s worker Wall, Wall: 49.083 s, Cores in use: 1.02, Total efficiency (on 16 cores): 6.4 % Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/tractor-i/cus/tractor-custom-212247m07045.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. Finished galaxy PGC1025201 in 0.863 minutes. END Rejected (1), keyrecords following the END keyrecord were ignored. Traceback (most recent call last): File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3771, in sys.exit(main()) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3739, in main run_brick(opt.brick, survey, **kwargs) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3295, in run_brick initial_args=initargs, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage R = stagefunc(stage, **Px) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3284, in mystagefunc R = stagefunc(stage, mp=mp, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__ rtn = super(CallGlobalTime, self).__call__(stage, **kwargs) File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__ return func(*self.args, **kwa) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 2839, in stage_writecat N_wise_epochs=15, motions=gaia_stars, gaia_tagalong=True) File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/survey.py", line 1098, in __exit__ f = open(self.tmpfn, 'wb') FileNotFoundError: [Errno 2] No such file or directory: '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/212/PGC1025201/tractor/cus/tmp-tractor-custom-212247m07045.fits' Something went wrong; please check the logfile. ERROR: galaxy PGC1025201; please check the logfile. Finished galaxy PGC1025201 in 0.657 minutes. Started working on galaxy PGC1025201 at Mon Aug 10 07:28:11 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 212.24746382963073 -7.046386054785208 --width 233 --height 233 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-10T07:28:12.562067 legacypipe git version: DR9.5.11-watershed-6-g6a2880fc Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '212.24746382963073', '-7.046386054785208', '--width', '233', '--height', '233', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 212.24746382963073 -7.046386054785208 --width 233 --height 233 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 212.24746382963073 -7.046386054785208 Starting process 15909 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059228 MB Starting process 15910 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 15911 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 15912 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 15913 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 15914 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 15915 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB Starting process 15916 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-10T07:28:13.061199 Keeping 7 CCDs: c4d_190307_065801_ooi_z_ls9-S29 z expnum 830719 exptime 80.0 propid 2014B-0404 seeing 0.93 MJD 58549.290 object DECaLS_17181_z decam/CP/V4.8.2a/CP20190306/c4d_190307_065801_ooi_z_ls9.fits.fz c4d_170304_075532_ooi_g_ls9-S29 g expnum 625787 exptime 154.0 propid 2014B-0404 seeing 1.30 MJD 57816.328 object DECaLS_17181_g decam/CP/V4.8.2/CP20170303/c4d_170304_075532_ooi_g_ls9.fits.fz c4d_160410_050652_ooi_r_ls9-S29 r expnum 536018 exptime 85.0 propid 2014B-0404 seeing 1.23 MJD 57488.212 object DECaLS_17181_r decam/CP/V4.8.2a/CP20160409/c4d_160410_050652_ooi_r_ls9.fits.fz c4d_160722_025838_ooi_z_ls9-S29 z expnum 553729 exptime 180.0 propid 2016A-0190 seeing 1.67 MJD 57591.121 object DECaLS_17181_z decam/CP/V4.8.2a/CP20160721/c4d_160722_025838_ooi_z_ls9.fits.fz c4d_170401_083230_ooi_g_ls9-N23 g expnum 636060 exptime 175.0 propid 2014B-0404 seeing 1.42 MJD 57844.353 object DECaLS_33067_g decam/CP/V4.8.2/CP20170331/c4d_170401_083230_ooi_g_ls9.fits.fz c4d_170505_013002_ooi_z_ls9-N23 z expnum 647369 exptime 196.0 propid 2014B-0404 seeing 0.71 MJD 57878.060 object DECaLS_33067_z decam/CP/V4.8.2/CP20170504/c4d_170505_013002_ooi_z_ls9.fits.fz c4d_170522_035509_ooi_r_ls9-N23 r expnum 650165 exptime 69.0 propid 2014B-0404 seeing 0.95 MJD 57895.162 object DECaLS_33067_r decam/CP/V4.8.2/CP20170521/c4d_170522_035509_ooi_r_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950f8db-g-20170304T075205_oi[S29,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c242-z-20160722T025448_oi[S29,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9535707-z-20190307T065801_oi[S29,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956cfb1-r-20160410T050436_oi[S29,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95173ea-g-20170401T082845_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951a62c-r-20170522T035307_oi[N23,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95155af-z-20170505T012557_oi[N23,noinherit][2046,4094][real]: DECaLS_ Stage tims : Wall: 28.65 s, CPU: 0.89 s, VmPeak: 1004 MB, VmSize: 1004 MB, VmRSS: 173 MB, VmData: 206 MB, maxrss: 0.176788 MB Stage tims finished: 2020-08-10T07:28:41.715753 Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_956cfb1-r-20160410T050436_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_950f8db-g-20170304T075205_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954c242-z-20160722T025448_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95173ea-g-20170401T082845_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95155af-z-20170505T012557_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9535707-z-20190307T065801_oi[S29,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Resources for stage tims : 4.490 s worker CPU, 185.767 s worker Wall, Wall: 28.656 s, Cores in use: 0.19, Total efficiency (on 8 cores): 2.3 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-tims.p Running stage refs Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951a62c-r-20170522T035307_oi[N23,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Running stage refs at 2020-08-10T07:28:41.732760 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06923.fits Matched 10 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/reference-custom-212247m07046.fits Stage refs : Wall: 16.81 s, CPU: 2.24 s, VmPeak: 1035 MB, VmSize: 1027 MB, VmRSS: 197 MB, VmData: 230 MB, maxrss: 0.201888 MB Stage refs finished: 2020-08-10T07:28:58.539474 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 16.808 s, Cores in use: 0.13, Total efficiency (on 8 cores): 1.7 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-10T07:28:58.557389 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/outlier-mask-custom-212247m07046.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/outliers-pre-custom-212247m07046.jpg Image c4d_170401_083230_ooi_g_ls9-N23 g : masking 344 positive outlier pixels and 0 negative outlier pixels Image c4d_160410_050652_ooi_r_ls9-S29 r : masking 112 positive outlier pixels and 0 negative outlier pixels Image c4d_160722_025838_ooi_z_ls9-S29 z : masking 123 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/outlier-mask-custom-212247m07046.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/outliers-post-custom-212247m07046.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/outliers-masked-pos-custom-212247m07046.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/outliers-masked-neg-custom-212247m07046.jpg Stage outliers : Wall: 0.86 s, CPU: 0.67 s, VmPeak: 1037 MB, VmSize: 1037 MB, VmRSS: 208 MB, VmData: 238 MB, maxrss: 0.2133 MB Stage outliers finished: 2020-08-10T07:28:59.420765 Resources for stage outliers : 0.438 s worker CPU, 0.457 s worker Wall, Wall: 0.864 s, Cores in use: 1.28, Total efficiency (on 8 cores): 16.1 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-10T07:28:59.486875 Stage halos : Wall: 1.10 s, CPU: 0.92 s, VmPeak: 1037 MB, VmSize: 1037 MB, VmRSS: 208 MB, VmData: 238 MB, maxrss: 0.213496 MB Stage halos finished: 2020-08-10T07:29:00.588443 Resources for stage halos : 0.102 s worker CPU, 0.715 s worker Wall, Wall: 1.103 s, Cores in use: 0.93, Total efficiency (on 8 cores): 11.6 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-10T07:29:00.611310 PSF sigmas in coadd pixels: ['2.10', '2.30'] Using average PSF sigma 2.2019974134807656 Gaussian PSF norm 0.12810859360999982 vs pixelized 0.11320412 average tim pixel scale / coadd scale: 1.0011475515473662 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.99', '1.54'] Using average PSF sigma 1.7656686059034938 Gaussian PSF norm 0.159766555757234 vs pixelized 0.1233821 average tim pixel scale / coadd scale: 1.001086816740869 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.52', '2.72', '1.16'] Using average PSF sigma 1.7969394431748915 Gaussian PSF norm 0.15698625395826574 vs pixelized 0.13192634 average tim pixel scale / coadd scale: 1.0011316555239373 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.14 s, CPU: 0.11 s, VmPeak: 1037 MB, VmSize: 1037 MB, VmRSS: 210 MB, VmData: 238 MB, maxrss: 0.215088 MB Stage fit_on_coadds finished: 2020-08-10T07:29:00.748669 Resources for stage fit_on_coadds : 0.099 s worker CPU, 0.099 s worker Wall, Wall: 0.138 s, Cores in use: 1.50, Total efficiency (on 8 cores): 18.8 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-10T07:29:00.761555 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 8 potential peaks Of 8 potential peaks: 1 in veto map, 1 cut by saddle test, 0 cut by aper test, 6 kept SED z : 6 new peaks Found 8 potential peaks Of 8 potential peaks: 7 in veto map, 1 cut by saddle test, 0 cut by aper test, 0 kept SED r : 0 new peaks Found 9 potential peaks Of 9 potential peaks: 7 in veto map, 0 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 13 potential peaks Of 13 potential peaks: 9 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 15 potential peaks Of 15 potential peaks: 10 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 12 new sources Median coadd sky for g : 0.00012445278 Median coadd sky for r : -1.5063742e-05 Median coadd sky for z : -8.224367e-05 Stage srcs : Wall: 0.87 s, CPU: 0.82 s, VmPeak: 1037 MB, VmSize: 1037 MB, VmRSS: 210 MB, VmData: 238 MB, maxrss: 0.215504 MB Stage srcs finished: 2020-08-10T07:29:01.634566 Resources for stage srcs : 0.073 s worker CPU, 0.073 s worker Wall, Wall: 0.874 s, Cores in use: 1.03, Total efficiency (on 8 cores): 12.8 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-10T07:29:01.645341 Blob 1 of 12, id: 6, sources: 1, size: 102x63, npix 4090, brick X: 65,167, Y: 82,145, one pixel: 117 82 Blob 2 of 12, id: 3, sources: 1, size: 49x40, npix 1237, brick X: 144,193, Y: 32,72, one pixel: 173 32 Blob 3 of 12, id: 0, sources: 2, size: 33x40, npix 917, brick X: 74,107, Y: 0,40, one pixel: 81 0 Blob 4 of 12, id: 10, sources: 1, size: 37x37, npix 889, brick X: 104,141, Y: 188,225, one pixel: 127 188 Blob 5 of 12, id: 7, sources: 1, size: 29x41, npix 704, brick X: 35,64, Y: 141,182, one pixel: 54 141 Blob 6 of 12, id: 8, sources: 1, size: 27x25, npix 427, brick X: 119,146, Y: 163,188, one pixel: 130 163 Blob 7 of 12, id: 11, sources: 1, size: 23x23, npix 325, brick X: 56,79, Y: 191,214, one pixel: 66 191 Blob 8 of 12, id: 1, sources: 1, size: 20x20, npix 220, brick X: 154,174, Y: 9,29, one pixel: 163 9 Blob 9 of 12, id: 5, sources: 1, size: 19x19, npix 217, brick X: 178,197, Y: 59,78, one pixel: 186 59 Blob 8 finished, total: Wall: 0.69 s, CPU: 0.67 s, VmPeak: 956 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.142956 MB Blob 10 of 12, id: 9, sources: 1, size: 18x19, npix 198, brick X: 85,103, Y: 177,196, one pixel: 93 177 Blob 9 finished, total: Wall: 0.09 s, CPU: 0.09 s, VmPeak: 956 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.142956 MB Blob 11 of 12, id: 4, sources: 1, size: 18x18, npix 171, brick X: 48,66, Y: 59,77, one pixel: 56 59 Blob 7 finished, total: Wall: 0.88 s, CPU: 0.70 s, VmPeak: 942 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.145076 MB Blob 12 of 12, id: 2, sources: 1, size: 17x17, npix 145, brick X: 44,61, Y: 23,40, one pixel: 52 23 Blob 11 finished, total: Wall: 0.07 s, CPU: 0.07 s, VmPeak: 942 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.145404 MB Blob 6 finished, total: Wall: 1.00 s, CPU: 1.00 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 111 MB, VmData: 166 MB, maxrss: 0.113492 MB Blob 10 finished, total: Wall: 0.36 s, CPU: 0.36 s, VmPeak: 956 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.142956 MB Blob 5 finished, total: Wall: 1.28 s, CPU: 1.28 s, VmPeak: 962 MB, VmSize: 725 MB, VmRSS: 140 MB, VmData: 180 MB, maxrss: 0.143428 MB Blob 12 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 942 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.145404 MB Measuring 1 iterative sources Blob 3 finished, total: Wall: 1.46 s, CPU: 1.46 s, VmPeak: 957 MB, VmSize: 725 MB, VmRSS: 141 MB, VmData: 180 MB, maxrss: 0.144784 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.59901, ee1=-0.857525, ee2=0.755496 Measuring 3 iterative sources Blob 4 finished, total: Wall: 1.97 s, CPU: 1.65 s, VmPeak: 956 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 180 MB, maxrss: 0.145048 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 2.31 s, CPU: 1.94 s, VmPeak: 939 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.14606 MB Blob 1 finished, total: Wall: 3.47 s, CPU: 3.45 s, VmPeak: 952 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146372 MB Old catalog: 15 New catalog: 14 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/blobs-custom-212247m07046.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/metrics/cus/all-models-custom-212247m07046.fits Stage fitblobs : Wall: 4.12 s, CPU: 0.62 s, VmPeak: 1037 MB, VmSize: 1037 MB, VmRSS: 211 MB, VmData: 238 MB, maxrss: 0.215668 MB Stage fitblobs finished: 2020-08-10T07:29:05.760508 Resources for stage fitblobs : 13.145 s worker CPU, 14.219 s worker Wall, Wall: 4.116 s, Cores in use: 3.34, Total efficiency (on 8 cores): 41.8 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-10T07:29:05.774294 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/tmp-legacysurvey-custom-212247m07046-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/tmp-legacysurvey-custom-212247m07046-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/tmp-legacysurvey-custom-212247m07046-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/tmp-legacysurvey-custom-212247m07046-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-resid.jpg Stage coadds : Wall: 1.90 s, CPU: 0.66 s, VmPeak: 1037 MB, VmSize: 1037 MB, VmRSS: 211 MB, VmData: 238 MB, maxrss: 0.21622 MB Stage coadds finished: 2020-08-10T07:29:07.673283 Resources for stage coadds : 4.361 s worker CPU, 4.811 s worker Wall, Wall: 1.900 s, Cores in use: 2.64, Total efficiency (on 8 cores): 33.0 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-10T07:29:07.683642 Cut to 1 unWISE tiles Reading WISE tile 2120m076 band 1 Reading WISE tile 2120m076 band 2 Reading WISE tile 2120m076 band 3 Reading WISE tile 2120m076 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 956 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.209612 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 952 MB, VmSize: 805 MB, VmRSS: 148 MB, VmData: 176 MB, maxrss: 0.19478 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 939 MB, VmSize: 799 MB, VmRSS: 142 MB, VmData: 170 MB, maxrss: 0.205328 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/212/2120m076/unwise-2120m076-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.03 s, CPU: 0.03 s, VmPeak: 942 MB, VmSize: 800 MB, VmRSS: 142 MB, VmData: 172 MB, maxrss: 0.205652 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/tmp-legacysurvey-custom-212247m07046-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/tmp-legacysurvey-custom-212247m07046-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.07 s, CPU: 0.41 s, VmPeak: 1037 MB, VmSize: 1037 MB, VmRSS: 212 MB, VmData: 238 MB, maxrss: 0.216924 MB Stage wise_forced finished: 2020-08-10T07:29:09.750318 Resources for stage wise_forced : 3.764 s worker CPU, 5.283 s worker Wall, Wall: 2.068 s, Cores in use: 2.02, Total efficiency (on 8 cores): 25.3 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-10T07:29:09.763013 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/coadd/cus/custom-212247m07046/legacysurvey-custom-212247m07046-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/tractor-i/cus/tractor-custom-212247m07046.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/tractor/cus/tractor-custom-212247m07046.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/tractor/cus/brick-custom-212247m07046.sha256sum Stage writecat : Wall: 1.77 s, CPU: 0.61 s, VmPeak: 1169 MB, VmSize: 1041 MB, VmRSS: 214 MB, VmData: 238 MB, maxrss: 0.349912 MB Stage writecat finished: 2020-08-10T07:29:11.537328 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 1.776 s, Cores in use: 0.35, Total efficiency (on 8 cores): 4.3 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/212/PGC1025201/PGC1025201-largegalaxy-writecat.p All done: 26.473 s worker CPU, 211.426 s worker Wall, Wall: 58.492 s, Cores in use: 0.59, Total efficiency (on 8 cores): 7.4 % Finished galaxy PGC1025201 in 1.005 minutes.