Started working on galaxy PGC1051678 at Wed Aug 5 19:38:23 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.341923 -4.9225598 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T19:38:27.123781 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '323.341923', '-4.9225598', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.341923 -4.9225598 --width 245 --height 245 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 323.341923 -4.9225598 Starting process 29900 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 29901 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 29902 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 29903 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 29904 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059348 MB Starting process 29905 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059336 MB Starting process 29906 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059336 MB Starting process 29907 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 29908 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 29909 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 29910 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 29911 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 29912 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 29913 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 29914 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 29915 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T19:38:28.903754 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 11 CCDs: c4d_180905_013038_ooi_g_ls9-S16 g expnum 768987 exptime 66.0 propid 2014B-0404 seeing 1.20 MJD 58366.063 object DECaLS_52078_g decam/CP/V4.8.2a/CP20180904/c4d_180905_013038_ooi_g_ls9.fits.fz c4d_180906_005235_ooi_g_ls9-S19 g expnum 769481 exptime 57.0 propid 2014B-0404 seeing 1.12 MJD 58367.037 object DECaLS_59215_g decam/CP/V4.8.2a/CP20180905/c4d_180906_005235_ooi_g_ls9.fits.fz c4d_170722_042032_ooi_r_ls9-N5 r expnum 660878 exptime 40.0 propid 2014B-0404 seeing 0.86 MJD 57956.180 object DECaLS_11759_r decam/CP/V4.8.2a/CP20170721/c4d_170722_042032_ooi_r_ls9.fits.fz c4d_160813_024709_ooi_z_ls9-N5 z expnum 563504 exptime 82.0 propid 2014B-0404 seeing 0.85 MJD 57613.114 object DECaLS_11759_z decam/CP/V4.8.2a/CP20160812/c4d_160813_024709_ooi_z_ls9.fits.fz c4d_160803_043122_ooi_g_ls9-N5 g expnum 558558 exptime 57.0 propid 2014B-0404 seeing 1.07 MJD 57603.187 object DECaLS_11759_g decam/CP/V4.8.2a/CP20160802/c4d_160803_043122_ooi_g_ls9.fits.fz c4d_181005_235712_ooi_z_ls9-N10 z expnum 780919 exptime 98.0 propid 2014B-0404 seeing 1.12 MJD 58396.998 object DECaLS_43503_z decam/CP/V4.8.2a/CP20181005/c4d_181005_235712_ooi_z_ls9.fits.fz c4d_170915_010700_ooi_r_ls9-N10 r expnum 677683 exptime 175.0 propid 2014B-0404 seeing 1.85 MJD 58011.047 object DECaLS_43503_r decam/CP/V4.8.2a/CP20170914/c4d_170915_010700_ooi_r_ls9.fits.fz c4d_180910_021336_ooi_r_ls9-N10 r expnum 771431 exptime 40.0 propid 2014B-0404 seeing 1.36 MJD 58371.093 object DECaLS_43503_r decam/CP/V4.8.2a/CP20180909/c4d_180910_021336_ooi_r_ls9.fits.fz c4d_170917_003033_ooi_r_ls9-N11 r expnum 678261 exptime 98.0 propid 2014B-0404 seeing 1.12 MJD 58013.021 object DECaLS_27631_r decam/CP/V4.8.2a/CP20170916/c4d_170917_003033_ooi_r_ls9.fits.fz c4d_170726_041020_ooi_g_ls9-N11 g expnum 662261 exptime 110.0 propid 2014B-0404 seeing 1.11 MJD 57960.172 object DECaLS_27631_g decam/CP/V4.8.2a/CP20170725/c4d_170726_041020_ooi_g_ls9.fits.fz c4d_160814_032505_ooi_z_ls9-N11 z expnum 563870 exptime 80.0 propid 2014B-0404 seeing 1.07 MJD 57614.141 object DECaLS_27631_z decam/CP/V4.8.2a/CP20160813/c4d_160814_032505_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9522538-r-20170915T010700_oi[N10,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954922e-z-20160813T024449_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95488c8-z-20160814T032253_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_N10-r-20180910T021336_oi[N10,noinherit][2046,4094][real]: DECaLS_4350 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530bc0-z-20181005T235712_oi[N10,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_964e77a-g-20180906T005235_oi[S19,noinherit][2046,4094][real]: DECaLS_ Masking 0 additional saturated pixels in DECam expnum 769481 S19 CCD, decam/CP/V4.8.2a/CP20180905/c4d_180906_005235_ooi_g_ls9.fits.fz slice (slice(3211, 3457, None), slice(365, 611, None)) The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951dcf7-r-20170722T041859_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95203e2-g-20170726T040738_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9534322-g-20180905T013038_oi[S16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9523600-r-20170917T003033_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95b90ac-g-20160803T042933_oi[N5,noinherit][2046,4094][real]: DECaLS_1 Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 47.11 s, CPU: 1.59 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 179 MB, VmData: 211 MB, maxrss: 0.183016 MB Stage tims finished: 2020-08-05T19:39:16.014373 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951dcf7-r-20170722T041859_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_964e77a-g-20180906T005235_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530bc0-z-20181005T235712_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95b90ac-g-20160803T042933_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9523600-r-20170917T003033_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9534322-g-20180905T013038_oi[S16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_N10-r-20180910T021336_oi[N10,noinherit][2046,4094][real]: DECaLS_4350 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954922e-z-20160813T024449_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9522538-r-20170915T010700_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 7.062 s worker CPU, 272.295 s worker Wall, Wall: 47.112 s, Cores in use: 0.18, Total efficiency (on 16 cores): 1.1 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95203e2-g-20170726T040738_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95488c8-z-20160814T032253_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T19:39:16.224457 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06707.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06706.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06835.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06579.fits Matched 1 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/reference-custom-323341m04922.fits Stage refs : Wall: 7.63 s, CPU: 4.23 s, VmPeak: 1051 MB, VmSize: 1045 MB, VmRSS: 213 MB, VmData: 244 MB, maxrss: 0.218356 MB Stage refs finished: 2020-08-05T19:39:23.857124 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 7.634 s, Cores in use: 0.55, Total efficiency (on 16 cores): 3.5 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T19:39:24.028488 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outlier-mask-custom-323341m04922.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-pre-custom-323341m04922.jpg Image c4d_160803_043122_ooi_g_ls9-N5 g : masking 440 positive outlier pixels and 0 negative outlier pixels Image c4d_180906_005235_ooi_g_ls9-S19 g : masking 101 positive outlier pixels and 0 negative outlier pixels Image c4d_170915_010700_ooi_r_ls9-N10 r : masking 3164 positive outlier pixels and 0 negative outlier pixels Image c4d_170917_003033_ooi_r_ls9-N11 r : masking 287 positive outlier pixels and 0 negative outlier pixels Image c4d_181005_235712_ooi_z_ls9-N10 z : masking 282 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outlier-mask-custom-323341m04922.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-post-custom-323341m04922.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-masked-pos-custom-323341m04922.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-masked-neg-custom-323341m04922.jpg Stage outliers : Wall: 1.36 s, CPU: 1.05 s, VmPeak: 1054 MB, VmSize: 1054 MB, VmRSS: 227 MB, VmData: 255 MB, maxrss: 0.232504 MB Stage outliers finished: 2020-08-05T19:39:25.387095 Resources for stage outliers : 0.941 s worker CPU, 1.042 s worker Wall, Wall: 1.360 s, Cores in use: 1.47, Total efficiency (on 16 cores): 9.2 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T19:39:25.549705 Stage halos : Wall: 2.22 s, CPU: 1.46 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 227 MB, VmData: 255 MB, maxrss: 0.232948 MB Stage halos finished: 2020-08-05T19:39:27.773090 Resources for stage halos : 0.423 s worker CPU, 2.623 s worker Wall, Wall: 2.225 s, Cores in use: 0.85, Total efficiency (on 16 cores): 5.3 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T19:39:27.813765 PSF sigmas in coadd pixels: ['1.95', '1.82', '1.73', '1.80'] Using average PSF sigma 1.8275361084147765 Gaussian PSF norm 0.154357985308739 vs pixelized 0.12569062 average tim pixel scale / coadd scale: 1.001099038683411 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.39', '3.01', '2.21', '1.82'] Using average PSF sigma 2.1075695296135075 Gaussian PSF norm 0.1338483916236963 vs pixelized 0.09821714 average tim pixel scale / coadd scale: 1.0010999384583223 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.38', '1.82', '1.74'] Using average PSF sigma 1.6443128746417466 Gaussian PSF norm 0.171557856247607 vs pixelized 0.14172818 average tim pixel scale / coadd scale: 1.0010343132085586 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.19 s, CPU: 0.16 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 230 MB, VmData: 256 MB, maxrss: 0.235468 MB Stage fit_on_coadds finished: 2020-08-05T19:39:28.004853 Resources for stage fit_on_coadds : 0.228 s worker CPU, 0.228 s worker Wall, Wall: 0.193 s, Cores in use: 2.04, Total efficiency (on 16 cores): 12.7 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T19:39:28.021353 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 31 potential peaks Of 31 potential peaks: 7 in veto map, 3 cut by saddle test, 0 cut by aper test, 21 kept SED z : 21 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 33 potential peaks Of 33 potential peaks: 30 in veto map, 2 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 36 potential peaks Of 36 potential peaks: 32 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 28 new sources Median coadd sky for g : 2.3440198e-05 Median coadd sky for r : -0.0001830734 Median coadd sky for z : 0.0004802092 Stage srcs : Wall: 1.10 s, CPU: 1.03 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 231 MB, VmData: 256 MB, maxrss: 0.236072 MB Stage srcs finished: 2020-08-05T19:39:29.119996 Resources for stage srcs : 0.082 s worker CPU, 0.082 s worker Wall, Wall: 1.100 s, Cores in use: 1.01, Total efficiency (on 16 cores): 6.3 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T19:39:29.146946 Blob 1 of 13, id: 8, sources: 10, size: 144x160, npix 10221, brick X: 71,215, Y: 64,224, one pixel: 103 64 Blob 2 of 13, id: 12, sources: 9, size: 110x83, npix 4345, brick X: 8,118, Y: 162,245, one pixel: 70 162 Blob 3 of 13, id: 3, sources: 3, size: 57x45, npix 1708, brick X: 0,57, Y: 29,74, one pixel: 45 29 Blob 4 of 13, id: 2, sources: 2, size: 39x52, npix 1340, brick X: 206,245, Y: 9,61, one pixel: 230 9 Blob 5 of 13, id: 10, sources: 2, size: 24x52, npix 905, brick X: 0,24, Y: 132,184, one pixel: 11 132 Blob 6 of 13, id: 4, sources: 2, size: 35x26, npix 549, brick X: 98,133, Y: 32,58, one pixel: 120 32 Blob 7 of 13, id: 9, sources: 1, size: 24x29, npix 495, brick X: 221,245, Y: 113,142, one pixel: 233 113 Blob 8 of 13, id: 11, sources: 1, size: 25x25, npix 402, brick X: 96,121, Y: 153,178, one pixel: 106 153 Blob 9 of 13, id: 0, sources: 1, size: 23x26, npix 361, brick X: 118,141, Y: 5,31, one pixel: 130 5 Blob 10 of 13, id: 6, sources: 1, size: 24x23, npix 318, brick X: 188,212, Y: 47,70, one pixel: 198 47 Blob 11 of 13, id: 1, sources: 1, size: 19x19, npix 199, brick X: 177,196, Y: 8,27, one pixel: 186 8 Blob 12 of 13, id: 5, sources: 1, size: 18x18, npix 180, brick X: 137,155, Y: 33,51, one pixel: 145 33 Blob 13 of 13, id: 7, sources: 1, size: 11x20, npix 142, brick X: 234,245, Y: 52,72, one pixel: 242 52 Blob 11 finished, total: Wall: 0.08 s, CPU: 0.08 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 88 MB, VmData: 141 MB, maxrss: 0.090512 MB Blob 12 finished, total: Wall: 0.33 s, CPU: 0.30 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.095184 MB Blob 13 finished, total: Wall: 0.45 s, CPU: 0.42 s, VmPeak: 950 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149656 MB Blob 10 finished, total: Wall: 0.62 s, CPU: 0.61 s, VmPeak: 693 MB, VmSize: 693 MB, VmRSS: 110 MB, VmData: 164 MB, maxrss: 0.1127 MB Blob 8 finished, total: Wall: 0.68 s, CPU: 0.66 s, VmPeak: 953 MB, VmSize: 737 MB, VmRSS: 152 MB, VmData: 186 MB, maxrss: 0.155596 MB Blob 6 finished, total: Wall: 0.67 s, CPU: 0.64 s, VmPeak: 960 MB, VmSize: 732 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.147944 MB Blob 9 finished, total: Wall: 0.90 s, CPU: 0.87 s, VmPeak: 958 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149728 MB Blob 7 finished, total: Wall: 1.15 s, CPU: 1.12 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 181 MB, maxrss: 0.151116 MB Blob 4 finished, total: Wall: 1.52 s, CPU: 1.52 s, VmPeak: 956 MB, VmSize: 732 MB, VmRSS: 147 MB, VmData: 181 MB, maxrss: 0.150068 MB Blob 5 finished, total: Wall: 1.73 s, CPU: 1.71 s, VmPeak: 956 MB, VmSize: 738 MB, VmRSS: 152 MB, VmData: 186 MB, maxrss: 0.156056 MB Blob 3 finished, total: Wall: 2.47 s, CPU: 2.44 s, VmPeak: 955 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.148248 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 7.55 s, CPU: 7.52 s, VmPeak: 921 MB, VmSize: 732 MB, VmRSS: 148 MB, VmData: 181 MB, maxrss: 0.15204 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.63642, ee1=-0.901032, ee2=-0.195639 Measuring 5 iterative sources Blob 1 finished, total: Wall: 12.77 s, CPU: 12.77 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 125 MB, VmData: 166 MB, maxrss: 0.128252 MB Old catalog: 41 New catalog: 38 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/blobs-custom-323341m04922.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/all-models-custom-323341m04922.fits Stage fitblobs : Wall: 15.92 s, CPU: 2.94 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 231 MB, VmData: 256 MB, maxrss: 0.236236 MB Stage fitblobs finished: 2020-08-05T19:39:45.070468 Resources for stage fitblobs : 30.894 s worker CPU, 32.637 s worker Wall, Wall: 15.925 s, Cores in use: 2.12, Total efficiency (on 16 cores): 13.3 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T19:39:45.088821 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-resid.jpg Stage coadds : Wall: 4.79 s, CPU: 2.02 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 231 MB, VmData: 256 MB, maxrss: 0.236752 MB Stage coadds finished: 2020-08-05T19:39:49.881796 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 9.553 s worker CPU, 18.970 s worker Wall, Wall: 4.795 s, Cores in use: 2.41, Total efficiency (on 16 cores): 15.1 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T19:39:49.901794 Cut to 1 unWISE tiles Reading WISE tile 3235m046 band 1 Reading WISE tile 3235m046 band 3 Reading WISE tile 3235m046 band 2 Reading WISE tile 3235m046 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 960 MB, VmSize: 809 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.211104 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 956 MB, VmSize: 812 MB, VmRSS: 152 MB, VmData: 177 MB, maxrss: 0.196264 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 956 MB, VmSize: 809 MB, VmRSS: 151 MB, VmData: 174 MB, maxrss: 0.212284 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 800 MB, VmSize: 800 MB, VmRSS: 140 MB, VmData: 172 MB, maxrss: 0.203032 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 4.26 s, CPU: 1.74 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 232 MB, VmData: 256 MB, maxrss: 0.23746 MB Stage wise_forced finished: 2020-08-05T19:39:54.165841 Resources for stage wise_forced : 3.214 s worker CPU, 6.731 s worker Wall, Wall: 4.266 s, Cores in use: 1.16, Total efficiency (on 16 cores): 7.3 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-05T19:39:54.179545 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/tractor-i/cus/tractor-custom-323341m04922.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/tractor/cus/tractor-custom-323341m04922.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/tractor/cus/brick-custom-323341m04922.sha256sum Stage writecat : Wall: 0.99 s, CPU: 0.62 s, VmPeak: 1187 MB, VmSize: 1059 MB, VmRSS: 234 MB, VmData: 256 MB, maxrss: 0.37034 MB Stage writecat finished: 2020-08-05T19:39:55.170850 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.993 s, Cores in use: 0.62, Total efficiency (on 16 cores): 3.9 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-writecat.p All done: 52.399 s worker CPU, 334.611 s worker Wall, Wall: 86.292 s, Cores in use: 0.80, Total efficiency (on 16 cores): 5.0 % Finished galaxy PGC1051678 in 1.531 minutes. Started working on galaxy PGC1051678 at Wed Aug 5 20:12:12 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.341923 -4.9225598 --width 245 --height 245 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-05T20:12:15.056201 legacypipe git version: DR9.5.10-12-gf22e529f Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '323.341923', '-4.9225598', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.341923 -4.9225598 --width 245 --height 245 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 323.341923 -4.9225598 Starting process 36914 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB Starting process 36915 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Starting process 36916 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 36917 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 36918 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059412 MB Starting process 36919 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05936 MB Starting process 36920 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05936 MB Starting process 36921 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 36922 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB Starting process 36923 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 36924 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB Starting process 36925 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 36926 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 36927 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB Starting process 36928 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 36929 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB Starting process 36930 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059276 MB Starting process 36931 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB Starting process 36932 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB Starting process 36933 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 36934 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 36935 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059288 MB Starting process 36936 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB Starting process 36937 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 36938 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 36939 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB Starting process 36940 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB Starting process 36941 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB Starting process 36942 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Starting process 36943 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Starting process 36944 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059312 MB Starting process 36945 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059316 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-05T20:12:20.501142 CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Keeping 11 CCDs: c4d_180905_013038_ooi_g_ls9-S16 g expnum 768987 exptime 66.0 propid 2014B-0404 seeing 1.20 MJD 58366.063 object DECaLS_52078_g decam/CP/V4.8.2a/CP20180904/c4d_180905_013038_ooi_g_ls9.fits.fz c4d_180906_005235_ooi_g_ls9-S19 g expnum 769481 exptime 57.0 propid 2014B-0404 seeing 1.12 MJD 58367.037 object DECaLS_59215_g decam/CP/V4.8.2a/CP20180905/c4d_180906_005235_ooi_g_ls9.fits.fz c4d_170722_042032_ooi_r_ls9-N5 r expnum 660878 exptime 40.0 propid 2014B-0404 seeing 0.86 MJD 57956.180 object DECaLS_11759_r decam/CP/V4.8.2a/CP20170721/c4d_170722_042032_ooi_r_ls9.fits.fz c4d_160813_024709_ooi_z_ls9-N5 z expnum 563504 exptime 82.0 propid 2014B-0404 seeing 0.85 MJD 57613.114 object DECaLS_11759_z decam/CP/V4.8.2a/CP20160812/c4d_160813_024709_ooi_z_ls9.fits.fz c4d_160803_043122_ooi_g_ls9-N5 g expnum 558558 exptime 57.0 propid 2014B-0404 seeing 1.07 MJD 57603.187 object DECaLS_11759_g decam/CP/V4.8.2a/CP20160802/c4d_160803_043122_ooi_g_ls9.fits.fz c4d_181005_235712_ooi_z_ls9-N10 z expnum 780919 exptime 98.0 propid 2014B-0404 seeing 1.12 MJD 58396.998 object DECaLS_43503_z decam/CP/V4.8.2a/CP20181005/c4d_181005_235712_ooi_z_ls9.fits.fz c4d_170915_010700_ooi_r_ls9-N10 r expnum 677683 exptime 175.0 propid 2014B-0404 seeing 1.85 MJD 58011.047 object DECaLS_43503_r decam/CP/V4.8.2a/CP20170914/c4d_170915_010700_ooi_r_ls9.fits.fz c4d_180910_021336_ooi_r_ls9-N10 r expnum 771431 exptime 40.0 propid 2014B-0404 seeing 1.36 MJD 58371.093 object DECaLS_43503_r decam/CP/V4.8.2a/CP20180909/c4d_180910_021336_ooi_r_ls9.fits.fz c4d_170917_003033_ooi_r_ls9-N11 r expnum 678261 exptime 98.0 propid 2014B-0404 seeing 1.12 MJD 58013.021 object DECaLS_27631_r decam/CP/V4.8.2a/CP20170916/c4d_170917_003033_ooi_r_ls9.fits.fz c4d_170726_041020_ooi_g_ls9-N11 g expnum 662261 exptime 110.0 propid 2014B-0404 seeing 1.11 MJD 57960.172 object DECaLS_27631_g decam/CP/V4.8.2a/CP20170725/c4d_170726_041020_ooi_g_ls9.fits.fz c4d_160814_032505_ooi_z_ls9-N11 z expnum 563870 exptime 80.0 propid 2014B-0404 seeing 1.07 MJD 57614.141 object DECaLS_27631_z decam/CP/V4.8.2a/CP20160813/c4d_160814_032505_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9534322-g-20180905T013038_oi[S16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95203e2-g-20170726T040738_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9523600-r-20170917T003033_oi[N11,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95b90ac-g-20160803T042933_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_964e77a-g-20180906T005235_oi[S19,noinherit][2046,4094][real]: DECaLS_ Masking 0 additional saturated pixels in DECam expnum 769481 S19 CCD, decam/CP/V4.8.2a/CP20180905/c4d_180906_005235_ooi_g_ls9.fits.fz slice (slice(3211, 3457, None), slice(365, 611, None)) The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9522538-r-20170915T010700_oi[N10,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_N10-r-20180910T021336_oi[N10,noinherit][2046,4094][real]: DECaLS_4350 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954922e-z-20160813T024449_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530bc0-z-20181005T235712_oi[N10,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951dcf7-r-20170722T041859_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95488c8-z-20160814T032253_oi[N11,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Warning: splinesky does not have sky-template version/run/scale values CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Stage tims : Wall: 82.59 s, CPU: 3.90 s, VmPeak: 1011 MB, VmSize: 1011 MB, VmRSS: 179 MB, VmData: 211 MB, maxrss: 0.183176 MB Stage tims finished: 2020-08-05T20:13:43.092093 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9523600-r-20170917T003033_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_964e77a-g-20180906T005235_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530bc0-z-20181005T235712_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954922e-z-20160813T024449_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951dcf7-r-20170722T041859_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9522538-r-20170915T010700_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95488c8-z-20160814T032253_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage tims : 6.433 s worker CPU, 438.831 s worker Wall, Wall: 82.594 s, Cores in use: 0.13, Total efficiency (on 32 cores): 0.4 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-tims.p Running stage refs WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95203e2-g-20170726T040738_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95b90ac-g-20160803T042933_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_N10-r-20180910T021336_oi[N10,noinherit][2046,4094][real]: DECaLS_4350 [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage refs at 2020-08-05T20:13:44.023269 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06707.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06706.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06835.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06579.fits Matched 1 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/reference-custom-323341m04922.fits Stage refs : Wall: 7.65 s, CPU: 4.14 s, VmPeak: 1051 MB, VmSize: 1045 MB, VmRSS: 213 MB, VmData: 244 MB, maxrss: 0.218452 MB Stage refs finished: 2020-08-05T20:13:51.676270 Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 7.655 s, Cores in use: 0.54, Total efficiency (on 32 cores): 1.7 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-05T20:13:51.839033 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outlier-mask-custom-323341m04922.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-pre-custom-323341m04922.jpg Image c4d_160803_043122_ooi_g_ls9-N5 g : masking 440 positive outlier pixels and 0 negative outlier pixels Image c4d_180906_005235_ooi_g_ls9-S19 g : masking 101 positive outlier pixels and 0 negative outlier pixels Image c4d_170917_003033_ooi_r_ls9-N11 r : masking 287 positive outlier pixels and 0 negative outlier pixels Image c4d_170915_010700_ooi_r_ls9-N10 r : masking 3164 positive outlier pixels and 0 negative outlier pixels Image c4d_181005_235712_ooi_z_ls9-N10 z : masking 282 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outlier-mask-custom-323341m04922.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-post-custom-323341m04922.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-masked-pos-custom-323341m04922.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/outliers-masked-neg-custom-323341m04922.jpg Stage outliers : Wall: 1.17 s, CPU: 0.87 s, VmPeak: 1054 MB, VmSize: 1054 MB, VmRSS: 228 MB, VmData: 256 MB, maxrss: 0.233416 MB Stage outliers finished: 2020-08-05T20:13:53.011632 WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9534322-g-20180905T013038_oi[S16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage outliers : 1.002 s worker CPU, 1.215 s worker Wall, Wall: 1.175 s, Cores in use: 1.60, Total efficiency (on 32 cores): 5.0 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-05T20:13:53.074014 Stage halos : Wall: 2.32 s, CPU: 0.34 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 228 MB, VmData: 256 MB, maxrss: 0.23376 MB Stage halos finished: 2020-08-05T20:13:55.394585 Resources for stage halos : 0.433 s worker CPU, 4.845 s worker Wall, Wall: 2.323 s, Cores in use: 0.33, Total efficiency (on 32 cores): 1.0 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-05T20:13:55.425366 PSF sigmas in coadd pixels: ['1.95', '1.82', '1.73', '1.80'] Using average PSF sigma 1.8275361084147765 Gaussian PSF norm 0.154357985308739 vs pixelized 0.12569062 average tim pixel scale / coadd scale: 1.001099038683411 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.39', '3.01', '2.21', '1.82'] Using average PSF sigma 2.1075695296135075 Gaussian PSF norm 0.1338483916236963 vs pixelized 0.09821714 average tim pixel scale / coadd scale: 1.0010999384583223 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.38', '1.82', '1.74'] Using average PSF sigma 1.6443128746417466 Gaussian PSF norm 0.171557856247607 vs pixelized 0.14172818 average tim pixel scale / coadd scale: 1.0010343132085586 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.18 s, CPU: 0.17 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 230 MB, VmData: 256 MB, maxrss: 0.235352 MB Stage fit_on_coadds finished: 2020-08-05T20:13:55.606344 Resources for stage fit_on_coadds : 0.194 s worker CPU, 0.194 s worker Wall, Wall: 0.183 s, Cores in use: 1.98, Total efficiency (on 32 cores): 6.2 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-05T20:13:55.618548 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 31 potential peaks Of 31 potential peaks: 7 in veto map, 3 cut by saddle test, 0 cut by aper test, 21 kept SED z : 21 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED r : 2 new peaks Found 29 potential peaks Of 29 potential peaks: 25 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED g : 2 new peaks Found 33 potential peaks Of 33 potential peaks: 30 in veto map, 2 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 36 potential peaks Of 36 potential peaks: 32 in veto map, 2 cut by saddle test, 0 cut by aper test, 2 kept SED Red : 2 new peaks Adding 28 new sources Median coadd sky for g : 2.3440125e-05 Median coadd sky for r : -0.00018307343 Median coadd sky for z : 0.0004802092 Stage srcs : Wall: 1.53 s, CPU: 1.47 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 230 MB, VmData: 256 MB, maxrss: 0.235828 MB Stage srcs finished: 2020-08-05T20:13:57.151941 Resources for stage srcs : 0.072 s worker CPU, 0.072 s worker Wall, Wall: 1.535 s, Cores in use: 1.00, Total efficiency (on 32 cores): 3.1 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-05T20:13:57.408252 Blob 1 of 13, id: 8, sources: 10, size: 144x160, npix 10221, brick X: 71,215, Y: 64,224, one pixel: 103 64 Blob 2 of 13, id: 12, sources: 9, size: 110x83, npix 4345, brick X: 8,118, Y: 162,245, one pixel: 70 162 Blob 3 of 13, id: 3, sources: 3, size: 57x45, npix 1708, brick X: 0,57, Y: 29,74, one pixel: 45 29 Blob 4 of 13, id: 2, sources: 2, size: 39x52, npix 1340, brick X: 206,245, Y: 9,61, one pixel: 230 9 Blob 5 of 13, id: 10, sources: 2, size: 24x52, npix 905, brick X: 0,24, Y: 132,184, one pixel: 11 132 Blob 6 of 13, id: 4, sources: 2, size: 35x26, npix 549, brick X: 98,133, Y: 32,58, one pixel: 120 32 Blob 7 of 13, id: 9, sources: 1, size: 24x29, npix 495, brick X: 221,245, Y: 113,142, one pixel: 233 113 Blob 8 of 13, id: 11, sources: 1, size: 25x25, npix 402, brick X: 96,121, Y: 153,178, one pixel: 106 153 Blob 9 of 13, id: 0, sources: 1, size: 23x26, npix 361, brick X: 118,141, Y: 5,31, one pixel: 130 5 Blob 10 of 13, id: 6, sources: 1, size: 24x23, npix 318, brick X: 188,212, Y: 47,70, one pixel: 198 47 Blob 11 of 13, id: 1, sources: 1, size: 19x19, npix 199, brick X: 177,196, Y: 8,27, one pixel: 186 8 Blob 12 of 13, id: 5, sources: 1, size: 18x18, npix 180, brick X: 137,155, Y: 33,51, one pixel: 145 33 Blob 13 of 13, id: 7, sources: 1, size: 11x20, npix 142, brick X: 234,245, Y: 52,72, one pixel: 242 52 Blob 11 finished, total: Wall: 0.09 s, CPU: 0.08 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 107 MB, VmData: 166 MB, maxrss: 0.110036 MB Blob 12 finished, total: Wall: 0.23 s, CPU: 0.20 s, VmPeak: 694 MB, VmSize: 694 MB, VmRSS: 112 MB, VmData: 165 MB, maxrss: 0.114548 MB Blob 10 finished, total: Wall: 0.41 s, CPU: 0.40 s, VmPeak: 955 MB, VmSize: 732 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.1481 MB Blob 13 finished, total: Wall: 0.42 s, CPU: 0.41 s, VmPeak: 954 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.1496 MB Blob 8 finished, total: Wall: 0.57 s, CPU: 0.56 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 112 MB, VmData: 166 MB, maxrss: 0.115104 MB Blob 6 finished, total: Wall: 0.66 s, CPU: 0.64 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 92 MB, VmData: 140 MB, maxrss: 0.09422 MB Blob 9 finished, total: Wall: 0.66 s, CPU: 0.65 s, VmPeak: 957 MB, VmSize: 732 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.14786 MB Blob 7 finished, total: Wall: 0.94 s, CPU: 0.92 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 114 MB, VmData: 166 MB, maxrss: 0.116328 MB Blob 4 finished, total: Wall: 1.31 s, CPU: 1.29 s, VmPeak: 950 MB, VmSize: 732 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149916 MB Blob 5 finished, total: Wall: 1.50 s, CPU: 1.48 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 91 MB, VmData: 139 MB, maxrss: 0.09366 MB Blob 3 finished, total: Wall: 2.17 s, CPU: 2.15 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 93 MB, VmData: 140 MB, maxrss: 0.095564 MB Measuring 1 iterative sources Blob 2 finished, total: Wall: 7.18 s, CPU: 7.17 s, VmPeak: 675 MB, VmSize: 672 MB, VmRSS: 97 MB, VmData: 143 MB, maxrss: 0.102348 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.63642, ee1=-0.901032, ee2=-0.195639 Measuring 5 iterative sources Blob 1 finished, total: Wall: 12.44 s, CPU: 12.42 s, VmPeak: 685 MB, VmSize: 679 MB, VmRSS: 104 MB, VmData: 150 MB, maxrss: 0.112712 MB Old catalog: 41 New catalog: 38 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/blobs-custom-323341m04922.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/metrics/cus/all-models-custom-323341m04922.fits Stage fitblobs : Wall: 14.33 s, CPU: 1.77 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 230 MB, VmData: 256 MB, maxrss: 0.235992 MB Stage fitblobs finished: 2020-08-05T20:14:11.734888 Resources for stage fitblobs : 28.555 s worker CPU, 29.207 s worker Wall, Wall: 14.329 s, Cores in use: 2.12, Total efficiency (on 32 cores): 6.6 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-05T20:14:11.767597 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-z.fits.fz CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-resid.jpg Stage coadds : Wall: 5.87 s, CPU: 2.36 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 231 MB, VmData: 256 MB, maxrss: 0.236508 MB Stage coadds finished: 2020-08-05T20:14:17.634388 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage coadds : 26.375 s worker CPU, 53.086 s worker Wall, Wall: 5.869 s, Cores in use: 4.90, Total efficiency (on 32 cores): 15.3 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-coadds.p Running stage wise_forced WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage wise_forced at 2020-08-05T20:14:17.649523 Cut to 1 unWISE tiles Reading WISE tile 3235m046 band 1 Reading WISE tile 3235m046 band 2 Reading WISE tile 3235m046 band 3 Reading WISE tile 3235m046 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.06 s, CPU: 0.06 s, VmPeak: 950 MB, VmSize: 809 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.211496 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 957 MB, VmSize: 812 MB, VmRSS: 151 MB, VmData: 177 MB, maxrss: 0.195912 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 954 MB, VmSize: 809 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.211172 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 956 MB, VmSize: 805 MB, VmRSS: 144 MB, VmData: 174 MB, maxrss: 0.206824 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.51 s, CPU: 0.68 s, VmPeak: 1055 MB, VmSize: 1055 MB, VmRSS: 232 MB, VmData: 256 MB, maxrss: 0.237212 MB Stage wise_forced finished: 2020-08-05T20:14:20.164583 WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Resources for stage wise_forced : 3.090 s worker CPU, 4.889 s worker Wall, Wall: 2.517 s, Cores in use: 1.50, Total efficiency (on 32 cores): 4.7 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-wise_forced.p Running stage writecat WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data] Running stage writecat at 2020-08-05T20:14:20.183210 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/tractor-i/cus/tractor-custom-323341m04922.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/tractor/cus/tractor-custom-323341m04922.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/tractor/cus/brick-custom-323341m04922.sha256sum Stage writecat : Wall: 1.31 s, CPU: 0.96 s, VmPeak: 1187 MB, VmSize: 1059 MB, VmRSS: 233 MB, VmData: 256 MB, maxrss: 0.370092 MB Stage writecat finished: 2020-08-05T20:14:21.496525 Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 1.316 s, Cores in use: 0.73, Total efficiency (on 32 cores): 2.3 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9delta/323/PGC1051678/PGC1051678-largegalaxy-writecat.p All done: 66.160 s worker CPU, 532.346 s worker Wall, Wall: 121.032 s, Cores in use: 0.69, Total efficiency (on 32 cores): 2.1 % Finished galaxy PGC1051678 in 2.162 minutes. Started working on galaxy PGC1051678 at Mon Aug 10 08:14:31 2020 Bands touching this brick, g r z python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.3415976283178 -4.922674735110007 --width 245 --height 245 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 runbrick.py starting at 2020-08-10T08:14:31.816620 legacypipe git version: DR9.5.11-watershed-6-g6a2880fc Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '323.3415976283178', '-4.922674735110007', '--width', '245', '--height', '245', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0'] python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 323.3415976283178 -4.922674735110007 --width 245 --height 245 --pixscale 0.262 --threads 8 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 Parsed RA,Dec 323.3415976283178 -4.922674735110007 Starting process 25094 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059228 MB Starting process 25095 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 25096 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 25097 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059232 MB Starting process 25098 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB Starting process 25099 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 25100 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059248 MB Starting process 25101 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059244 MB Runstage writecat Runstage wise_forced Runstage coadds Runstage fitblobs Runstage srcs Runstage fit_on_coadds Runstage halos Runstage outliers Runstage refs Runstage tims Running stage tims Running stage tims at 2020-08-10T08:14:32.322817 Keeping 11 CCDs: c4d_180905_013038_ooi_g_ls9-S16 g expnum 768987 exptime 66.0 propid 2014B-0404 seeing 1.20 MJD 58366.063 object DECaLS_52078_g decam/CP/V4.8.2a/CP20180904/c4d_180905_013038_ooi_g_ls9.fits.fz c4d_180906_005235_ooi_g_ls9-S19 g expnum 769481 exptime 57.0 propid 2014B-0404 seeing 1.12 MJD 58367.037 object DECaLS_59215_g decam/CP/V4.8.2a/CP20180905/c4d_180906_005235_ooi_g_ls9.fits.fz c4d_170722_042032_ooi_r_ls9-N5 r expnum 660878 exptime 40.0 propid 2014B-0404 seeing 0.86 MJD 57956.180 object DECaLS_11759_r decam/CP/V4.8.2a/CP20170721/c4d_170722_042032_ooi_r_ls9.fits.fz c4d_160813_024709_ooi_z_ls9-N5 z expnum 563504 exptime 82.0 propid 2014B-0404 seeing 0.85 MJD 57613.114 object DECaLS_11759_z decam/CP/V4.8.2a/CP20160812/c4d_160813_024709_ooi_z_ls9.fits.fz c4d_160803_043122_ooi_g_ls9-N5 g expnum 558558 exptime 57.0 propid 2014B-0404 seeing 1.07 MJD 57603.187 object DECaLS_11759_g decam/CP/V4.8.2a/CP20160802/c4d_160803_043122_ooi_g_ls9.fits.fz c4d_181005_235712_ooi_z_ls9-N10 z expnum 780919 exptime 98.0 propid 2014B-0404 seeing 1.12 MJD 58396.998 object DECaLS_43503_z decam/CP/V4.8.2a/CP20181005/c4d_181005_235712_ooi_z_ls9.fits.fz c4d_170915_010700_ooi_r_ls9-N10 r expnum 677683 exptime 175.0 propid 2014B-0404 seeing 1.85 MJD 58011.047 object DECaLS_43503_r decam/CP/V4.8.2a/CP20170914/c4d_170915_010700_ooi_r_ls9.fits.fz c4d_180910_021336_ooi_r_ls9-N10 r expnum 771431 exptime 40.0 propid 2014B-0404 seeing 1.36 MJD 58371.093 object DECaLS_43503_r decam/CP/V4.8.2a/CP20180909/c4d_180910_021336_ooi_r_ls9.fits.fz c4d_170917_003033_ooi_r_ls9-N11 r expnum 678261 exptime 98.0 propid 2014B-0404 seeing 1.12 MJD 58013.021 object DECaLS_27631_r decam/CP/V4.8.2a/CP20170916/c4d_170917_003033_ooi_r_ls9.fits.fz c4d_170726_041020_ooi_g_ls9-N11 g expnum 662261 exptime 110.0 propid 2014B-0404 seeing 1.11 MJD 57960.172 object DECaLS_27631_g decam/CP/V4.8.2a/CP20170725/c4d_170726_041020_ooi_g_ls9.fits.fz c4d_160814_032505_ooi_z_ls9-N11 z expnum 563870 exptime 80.0 propid 2014B-0404 seeing 1.07 MJD 57614.141 object DECaLS_27631_z decam/CP/V4.8.2a/CP20160813/c4d_160814_032505_ooi_z_ls9.fits.fz The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951dcf7-r-20170722T041859_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9534322-g-20180905T013038_oi[S16,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954922e-z-20160813T024449_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_N10-r-20180910T021336_oi[N10,noinherit][2046,4094][real]: DECaLS_4350 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_964e77a-g-20180906T005235_oi[S19,noinherit][2046,4094][real]: DECaLS_ Masking 0 additional saturated pixels in DECam expnum 769481 S19 CCD, decam/CP/V4.8.2a/CP20180905/c4d_180906_005235_ooi_g_ls9.fits.fz slice (slice(3207, 3452, None), slice(367, 612, None)) The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95b90ac-g-20160803T042933_oi[N5,noinherit][2046,4094][real]: DECaLS_1 The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9522538-r-20170915T010700_oi[N10,noinherit][2046,4094][real]: DECaLS_ The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530bc0-z-20181005T235712_oi[N10,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9523600-r-20170917T003033_oi[N11,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95203e2-g-20170726T040738_oi[N11,noinherit][2046,4094][real]: DECaLS_ Warning: splinesky does not have sky-template version/run/scale values The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95488c8-z-20160814T032253_oi[N11,noinherit][2046,4094][real]: DECaLS_ Stage tims : Wall: 84.68 s, CPU: 1.17 s, VmPeak: 1005 MB, VmSize: 1005 MB, VmRSS: 177 MB, VmData: 211 MB, maxrss: 0.181032 MB Stage tims finished: 2020-08-10T08:15:57.002929 Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95b90ac-g-20160803T042933_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9530bc0-z-20181005T235712_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9534322-g-20180905T013038_oi[S16,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_N10-r-20180910T021336_oi[N10,noinherit][2046,4094][real]: DECaLS_4350 [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_951dcf7-r-20170722T041859_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9523600-r-20170917T003033_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_964e77a-g-20180906T005235_oi[S19,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95488c8-z-20160814T032253_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_9522538-r-20170915T010700_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Resources for stage tims : 6.145 s worker CPU, 388.089 s worker Wall, Wall: 84.681 s, Cores in use: 0.09, Total efficiency (on 8 cores): 1.1 % Stage tims finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-tims.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-tims.p Running stage refs Warning: splinesky does not have sky-template version/run/scale values WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_954922e-z-20160813T024449_oi[N5,noinherit][2046,4094][real]: DECaLS_1 [astropy.io.fits.card] WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention: DECALS_DR9_95203e2-g-20170726T040738_oi[N11,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card] Running stage refs at 2020-08-10T08:15:57.146659 Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06707.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06706.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06835.fits Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06579.fits Matched 1 large galaxies to Gaia stars. Matched 0 large galaxies to Tycho-2 stars. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/reference-custom-323341m04922.fits Stage refs : Wall: 10.34 s, CPU: 2.91 s, VmPeak: 1044 MB, VmSize: 1038 MB, VmRSS: 211 MB, VmData: 244 MB, maxrss: 0.216184 MB Stage refs finished: 2020-08-10T08:16:07.484139 Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 10.339 s, Cores in use: 0.28, Total efficiency (on 8 cores): 3.5 % Stage refs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-refs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-refs.p Running stage outliers Running stage outliers at 2020-08-10T08:16:07.508632 Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/outlier-mask-custom-323341m04922.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/outliers-pre-custom-323341m04922.jpg Image c4d_160803_043122_ooi_g_ls9-N5 g : masking 450 positive outlier pixels and 0 negative outlier pixels Image c4d_180906_005235_ooi_g_ls9-S19 g : masking 101 positive outlier pixels and 0 negative outlier pixels Image c4d_170915_010700_ooi_r_ls9-N10 r : masking 3186 positive outlier pixels and 0 negative outlier pixels Image c4d_170722_042032_ooi_r_ls9-N5 r : masking 40 positive outlier pixels and 0 negative outlier pixels Image c4d_170917_003033_ooi_r_ls9-N11 r : masking 274 positive outlier pixels and 0 negative outlier pixels Image c4d_181005_235712_ooi_z_ls9-N10 z : masking 291 positive outlier pixels and 0 negative outlier pixels Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/outlier-mask-custom-323341m04922.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/outliers-post-custom-323341m04922.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/outliers-masked-pos-custom-323341m04922.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/outliers-masked-neg-custom-323341m04922.jpg Stage outliers : Wall: 1.14 s, CPU: 0.91 s, VmPeak: 1048 MB, VmSize: 1048 MB, VmRSS: 225 MB, VmData: 254 MB, maxrss: 0.230716 MB Stage outliers finished: 2020-08-10T08:16:08.653083 Resources for stage outliers : 0.895 s worker CPU, 0.895 s worker Wall, Wall: 1.145 s, Cores in use: 1.58, Total efficiency (on 8 cores): 19.7 % Stage outliers finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-outliers.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-outliers.p Running stage halos Running stage halos at 2020-08-10T08:16:08.683242 Stage halos : Wall: 1.94 s, CPU: 0.54 s, VmPeak: 1048 MB, VmSize: 1048 MB, VmRSS: 226 MB, VmData: 255 MB, maxrss: 0.231012 MB Stage halos finished: 2020-08-10T08:16:10.620881 Resources for stage halos : 0.369 s worker CPU, 3.775 s worker Wall, Wall: 1.939 s, Cores in use: 0.47, Total efficiency (on 8 cores): 5.9 % Stage halos finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-halos.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-halos.p Running stage fit_on_coadds Running stage fit_on_coadds at 2020-08-10T08:16:10.658757 PSF sigmas in coadd pixels: ['1.95', '1.82', '1.73', '1.80'] Using average PSF sigma 1.8275361084147765 Gaussian PSF norm 0.154357985308739 vs pixelized 0.12569213 average tim pixel scale / coadd scale: 1.001099038683411 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-g.fits.fz PSF sigmas in coadd pixels: ['1.39', '3.01', '2.21', '1.82'] Using average PSF sigma 2.1075695296135075 Gaussian PSF norm 0.1338483916236963 vs pixelized 0.09822333 average tim pixel scale / coadd scale: 1.0010999384583223 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-r.fits.fz PSF sigmas in coadd pixels: ['1.38', '1.82', '1.74'] Using average PSF sigma 1.6443128746417466 Gaussian PSF norm 0.171557856247607 vs pixelized 0.1417296 average tim pixel scale / coadd scale: 1.0010343132085586 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-copsf-z.fits.fz Stage fit_on_coadds : Wall: 0.21 s, CPU: 0.17 s, VmPeak: 1048 MB, VmSize: 1048 MB, VmRSS: 227 MB, VmData: 255 MB, maxrss: 0.2329 MB Stage fit_on_coadds finished: 2020-08-10T08:16:10.866177 Resources for stage fit_on_coadds : 0.231 s worker CPU, 0.231 s worker Wall, Wall: 0.208 s, Cores in use: 1.94, Total efficiency (on 8 cores): 24.2 % Stage fit_on_coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-fit_on_coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-fit_on_coadds.p Running stage srcs Running stage srcs at 2020-08-10T08:16:10.876306 SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])] Found 32 potential peaks Of 32 potential peaks: 6 in veto map, 3 cut by saddle test, 0 cut by aper test, 23 kept SED z : 23 new peaks Found 30 potential peaks Of 30 potential peaks: 26 in veto map, 3 cut by saddle test, 0 cut by aper test, 1 kept SED r : 1 new peaks Found 29 potential peaks Of 29 potential peaks: 24 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept SED g : 3 new peaks Found 35 potential peaks Of 35 potential peaks: 32 in veto map, 2 cut by saddle test, 0 cut by aper test, 1 kept SED Flat : 1 new peaks Found 38 potential peaks Of 38 potential peaks: 33 in veto map, 2 cut by saddle test, 0 cut by aper test, 3 kept SED Red : 3 new peaks Adding 31 new sources Median coadd sky for g : 5.9035415e-07 Median coadd sky for r : -0.00019770817 Median coadd sky for z : 0.00039704278 Stage srcs : Wall: 0.54 s, CPU: 0.50 s, VmPeak: 1048 MB, VmSize: 1048 MB, VmRSS: 228 MB, VmData: 255 MB, maxrss: 0.233424 MB Stage srcs finished: 2020-08-10T08:16:11.416278 Resources for stage srcs : 0.079 s worker CPU, 0.079 s worker Wall, Wall: 0.541 s, Cores in use: 1.08, Total efficiency (on 8 cores): 13.4 % Stage srcs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-srcs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-srcs.p Running stage fitblobs Running stage fitblobs at 2020-08-10T08:16:12.112955 Blob 1 of 14, id: 8, sources: 10, size: 144x161, npix 10240, brick X: 67,211, Y: 65,226, one pixel: 100 65 Blob 2 of 14, id: 12, sources: 9, size: 110x81, npix 4308, brick X: 3,113, Y: 164,245, one pixel: 66 164 Blob 3 of 14, id: 3, sources: 3, size: 52x45, npix 1616, brick X: 0,52, Y: 31,76, one pixel: 31 31 Blob 4 of 14, id: 1, sources: 3, size: 44x55, npix 1543, brick X: 201,245, Y: 7,62, one pixel: 238 7 Blob 5 of 14, id: 10, sources: 3, size: 19x46, npix 632, brick X: 0,19, Y: 134,180, one pixel: 6 134 Blob 6 of 14, id: 9, sources: 1, size: 29x29, npix 579, brick X: 216,245, Y: 114,143, one pixel: 230 114 Blob 7 of 14, id: 4, sources: 2, size: 36x26, npix 563, brick X: 93,129, Y: 33,59, one pixel: 116 33 Blob 8 of 14, id: 11, sources: 1, size: 25x26, npix 404, brick X: 91,116, Y: 154,180, one pixel: 104 154 Blob 9 of 14, id: 0, sources: 1, size: 23x25, npix 358, brick X: 113,136, Y: 7,32, one pixel: 124 7 Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (323.35082, -4.91942), pm (-12.2, -7.6), parallax 1.544 with NanoMaggies: g=22.4, r=21.1, z=19.1 Reset brightness to NanoMaggies: g=20.8, r=19.3, z=17.9 Blob 8 finished, total: Wall: 0.82 s, CPU: 0.81 s, VmPeak: 953 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 180 MB, maxrss: 0.144964 MB Blob 10 of 14, id: 7, sources: 1, size: 17x29, npix 354, brick X: 228,245, Y: 49,78, one pixel: 238 49 Blob 7 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 968 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.145484 MB Blob 11 of 14, id: 6, sources: 1, size: 23x22, npix 316, brick X: 184,207, Y: 49,71, one pixel: 193 49 Blob 5 finished, total: Wall: 1.25 s, CPU: 1.24 s, VmPeak: 958 MB, VmSize: 726 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.146968 MB Blob 12 of 14, id: 2, sources: 1, size: 19x19, npix 199, brick X: 172,191, Y: 10,29, one pixel: 181 10 Blob 6 finished, total: Wall: 1.33 s, CPU: 1.33 s, VmPeak: 950 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 180 MB, maxrss: 0.147544 MB Blob 13 of 14, id: 5, sources: 1, size: 19x18, npix 198, brick X: 132,151, Y: 35,53, one pixel: 140 35 Blob 11 finished, total: Wall: 0.33 s, CPU: 0.33 s, VmPeak: 958 MB, VmSize: 726 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.146968 MB Blob 14 of 14, id: 13, sources: 1, size: 17x17, npix 145, brick X: 227,244, Y: 223,240, one pixel: 235 223 Blob 9 finished, total: Wall: 0.85 s, CPU: 0.85 s, VmPeak: 953 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 180 MB, maxrss: 0.144964 MB Blob 10 finished, total: Wall: 0.83 s, CPU: 0.83 s, VmPeak: 968 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.145484 MB Blob 12 finished, total: Wall: 0.39 s, CPU: 0.39 s, VmPeak: 950 MB, VmSize: 725 MB, VmRSS: 144 MB, VmData: 180 MB, maxrss: 0.147672 MB Blob 13 finished, total: Wall: 0.27 s, CPU: 0.27 s, VmPeak: 958 MB, VmSize: 726 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.146968 MB Blob 14 finished, total: Wall: 0.24 s, CPU: 0.24 s, VmPeak: 953 MB, VmSize: 725 MB, VmRSS: 142 MB, VmData: 180 MB, maxrss: 0.144964 MB Blob 4 finished, total: Wall: 2.39 s, CPU: 2.39 s, VmPeak: 921 MB, VmSize: 725 MB, VmRSS: 143 MB, VmData: 180 MB, maxrss: 0.146424 MB Measuring 1 iterative sources Blob 3 finished, total: Wall: 3.52 s, CPU: 3.52 s, VmPeak: 960 MB, VmSize: 726 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.147592 MB Measuring 2 iterative sources Blob 2 finished, total: Wall: 7.12 s, CPU: 7.12 s, VmPeak: 968 MB, VmSize: 726 MB, VmRSS: 145 MB, VmData: 181 MB, maxrss: 0.14832 MB Known galaxy. Initial shape: EllipseWithPriors(0.25): log r_e=1.34415, ee1=-0.735936, ee2=-0.158802 Measuring 3 iterative sources Blob 1 finished, total: Wall: 9.59 s, CPU: 9.59 s, VmPeak: 969 MB, VmSize: 726 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149484 MB Old catalog: 45 New catalog: 40 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/blobs-custom-323341m04922.fits.gz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/metrics/cus/all-models-custom-323341m04922.fits Stage fitblobs : Wall: 10.00 s, CPU: 0.40 s, VmPeak: 1048 MB, VmSize: 1048 MB, VmRSS: 228 MB, VmData: 255 MB, maxrss: 0.233588 MB Stage fitblobs finished: 2020-08-10T08:16:22.109936 Resources for stage fitblobs : 29.921 s worker CPU, 29.995 s worker Wall, Wall: 9.998 s, Cores in use: 3.03, Total efficiency (on 8 cores): 37.9 % Stage fitblobs finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-fitblobs.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-fitblobs.p Running stage coadds Running stage coadds at 2020-08-10T08:16:22.123404 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-ccds.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-g.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-r.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-nexp-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-galdepth-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-psfsize-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-chi2-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel-z.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-depth.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-blobmodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-resid.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-resid.jpg Stage coadds : Wall: 1.54 s, CPU: 0.65 s, VmPeak: 1048 MB, VmSize: 1048 MB, VmRSS: 229 MB, VmData: 255 MB, maxrss: 0.234108 MB Stage coadds finished: 2020-08-10T08:16:23.660524 Resources for stage coadds : 3.987 s worker CPU, 4.573 s worker Wall, Wall: 1.538 s, Cores in use: 3.02, Total efficiency (on 8 cores): 37.7 % Stage coadds finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-coadds.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-coadds.p Running stage wise_forced Running stage wise_forced at 2020-08-10T08:16:23.685818 Cut to 1 unWISE tiles Reading WISE tile 3235m046 band 1 Reading WISE tile 3235m046 band 2 Reading WISE tile 3235m046 band 3 Reading WISE tile 3235m046 band 4 Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w4-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.07 s, CPU: 0.07 s, VmPeak: 969 MB, VmSize: 807 MB, VmRSS: 150 MB, VmData: 178 MB, maxrss: 0.195296 MB Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w3-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 958 MB, VmSize: 803 MB, VmRSS: 148 MB, VmData: 174 MB, maxrss: 0.210184 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w2-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 968 MB, VmSize: 803 MB, VmRSS: 146 MB, VmData: 174 MB, maxrss: 0.209708 MB Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-img-m.fits Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-invvar-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-n-m.fits.gz Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/323/3235m046/unwise-3235m046-w1-n-u.fits.gz Reading /usr/local/lib/python/wise/wise-psf-avg.fits unWISE forced photometry took Wall: 0.05 s, CPU: 0.05 s, VmPeak: 960 MB, VmSize: 803 MB, VmRSS: 145 MB, VmData: 174 MB, maxrss: 0.2091 MB Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W1.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W2.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W3.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-image-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-model-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-invvar-W4.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-wise.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/tmp-legacysurvey-custom-323341m04922-wisemodel.jpg Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-wisemodel.jpg Setting WISE apphot Setting WISE apphot Setting WISE apphot Setting WISE apphot Stage wise_forced : Wall: 2.72 s, CPU: 0.51 s, VmPeak: 1048 MB, VmSize: 1048 MB, VmRSS: 229 MB, VmData: 255 MB, maxrss: 0.234812 MB Stage wise_forced finished: 2020-08-10T08:16:26.408219 Resources for stage wise_forced : 3.164 s worker CPU, 5.319 s worker Wall, Wall: 2.723 s, Cores in use: 1.35, Total efficiency (on 8 cores): 16.9 % Stage wise_forced finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-wise_forced.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-wise_forced.p Running stage writecat Running stage writecat at 2020-08-10T08:16:26.439139 Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/coadd/cus/custom-323341m04922/legacysurvey-custom-323341m04922-maskbits.fits.fz Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/tractor-i/cus/tractor-custom-323341m04922.fits Reading SFD maps... END Rejected (1), keyrecords following the END keyrecord were ignored. END Rejected (1), keyrecords following the END keyrecord were ignored. Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/tractor/cus/tractor-custom-323341m04922.fits Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/tractor/cus/brick-custom-323341m04922.sha256sum Stage writecat : Wall: 1.00 s, CPU: 0.63 s, VmPeak: 1181 MB, VmSize: 1053 MB, VmRSS: 231 MB, VmData: 255 MB, maxrss: 0.367668 MB Stage writecat finished: 2020-08-10T08:16:27.435008 Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 0.997 s, Cores in use: 0.64, Total efficiency (on 8 cores): 7.9 % Stage writecat finished Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-writecat.p Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/323/PGC1051678/PGC1051678-largegalaxy-writecat.p All done: 44.793 s worker CPU, 432.958 s worker Wall, Wall: 115.135 s, Cores in use: 0.46, Total efficiency (on 8 cores): 5.8 % Finished galaxy PGC1051678 in 1.947 minutes.