Started working on galaxy PGC3441840 at Sun Aug  2 08:41:37 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 307.6599855 -0.26497950000000003 --width 403 --height 403 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-02T08:41:38.238727
legacypipe git version: DR9.5.10-2-ge83842ef
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '307.6599855', '-0.26497950000000003', '--width', '403', '--height', '403', '--pixscale', '0.262', '--threads', '16', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-%%(stage)s.p', '--force-all', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 307.6599855 -0.26497950000000003 --width 403 --height 403 --pixscale 0.262 --threads 16 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-%%(stage)s.p --force-all --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 307.6599855 -0.26497950000000003
Starting process 43845 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058892 MB
Starting process 43846 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058964 MB
Starting process 43847 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058992 MB
Starting process 43848 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058896 MB
Starting process 43849 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058896 MB
Starting process 43850 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058896 MB
Starting process 43851 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058976 MB
Starting process 43852 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058976 MB
Starting process 43853 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058956 MB
Starting process 43854 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058956 MB
Starting process 43855 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058912 MB
Starting process 43856 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB
Starting process 43857 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB
Starting process 43858 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB
Starting process 43859 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.058916 MB
Starting process 43860 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 635 MB, VmSize: 635 MB, VmRSS: 58 MB, VmData: 128 MB, maxrss: 0.05892 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-02T08:41:39.582047
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Keeping 7 CCDs:
  c4d_170822_011754_ooi_g_ls9-N9 g expnum 670247 exptime 189.0 propid 2014B-0404 seeing 1.22 MJD 57987.054 object DECaLS_27535_g 
    decam/CP/V4.8.2a/CP20170821/c4d_170822_011754_ooi_g_ls9.fits.fz
  c4d_170822_011608_ooi_r_ls9-N9 r expnum 670246 exptime 77.0 propid 2014B-0404 seeing 1.17 MJD 57987.053 object DECaLS_27535_r 
    decam/CP/V4.8.2a/CP20170821/c4d_170822_011608_ooi_r_ls9.fits.fz
  c4d_180812_023009_ooi_g_ls9-N14 g expnum 764551 exptime 88.0 propid 2014B-0404 seeing 1.04 MJD 58342.104 object DECaLS_43407_g 
    decam/CP/V4.8.2a/CP20180811/c4d_180812_023009_ooi_g_ls9.fits.fz
  c4d_160813_022210_ooi_z_ls9-N14 z expnum 563496 exptime 180.0 propid 2014B-0404 seeing 1.74 MJD 57613.096 object DECaLS_43407_z 
    decam/CP/V4.8.2a/CP20160812/c4d_160813_022210_ooi_z_ls9.fits.fz
  c4d_180808_030207_ooi_r_ls9-N14 r expnum 762846 exptime 140.0 propid 2014B-0404 seeing 1.95 MJD 58338.126 object DECaLS_43407_r 
    decam/CP/V4.8.2a/CP20180807/c4d_180808_030207_ooi_r_ls9.fits.fz
  c4d_160813_013508_ooi_z_ls9-N10 z expnum 563475 exptime 106.0 propid 2014B-0404 seeing 0.87 MJD 57613.064 object DECaLS_11663_z 
    decam/CP/V4.8.2a/CP20160812/c4d_160813_013508_ooi_z_ls9.fits.fz
  c4d_160724_065654_ooi_z_ls9-N9 z expnum 554225 exptime 118.0 propid 2016A-0190 seeing 1.23 MJD 57593.288 object DECaLS_27535_z 
    decam/CP/V4.8.2a/CP20160723/c4d_160724_065654_ooi_z_ls9.fits.fz
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T013233_oi[N10,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9521d7c-r-20170822T011608_oi[N9,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954b419-z-20160724T065406_oi[N9,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95220ee-g-20170822T011754_oi[N9,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T021821_oi[N14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9560feb-r-20180808T030207_oi[N14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9532e22-g-20180812T023009_oi[N14,noinherit][2046,4094][real]: DECaLS_
Clipping 3 pixels with anomalously large oow values: max 0.0485352 vs median 0.0364959
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Warning: splinesky does not have sky-template version/run/scale values
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Stage tims : Wall: 11.83 s, CPU: 1.44 s, VmPeak: 1004 MB, VmSize: 1004 MB, VmRSS: 180 MB, VmData: 212 MB, maxrss: 0.18406 MB
Stage tims finished: 2020-08-02T08:41:51.409796
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9521d7c-r-20170822T011608_oi[N9,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9560feb-r-20180808T030207_oi[N14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95220ee-g-20170822T011754_oi[N9,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9532e22-g-20180812T023009_oi[N14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954b419-z-20160724T065406_oi[N9,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T021821_oi[N14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T013233_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage tims : 4.181 s worker CPU, 52.322 s worker Wall, Wall: 11.829 s, Cores in use: 0.48, Total efficiency (on 16 cores): 3.0 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-tims.p
Running stage refs
Running stage refs at 2020-08-02T08:41:51.660637
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06189.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06317.fits
Matched 3 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/reference-custom-307659m00264.fits
Stage refs : Wall: 10.00 s, CPU: 9.60 s, VmPeak: 1140 MB, VmSize: 1134 MB, VmRSS: 312 MB, VmData: 342 MB, maxrss: 0.319356 MB
Stage refs finished: 2020-08-02T08:42:01.660699
Resources for stage refs : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 10.002 s, Cores in use: 0.96, Total efficiency (on 16 cores): 6.0 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-02T08:42:01.807910
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outlier-mask-custom-307659m00264.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-pre-custom-307659m00264.jpg
Image c4d_170822_011754_ooi_g_ls9-N9 g : masking 385 positive outlier pixels and 0 negative outlier pixels
Image c4d_160813_022210_ooi_z_ls9-N14 z : masking 590 positive outlier pixels and 0 negative outlier pixels
Image c4d_160813_013508_ooi_z_ls9-N10 z : masking 1135 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outlier-mask-custom-307659m00264.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-post-custom-307659m00264.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-masked-pos-custom-307659m00264.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-masked-neg-custom-307659m00264.jpg
Stage outliers : Wall: 2.33 s, CPU: 1.30 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 335 MB, VmData: 362 MB, maxrss: 0.343176 MB
Stage outliers finished: 2020-08-02T08:42:04.133457
Resources for stage outliers : 1.080 s worker CPU, 1.280 s worker Wall, Wall: 2.328 s, Cores in use: 1.02, Total efficiency (on 16 cores): 6.4 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-02T08:42:04.379017
Stage halos : Wall: 1.21 s, CPU: 0.92 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 335 MB, VmData: 363 MB, maxrss: 0.343376 MB
Stage halos finished: 2020-08-02T08:42:05.590336
Resources for stage halos : 0.587 s worker CPU, 1.032 s worker Wall, Wall: 1.213 s, Cores in use: 1.24, Total efficiency (on 16 cores): 7.8 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-02T08:42:05.835357
PSF sigmas in coadd pixels: ['1.98', '1.69']
Using average PSF sigma 1.8355417243605892
Gaussian PSF norm 0.15368476130508332 vs pixelized 0.13155568
average tim pixel scale / coadd scale: 1.0010929152152663
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.90', '3.17']
Using average PSF sigma 2.539443255773545
Gaussian PSF norm 0.11108529049921563 vs pixelized 0.08840066
average tim pixel scale / coadd scale: 1.0011148097381024
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.82', '1.42', '1.99']
Using average PSF sigma 2.0781631859058436
Gaussian PSF norm 0.13574236791752078 vs pixelized 0.120690934
average tim pixel scale / coadd scale: 1.0010729702047378
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.25 s, CPU: 0.20 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 337 MB, VmData: 363 MB, maxrss: 0.344912 MB
Stage fit_on_coadds finished: 2020-08-02T08:42:06.082778
Resources for stage fit_on_coadds : 0.262 s worker CPU, 0.262 s worker Wall, Wall: 0.249 s, Cores in use: 1.85, Total efficiency (on 16 cores): 11.6 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-02T08:42:06.168616
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 123 potential peaks
Of 123 potential peaks: 35 in veto map, 16 cut by saddle test, 0 cut by aper test, 72 kept
SED z : 72 new peaks
Found 78 potential peaks
Of 78 potential peaks: 71 in veto map, 7 cut by saddle test, 0 cut by aper test, 0 kept
SED r : 0 new peaks
Found 84 potential peaks
Of 84 potential peaks: 67 in veto map, 11 cut by saddle test, 0 cut by aper test, 6 kept
SED g : 6 new peaks
Found 111 potential peaks
Of 111 potential peaks: 100 in veto map, 9 cut by saddle test, 0 cut by aper test, 2 kept
SED Flat : 2 new peaks
Found 130 potential peaks
Of 130 potential peaks: 113 in veto map, 12 cut by saddle test, 0 cut by aper test, 5 kept
SED Red : 5 new peaks
Adding 85 new sources
Median coadd sky for g : 9.068323e-05
Median coadd sky for r : 0.00013714924
Median coadd sky for z : 0.00020202858
Stage srcs : Wall: 2.37 s, CPU: 2.28 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 337 MB, VmData: 363 MB, maxrss: 0.34542 MB
Stage srcs finished: 2020-08-02T08:42:08.537176
Resources for stage srcs : 0.200 s worker CPU, 0.200 s worker Wall, Wall: 2.370 s, Cores in use: 1.05, Total efficiency (on 16 cores): 6.5 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-02T08:42:09.104077
Blob 1 of 28, id: 3, sources: 52, size: 307x403, npix 51182, brick X: 80,387, Y: 0,403, one pixel: 229 0
Blob 2 of 28, id: 7, sources: 7, size: 93x99, npix 4848, brick X: 63,156, Y: 49,148, one pixel: 90 49
Blob 3 of 28, id: 22, sources: 10, size: 79x100, npix 4189, brick X: 271,350, Y: 301,401, one pixel: 317 301
Blob 4 of 28, id: 1, sources: 6, size: 108x49, npix 3499, brick X: 88,196, Y: 0,49, one pixel: 93 0
Blob 5 of 28, id: 14, sources: 3, size: 65x33, npix 1291, brick X: 21,86, Y: 148,181, one pixel: 52 148
Blob 6 of 28, id: 0, sources: 2, size: 46x37, npix 1247, brick X: 6,52, Y: 0,37, one pixel: 10 0
Blob 7 of 28, id: 17, sources: 1, size: 41x39, npix 1138, brick X: 40,81, Y: 194,233, one pixel: 56 194
Blob 8 of 28, id: 4, sources: 1, size: 44x35, npix 1128, brick X: 296,340, Y: 0,35, one pixel: 306 0
Blob 9 of 28, id: 24, sources: 2, size: 39x38, npix 854, brick X: 345,384, Y: 310,348, one pixel: 369 310
Blob 10 of 28, id: 8, sources: 3, size: 37x46, npix 798, brick X: 278,315, Y: 54,100, one pixel: 287 54
Blob 11 of 28, id: 6, sources: 1, size: 37x31, npix 778, brick X: 4,41, Y: 45,76, one pixel: 18 45
Blob 12 of 28, id: 18, sources: 1, size: 30x34, npix 770, brick X: 373,403, Y: 222,256, one pixel: 386 222
Blob 13 of 28, id: 9, sources: 1, size: 32x32, npix 690, brick X: 319,351, Y: 72,104, one pixel: 332 72
Blob 14 of 28, id: 11, sources: 2, size: 35x34, npix 571, brick X: 4,39, Y: 108,142, one pixel: 14 108
Blob 15 of 28, id: 13, sources: 3, size: 20x36, npix 525, brick X: 0,20, Y: 147,183, one pixel: 5 147
Blob 16 of 28, id: 16, sources: 1, size: 28x27, npix 502, brick X: 89,117, Y: 193,220, one pixel: 100 193
Blob 17 of 28, id: 5, sources: 1, size: 18x39, npix 478, brick X: 385,403, Y: 26,65, one pixel: 395 26
Blob 14 finished, total: Wall: 0.55 s, CPU: 0.53 s, VmPeak: 689 MB, VmSize: 689 MB, VmRSS: 115 MB, VmData: 167 MB, maxrss: 0.117764 MB
Blob 18 of 28, id: 21, sources: 2, size: 18x40, npix 464, brick X: 0,18, Y: 262,302, one pixel: 8 262
Blob 16 finished, total: Wall: 0.95 s, CPU: 0.95 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 97 MB, VmData: 142 MB, maxrss: 0.099408 MB
Blob 19 of 28, id: 15, sources: 1, size: 20x29, npix 408, brick X: 383,403, Y: 151,180, one pixel: 396 151
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (307.69019, -0.26706), pm (4.4, -9.9), parallax 1.122 with NanoMaggies: g=13.2, r=12.9, z=12.9
Reset brightness to NanoMaggies: g=13.2, r=12.9, z=12.9
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (307.67472, -0.26760), pm (0.9, -7.3), parallax 0.431 with NanoMaggies: g=20.6, r=19.8, z=18.7
Reset brightness to NanoMaggies: g=20.8, r=19.5, z=18.8
Blob 15 finished, total: Wall: 1.09 s, CPU: 1.08 s, VmPeak: 690 MB, VmSize: 690 MB, VmRSS: 116 MB, VmData: 167 MB, maxrss: 0.11852 MB
Blob 20 of 28, id: 19, sources: 1, size: 26x24, npix 379, brick X: 75,101, Y: 225,249, one pixel: 87 225
Blob 10 finished, total: Wall: 1.36 s, CPU: 1.35 s, VmPeak: 953 MB, VmSize: 731 MB, VmRSS: 152 MB, VmData: 187 MB, maxrss: 0.156008 MB
Blob 21 of 28, id: 25, sources: 1, size: 25x24, npix 377, brick X: 95,120, Y: 343,367, one pixel: 105 343
Measuring 1 iterative sources
Blob 13 finished, total: Wall: 1.46 s, CPU: 1.46 s, VmPeak: 951 MB, VmSize: 731 MB, VmRSS: 152 MB, VmData: 186 MB, maxrss: 0.156092 MB
Blob 22 of 28, id: 12, sources: 1, size: 19x24, npix 305, brick X: 384,403, Y: 123,147, one pixel: 395 123
Measuring 1 iterative sources
Blob 18 finished, total: Wall: 1.04 s, CPU: 1.04 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 97 MB, VmData: 142 MB, maxrss: 0.09962 MB
Blob 23 of 28, id: 10, sources: 1, size: 21x21, npix 268, brick X: 381,402, Y: 87,108, one pixel: 391 87
Blob 20 finished, total: Wall: 0.76 s, CPU: 0.76 s, VmPeak: 953 MB, VmSize: 732 MB, VmRSS: 153 MB, VmData: 187 MB, maxrss: 0.156436 MB
Blob 24 of 28, id: 27, sources: 1, size: 18x23, npix 265, brick X: 0,18, Y: 363,386, one pixel: 5 363
Blob 7 finished, total: Wall: 2.21 s, CPU: 2.18 s, VmPeak: 952 MB, VmSize: 732 MB, VmRSS: 153 MB, VmData: 187 MB, maxrss: 0.156584 MB
Blob 25 of 28, id: 23, sources: 1, size: 19x20, npix 227, brick X: 36,55, Y: 304,324, one pixel: 44 304
Blob 21 finished, total: Wall: 0.77 s, CPU: 0.77 s, VmPeak: 951 MB, VmSize: 731 MB, VmRSS: 153 MB, VmData: 186 MB, maxrss: 0.156828 MB
Blob 26 of 28, id: 26, sources: 1, size: 20x19, npix 218, brick X: 23,43, Y: 349,368, one pixel: 31 349
Blob 22 finished, total: Wall: 0.32 s, CPU: 0.32 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 97 MB, VmData: 142 MB, maxrss: 0.09962 MB
Blob 27 of 28, id: 2, sources: 1, size: 20x15, npix 190, brick X: 197,217, Y: 0,15, one pixel: 201 0
Blob 9 finished, total: Wall: 2.41 s, CPU: 2.40 s, VmPeak: 951 MB, VmSize: 725 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149236 MB
Blob 28 of 28, id: 20, sources: 1, size: 18x18, npix 180, brick X: 316,334, Y: 249,267, one pixel: 324 249
Blob 8 finished, total: Wall: 2.42 s, CPU: 2.41 s, VmPeak: 661 MB, VmSize: 661 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094444 MB
Blob 23 finished, total: Wall: 0.30 s, CPU: 0.30 s, VmPeak: 953 MB, VmSize: 732 MB, VmRSS: 153 MB, VmData: 187 MB, maxrss: 0.156436 MB
Blob 17 finished, total: Wall: 1.87 s, CPU: 1.87 s, VmPeak: 689 MB, VmSize: 689 MB, VmRSS: 115 MB, VmData: 167 MB, maxrss: 0.118104 MB
Blob 11 finished, total: Wall: 2.53 s, CPU: 2.53 s, VmPeak: 950 MB, VmSize: 731 MB, VmRSS: 152 MB, VmData: 186 MB, maxrss: 0.155904 MB
Blob 24 finished, total: Wall: 0.37 s, CPU: 0.37 s, VmPeak: 952 MB, VmSize: 732 MB, VmRSS: 153 MB, VmData: 187 MB, maxrss: 0.156584 MB
Blob 25 finished, total: Wall: 0.41 s, CPU: 0.41 s, VmPeak: 951 MB, VmSize: 731 MB, VmRSS: 153 MB, VmData: 186 MB, maxrss: 0.156828 MB
Blob 28 finished, total: Wall: 0.54 s, CPU: 0.54 s, VmPeak: 661 MB, VmSize: 661 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094592 MB
Blob 26 finished, total: Wall: 0.70 s, CPU: 0.70 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 97 MB, VmData: 142 MB, maxrss: 0.099656 MB
Blob 19 finished, total: Wall: 2.00 s, CPU: 2.00 s, VmPeak: 690 MB, VmSize: 690 MB, VmRSS: 117 MB, VmData: 168 MB, maxrss: 0.119424 MB
Blob 6 finished, total: Wall: 3.18 s, CPU: 3.17 s, VmPeak: 664 MB, VmSize: 664 MB, VmRSS: 96 MB, VmData: 141 MB, maxrss: 0.09794 MB
Blob 27 finished, total: Wall: 0.75 s, CPU: 0.75 s, VmPeak: 951 MB, VmSize: 725 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149968 MB
Measuring 2 iterative sources
Measuring 1 iterative sources
Blob 5 finished, total: Wall: 3.59 s, CPU: 3.58 s, VmPeak: 949 MB, VmSize: 732 MB, VmRSS: 154 MB, VmData: 187 MB, maxrss: 0.157312 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (307.66507, -0.28294), pm (-1.6, -8.0), parallax 0.606 with NanoMaggies: g=18.8, r=18.2, z=18
Reset brightness to NanoMaggies: g=18.8, r=18.2, z=18
Blob 4 finished, total: Wall: 3.83 s, CPU: 3.82 s, VmPeak: 954 MB, VmSize: 726 MB, VmRSS: 146 MB, VmData: 181 MB, maxrss: 0.149604 MB
Blob 12 finished, total: Wall: 4.19 s, CPU: 4.19 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 102 MB, VmData: 146 MB, maxrss: 0.104152 MB
Measuring 1 iterative sources
Blob 3 finished, total: Wall: 8.23 s, CPU: 8.22 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 102 MB, VmData: 147 MB, maxrss: 0.10492 MB
Measuring 1 iterative sources
Blob 2 finished, total: Wall: 8.92 s, CPU: 8.92 s, VmPeak: 668 MB, VmSize: 665 MB, VmRSS: 98 MB, VmData: 143 MB, maxrss: 0.10382 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=2.09661, ee1=0.249376, ee2=0.506751
Measuring 5 iterative sources
Blob 1 finished, total: Wall: 58.17 s, CPU: 58.16 s, VmPeak: 750 MB, VmSize: 738 MB, VmRSS: 180 MB, VmData: 215 MB, maxrss: 0.193436 MB
Old catalog: 116
New catalog: 110
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/blobs-custom-307659m00264.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/all-models-custom-307659m00264.fits
Stage fitblobs : Wall: 59.84 s, CPU: 1.47 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 340 MB, VmData: 365 MB, maxrss: 0.348084 MB
Stage fitblobs finished: 2020-08-02T08:43:08.947238
Resources for stage fitblobs : 115.200 s worker CPU, 117.814 s worker Wall, Wall: 59.845 s, Cores in use: 1.95, Total efficiency (on 16 cores): 12.2 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-02T08:43:09.033634
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel-z.fits.fz
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock'
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-resid.jpg
Stage coadds : Wall: 6.51 s, CPU: 3.27 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 340 MB, VmData: 365 MB, maxrss: 0.348536 MB
Stage coadds finished: 2020-08-02T08:43:15.547177
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Resources for stage coadds : 15.142 s worker CPU, 22.774 s worker Wall, Wall: 6.515 s, Cores in use: 2.83, Total efficiency (on 16 cores): 17.7 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-coadds.p
Running stage wise_forced
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/homedir/.astropy/cache/download/py3/lock' [astropy.utils.data]
Running stage wise_forced at 2020-08-02T08:43:15.610397
Cut to 1 unWISE tiles
Reading WISE tile 3070p000 band 1
Reading WISE tile 3070p000 band 2
Reading WISE tile 3070p000 band 3
Reading WISE tile 3070p000 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.19 s, CPU: 0.48 s, VmPeak: 1008 MB, VmSize: 1008 MB, VmRSS: 155 MB, VmData: 188 MB, maxrss: 0.21816 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.44 s, CPU: 1.46 s, VmPeak: 1075 MB, VmSize: 1011 MB, VmRSS: 154 MB, VmData: 190 MB, maxrss: 0.197212 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.20 s, CPU: 0.52 s, VmPeak: 1008 MB, VmSize: 1008 MB, VmRSS: 151 MB, VmData: 187 MB, maxrss: 0.212768 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.20 s, CPU: 0.50 s, VmPeak: 1068 MB, VmSize: 1004 MB, VmRSS: 148 MB, VmData: 186 MB, maxrss: 0.2101 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 4.55 s, CPU: 2.62 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 341 MB, VmData: 365 MB, maxrss: 0.349048 MB
Stage wise_forced finished: 2020-08-02T08:43:20.156714
Resources for stage wise_forced : 6.157 s worker CPU, 5.911 s worker Wall, Wall: 4.548 s, Cores in use: 1.93, Total efficiency (on 16 cores): 12.1 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-02T08:43:20.303256
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/tractor-i/cus/tractor-custom-307659m00264.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/tractor/cus/tractor-custom-307659m00264.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/tractor/cus/brick-custom-307659m00264.sha256sum
Stage writecat : Wall: 3.23 s, CPU: 2.66 s, VmPeak: 1276 MB, VmSize: 1148 MB, VmRSS: 342 MB, VmData: 365 MB, maxrss: 0.481796 MB
Stage writecat finished: 2020-08-02T08:43:23.533928
Resources for stage writecat : 0.000 s worker CPU, 0.000 s worker Wall, Wall: 3.232 s, Cores in use: 0.82, Total efficiency (on 16 cores): 5.1 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-writecat.p
All done: 142.814 s worker CPU, 201.601 s worker Wall, Wall: 104.092 s, Cores in use: 1.62, Total efficiency (on 16 cores): 10.1 %
Finished galaxy PGC3441840 in 1.788 minutes.
Started working on galaxy PGC3441840 at Thu Aug  6 07:52:55 2020
Bands touching this brick, g r z
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 307.6599855 -0.26497950000000003 --width 403 --height 403 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0 

runbrick.py starting at 2020-08-06T07:52:57.758304
legacypipe git version: DR9.5.10-12-gf22e529f
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '307.6599855', '-0.26497950000000003', '--width', '403', '--height', '403', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840', '--survey-dir', '/global/cfs/cdirs/cosmo/work/legacysurvey/dr9', '--run', 'south', '--skip-calibs', '--checkpoint', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-checkpoint.p', '--pickle', '/global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-%%(stage)s.p', '--fit-on-coadds', '--saddle-fraction', '0.2', '--saddle-min', '4.0']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 307.6599855 -0.26497950000000003 --width 403 --height 403 --pixscale 0.262 --threads 32 --outdir /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840 --survey-dir /global/cfs/cdirs/cosmo/work/legacysurvey/dr9 --run south --skip-calibs --checkpoint /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-checkpoint.p --pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-%%(stage)s.p --fit-on-coadds --saddle-fraction 0.2 --saddle-min 4.0

Parsed RA,Dec 307.6599855 -0.26497950000000003
Starting process 35747 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059236 MB
Starting process 35748 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 35749 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 35750 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05924 MB
Starting process 35751 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059536 MB
Starting process 35752 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB
Starting process 35753 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059548 MB
Starting process 35754 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 35755 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059252 MB
Starting process 35756 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05926 MB
Starting process 35757 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 35758 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 35759 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 35760 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 35761 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 35762 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059264 MB
Starting process 35763 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059268 MB
Starting process 35764 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059272 MB
Starting process 35765 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 35766 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 35767 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.05928 MB
Starting process 35768 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059284 MB
Starting process 35769 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 35770 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 35771 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 35772 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 35773 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059296 MB
Starting process 35774 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.0593 MB
Starting process 35775 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 35776 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 35777 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059304 MB
Starting process 35778 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 642 MB, VmSize: 642 MB, VmRSS: 58 MB, VmData: 129 MB, maxrss: 0.059308 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage fit_on_coadds
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-08-06T07:52:58.849930
Keeping 7 CCDs:
  c4d_170822_011754_ooi_g_ls9-N9 g expnum 670247 exptime 189.0 propid 2014B-0404 seeing 1.22 MJD 57987.054 object DECaLS_27535_g 
    decam/CP/V4.8.2a/CP20170821/c4d_170822_011754_ooi_g_ls9.fits.fz
  c4d_170822_011608_ooi_r_ls9-N9 r expnum 670246 exptime 77.0 propid 2014B-0404 seeing 1.17 MJD 57987.053 object DECaLS_27535_r 
    decam/CP/V4.8.2a/CP20170821/c4d_170822_011608_ooi_r_ls9.fits.fz
  c4d_180812_023009_ooi_g_ls9-N14 g expnum 764551 exptime 88.0 propid 2014B-0404 seeing 1.04 MJD 58342.104 object DECaLS_43407_g 
    decam/CP/V4.8.2a/CP20180811/c4d_180812_023009_ooi_g_ls9.fits.fz
  c4d_160813_022210_ooi_z_ls9-N14 z expnum 563496 exptime 180.0 propid 2014B-0404 seeing 1.74 MJD 57613.096 object DECaLS_43407_z 
    decam/CP/V4.8.2a/CP20160812/c4d_160813_022210_ooi_z_ls9.fits.fz
  c4d_180808_030207_ooi_r_ls9-N14 r expnum 762846 exptime 140.0 propid 2014B-0404 seeing 1.95 MJD 58338.126 object DECaLS_43407_r 
    decam/CP/V4.8.2a/CP20180807/c4d_180808_030207_ooi_r_ls9.fits.fz
  c4d_160813_013508_ooi_z_ls9-N10 z expnum 563475 exptime 106.0 propid 2014B-0404 seeing 0.87 MJD 57613.064 object DECaLS_11663_z 
    decam/CP/V4.8.2a/CP20160812/c4d_160813_013508_ooi_z_ls9.fits.fz
  c4d_160724_065654_ooi_z_ls9-N9 z expnum 554225 exptime 118.0 propid 2016A-0190 seeing 1.23 MJD 57593.288 object DECaLS_27535_z 
    decam/CP/V4.8.2a/CP20160723/c4d_160724_065654_ooi_z_ls9.fits.fz
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9521d7c-r-20170822T011608_oi[N9,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T013233_oi[N10,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954b419-z-20160724T065406_oi[N9,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9560feb-r-20180808T030207_oi[N14,noinherit][2046,4094][real]: DECaLS_
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95220ee-g-20170822T011754_oi[N9,noinherit][2046,4094][real]: DECaLS_2
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9532e22-g-20180812T023009_oi[N14,noinherit][2046,4094][real]: DECaLS_
Clipping 3 pixels with anomalously large oow values: max 0.0485352 vs median 0.0364959
The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T021821_oi[N14,noinherit][2046,4094][real]: DECaLS_
Stage tims : Wall: 6.45 s, CPU: 0.78 s, VmPeak: 1004 MB, VmSize: 1004 MB, VmRSS: 180 MB, VmData: 214 MB, maxrss: 0.184372 MB
Stage tims finished: 2020-08-06T07:53:05.304451
Warning: splinesky does not have sky-template version/run/scale values
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9521d7c-r-20170822T011608_oi[N9,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
Warning: splinesky does not have sky-template version/run/scale values
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T021821_oi[N14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Warning: splinesky does not have sky-template version/run/scale values
Warning: splinesky does not have sky-template version/run/scale values
Warning: splinesky does not have sky-template version/run/scale values
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954922e-z-20160813T013233_oi[N10,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_95220ee-g-20170822T011754_oi[N9,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9560feb-r-20180808T030207_oi[N14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Resources for stage tims : 3.872 s worker CPU, 35.318 s worker Wall, Wall: 6.457 s, Cores in use: 0.72, Total efficiency (on 32 cores): 2.3 %
Stage tims finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-tims.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-tims.p
Running stage refs
Warning: splinesky does not have sky-template version/run/scale values
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_954b419-z-20160724T065406_oi[N9,noinherit][2046,4094][real]: DECaLS_2 [astropy.io.fits.card]
Warning: splinesky does not have sky-template version/run/scale values
WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:
DECALS_DR9_9532e22-g-20180812T023009_oi[N14,noinherit][2046,4094][real]: DECaLS_ [astropy.io.fits.card]
Running stage refs at 2020-08-06T07:53:05.335680
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06189.fits
Reading /global/cfs/cdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom-2/chunk-06317.fits
Matched 3 large galaxies to Gaia stars.
Matched 0 large galaxies to Tycho-2 stars.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/reference-custom-307659m00264.fits
Stage refs : Wall: 9.51 s, CPU: 9.18 s, VmPeak: 1140 MB, VmSize: 1134 MB, VmRSS: 312 MB, VmData: 343 MB, maxrss: 0.319472 MB
Stage refs finished: 2020-08-06T07:53:14.849421
Resources for stage refs : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 9.516 s, Cores in use: 0.96, Total efficiency (on 32 cores): 3.0 %
Stage refs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-refs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-refs.p
Running stage outliers
Running stage outliers at 2020-08-06T07:53:14.983418
Failed to apply outlier mask: No such file: /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outlier-mask-custom-307659m00264.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-pre-custom-307659m00264.jpg
Image c4d_170822_011754_ooi_g_ls9-N9 g : masking 385 positive outlier pixels and 0 negative outlier pixels
Image c4d_160813_013508_ooi_z_ls9-N10 z : masking 1135 positive outlier pixels and 0 negative outlier pixels
Image c4d_160813_022210_ooi_z_ls9-N14 z : masking 590 positive outlier pixels and 0 negative outlier pixels
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outlier-mask-custom-307659m00264.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-post-custom-307659m00264.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-masked-pos-custom-307659m00264.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/outliers-masked-neg-custom-307659m00264.jpg
Stage outliers : Wall: 1.40 s, CPU: 0.82 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 334 MB, VmData: 365 MB, maxrss: 0.342404 MB
Stage outliers finished: 2020-08-06T07:53:16.382314
Resources for stage outliers : 1.250 s worker CPU, 1.508 s worker Wall, Wall: 1.401 s, Cores in use: 1.48, Total efficiency (on 32 cores): 4.6 %
Stage outliers finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-outliers.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-outliers.p
Running stage halos
Running stage halos at 2020-08-06T07:53:16.414293
Stage halos : Wall: 0.41 s, CPU: 0.24 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 335 MB, VmData: 365 MB, maxrss: 0.342804 MB
Stage halos finished: 2020-08-06T07:53:16.827620
Resources for stage halos : 0.600 s worker CPU, 0.897 s worker Wall, Wall: 0.416 s, Cores in use: 2.02, Total efficiency (on 32 cores): 6.3 %
Stage halos finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-halos.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-halos.p
Running stage fit_on_coadds
Running stage fit_on_coadds at 2020-08-06T07:53:17.189437
PSF sigmas in coadd pixels: ['1.98', '1.69']
Using average PSF sigma 1.8355417243605892
Gaussian PSF norm 0.15368476130508332 vs pixelized 0.13155568
average tim pixel scale / coadd scale: 1.0010929152152663
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-copsf-g.fits.fz
PSF sigmas in coadd pixels: ['1.90', '3.17']
Using average PSF sigma 2.539443255773545
Gaussian PSF norm 0.11108529049921563 vs pixelized 0.08840066
average tim pixel scale / coadd scale: 1.0011148097381024
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-copsf-r.fits.fz
PSF sigmas in coadd pixels: ['2.82', '1.42', '1.99']
Using average PSF sigma 2.0781631859058436
Gaussian PSF norm 0.13574236791752078 vs pixelized 0.120690934
average tim pixel scale / coadd scale: 1.0010729702047378
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-copsf-z.fits.fz
Stage fit_on_coadds : Wall: 0.22 s, CPU: 0.20 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 336 MB, VmData: 365 MB, maxrss: 0.344268 MB
Stage fit_on_coadds finished: 2020-08-06T07:53:17.409833
Resources for stage fit_on_coadds : 0.238 s worker CPU, 0.238 s worker Wall, Wall: 0.223 s, Cores in use: 1.99, Total efficiency (on 32 cores): 6.2 %
Stage fit_on_coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fit_on_coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fit_on_coadds.p
Running stage srcs
Running stage srcs at 2020-08-06T07:53:17.546072
SED-matched filters: [('z', array([0., 0., 1.])), ('r', array([0., 1., 0.])), ('g', array([1., 0., 0.])), ('Flat', [1.0, 1.0, 1.0]), ('Red', [2.5, 1.0, 0.4])]
Found 123 potential peaks
Of 123 potential peaks: 35 in veto map, 16 cut by saddle test, 0 cut by aper test, 72 kept
SED z : 72 new peaks
Found 78 potential peaks
Of 78 potential peaks: 71 in veto map, 7 cut by saddle test, 0 cut by aper test, 0 kept
SED r : 0 new peaks
Found 84 potential peaks
Of 84 potential peaks: 67 in veto map, 11 cut by saddle test, 0 cut by aper test, 6 kept
SED g : 6 new peaks
Found 111 potential peaks
Of 111 potential peaks: 100 in veto map, 9 cut by saddle test, 0 cut by aper test, 2 kept
SED Flat : 2 new peaks
Found 130 potential peaks
Of 130 potential peaks: 113 in veto map, 12 cut by saddle test, 0 cut by aper test, 5 kept
SED Red : 5 new peaks
Adding 85 new sources
Median coadd sky for g : 9.068323e-05
Median coadd sky for r : 0.00013714924
Median coadd sky for z : 0.00020202882
Stage srcs : Wall: 1.16 s, CPU: 1.10 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 337 MB, VmData: 365 MB, maxrss: 0.344788 MB
Stage srcs finished: 2020-08-06T07:53:18.708404
Resources for stage srcs : 0.172 s worker CPU, 0.173 s worker Wall, Wall: 1.165 s, Cores in use: 1.09, Total efficiency (on 32 cores): 3.4 %
Stage srcs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-srcs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-srcs.p
Running stage fitblobs
Running stage fitblobs at 2020-08-06T07:53:18.776000
Blob 1 of 28, id: 3, sources: 52, size: 307x403, npix 51182, brick X: 80,387, Y: 0,403, one pixel: 229 0
Blob 2 of 28, id: 7, sources: 7, size: 93x99, npix 4848, brick X: 63,156, Y: 49,148, one pixel: 90 49
Blob 3 of 28, id: 22, sources: 10, size: 79x100, npix 4189, brick X: 271,350, Y: 301,401, one pixel: 317 301
Blob 4 of 28, id: 1, sources: 6, size: 108x49, npix 3499, brick X: 88,196, Y: 0,49, one pixel: 93 0
Blob 5 of 28, id: 14, sources: 3, size: 65x33, npix 1291, brick X: 21,86, Y: 148,181, one pixel: 52 148
Blob 6 of 28, id: 0, sources: 2, size: 46x37, npix 1247, brick X: 6,52, Y: 0,37, one pixel: 10 0
Blob 7 of 28, id: 17, sources: 1, size: 41x39, npix 1138, brick X: 40,81, Y: 194,233, one pixel: 56 194
Blob 8 of 28, id: 4, sources: 1, size: 44x35, npix 1128, brick X: 296,340, Y: 0,35, one pixel: 306 0
Blob 9 of 28, id: 24, sources: 2, size: 39x38, npix 854, brick X: 345,384, Y: 310,348, one pixel: 369 310
Blob 10 of 28, id: 8, sources: 3, size: 37x46, npix 798, brick X: 278,315, Y: 54,100, one pixel: 287 54
Blob 11 of 28, id: 6, sources: 1, size: 37x31, npix 778, brick X: 4,41, Y: 45,76, one pixel: 18 45
Blob 12 of 28, id: 18, sources: 1, size: 30x34, npix 770, brick X: 373,403, Y: 222,256, one pixel: 386 222
Blob 13 of 28, id: 9, sources: 1, size: 32x32, npix 690, brick X: 319,351, Y: 72,104, one pixel: 332 72
Blob 14 of 28, id: 11, sources: 2, size: 35x34, npix 571, brick X: 4,39, Y: 108,142, one pixel: 14 108
Blob 15 of 28, id: 13, sources: 3, size: 20x36, npix 525, brick X: 0,20, Y: 147,183, one pixel: 5 147
Blob 16 of 28, id: 16, sources: 1, size: 28x27, npix 502, brick X: 89,117, Y: 193,220, one pixel: 100 193
Blob 17 of 28, id: 5, sources: 1, size: 18x39, npix 478, brick X: 385,403, Y: 26,65, one pixel: 395 26
Blob 18 of 28, id: 21, sources: 2, size: 18x40, npix 464, brick X: 0,18, Y: 262,302, one pixel: 8 262
Blob 19 of 28, id: 15, sources: 1, size: 20x29, npix 408, brick X: 383,403, Y: 151,180, one pixel: 396 151
Blob 20 of 28, id: 19, sources: 1, size: 26x24, npix 379, brick X: 75,101, Y: 225,249, one pixel: 87 225
Blob 21 of 28, id: 25, sources: 1, size: 25x24, npix 377, brick X: 95,120, Y: 343,367, one pixel: 105 343
Blob 22 of 28, id: 12, sources: 1, size: 19x24, npix 305, brick X: 384,403, Y: 123,147, one pixel: 395 123
Blob 23 of 28, id: 10, sources: 1, size: 21x21, npix 268, brick X: 381,402, Y: 87,108, one pixel: 391 87
Blob 24 of 28, id: 27, sources: 1, size: 18x23, npix 265, brick X: 0,18, Y: 363,386, one pixel: 5 363
Blob 25 of 28, id: 23, sources: 1, size: 19x20, npix 227, brick X: 36,55, Y: 304,324, one pixel: 44 304
Blob 26 of 28, id: 26, sources: 1, size: 20x19, npix 218, brick X: 23,43, Y: 349,368, one pixel: 31 349
Blob 27 of 28, id: 2, sources: 1, size: 20x15, npix 190, brick X: 197,217, Y: 0,15, one pixel: 201 0
Blob 28 of 28, id: 20, sources: 1, size: 18x18, npix 180, brick X: 316,334, Y: 249,267, one pixel: 324 249
Blob 22 finished, total: Wall: 0.34 s, CPU: 0.33 s, VmPeak: 955 MB, VmSize: 726 MB, VmRSS: 140 MB, VmData: 181 MB, maxrss: 0.143488 MB
Blob 23 finished, total: Wall: 0.34 s, CPU: 0.34 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 96 MB, VmData: 142 MB, maxrss: 0.097832 MB
Blob 25 finished, total: Wall: 0.39 s, CPU: 0.39 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 94 MB, VmData: 141 MB, maxrss: 0.09658 MB
Blob 24 finished, total: Wall: 0.42 s, CPU: 0.42 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 96 MB, VmData: 142 MB, maxrss: 0.09832 MB
Blob 14 finished, total: Wall: 0.56 s, CPU: 0.56 s, VmPeak: 957 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.14656 MB
Blob 20 finished, total: Wall: 0.56 s, CPU: 0.56 s, VmPeak: 959 MB, VmSize: 726 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.147276 MB
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (307.69019, -0.26706), pm (4.4, -9.9), parallax 1.122 with NanoMaggies: g=13.2, r=12.9, z=12.9
Reset brightness to NanoMaggies: g=13.2, r=12.9, z=12.9
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (307.67472, -0.26760), pm (0.9, -7.3), parallax 0.431 with NanoMaggies: g=20.6, r=19.8, z=18.7
Reset brightness to NanoMaggies: g=20.8, r=19.5, z=18.8
Blob 15 finished, total: Wall: 0.68 s, CPU: 0.68 s, VmPeak: 961 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146652 MB
Blob 28 finished, total: Wall: 0.75 s, CPU: 0.74 s, VmPeak: 957 MB, VmSize: 726 MB, VmRSS: 142 MB, VmData: 181 MB, maxrss: 0.145244 MB
Blob 21 finished, total: Wall: 0.79 s, CPU: 0.79 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 97 MB, VmData: 143 MB, maxrss: 0.09904 MB
Blob 16 finished, total: Wall: 0.81 s, CPU: 0.81 s, VmPeak: 959 MB, VmSize: 726 MB, VmRSS: 143 MB, VmData: 181 MB, maxrss: 0.146848 MB
Measuring 1 iterative sources
Blob 27 finished, total: Wall: 0.88 s, CPU: 0.88 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.093884 MB
Blob 26 finished, total: Wall: 1.03 s, CPU: 1.03 s, VmPeak: 673 MB, VmSize: 673 MB, VmRSS: 99 MB, VmData: 145 MB, maxrss: 0.101492 MB
Blob 18 finished, total: Wall: 1.07 s, CPU: 1.06 s, VmPeak: 676 MB, VmSize: 676 MB, VmRSS: 101 MB, VmData: 147 MB, maxrss: 0.103796 MB
Blob 13 finished, total: Wall: 1.20 s, CPU: 1.16 s, VmPeak: 667 MB, VmSize: 667 MB, VmRSS: 90 MB, VmData: 138 MB, maxrss: 0.091836 MB
Blob 10 finished, total: Wall: 1.22 s, CPU: 1.22 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096444 MB
Blob 19 finished, total: Wall: 1.24 s, CPU: 1.24 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 95 MB, VmData: 141 MB, maxrss: 0.097656 MB
Blob 7 finished, total: Wall: 1.36 s, CPU: 1.36 s, VmPeak: 671 MB, VmSize: 671 MB, VmRSS: 97 MB, VmData: 142 MB, maxrss: 0.099528 MB
Measuring 1 iterative sources
Blob 17 finished, total: Wall: 1.62 s, CPU: 1.62 s, VmPeak: 669 MB, VmSize: 669 MB, VmRSS: 94 MB, VmData: 140 MB, maxrss: 0.096084 MB
Blob 9 finished, total: Wall: 1.82 s, CPU: 1.82 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 114 MB, VmData: 166 MB, maxrss: 0.116588 MB
Blob 8 finished, total: Wall: 1.92 s, CPU: 1.92 s, VmPeak: 670 MB, VmSize: 670 MB, VmRSS: 96 MB, VmData: 141 MB, maxrss: 0.098132 MB
Blob 11 finished, total: Wall: 2.08 s, CPU: 2.08 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 92 MB, VmData: 139 MB, maxrss: 0.094068 MB
Blob 6 finished, total: Wall: 2.11 s, CPU: 2.11 s, VmPeak: 672 MB, VmSize: 672 MB, VmRSS: 98 MB, VmData: 143 MB, maxrss: 0.100084 MB
Measuring 2 iterative sources
Dropped reference star: GaiaSource at GaiaPosition: RA, Dec = (307.66507, -0.28294), pm (-1.6, -8.0), parallax 0.606 with NanoMaggies: g=18.8, r=18.2, z=18
Reset brightness to NanoMaggies: g=18.8, r=18.2, z=18
Blob 4 finished, total: Wall: 2.36 s, CPU: 2.36 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 114 MB, VmData: 166 MB, maxrss: 0.116548 MB
Measuring 1 iterative sources
Blob 5 finished, total: Wall: 2.81 s, CPU: 2.80 s, VmPeak: 668 MB, VmSize: 668 MB, VmRSS: 94 MB, VmData: 139 MB, maxrss: 0.096032 MB
Blob 12 finished, total: Wall: 2.98 s, CPU: 2.98 s, VmPeak: 958 MB, VmSize: 726 MB, VmRSS: 144 MB, VmData: 181 MB, maxrss: 0.14708 MB
Measuring 1 iterative sources
Blob 3 finished, total: Wall: 7.26 s, CPU: 7.26 s, VmPeak: 695 MB, VmSize: 695 MB, VmRSS: 114 MB, VmData: 166 MB, maxrss: 0.116952 MB
Measuring 2 iterative sources
Blob 2 finished, total: Wall: 8.42 s, CPU: 8.42 s, VmPeak: 674 MB, VmSize: 672 MB, VmRSS: 98 MB, VmData: 143 MB, maxrss: 0.102616 MB
Known galaxy.  Initial shape: EllipseWithPriors(0.25): log r_e=1.98075, ee1=0.228868, ee2=0.530994
Measuring 5 iterative sources
Blob 1 finished, total: Wall: 54.77 s, CPU: 54.77 s, VmPeak: 754 MB, VmSize: 744 MB, VmRSS: 180 MB, VmData: 215 MB, maxrss: 0.188304 MB
Old catalog: 116
New catalog: 110
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/blobs-custom-307659m00264.fits.gz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/metrics/cus/all-models-custom-307659m00264.fits
Stage fitblobs : Wall: 55.30 s, CPU: 0.57 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 338 MB, VmData: 365 MB, maxrss: 0.3461 MB
Stage fitblobs finished: 2020-08-06T07:54:14.076258
Resources for stage fitblobs : 102.225 s worker CPU, 102.365 s worker Wall, Wall: 55.303 s, Cores in use: 1.86, Total efficiency (on 32 cores): 5.8 %
Stage fitblobs finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fitblobs.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-fitblobs.p
Running stage coadds
Running stage coadds at 2020-08-06T07:54:14.212964
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-ccds.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-nexp-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-galdepth-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-psfsize-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-chi2-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel-g.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-nexp-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-galdepth-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-psfsize-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-chi2-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel-r.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-nexp-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-galdepth-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-psfsize-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-chi2-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel-z.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-depth.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-blobmodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-resid.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-resid.jpg
Stage coadds : Wall: 5.65 s, CPU: 1.17 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 338 MB, VmData: 365 MB, maxrss: 0.346424 MB
Stage coadds finished: 2020-08-06T07:54:19.860917
Resources for stage coadds : 37.165 s worker CPU, 40.831 s worker Wall, Wall: 5.650 s, Cores in use: 6.79, Total efficiency (on 32 cores): 21.2 %
Stage coadds finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-coadds.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-coadds.p
Running stage wise_forced
Running stage wise_forced at 2020-08-06T07:54:19.887167
Cut to 1 unWISE tiles
Reading WISE tile 3070p000 band 2
Reading WISE tile 3070p000 band 1
Reading WISE tile 3070p000 band 3
Reading WISE tile 3070p000 band 4
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w4-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.34 s, CPU: 1.04 s, VmPeak: 1007 MB, VmSize: 1007 MB, VmRSS: 147 MB, VmData: 190 MB, maxrss: 0.187956 MB
Does not exist: /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/allwise/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w3-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.21 s, CPU: 0.53 s, VmPeak: 1066 MB, VmSize: 1066 MB, VmRSS: 208 MB, VmData: 248 MB, maxrss: 0.213108 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w2-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.15 s, CPU: 0.35 s, VmPeak: 1006 MB, VmSize: 1006 MB, VmRSS: 144 MB, VmData: 185 MB, maxrss: 0.207176 MB
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-img-m.fits
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-invvar-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-n-m.fits.gz
Reading /global/cfs/cdirs/cosmo/data/unwise/neo6/unwise-coadds/fulldepth/307/3070p000/unwise-3070p000-w1-n-u.fits.gz
Reading /usr/local/lib/python/wise/wise-psf-avg.fits
unWISE forced photometry took Wall: 0.16 s, CPU: 0.37 s, VmPeak: 1003 MB, VmSize: 1003 MB, VmRSS: 142 MB, VmData: 186 MB, maxrss: 0.204352 MB
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W1.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W2.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W3.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-image-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-model-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-invvar-W4.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-wise.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/tmp-legacysurvey-custom-307659m00264-wisemodel.jpg
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-wisemodel.jpg
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Setting WISE apphot
Stage wise_forced : Wall: 3.29 s, CPU: 1.18 s, VmPeak: 1144 MB, VmSize: 1144 MB, VmRSS: 339 MB, VmData: 365 MB, maxrss: 0.346936 MB
Stage wise_forced finished: 2020-08-06T07:54:23.177049
Resources for stage wise_forced : 5.471 s worker CPU, 6.331 s worker Wall, Wall: 3.292 s, Cores in use: 2.02, Total efficiency (on 32 cores): 6.3 %
Stage wise_forced finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-wise_forced.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-wise_forced.p
Running stage writecat
Running stage writecat at 2020-08-06T07:54:23.912395
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/coadd/cus/custom-307659m00264/legacysurvey-custom-307659m00264-maskbits.fits.fz
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/tractor-i/cus/tractor-custom-307659m00264.fits
Reading SFD maps...
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
END                                                                             
  Rejected (1), keyrecords following the END keyrecord were ignored.
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/tractor/cus/tractor-custom-307659m00264.fits
Wrote /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/tractor/cus/brick-custom-307659m00264.sha256sum
Stage writecat : Wall: 1.09 s, CPU: 0.83 s, VmPeak: 1276 MB, VmSize: 1148 MB, VmRSS: 341 MB, VmData: 365 MB, maxrss: 0.479796 MB
Stage writecat finished: 2020-08-06T07:54:25.003438
Resources for stage writecat : 0.000 s worker CPU, 0.001 s worker Wall, Wall: 1.093 s, Cores in use: 0.76, Total efficiency (on 32 cores): 2.4 %
Stage writecat finished
Saving pickle /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-writecat.p
Saved /global/cscratch1/sd/ioannis/SGA-data-dr9alpha/307/PGC3441840/PGC3441840-largegalaxy-writecat.p
All done: 150.999 s worker CPU, 187.666 s worker Wall, Wall: 86.282 s, Cores in use: 1.94, Total efficiency (on 32 cores): 6.1 %
Finished galaxy PGC3441840 in 1.511 minutes.