Started working on galaxy NGC2976_GROUP at Sat Feb  8 08:00:32 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:00:33.389136
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '146.82392155320917', '67.91592622424893', '--width', '2681', '--height', '2681', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 146.82392155320917 67.91592622424893
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 6775 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 6776 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 6777 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 6778 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 6779 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 6780 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 6781 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 6782 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:00:34.804457
Keeping 19 CCDs:
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD4 g exptime 112.798 propid NOTBASS    seeing 1.28 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD4 r exptime 143.411 propid NOTBASS    seeing 1.17 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD2 g exptime 112.798 propid NOTBASS    seeing 1.31 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD2 r exptime 143.411 propid NOTBASS    seeing 1.32 MJD 58511.464 object 
   ksb_190128_063157_ooi_g_ls9.fits.fz ksb_190128_063157_ooi_g_ls9-CCD1 g exptime 97.442 propid NOTBASS    seeing 1.40 MJD 58511.272 object 
   ksb_190209_093733_ooi_r_ls9.fits.fz ksb_190209_093733_ooi_r_ls9-CCD1 r exptime 122.645 propid NOTBASS    seeing 1.57 MJD 58523.401 object 
   ksb_180410_063935_ooi_g_ls9.fits.fz ksb_180410_063935_ooi_g_ls9-CCD4 g exptime 224.25 propid BASS       seeing 1.46 MJD 58218.277 object 44302
   ksb_190131_115514_ooi_r_ls9.fits.fz ksb_190131_115514_ooi_r_ls9-CCD4 r exptime 200.6 propid NOTBASS    seeing 1.54 MJD 58514.497 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD1 r exptime 143.411 propid NOTBASS    seeing 1.29 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD1 g exptime 112.798 propid NOTBASS    seeing 1.40 MJD 58495.436 object 
   k4m_171204_074619_ooi_zd_ls9.fits.fz k4m_171204_074619_ooi_zd_ls9-CCD2 z exptime 246.105 propid 2016A-0453 seeing 1.63 MJD 58091.324 object MzLS_708020_z
   k4m_160127_085335_ooi_zd_ls9.fits.fz k4m_160127_085335_ooi_zd_ls9-CCD2 z exptime 129.104 propid 2016A-0453 seeing 1.05 MJD 57414.371 object MzLS_708020_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD1 z exptime 117.106 propid 2016B-0453 seeing 1.02 MJD 57412.392 object MzLS_709454_z
   k4m_171204_074135_ooi_zd_ls9.fits.fz k4m_171204_074135_ooi_zd_ls9-CCD4 z exptime 246.106 propid 2016A-0453 seeing 1.35 MJD 58091.321 object MzLS_708019_z
   k4m_160127_084815_ooi_zd_ls9.fits.fz k4m_160127_084815_ooi_zd_ls9-CCD4 z exptime 110.104 propid 2016A-0453 seeing 1.15 MJD 57414.367 object MzLS_708019_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD3 z exptime 117.106 propid 2016B-0453 seeing 0.97 MJD 57412.392 object MzLS_709454_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD4 z exptime 80.103 propid 2016A-0453 seeing 1.08 MJD 57464.258 object MzLS_708979_z
   k4m_160320_055942_ooi_zd_ls9.fits.fz k4m_160320_055942_ooi_zd_ls9-CCD4 z exptime 93.105 propid 2016A-0453 seeing 0.77 MJD 57467.250 object MzLS_708501_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD3 z exptime 80.103 propid 2016A-0453 seeing 1.13 MJD 57464.258 object MzLS_708979_z
Faked up EXPNUM 82180085
Faked up EXPNUM 82180085
Faked up EXPNUM 82180085
90prime get_fwhm: self.fwhm = 3.21
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
90prime get_fwhm: self.fwhm = 3.47
WARNING: EXPNUM None!=160125092505 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092505_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 4.15
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 2.89
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 3.09
mosaic get_fwhm: self.fwhm = 5.19
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.91
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.85
mosaic get_fwhm: self.fwhm = 4.41
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2976_GROUP; please check the logfile.
Finished galaxy NGC2976_GROUP in 0.125 minutes.
Started working on galaxy NGC2976_GROUP at Sat Feb  8 12:48:58 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T12:48:59.091814
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '146.82392155320917', '67.91592622424893', '--width', '2681', '--height', '2681', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 146.82392155320917 67.91592622424893
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 31628 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31629 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31630 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31631 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31632 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 31633 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 31634 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 31635 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T12:49:00.435487
Keeping 19 CCDs:
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD4 g exptime 112.798 propid NOTBASS    seeing 1.28 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD4 r exptime 143.411 propid NOTBASS    seeing 1.17 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD2 g exptime 112.798 propid NOTBASS    seeing 1.31 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD2 r exptime 143.411 propid NOTBASS    seeing 1.32 MJD 58511.464 object 
   ksb_190128_063157_ooi_g_ls9.fits.fz ksb_190128_063157_ooi_g_ls9-CCD1 g exptime 97.442 propid NOTBASS    seeing 1.40 MJD 58511.272 object 
   ksb_190209_093733_ooi_r_ls9.fits.fz ksb_190209_093733_ooi_r_ls9-CCD1 r exptime 122.645 propid NOTBASS    seeing 1.57 MJD 58523.401 object 
   ksb_180410_063935_ooi_g_ls9.fits.fz ksb_180410_063935_ooi_g_ls9-CCD4 g exptime 224.25 propid BASS       seeing 1.46 MJD 58218.277 object 44302
   ksb_190131_115514_ooi_r_ls9.fits.fz ksb_190131_115514_ooi_r_ls9-CCD4 r exptime 200.6 propid NOTBASS    seeing 1.54 MJD 58514.497 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD1 r exptime 143.411 propid NOTBASS    seeing 1.29 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD1 g exptime 112.798 propid NOTBASS    seeing 1.40 MJD 58495.436 object 
   k4m_171204_074619_ooi_zd_ls9.fits.fz k4m_171204_074619_ooi_zd_ls9-CCD2 z exptime 246.105 propid 2016A-0453 seeing 1.63 MJD 58091.324 object MzLS_708020_z
   k4m_160127_085335_ooi_zd_ls9.fits.fz k4m_160127_085335_ooi_zd_ls9-CCD2 z exptime 129.104 propid 2016A-0453 seeing 1.05 MJD 57414.371 object MzLS_708020_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD1 z exptime 117.106 propid 2016B-0453 seeing 1.02 MJD 57412.392 object MzLS_709454_z
   k4m_171204_074135_ooi_zd_ls9.fits.fz k4m_171204_074135_ooi_zd_ls9-CCD4 z exptime 246.106 propid 2016A-0453 seeing 1.35 MJD 58091.321 object MzLS_708019_z
   k4m_160127_084815_ooi_zd_ls9.fits.fz k4m_160127_084815_ooi_zd_ls9-CCD4 z exptime 110.104 propid 2016A-0453 seeing 1.15 MJD 57414.367 object MzLS_708019_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD3 z exptime 117.106 propid 2016B-0453 seeing 0.97 MJD 57412.392 object MzLS_709454_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD4 z exptime 80.103 propid 2016A-0453 seeing 1.08 MJD 57464.258 object MzLS_708979_z
   k4m_160320_055942_ooi_zd_ls9.fits.fz k4m_160320_055942_ooi_zd_ls9-CCD4 z exptime 93.105 propid 2016A-0453 seeing 0.77 MJD 57467.250 object MzLS_708501_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD3 z exptime 80.103 propid 2016A-0453 seeing 1.13 MJD 57464.258 object MzLS_708979_z
Faked up EXPNUM 82180085
Faked up EXPNUM 82180085
Faked up EXPNUM 82180085
90prime get_fwhm: self.fwhm = 3.21
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 2.89
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.85
WARNING: EXPNUM None!=160125092505 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092505_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 4.41
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.57
mosaic get_fwhm: self.fwhm = 6.29
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 2.83
mosaic get_fwhm: self.fwhm = 4.05
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
90prime get_fwhm: self.fwhm = 3.47
WARNING: EXPNUM None!=160125092505 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092505_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 4.15
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.91
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 3.09
mosaic get_fwhm: self.fwhm = 5.19
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2976_GROUP; please check the logfile.
Finished galaxy NGC2976_GROUP in 0.121 minutes.
Started working on galaxy NGC2976_GROUP at Sun Feb  9 16:11:16 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T16:11:16.734670
legacypipe git version: dr9e-pre1-8-gdd90f8de
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '146.82392155320917', '67.91592622424893', '--width', '2681', '--height', '2681', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 146.82392155320917 67.91592622424893
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 53770 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07116 MB
Starting process 53771 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07116 MB
Starting process 53772 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07116 MB
Starting process 53773 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07116 MB
Starting process 53774 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071164 MB
Starting process 53775 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071168 MB
Starting process 53776 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071168 MB
Starting process 53777 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071172 MB
Starting process 53778 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071172 MB
Starting process 53779 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071172 MB
Starting process 53780 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071176 MB
Starting process 53781 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071176 MB
Starting process 53782 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071176 MB
Starting process 53783 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071176 MB
Starting process 53784 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07118 MB
Starting process 53785 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07118 MB
Starting process 53786 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 141 MB, maxrss: 0.071184 MB
Starting process 53787 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 141 MB, maxrss: 0.072196 MB
Starting process 53788 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 141 MB, maxrss: 0.072196 MB
Starting process 53789 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07192 MB
Starting process 53790 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071976 MB
Starting process 53791 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071544 MB
Starting process 53792 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07154 MB
Starting process 53793 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07154 MB
Starting process 53794 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071476 MB
Starting process 53795 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071476 MB
Starting process 53796 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071476 MB
Starting process 53797 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071476 MB
Starting process 53798 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 53799 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.07148 MB
Starting process 53800 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071484 MB
Starting process 53801 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 141 MB, maxrss: 0.071484 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T16:11:18.470158
Keeping 19 CCDs:
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD4 g exptime 112.798 propid NOTBASS    seeing 1.28 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD4 r exptime 143.411 propid NOTBASS    seeing 1.17 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD2 g exptime 112.798 propid NOTBASS    seeing 1.31 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD2 r exptime 143.411 propid NOTBASS    seeing 1.32 MJD 58511.464 object 
   ksb_190128_063157_ooi_g_ls9.fits.fz ksb_190128_063157_ooi_g_ls9-CCD1 g exptime 97.442 propid NOTBASS    seeing 1.40 MJD 58511.272 object 
   ksb_190209_093733_ooi_r_ls9.fits.fz ksb_190209_093733_ooi_r_ls9-CCD1 r exptime 122.645 propid NOTBASS    seeing 1.57 MJD 58523.401 object 
   ksb_180410_063935_ooi_g_ls9.fits.fz ksb_180410_063935_ooi_g_ls9-CCD4 g exptime 224.25 propid BASS       seeing 1.46 MJD 58218.277 object 44302
   ksb_190131_115514_ooi_r_ls9.fits.fz ksb_190131_115514_ooi_r_ls9-CCD4 r exptime 200.6 propid NOTBASS    seeing 1.54 MJD 58514.497 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD1 r exptime 143.411 propid NOTBASS    seeing 1.29 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD1 g exptime 112.798 propid NOTBASS    seeing 1.40 MJD 58495.436 object 
   k4m_171204_074619_ooi_zd_ls9.fits.fz k4m_171204_074619_ooi_zd_ls9-CCD2 z exptime 246.105 propid 2016A-0453 seeing 1.63 MJD 58091.324 object MzLS_708020_z
   k4m_160127_085335_ooi_zd_ls9.fits.fz k4m_160127_085335_ooi_zd_ls9-CCD2 z exptime 129.104 propid 2016A-0453 seeing 1.05 MJD 57414.371 object MzLS_708020_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD1 z exptime 117.106 propid 2016B-0453 seeing 1.02 MJD 57412.392 object MzLS_709454_z
   k4m_171204_074135_ooi_zd_ls9.fits.fz k4m_171204_074135_ooi_zd_ls9-CCD4 z exptime 246.106 propid 2016A-0453 seeing 1.35 MJD 58091.321 object MzLS_708019_z
   k4m_160127_084815_ooi_zd_ls9.fits.fz k4m_160127_084815_ooi_zd_ls9-CCD4 z exptime 110.104 propid 2016A-0453 seeing 1.15 MJD 57414.367 object MzLS_708019_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD3 z exptime 117.106 propid 2016B-0453 seeing 0.97 MJD 57412.392 object MzLS_709454_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD4 z exptime 80.103 propid 2016A-0453 seeing 1.08 MJD 57464.258 object MzLS_708979_z
   k4m_160320_055942_ooi_zd_ls9.fits.fz k4m_160320_055942_ooi_zd_ls9-CCD4 z exptime 93.105 propid 2016A-0453 seeing 0.77 MJD 57467.250 object MzLS_708501_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD3 z exptime 80.103 propid 2016A-0453 seeing 1.13 MJD 57464.258 object MzLS_708979_z
WARNING: EXPNUM None!=160125092505 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092505_ooi_zd_ls9.fits.fz header
WARNING: EXPNUM None!=160125092505 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092505_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.85
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 3.09
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 2.89
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.91
mosaic get_fwhm: self.fwhm = 5.19
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.57
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
90prime get_fwhm: self.fwhm = 3.47
mosaic get_fwhm: self.fwhm = 6.29
mosaic get_fwhm: self.fwhm = 4.05
Faked up EXPNUM 85110143
Faked up EXPNUM 85110143
Faked up EXPNUM 85110143
90prime get_fwhm: self.fwhm = 3.09
Faked up EXPNUM 85140230
Faked up EXPNUM 85140230
Faked up EXPNUM 85140230
90prime get_fwhm: self.fwhm = 3.39
mosaic get_fwhm: self.fwhm = 4.15
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2976_GROUP; please check the logfile.
Finished galaxy NGC2976_GROUP in 0.114 minutes.
Started working on galaxy NGC2976_GROUP at Mon Feb 10 06:18:27 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-10T06:18:28.297709
legacypipe git version: dr9e-pre1-9-g4a51be93
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '146.82392155320917', '67.91592622424893', '--width', '2681', '--height', '2681', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 146.82392155320917 67.91592622424893 --width 2681 --height 2681 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/146/NGC2976_GROUP/NGC2976_GROUP-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 146.82392155320917 67.91592622424893
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 15911 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15912 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15913 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15914 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15915 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 15916 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 15917 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 15918 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 15919 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15920 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15921 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 15922 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15923 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15924 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15925 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15926 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15927 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 15928 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072156 MB
Starting process 15929 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072156 MB
Starting process 15930 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071864 MB
Starting process 15931 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071868 MB
Starting process 15932 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 15933 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071504 MB
Starting process 15934 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071504 MB
Starting process 15935 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15936 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15937 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15938 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15939 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15940 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15941 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Starting process 15942 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-10T06:18:29.537717
Keeping 19 CCDs:
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD4 g exptime 112.798 propid NOTBASS    seeing 1.28 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD4 r exptime 143.411 propid NOTBASS    seeing 1.17 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD2 g exptime 112.798 propid NOTBASS    seeing 1.31 MJD 58495.436 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD2 r exptime 143.411 propid NOTBASS    seeing 1.32 MJD 58511.464 object 
   ksb_190128_063157_ooi_g_ls9.fits.fz ksb_190128_063157_ooi_g_ls9-CCD1 g exptime 97.442 propid NOTBASS    seeing 1.40 MJD 58511.272 object 
   ksb_190209_093733_ooi_r_ls9.fits.fz ksb_190209_093733_ooi_r_ls9-CCD1 r exptime 122.645 propid NOTBASS    seeing 1.57 MJD 58523.401 object 
   ksb_180410_063935_ooi_g_ls9.fits.fz ksb_180410_063935_ooi_g_ls9-CCD4 g exptime 224.25 propid BASS       seeing 1.46 MJD 58218.277 object 44302
   ksb_190131_115514_ooi_r_ls9.fits.fz ksb_190131_115514_ooi_r_ls9-CCD4 r exptime 200.6 propid NOTBASS    seeing 1.54 MJD 58514.497 object 
   ksb_190128_110823_ooi_r_ls9.fits.fz ksb_190128_110823_ooi_r_ls9-CCD1 r exptime 143.411 propid NOTBASS    seeing 1.29 MJD 58511.464 object 
   ksb_190112_102743_ooi_g_ls9.fits.fz ksb_190112_102743_ooi_g_ls9-CCD1 g exptime 112.798 propid NOTBASS    seeing 1.40 MJD 58495.436 object 
   k4m_171204_074619_ooi_zd_ls9.fits.fz k4m_171204_074619_ooi_zd_ls9-CCD2 z exptime 246.105 propid 2016A-0453 seeing 1.63 MJD 58091.324 object MzLS_708020_z
   k4m_160127_085335_ooi_zd_ls9.fits.fz k4m_160127_085335_ooi_zd_ls9-CCD2 z exptime 129.104 propid 2016A-0453 seeing 1.05 MJD 57414.371 object MzLS_708020_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD1 z exptime 117.106 propid 2016B-0453 seeing 1.02 MJD 57412.392 object MzLS_709454_z
   k4m_171204_074135_ooi_zd_ls9.fits.fz k4m_171204_074135_ooi_zd_ls9-CCD4 z exptime 246.106 propid 2016A-0453 seeing 1.35 MJD 58091.321 object MzLS_708019_z
   k4m_160127_084815_ooi_zd_ls9.fits.fz k4m_160127_084815_ooi_zd_ls9-CCD4 z exptime 110.104 propid 2016A-0453 seeing 1.15 MJD 57414.367 object MzLS_708019_z
   k4m_160125_092505_ooi_zd_ls9.fits.fz k4m_160125_092505_ooi_zd_ls9-CCD3 z exptime 117.106 propid 2016B-0453 seeing 0.97 MJD 57412.392 object MzLS_709454_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD4 z exptime 80.103 propid 2016A-0453 seeing 1.08 MJD 57464.258 object MzLS_708979_z
   k4m_160320_055942_ooi_zd_ls9.fits.fz k4m_160320_055942_ooi_zd_ls9-CCD4 z exptime 93.105 propid 2016A-0453 seeing 0.77 MJD 57467.250 object MzLS_708501_z
   k4m_160317_061131_ooi_zd_ls9.fits.fz k4m_160317_061131_ooi_zd_ls9-CCD3 z exptime 80.103 propid 2016A-0453 seeing 1.13 MJD 57464.258 object MzLS_708979_z
WARNING: EXPNUM None!=160125092505 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092505_ooi_zd_ls9.fits.fz header
WARNING: EXPNUM None!=160125092505 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_092505_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 3.09
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.85
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
Faked up EXPNUM 84950061
90prime get_fwhm: self.fwhm = 2.89
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.91
mosaic get_fwhm: self.fwhm = 5.19
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
Faked up EXPNUM 85110246
90prime get_fwhm: self.fwhm = 2.57
Faked up EXPNUM 85110143
Faked up EXPNUM 85110143
Faked up EXPNUM 85110143
90prime get_fwhm: self.fwhm = 3.09
mosaic get_fwhm: self.fwhm = 6.29
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
Faked up EXPNUM 85230159
90prime get_fwhm: self.fwhm = 3.47
Faked up EXPNUM 82180085
Faked up EXPNUM 82180085
Faked up EXPNUM 82180085
90prime get_fwhm: self.fwhm = 3.21
mosaic get_fwhm: self.fwhm = 4.05
Faked up EXPNUM 85140230
Faked up EXPNUM 85140230
Faked up EXPNUM 85140230
90prime get_fwhm: self.fwhm = 3.39
mosaic get_fwhm: self.fwhm = 4.15
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2976_GROUP; please check the logfile.
Finished galaxy NGC2976_GROUP in 0.093 minutes.