Started working on galaxy NGC2787 at Sat Feb  8 08:08:44 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:08:44.729449
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '139.82681100000002', '69.2033065', '--width', '2225', '--height', '2225', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 139.82681100000002 69.2033065
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 57709 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 57710 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 57711 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 57712 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 57713 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 57714 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 57715 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 57716 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:08:47.145942
Keeping 16 CCDs:
   ksb_190117_102440_ooi_r_ls9.fits.fz ksb_190117_102440_ooi_r_ls9-CCD3 r exptime 250.0 propid NOTBASS    seeing 1.82 MJD 58500.434 object 
   ksb_190111_104852_ooi_g_ls9.fits.fz ksb_190111_104852_ooi_g_ls9-CCD3 g exptime 198.403 propid NOTBASS    seeing 1.68 MJD 58494.451 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD3 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD3 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   ksb_190128_060634_ooi_g_ls9.fits.fz ksb_190128_060634_ooi_g_ls9-CCD4 g exptime 155.187 propid NOTBASS    seeing 1.43 MJD 58511.255 object 
   ksb_190209_085835_ooi_r_ls9.fits.fz ksb_190209_085835_ooi_r_ls9-CCD4 r exptime 152.5 propid NOTBASS    seeing 1.58 MJD 58523.374 object 
   ksb_190209_085247_ooi_r_ls9.fits.fz ksb_190209_085247_ooi_r_ls9-CCD1 r exptime 146.284 propid NOTBASS    seeing 1.48 MJD 58523.370 object 
   ksb_190127_063512_ooi_g_ls9.fits.fz ksb_190127_063512_ooi_g_ls9-CCD1 g exptime 220.0 propid NOTBASS    seeing 2.34 MJD 58510.274 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD4 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD4 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016A-0453 seeing 1.16 MJD 57466.232 object MzLS_711760_z
   k4m_160125_084615_ooi_zd_ls9.fits.fz k4m_160125_084615_ooi_zd_ls9-CCD3 z exptime 128.103 propid 2016B-0453 seeing 0.93 MJD 57412.365 object MzLS_712210_z
   k4m_160319_053919_ooi_zd_ls9.fits.fz k4m_160319_053919_ooi_zd_ls9-CCD2 z exptime 106.106 propid 2016A-0453 seeing 1.18 MJD 57466.236 object MzLS_711306_z
   k4m_160319_053642_ooi_zd_ls9.fits.fz k4m_160319_053642_ooi_zd_ls9-CCD1 z exptime 117.103 propid 2016A-0453 seeing 1.20 MJD 57466.234 object MzLS_712657_z
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD4 z exptime 128.103 propid 2016A-0453 seeing 1.11 MJD 57466.232 object MzLS_711760_z
   k4m_160125_085700_ooi_zd_ls9.fits.fz k4m_160125_085700_ooi_zd_ls9-CCD1 z exptime 160.1 propid 2016B-0453 seeing 1.05 MJD 57412.373 object MzLS_712211_z
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
90prime get_fwhm: self.fwhm = 3.27
WARNING: EXPNUM None!=160125085700 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_085700_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
90prime get_fwhm: self.fwhm = 5.17
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
90prime get_fwhm: self.fwhm = 3.15
mosaic get_fwhm: self.fwhm = 4.27
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
90prime get_fwhm: self.fwhm = 3.71
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2787; please check the logfile.
Finished galaxy NGC2787 in 0.127 minutes.
Started working on galaxy NGC2787 at Sat Feb  8 08:35:02 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:35:02.595058
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '139.82681100000002', '69.2033065', '--width', '2225', '--height', '2225', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 139.82681100000002 69.2033065
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 26933 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 26934 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 26935 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 26936 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 26937 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 26938 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 26939 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 26940 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:35:03.880764
Keeping 16 CCDs:
   ksb_190117_102440_ooi_r_ls9.fits.fz ksb_190117_102440_ooi_r_ls9-CCD3 r exptime 250.0 propid NOTBASS    seeing 1.82 MJD 58500.434 object 
   ksb_190111_104852_ooi_g_ls9.fits.fz ksb_190111_104852_ooi_g_ls9-CCD3 g exptime 198.403 propid NOTBASS    seeing 1.68 MJD 58494.451 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD3 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD3 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   ksb_190128_060634_ooi_g_ls9.fits.fz ksb_190128_060634_ooi_g_ls9-CCD4 g exptime 155.187 propid NOTBASS    seeing 1.43 MJD 58511.255 object 
   ksb_190209_085835_ooi_r_ls9.fits.fz ksb_190209_085835_ooi_r_ls9-CCD4 r exptime 152.5 propid NOTBASS    seeing 1.58 MJD 58523.374 object 
   ksb_190209_085247_ooi_r_ls9.fits.fz ksb_190209_085247_ooi_r_ls9-CCD1 r exptime 146.284 propid NOTBASS    seeing 1.48 MJD 58523.370 object 
   ksb_190127_063512_ooi_g_ls9.fits.fz ksb_190127_063512_ooi_g_ls9-CCD1 g exptime 220.0 propid NOTBASS    seeing 2.34 MJD 58510.274 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD4 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD4 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016A-0453 seeing 1.16 MJD 57466.232 object MzLS_711760_z
   k4m_160125_084615_ooi_zd_ls9.fits.fz k4m_160125_084615_ooi_zd_ls9-CCD3 z exptime 128.103 propid 2016B-0453 seeing 0.93 MJD 57412.365 object MzLS_712210_z
   k4m_160319_053919_ooi_zd_ls9.fits.fz k4m_160319_053919_ooi_zd_ls9-CCD2 z exptime 106.106 propid 2016A-0453 seeing 1.18 MJD 57466.236 object MzLS_711306_z
   k4m_160319_053642_ooi_zd_ls9.fits.fz k4m_160319_053642_ooi_zd_ls9-CCD1 z exptime 117.103 propid 2016A-0453 seeing 1.20 MJD 57466.234 object MzLS_712657_z
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD4 z exptime 128.103 propid 2016A-0453 seeing 1.11 MJD 57466.232 object MzLS_711760_z
   k4m_160125_085700_ooi_zd_ls9.fits.fz k4m_160125_085700_ooi_zd_ls9-CCD1 z exptime 160.1 propid 2016B-0453 seeing 1.05 MJD 57412.373 object MzLS_712211_z
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
90prime get_fwhm: self.fwhm = 3.49
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
90prime get_fwhm: self.fwhm = 5.17
WARNING: EXPNUM None!=160125085700 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_085700_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
90prime get_fwhm: self.fwhm = 3.71
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
90prime get_fwhm: self.fwhm = 3.27
mosaic get_fwhm: self.fwhm = 4.27
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.45
mosaic get_fwhm: self.fwhm = 4.45
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
90prime get_fwhm: self.fwhm = 4.01
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.43
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2787; please check the logfile.
Finished galaxy NGC2787 in 0.116 minutes.
Started working on galaxy NGC2787 at Sat Feb  8 13:07:14 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T13:07:15.022624
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '139.82681100000002', '69.2033065', '--width', '2225', '--height', '2225', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 139.82681100000002 69.2033065
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 2012 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 2013 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 2014 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 2015 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 2016 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 2017 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 2018 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 2019 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T13:07:16.341569
Keeping 16 CCDs:
   ksb_190117_102440_ooi_r_ls9.fits.fz ksb_190117_102440_ooi_r_ls9-CCD3 r exptime 250.0 propid NOTBASS    seeing 1.82 MJD 58500.434 object 
   ksb_190111_104852_ooi_g_ls9.fits.fz ksb_190111_104852_ooi_g_ls9-CCD3 g exptime 198.403 propid NOTBASS    seeing 1.68 MJD 58494.451 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD3 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD3 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   ksb_190128_060634_ooi_g_ls9.fits.fz ksb_190128_060634_ooi_g_ls9-CCD4 g exptime 155.187 propid NOTBASS    seeing 1.43 MJD 58511.255 object 
   ksb_190209_085835_ooi_r_ls9.fits.fz ksb_190209_085835_ooi_r_ls9-CCD4 r exptime 152.5 propid NOTBASS    seeing 1.58 MJD 58523.374 object 
   ksb_190209_085247_ooi_r_ls9.fits.fz ksb_190209_085247_ooi_r_ls9-CCD1 r exptime 146.284 propid NOTBASS    seeing 1.48 MJD 58523.370 object 
   ksb_190127_063512_ooi_g_ls9.fits.fz ksb_190127_063512_ooi_g_ls9-CCD1 g exptime 220.0 propid NOTBASS    seeing 2.34 MJD 58510.274 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD4 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD4 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016A-0453 seeing 1.16 MJD 57466.232 object MzLS_711760_z
   k4m_160125_084615_ooi_zd_ls9.fits.fz k4m_160125_084615_ooi_zd_ls9-CCD3 z exptime 128.103 propid 2016B-0453 seeing 0.93 MJD 57412.365 object MzLS_712210_z
   k4m_160319_053919_ooi_zd_ls9.fits.fz k4m_160319_053919_ooi_zd_ls9-CCD2 z exptime 106.106 propid 2016A-0453 seeing 1.18 MJD 57466.236 object MzLS_711306_z
   k4m_160319_053642_ooi_zd_ls9.fits.fz k4m_160319_053642_ooi_zd_ls9-CCD1 z exptime 117.103 propid 2016A-0453 seeing 1.20 MJD 57466.234 object MzLS_712657_z
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD4 z exptime 128.103 propid 2016A-0453 seeing 1.11 MJD 57466.232 object MzLS_711760_z
   k4m_160125_085700_ooi_zd_ls9.fits.fz k4m_160125_085700_ooi_zd_ls9-CCD1 z exptime 160.1 propid 2016B-0453 seeing 1.05 MJD 57412.373 object MzLS_712211_z
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
90prime get_fwhm: self.fwhm = 5.17
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
90prime get_fwhm: self.fwhm = 3.27
WARNING: EXPNUM None!=160125085700 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_085700_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
90prime get_fwhm: self.fwhm = 3.49
mosaic get_fwhm: self.fwhm = 4.27
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
90prime get_fwhm: self.fwhm = 4.01
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.43
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.45
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2787; please check the logfile.
Finished galaxy NGC2787 in 0.096 minutes.
Started working on galaxy NGC2787 at Sun Feb  9 08:25:00 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T08:25:00.706533
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '139.82681100000002', '69.2033065', '--width', '2225', '--height', '2225', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 139.82681100000002 69.2033065
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 28847 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 28848 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 28849 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 28850 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 28851 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 28852 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 28853 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 28854 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 28855 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 28856 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 28857 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 28858 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 28859 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 28860 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 28861 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 28862 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 28863 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071192 MB
Starting process 28864 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072204 MB
Starting process 28865 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072204 MB
Starting process 28866 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071912 MB
Starting process 28867 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071912 MB
Starting process 28868 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB
Starting process 28869 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 28870 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 28871 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 28872 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 28873 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 28874 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 28875 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 28876 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 28877 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 28878 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071488 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T08:25:01.962608
Keeping 16 CCDs:
   ksb_190117_102440_ooi_r_ls9.fits.fz ksb_190117_102440_ooi_r_ls9-CCD3 r exptime 250.0 propid NOTBASS    seeing 1.82 MJD 58500.434 object 
   ksb_190111_104852_ooi_g_ls9.fits.fz ksb_190111_104852_ooi_g_ls9-CCD3 g exptime 198.403 propid NOTBASS    seeing 1.68 MJD 58494.451 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD3 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD3 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   ksb_190128_060634_ooi_g_ls9.fits.fz ksb_190128_060634_ooi_g_ls9-CCD4 g exptime 155.187 propid NOTBASS    seeing 1.43 MJD 58511.255 object 
   ksb_190209_085835_ooi_r_ls9.fits.fz ksb_190209_085835_ooi_r_ls9-CCD4 r exptime 152.5 propid NOTBASS    seeing 1.58 MJD 58523.374 object 
   ksb_190209_085247_ooi_r_ls9.fits.fz ksb_190209_085247_ooi_r_ls9-CCD1 r exptime 146.284 propid NOTBASS    seeing 1.48 MJD 58523.370 object 
   ksb_190127_063512_ooi_g_ls9.fits.fz ksb_190127_063512_ooi_g_ls9-CCD1 g exptime 220.0 propid NOTBASS    seeing 2.34 MJD 58510.274 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD4 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD4 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016A-0453 seeing 1.16 MJD 57466.232 object MzLS_711760_z
   k4m_160125_084615_ooi_zd_ls9.fits.fz k4m_160125_084615_ooi_zd_ls9-CCD3 z exptime 128.103 propid 2016B-0453 seeing 0.93 MJD 57412.365 object MzLS_712210_z
   k4m_160319_053919_ooi_zd_ls9.fits.fz k4m_160319_053919_ooi_zd_ls9-CCD2 z exptime 106.106 propid 2016A-0453 seeing 1.18 MJD 57466.236 object MzLS_711306_z
   k4m_160319_053642_ooi_zd_ls9.fits.fz k4m_160319_053642_ooi_zd_ls9-CCD1 z exptime 117.103 propid 2016A-0453 seeing 1.20 MJD 57466.234 object MzLS_712657_z
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD4 z exptime 128.103 propid 2016A-0453 seeing 1.11 MJD 57466.232 object MzLS_711760_z
   k4m_160125_085700_ooi_zd_ls9.fits.fz k4m_160125_085700_ooi_zd_ls9-CCD1 z exptime 160.1 propid 2016B-0453 seeing 1.05 MJD 57412.373 object MzLS_712211_z
WARNING: EXPNUM None!=160125084615 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_084615_ooi_zd_ls9.fits.fz header
WARNING: EXPNUM None!=160125085700 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_085700_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
90prime get_fwhm: self.fwhm = 3.71
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.45
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
mosaic get_fwhm: self.fwhm = 4.27
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
90prime get_fwhm: self.fwhm = 5.17
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
90prime get_fwhm: self.fwhm = 3.27
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
90prime get_fwhm: self.fwhm = 3.49
mosaic get_fwhm: self.fwhm = 4.61
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2787; please check the logfile.
Finished galaxy NGC2787 in 0.080 minutes.
Started working on galaxy NGC2787 at Sun Feb  9 14:12:29 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T14:12:30.110790
legacypipe git version: dr9e-pre1-8-gdd90f8de
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '139.82681100000002', '69.2033065', '--width', '2225', '--height', '2225', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 139.82681100000002 69.2033065
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 1717 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 1718 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 1719 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 1720 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 1721 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 1722 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 1723 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 1724 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 1725 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 1726 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 1727 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 1728 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 1729 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 1730 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 1731 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 1732 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 1733 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 1734 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB
Starting process 1735 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB
Starting process 1736 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071908 MB
Starting process 1737 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071904 MB
Starting process 1738 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071544 MB
Starting process 1739 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB
Starting process 1740 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB
Starting process 1741 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB
Starting process 1742 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB
Starting process 1743 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071536 MB
Starting process 1744 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071472 MB
Starting process 1745 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071472 MB
Starting process 1746 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071472 MB
Starting process 1747 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071476 MB
Starting process 1748 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071476 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T14:12:33.206231
Keeping 16 CCDs:
   ksb_190117_102440_ooi_r_ls9.fits.fz ksb_190117_102440_ooi_r_ls9-CCD3 r exptime 250.0 propid NOTBASS    seeing 1.82 MJD 58500.434 object 
   ksb_190111_104852_ooi_g_ls9.fits.fz ksb_190111_104852_ooi_g_ls9-CCD3 g exptime 198.403 propid NOTBASS    seeing 1.68 MJD 58494.451 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD3 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD3 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   ksb_190128_060634_ooi_g_ls9.fits.fz ksb_190128_060634_ooi_g_ls9-CCD4 g exptime 155.187 propid NOTBASS    seeing 1.43 MJD 58511.255 object 
   ksb_190209_085835_ooi_r_ls9.fits.fz ksb_190209_085835_ooi_r_ls9-CCD4 r exptime 152.5 propid NOTBASS    seeing 1.58 MJD 58523.374 object 
   ksb_190209_085247_ooi_r_ls9.fits.fz ksb_190209_085247_ooi_r_ls9-CCD1 r exptime 146.284 propid NOTBASS    seeing 1.48 MJD 58523.370 object 
   ksb_190127_063512_ooi_g_ls9.fits.fz ksb_190127_063512_ooi_g_ls9-CCD1 g exptime 220.0 propid NOTBASS    seeing 2.34 MJD 58510.274 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD4 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD4 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016A-0453 seeing 1.16 MJD 57466.232 object MzLS_711760_z
   k4m_160125_084615_ooi_zd_ls9.fits.fz k4m_160125_084615_ooi_zd_ls9-CCD3 z exptime 128.103 propid 2016B-0453 seeing 0.93 MJD 57412.365 object MzLS_712210_z
   k4m_160319_053919_ooi_zd_ls9.fits.fz k4m_160319_053919_ooi_zd_ls9-CCD2 z exptime 106.106 propid 2016A-0453 seeing 1.18 MJD 57466.236 object MzLS_711306_z
   k4m_160319_053642_ooi_zd_ls9.fits.fz k4m_160319_053642_ooi_zd_ls9-CCD1 z exptime 117.103 propid 2016A-0453 seeing 1.20 MJD 57466.234 object MzLS_712657_z
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD4 z exptime 128.103 propid 2016A-0453 seeing 1.11 MJD 57466.232 object MzLS_711760_z
   k4m_160125_085700_ooi_zd_ls9.fits.fz k4m_160125_085700_ooi_zd_ls9-CCD1 z exptime 160.1 propid 2016B-0453 seeing 1.05 MJD 57412.373 object MzLS_712211_z
WARNING: EXPNUM None!=160125085700 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_085700_ooi_zd_ls9.fits.fz header
WARNING: EXPNUM None!=160125084615 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_084615_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
90prime get_fwhm: self.fwhm = 4.01
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
mosaic get_fwhm: self.fwhm = 4.27
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
90prime get_fwhm: self.fwhm = 3.71
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.45
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
Faked up EXPNUM 85100117
90prime get_fwhm: self.fwhm = 5.17
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
90prime get_fwhm: self.fwhm = 3.15
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
90prime get_fwhm: self.fwhm = 3.49
mosaic get_fwhm: self.fwhm = 4.45
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.43
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2787; please check the logfile.
Finished galaxy NGC2787 in 0.120 minutes.
Started working on galaxy NGC2787 at Sun Feb  9 14:45:19 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T14:45:20.112453
legacypipe git version: dr9e-pre1-8-gdd90f8de
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '139.82681100000002', '69.2033065', '--width', '2225', '--height', '2225', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 139.82681100000002 69.2033065
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 50734 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 50735 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 50736 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 50737 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 50738 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 50739 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 50740 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 50741 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 50742 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 50743 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 50744 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 50745 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 50746 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 50747 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 50748 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 50749 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 50750 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071156 MB
Starting process 50751 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072168 MB
Starting process 50752 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072168 MB
Starting process 50753 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072164 MB
Starting process 50754 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072164 MB
Starting process 50755 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071152 MB
Starting process 50756 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50757 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50758 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50759 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50760 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50761 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50762 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50763 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50764 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 50765 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071512 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T14:45:21.464935
Keeping 16 CCDs:
   ksb_190117_102440_ooi_r_ls9.fits.fz ksb_190117_102440_ooi_r_ls9-CCD3 r exptime 250.0 propid NOTBASS    seeing 1.82 MJD 58500.434 object 
   ksb_190111_104852_ooi_g_ls9.fits.fz ksb_190111_104852_ooi_g_ls9-CCD3 g exptime 198.403 propid NOTBASS    seeing 1.68 MJD 58494.451 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD3 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD3 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   ksb_190128_060634_ooi_g_ls9.fits.fz ksb_190128_060634_ooi_g_ls9-CCD4 g exptime 155.187 propid NOTBASS    seeing 1.43 MJD 58511.255 object 
   ksb_190209_085835_ooi_r_ls9.fits.fz ksb_190209_085835_ooi_r_ls9-CCD4 r exptime 152.5 propid NOTBASS    seeing 1.58 MJD 58523.374 object 
   ksb_190209_085247_ooi_r_ls9.fits.fz ksb_190209_085247_ooi_r_ls9-CCD1 r exptime 146.284 propid NOTBASS    seeing 1.48 MJD 58523.370 object 
   ksb_190127_063512_ooi_g_ls9.fits.fz ksb_190127_063512_ooi_g_ls9-CCD1 g exptime 220.0 propid NOTBASS    seeing 2.34 MJD 58510.274 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD4 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD4 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016A-0453 seeing 1.16 MJD 57466.232 object MzLS_711760_z
   k4m_160125_084615_ooi_zd_ls9.fits.fz k4m_160125_084615_ooi_zd_ls9-CCD3 z exptime 128.103 propid 2016B-0453 seeing 0.93 MJD 57412.365 object MzLS_712210_z
   k4m_160319_053919_ooi_zd_ls9.fits.fz k4m_160319_053919_ooi_zd_ls9-CCD2 z exptime 106.106 propid 2016A-0453 seeing 1.18 MJD 57466.236 object MzLS_711306_z
   k4m_160319_053642_ooi_zd_ls9.fits.fz k4m_160319_053642_ooi_zd_ls9-CCD1 z exptime 117.103 propid 2016A-0453 seeing 1.20 MJD 57466.234 object MzLS_712657_z
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD4 z exptime 128.103 propid 2016A-0453 seeing 1.11 MJD 57466.232 object MzLS_711760_z
   k4m_160125_085700_ooi_zd_ls9.fits.fz k4m_160125_085700_ooi_zd_ls9-CCD1 z exptime 160.1 propid 2016B-0453 seeing 1.05 MJD 57412.373 object MzLS_712211_z
WARNING: EXPNUM None!=160125085700 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_085700_ooi_zd_ls9.fits.fz header
WARNING: EXPNUM None!=160125084615 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_084615_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 4.27
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
90prime get_fwhm: self.fwhm = 3.71
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.45
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
90prime get_fwhm: self.fwhm = 3.15
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
mosaic get_fwhm: self.fwhm = 4.61
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
90prime get_fwhm: self.fwhm = 3.49
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.43
mosaic get_fwhm: self.fwhm = 4.55
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
90prime get_fwhm: self.fwhm = 3.27
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2787; please check the logfile.
Finished galaxy NGC2787 in 0.087 minutes.
Started working on galaxy NGC2787 at Mon Feb 10 06:18:23 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-10T06:18:23.577531
legacypipe git version: dr9e-pre1-9-g4a51be93
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '139.82681100000002', '69.2033065', '--width', '2225', '--height', '2225', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 139.82681100000002 69.2033065 --width 2225 --height 2225 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/139/NGC2787/NGC2787-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 139.82681100000002 69.2033065
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 15865 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15866 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15867 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15868 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15869 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07112 MB
Starting process 15870 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 15871 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071128 MB
Starting process 15872 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15873 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15874 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 15875 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 15876 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 15877 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15878 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15879 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15880 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07114 MB
Starting process 15881 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 15882 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072156 MB
Starting process 15883 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072156 MB
Starting process 15884 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071864 MB
Starting process 15885 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.07186 MB
Starting process 15886 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15887 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15888 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15889 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15890 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15891 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15892 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071436 MB
Starting process 15893 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15894 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15895 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 15896 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-10T06:18:24.816682
Keeping 16 CCDs:
   ksb_190117_102440_ooi_r_ls9.fits.fz ksb_190117_102440_ooi_r_ls9-CCD3 r exptime 250.0 propid NOTBASS    seeing 1.82 MJD 58500.434 object 
   ksb_190111_104852_ooi_g_ls9.fits.fz ksb_190111_104852_ooi_g_ls9-CCD3 g exptime 198.403 propid NOTBASS    seeing 1.68 MJD 58494.451 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD3 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD3 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   ksb_190128_060634_ooi_g_ls9.fits.fz ksb_190128_060634_ooi_g_ls9-CCD4 g exptime 155.187 propid NOTBASS    seeing 1.43 MJD 58511.255 object 
   ksb_190209_085835_ooi_r_ls9.fits.fz ksb_190209_085835_ooi_r_ls9-CCD4 r exptime 152.5 propid NOTBASS    seeing 1.58 MJD 58523.374 object 
   ksb_190209_085247_ooi_r_ls9.fits.fz ksb_190209_085247_ooi_r_ls9-CCD1 r exptime 146.284 propid NOTBASS    seeing 1.48 MJD 58523.370 object 
   ksb_190127_063512_ooi_g_ls9.fits.fz ksb_190127_063512_ooi_g_ls9-CCD1 g exptime 220.0 propid NOTBASS    seeing 2.34 MJD 58510.274 object 
   ksb_180411_060955_ooi_g_ls9.fits.fz ksb_180411_060955_ooi_g_ls9-CCD4 g exptime 193.2 propid BASS       seeing 1.56 MJD 58219.257 object 45062
   ksb_190131_111024_ooi_r_ls9.fits.fz ksb_190131_111024_ooi_r_ls9-CCD4 r exptime 134.16 propid NOTBASS    seeing 1.49 MJD 58514.466 object 
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD2 z exptime 128.103 propid 2016A-0453 seeing 1.16 MJD 57466.232 object MzLS_711760_z
   k4m_160125_084615_ooi_zd_ls9.fits.fz k4m_160125_084615_ooi_zd_ls9-CCD3 z exptime 128.103 propid 2016B-0453 seeing 0.93 MJD 57412.365 object MzLS_712210_z
   k4m_160319_053919_ooi_zd_ls9.fits.fz k4m_160319_053919_ooi_zd_ls9-CCD2 z exptime 106.106 propid 2016A-0453 seeing 1.18 MJD 57466.236 object MzLS_711306_z
   k4m_160319_053642_ooi_zd_ls9.fits.fz k4m_160319_053642_ooi_zd_ls9-CCD1 z exptime 117.103 propid 2016A-0453 seeing 1.20 MJD 57466.234 object MzLS_712657_z
   k4m_160319_053354_ooi_zd_ls9.fits.fz k4m_160319_053354_ooi_zd_ls9-CCD4 z exptime 128.103 propid 2016A-0453 seeing 1.11 MJD 57466.232 object MzLS_711760_z
   k4m_160125_085700_ooi_zd_ls9.fits.fz k4m_160125_085700_ooi_zd_ls9-CCD1 z exptime 160.1 propid 2016B-0453 seeing 1.05 MJD 57412.373 object MzLS_712211_z
WARNING: EXPNUM None!=160125085700 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_085700_ooi_zd_ls9.fits.fz header
WARNING: EXPNUM None!=160125084615 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_084615_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 4.27
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
Faked up EXPNUM 84940070
90prime get_fwhm: self.fwhm = 3.71
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
Faked up EXPNUM 85000182
90prime get_fwhm: self.fwhm = 4.01
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
Faked up EXPNUM 82190081
90prime get_fwhm: self.fwhm = 3.45
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
Faked up EXPNUM 85110134
90prime get_fwhm: self.fwhm = 3.15
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
Faked up EXPNUM 85140216
90prime get_fwhm: self.fwhm = 3.29
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
Faked up EXPNUM 85230146
90prime get_fwhm: self.fwhm = 3.49
mosaic get_fwhm: self.fwhm = 4.61
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
Faked up EXPNUM 85230144
90prime get_fwhm: self.fwhm = 3.27
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy NGC2787; please check the logfile.
Finished galaxy NGC2787 in 0.078 minutes.