Started working on galaxy PGC027461 at Sat Feb  8 08:00:52 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T08:00:53.339312
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '144.55174950000003', '68.8334167', '--width', '517', '--height', '517', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 144.55174950000003 68.8334167
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 29868 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 29869 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 29870 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 29871 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 29872 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 29873 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 29874 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 29875 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T08:00:57.286906
Keeping 13 CCDs:
   ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD4 g exptime 124.63 propid NOTBASS    seeing 1.41 MJD 58495.427 object 
   ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD4 r exptime 163.914 propid NOTBASS    seeing 1.18 MJD 58511.457 object 
   ksb_190131_112908_ooi_r_ls9.fits.fz ksb_190131_112908_ooi_r_ls9-CCD2 r exptime 117.431 propid NOTBASS    seeing 1.41 MJD 58514.479 object 
   ksb_180411_065359_ooi_g_ls9.fits.fz ksb_180411_065359_ooi_g_ls9-CCD2 g exptime 220.374 propid BASS       seeing 1.59 MJD 58219.287 object 45072
   ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD3 g exptime 118.43 propid NOTBASS    seeing 1.32 MJD 58511.267 object 
   ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD3 r exptime 140.178 propid NOTBASS    seeing 1.58 MJD 58523.391 object 
   k4m_160317_060522_ooi_zd_ls9.fits.fz k4m_160317_060522_ooi_zd_ls9-CCD2 z exptime 80.104 propid 2016A-0453 seeing 1.19 MJD 57464.254 object MzLS_710389_z
   k4m_180118_054656_ooi_zd_ls9.fits.fz k4m_180118_054656_ooi_zd_ls9-CCD4 z exptime 120.103 propid 2016A-0453 seeing 1.38 MJD 58136.241 object MzLS_710851_z
   k4m_160125_091118_ooi_zd_ls9.fits.fz k4m_160125_091118_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 1.04 MJD 57412.383 object MzLS_710851_z
   k4m_170422_043305_ooi_zd_ls9.fits.fz k4m_170422_043305_ooi_zd_ls9-CCD1 z exptime 246.105 propid 2016A-0453 seeing 1.32 MJD 57865.190 object MzLS_711763_z
   k4m_170403_060320_ooi_zd_ls9.fits.fz k4m_170403_060320_ooi_zd_ls9-CCD1 z exptime 100.105 propid 2016A-0453 seeing 0.90 MJD 57846.252 object MzLS_711309_z
   k4m_170403_050712_ooi_zd_ls9.fits.fz k4m_170403_050712_ooi_zd_ls9-CCD4 z exptime 106.105 propid 2016A-0453 seeing 1.20 MJD 57846.213 object MzLS_710388_z
   k4m_160317_055914_ooi_zd_ls9.fits.fz k4m_160317_055914_ooi_zd_ls9-CCD4 z exptime 80.104 propid 2016A-0453 seeing 1.20 MJD 57464.249 object MzLS_710388_z
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
90prime get_fwhm: self.fwhm = 2.61
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
90prime get_fwhm: self.fwhm = 3.11
mosaic get_fwhm: self.fwhm = 5.31
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
90prime get_fwhm: self.fwhm = 3.51
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy PGC027461; please check the logfile.
Finished galaxy PGC027461 in 0.126 minutes.
Started working on galaxy PGC027461 at Sat Feb  8 12:50:06 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-08T12:50:06.493803
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '144.55174950000003', '68.8334167', '--width', '517', '--height', '517', '--pixscale', '0.262', '--threads', '8', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 8 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 144.55174950000003 68.8334167
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 64033 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 64034 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 64035 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 64036 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 64037 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 64038 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 64039 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 64040 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-08T12:50:07.730127
Keeping 13 CCDs:
   ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD4 g exptime 124.63 propid NOTBASS    seeing 1.41 MJD 58495.427 object 
   ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD4 r exptime 163.914 propid NOTBASS    seeing 1.18 MJD 58511.457 object 
   ksb_190131_112908_ooi_r_ls9.fits.fz ksb_190131_112908_ooi_r_ls9-CCD2 r exptime 117.431 propid NOTBASS    seeing 1.41 MJD 58514.479 object 
   ksb_180411_065359_ooi_g_ls9.fits.fz ksb_180411_065359_ooi_g_ls9-CCD2 g exptime 220.374 propid BASS       seeing 1.59 MJD 58219.287 object 45072
   ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD3 g exptime 118.43 propid NOTBASS    seeing 1.32 MJD 58511.267 object 
   ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD3 r exptime 140.178 propid NOTBASS    seeing 1.58 MJD 58523.391 object 
   k4m_160317_060522_ooi_zd_ls9.fits.fz k4m_160317_060522_ooi_zd_ls9-CCD2 z exptime 80.104 propid 2016A-0453 seeing 1.19 MJD 57464.254 object MzLS_710389_z
   k4m_180118_054656_ooi_zd_ls9.fits.fz k4m_180118_054656_ooi_zd_ls9-CCD4 z exptime 120.103 propid 2016A-0453 seeing 1.38 MJD 58136.241 object MzLS_710851_z
   k4m_160125_091118_ooi_zd_ls9.fits.fz k4m_160125_091118_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 1.04 MJD 57412.383 object MzLS_710851_z
   k4m_170422_043305_ooi_zd_ls9.fits.fz k4m_170422_043305_ooi_zd_ls9-CCD1 z exptime 246.105 propid 2016A-0453 seeing 1.32 MJD 57865.190 object MzLS_711763_z
   k4m_170403_060320_ooi_zd_ls9.fits.fz k4m_170403_060320_ooi_zd_ls9-CCD1 z exptime 100.105 propid 2016A-0453 seeing 0.90 MJD 57846.252 object MzLS_711309_z
   k4m_170403_050712_ooi_zd_ls9.fits.fz k4m_170403_050712_ooi_zd_ls9-CCD4 z exptime 106.105 propid 2016A-0453 seeing 1.20 MJD 57846.213 object MzLS_710388_z
   k4m_160317_055914_ooi_zd_ls9.fits.fz k4m_160317_055914_ooi_zd_ls9-CCD4 z exptime 80.104 propid 2016A-0453 seeing 1.20 MJD 57464.249 object MzLS_710388_z
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
90prime get_fwhm: self.fwhm = 2.61
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
90prime get_fwhm: self.fwhm = 3.11
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
90prime get_fwhm: self.fwhm = 3.51
mosaic get_fwhm: self.fwhm = 5.31
mosaic get_fwhm: self.fwhm = 4.57
WARNING: EXPNUM None!=160125091118 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_091118_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 5.07
Swapping in SKY_JOHN values for 1 splinesky cells; 111.556076 -> 111.52908
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy PGC027461; please check the logfile.
Finished galaxy PGC027461 in 0.066 minutes.
Started working on galaxy PGC027461 at Sun Feb  9 08:13:28 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T08:13:28.770472
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '144.55174950000003', '68.8334167', '--width', '517', '--height', '517', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 144.55174950000003 68.8334167
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 27661 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 27662 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 27663 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 27664 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 27665 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 27666 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 27667 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 27668 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 27669 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 27670 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 27671 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 27672 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 27673 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 27674 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 27675 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 27676 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 27677 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 27678 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072204 MB
Starting process 27679 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.072204 MB
Starting process 27680 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071912 MB
Starting process 27681 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071912 MB
Starting process 27682 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27683 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27684 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27685 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27686 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27687 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27688 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27689 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 27690 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Starting process 27691 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071548 MB
Starting process 27692 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071484 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T08:13:30.018929
Keeping 13 CCDs:
   ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD4 g exptime 124.63 propid NOTBASS    seeing 1.41 MJD 58495.427 object 
   ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD4 r exptime 163.914 propid NOTBASS    seeing 1.18 MJD 58511.457 object 
   ksb_190131_112908_ooi_r_ls9.fits.fz ksb_190131_112908_ooi_r_ls9-CCD2 r exptime 117.431 propid NOTBASS    seeing 1.41 MJD 58514.479 object 
   ksb_180411_065359_ooi_g_ls9.fits.fz ksb_180411_065359_ooi_g_ls9-CCD2 g exptime 220.374 propid BASS       seeing 1.59 MJD 58219.287 object 45072
   ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD3 g exptime 118.43 propid NOTBASS    seeing 1.32 MJD 58511.267 object 
   ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD3 r exptime 140.178 propid NOTBASS    seeing 1.58 MJD 58523.391 object 
   k4m_160317_060522_ooi_zd_ls9.fits.fz k4m_160317_060522_ooi_zd_ls9-CCD2 z exptime 80.104 propid 2016A-0453 seeing 1.19 MJD 57464.254 object MzLS_710389_z
   k4m_180118_054656_ooi_zd_ls9.fits.fz k4m_180118_054656_ooi_zd_ls9-CCD4 z exptime 120.103 propid 2016A-0453 seeing 1.38 MJD 58136.241 object MzLS_710851_z
   k4m_160125_091118_ooi_zd_ls9.fits.fz k4m_160125_091118_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 1.04 MJD 57412.383 object MzLS_710851_z
   k4m_170422_043305_ooi_zd_ls9.fits.fz k4m_170422_043305_ooi_zd_ls9-CCD1 z exptime 246.105 propid 2016A-0453 seeing 1.32 MJD 57865.190 object MzLS_711763_z
   k4m_170403_060320_ooi_zd_ls9.fits.fz k4m_170403_060320_ooi_zd_ls9-CCD1 z exptime 100.105 propid 2016A-0453 seeing 0.90 MJD 57846.252 object MzLS_711309_z
   k4m_170403_050712_ooi_zd_ls9.fits.fz k4m_170403_050712_ooi_zd_ls9-CCD4 z exptime 106.105 propid 2016A-0453 seeing 1.20 MJD 57846.213 object MzLS_710388_z
   k4m_160317_055914_ooi_zd_ls9.fits.fz k4m_160317_055914_ooi_zd_ls9-CCD4 z exptime 80.104 propid 2016A-0453 seeing 1.20 MJD 57464.249 object MzLS_710388_z
WARNING: EXPNUM None!=160125091118 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_091118_ooi_zd_ls9.fits.fz header
Faked up EXPNUM 85230154
Faked up EXPNUM 85230154
Faked up EXPNUM 85230154
90prime get_fwhm: self.fwhm = 3.49
mosaic get_fwhm: self.fwhm = 4.57
Faked up EXPNUM 85110140
Faked up EXPNUM 85110140
Faked up EXPNUM 85110140
90prime get_fwhm: self.fwhm = 2.91
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
90prime get_fwhm: self.fwhm = 2.61
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
90prime get_fwhm: self.fwhm = 3.11
mosaic get_fwhm: self.fwhm = 5.07
Swapping in SKY_JOHN values for 1 splinesky cells; 111.556076 -> 111.52908
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
90prime get_fwhm: self.fwhm = 3.11
mosaic get_fwhm: self.fwhm = 4.61
mosaic get_fwhm: self.fwhm = 5.31
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy PGC027461; please check the logfile.
Finished galaxy PGC027461 in 0.071 minutes.
Started working on galaxy PGC027461 at Sun Feb  9 10:04:49 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-09T10:04:49.739974
legacypipe git version: dr9e-pre1-7-g54c974a3
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '144.55174950000003', '68.8334167', '--width', '517', '--height', '517', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 144.55174950000003 68.8334167
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 39888 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 39889 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 39890 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 39891 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 39892 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071164 MB
Starting process 39893 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071168 MB
Starting process 39894 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071172 MB
Starting process 39895 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 39896 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 39897 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071176 MB
Starting process 39898 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07118 MB
Starting process 39899 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 39900 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 39901 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 39902 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 39903 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071184 MB
Starting process 39904 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071188 MB
Starting process 39905 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB
Starting process 39906 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.0722 MB
Starting process 39907 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071908 MB
Starting process 39908 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071908 MB
Starting process 39909 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 39910 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 39911 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 39912 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 39913 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07154 MB
Starting process 39914 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 39915 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 39916 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 39917 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 39918 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Starting process 39919 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07148 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-09T10:04:51.039091
Keeping 13 CCDs:
   ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD4 g exptime 124.63 propid NOTBASS    seeing 1.41 MJD 58495.427 object 
   ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD4 r exptime 163.914 propid NOTBASS    seeing 1.18 MJD 58511.457 object 
   ksb_190131_112908_ooi_r_ls9.fits.fz ksb_190131_112908_ooi_r_ls9-CCD2 r exptime 117.431 propid NOTBASS    seeing 1.41 MJD 58514.479 object 
   ksb_180411_065359_ooi_g_ls9.fits.fz ksb_180411_065359_ooi_g_ls9-CCD2 g exptime 220.374 propid BASS       seeing 1.59 MJD 58219.287 object 45072
   ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD3 g exptime 118.43 propid NOTBASS    seeing 1.32 MJD 58511.267 object 
   ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD3 r exptime 140.178 propid NOTBASS    seeing 1.58 MJD 58523.391 object 
   k4m_160317_060522_ooi_zd_ls9.fits.fz k4m_160317_060522_ooi_zd_ls9-CCD2 z exptime 80.104 propid 2016A-0453 seeing 1.19 MJD 57464.254 object MzLS_710389_z
   k4m_180118_054656_ooi_zd_ls9.fits.fz k4m_180118_054656_ooi_zd_ls9-CCD4 z exptime 120.103 propid 2016A-0453 seeing 1.38 MJD 58136.241 object MzLS_710851_z
   k4m_160125_091118_ooi_zd_ls9.fits.fz k4m_160125_091118_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 1.04 MJD 57412.383 object MzLS_710851_z
   k4m_170422_043305_ooi_zd_ls9.fits.fz k4m_170422_043305_ooi_zd_ls9-CCD1 z exptime 246.105 propid 2016A-0453 seeing 1.32 MJD 57865.190 object MzLS_711763_z
   k4m_170403_060320_ooi_zd_ls9.fits.fz k4m_170403_060320_ooi_zd_ls9-CCD1 z exptime 100.105 propid 2016A-0453 seeing 0.90 MJD 57846.252 object MzLS_711309_z
   k4m_170403_050712_ooi_zd_ls9.fits.fz k4m_170403_050712_ooi_zd_ls9-CCD4 z exptime 106.105 propid 2016A-0453 seeing 1.20 MJD 57846.213 object MzLS_710388_z
   k4m_160317_055914_ooi_zd_ls9.fits.fz k4m_160317_055914_ooi_zd_ls9-CCD4 z exptime 80.104 propid 2016A-0453 seeing 1.20 MJD 57464.249 object MzLS_710388_z
WARNING: EXPNUM None!=160125091118 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_091118_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 4.57
Faked up EXPNUM 85230154
Faked up EXPNUM 85230154
Faked up EXPNUM 85230154
90prime get_fwhm: self.fwhm = 3.49
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
90prime get_fwhm: self.fwhm = 2.61
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
90prime get_fwhm: self.fwhm = 3.11
Faked up EXPNUM 85110140
Faked up EXPNUM 85110140
Faked up EXPNUM 85110140
90prime get_fwhm: self.fwhm = 2.91
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
90prime get_fwhm: self.fwhm = 3.11
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
90prime get_fwhm: self.fwhm = 3.51
mosaic get_fwhm: self.fwhm = 5.07
Swapping in SKY_JOHN values for 1 splinesky cells; 111.556076 -> 111.52908
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy PGC027461; please check the logfile.
Finished galaxy PGC027461 in 0.060 minutes.
Started working on galaxy PGC027461 at Mon Feb 10 06:14:39 2020
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds 

runbrick.py starting at 2020-02-10T06:14:39.843213
legacypipe git version: dr9e-pre1-9-g4a51be93
Command-line args: ['/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py', '--radec', '144.55174950000003', '68.8334167', '--width', '517', '--height', '517', '--pixscale', '0.262', '--threads', '32', '--outdir', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461', '--survey-dir', '/global/cscratch1/sd/desimpp/dr9e', '--run', '90prime-mosaic', '--largegalaxy-preburner', '--write-stage', 'tims', '--write-stage', 'srcs', '--skip-calibs', '--checkpoint', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p', '--pickle', '/global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p', '--unwise-coadds']
python /global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py --radec 144.55174950000003 68.8334167 --width 517 --height 517 --pixscale 0.262 --threads 32 --outdir /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461 --survey-dir /global/cscratch1/sd/desimpp/dr9e --run 90prime-mosaic --largegalaxy-preburner --write-stage tims --write-stage srcs --skip-calibs --checkpoint /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-checkpoint.p --pickle /global/projecta/projectdirs/desi/users/ioannis/LSLGA-data-dr9e/144/PGC027461/PGC027461-runbrick-%%(stage)s.p --unwise-coadds

WARNING: no WISE sky background maps in /global/cscratch1/sd/desimpp/dr9e/calib/wise/modelsky
Parsed RA,Dec 144.55174950000003 68.8334167
WARNING: CacheMissingWarning: Not clearing data cache - cache inaccessible due to OSError: [Errno 30] Read-only file system: '/home/legacyhalos/.astropy/cache/download/py3/lock' [astropy.utils.data]
Starting process 13210 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 13211 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 13212 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 13213 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 13214 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071124 MB
Starting process 13215 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 69 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 13216 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071132 MB
Starting process 13217 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 13218 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 13219 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071136 MB
Starting process 13220 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 13221 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 13222 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 13223 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 13224 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071144 MB
Starting process 13225 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 13226 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 71 MB, VmData: 140 MB, maxrss: 0.071148 MB
Starting process 13227 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072164 MB
Starting process 13228 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072164 MB
Starting process 13229 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.072036 MB
Starting process 13231 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.0721 MB
Starting process 13232 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071508 MB
Starting process 13233 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071504 MB
Starting process 13234 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071504 MB
Starting process 13235 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071504 MB
Starting process 13236 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071504 MB
Starting process 13237 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 13238 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 13239 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 13240 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 13241 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.07144 MB
Starting process 13242 Wall: -0.00 s, CPU: -0.00 s, VmPeak: 665 MB, VmSize: 665 MB, VmRSS: 70 MB, VmData: 140 MB, maxrss: 0.071444 MB
Runstage writecat
Runstage wise_forced
Runstage coadds
Runstage fitblobs
Runstage srcs
Runstage largegalaxies
Runstage halos
Runstage outliers
Runstage refs
Runstage tims
Running stage tims
Running stage tims at 2020-02-10T06:14:41.093949
Keeping 13 CCDs:
   ksb_190112_101522_ooi_g_ls9.fits.fz ksb_190112_101522_ooi_g_ls9-CCD4 g exptime 124.63 propid NOTBASS    seeing 1.41 MJD 58495.427 object 
   ksb_190128_105804_ooi_r_ls9.fits.fz ksb_190128_105804_ooi_r_ls9-CCD4 r exptime 163.914 propid NOTBASS    seeing 1.18 MJD 58511.457 object 
   ksb_190131_112908_ooi_r_ls9.fits.fz ksb_190131_112908_ooi_r_ls9-CCD2 r exptime 117.431 propid NOTBASS    seeing 1.41 MJD 58514.479 object 
   ksb_180411_065359_ooi_g_ls9.fits.fz ksb_180411_065359_ooi_g_ls9-CCD2 g exptime 220.374 propid BASS       seeing 1.59 MJD 58219.287 object 45072
   ksb_190128_062355_ooi_g_ls9.fits.fz ksb_190128_062355_ooi_g_ls9-CCD3 g exptime 118.43 propid NOTBASS    seeing 1.32 MJD 58511.267 object 
   ksb_190209_092322_ooi_r_ls9.fits.fz ksb_190209_092322_ooi_r_ls9-CCD3 r exptime 140.178 propid NOTBASS    seeing 1.58 MJD 58523.391 object 
   k4m_160317_060522_ooi_zd_ls9.fits.fz k4m_160317_060522_ooi_zd_ls9-CCD2 z exptime 80.104 propid 2016A-0453 seeing 1.19 MJD 57464.254 object MzLS_710389_z
   k4m_180118_054656_ooi_zd_ls9.fits.fz k4m_180118_054656_ooi_zd_ls9-CCD4 z exptime 120.103 propid 2016A-0453 seeing 1.38 MJD 58136.241 object MzLS_710851_z
   k4m_160125_091118_ooi_zd_ls9.fits.fz k4m_160125_091118_ooi_zd_ls9-CCD4 z exptime 128.104 propid 2016B-0453 seeing 1.04 MJD 57412.383 object MzLS_710851_z
   k4m_170422_043305_ooi_zd_ls9.fits.fz k4m_170422_043305_ooi_zd_ls9-CCD1 z exptime 246.105 propid 2016A-0453 seeing 1.32 MJD 57865.190 object MzLS_711763_z
   k4m_170403_060320_ooi_zd_ls9.fits.fz k4m_170403_060320_ooi_zd_ls9-CCD1 z exptime 100.105 propid 2016A-0453 seeing 0.90 MJD 57846.252 object MzLS_711309_z
   k4m_170403_050712_ooi_zd_ls9.fits.fz k4m_170403_050712_ooi_zd_ls9-CCD4 z exptime 106.105 propid 2016A-0453 seeing 1.20 MJD 57846.213 object MzLS_710388_z
   k4m_160317_055914_ooi_zd_ls9.fits.fz k4m_160317_055914_ooi_zd_ls9-CCD4 z exptime 80.104 propid 2016A-0453 seeing 1.20 MJD 57464.249 object MzLS_710388_z
WARNING: EXPNUM None!=160125091118 in /global/cscratch1/sd/desimpp/dr9e/images/mosaic/CP/V4.3/CP20160124/k4m_160125_091118_ooi_zd_ls9.fits.fz header
mosaic get_fwhm: self.fwhm = 4.57
Faked up EXPNUM 85110140
Faked up EXPNUM 85110140
Faked up EXPNUM 85110140
90prime get_fwhm: self.fwhm = 2.91
Faked up EXPNUM 85230154
Faked up EXPNUM 85230154
Faked up EXPNUM 85230154
90prime get_fwhm: self.fwhm = 3.49
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
Faked up EXPNUM 85140222
90prime get_fwhm: self.fwhm = 3.11
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
Faked up EXPNUM 84950057
90prime get_fwhm: self.fwhm = 3.11
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
Faked up EXPNUM 85110243
90prime get_fwhm: self.fwhm = 2.61
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
Faked up EXPNUM 82190093
90prime get_fwhm: self.fwhm = 3.51
mosaic get_fwhm: self.fwhm = 5.31
Traceback (most recent call last):
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3605, in <module>
    sys.exit(main())
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3575, in main
    run_brick(opt.brick, survey, **kwargs)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3161, in run_brick
    initial_args=initargs, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 113, in runstage
    write=write, initial_args=initial_args, **kwargs)
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python/astrometry/util/stages.py", line 124, in runstage
    R = stagefunc(stage, **Px)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 3150, in mystagefunc
    R = stagefunc(stage, mp=mp, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 40, in __call__
    rtn = super(CallGlobalTime, self).__call__(stage, **kwargs)
  File "/usr/local/lib/python/astrometry/util/stages.py", line 32, in __call__
    return func(*self.args, **kwa)
  File "/global/homes/i/ioannis/repos/git/legacypipe/py/legacypipe/runbrick.py", line 271, in stage_tims
    tims = list(mp.map(read_one_tim, args))
  File "/usr/local/lib/python/astrometry/util/multiproc.py", line 89, in map
    return self.pool.map(f, args, cs)
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AssertionError
Something went wrong; please check the logfile.
ERROR: galaxy PGC027461; please check the logfile.
Finished galaxy PGC027461 in 0.065 minutes.