unwise_coadd.py starting: args: ['/global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py', '--tile', '1294m621', '--band', '1', '--outdir', '/global/cscratch1/sd/ameisner/unwise-coadds/tr_neo2/e000', '--recover_warped', '--no_irsa_dl', '--reference_dir', '/global/cscratch1/sd/ameisner/unwise-coadds/fulldepth_neo2_ref_w1', '--epoch', '0', '--warp_all', '--no_output_masks'] PBS_ARRAYID: 0 opt: {'threads1': None, 'reference_dir': '/global/cscratch1/sd/ameisner/unwise-coadds/fulldepth_neo2_ref_w1', 'plotprefix': None, 'force': False, 'no_sanity_check': False, 'height': 0, 'hi_lo_rej': False, 'medfilt': None, 'minmax': None, 'size': 2048, 'plots2': False, 'cube1': False, 'bgmatch': False, 'pixscale': 2.75, 'int_gz': False, 'nframes': 0, 'width': 0, 'epoch': 0, 'ra': None, 'dsky': False, 'preprocess': False, 'try_download': False, 'before': 100000.0, 'do_rebin': True, 'md5': False, 'dataset': 'sequels', 'cube': False, 'after': None, 'tile': '1294m621', 'threads': None, 'allmd5': False, 'rchi_fraction': 0.01, 'plots': False, 'frame0': 0, 'wishlist': False, 'maxmem': 0, 'compare_moon_all': False, 'center': False, 'band': [1], 'output_masks': False, 'use_zp_meta': False, 'outdir': '/global/cscratch1/sd/ameisner/unwise-coadds/tr_neo2/e000', 'pdf': False, 'warp_all': True, 'dec': None, 'outdir2': None, 'recover_warped': True} ['__cmp__', '__doc__', '__init__', '__module__', '__repr__', '__str__', '_update', '_update_careful', '_update_loose', 'after', 'allmd5', 'band', 'before', 'bgmatch', 'center', 'compare_moon_all', 'cube', 'cube1', 'dataset', 'dec', 'do_rebin', 'dsky', 'ensure_value', 'epoch', 'force', 'frame0', 'height', 'hi_lo_rej', 'int_gz', 'maxmem', 'md5', 'medfilt', 'minmax', 'nframes', 'no_sanity_check', 'outdir', 'outdir2', 'output_masks', 'pdf', 'pixscale', 'plotprefix', 'plots', 'plots2', 'preprocess', 'ra', 'rchi_fraction', 'read_file', 'read_module', 'recover_warped', 'reference_dir', 'size', 'threads', 'threads1', 'tile', 'try_download', 'use_zp_meta', 'warp_all', 'width', 'wishlist'] Running on host: nid02236 Running as user: ameisner MKL_NUM_THREADS: 1 Tile RA,Dec 129.4 -62.1 Looking for file 1294m621-atlas.fits Reading /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/wise_allsky_4band_p3as_cdd.fits Read 18240 Atlas tiles cosdec: 0.467914389569 Cut to 1 Atlas tiles near RA,Dec box Wrote 1294m621-atlas.fits Read 7904572 WISE L1b frames from /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-index-L1b_w1.fits /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_utils.py:268: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future dangle = degrees_between(ra[ind[0]:ind[1]], dec[ind[0]:ind[1]], racen, deccen) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_utils.py:270: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future nearby[ind[0]:ind[1]] = (dangle <= margin) figuring out which frames are nearby took 1.35676193237 seconds Cut to 3617 WISE frames near RA,Dec box Reading /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-4band.fits Read 1491686 from /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-4band.fits Cut to 2324 near RA,Dec box Matched 778 WISE-index-L1b scan_id: |S6 frame_num: >i8 WISE-metadata scan_id: |S6 frame_num: >i8 Cut to 590 matching scan/frame Band 1 : 590 Band 2 : 0 Band 3 : 0 Band 4 : 0 Reading /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-3band.fits Read 392879 from /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-3band.fits Cut to 0 near RA,Dec box Reading /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-2band.fits Read 901271 from /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-2band.fits Cut to 2726 near RA,Dec box Matched 936 WISE-index-L1b scan_id: |S6 frame_num: >i8 WISE-metadata scan_id: |S6 frame_num: >i8 Cut to 751 matching scan/frame Band 1 : 751 Band 2 : 0 Reading /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-neowiser.fits Read 2497867 from /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-neowiser.fits Cut to 4667 near RA,Dec box Matched 1525 WISE-index-L1b scan_id: |S6 frame_num: >i8 WISE-metadata scan_id: |S6 frame_num: >i2 Cut to 1197 matching scan/frame Band 1 : 1197 Band 2 : 0 Reading /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-neowiser2.fits Read 2620896 from /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/WISE-l1b-metadata-neowiser2.fits Cut to 4593 near RA,Dec box Matched 1411 WISE-index-L1b scan_id: |S6 frame_num: >i8 WISE-metadata scan_id: |S6 frame_num: >i4 Cut to 1079 matching scan/frame Band 1 : 1079 Band 2 : 0 3617 of 3617 matched to metadata tables Doing coadd tile 1294m621 band 1 Coadd tile 1294m621 RA,Dec 129.473546 -62.092222 Band 1 "git describe" version info: v3.5-15-gdebad9c will attempt to recover frames using per-quadrant polynomial warps ... mkdir /global/cscratch1/sd/ameisner/unwise-coadds/tr_neo2/e000/129/1294m621 Coadd IWC polygon: [[ 7.89479167e-01 0.00000000e+00] [ 7.89479167e-01 7.89479167e-01] [ 1.27234264e-14 7.89479167e-01] [ -7.89479167e-01 7.89479167e-01] [ -7.89479167e-01 -3.18085660e-15] [ -7.89479167e-01 -7.89479167e-01] [ 1.27234264e-14 -7.89479167e-01] [ 7.89479167e-01 -7.89479167e-01]] Found 3617 WISE frames in range and in band W1 cut to 2862 in RA,Dec box Defined epoch breaks [55281.933829524998, 55461.924639584999, 56105.658449484996, 56742.830496215, 56925.225719645001, 57104.363201775006] Found 6 epoch breaks Cut to 2 within epoch Cut to 2 frames before MJD 100000.0 Cut out qual_frame = 0; 2 remaining Cut out planets != 0; 2 remaining Frames: 0 00716a 144 MJD 55203.2335262 1 00716a 145 MJD 55203.2336536 Reading L1b top-level directory locations from /global/cscratch1/sd/ameisner/code/unwise-coadds/etc_p9/l1b_dirs.yml L1b top-level directories are: {'2band': '/global/cscratch1/sd/desiproc/run_unwise/staging/2band_p1bm_frm', '3band': '/global/cscratch1/sd/desiproc/run_unwise/staging/3band_p1bm_frm', '4band': '/global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm', 'missing': 'merge_p1bm_frm', 'neo1': '/global/cscratch1/sd/desiproc/run_unwise/staging/p1bm_frm', 'neo2': '/global/cscratch1/sd/desiproc/run_unwise/staging/p1bm_frm'} 1 of 2 scan 00716a frame 144 band 1 intfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/144/00716a144-w1-int-1b.fits XTENSION: '(null)' Image extent: [1001 1015 88 144] Coadd extent: [ 0 39 0 16] Quadrant 1 does not intersect target Quadrant 2 does not intersect target Quadrant 3 does not intersect target Total pixels in coadd space: 680.0 2 of 2 scan 00716a frame 145 band 1 intfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/145/00716a145-w1-int-1b.fits XTENSION: '(null)' Image extent: [1011 1015 999 1015] Coadd extent: [15 46 0 2] Quadrant 2 does not intersect target Quadrant 3 does not intersect target Quadrant 4 does not intersect target Total pixels in coadd space: 776.0 Cut to 2 frames intersecting target Up to coadd_wise: Wall: 1.64 s, CPU: 0.06 s, VmPeak: 845 MB, VmSize: 373 MB, VmRSS: 75 MB, VmData: 59 MB, maxrss: 0.560268 MB Creating reference image from files: /global/cscratch1/sd/ameisner/unwise-coadds/fulldepth_neo2_ref_w1/129/1294m621/unwise-1294m621-w1-img-u.fits /global/cscratch1/sd/ameisner/unwise-coadds/fulldepth_neo2_ref_w1/129/1294m621/unwise-1294m621-w1-std-u.fits /global/cscratch1/sd/ameisner/unwise-coadds/fulldepth_neo2_ref_w1/129/1294m621/unwise-1294m621-w1-n-u.fits /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:2575: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future coimg = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:2576: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future coimgsq = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:2577: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future cow = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:2578: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future con1 = np.zeros((H,W)) Reading zero point polynomial coefficients for phase : 3band, W1 Reading zero point polynomial coefficients for phase : 2band, W1 Reading zero point polynomial coefficients for phase : neowiser, W1 Reading zero point polynomial coefficients for phase : neowiser2, W1 Warping all quadrants relative to reference image Creating per-quadrant FirstRoundImage objects Coadd round 1, image 1 of 2 intfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/144/00716a144-w1-int-1b.fits uncfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/144/00716a144-w1-unc-1b.fits.gz maskfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/144/00716a144-w1-msk-1b.fits.gz Zeropoint: 20.752 -> scale 5.00264862421 sig1: 3.82202148438 Pixels to patch: 11 Estimated sky: 14.5447985012 Image median: 15.1408 Image median w/ noise: 15.5141 Pixels in range: 1 Wall: 0.35 s, CPU: 0.30 s, VmPeak: 845 MB, VmSize: 603 MB, VmRSS: 143 MB, VmData: 254 MB, maxrss: 0.560268 MB Skipping quadrant 1 Skipping quadrant 2 Skipping quadrant 3 Too few pixels : 1, not computing warp deleting x, y coordinates for quadrant 4 deleting x, y coordinates for quadrant -1 Coadd round 1, image 2 of 2 intfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/145/00716a145-w1-int-1b.fits uncfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/145/00716a145-w1-unc-1b.fits.gz maskfn /global/cscratch1/sd/desiproc/run_unwise/staging/4band_p1bm_frm/6a/00716a/145/00716a145-w1-msk-1b.fits.gz Zeropoint: 20.752 -> scale 5.00264862421 sig1: 3.80908203125 Pixels to patch: 1 Estimated sky: 14.0930797566 Image median: 14.5889 Image median w/ noise: 14.904 No overlap; skipping Accumulating first-round coadds... Wall: 0.00 s, CPU: 0.00 s, VmPeak: 845 MB, VmSize: 603 MB, VmRSS: 143 MB, VmData: 254 MB, maxrss: 0.560268 MB Before garbage collection: Wall: 0.00 s, CPU: 0.00 s, VmPeak: 845 MB, VmSize: 667 MB, VmRSS: 240 MB, VmData: 318 MB, maxrss: 0.560268 MB After garbage collection: Wall: 0.01 s, CPU: 0.01 s, VmPeak: 845 MB, VmSize: 667 MB, VmRSS: 240 MB, VmData: 318 MB, maxrss: 0.560268 MB /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1746: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.coimg = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1747: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.coimgsq = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1748: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.cow = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1749: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.con = np.zeros((H,W), np.int16) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1750: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.coimgb = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1751: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.coimgsqb = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1752: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.cowb = np.zeros((H,W)) /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1753: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future self.conb = np.zeros((H,W), np.int16) Coadd round 2, image 1 of 1 /global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py:1593: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future badpixmask.dtype) Wall: 0.00 s, CPU: 0.00 s, VmPeak: 859 MB, VmSize: 859 MB, VmRSS: 240 MB, VmData: 510 MB, maxrss: 0.560268 MB Before garbage collection: Wall: 0.00 s, CPU: 0.00 s, VmPeak: 859 MB, VmSize: 859 MB, VmRSS: 240 MB, VmData: 510 MB, maxrss: 0.560268 MB After garbage collection: Wall: 0.01 s, CPU: 0.01 s, VmPeak: 859 MB, VmSize: 859 MB, VmRSS: 240 MB, VmData: 510 MB, maxrss: 0.560268 MB /global/cscratch1/sd/ameisner/code/astrometry.net-0.64/astrometry/util/miscutils.py:48: RuntimeWarning: invalid value encountered in divide x = (bincenters - x0) / xscale coadd_wise failed: Traceback (most recent call last): File "/global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py", line 1123, in one_coadd warp_all=warp_all, reference_dir=reference_dir, hi_lo_rej=hi_lo_rej) File "/global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py", line 2294, in coadd_wise coimg, coimgb, sky = subtract_coadd_sky(coimg, coimgb, con) File "/global/cscratch1/sd/ameisner/code/unwise-coadds/unwise_coadd.py", line 2339, in subtract_coadd_sky sky = estimate_mode(coimgb[con != 0]) # ignore zero coverage regions File "/global/cscratch1/sd/ameisner/code/astrometry.net-0.64/astrometry/util/miscutils.py", line 54, in estimate_mode res = np.linalg.lstsq(A, b) File "/global/common/cori/software/python/2.7-anaconda/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 1919, in lstsq 0, work, lwork, iwork, 0) ValueError: On entry to DGELSD parameter number 4 had an illegal value time up to failure: Wall: 2.01 s, CPU: 1.41 s, VmPeak: 995 MB, VmSize: 931 MB, VmRSS: 376 MB, VmData: 582 MB, maxrss: 0.560268 MB Tile 1294m621 band 1 took: Wall: 4.57 s, CPU: 2.29 s, VmPeak: 995 MB, VmSize: 435 MB, VmRSS: 103 MB, VmData: 86 MB, maxrss: 0.560268 MB