E4S 21.02 - Cori¶
The E4S 21.02 stack is built using e4s-21.02 tag release in spack that is built for Haswell architecture. The spack configuration can be found at https://software.nersc.gov/NERSC/e4s-2102.
In order to access this stack you need to load the following module:
module load e4s/21.02
This module will setup a source spack in your user shell and provide the spack command in your $PATH.
Overview¶
This release comes with 45 E4S products, with most packages built with intel@19.1.2.254 compiler and a subset of packages were installed using gcc@10.1.0. Some of these packages were installed with gcc because spack could not concretize or install the package with intel compiler.
Shown below is the list of installed packages provided by spack.
$ spack find --explicit
==> In environment e4s
==> Root specs
-- no arch / gcc@10.1.0 -----------------------------------------
darshan-runtime@3.2.1%gcc@10.1.0 +slurm dyninst@10.2.1%gcc@10.1.0 plasma@20.9.20%gcc@10.1.0
darshan-util@3.2.1%gcc@10.1.0 +bzip2 legion@20.03.0%gcc@10.1.0 slate@2020.10.00%gcc@10.1.0 ~cuda
-- no arch / intel@19.1.2.254 -----------------------------------
adios2@2.7.1%intel@19.1.2.254 +hdf5
aml@0.1.0%intel@19.1.2.254
arborx@0.9-beta%intel@19.1.2.254 +openmp
bolt@2.0%intel@19.1.2.254
caliper@2.5.0%intel@19.1.2.254 +fortran
faodel@1.1906.1%intel@19.1.2.254
flecsi@1.4%intel@19.1.2.254 +caliper+cinch+graphviz+tutorial
flit@2.1.0%intel@19.1.2.254
gasnet@2020.3.0%intel@19.1.2.254 +udp
ginkgo@1.3.0%intel@19.1.2.254
gotcha@1.0.3%intel@19.1.2.254 +test
hdf5@1.10.7%intel@19.1.2.254
hypre@2.20.0%intel@19.1.2.254 +mixedint+openmp+superlu-dist
libnrm@0.1.0%intel@19.1.2.254
libquo@1.3.1%intel@19.1.2.254
mercury@2.0.0%intel@19.1.2.254 +udreg
mfem@4.2.0%intel@19.1.2.254 +examples+gnutls+gslib+lapack+libunwind+openmp+pumi+threadsafe+umpire
ninja@1.10.2%intel@19.1.2.254
omega-h@9.32.5%intel@19.1.2.254 ~trilinos
openpmd-api@0.13.2%intel@19.1.2.254
papi@6.0.0.1%intel@19.1.2.254 +example+infiniband+powercap+static_tools
papyrus@1.0.1%intel@19.1.2.254
pdt@3.25.1%intel@19.1.2.254 +pic
precice@2.2.0%intel@19.1.2.254 +python
pumi@2.2.5%intel@19.1.2.254 +fortran
qthreads@1.16%intel@19.1.2.254 ~hwloc
raja@0.13.0%intel@19.1.2.254 +tests
slepc@3.14.2%intel@19.1.2.254
strumpack@5.1.1%intel@19.1.2.254 +shared
sundials@5.7.0%intel@19.1.2.254 +examples-cxx+hypre+klu+lapack
superlu@5.2.1%intel@19.1.2.254
superlu-dist@6.4.0%intel@19.1.2.254 +openmp
swig@4.0.2-fortran%intel@19.1.2.254
tasmanian@7.3%intel@19.1.2.254 +blas+fortran+mpi+python+xsdkflags
tau@2.30.1%intel@19.1.2.254 +mpi~pdt
turbine@1.2.3%intel@19.1.2.254 +hdf5+python
umap@2.1.0%intel@19.1.2.254 +tests
umpire@4.1.2%intel@19.1.2.254 +fortran+numa+openmp
upcxx@2020.10.0%intel@19.1.2.254
zfp@0.5.5%intel@19.1.2.254 +aligned+c+fortran+openmp+profile
==> 45 installed packages
-- cray-cnl7-haswell / gcc@10.1.0 -------------------------------
darshan-runtime@3.2.1 darshan-util@3.2.1 dyninst@10.2.1 legion@20.03.0 plasma@20.9.20 slate@2020.10.00
-- cray-cnl7-haswell / intel@19.1.2.254 -------------------------
adios2@2.7.1 flecsi@1.4 libnrm@0.1.0 openpmd-api@0.13.2 qthreads@1.16 superlu-dist@6.4.0 umpire@4.1.2
aml@0.1.0 flit@2.1.0 libquo@1.3.1 papi@6.0.0.1 raja@0.13.0 swig@4.0.2-fortran upcxx@2020.10.0
arborx@0.9-beta gasnet@2020.3.0 mercury@2.0.0 papyrus@1.0.1 slepc@3.14.2 tasmanian@7.3 zfp@0.5.5
bolt@2.0 ginkgo@1.3.0 mfem@4.2.0 pdt@3.25.1 strumpack@5.1.1 tau@2.30.1
caliper@2.5.0 gotcha@1.0.3 ninja@1.10.2 precice@2.2.0 sundials@5.7.0 turbine@1.2.3
faodel@1.1906.1 hypre@2.20.0 omega-h@9.32.5 pumi@2.2.5 superlu@5.2.1 umap@2.1.0
Note
The spack find --explicit will show installed specs explicitly which is all of the E4S products which can be useful to filter output by E4S products. If you to see all packages (explicit and implicit) then please use spack find.
Accessing E4S Software Stack¶
When you load e4s/21.02 we expose a MODULEPATH to the location where spack generates modules. The full path to module root is /global/common/software/spackecp/e4s-21.02/modules/cray-cnl7-haswell/. In this release we provide all modules without hash in the form: {name}/{version}-{compiler.name}-{compiler.version}.
Note
We only generate modules for root specs which means packages installed implicitly won't have a generated module.
/global/common/software/spackecp/e4s-21.02/modules/cray-cnl7-haswell/ ---------------------------------------------------
adios2/2.7.1-intel-19.1.2.254 ginkgo/1.3.0-intel-19.1.2.254 papyrus/1.0.1-intel-19.1.2.254 superlu-dist/6.4.0-intel-19.1.2.254
aml/0.1.0-intel-19.1.2.254 gotcha/1.0.3-intel-19.1.2.254 pdt/3.25.1-intel-19.1.2.254 swig/4.0.2-fortran-intel-19.1.2.254
arborx/0.9-beta-intel-19.1.2.254 hypre/2.20.0-intel-19.1.2.254 plasma/20.9.20-gcc-10.1.0 tasmanian/7.3-intel-19.1.2.254
bolt/2.0-intel-19.1.2.254 legion/20.03.0-gcc-10.1.0 precice/2.2.0-intel-19.1.2.254 tau/2.30.1-intel-19.1.2.254
caliper/2.5.0-intel-19.1.2.254 libnrm/0.1.0-intel-19.1.2.254 pumi/2.2.5-intel-19.1.2.254 turbine/1.2.3-intel-19.1.2.254
darshan-runtime/3.2.1-gcc-10.1.0 libquo/1.3.1-intel-19.1.2.254 qthreads/1.16-intel-19.1.2.254 umap/2.1.0-intel-19.1.2.254
darshan-util/3.2.1-gcc-10.1.0 mercury/2.0.0-intel-19.1.2.254 raja/0.13.0-intel-19.1.2.254 umpire/4.1.2-intel-19.1.2.254
dyninst/10.2.1-gcc-10.1.0 mfem/4.2.0-intel-19.1.2.254 slate/2020.10.00-gcc-10.1.0 upcxx/2020.10.0-intel-19.1.2.254
faodel/1.1906.1-intel-19.1.2.254 ninja/1.10.2-intel-19.1.2.254 slepc/3.14.2-intel-19.1.2.254 zfp/0.5.5-intel-19.1.2.254
flecsi/1.4-intel-19.1.2.254 omega-h/9.32.5-intel-19.1.2.254 strumpack/5.1.1-intel-19.1.2.254
flit/2.1.0-intel-19.1.2.254 openpmd-api/0.13.2-intel-19.1.2.254 sundials/5.7.0-intel-19.1.2.254
gasnet/2020.3.0-intel-19.1.2.254 papi/6.0.0.1-intel-19.1.2.254 superlu/5.2.1-intel-19.1.2.254
If you are unsure about the module file you can use spack module tcl find <spec> to see a correlation of spec with its modulefile. If you want to know the full path to modulefile you can use the --full-path option in spack module tcl find command.
You can access the software via module load or spack load depending on your preference. For instance we load the papi module and you can run papi_mem_info to report Memory Cache details.
$ module load papi/6.0.0.1-intel-19.1.2.254
$ which papi_mem_info
/global/common/software/spackecp/e4s-21.02/software/cray-cnl7-haswell/intel-19.1.2.254/papi-6.0.0.1-ghgds2oiifqz4ulxpfpldp6br5rnwb2w/bin/papi_mem_info
$ papi_mem_info
Memory Cache and TLB Hierarchy Information.
------------------------------------------------------------------------
TLB Information.
There may be multiple descriptors for each level of TLB
if multiple page sizes are supported.
L1 Data TLB:
Page Size: 4 KB
Number of Entries: 64
Associativity: 4
Cache Information.
L1 Data Cache:
Total size: 32 KB
Line size: 64 B
Number of Lines: 512
Associativity: 8
L1 Instruction Cache:
Total size: 32 KB
Line size: 64 B
Number of Lines: 512
Associativity: 8
L2 Unified Cache:
Total size: 256 KB
Line size: 64 B
Number of Lines: 4096
Associativity: 8
L3 Unified Cache:
Total size: 40960 KB
Line size: 64 B
Number of Lines: 655360
Associativity: 20
Alternately, you can use spack load to change your user environment similar to module load but you have access to all the spack packages whereas the modules only expose explitcit package.
Let's say we want to use darshan tool so we can load darshan-util and darshan-runtime package using the following. darshan-parser is one of the tools provided by darshan note that loading the spack package updated the $PATH to point to the E4S install.
$ spack load darshan-util
$ spack load darshan-runtime
$ which darshan-parser
/global/common/software/spackecp/e4s-21.02/software/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx/bin/darshan-parser
$ darshan-parser --help
Usage: darshan-parser [options] <filename>
--all : all sub-options are enabled
--base : darshan log field data [default]
--file : total file counts
--file-list : per-file summaries
--file-list-detailed : per-file summaries with additional detail
--perf : derived perf data
--total : aggregated darshan field data
If you want to see list of specs loaded in your user environment use the spack find --loaded command, which is akin to module list command which reports active modules.
Similarly you can use spack unload <spec> to remove spec from user environment or run spack unload -a to remove all loaded spack packages from user environment which is equivalent to module purge.
Installing Software via spack¶
We provide a mirror named cori-e4s-21.02 that provides a list of specs that can be installed from buildcache. To see the list of mirrors you can run the following
$ spack mirror list
cori-e4s-21.02 /global/common/software/spackecp/mirrors/cori-e4s-21.02
spack-public https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/
To see a list of packages in the buildcache you can run, the -Lv option can be useful when you want to see specs with hash and list of variants for each spec:
spack buildcache list -Lv
If you want to install some packages, we recommend you create an environment in your user space such as $HOME as follows:
Note
You may need to deactivate from e4s environment by running despactivate or spack env deactivate in order to create new environment.
elvis@cori08:~> spack env create -d $HOME/myproject
==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Created environment in /global/u2/y/elvis/myproject
==> You can activate this environment with:
==> spack env activate /global/u2/y/elvis/myproject
elvis@cori08:~> spack env activate -d $HOME/myproject
elvis@cori10:~> spack env st
==> In environment /global/u2/y/elvis/myproject
Error
You cannot create spack environment using the name format because spack will try to write environment in $SPACK_ROOT/var/spack/environments which is not world writable
elvis@cori03:~/myproject> spack env create demo
==> Error: [Errno 13] Permission denied: '/global/common/software/spackecp/e4s-21.02/spack/var/spack/environments/demo'
We have configured our spack instance with site default in order to help facilitate spack install for multi-user. The spack configuration can be found at $SPACK_ROOT/etc/spack that contains files like compilers.yaml, config.yaml, mirrors.yaml, modules.yaml, and packages.yaml
For instance, we have configured the mirror cori-e4s-21.02 so that user can automatically install specs from buildcache. This configuration is set in $SPACK_ROOT/etc/spack/mirrors.yaml.
elvis@cori08:~> spack mirror list
cori-e4s-21.02 /global/common/software/spackecp/mirrors/cori-e4s-21.02
spack-public https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/
Now let's try to install darshan-runtime and darshan-util by copying this content into your spack.yaml
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
view: true
specs:
- darshan-runtime@3.2.1%gcc@10.1.0 +slurm
- darshan-util@3.2.1%gcc +bzip2
We need to trust the gpg key in-order for spack to install specs from buildcache. This is a one-time operation, we recommend you set the following which tells spack to look for gpg keys in $HOME/.gnupg instead of $SPACK_ROOT.
export SPACK_GNUPGHOME=$HOME/.gnupg
Next you should trust the gpg key by running the following:
$ spack gpg trust /global/common/software/spackecp/gpgkeys/e4s2010.pub
gpgconf: socketdir is '/run/user/92503/gnupg'
gpg: key 0140A256659E0CBD: public key "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" imported
gpg: Total number processed: 1
gpg: imported: 1
Next let's concretize the environment to see package preferences.
Running spack concretize -f
elvis@cori10:~/myproject> spack concretize -f
==> Concretized darshan-runtime@3.2.1%gcc@10.1.0+slurm
[+] bqigqvv darshan-runtime@3.2.1%gcc@10.1.0~cobalt+mpi~pbs+slurm arch=cray-cnl7-haswell
[+] 4owqxmf ^mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell
[+] 5xqcgln ^zlib@1.2.11%gcc@10.1.0+optimize+pic+shared arch=cray-cnl7-haswell
==> Concretized darshan-util@3.2.1%gcc+bzip2
- wgigpno darshan-util@3.2.1%gcc@10.1.0+bzip2+shared patches=d1a0e581fb72830089302c94f77ccfcbaae416478f5241c4213fbe8b3f63b3b7 arch=cray-cnl7-haswell
- ssdgsz7 ^bzip2@1.0.8%gcc@10.1.0+shared arch=cray-cnl7-haswell
- 3rcgfbw ^diffutils@3.7%gcc@10.1.0 arch=cray-cnl7-haswell
- fu3r6gy ^libiconv@1.16%gcc@10.1.0 arch=cray-cnl7-haswell
[+] 5xqcgln ^zlib@1.2.11%gcc@10.1.0+optimize+pic+shared arch=cray-cnl7-haswell
==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Warning: [/global/u2/y/elvis/myproject/.spack-env/view] Skipping external package: mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell/4owqxmf
Please note we have defined some package preferences to help you satisfy some common package dependency (compiler, mpi, mkl), this can be found in $SPACK_ROOT/etc/spack/packages.yaml. We recommend you specify your own packages section in your spack.yaml to help tune your package preference. Note, if your package preference differs from specs in buildcache or slight difference in variants, then spack will end up installing packages from source so please be mindful of your selection. If you want to reuse existing specs in buildcache, you will need to specify preference with the same variants so that spack will reuse spec from buildcache. This can be done by comparing spec and its variant in buildcache spack buildcache list -Lv <spec> with the concretized output via spack concretize -f or spack spec -L <spec>
Running spack install
elvis@cori10:~/myproject> spack install
==> Installing environment /global/u2/y/elvis/myproject
==> Installing libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/libiconv-1.16/cray-cnl7-haswell-gcc-10.1.0-libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi.spack
################################################################################################################################################################### 100.0%
==> Extracting libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi from binary cache
gpgconf: socketdir is '/global/homes/y/elvis/.gnupg'
gpgconf: no /run/user dir
gpgconf: using homedir as fallback
gpg: Signature made Tue Jun 1 13:13:04 2021 PDT
gpg: using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56 522F 0140 A256 659E 0CBD
Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628 CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/zlib-1.2.11-5xqcglnrl3hl4wvyf4w3eafeefze5gcu
==> Installing diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/diffutils-3.7/cray-cnl7-haswell-gcc-10.1.0-diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx.spack
################################################################################################################################################################### 100.0%
==> Extracting diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx from binary cache
gpg: Signature made Tue Jun 1 14:03:15 2021 PDT
gpg: using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56 522F 0140 A256 659E 0CBD
Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628 CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx
==> Installing bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/bzip2-1.0.8/cray-cnl7-haswell-gcc-10.1.0-bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg.spack
################################################################################################################################################################### 100.0%
==> Extracting bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg from binary cache
gpg: Signature made Tue Jun 1 14:35:43 2021 PDT
gpg: using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56 522F 0140 A256 659E 0CBD
Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628 CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg
==> Installing darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1/cray-cnl7-haswell-gcc-10.1.0-darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx.spack
################################################################################################################################################################### 100.0%
==> Extracting darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx from binary cache
gpg: Signature made Tue Jun 1 16:32:50 2021 PDT
gpg: using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56 522F 0140 A256 659E 0CBD
Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628 CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx
==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Warning: [/global/u2/y/elvis/myproject/.spack-env/view] Skipping external package: mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell/4owqxmf
==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Warning: [/global/u2/y/elvis/myproject/.spack-env/view] Skipping external package: mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell/4owqxmf
Module Generation¶
Now we have successfully installed packages, if you are curious where they are installed. By default we have set the install location in $HOME/spack-workspace/software which is set in $SPACK_ROOT/etc/spack/config.yaml. Let's do a quick check to see where software is installed by running the following
elvis@cori10:~/myproject> spack find -Lp
==> In environment /global/u2/y/elvis/myproject
==> Root specs
-- no arch / gcc ------------------------------------------------
-------------------------------- darshan-util@3.2.1%gcc +bzip2
-- no arch / gcc@10.1.0 -----------------------------------------
-------------------------------- darshan-runtime@3.2.1%gcc@10.1.0 +slurm
==> 7 installed packages
-- cray-cnl7-haswell / gcc@10.1.0 -------------------------------
ssdgsz7hansfksagqhzaoxcvb3aug3bg bzip2@1.0.8 /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg
bqigqvvhf5rsdwa3yg53bfap7pcodfcw darshan-runtime@3.2.1 /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/darshan-runtime-3.2.1-bqigqvvhf5rsdwa3yg53bfap7pcodfcw
wgigpnoiosrtmizts4pn54of4lkh2ubx darshan-util@3.2.1 /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx
3rcgfbwwpvu7oxlsm2kh53trxjyilzdx diffutils@3.7 /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx
fu3r6gykbpgc3mayw34uhoh5q6pvjbhi libiconv@1.16 /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi
4owqxmfeiur5ctos6iwzfsmevyqvtl7e mpich@3.1 /opt/cray/pe/mpt/7.7.10/gni/mpich-gnu/8.2
5xqcglnrl3hl4wvyf4w3eafeefze5gcu zlib@1.2.11 /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/zlib-1.2.11-5xqcglnrl3hl4wvyf4w3eafeefze5gcu
Next, let's generate the modules for this environment. We have set a default location for modules in $HOME/spack-workspace/modules. If you want to generate the modules you can run the following:
elvis@cori10:~/myproject> spack module tcl refresh -y
==> Regenerating tcl module files
Let's check where the module files are located by checking the path to one of the spec.
elvis@cori10:~/myproject> spack module tcl find --full-path darshan-runtime
/global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell/darshan-runtime/3.2.1-gcc-10.1.0
We see that the root of module tree is /global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell/ therefore we should add this path to MODULEPATH by running the following:
elvis@cori10:~/myproject> module use /global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell
The module use will prepend the directory in front of MODULEPATH.
If all is done correctly you should see a module tree as follows:
$ module av
----------------------------------------------------- /global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell ------------------------------------------------------
bzip2/1.0.8-gcc-10.1.0 darshan-util/3.2.1-gcc-10.1.0 libiconv/1.16-gcc-10.1.0 zlib/1.2.11-gcc-10.1.0
darshan-runtime/3.2.1-gcc-10.1.0 diffutils/3.7-gcc-10.1.0 mpich/3.1-gcc-10.1.0
Note
We have the same name format as our production stack {name}/{version}-{compiler.name}-{compiler.version}. This is set in our site configuration $SPACK_ROOT/etc/spack/modules.yaml which can be overridden in your spack.yaml