When you execute a (non-builtin) command in bash, the location of the program corresponding to that command is looked up in the filesystem according to the $PATH environment variable. For instance, on a typical Linux machine, one might see
Thus, when the command 'minctracc' is typed into a Bash session, the above ':'-separated list of directories will be searched in left-to-right order for an executable file (script or binary) called 'minctracc'.
On most systems (NixOS/nixpkgs and Guix being the notable exceptions), software is simply upgraded in-place. However, for scientific computing, we must be able to install multiple versions of the same software (together with their dependencies, of course). For instance, users may want to test a new version or use features not available in previous versions, but must also be able to use old versions to continue and re-run previous analyses. On the other hand, upgrading system software such as that in /bin and /sbin is necessary for security reasons. We currently resolve these issues by providing a collection of environment modules (modules for short), although arguably use of lightweight containers (Singularity/Apptainer, Docker, Podman, etc.) or a Nixpkgs-like installation is a better solution.
Currently, (August 2022) we have Ubuntu 20.04 (focal) installed, and our science software available via modules resides here:
Accessing the module system
Here StdEnv (formerly called mice-env, and still called that on non-MICe systems such as HPF, Graham, etc., which may define their own StdEnv) is an omnibus module which loads a variety of other modules.
StdEnv is no longer loaded automatically, as used to be the case during mice-env days. The main reason is that StdEnv provides many more modules in order to increase isolation from the "mutable" base OS packages, and is slower to load as a result. If you are fine with every terminal taking several seconds to start, feel free to add 'module load StdEnv' to your ~/.bashrc – there are no other known issues involving this (although in principle some system libraries such as bz, lz4 etc might break as a few modules set LD_LIBRARY_PATH in order for Python cdll to find their contents). We ought to investigate ways to optimize this load time.
You can list available modules by using the following command:
Similarly, `module list` lists 'loaded' modules, 'module purge' unloads all modules. You can also use 'ml' as a shorthand, e.g. 'ml mice-env', 'ml list', etc., but this doesn't work with older module systems such as at HPF. For more details, see https://lmod.readthedocs.io/en/latest/.
By default, the most recent version of the StdEnv module is loaded. When beginning a project, it is likely a good idea to run 'module avail' and note the version of this module in order to reload the same version for the remainder of the project in order to prevent, e.g., unexpected type errors in project-specific R or Python code or even command-line tools, changes in numerics affecting your results, etc. You can create a bash script to reload this module and even use the
direnv program to automatically reload whenever you re-enter this project directory.
The following lines are run by default on the MICe desktops, making the environment module system available at the command line:
Modules and Pydpiper
The following link is mostly deprecated as it's largely redundant with the current page.
For information about registration quarantines and pydpiper versions look here: PydPiper at MICe with quarantines