Source shell script in modulefile
When working with large software suite providing a shell script for their
enablement in user environment, it is usually desired to also provide access
to these software through module
. However these software enablement may be
complex and it may be wise to keep using the shell script provided by software
editor rather crafting a modulefile from scratch.
This recipe describes how to make modulefiles for such software by using the enablement shell script provided with them.
Implementation
Modules version 4.6 introduces a new sub-command named sh-to-mod
and
a new modulefile command named source-sh
. The sh-to-mod
outputs as a modulefile content the environment changes done by the evaluation
of a shell script passed as argument. On the other hand, the source-sh
modulefile command sources environment changes done by the evaluation of a
shell script passed as argument.
Both new features relies on the same mechanism that starts a designated shell to:
get current environment state (environment variables, shell aliases, shell functions and current working directory)
source designated shell script with defined arguments
get resulting environment state
Once done, environment prior and after script source are compared to determine
the corresponding environment changes and translate those changes into
modulefile commands (setenv
, prepend-path
,
set-alias
, set-function
, ...).
sh-to-mod
outputs these resulting modulefile commands. This output
can be redirected into a file to create a modulefile. source-sh
on
the other hand sources the resulting modulefile commands to evaluate them as
if they were written in the modulefile calling source-sh
.
sh-to-mod
and source-sh
support the following shells: sh, dash,
csh, tcsh, bash, ksh, ksh93, zsh and fish.
Compatible with Modules v4.6+
Basic usage example
For this recipe, a dummy software named foo is used as example. foo is
installed in version 1.2 in example/source-script-in-modulefile/foo-1.2
directory and it provides a foo-setup.sh
script to activate itself in
user environment:
#!/bin/bash
export FOOENV="$1"
PATH=$(dirname "${BASH_SOURCE[0]}")/bin:$PATH
export PATH
alias foo='foobin -q -l'
First line of foo-setup.sh
script helps to identify which shell needs to
be used to evaluate it: bash
.
sh-to-mod
may be used to get this script translated as a
modulefile:
$ module sh-to-mod bash example/source-script-in-modulefile/foo-1.2/foo-setup.sh arg1 #%Module prepend-path PATH example/source-script-in-modulefile/foo-1.2/bin set-alias foo {foobin -q -l} setenv FOOENV arg1
Output could be redirected into a foo/1.2
file and make it the modulefile
to enable software foo:
$ mkdir -p modulefiles/foo $ module sh-to-mod bash example/source-script-in-modulefile/foo-1.2/foo-setup.sh arg1 >modulefiles/foo/1.2 $ module use ./modulefiles $ module show foo ------------------------------------------------------------------- modulefiles/foo/1.2: prepend-path PATH example/source-script-in-modulefile/foo-1.2/bin set-alias foo {foobin -q -l} setenv FOOENV arg1 -------------------------------------------------------------------
Instead of transforming shell script in modulefile, a modulefile using
source-sh
modulefile command to evaluate shell script at modulefile
evaluation time may be written:
#%Module4.6
source-sh bash example/source-script-in-modulefile/foo-1.2/foo-setup.sh arg1
When displaying a modulefile using source-sh
modulefile command,
modulefile commands resulting from source-sh
evaluation are reported:
$ module show foo/1.2 ------------------------------------------------------------------- .../modulefiles/foo/1.2: prepend-path PATH example/source-script-in-modulefile/foo-1.2/bin set-alias foo {foobin -q -l} setenv FOOENV arg1 -------------------------------------------------------------------
Loading this foo/1.2
module will enable access to software foo:
$ module load foo/1.2 $ alias foo alias foo='foobin -q -l' $ foo foo, version 1.2
Unloading foo/1.2
module will properly revert these environment settings:
$ module unload foo/1.2 $ alias foo bash: alias: foo: not found $ foobin bash: foobin: command not found
As conclusion, these new features enable to leverage the setup scripts that
are provided along with software to make them reachable from the module
environment.
Usage with shell-specific scripts
When the initialization script provided by software only defines environment
variables, this script could be used to setup the user environment through the
use of source-sh
in a modulefile whatever the shell ran by user, as the
module
command will accurately translate script changes into the language
of the running shell.
For instance the foo/1.2
module, that uses the source-sh
modulefile
command over the foo-setup.sh
bash script, could also be used when running
the tcsh
or fish
shell:
$ echo $version tcsh 6.22.03 (Astron) 2020-11-18 (x86_64-unknown-linux) ... $ module show foo/1.2 ------------------------------------------------------------------- .../modulefiles/foo/1.2: prepend-path PATH example/source-script-in-modulefile/foo-1.2/bin set-alias foo {foobin -q -l} setenv FOOENV arg1 ------------------------------------------------------------------- $ module load foo/1.2 $ foo foo, version 1.2
Software may sometimes provide a specific script for each shell they support as they do not perform their initialization the same way on every shell. Quite often a shell function is defined for sh shells whereas an alias is setup for csh shells (as such shells do not support shell function).
Dummy software bar is used to demonstrate this situation. bar is installed
in version 2.1 in example/source-script-in-modulefile/bar-2.1
directory
and it provides a bar-setup.sh
and a bar-setup.csh
scripts to activate
itself in user environment, depending on the shell kind used.
#!/bin/bash
PATH=$(dirname "${BASH_SOURCE[0]}")/bin:$PATH
export PATH
bar() {
barbin -q -l
}
#!/bin/tcsh
setenv PATH "example/source-script-in-modulefile/bar-2.1/bin:$PATH"
alias bar 'barbin -q -l'
To accurately initialize environment for bar software, the bar
module
needs to call the .sh
script if user is currently running a shell from the
sh family, or to call the .csh
script if user runs a csh-kind shell.
#%Module4.6
set scriptpath example/source-script-in-modulefile/bar-2.1
switch -- [module-info shelltype] {
sh {
source-sh bash $scriptpath/bar-setup.sh
}
csh {
source-sh tcsh $scriptpath/bar-setup.csh
}
}
This way the bar
shell function is initialized when loading module from a user
environment running a sh shell:
$ echo $BASH_VERSION 5.1.0(1)-release $ module use example/source-script-in-modulefile/modulefiles $ module show bar ------------------------------------------------------------------- .../modulefiles/bar/2.1: prepend-path PATH example/source-script-in-modulefile/bar-2.1/bin set-function bar { barbin -q -l} ------------------------------------------------------------------- $ module load bar $ type bar bar is a function bar () { barbin -q -l } $ bar bar, version 2.1
Whereas the bar
shell alias is setup on csh shell environment:
$ echo $version tcsh 6.22.03 (Astron) 2020-11-18 (x86_64-unknown-linux) ... $ module use example/source-script-in-modulefile/modulefiles $ module show bar ------------------------------------------------------------------- .../modulefiles/bar/2.1: prepend-path PATH example/source-script-in-modulefile/bar-2.1/bin set-alias bar {barbin -q -l} ------------------------------------------------------------------- $ module load bar $ alias bar barbin -q -l $ bar bar, version 2.1