Use new features without breaking old module command

When working on large infrastructure, sometimes the module command is not deployed with the same version everywhere. You may have for instance some old cluster that still uses Modules version 3.2 and a newer supercomputer where the latest version of Modules 4 is available. In such situation it may however be desired to share the same modulefile catalog.

People providing software that build software and generate the modulefiles to access them fall in a similar situation: a large variety of Modules version may be in use by the end-users of their build product.

This recipe describes how to use features from the latest version of Modules 4 in modulefiles without breaking the use of these modulefiles from older Modules version.

Implementation

The Tcl language provides some introspection mechanisms that help to know what Tcl procedures are available. In the modulefile or modulerc evaluation context, it helps to determine what Tcl modulefile commands are known. So by crafting a conditional test using such mechanism (with Tcl info commands command) it is possible to test if a new feature is available prior using it.

.modulerc

#%Module

# use module-hide only if available
if {[info commands module-hide] eq {module-hide}} {
   module-hide --after 2020-10-01 foo/1.1
}
# use module-forbid only if available
if {[info commands module-forbid] eq {module-forbid}} {
   module-forbid --after 2020-10-01 foo/1.1
}

Compatible with Modules v3.2+

Starting version 4.7 of Modules, two new Tcl variables are introduced in the modulefile and modulerc evaluation context: ModuleTool and ModuleToolVersion. These two variables help to determine respectively what is the module implementation running and what is its version. With this knowledge it is possible to adapt modulefile and modulerc code to cope with a behavior changing over module versions or with different behaviors between different module implementation. The Tcl modulefile command versioncmp has been added along to help comparing software version number (e.g. 4.10 is newer than 4.7).

Starting its version 8.4.8, the Lmod project also supports the ModuleTool and ModuleToolVersion variables and the versioncmp modulefile command. It enables having modulefiles compatible with both module implementations without restricting yourself from using the advanced features from both projects.

Compatible with Modules v4.7+

Usage example

For this recipe, a foo module is available in version 1.1 and 1.2. Version 1.1 is outdated and it has been decided to hide and forbid it starting October 2020. For that, the new module-hide and module-forbid modulefile commands introduced in Modules 4.6 are used.

Enable the modulepath where the example modulefiles are located:

$ module use example/new-features-without-breaking-old-module/modulefiles

By using the conditional test code in the modulepath rc file (see Implementation section above), the two new modulefile commands are not used. So older versions of Modules still in use do not benefit from the dynamic hiding and forbidding features however no error are obtained on these setups:

$ module -V | grep ^VERSION=
VERSION=3.2.13
$ module avail -t foo
/path/to/example/new-features-without-breaking-old-module/modulefiles:
foo/1.1
foo/1.2

Yet the dynamic hiding and forbidding features are enabled for setup using Modules 4.6 or newer version:

$ module -V
Modules Release 4.6.0 (2020-09-16)
$ module avail -t foo
/path/to/example/new-features-without-breaking-old-module/modulefiles:
foo/1.2

Now take a look at bar module which provides a version for each Unix group the current user is member of. User group membership can be retrieved with the usergroups sub-command of module-info starting Modules version 4.6. With older version of Modules, the external command groups has to be used to get this information. By using the ModuleTool and ModuleToolVersion Tcl variables it will be possible to determine if usergroups sub-command is available on module-info.

bar/.modulerc

#%Module

if {[info exists ModuleTool] && $ModuleTool eq {Modules}
   && [versioncmp $ModuleToolVersion 4.6] >= 0} {
   set grouplist [module-info usergroups]
} else {
   set grouplist [exec groups]
}

foreach grp $grouplist {
    module-virtual bar/$grp ./.common
}

Querying available bar module versions should match the list of groups of current user:

$ groups
grp1 grp2
$ module avail -t bar
/path/to/example/new-features-without-breaking-old-module/modulefiles:
bar/grp1
bar/grp2

Note

As the new Tcl variables are introduced in Modules 4.7, the use of the new usergroups sub-command will only be triggered starting Modules 4.7.