Grid software

In this page we will talk about the options to run your software on the Grid worker nodes:

Softdrive

Softdrive is the service that allows you to install software in a central place and distribute it automagically on the Grid. You install the software once, and it will be available on all clusters, to all users. This means that you no longer need to supply your software in your input sandbox, or download your software in your job.

This page is about using Softdrive on the grid infrastructure. The use of Softdrive is however not limited to grid alone, as it can equally well be applied to your own clusters, your own computer or in cloud environments.

CVMFS

Softdrive is using the CVMFS service (short for CernVM File System) on the background. CVMFS is a network file system based on HTTP and optimised to deliver experiment software in a fast, scalable, and reliable way.

Quickstart

In this example, we will distribute a few small files to all nodes in the Dutch Grid. This should give you an idea of what is possible with Softdrive.

Softdrive works by logging in the software distribution node, and putting your files there. Next, you tell the software distribution system that you are ready installing files. These files will be made available on all nodes in the Life Science Grid and on all other nodes on the Dutch National Grid.

Access

Users of the National e-Infrastructure are entitled to use Softdrive without the need for a separate resource request. You can request access by sending an e-mail with your current project allocation id to helpdesk@surfsara.nl.

Logging in on the softdrive

Once access has been arranged, you can log in on the software distribution node, using your Grid UI username and password:

$ssh homer@softdrive.grid.sara.nl # replace homer with your username

In your home-directory (e.g. /home/homer), you will find a README file with detailed information about the Softdrive usage.

Distributing an example file

To demonstrate distributing files to all Grid nodes, create a file and a directory within your home directory:

# a test directory and a file
softdrive.grid.sara.nl:/home/homer$ mkdir -p test_dir
softdrive.grid.sara.nl:/home/homer$ echo "Hello world" > test_dir/hello.txt

To make this directory file available on all nodes on the Grid, you have to copy the test_dir under /cvmfs/softdrive.nl/$USER:

softdrive.grid.sara.nl:/home/homer$ cp -r test_dir /cvmfs/softdrive.nl/homer # replace homer with your username
  • To force the update everywhere in the Grid, trigger publication by executing command:
publish-my-softdrive

Updating on all Grid nodes can take up to two hours.

Note

You need to run the command publish-my-softdrive each time you make a change in your /cvmfs/softdrive.nl/$USER directory in order to take effect on the Grid sites.

Finding your files on the Grid nodes

On Dutch Grid nodes, your Softdrive files will be available under:

/cvmfs/softdrive.nl/homer/ # replace homer with your username

Log in to your UI account and check whether your files are there:

ui.grid.sara.nl:/home/homer$ ls /cvmfs/softdrive.nl/homer/
drwxr-xr-x 17 cvmfs cvmfs 4096 Dec 16 12:11 test_dir

Note

If your software is statically compiled, then copying the executables from your home directory to /cvmfs/softdrive.nl/$USER/ should work. Just remember to export the /cvmfs/softdrive.nl/$USER software paths into your Grid scripts or UI .bashrc. In other cases with library path dependencies, we advice you to install your software directly under /cvmfs/softdrive.nl/$USER or use a prefix. An example of software installation in Softdrive can be found in section anaconda on Grid.

Python on the Grid

If you want to use a different python version to the existing on the Grid nodes or additional packages, we recommend you to install Anaconda python in your UI or Softdrive account.

Next is an example of installing the Anaconda python distribution in Softdrive.

Softdrive anaconda

  • Log in to Softdrive with your account:
$ssh homer@softdrive.grid.sara.nl # replace homer with your username

$wget https://repo.continuum.io/archive/Anaconda2-5.1.0-Linux-x86_64.sh
  • Run the installer (read and approve the license terms) in Softdrive:
$bash Anaconda2-5.1.0-Linux-x86_64.sh

Note here! The installer will ask you to which location to install the software. Do not accept the default but change it to: /cvmfs/softdrive.nl/$USER/anaconda-2-5.1.0/:

Anaconda2 will now be installed into this location:
/home/homer/anaconda2
- Press ENTER to confirm the location
- Press CTRL-C to abort the installation
- Or specify a different location below

[/home/homer/anaconda2] >>> /cvmfs/softdrive.nl/homer/anaconda-2-5.1.0/
...

That was it! You can now publish the software that is installed in your /cvmfs/softdrive.nl/homer/anaconda-2-5.1.0 directory. To do so, run this command in Softdrive:

$publish-my-softdrive

Then check after 1-2 hours from the UI if the /cvmfs/softdrive.nl/homer/anaconda-2-5.1.0 exists.

Finally, remember to include the installation path in your scripts as:

$export PATH=/cvmfs/softdrive.nl/homer/anaconda-2-5.1.0/bin:$PATH # replace homer with your username

Docker

At the moment it is not possible to run Docker containers on the Dutch National Grid or Life Science Grid. We are currently investigating different possibilities and we offer Singularity as a container service. Please contact us at helpdesk@surfsara.nl to discuss about the available options.