Grid software

In this page we will talk about the options to run your software on the Grid worker nodes:

Softdrive

Softdrive is the service that allows you to install software in a central place and distribute it automagically on the Grid. You install the software once, and it will be available on all clusters, to all users. This means that you no longer need to supply your software in your input sandbox, or download your software in your job.

CVMFS

Softdrive is using the CVMFS (short for CernVM File System) tool on the background. CVMFS is a network file system based on HTTP and optimized to deliver experiment software in a fast, scalable, and reliable way.

Quickstart

In this example, we will distribute a few small files to all nodes in the Life Sciences Grid. This should give you an idea of what is possible with Softdrive.

Softdrive works by logging in the software distribution node, and putting your files there. Next, you tell the software distribution system that you are ready installing files. These files will be made available on all nodes in the Life Science Grid and on all other nodes on the Dutch National Grid.

CVMFS group membership

To distribute your files using Softdrive, you must be a member of the group softdrive. If you are not a member yet, send an e-mail to helpdesk@surfsara.nl with a membership request.

Logging in on the softdrive

After you have been added to the softdrive group, you can log in on the software distribution node, using your Grid UI username and password:

$ssh homer@softdrive.grid.sara.nl # replace homer with your username

In your home-directory (e.g. /home/homer), you will find a README file with detailed information about the Softdrive usage.

Distributing an example file

To demonstrate distributing files to all Grid nodes, create a file and a directory within your home directory:

# a test directory and a file
softdrive.grid.sara.nl:/home/homer$ mkdir -p test_dir
softdrive.grid.sara.nl:/home/homer$ echo "Hello world" > test_dir/hello.txt

To make this directory file available on all nodes on the Grid, you have to copy the test_dir under /cvmfs/softdrive.nl/$USER:

softdrive.grid.sara.nl:/home/homer$ cp -r test_dir /cvmfs/softdrive.nl/homer # replace homer with your username
  • To force the update everywhere in the Grid, trigger publication by executing command:
publish-my-softdrive

Updating on all Grid nodes can take up to two hours.

Note

You need to run the command publish-my-softdrive each time you make a change in your /cvmfs/softdrive.nl/$USER directory in order to take effect on the Grid sites.

Finding your files on the Grid nodes

On nodes, your Softdrive files will be available under:

/cvmfs/softdrive.nl/homer/ # replace homer with your username

Log in to your UI account and check whether your files are there:

ui.grid.sara.nl:/home/homer$ ls /cvmfs/softdrive.nl/homer/
drwxr-xr-x 17 cvmfs cvmfs 4096 Dec 16 12:11 test_dir

Note

If your software is statically compiled, then copying the executables from your home directory to /cvmfs/softdrive.nl/$USER/ should work. Just remember to export the /cvmfs/softdrive.nl/$USER software paths into your Grid scripts or UI .bashrc. In other cases with library path dependencies, we advice you to install your software directly under /cvmfs/softdrive.nl/$USER or use a prefix. An example of software installation in Softdrive can be found in section anaconda on Grid.

Python on the Grid

On the local Grid clusters the python version installed is Python 2.6.6. If you need a different python version or additional packages, we recommend you to install Anaconda python in your UI or Softdrive account.

Next is an example of installing the Anaconda python distribution in Softdrive.

Softdrive anaconda

  • Log in to Softdrive with your account:
$ssh homer@softdrive.grid.sara.nl # replace homer with your username
  • Download in your home account the latest version of Anaconda installer for linux, e.g.:
$wget https://3230d63b5fc54e62148e-c95ac804525aac4b6dba79b00b39d1d3.ssl.cf1.rackcdn.com/Anaconda2-2.4.0-Linux-x86_64.sh
  • Run the installer (read and approve the license terms) in Softdrive:
$bash Anaconda2-2.4.0-Linux-x86_64.sh

Note here! The installer will ask you to which location to install the software. Do not accept the default but change it to: /cvmfs/softdrive.nl/$USER/anaconda-2-2.4.0/:

Anaconda2 will now be installed into this location:
/home/homer/anaconda2
- Press ENTER to confirm the location
- Press CTRL-C to abort the installation
- Or specify a different location below

[/home/homer/anaconda2] >>> /cvmfs/softdrive.nl/homer/anaconda-2-2.4.0/
...

That was it! You can now publish the software that is installed in your /cvmfs/softdrive.nl/homer/anaconda-2-2.4.0 directory. To do so, run this command in Softdrive:

$publish-my-softdrive

Then check after 1-2 hours from the UI if the /cvmfs/softdrive.nl/homer/anaconda-2-2.4.0 exists.

Finally, remember to include the installation path in your scripts as:

$export PATH=/cvmfs/softdrive.nl/homer/anaconda-2-2.4.0/bin:$PATH # replace homer with your username

Docker

At the moment it is not possible to run Docker containers on the Dutch National Grid or Life Science Grid. We are currently investigating different possibilities. Please contact us at helpdesk@surfsara.nl to discuss about the available options.