Grid software¶
In this page we will talk about the options to run your software on the Grid worker nodes:
Contents
Softdrive¶
Softdrive is the service that allows you to install software in a central place and distribute it automagically on the Grid. You install the software once, and it will be available on all clusters, to all users. This means that you no longer need to supply your software in your input sandbox, or download your software in your job.
This page is about using Softdrive on the grid infrastructure. The use of Softdrive is however not limited to grid alone, as it can equally well be applied to your own clusters, your own computer or in cloud environments.
CVMFS¶
Softdrive is using the CVMFS service (short for CernVM File System) on the background. CVMFS is a network file system based on HTTP and optimised to deliver experiment software in a fast, scalable, and reliable way.
Quickstart¶
In this example, we will distribute a few small files to all nodes in the Dutch Grid. This should give you an idea of what is possible with Softdrive.
Softdrive works by logging in the software distribution node, and putting your files there. Next, you tell the software distribution system that you are ready installing files. These files will be made available on all nodes in the Life Science Grid and on all other nodes on the Dutch National Grid.
Access¶
Users of the National e-Infrastructure are entitled to use Softdrive without the need for a separate resource request. You can request access by sending an e-mail with your current project allocation id to helpdesk@surfsara.nl.
Logging in on the softdrive¶
Once access has been arranged, you can log in on the software distribution node, using your Grid UI username and password:
$ssh homer@softdrive.grid.sara.nl # replace homer with your username
In your home-directory (e.g. /home/homer
), you will find a README file with detailed information about the Softdrive usage.
Distributing an example file¶
To demonstrate distributing files to all Grid nodes, create a file and a directory within your home directory:
# a test directory and a file
softdrive.grid.sara.nl:/home/homer$ mkdir -p test_dir
softdrive.grid.sara.nl:/home/homer$ echo "Hello world" > test_dir/hello.txt
To make this directory file available on all nodes on the Grid, you have to copy the test_dir
under /cvmfs/softdrive.nl/$USER
:
softdrive.grid.sara.nl:/home/homer$ cp -r test_dir /cvmfs/softdrive.nl/homer # replace homer with your username
- To force the update everywhere in the Grid, trigger publication by executing command:
publish-my-softdrive
Updating on all Grid nodes can take up to two hours.
Note
You need to run the command publish-my-softdrive
each time you make a change in your /cvmfs/softdrive.nl/$USER
directory in order to take effect on the Grid sites.
Finding your files on the Grid nodes¶
On Dutch Grid nodes, your Softdrive files will be available under:
/cvmfs/softdrive.nl/homer/ # replace homer with your username
Log in to your UI account and check whether your files are there:
ui.grid.sara.nl:/home/homer$ ls /cvmfs/softdrive.nl/homer/
drwxr-xr-x 17 cvmfs cvmfs 4096 Dec 16 12:11 test_dir
Note
If your software is statically compiled, then copying the executables from your home directory to /cvmfs/softdrive.nl/$USER/
should work. Just remember to export the /cvmfs/softdrive.nl/$USER
software paths into your Grid scripts or UI .bashrc
. In other cases with library path dependencies, we advice you to install your software directly under /cvmfs/softdrive.nl/$USER
or use a prefix. An example of software installation in Softdrive can be found in section anaconda on Grid.
Python on the Grid¶
If you want to use a different python version to the existing on the Grid nodes or additional packages, we recommend you to install Anaconda python in your UI or Softdrive account.
Next is an example of installing the Anaconda python distribution in Softdrive.
Softdrive anaconda¶
- Log in to Softdrive with your account:
$ssh homer@softdrive.grid.sara.nl # replace homer with your username
- Download in your home account the latest version of Anaconda installer for linux, e.g.:
$wget https://repo.continuum.io/archive/Anaconda2-5.1.0-Linux-x86_64.sh
- Run the installer (read and approve the license terms) in Softdrive:
$bash Anaconda2-5.1.0-Linux-x86_64.sh
Note here! The installer will ask you to which location to install the software. Do not accept the default but change it to: /cvmfs/softdrive.nl/$USER/anaconda-2-5.1.0/
:
Anaconda2 will now be installed into this location:
/home/homer/anaconda2
- Press ENTER to confirm the location
- Press CTRL-C to abort the installation
- Or specify a different location below
[/home/homer/anaconda2] >>> /cvmfs/softdrive.nl/homer/anaconda-2-5.1.0/
...
That was it! You can now publish the software that is installed in your /cvmfs/softdrive.nl/homer/anaconda-2-5.1.0
directory. To do so, run this command in Softdrive:
$publish-my-softdrive
Then check after 1-2 hours from the UI if the /cvmfs/softdrive.nl/homer/anaconda-2-5.1.0
exists.
Finally, remember to include the installation path in your scripts as:
$export PATH=/cvmfs/softdrive.nl/homer/anaconda-2-5.1.0/bin:$PATH # replace homer with your username
Docker¶
At the moment it is not possible to run Docker containers on the Dutch National Grid or Life Science Grid. We are currently investigating different possibilities and we offer Singularity as a container service. Please contact us at helpdesk@surfsara.nl to discuss about the available options.