Loughborough University
Leicestershire, UK
LE11 3TU
+44 (0)1509 222222
Loughborough University

IT Services : High Performance Computing

Storage


Introduction

None of the HPC user filestores are backed up. It is the responsibility of users to ensure that they keep copies of their critical files somewhere safe.

Home Directories

User home directories are mounted on all compute nodes via Lustre and can be used in run scripts or programs without any changes to directory paths.

However, if you are using heavy input or output you may wish to use the scratch space which is mounted on each node under /scratch

Directories are not backed up. It is the user's responsibility to ensure they keep copies of any critical files elsewhere.

Quotas

Users have a 1TB quota set on their home directory. This is to ensure that runaway jobs do not fill up the disk space and so interrupt the service for other users. To check your current disk usage you can run the lfs quota command: lfs quota -h /home to see the quota on the home directory area. The following example output is for a fictitious user. It also shows how much data people in your group (normally department) is using across all users in that group.

Disk quotas for user blah (uid 94388):
     Filesystem    used   quota   limit   grace   files   quota   limit   grace
          /home  418.6G      1T   1.25T       - 1050591       0       0       -
Disk quotas for group thing (gid 2100):
     Filesystem    used   quota   limit   grace   files   quota   limit   grace
          /home  841.7G      0k      0k       - 1262549       0       0       -
      

Note that in the group section a quota of 0k is shown. This actually means that there is no limit.

Accessing Home Directories via CIFS

User home directories can be accessed from campus networks (or for external clients via the VPN client: VPN client as:

\\hydra5\your-user-name
      

if the domain is not set to be LUNET then ensure that it is.

For Macintosh or Linux users, choose "Connect to Server" and then enter:

smb://hydra5/your-user-name
      

and ensure when prompted for a password (the details vary between versions of Linux and OSX) that the Domain is set to LUNET. Below are examples for SUSE.

Create a server connection

Username, password, domain

/tmp

The /tmp area is local to each compute node. It is not recommended to utilise this space.

Scratch Space

The Lustre scratch space is a high-performance, global filesystem intended for use as working storage by running jobs. It is not intended for saving results between jobs.

To use the scratch space, we suggest you do something similar to this within your run scripts:

mkdir /scratch/tmp/
job.$SLURM_JOB_ID
export TMPDIR=/scratch/tmp/job.$SLURM_JOB_ID
      

The first line creates a directory for use by your job. The second sets the TMPDIR environmentvariable which it is standard for unix programs to set where they keep their temporary files.

Within /scratch/tmp, files and directories older than 20 days are deleted automatically.

More details are available in the sections on SLURM .

Transferring Data

There are two ways to effect this. Either mount the hydra file system as noted above, or use secure file transfer tools as detailed in File Transfer

Archiving and Curation of Data

Many research councils require that data related to publications be conserved to facilitate reproducibility of results and to back papers. The same is true now of many journals. This may require that your archive data, but please take not of the section below on Data Export

Please note that the HPC Service does not offer long-term data storage and does not offer a service suitable for the public access of data sets. You are advised to make provision for copying critical data to suitable locations for permanent curation and the pages on Archiving Data provide some information on how to do this.

If you have requirements of this type which cannot be met by other, existing services within the University then please contact the support team to see if it can help. See Contacting Support .

Security

You are recommended to look at how Linux deals with permissions on files via chmod and chown to ensure that your files are visible only to those you wish them to be visible too. By default they are visible to everyone.

For example, chmod -R og-rwx directoryname will set the directory directoryname and all files within it it such that noone but you can see inside it, read, execute or modify any of the files.

Alternatively you may wish to modify your home directory via cd; chmod og-rwx ..

Files in the .ssh are subject to special requirements and should not be made globally readable even if your other files are.

For details of chmod see Wikipedia

Data Export

Please be aware that some data may have restrictions, e.g. it may not be made available outside campus, the UK, Europe. Please check carefully with your industrial and scientific partners before making data available.

Data Import

Please be aware that some data may not be suitable for use on a shared system, e.g. medical data which is not anonymised, or commercially sensitive data. Even scientific data may have restructions. Please check carefully with your industrial and scientific partners before copying data onto the system.