None of the HPC user filestores are backed up. It is the responsibility of users to ensure that they keep copies of their critical files somewhere safe.
User home directories are mounted on all compute nodes via Lustre and can be used in run scripts or programs without any changes to directory paths.
However, if you are using heavy input or output you may wish to use the
scratch space which is mounted on each node under
Directories are not backed up. It is the user's responsibility to ensure they keep copies of any critical files elsewhere.
Users have a 1TB quota set on their home directory. This is to ensure that
runaway jobs do not fill up the disk space and so interrupt the service
for other users. To check your current disk usage you can run the
lfs quota command:
lfs quota -h
/home to see the quota on the home directory area. The
following example output is for a fictitious user. It also shows how much
data people in your group (normally department) is using across all users
in that group.
Disk quotas for user blah (uid 94388): Filesystem used quota limit grace files quota limit grace /home 418.6G 1T 1.25T - 1050591 0 0 - Disk quotas for group thing (gid 2100): Filesystem used quota limit grace files quota limit grace /home 841.7G 0k 0k - 1262549 0 0 -
Note that in the group section a quota of 0k is shown. This actually means that there is no limit.
User home directories can be accessed from campus networks (or for external clients via the VPN client: VPN client as:
if the domain is not set to be
LUNET then ensure
that it is.
For Macintosh or Linux users, choose "Connect to Server" and then enter:
and ensure when prompted for a password (the details vary between
versions of Linux and OSX) that the
is set to
LUNET. Below are examples for SUSE.
The Lustre scratch space is a high-performance, global filesystem intended for use as working storage by running jobs. It is not intended for saving results between jobs.
To use the scratch space, we suggest you do something similar to this within your run scripts:
mkdir /scratch/tmp/ job.$SLURM_JOB_ID export TMPDIR=/scratch/tmp/job.$SLURM_JOB_ID
The first line creates a directory for use by your job. The second sets
TMPDIR environmentvariable which it
is standard for unix programs to set where they keep their temporary
/scratch/tmp, files and
directories older than 20 days are deleted automatically.
More details are available in the sections on SLURM .
There are two ways to effect this. Either mount the hydra file system as noted above, or use secure file transfer tools as detailed in File Transfer
Many research councils require that data related to publications be conserved to facilitate reproducibility of results and to back papers. The same is true now of many journals. This may require that your archive data, but please take not of the section below on Data Export
Please note that the HPC Service does not offer long-term data storage and does not offer a service suitable for the public access of data sets. You are advised to make provision for copying critical data to suitable locations for permanent curation and the pages on Archiving Data provide some information on how to do this.
If you have requirements of this type which cannot be met by other, existing services within the University then please contact the support team to see if it can help. See Contacting Support .
You are recommended to look at how Linux deals with permissions on files
ensure that your files are visible only to those you wish them to be
visible too. By default they are visible to everyone.
chmod -R og-rwx directoryname will set
directoryname and all
files within it it such that noone but you can see inside it, read,
execute or modify any of the files.
Alternatively you may wish to modify your home directory via
cd; chmod og-rwx ..
Files in the
.ssh are subject to
special requirements and should not be made globally readable even if your
other files are.
For details of
Please be aware that some data may have restrictions, e.g. it may not be made available outside campus, the UK, Europe. Please check carefully with your industrial and scientific partners before making data available.
Please be aware that some data may not be suitable for use on a shared system, e.g. medical data which is not anonymised, or commercially sensitive data. Even scientific data may have restructions. Please check carefully with your industrial and scientific partners before copying data onto the system.