BEAR Data Transfer¶
If you need to transfer large amounts of research data with either external people or organisations, there are a number of BEAR services available to support this. In all cases, when transferring data, you should take appropriate measures to ensure that sensitive data is protected or encrypted prior to the transfer being undertaken.
Globus Connect Server¶
Globus Connect Server is a tool used for transferring multi-TB data sets. A data transfer node (DTN) is required at each end for the transfer to work. The data transfer is usually controlled using the Globus web service which automatically manages background data transfers to be completed. The BEAR Globus endpoint is directly connected to the Research Data Store (BEAR RDS) storage facility. Globus is used in a number of large international scientific organisations, for example ORNL, CERN, Sanger, Crick, Archer (RDF) and so is well suited for large inter-institutional data transfers.
It is also possible to use Globus Connect Personal on other devices (such as laptops) to transfer data quickly to and from the Research Data Store. Note that we recommend you are not connected to the University’s Remote Access Service (VPN) when you try to transfer data using Globus Connect Personal.
Access to Globus is normally via the Globus Web App. Any researcher registered on a BEAR project with Research Data Store (RDS) facility can use the Globus end point to transfer data without prior registration.
Note
Globus Connect Server is the preferred solution for large scale data transfers as the transfer can occur in the background with integrity checks built in to ensure successful transfer of the data.
Info
The Birmingham endpoint currently only supports encrypted transfers, if the remote endpoint does not support encrypted transfers then your transfer may fail.
Please see our quick start guide on Using Globus.
For sharing data with external collaborators please see the External User (Globus) page.
Command line tools (rsync, sftp, scp)¶
Other command line tools may be necessary to transfer data from some organisations. Where this is required, it is preferable that the data transfer is done from a system which directly accesses the Research Data Store. For example on BlueBEAR login nodes.
Warning
It is not possible to perform transfers using the above methods from BlueBEAR compute nodes (e.g. in a batch job). Whilst computational work is not permitted on BlueBEAR login nodes there are specific exemptions for these data transfer tools.