Using Globus¶

Globus is the preferred tool for transferring large amounts of research data with external partners/research organisations. Both Research Data Store storage are accessible using Globus and any researcher registered on a BEAR project with storage will automatically have access to Globus for data transfers.

Whilst Globus can be used to transfer data to other Data Transfer Nodes, this guide is focussed on getting started using the BEAR Globus endpoint.

In addition to using the Globus Web App, the Globus-CLI (command line tools) are also available on BlueBEAR login nodes.

Getting started using Globus¶

Globus is a data transfer tool which allows large amounts of data to be scheduled for transfer and is typically used for moving data between research organisations. As part of the BEAR services, we offer a Globus endpoint/data transfer node (DTN) which enables BEAR storage to be accessed via Globus services. When the data transfer occurs, unlike a drag and drop in e.g. Windows, the transfer takes place directly between two Globus DTNs. Parallel streams are used to make the data transfer significantly faster and ensures the transfer is completed successfully in the background.

To access the DTN, you will need to be registered on a BEAR project (see BEAR Project Registration). Globus works by creating data transfer control channels via the web services, which directly connect the DTN systems to perform the data transfer reliably and in the background. If you are doing a large transfer it may take some time to complete but closing the Globus Web App in your browser will not stop the transfer as this happens directly between the DTN systems.

Logging In¶

The login process uses the University of Birmingham’s single sign-on (SSO) service and is connected to Globus via a broker called CILogon. When you access the service, you should only enter your University username and password at the University single sign-on portal, all authentication beyond this is done using secure tokens passed by your web-browser. When you first use Globus, you will be prompted to authorise passing of these credentials, which will create and link a Globus account to your University account.

To access Globus, you should browse to Globus Web App.

The steps below show you how to login to Globus for the first time and how to perform this account linking.

When visiting the Globus site, you will be asked to choose how to login:
Search for “University of Birmingham” in the “Look-up your organisation” box:

(Please ensure that you select “University of Birmingham” and NOT “University of Birmingham Development IDP”.)
Globus uses federated authentication via a broker called CILogon, when you click “Continue” you will be redirected via CILogon to the University’s SSO web page:

Use your normal University username and password (ADF), you will then be redirected back to the Globus website:

(If you have previously used Globus at another organisation then you can “Link to an existing account”, otherwise just click the “Continue” button.)
For you to use Globus, you need to confirm you are happy for your email address etc. to be passed to Globus to create an account:

(You can opt in to receive Globus email updates if you wish.)
Once you click continue you will see:

You need to click Allow to permit Globus to transfer data files.

Once you have logged in for the first time, you will not need to link your account again and usually the web browser will remember your preferred organisation to sign in via.

Accessing the BEAR endpoint¶

To transfer data to/from Research Data Store storage, you need to connect to the “University of Birmingham BEAR” endpoint.

Once logged into Globus, select the “File Manager” icon:
In the “Collection” box start search for “bear”:

In this example, you can see “University of Birmingham BEAR” as the second item found, when you click on this endpoint, you should automatically be connected to the BEAR endpoint.
By default you will start in your BlueBEAR home directory (this may be empty if you are only a user of RDS storage):

To access other parts of the storage you can type the path to your RDS folder in the “Path” box, e.g. /rds/projects/2015/XXX, noting that this is the “BEAR Linux path” rather than the path you might use on Windows or macOS. (Please refer to the Connecting to the RDS via Linux page for information on locating the BEAR Linux path for a given project.)

Transferring data using Globus¶

When you transfer data using the Globus web application, this is a scheduled data transfer that happens in the background. You can transfer data both from and to Birmingham’s DTN.

To start a data transfer, select the File Manager in the Globus web application and connect to the BEAR DTN. To the right of the window, you should see a “Transfer or Sync to…” link. When you click this, the window should split:
You can then click the “Transfer or sync to…” box on the right which will allow you to select another Globus endpoint. Search for the Globus DTN you are wanting to transfer data from/to in the “Collection” box. In the following example we are using the ESnet test endpoint at CERN – this is a public endpoint which can be used for testing downloads from another endpoint but you cannot transfer to it. You may wish to try using the "Shared EMBL-EBI public endpoint" to do test transfers as the ESnet test endpoint currently fails:

Select the file(s) that you wish to transfer either from Birmingham on the left or to Birmingham on the right (hold the CTRL key to select multiple files) Note: the "Shared EMBL-EBI public endpoint" is read-only meaning you can only transfer data from them to Birmingham. You should then click the “Start” button under the right hand column to indicate you want to transfer from "Shared EMBL-EBI public endpoint" to BEAR. This will then schedule the data transfer.
You can check on the progress of the transfer by selecting “Activity” on the left hand tool bar, depending on your account settings, you may also receive an email once the data transfer has completed:

In this example you can see one completed transfer and one transfer which has just started.

Note

When accessing Globus endpoints, you may need to use local credentials to access the endpoint, for example if you are transferring data from Diamond Light Source, you may be asked to enter your Diamond Light Source username and password:

Globus allows you to share directories/folders with external users without them needing a University account and this sharing facility is enabled on the BEAR endpoint.

For further information we recommend following the Globus documentation: How To Share Data Using Globus

Note

Please note that if the person you are sharing data with has never used Globus before, we recommend that you ask them to login to Globus first. Whilst they may receive the sharing email from Globus, they may get “Access Denied” errors to the folder after login. This is due to the way identities sometimes appear to Globus. If this happens, once your collaborator has logged into a Globus account, you need to edit the permissions on the share. You should enter the email address in the box and click “Search” rather than “Add”. This should then find the Globus account and access should then be granted to the collaborator.

Warning

When you share a folder with an external partner, they inherit your access permissions for the entire folder tree and so will be able to read any file you can access. We therefore strongly recommend that you do not share the “top” of your storage folder but create a folder specifically for sharing.

Information for External (non-UoB) users¶

The following page covers how non-UoB users can access shared data: Globus Guide for External (non-UoB) Users

Further Assistance¶

If you need further help, please raise an Other BEAR Request ticket on the Service Desk.

If you need to report a fault with the service, please raise a Report a Fault with BEAR ticket on the Service Desk.