GitLab Continuous Integration

BEAR GitLab provides a CI (Continuous Integration) framework that allows users to undertake automated tasks when code is committed to a GitLab project. These tasks can include code validation, running test cases, or even building packages for deployment. The CI framework is defined in a file called .gitlab-ci.yml, which lives in the root directory of the GitLab project repository.
Within GitLab, this feature is provided by GitLab runners. Whilst researchers are able to setup their own runners outside of the BEAR environment, we provide several shared runners that are available for all BEAR GitLab projects to use. As these runners are included as part of the service, we ensure that they are monitored for availability, providing a reliable environment for CI tasks.

Using the shared runners

Initial project config

1. To use the BEAR GitLab shared runners, they first need to be enabled within the project in GitLab. From the GitLab web interface, select the "Settings" item from the left hand menu and chose "CI/CD":
   

2. Scroll down until you find the "Shared Runners" section and click the "Enable shared Runners" button:
   

You've now enabled the shared runners for your repository. You'll need to do this for each git repository you want to use CI for.

To actually do something as part of CI, you now need to define the config.

Enabling CI tests

A GitLab CI workflow is enabled by creating a file called .gitlab-ci.yml at the root of the repository. It is important to note that to use the shared runners, you need to include the tags keyword, telling GitLab which runner to use. This can be done by either adding tags to the default: section of the config file, or by adding it to each task that you want to run on the shared runners. The tag to use is bear-gitlab-runners.
See the following file for an example of a .gitlab-ci.yml file that uses the shared runners:

.gitlab-ci.yml

---
default:
  image: python:3.11
  tags:
    - bear-gitlab-runners

stages: # (1)!
  - linting
  - testing

before_script: # (2)!
  - python -V

flake8:
  stage: linting
  script:
    - pip install flake8
    - flake8 --version
    - flake8 -v

test:
  stage: testing
  script:
    - pip install -r requirements.txt
    - python get_globus_data.py --help

1. Including the stages: section is optional but it is a good practice to define the stages of your pipeline. "Stages" provide a logical grouping of tasks and control their execution order. In this example, we have two stages: linting and testing.

   Tasks within a stage can run in parallel, but stages themselves are executed sequentially. This means that all tasks in the linting stage will run before any tasks in the testing stage. If the linting stage fails, the testing stage will not run.

2. The before_script: section is optional and defines commands that will run before any task in the pipeline. In this case, it checks the Python version.

By default, the CI will start by running a git fetch command to pull either main/master or the relevant branch. This will take place whenever code is pushed to the repo. In the above example we perform a sequential two-stage pipeline with both tasks running a Python 3.11 image (from Docker Hub).
The first task, named "flake8", has three steps: install flake8; print the flake8 version, and; run the flake8 command in verbose mode. (Flake8 is a tool used for checking the style and basic syntax of Python code.)
The second task, named "test", has two steps: install the required Python packages from a requirements.txt file, and; run a test Python script called get_globus_data.py with the --help option.

Accessing the test reports

To access a test report from a CI pipeline, start by viewing the Pipelines page in the CI/CD menu:

This will show you the pipelines run in the repository, with the most recent one listed at the top. Selecting the pipeline will take you to a page with details about the stages of the pipeline.

Finally, you can select an individual stage to see the report of that stage of the pipeline, which will help you to debug any failures.

Further information

For more help see the GitLab runners documentation and the general GitLab CI/CD documentation.