GitLab Continuous Integration¶
BEAR GitLab provides a CI (Continuous Integration) framework that allows users to undertake automated tasks when code is committed to a GitLab project. These tasks can
include code validation, running test cases, or even building packages for deployment. The CI framework is defined in a file called .gitlab-ci.yml
, which lives in
the root directory of the GitLab project repository.
Within GitLab, this feature is provided by GitLab runners. Whilst researchers are able to setup their own runners outside of the BEAR environment, we provide several
shared runners that are available for all BEAR GitLab projects to use. As these runners are included as part of the service, we ensure that they are monitored for
availability, providing a reliable environment for CI tasks.
Using the shared runners¶
Initial project config¶
-
To use the BEAR GitLab shared runners, they first need to be enabled within the project in GitLab. From the GitLab web interface, select the "Settings" item from the left hand menu and chose "CI/CD":
-
Scroll down until you find the "Shared Runners" section and click the "Enable shared Runners" button:
You've now enabled the shared runners for your repository. You'll need to do this for each git repository you want to use CI for.
To actually do something as part of CI, you now need to define the config.
Enabling CI tests¶
A GitLab CI workflow is enabled by creating a file called .gitlab-ci.yml
at the root of the repository. It is important to note that to use the shared runners, you need to include the tags
keyword, telling GitLab which runner to use. This can be done by either adding tags
to the default:
section of the config file, or by adding it to each task that you want to run on the shared runners. The tag to use is bear-gitlab-runners
.
See the following file for an example of a .gitlab-ci.yml
file that uses the shared runners:
---
default:
image: python:3.11
tags:
- bear-gitlab-runners
stages: # (1)!
- linting
- testing
before_script: # (2)!
- python -V
flake8:
stage: linting
script:
- pip install flake8
- flake8 --version
- flake8 -v
test:
stage: testing
script:
- pip install -r requirements.txt
- python get_globus_data.py --help
-
Including the
stages:
section is optional but it is a good practice to define the stages of your pipeline. "Stages" provide a logical grouping of tasks and control their execution order. In this example, we have two stages:linting
andtesting
.Tasks within a stage can run in parallel, but stages themselves are executed sequentially. This means that all tasks in the
linting
stage will run before any tasks in thetesting
stage. If thelinting
stage fails, thetesting
stage will not run. -
The
before_script:
section is optional and defines commands that will run before any task in the pipeline. In this case, it checks the Python version.
By default, the CI will start by running a git fetch
command to pull either main
/master
or the relevant branch. This will take place whenever code is pushed to the repo.
In the above example we perform a sequential two-stage pipeline with both tasks running a Python 3.11 image (from Docker Hub).
The first task, named "flake8", has three steps: install flake8; print the flake8 version, and; run the flake8
command in verbose mode.
(Flake8 is a tool used for checking the style and basic syntax of Python code.)
The second task, named "test", has two steps: install the required Python packages from a requirements.txt
file, and; run a test Python script called get_globus_data.py
with the --help
option.
Accessing the test reports¶
To access a test report from a CI pipeline, start by viewing the Pipelines page in the CI/CD menu:
This will show you the pipelines run in the repository, with the most recent one listed at the top. Selecting the pipeline will take you to a page with details about
the stages of the pipeline.
Finally, you can select an individual stage to see the report of that stage of the pipeline, which will help you to debug any failures.
Further information¶
For more help see the GitLab runners documentation and the general GitLab CI/CD documentation.