Run Your Dockerized Tests in Gitlab CI-CD Pipeline

Test Dockerization, Run and Report: Part-2 | Expertise: Genin

You may want to run your dockerized tests as it is on your CI/CD pipeline to verify any change introduced by your developers.

In this article, we will use the Docker Compose image along with the Docker-In-Docker service to run our dockerized tests in the Gitlab CI/CD pipeline.

Note: As a thumb rule Docker in Docker image should be avoided for reasons briefly articulated in this article. This article merely shows how we can use it to our advantage.

Gitlab + Docker

Below is a quick intro to CI/CD Pipelines.

Prerequisites:
Gitlab Account
Dockerized Test Project

You may view my previous article on how to dockerize a Test Automation project or simply visit the Gitlab project used in this article.

Step 1: Add a .gitlab-ci.yml file to your project

We will use this .gitlab-ci.yml file configure specific instructions for our GitLab CI/CD pipeline.

#Use the official docker image.
docker-test:
  image: docker/compose:latest
  stage: test
  services:
    - docker:dind

  #Instructions to run tests
  script:
    - docker-compose up -d --build

  #Run this job in a branch where a Dockerfile exists
  rules:
    - if: $CI_COMMIT_BRANCH
      exists:
        - Dockerfile
        - docker-compose.yml

Step 2: Add a script.sh to copy test results from the running docker container

The below script will first try to find the test container-id and then issue a copy command.

apk add bash
var=$(docker ps -alq)
export CONTAINER_ID="$var"
echo $CONTAINER_ID
docker cp $CONTAINER_ID:./app .
docker cp $CONTAINER_ID:./jar .

Step 3: Modify your .gitlab-ci.yml file to run the earlier added script.

Add the below lines under script:

#Instructions to run tests
script:
  - docker-compose up -d --build
  - docker wait test-run
  - chmod +x ./script.sh
  - ./script.sh

Step 4: Configure Junit report to display Test Results in Gitlab CI/CD pipeline

Add the below steps in .gitlab-ci.yml. Modify the report path wherever your JUnit reports are available.

artifacts:
  reports:
    junit:
      - jar/test-output/junitreports/TEST-*.xml

Step 5: Push the code with .gitlab-ci.yml

It should run your docker container which contains your tests.

Dockerized Tests Status: Running

Once completed, you can navigate to the Tests sections to view the test results.

Dockerized Tests: Test Results

Bonus: Gitlab has very limited features to run tests and then fail the pipeline following a test-run failure.

However, we can write our own custom logic to achieve this. In the below example I have used the TestNG test runner to run my tests. TestNG generates various files including a testng-failed.xml in case of test failures. We can check the presence of this file to fail the pipeline.

variables:
  TESTNG_FAILED_XML: jar/test-output/testng-failed.xml
script:
  - docker-compose up -d --build
  - docker wait test-run
  - chmod +x ./script.sh
  - ./script.sh
  - |
    if [[ -f "$TESTNG_FAILED_XML" ]]; then
      echo "Test failures observed as TestNG-Failed file exists."
      exit 1
    else
      echo "All tests passed Successfully."
    fi

Links:

Link to my GitLab project used in this example.
The project contains a lightweight framework capable of running both UI and API tests.

Suryajit / my-blog-topics