Introduction to Continuous Testing and Cloud Test Execution on AWS EC2 using GitLab

In this article, we will learn what Continuous testing is and how we can implement cloud test execution in AWS EC2 using GitLab CI.

Let’s first understand the basic concepts of continuous integration, continuous delivery and continuous testing.

Continuous Integration, Continuous Delivery and Continuous testing

CI/CD introduces ongoing automation and continuous monitoring throughout the lifecycle of a release, from integration and testing phases to delivery and deployment.

Continuous Integration is a coding philosophy and set of practices that drive development teams to implement small changes and check in code to version control repositories frequently.

Continuous Delivery picks up where continuous integration ends. CD automates the delivery of applications to selected infrastructure environments. Most teams work with multiple environments other than the production, such as development and testing environments, and CD ensures there is an automated way to push code changes to them.

Continuous testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible. Continuous testing is often implemented as a set of automated regression, performance, and other tests that are executed in the CI/CD pipeline.

Continuous integration and continuous delivery require continuous testing because the objective is to deliver quality applications and code to users.

Cloud Test Execution

Cloud testing refers to the verification of software quality on the cloud. Essentially, this translates to running manual and automation testing on a cloud computing environment with the requisite infrastructure.
This makes the entire testing process online, sparing QAs the hassle of limited device/browsers/OS coverage, geographical limitations, extensive setup and maintenance processes.

Why is Cloud Testing needed?

With cloud automated testing, the process is simplified for the following reasons:

• Cloud testing platforms are set up to facilitate tests for multiple users and teams on multiple devices simultaneously.
• With Cloud Testing, testing becomes faster, easier, and infinitely more manageable.
• With automated testing and parallel testing, testing in the cloud allows QAs to accelerate test execution and results significantly.
• The hardware & software resources that provide access to cloud-based testing automation tools are accessible 24/7.

Implementing cloud test execution

This article does not provide steps to set up AWS EC2 Linux. There are plenty of documents and YouTube tutorials out there which you can follow. If you need a boiler-plate code base, you can use my Selenium project given here.

Now that we understood the basic concepts, let’s dive right in and implement what we learnt. For this project, we will use GitLab as our CI and code repository. We will trigger tests from GitLab using Gitlab Runner. When a test is triggered from GitLab, it will execute in our local machine or in AWS EC2 Linux as specified.

Test Infrastructure and Execution Flow

A GitLab CI/CD pipeline consists of two major components: A .gitlab-ci.yml file describing a pipeline’s jobs, and a Gitlab Runner, an application that executes the pipeline jobs. We will use the below simple .gitlab-ci.yml file for this project. This file should be pushed to your project code in GitLab. GitLab will recognize this file automatically and build a job pipeline.

test:
stage: test
script:
— mvn test -DsuiteFile=”$xml”

My project uses Maven as the build automation tool and hence the last line. You can change this based on the tools you are using in your project.

Once you have your EC2 set up, next step is to install GitLab Runner on EC2 and register it to the GitLab repository. Follow the below steps to get your EC2 test-ready.

1: Add the Official GitLab Repository

First add the official GitLab Repository using below command, to check latest Gitlab Repository visit the official GitLab Runner page

curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash

2: Install GitLab Runner on Amazon Linux 2

Run below command to install latest GitLab Runner on Amazon Linux 2

sudo yum install gitlab-runner

use below commands to install specific version of GitLab Runner on Amazon Linux 2

yum list gitlab-runner --showduplicates | sort -r
sudo yum install gitlab-runner-10.0.0-1

Command to check GitLab Runner version

sudo gitlab-runner -version

Check the status of the runner

sudo gitlab-runner status

Output:

Runtime platform                                    arch=amd64 os=linux pid=29368 revision=e95f89a0 version=13.4.1
gitlab-runner: Service is running!

3: Now Start the GitLab Runner

sudo gitlab-runner start

4: Grant sudo Permission to GitLab Runner User

After install GitLab Runner you will see gitlab-runner user in /home directory

cd /home
ls

Output:

gitlab-runner

To grant sudo permission to gitlab-runner user, ope the visudo file

sudo visudo

Add the gitlab-runner user in sudoers group and set NOPASSWD as shown below

gitlab-runner ALL=(ALL:ALL) ALL
gitlab-runner ALL=(ALL) NOPASSWD: ALL

5: Register GitLab Runner to GitLab on Amazon Linux 2

- Run the below command in Terminal

sudo gitlab-runner register
Output: Enter the GitLab instance URL

- Open your Gitlab Repo > Settings > CI/CD > Runners > copy the url given under the step ‘2. Register the runner with this URL:'

- Paste the URL in Terminal and hit Enter
Output: Enter the registration token

- Copy the token from the repo given under the step '2. And this registration token:'

- Paste the token in Terminal and hit Enter
Output: Enter a description for the runner:

- Give your name + AWS Runner (Example: Afsal’s AWS Runner). Hit Enter.
Output: Enter tags for the runner (comma-separated):

- Leave it blank and hit Enter.
Output: Enter an executor:

- Type “Shell“ and hit Enter.
Output: Runner registered successfully.

- Run ./gitlab-runner.exe start
Output: Runtime platform…

- Go back to the project page and refresh. Under Runners section, under Available specific runners, verify the runner you registered is displayed and the icon is green.

Reference: Install GitLab Runner on Windows | GitLab

With that, our GitLab runner set up in AWS and registration with GitLab CI is complete. Now it’s time to run tests.

Executing a Test

Once the setup is complete, follow the below steps to run a test.

• Open your GitLab repository
• CI/CD > Pipelines > Run pipeline
• Select your branch (master) from the ‘Run for branch name or tag’ dropdown.
• Click ‘Run pipeline’.
  (If you are following my .gitlab-ci.yml mentioned above and want to run a specific test by passing the test xml file as parameter to $xml, Select ‘Variable’ from the ‘Variables’ dropdown and add input variable key and input variable value specific to your project.
  Example: input variable key : xml
  input variable value: testng.xml)
• You can see a pipeline job is triggered with the stages mentioned in your .yml file and your test will be spawned in AWS EC2.

Summary

Everything we’ve done so far was to setup the environment for the CI to work. All the Continuous Integration (CI) will work through the .gitlab-ci.yml file that you created in the project. Now whenever a commit is made on Master branch, specified tests will kick-off.

This process can be complicated to set-up but makes deployments much easier in the end. Your developers will be able to spend less time worrying about things like how their application is deployed, permission issues, testing, and QA, and more time focusing on bringing innovation to your applications.