Develop C++ unit testing with Catch2, JUnit, and GitLab CI

Continuous integration (CI) and automated testing are important DevSecOps workflows for software developers to detect bugs early, improve code quality, and streamline their development processes.

In this tutorial, you'll learn how to set up unit testing on a C++ project with Catch2 and GitLab CI for continuous integration. You'll also see how the AI-powered features of GitLab Duo can help. We’ll use an air quality monitoring application as our reference project.

Prerequisites

• Ensure you have CMake installed on your machine.
• A modern C++ compiler such as GCC or Clang is required.
• An API key from OpenWeatherMap - requires signing up for a free account (1,000/calls per day are included for free).

Set up the application for testing

The reference project we’ll be using for demonstrating testing in this blog post is an air quality monitoring application that fetches air quality data from the OpenWeatherMap API based on the U.S zip codes only provided by the user.

Here are the steps to set up the application for testing:

1. Fork the the reference project and clone the fork to your local environment.

2. Generate an API key from OpenWeatherMap and export it into the environment.

export API_KEY="YOURAPIKEY_HERE"

3. Alternatively, you can add the key into your .env configuration, and source it with source ~/.env, or use a different mechanism to populate the environment.

4. Compile and build the project code with the following instructions:

cmake -S . -B build
cmake --build build

5. Run the application using the executable and passing in a U.S zip code (90210 as an example):

./build/air_quality_app 90210

Here’s an example of what running the program will look like in your terminal:

❯ ./build/air_quality_app 90210
Air Quality Index (AQI) for Zip Code 90210: 2 (Fair)

Install Catch2

Now that the application is set up and working, let's start working on adding testing using Catch2. Catch2 is a modern, C++-native testing framework for unit tests.

You can also ask GitLab Duo Chat within your IDE for an introduction to getting started with Catch2 as a C++ testing framework. GitLab Duo Chat will provide getting started steps as well as an example test:

1. First navigate to your project’s root directory and create an externals folder using the mkdir command.

mkdir externals

2. There are several ways to install Catch2 via its CMake integration. We will use the option of installing it as a submodule and including it as part of the source code to simplify dependency management. To add Catch2 to your project in the externals folder:

git submodule add https://github.com/catchorg/Catch2.git externals/Catch2
git submodule update --init --recursive

3. Update CMakeLists.txt to include Catch2’s directory as a subdirectory. This allows CMake to find and build Catch2 as a part of our project.

# Assuming Catch2 in externals/Catch2
add_subdirectory(externals/Catch2)

4. Create a tests.cpp file in your project root to write our tests to:

touch tests.cpp

5. Update CMakeLists.txt Link against Catch2. When defining your test executable in CMake, link it against Catch2:

# Add tests executable and link it to Catch2
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)

Structure the project for testing

Before we start writing our tests, we should separate our application logic into separate files in order to maintain and test our code more efficiently. At the end of this section we should have:

main.cpp containing only the main() function and application setup
includes/functions.cpp containing all functional code such as API calls and data processing: 
includes/functions.h containing the declarations for the functions defined in functions.cpp.  It needs to define the preprocessor macro guards, and include all necessary headers. 

Apply the following changes to the files:

1. main.cpp

#include <iostream>
#include "functions.h"

int main(int argc, char* argv[]) {
   if (argc < 2) {
       std::cerr << "Usage: " << argv[0] << " <Zip Code>" << std::endl;
       return 1;
   }

   std::string zipCode = argv[1];
   std::string apiKey = getApiKey();
   if (apiKey.empty()) {
       std::cerr << "API key not found." << std::endl;
       return 1;
   }

   auto [lat, lon] = geocodeZipcode(zipCode, apiKey);
   if (lat == 0 && lon == 0) {
       std::cerr << "Failed to geocode zipcode." << std::endl;
       return 1;
   }

   std::string response = fetchAirQuality(lat, lon, apiKey);
   std::string airQualityInfo = parseAirQualityResponse(response);

   std::cout << "Air Quality Index for Zip Code " << zipCode << ": " << airQualityInfo << std::endl;

   return 0;
}

2. Create a functions.h: in the includes folder:

#ifndef FUNCTIONS_H
#define FUNCTIONS_H

#include <string>
#include <utility>
#include <vector>

// Declare the function prototype
std::string httpRequest(const std::string& url);
bool loadEnvFile(const std::string& filename);
std::string getApiKey();
std::pair<double, double> geocodeZipcode(const std::string& zipCode, const std::string& apiKey);
std::string fetchAirQuality(double lat, double lon, const std::string& apiKey);
std::string parseAirQualityResponse(const std::string& response);

#endif

3. Create a functions.cpp in the includes folder:

#include "functions.h"
#include <fstream>
#include <elnormous/HTTPRequest.hpp>
#include <nlohmann/json.hpp>
#include <iostream>
#include <cstdlib> // For getenv

std::string httpRequest(const std::string& url) {
   try {
       http::Request request{url};
       const auto response = request.send("GET");
       return std::string{response.body.begin(), response.body.end()};
   } catch (const std::exception& e) {
       std::cerr << "Request failed, error: " << e.what() << std::endl;
       return "";
   }
}
std::string getApiKey() {
   const char* envApiKey = std::getenv("API_KEY");
   if (envApiKey) {
       return std::string(envApiKey);
   }
   // If the environment variable is not set, fallback to the config file
   std::ifstream configFile("config.txt");
   std::string line;
   if (getline(configFile, line)) {
       return line.substr(line.find('=') + 1);
   }
   return "";
}

std::pair<double, double> geocodeZipcode(const std::string& zipCode, const std::string& apiKey) {
   std::string url = "http://api.openweathermap.org/geo/1.0/zip?zip=" + zipCode + ",US&appid=" + apiKey;
   std::string response = httpRequest(url);
   try {
       auto json = nlohmann::json::parse(response);
       if (json.contains("lat") && json.contains("lon")) {
           double lat = json["lat"];
           double lon = json["lon"];
           return {lat, lon};
       } else {
           std::cerr << "Geocode response missing 'lat' or 'lon' fields: " << response << std::endl;
       }
   } catch (const nlohmann::json::parse_error& e) {
       std::cerr << "Failed to parse geocode response: " << e.what() << " - Response: " << response << std::endl;
   }
   return {0, 0};
}

std::string fetchAirQuality(double lat, double lon, const std::string& apiKey) {
   std::string url = "http://api.openweathermap.org/data/2.5/air_pollution?lat=" + std::to_string(lat) + "&lon=" + std::to_string(lon) + "&appid=" + apiKey;
   std::string response = httpRequest(url);
   return response;
}

std::string parseAirQualityResponse(const std::string& response) {
   try {
       auto json = nlohmann::json::parse(response);
       if (json.contains("list") && !json["list"].empty() && json["list"][0].contains("main")) {
           int aqi = json["list"][0]["main"]["aqi"];
           std::string aqiCategory;
           switch (aqi) {
               case 1:
                   aqiCategory = "Good";
                   break;
               case 2:
                   aqiCategory = "Fair";
                   break;
               case 3:
                   aqiCategory = "Moderate";
                   break;
               case 4:
                   aqiCategory = "Poor";
                   break;
               case 5:
                   aqiCategory = "Very Poor";
                   break;
               default:
                   aqiCategory = "Unknown";
                   break;
           }
           return std::to_string(aqi) + " (" + aqiCategory + ")";
       } else {
           return "No AQI data available";
       }
   } catch (const std::exception& e) {
       std::cerr << "Failed to parse JSON response: " << e.what() << std::endl;
       return "Error parsing AQI data";
   }
}

4. Now that we have separated the source files, we also need to update our CMakeLists.txt to include functions.cpp in the add_executable() calls:

cmake_minimum_required(VERSION 3.14)
project(air-quality-app)

# Set the C++ standard for the project
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

include_directories(${CMAKE_SOURCE_DIR}/includes)

# Define the main program executable
add_executable(air_quality_app main.cpp includes/functions.cpp)

# Assuming Catch2 in externals/Catch2
add_subdirectory(externals/Catch2)

# Add tests executable and link it to Catch2
add_executable(tests tests.cpp includes/functions.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)

To verify that the changes are working, regenerate the CMake configuration and rebuild the source code with the following commands. The build will take longer now that we're compiling Catch2 files.

rm -rf build # delete existing build files
cmake -S . -B build 
cmake --build build  

You should be able to run the application without any errors.

./build/air_quality_app 90210

Write tests in Catch2

Catch2 tests are made up of macros and assertions. Macros in Catch2 are used to define test cases and sections within those test cases. They help in organizing and structuring the tests. Assertions are used to verify that the code behaves as expected. If an assertion fails, the test case will fail, and Catch2 will report the failure.

Let’s review a basic test scenario for an addition function to understand. Note: This test is read-only, as an example.

int add(int a, int b) {
   return a + b;
}

TEST_CASE("Addition works correctly", "[math]") {
   REQUIRE(add(1, 1) == 2);  // Test passes if 1+1 equals 2
   REQUIRE(add(2, 2) != 5);  // Test passes if 2+2 does not equal 5
}

• Each test begins with the TEST_CASE macro, which defines a test case container. The macro accepts two parameters: a string describing the test case and optionally a second string for tagging the test for easy filtering.
• Tests are also composed of assertions, which are statements that check if conditions are true. Catch2 provides macros for assertion that include REQUIRE, which aborts the current test if the assertion fails, and CHECK, which logs the failure but continues with the current test.

Prepare to write tests with Catch2

To test the API retrieval functions in our air quality application, we’ll be using mock API requests. Mock API testing is a technique used to test how your application will interact with an external API without making any real API calls. Instead of sending requests to a live API server, we can simulate the responses using predefined data. Mock requests allow us to control the input data and specify exactly what the API would return for different requests, making sure that our tests aren't affected by changes in the real API responses or unexpected data. This also makes it easier for us to simulate and catch different failures.

In our tests.cpp file, let’s define the following function to run mock API requests.

#include "includes/functions.h"
#include <catch2/catch_test_macros.hpp>
#include <string>

// Mock HTTP request function that simulates API responses
std::string mockHttpRequest(const std::string& url) {
   if (url.find("geo") != std::string::npos) {
       // Mock response for geocoding
       return R"({"lat": 40.7128, "lon": -74.0060})"; 
   } else if (url.find("air_pollution") != std::string::npos) {
       // Mock response for air quality
       return R"({"list": [{"main": {"aqi": 2}}]})";
   }
   // Default mock response for unmatched endpoints
   return "{}";
}
// Overriding the actual httpRequest function with the mockHttpRequest for testing
std::string httpRequest(const std::string& url) {
   return mockHttpRequest(url);
}

• This function simulates HTTP requests and returns predefined JSON responses based on the URL given as input.
• It also checks the URL to determine which type of data is being requested based on the functionality of the application (geocoding, air pollution, or forecast data). If the URL doesn’t match the expected endpoint, it returns an empty JSON object.

Don't compile the code just yet, as you'll see a linker error. Since we're overriding the original httpRequest function with our mock function for testing, we'll need a preprocessor macro to enable conditional compilation - indicating which httpRequest function should run when we're compiling tests.

Define a preprocessor macro for testing

Because we’ve overridden httpRequest in our tests.cpp, we need to exclude that code from functions.cpp when we’re testing. When building tests, we may need to ensure that certain parts of our code behave differently or are excluded. We can do this by defining a preprocessor macro TESTING which enables conditional compilation, allowing us to selectively include or exclude code when compiling the test target:

We define the TESTING macro in our CMakeLists.txt at the end:

# Define TESTING macro for this target
target_compile_definitions(tests PRIVATE TESTING)

And add the macro wrapper in functions.cpp around the original httpRequest function:

#ifndef TESTING  // Exclude this part when TESTING is defined
std::string httpRequest(const std::string& url) {
   try {
       http::Request request{url};
       const auto response = request.send("GET");
       return std::string{response.body.begin(), response.body.end()};
   } catch (const std::exception& e) {
       std::cerr << "Request failed, error: " << e.what() << std::endl;
       return "";
   }
}
#endif

Regenerate the CMake configuration and rebuild the source code to verify it works.

cmake --build build  

Write the first tests

Now, let’s write some tests for our air quality application.

Test 1: Verify API key retrieval

This test ensures that the getApiKey function retrieves the API key correctly from the environment variable or the configuration file. Add the test case to our tests.cpp:

TEST_CASE("API Key Retrieval", "[api]") {
   // Set the API_KEY environment variable for testing
   setenv("API_KEY", "test_key", 1);
   // Test if the key is retrieved correctly
   REQUIRE(getApiKey() == "test_key");
}

You can verify that this tests passes by rebuilding the code and running the tests:

cmake --build build
./build/tests

Test 2: Geocode the zip code

This test ensures that the geocodeZipcode function returns the correct latitude and longitude for a given zip code using the mock API response function we set up earlier. The geocodeZipcode function is supposed to hit an API that returns geographic coordinates based on a zip code.

In tests.cpp, add this test case for the zip code 90210:

TEST_CASE("Geocode Zip code", "[geocode]") {
   std::string apiKey = "test_key";
   std::pair<double, double> coordinates = geocodeZipcode("90210", apiKey);
   // Check latitude
   REQUIRE(coordinates.first == 40.7128);
   // Check longitude 
   REQUIRE(coordinates.second == -74.0060);
}

The purpose of this test is to verify that the function geocodeZipcode can correctly parse the latitude and longitude from the API response. By hardcoding the expected response, we ensure that the test environment is controlled and predictable.

Test 3: Air quality API test

This test ensures that the fetchAirQuality function correctly fetches air quality data using the mock API response function we set up earlier. It verifies that the function constructs the API request properly, sends it, and accurately parses the air quality index (AQI) from the mock JSON response. This validation helps ensure that the overall process of fetching and interpreting air quality data works as intended.

TEST_CASE("Fetch Air Quality", "[airquality]") {
   std::string apiKey = "test_key";
   double lat = 40.7128;
   double lon = -74.0060;
   std::string response = fetchAirQuality(lat, lon, apiKey);
   // Check the response
   REQUIRE(response == R"({"list": [{"main": {"aqi": 2}}]})");
}

Build and run the tests

To build and compile our application, we'll use the same CMake commands as before:

cmake -S . -B build
cmake --build build

After building, we can run our tests by executing the test binary:

./build/tests

Running this command will execute all defined tests, and you will see output indicating whether each test has passed or failed.

Set up GitLab CI/CD

To automate the testing process each time we push some new code to our repository, let’s set up GitLab CI/CD. Create a new .gitlab-ci.yml configuration file in the root directory.

image: gcc:latest

variables:
 GIT_SUBMODULE_STRATEGY: recursive

stages:
 - build
 - test

before_script:
 - apt-get update && apt-get install -y cmake

compile:
 stage: build
 script:
   - cmake -S . -B build
   - cmake --build build
 artifacts:
   paths:
     - build/

test:
 stage: test
 script:
   - ./build/tests --reporter junit -o test-results.xml
 artifacts:
   reports:
     junit: test-results.xml

This CI/CD configuration will compile both the main application and the test suite, then run the tests, generating a JUnit XML report which GitLab uses to display the test results.

• In before_script, we added an installation for cmake, and git submodule sync --recursive which initializes and updates our submodules (catch2).
• In the test stage, --reporter junit -o test-results.xml specifies that the test results should be treated as a JUnit report which allows GitLab CI to display results in the UI. This is super helpful when you have several tests in your application.

We also need to add an environmental variable with the API_KEY in project settings on GitLab.

Don’t forget to add all new files to Git, and commit and push the changes in a new MR:

git checkout -b tests-catch2-cicd

git add includes/functions.{h,cpp} tests.cpp .gitlab-ci.yml 
git add CMakeLists.txt main.cpp 

git commit -vm “Add Catch2 tests and CI/CD configuration”
git push 

View the test report

After pushing our code changes, we can review the results of our tests in the GitLab UI in the Pipeline view in the Tests tab:

Simulate a test failure

To demonstrate how our UI will handle test failures, we can intentionally introduce a bug into our code and observe the resulting behavior.

Let's modify our parseAirQualityResponse function to introduce an error. We can change the AQI category for an AQI value of 2 from "Fair" to "Poor." This change will cause the related test to fail, allowing us to see the test failure in the GitLab UI.

In functions.cpp, find the parseAirQualityResponse function and modify the switch statement for case 2 to set the Poor value instead of Fair:

               // Intentional bug:
               case 2:
                   aqiCategory = "Poor";
                   break;

In tests.cpp, add a new test case that directly checks the output of the parseAirQualityResponse function. This test ensures that the parseAirQualityResponse function correctly parses and categorizes the air quality data from the mock API response. This function takes a JSON response, extracts the AQI value, and translates it into a human-readable category.

TEST_CASE("Parse Air Quality Response", "[airquality]") {
   std::string mockResponse = R"({"list": [{"main": {"aqi": 2}}]})";
   std::string result = parseAirQualityResponse(mockResponse);
   // This should fail due to the intentional bug
   REQUIRE(result == "2 (Fair)");
}

Commit the changes, and push them into the MR. Open the MR in your browser.

By introducing an intentional bug in this function, we can see how a test failure is reported in GitLab's pipelines UI. We must add, commit, and push the changes to our repository to view the test failure in the pipeline.

Once we've verified this simulated test failure, we can use git revert to roll back that commit.

git revert

Add and test a new feature

Let’s put what you've learned together by creating a new feature in the air quality application and then writing a test for that feature using Catch2. The new feature will fetch the current weather forecast for the provided zip code.

First, we'll define a Weather struct and add the function prototype in our functions.h file (inside the #endif):

struct Weather {
   std::string main;
   std::string description;
   double temperature;
};

Weather getCurrentWeather(const std::string& apiKey, double lat, double lon);

Then, we implement the getCurrentWeather function in functions.cpp. This function calls the OpenWeatherMap API to retrieve the current weather and parses the JSON response. This code was generated using GitLab Duo. If you start typing Weather getCurrentWeather(const std::string& apiKey, double lat, double lon) { to complete the function, GitLab Duo will provide the function contents for you, line by line.

Here's what your getCurrentWeather() function can look like:

Weather getCurrentWeather(const std::string& apiKey, double lat, double lon) {
   std::string url = "http://api.openweathermap.org/data/2.5/weather?lat=" + std::to_string(lat) + "&lon=" + std::to_string(lon) + "&appid=" + apiKey;
   std::string response = httpRequest(url);
   auto json = nlohmann::json::parse(response);
   Weather weather;
   if (!json.is_null()) {
       weather.main = json["weather"][0]["main"];
       weather.description = json["weather"][0]["description"];
       weather.temperature = json["main"]["temp"];
   }
   return weather;
}

And, finally, we update our main.cpp file in the main function to output the current forecast (and converting Kelvin to Celsius for the output):

   Weather currentWeather = getCurrentWeather(apiKey, lat, lon);
   if (currentWeather.main.empty()) {
       std::cerr << "Failed to fetch current weather." << std::endl;
       return 1;
   }

   std::cout << "Current Weather: " << currentWeather.main << ", " << currentWeather.description
       << ", temperature " << currentWeather.temperature - 273.15 << " °C" << std::endl;

We can confirm that our new feature is working by building and running the application:

cmake --build build
./build/air_quality_app 

And we should see the following output or similar in case the weather is different on the day the code is run :)

Air Quality Index for Zip Code 90210: 2 (Poor)
Current Weather: Clouds, broken clouds, temperature 23.2 °C

With all new functionality, there should be testing! We can also write a test to check whether the application is fetching and parsing a weather forecast correctly. This test checks that the function returns a list containing the correct number of forecast entries and that each entry has accurate data regarding time and temperature.

TEST_CASE("Current Weather functionality", "[api]") {
   auto weather = getCurrentWeather("dummyApiKey", 40.7128, -74.0060);
   // Ensure main weather description is not empty
   REQUIRE_FALSE(weather.main.empty());
   // Validate that temperature is a reasonable value
   REQUIRE(weather.temperature > 0); 
}

We’ll also have to update our mockHTTPRequest function in tests.cpp to account for this new test. Modify the if-condition with a new else-if branch checking for the weather string in the URL:

// Mock HTTP request function that simulates API responses
std::string mockHttpRequest(const std::string &url)
{
   if (url.find("geo") != std::string::npos)
   {
       // Mock response for geocoding
       return R"({"lat": 40.7128, "lon": -74.0060})";
   }
   else if (url.find("air_pollution") != std::string::npos)
   {
       // Mock response for air quality
       return R"({"list": [{"main": {"aqi": 2}}]})";
   }
   else if (url.find("weather") != std::string::npos)
   {
       // Mock response for current weather
       return R"({
          "weather": [{"main": "Clear", "description": "clear sky"}],
          "main": {"temp": 298.55}
      })";
   }
   return "{}";
}

And verify that our tests are working by rebuilding and running our tests:

cmake --build build 
./build/tests

All tests should pass, including the new one for Current Weather Functionality.

Optimize tests.cpp with sections

To better organize our tests as the project grows and categorize each functionality, we can use Catch2’s SECTION macro. The SECTION macro allows you to define logically separate test scenarios within a single test case, providing a clean way to test different behaviors or conditions without requiring multiple separate test cases or multiple files. This approach keeps related tests bundled together and also improves test maintainability by allowing shared setup code to be executed repeatedly for each section.

Since some of our functionality is preprocessing data to retrieve information, let’s section our tests as such:

• preprocessing steps:
  • API key validation
  • geocoding validation
• API data retrieval:
  • air pollution retrieval
  • forecast retrieval

Here’s what our tests.cpp will look like if organized by sections:

#include "functions.h"
#include <catch2/catch_test_macros.hpp>
#include <string>

// Mock HTTP request function that simulates API responses
std::string mockHttpRequest(const std::string &url)
{
   if (url.find("geo") != std::string::npos)
   {
       // Mock response for geocoding
       return R"({"lat": 40.7128, "lon": -74.0060})";
   }
   else if (url.find("air_pollution") != std::string::npos)
   {
       // Mock response for air quality
       return R"({"list": [{"main": {"aqi": 2}}]})";
   }
   else if (url.find("weather") != std::string::npos)
   {
       // Mock response for current weather
       return R"({
          "weather": [{"main": "Clear", "description": "clear sky"}],
          "main": {"temp": 298.55}
      })";
   }
   return "{}";
}

// Overriding the actual httpRequest function with the mockHttpRequest for testing
std::string httpRequest(const std::string &url)
{
   return mockHttpRequest(url);
}

// Preprocessing Steps
TEST_CASE("Preprocessing Steps", "[preprocessing]") {
   SECTION("API Key Retrieval") {
       // Set the API_KEY environment variable for testing
       setenv("API_KEY", "test_key", 1);
       // Test if the key is retrieved correctly
       REQUIRE_FALSE(getApiKey().empty());
   }

   SECTION("Geocode Functionality") {
       std::string apiKey = "test_key";
       std::pair<double, double> coordinates = geocodeZipcode("90210", apiKey);
       // Check latitude
       REQUIRE(coordinates.first == 40.7128);
       // Check longitude 
       REQUIRE(coordinates.second == -74.0060);
   }
}

// API Data Retrieval
TEST_CASE("API Data Retrieval", "[data_retrieval]") {
   SECTION("Air Quality Functionality") {
       std::string apiKey = "test_key";
       double lat = 40.7128;
       double lon = -74.0060;
       std::string response = fetchAirQuality(lat, lon, apiKey);
       // Check the response
       REQUIRE(response == R"({"list": [{"main": {"aqi": 2}}]})");
   }

   SECTION("Current Weather Functionality") {
       auto weather = getCurrentWeather("dummyApiKey", 40.7128, -74.0060);
       // Ensure main weather description is not empty
       REQUIRE_FALSE(weather.main.empty());
       // Validate that temperature is a reasonable value
       REQUIRE(weather.temperature > 0);
   }
}

Rebuild the code and run the tests again to verify.

cmake --build build 
./build/tests

Next steps

In this post, we covered how to integrate unit testing into a C++ project using Catch2 testing framework and GitLab CI/CD and set up basic tests for our reference air quality application project.

To explore these concepts further, you can check out the Catch2 documentation and GitLab's Unit test report examples documentation.

For an advanced async exercise, you could build upon this project by using GitLab Duo to implement a feature that retrieves and analyzes historical air quality data and add code quality checks into the CI/CD pipeline. Happy coding!