Getting Started

These notes are from The Construct and was originally for ROS. Since they didn't have a couse on testing with ROS2, I referenced the ROS2 documentation and ChatGPT while doing this to fill in the gaps for ROS2 Testing. I used the docker image osrf/ros:humble-desktop for my environment.

This note will go over:

• Unit Testing
• Node Level Tests
• Integration Tests

Testing Frameworks for ROS2

Testing Framework	Language	Purpose
Google Test	C++	Unit testing with Google Test
pytest	Python	Unit testing for Python-based ROS 2 nodes and library code
launch_testing	Python/C++	Testing ROS 2 launch files
mock, unittest	Python	Unit testing with mock and unittest
ros2-benchmarks	C++/Python	Performance testing in ROS 2
Cucumber, Behave	Python	Behavior-driven testing (BDD)

I did not do any testing with gtest as of 12/25/24 and this was done using unittest.

Unit Tests

Unit tests are mainly to ensure correct functionality of a stand-alone unit or function. This means that they could test a single function, node, or process. Library unit tests are for testing non-ROS2 related issues while ROS2 unit tests are for ROS2 single nodes.

Integration Tests

Integration tests are to ensure correct functionality of the system as a whole. For ROS2, it means starting up multiple nodes and ensuring that they all work as intended.

I will be using the code below for all testing purposes.

#!/usr/bin/env python3

import rclpy
from rclpy.node import Node
from geometry_msgs.msg import Twist
from sensor_msgs.msg import LaserScan
import time

class RobotControl(Node):

    def __init__(self):
        super().__init__('robot_control_node')
        self.vel_publisher = self.create_publisher(Twist, '/cmd_vel', 10)
        self.cmd = Twist()
        self.ctrl_c = False
        self.rate = self.create_rate(10)
        self.create_timer(0.1, self.shutdownhook)

    def publish_once_in_cmd_vel(self):
        """
        Publish a single Twist message to ensure command receipt.
        """
        while not self.ctrl_c:
            if self.vel_publisher.get_subscription_count() > 0:
                self.vel_publisher.publish(self.cmd)
                self.get_logger().info("Command published")
                break
	     else:
                self.rate.sleep()

    def shutdownhook(self):
        self.stop_robot()
        self.ctrl_c = True

    def stop_robot(self):
        self.cmd.linear.x = 0.0
        self.cmd.angular.z = 0.0
        self.publish_once_in_cmd_vel()

    def get_inputs_rotate(self):
        angular_speed = float(input('Enter angular speed (degrees/sec): '))
        angle = float(input('Enter angle (degrees): '))
        clockwise = input('Rotate clockwise? (y/n): ').strip().lower() == 'y'
        return angular_speed, angle, clockwise

    def convert_degree_to_rad(self, speed_deg, angle_deg):
        angle = float(input('Enter angle (degrees): '))
        clockwise = input('Rotate clockwise? (y/n): ').strip().lower() == 'y'
        return angular_speed, angle, clockwise

    def convert_degree_to_rad(self, speed_deg, angle_deg):
        return speed_deg * 3.141592653589793 / 180, angle_deg * 3.141592653589793 / 180

    def rotate(self):
        angular_speed_d, angle_d, clockwise = self.get_inputs_rotate()
        angular_speed_r, angle_r = self.convert_degree_to_rad(angular_speed_d, angle_d)

        self.cmd.angular.z = -angular_speed_r if clockwise else angular_speed_r

        start_time = self.get_clock().now()
        current_angle = 0.0

        while current_angle < angle_r:
            self.vel_publisher.publish(self.cmd)
            elapsed_time = self.get_clock().now() - start_time
            current_angle = angular_speed_r * elapsed_time.nanoseconds * 1e-9
            self.rate.sleep()

        self.stop_robot()

def main(args=None):
    rclpy.init(args=args)
    robot_control = RobotControl()
    try:
        robot_control.rotate()
    except KeyboardInterrupt:
        robot_control.get_logger().info("Shutting down gracefully.")
    finally:
        robot_control.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

note

All tests must be within test directory and must start as test_*.py conventionally.

Library Unit Tests

unittest

Let's go ahead and create a simple unit test to see if our RobotControl is behaving as correctly.

#!/usr/bin/env python3

import unittest # Testing framework
import rclpy
from robot_control_py.robot_control import RobotControl

class TestRobotControl(unittest.TestCase):

    @classmethod
    def setUpClass(cls): # Part of the unittest framework where we override to specify logic when we instantiate the Test Node.
        """Initialize rclpy once for the entire test class."""
        rclpy.init()

    @classmethod
    def tearDownClass(cls): # Part of the unittest framework where we override to specify logic for shutting down
        """Shutdown rclpy once after all tests are complete."""
        rclpy.shutdown()

    def setUp(self): # Applies to each individual test cases and is ran before each test.
        """Set up the test environment by creating a RobotControl node."""
        self.rc = RobotControl()

    def tearDown(self): # Applies to each after individual test case.
        """Clean up resources after each test."""
        self.rc.destroy_node()

    def test_deg_rad_conversion(self):
        """Test degree-to-radian conversion."""
        speed, angle = self.rc.convert_degree_to_rad(60, 90)
        self.assertAlmostEqual(speed, 1.0472, places=4, msg="Speed conversion failed")
        self.assertAlmostEqual(angle, 1.57, places=2, msg="Angle conversion failed")

if __name__ == '__main__':
    unittest.main()

ROS2 has a built-in testing system using it's build system colcon, however, to set it up, we have to configure some files. As a general rule of thumb, while ROS2 ament_python is available as a build type, it's in your best interest to work with ament_cmake as most ROS2 features require a CMakeLists.txt.

CMakeList

if(BUILD_TESTING)
    find_package(ament_cmake_pytest REQUIRED)
    ament_add_pytest_test(test_robot_control test/test_robot_control.py)
endif()

package.xml

<test_depend>ament_cmake_pytest</test_depend>

setup.py

This is only if you are using an ament_python package, and is to provide the ROS2 environment where the test files are after building. Note that you will need the two above files as well.

(os.path.join('share', package_name, 'test'), glob('test/*.py'))

If you don't want to utilize the colcon build system, you can run the python file as a stand-alone script.

python3 test_robot_control.py # Run without ROS2 environment
colcon test --packages-select <package_name> # Run with ROS2 environment
colcon test-result --all --verbose # Run to see results

The outputs for success and failing:

.
----------------------------------------------------------------------
Ran 1 test in 0.024s

OK

######################################################################

F
======================================================================
FAIL: test_deg_rad_conversion (__main__.TestRobotControl)
Test degree-to-radian conversion.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ros2_ws/src/robot_control_py/test/test_robot_control.py", line 30, in test_deg_rad_conversion
    self.assertAlmostEqual(speed, 1.0472, places=4, msg="Speed conversion failed")
AssertionError: 0.5235987755982988 != 1.0472 within 4 places (0.5236012244017011 difference) : Speed conversion failed
Finished <<< robot_control_py [0.63s]
Summary: 1 package finished [0.83s]
root@9f3977f822c1:/ros2_ws# colcon test-result --verbose
Summary: 1 test, 0 errors, 0 failures, 0 skipped

Test Suites

unittest provides what's known as a Test Suite which allows us to modularize our test cases for different functionalities. Test Suites contain test cases or even other test suites.

We can create a TestSuite via below:

#!/usr/bin/env python3

import unittest
import rclpy
from robot_control_py.robot_control import RobotControl

class TestCaseA(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        rclpy.init()

    @classmethod
    def tearDownClass(cls):
        rclpy.shutdown()

    def setUp(self):
        self.rc = RobotControl()

    def test_case_a(self):
        speed, angle = self.rc.convert_degree_to_rad(60, 90)
        self.assertAlmostEqual(angle, 1.57, places=2, msg="Test Case A failed")

class TestCaseB(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        rclpy.init()

    @classmethod
    def tearDownClass(cls):
        rclpy.shutdown()

    def setUp(self):
        self.rc = RobotControl()

    def test_case_b(self):
        speed, angle = self.rc.convert_degree_to_rad(60, -90)
        self.assertAlmostEqual(angle, 1.57, places=2, msg="Test Case B failed")

If we don't want to use the colcon build system, we need a runner for this.

import unittest
from test_robot_control_cases import TestCaseA, TestCaseB

if __name__ == '__main__':
    suite = unittest.TestSuite()
    suite.addTest(TestCaseA('test_case_a'))
    suite.addTest(TestCaseB('test_case_b'))
    runner = unittest.TextTestRunner()
    runner.run(suite)

Output:

root@9f3977f822c1:/ros2_ws/src/robot_control_py/test# python3 run_tests.py
.F
======================================================================
FAIL: test_case_b (test_robot_control_cases.TestCaseB)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ros2_ws/src/robot_control_py/test/test_robot_control_cases.py", line 39, in test_case_b
    self.assertAlmostEqual(angle, 1.57, places=2, msg="Test Case B failed")
AssertionError: -1.5707963267948966 != 1.57 within 2 places (3.140796326794897 difference) : Test Case B failed                                                                                             ----------------------------------------------------------------------
Ran 2 tests in 0.054s
FAILED (failures=1)

The tests are failing because we only want positive angles but are accounting for negative angles. We can fix it by adjusting our core code and rebuilding, and doing so will pass our tests.

info

If you only want to see tests for one package, run colcon test-result --test-result-base build/<package_name>

If you want to color code it, run this command or put it inside a script file:

#!/bin/bash
colcon test --packages-select <package_name> --pytest-args="--color=yes" --pytest-args="-v" --event-handlers console_direct+

root@9f3977f822c1:/ros2_ws# ./tests.sh
Starting >>> robot_control_py
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /usr/bin/python3
cachedir: /ros2_ws/build/robot_control_py/.pytest_cache
rootdir: /ros2_ws/src/robot_control_py, configfile: pytest.ini
plugins: ament-flake8-0.12.11, ament-pep257-0.12.11, ament-xmllint-0.12.11, ament-copyright-0.12.11, ament-lint-0.12.11, launch-testing-ros-0.19.8, launch-testing-1.0.7, colcon-core-0.18.4
collecting ...
collected 3 items
test/test_robot_control_cases.py::TestCaseA::test_case_a PASSED          [ 33%]
test/test_robot_control_cases.py::TestCaseB::test_case_b PASSED          [ 66%]
test/test_robot_control_cases.py::TestCaseB::test_case_string PASSED     [100%]
-------- generated xml file: /ros2_ws/build/robot_control_py/pytest.xml --------
============================== 3 passed in 0.12s ===============================
Finished <<< robot_control_py [0.63s]                                                                                                                                                                       Summary: 1 package finished [0.83s]

Node Tests

Let's do some Node Testing, which is just unit test for ROS2 nodes.

#!/usr/bin/env python3

from robot_control_py.robot_control import RobotControl
from rclpy.node import Node
from geometry_msgs.msg import Twist
import time
import rclpy
import unittest


class TestRobotControl(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        rclpy.init()

    @classmethod
    def tearDownClass(cls):
        rclpy.shutdown()

    def setUp(self):
        self.node = rclpy.create_node('test_robot_control_node')
        self.rc = RobotControl()
        self.success = False

    def callback(self, msg):
        print(f"Received Angular Speed: {msg.angular.z}")
        self.success = msg.angular.z == 1.0

    def test_publish_cmd_vel(self):
        test_sub = self.node.create_subscription(
            Twist, "/cmd_vel", self.callback, 10
        )
        self.rc.cmd.angular.z = 1.0
        self.rc.publish_once_in_cmd_vel()
        timeout_t = time.time() + 10.0  # 10 seconds

        rclpy.spin_once(self.node, timeout_sec=0.1)
        while not self.success and time.time() < timeout_t:
            rclpy.spin_once(self.node, timeout_sec=0.1)

        self.assertTrue(self.success)

    def tearDown(self):
        self.node.destroy_node()


if __name__ == '__main__':
    unittest.main()

The only difference is that before each test, we are instantiating a ROS2 node and using it within our test function. In our case, we're subscribing to /cmd_vel, and seeing if we do get a message back.

python3 test_robot_control_node.py
[INFO] [1735008562.793690945] [robot_control_node]: Command published
Received Angular Speed: 1.0
.
----------------------------------------------------------------------
Ran 1 test in 0.036s

OK

======================================================================
./tests.sh robot_control_py
./tests.sh robot_control_py
Starting >>> robot_control_py
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /usr/bin/python3
cachedir: /ros2_ws/build/robot_control_py/.pytest_cache
rootdir: /ros2_ws/src/robot_control_py, configfile: pytest.ini
plugins: ament-flake8-0.12.11, ament-pep257-0.12.11, ament-xmllint-0.12.11, ament-copyright-0.12.11, ament-lint-0.12.11, launch-testing-ros-0.19.8, launch-testing-1.0.7, colcon-core-0.18.4
collecting ...
collected 4 items

test/test_robot_control_cases.py::TestCaseA::test_case_a PASSED          [ 25%]
test/test_robot_control_cases.py::TestCaseB::test_case_b PASSED          [ 50%]
test/test_robot_control_cases.py::TestCaseB::test_case_string PASSED     [ 75%]
test/test_robot_control_node.py::TestRobotControl::test_publish_cmd_vel PASSED [100%]

-------- generated xml file: /ros2_ws/build/robot_control_py/pytest.xml --------
============================== 4 passed in 0.15s ===============================
Finished <<< robot_control_py [0.67s]

Summary: 1 package finished [0.89s]

Automation

There's actually a way simpler way where we just modify the CMakeLists and package.xml, and remove all the unnecessary stuff from main. From the ROS2 documentation,

cmake_minimum_required(VERSION 3.8)
project(app)

if(BUILD_TESTING)
  # Integration tests
  find_package(ament_cmake_ros REQUIRED)
  find_package(launch_testing_ament_cmake REQUIRED)
  function(add_ros_isolated_launch_test path)
    set(RUNNER "${ament_cmake_ros_DIR}/run_test_isolated.py")
    add_launch_test("${path}" RUNNER "${RUNNER}" ${ARGN})
  endfunction()
  add_ros_isolated_launch_test(test/<any_test>.py)
endif()

<test_depend>ament_cmake_ros</test_depend>
<test_depend>launch</test_depend>
<test_depend>launch_ros</test_depend>
<test_depend>launch_testing</test_depend>
<test_depend>launch_testing_ament_cmake</test_depend>
<test_depend>rclpy</test_depend>

And you can just run colcon test to get your tests. I came upon this after attempting to figure out how to create launch files for testing.

Integration Tests

We can use the same CMake file for configuring an integration test as it was built for that purpose. The add_ros_isolated_launch_test isolates the nodes in order to test them. In fact, as long as our CMake file contains those lines of code, we don't have to worry about tests as long as all of our tests are in the test directory and starts with test_*.py