Testing Pyramid: The Blueprint to Grouping Software Tests

It’s 2024, so what’s new in the world of software testing? And what is the right way to approach testing for your application?

If you want to ensure your team builds a high-quality product, then you need a high-quality testing strategy. If you’re lost on where to begin or just looking for new ideas to bring to the table, continue reading to learn about a blueprint for grouping your software tests, called the testing pyramid.

This blueprint isn’t rigid and can be adjusted to meet your unique needs, but always cover unit testing, integration testing, and end-to-end testing. Learn about the value of automating your test cases and how to approach each type of testing.

Remember that testing doesn’t have to be an overwhelming process, as there are plenty of tools at your disposal. Use Instatus to keep an eye on all your services during the final stages of testing. Check out Selenium, an open-source tool for automated web browser testing, if you’re building a web application. Try Wiremock if you need a mock API for testing purposes.

Now, let’s take a closer look at manual and automated testing and the blueprint for grouping software tests through the testing pyramid.

Automated Testing

The age of manual testing is coming to a close. The software industry has been transitioning to automated testing for the last 8-10 years, with a heavy ramp-up in the last 5 years. Manual testing requires a QA engineer to manually run unit, integration, and interface tests to check for bugs and other problems. This is time-consuming and imperfect in many ways. Manual testing requires users to log all the issues they find and review all associated services for errors.

When you decide to focus on an automated testing strategy, your team's resources will be less focused on doing the software testing itself. This allows more focus on writing and developing high-quality tests. Let the automation tools do the heavy lifting. If you’re just getting started in this space, research and pick out a few well-reviewed automation tools to invest in. And don’t forget, there are many open-source automation testing tools available that are really great.

Lean into the strengths of your team. If your team isn’t comfortable with complicated automation tools, find simpler tools, like Instatus, that can still help in tangible ways. If your team is not good at building API integration tests, find a tool that helps in that area.

What is the Test Pyramid?

The Test Pyramid or Test Automation Pyramid was first conceptualized in Mike Cohn’s book, Succeeding with Agile. In the book, Cohn introduces the three tiers of software testing:

Unit Testing
Service Testing
End-to-End Testing

This pyramid provides a wonderful visualization of how your team should be structuring their testing strategy. Expect unit testing to take up most of your testing time and energy while end-to-end testing should be the shortest testing period.

If you want to practice test-driven development (TDD), the test pyramid is a great place to start. It will provide a blueprint to help you group different types of tests together. If you are randomly testing pieces of your software, you won’t know up from down when it comes to what works and what has been thoroughly tested.

Modernize the Test Pyramid

The test pyramid is not perfect, especially from a modern perspective. While it’s a significant improvement over the ice cream cone software testing workflow of the past, the test pyramid sometimes needs to be adjusted to suit your unique needs.

For example, Cohn called the second tier in the pyramid the ‘service’ tier. In practice, this tier is usually integration testing, which we’ll be addressing later in this article. Feel free to change the names of the test layers while keeping the bottom-up concept for testing.

Unit Testing

Unit testing is the base layer of the Test Automation Pyramid. To build a solid testing foundation, start with a robust unit testing phase. Unit tests are extremely narrow in scope but will make up the majority of your test suite.

So what exactly is the ‘unit’ in unit testing? Well, there’s no need to get too far into semantics. You and your team should already have a working knowledge of unit testing. But for the purposes of this guide, a ‘unit’ is the smallest piece of your software. Typically, this is a function but can also extend to a method or class, depending on the programming language your application uses.

Image source: Martinfowler.com

The best thing about unit tests is that once your developers are familiar with them, hundreds of unit tests can be created fairly quickly. Unit tests also execute quickly, and you learn the exact unit that is not functioning correctly. You can efficiently hone in on that unit and begin debugging.

Solitary or Sociable Unit Tests?

You will also have to decide if you want to use solitary unit tests or sociable unit tests.

Suppose you need to test a class that relies on other functions. In that case, you must decide whether you want to use the collaborators or isolate the unit by using Test Doubles. Using a Test Double will reduce the complexity of the test, and sometimes you need simplicity if there are some weird interactions with collaborators. However, to better understand the behaviors of your units, you will want to use sociable tests frequently.

Integration Testing

Integration testing is quickly becoming a larger piece of your testing suite. As microservices and other distributed systems become more popular, expect this type of software testing to become increasingly important.

Integration testing usually replaces the ‘service’ tier from the original automated testing pyramid proposed by Mike Cohn. Compared to unit testing, expect this section to take longer, even though the number of tests you are running will be far fewer.

The goal of integration testing is to see how different pieces of code function together. Your unit tests are not testing for this functionality to keep your test suite fast and light. Your application still has many moving parts that live within your application and those that communicate outside your application.

Single Point Integration Testing

It’s best to test a single integration at a time. For example, if you are testing database integration, any other services should be replaced with test doubles. Once the database integration has been thoroughly vetted, you can move on to the subsequent integration. You can then start lumping integrations together and checking how they work together.

Aim to write your integration tests with any dependencies running locally. This prevents your testing from disrupting any service that is actually serving customers.

Integration Testing Tools

Instatus

Once you start integration testing, you will likely work in a pseudo production environment. If you’re developing using the continuous integration/continuous deployment methodology, then you may even be integration testing during production.

Companies like Netflix and Facebook integrate during production and have excellent track records of deploying new features to millions of users. When testing within a production environment, prepare for the worst by using beautiful status pages from Instatus. This will keep your users and team updated if you experience any disruptions to your services.

Protractor

Protractor is an open-source tool for your web-based integration testing needs. It’s most commonly used for Angular or AngularJS applications but can be extended to other use cases. Protractor will automate the testing process and mimic an end-user to get the best test results. Check out this tutorial to get started.

Wiremock

Wiremock provides a way to create mock APIs for all your integration testing needs. Wiremock will construct an HTTPS server that mimics an actual web service so that you can verify the behaviors of your software. Wiremock is open-source and highly configurable.

Citrus

Citrus is another integration testing tool that can help with your complex integration testing needs. Citrus can handle a variety of protocols, including HTTPS, JMS, TCP/IP, XML, and JSON. This tool can exchange actual request and response messages.

Selenium

Selenium is a sophisticated tool to automate integration tests for web applications. Selenium can help you scale your testing suite while managing multiple environments simultaneously. You can test your software across a variety of browsers and operating systems. Selenium is open-source and can be used for all types of software testing, not just integration testing.

Image source: Tatvasoft.com

End-to-End Testing

At the top of the testing pyramid is end-to-end testing. End-to-end testing is one of the final steps before your software is greenlighted. When you’ve completed end-to-end testing, you may move into acceptance testing.

The goal of end-to-end testing is to test your application as it is intended to be used. This phase of testing is often called interface testing. More precisely, your end-to-end testing needs to be tested through the UI. Testing the interface is relatively straightforward as many web application frameworks have basic tests that include UI testing. You can turn to Selenium for testing applications that render on the server-side.

Ultimately, end-to-end testing will help give your team confidence that the product is complete and working as intended. You may not have many end-to-end tests to run, but expect them to be clunky, flaky, and time-consuming. Streamline end-to-end tests when possible, focusing on the most critical interactions your users have with the application.

For example, if you offer a subscription service for video streaming, then the focus of your end-to-end testing may center around the actual subscribing process and the functionality of the video player.

Do You Still Need To Test Manually?

While the testing pyramid does not explicitly include manual testing in all versions, it’s still a valuable testing method to this day. Manual testing isn’t automated at a high level yet, so this method is time-consuming and labor-intensive.

But there is something about a real user testing your application that is not replicated by a computer. A user may bring about edge cases that your team never considered and find bugs you couldn’t have dreamed up. If you have the time and resources, devote some effort to this area.

Test Pyramid – A Blueprint for Testing Success

So, your team can choose to develop any kind of testing strategy they want, but following a blueprint can prevent your team from missing key areas. The automated testing pyramid is an excellent place to start. You will want to adjust it to your individual team and application, but you will generally have 3-4 testing phases.

Move through unit testing, integration testing, and end-to-end testing, to ensure you have a great product that works the way you want it to. You can even throw in some manual testing for a more humanized analysis. However, automating as much of the testing process as you can leaves your developers more time to focus on building a quality test suite.

Use tools like Instatus to create simple but effective status pages or Selenium for automated integration testing. Build a better product for your users by leading with testing-driven development.