Whitepaper: Laser-Focus Your Testing With Change-Based Testing

Reduce test execution and integration time while accelerating the continuous integration process. Change-based testing is the way there.

Discover the key to focusing on the precise set of tests that are needed to save time, money and speed up continuous integration.

change based testing

Download the whitepaper to get all the facts on adopting change-based testing technology to identify exactly what code must be tested after every new build.

Focus areas:
  • Obstacles of unit testing
  • How to use change-based testing to evolve from status quo
  • Applying change-based testing to maintain momentum in agile

While experienced developers can structure their tests in well-organized suites and manually execute only those which are related to the changed code, the situation is less flexible when dealing with CI infrastructure because CI systems typically execute all tests every time.

To evolve from status quo, teams can leverage change-based testing. Using change-based testing, test execution time can be significantly reduced, in turn reducing the integration time and accelerating the CI process. Change-based testing is made possible by the Parasoft reporting and analytics product, called Parasoft DTP.

DTP engines support change-based testing by measuring code coverage during test execution over time for each build while associating it with each individual test and code unit. Unlike ordinary code coverage tools available on the market today, DTP technology allows you to store contextual information about executed tests to associated code units.

  • The change-based testing slice selects tests to run based on the impact of a submitted code change to the project, using collected code coverage metrics.
  • During every run of unit and functional tests, Parasoft DTP engine traces the execution of each test individually and collects code coverage associated with each of them. This association is persisted in a mapping and stored in DTP’s centralized repository as a “coverage image.”
  • When new source code changes are submitted into a source control system and a new build is triggered by the CI process, Parasoft Process Intelligence Engine (PIE) can detect all changed lines of code between the current and previous builds.
  • With cross-referencing the coverage image with the mapped tests and knowledge of which lines/methods were modified, identification of the relevant unit tests is possible. This subset of tests is smaller than an all-unit test execution, and thus the execution time is greatly reduced, as well as the time to validate test failures and update necessary assertions that reflect intended changes in code functionality.
  • DTP engines expose their functionality through a REST API, so any external orchestration system that runs unit/functional tests can manage association of those tests with collected code coverage metrics. This approach allows you to integrate change-based testing technology with third-party testing tools to support your entire testing practice. Data is collected in the same fashion as Parasoft’s own tools, on a per test, unit and build basis.
  • During execution of the subset of tests impacted by change, coverage information is collected again and merged back to the original coverage image to reflect all recent changes.
  • DTP engines are capable of monitoring and collecting coverage metrics in multi-session mode. It means that DTP engine can collect coverage information per session independently from each other and associate it with corresponding tests. This approach speeds up the verification test sessions, running them in parallel, which might be a typical scenario for an application under test being deployed on a centralized server or in clouds.
  • Because DTP collects coverage information during manual and automated testing per build on a continuous basis, change-based testing technology will always be in sync with the latest source changes and accumulate coverage information.
  • Change-based testing can be extended with plugins for CI systems (Jenkins, TeamCity, Bamboo, etc.), for advanced functionality to help software development organizations reduce bottlenecks associated with running continuous builds.
  • Local “workstation” adoption of change-based testing reinforces the best software engineering practices – to run local tests often – between code modifications and before committing any changes to the source control system.
  • Change-based testing can trace execution of manual tests against web applications and standalone products and associate captured code coverage information with those tests.

When software development teams do thorough unit, functional and manual testing, they can be easily overwhelmed as their test execution suite grows in proportion to project code. As code is modified for each new story, feature, or cycle, teams must either run all tests or guess which ones need to be run.

To solve this challenge, the change-based testing “PIE slice” provides software teams with accurate information on exactly what code needs to be tested after each new build. Focusing on the exact set of tests that are needed saves time and money, plus accelerates continuous integration.

Change-based testing enables the team to discover regressions sooner and avoid last minute delays and late nights before a new release.

About Parasoft

Parasoft’s software testing solutions support the entire software development process, from when the developer writes the first line of code all the way through unit and functional testing, to performance and security testing, leveraging simulated test environments along the way. Parasoft's unique analytics platform aggregates data from across all testing practices, providing insights up and down the testing pyramid to enable organizations to succeed in today's most strategic development initiatives, including Agile/DevOps, Continuous Testing, and the complexities of IoT. Parasoft’s automated software testing innovations fuel software development organizations, helping teams reduce the time, cost, and effort of delivering high-quality software to the market.