Practical regression test selection with dynamic file dependencies
190
Citation
45
Reference
10
Related Paper
Citation Trend
Abstract:
Regression testing is important but can be time-intensive. One approach to speed it up is regression test selection (RTS), which runs only a subset of tests. RTS was proposed over three decades ago but has not been widely adopted in practice. Meanwhile, testing frameworks, such as JUnit, are widely adopted and well integrated with many popular build systems. Hence, integrating RTS in a testing framework already used by many projects would increase the likelihood that RTS is adopted. We propose a new, lightweight RTS technique, called Ekstazi, that can integrate well with testing frameworks. Ekstazi tracks dynamic dependencies of tests on files, and unlike most prior RTS techniques, Ekstazi requires no integration with version-control systems. We implemented Ekstazi for Java and JUnit, and evaluated it on 615 revisions of 32 open-source projects (totaling almost 5M LOC) with shorter- and longer-running test suites. The results show that Ekstazi reduced the end-to-end testing time 32% on average, and 54% for longer-running test suites, compared to executing all tests. Ekstazi also has lower end-to-end time than the existing techniques, despite the fact that it selects more tests. Ekstazi has been adopted by several popular open source projects, including Apache Camel, Apache Commons Math, and Apache CXF.Keywords:
Regression testing
Integration testing
Unit testing
White-box testing
Integration testing
Unit testing
Test Management Approach
Black-box testing
Regression testing
Manual testing
Non-regression testing
Test strategy
System under test
Test harness
System integration testing
Test script
Keyword-driven testing
Model-Based Testing
Cite
Citations (11)
In this thesis, I investigate the problem of automatically generating test cases. In particular, I focus on the problem of automatic generation of integration test cases from unit test cases. I start from the observation that software is usually provided with many unit test cases, and that unit test cases carry a lot of information about the unit execution that can be used to generate integration test cases. This paper illustrates the approach that I am investigating and that consists in capturing information in unit test cases with static analysis techniques to automatically merge unit test cases to produce useful integration test cases. The preliminary results reported in this paper provide evidence of the effectiveness of the approach. My current research is developing the approach further and producing additional experimental evidence. I expect to complete the research by defining a set of design for testability rules to produce software that facilitates the automatic generation of integration test cases.
Integration testing
Unit testing
Test Management Approach
Test harness
Merge (version control)
White-box testing
Keyword-driven testing
Code coverage
Cite
Citations (3)
This paper presents an automatic technique for generating maintainable regression unit tests for programs. We found previous test generation techniques inadequate for two main reasons. First. they were designed for and evaluated upon libraries rather than applications. Second, they were designed to find bugs rather than to create maintainable regression test suites: the test suites that they generated were brittle and hard to understand. This paper presents a suite of techniques that address these problems by enhancing an existing unit test generation system. In experiments using an industrial system, the generated tests achieved good coverage and mutation kill score, were readable by the product's developers, and required few edits as the system under test evolved. While our evaluation is in the context of one test generator, we are aware of many research systems that suffer similar limitations, so our approach and observations are more generally relevant.
Regression testing
Test suite
Unit testing
Test Management Approach
Keyword-driven testing
White-box testing
Cite
Citations (53)
Regression testing
Unit testing
Non-regression testing
White-box testing
Manual testing
Integration testing
Cite
Citations (3)
Any incremental class testing approach has to answer the two following questions: What integration process, indicating in which order classes are (integration) tested, should be selected? Which test design techniques should be applied to unit and integration test classes? Although there is a fairly large number of reported works on both questions, much remains to be done. On the one hand, class unit/integration testing techniques have mostly been described without any consideration for any integration process. On the other hand, integration processes have been suggested without much consideration for how they could be used in practice, along with class unit/integration testing techniques. There is a lack of research and practical results on how a class integration process can be used to conduct class unit/integration testing. This paper summarizes the problem and suggests an experiment-driven research approach to solve it.
Integration testing
Unit testing
White-box testing
System Integration
System integration testing
Test strategy
Cite
Citations (7)
H. Leung and L. White (1990) introduced the firewall concept for control flow in regression testing. In order to define this firewall, they assumed that unit testing and integration testing are both reliable. The paper continues to assume that unit testing can be carried out in an entirely reliable fashion, while integration testing may not be reliable. The paper is designed to fulfill the following objectives: identify the error cases that the testing process may have missed, making integration testing unreliable; attempt to solve the problems that make integration testing unreliable, designing procedures to improve reliability; and determine the effects of unreliable integration testing on the firewall, indicating how the firewall construction should subsequently change.
Integration testing
Regression testing
White-box testing
Non-regression testing
Firewall (physics)
Test strategy
Manual testing
Unit testing
System testing
Software performance testing
Risk-based testing
Cite
Citations (13)
Already from the early days of testing, practitioners distinguish between unit tests and integration tests as a strategy to locate defects. Unfortunately, the mining software engineering community rarely distinguishes between these two strategies, mainly because it is not straightforward to separate them in the code repositories under study. In this paper we exploited the TravisTorrent dataset provided for the MSR 2017 mining challenge, separated unit tests from integration tests, and correlated these against the workflow as recorded in the corresponding issue reports. Further analysis confirmed that it is worthwhile to treat unit tests and integration tests differently: we discovered that unit tests cause more breaking builds, that fixing the defects exposed by unit tests takes longer and implies more coordination between team members.
Unit testing
Integration testing
Software testing
Test strategy
White-box testing
Cite
Citations (11)
Unit testing aims to ensure that methods correctly implement the specified and implied pre- and post-conditions, while integration testing ensures that modules correctly follow interaction protocols. While the generation of unit test cases has been explored extensively in the literature, there is still little work on the generation of integration test cases. In this paper we present a new technique to generate integration test cases that leverages existing unit test cases. Our key observation is that both, unit and integration testing, use method calls as the atoms to construct test cases from. Unit tests contain information on how to instantiate classes in meaningful ways, how to construct arguments for method calls, and what the resulting system state should be after calling methods with those arguments. We use this information to construct more complex test cases that focus on class interactions rather than on individual state transformations caused by single method calls. This paper presents the approach and shows that the generated test cases can find interesting faults, compared to test suites generated with state of the art approaches.
Unit testing
Integration testing
Cite
Citations (24)
AspectJ
Unit testing
Regression testing
Code refactoring
Software inspection
Cite
Citations (3)
A guide to unit testing Java applications (including J2EE applications) using the JUnit framework and its extensions, this book provides techniques for solving real-world problems such as unit testing legacy applications, writing real tests for real objects, automating tests, testing in isolation, and unit testing J2EE and database applications. Using a sample-driven approach, various unit testing strategies are covered, such as how to unit test EJBs, database applications, JSPs, and Taglibs. Also addressed are testing strategies using freely available open source frameworks and tools, and how to unit test in isolation with Mock Objects. Testing J2EE applications by running tests from inside the container for performing integration unit tests is discussed, as is how to automate unit testing in automated builds (such as Ant and Maven) for performing continuous integration.
Unit testing
Integration testing
Test strategy
Isolation
Keyword-driven testing
Manual testing
Cite
Citations (79)