Darko Marinov

University of Illinois Urbana-Champaign

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Sarfraz Khurshid

The University of Texas at Austin

Milos Gligoric

The University of Texas at Austin

August Shi

University of Illinois Urbana-Champaign

Tao Xie

Peking University

Lingming Zhang

University of Illinois Urbana-Champaign

Owolabi Legunsen

Cornell University

Vilas Jagannath

University of Illinois Urbana-Champaign

Danny Dig

University of Colorado Boulder

Steven Lauterburg

Salisbury University

Marcelo d’Amorim

North Carolina State University

Cooperative Institutions

University of Illinois Urbana-Champaign

130

University of Illinois System

Urbana University

Massachusetts Institute of Technology

The University of Texas at Austin

IBM (United States)

International University of the Caribbean

Peking University

Carnegie Mellon University

IBM Research - Thomas J. Watson Research Center

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

TransDPOR: A Novel Dynamic Partial-Order Reduction Technique for Testing Actor Programs

Lecture notes in computer science (2012)

Samira Tasharofi Rajesh K. Karmani Steven Lauterburg Axel Legay Darko Marinov

Partial order reduction

Pruning

10.1007/978-3-642-30793-5_14

Cite

Citations (70)

iDFlakies: A Framework for Detecting and Partially Classifying Flaky Tests

Wing Lam Reed Oei August Shi Darko Marinov Tao Xie

Regression testing is increasingly important with the wide use of continuous integration. A desirable requirement for regression testing is that a test failure reliably indicates a problem in the code under test and not a false alarm from the test code or the testing infrastructure. However, some test failures are unreliable, stemming from flaky tests that can nondeterministically pass or fail for the same code under test. There are many types of flaky tests, with order-dependent tests being a prominent type. To help advance research on flaky tests, we present (1) a framework, iDFlakies, to detect and partially classify flaky tests; (2) a dataset of flaky tests in open-source projects; and (3) a study with our dataset. iDFlakies automates experimentation with our tool for Maven-based Java projects. Using iDFlakies, we build a dataset of 422 flaky tests, with 50.5% order-dependent and 49.5% not. Our study of these flaky tests finds the prevalence of two types of flaky tests, probability of a test-suite run to have at least one failure due to flaky tests, and how different test reorderings affect the number of detected flaky tests. We envision that our work can spur research to alleviate the problem of flaky tests.

Regression testing

Test suite

Code (set theory)

False alarm

10.1109/icst.2019.00038

Cite

Citations (117)

Practical regression test selection with dynamic file dependencies

Milos Gligoric Lamyaa Eloussi Darko Marinov

Regression testing is important but can be time-intensive. One approach to speed it up is regression test selection (RTS), which runs only a subset of tests. RTS was proposed over three decades ago but has not been widely adopted in practice. Meanwhile, testing frameworks, such as JUnit, are widely adopted and well integrated with many popular build systems. Hence, integrating RTS in a testing framework already used by many projects would increase the likelihood that RTS is adopted. We propose a new, lightweight RTS technique, called Ekstazi, that can integrate well with testing frameworks. Ekstazi tracks dynamic dependencies of tests on files, and unlike most prior RTS techniques, Ekstazi requires no integration with version-control systems. We implemented Ekstazi for Java and JUnit, and evaluated it on 615 revisions of 32 open-source projects (totaling almost 5M LOC) with shorter- and longer-running test suites. The results show that Ekstazi reduced the end-to-end testing time 32% on average, and 54% for longer-running test suites, compared to executing all tests. Ekstazi also has lower end-to-end time than the existing techniques, despite the fact that it selects more tests. Ekstazi has been adopted by several popular open source projects, including Apache Camel, Apache Commons Math, and Apache CXF.

Regression testing

Integration testing

Unit testing

10.1145/2771783.2771784

Cite

Citations (190)

Rostra: a framework for detecting redundant object-oriented unit tests

Tao Xie David Notkin Darko Marinov

Unit testing

10.1109/ase.2004.61

Cite

Citations (139)

Wanted: standards for automatic reproducibility of computational experiments

arXiv (Cornell University) (2023)

Samuel Grayson Reed Milewicz Joshua Teves Daniel S. Katz Darko Marinov

Those seeking to reproduce a computational experiment often need to manually look at the code to see how to build necessary libraries, configure parameters, find data, and invoke the experiment; it is not automatic. Automatic reproducibility is a more stringent goal, but working towards it would benefit the community. This work discusses a machine-readable language for specifying how to execute a computational experiment. We invite interested stakeholders to discuss this language at https://github.com/charmoniumQ/execution-description .

Code (set theory)

10.48550/arxiv.2307.11383

Cite

Citations (0)

Research Artifact: Test-Case Prioritization for Configuration Testing

Zenodo (CERN European Organization for Nuclear Research) (2021)

Runxiang Cheng Lingming Zhang Darko Marinov Tianyin Xu

This is the artifact of the research paper, "Test-Case Prioritization for Configuration Testing", at the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2021. It is also available on Github: https://github.com/xlab-uiuc/ctest_prio_art To use the artifact, please first download and decompress ctest_prio_art-main.zip. Then refer to README.md for using our data, reproducing our experiment results, and running new experiments.

Artifact (error)

Prioritization

10.5281/zenodo.4728985

Cite

Citations (0)

Integrating model checking and theorem proving for relational reasoning

Lecture notes in computer science (2004)

Konstantine Arkoudas Sarfraz Khurshid Darko Marinov Martin Rinard

Counterexample

Scope (computer science)

Soundness

Proof complexity

Source

Cite

Citations (2)

Comparing non-adequate test suites using coverage criteria

Milos Gligoric Alex Groce Chaoqiang Zhang Rohan Sharma Mohammad Amin Alipour

A fundamental question in software testing research is how to compare test suites, often as a means for comparing test-generation techniques. Researchers frequently compare test suites by measuring their coverage. A coverage criterion C provides a set of test requirements and measures how many requirements a given suite satisfies. A suite that satisfies 100% of the (feasible) requirements is C-adequate.

Test suite

Code coverage

Software testing

10.1145/2483760.2483769

Cite

Citations (116)

Comparing and combining test-suite reduction and regression test selection

August Shi Tifany Yung Alex Gyori Darko Marinov

Regression testing is widely used to check that changes made to software do not break existing functionality, but regression test suites grow, and running them fully can become costly. Researchers have proposed test-suite reduction and regression test selection as two approaches to reduce this cost by not running some of the tests from the test suite. However, previous research has not empirically evaluated how the two approaches compare to each other, and how well a combination of these approaches performs. We present the first extensive study that compares test-suite reduction and regression test selection approaches individually, and also evaluates a combination of the two approaches. We also propose a new criterion to measure the quality of tests with respect to software changes. Our experiments on 4,793 commits from 17 open-source projects show that regression test selection runs on average fewer tests (by 40.15pp) than test-suite reduction. However, test-suite reduction can have a high loss in fault-detection capability with respect to the changes, whereas a (safe) regression test selection has no loss. The experiments also show that a combination of the two approaches runs even fewer tests (on average 5.34pp) than regression test selection, but these tests still have a loss in fault-detection capability with respect to the changes.

Test suite

Regression testing

Software regression

Test Management Approach

10.1145/2786805.2786878

Cite

Citations (81)

Korat

ACM SIGSOFT Software Engineering Notes (2002)

Chandrasekhar Boyapati Sarfraz Khurshid Darko Marinov

This paper presents Korat, a novel framework for automated testing of Java programs. Given a formal specification for a method, Korat uses the method precondition to automatically generate all (nonisomorphic) test cases up to a given small size. Korat then executes the method on each test case, and uses the method postcondition as a test oracle to check the correctness of each output.To generate test cases for a method, Korat constructs a Java predicate (i.e., a method that returns a boolean) from the method's pre-condition. The heart of Korat is a technique for automatic test case generation: given a predicate and a bound on the size of its inputs, Korat generates all (nonisomorphic) inputs for which the predicate returns true. Korat exhaustively explores the bounded input space of the predicate but does so efficiently by monitoring the predicate's executions and pruning large portions of the search space.This paper illustrates the use of Korat for testing several data structures, including some from the Java Collections Framework. The experimental results show that it is feasible to generate test cases from Java predicates, even when the search space for inputs is very large. This paper also compares Korat with a testing framework based on declarative specifications. Contrary to our initial expectation, the experiments show that Korat generates test cases much faster than the declarative framework.

Predicate (mathematical logic)

Predicate transformer semantics

Predicate abstraction

10.1145/566189.566191

Cite

Citations (81)