Detecting and analyzing I/O performance regressions

Regression testing can be performed by reexecuting a test suite on different software versions and comparing the outcome. For functional testing, the outcome of such tests is either pass correct behavior or fail incorrect behavior. For nonfunctional testing, such as performance testing, this is more challenging as correct and incorrect are not clearly defined concepts for these types of testing.