A study on the errors and uncertainties of file system trace capture methods

Despite the popularity of trace-based file system performance evaluation, there is currently no accepted methodology to capture and accurately replay file system traces. The less we know about the limitations of trace-based methodologies, the less we can rely on the obtained results. In this paper, we present a case study analyzing the two most popular trace capture methods. The results of the case study indicate that the two evaluated methods provide good precision, but significant bias in some cases. In addition to providing guidelines on how to improve the quality of trace capture, our results allow us to draw important observations about current practice. We show that bias can be corrected by the execution of a calibration procedure, a practice that is mostly absent in the methodology used in the area. Our results also revealed that, to achieve correct calibration, it is crucial to collect information about the background activity and about the operating system layer running in the experimental environment. Finally, our results allow to quantify the overall trace capture uncertainty, providing means for researches to decide on the suitability of trace capture tools for their purposes.