15 Years of Reproducible Research in Computational Harmonic Analysis

Scientific Computation is emerging as absolutely central to the scientific method. Unfortunately, it is error-prone and currently immature: traditional scientific publication is incapable of finding and rooting out errors in scientific computation; this must be recognized as a crisis. Reproducible computational research, in which the full computational environment that produces a result is published along with the article, is an important recent development, and a necessary response to this crisis. We have been practicing reproducible computational research for 15 years and integrated it with our scientific research, and with doctoral and postdoctoral education. In this article, we review our approach, how the approach has spread over time, and then discuss some of the arguments for and against working reproducibly.