Reproducible Research in Signal Processing - What, why, and how
Vandewalle, Patrick and Kovacevic, Jelena and Vetterli, Martin
Have you ever tried to reproduce the results presented in a research paper? For many of our current publications, this would unfortunately be a challenging task. For a computational algorithm, details such as the exact data set, initialization or termination procedures, and precise parameter values are often omitted in the publication for various reasons, such as a lack of space, a lack of self-discipline, or an apparent lack of interest to the readers, to name a few. This makes it difficult, if not impossible, for someone else to obtain the same results. In our experience, it is often even worse as even we are not always able to reproduce our own experiments, making it difficult to answer questions from colleagues about details. Following are some examples of e-mails we have received: "I just read your paper X. It is very completely described, however I am confused by Y. Could you provide the implementation code to me for reference if possible?" "Hi! I am also working on a project related to X. I have implemented your algorithm but cannot get the same results as described in your paper. Which values should I use for parameters Y and Z?"
Vandewalle, Patrick and Kovacevic, Jelena and Vetterli, Martin (2009) Reproducible Research in Signal Processing - What, why, and how. IEEE Signal Processing Magazine, 26 (3). pp. 37-47. [BibTeX]
We are grateful to Jon Claerbout for getting the reproducible research efforts started and to Mauro Barni and Fernando Perez-Gonzalez for starting the discussions in our signal processing community. We would like to thank the 90 reviewers for their great help in our reproducibility study. We also thank the editorial board members and the other reviewers of this article for the valuable discussions and feedback on early versions of the manuscript.
Here you can find the code and data to reproduce all images and tables from the paper. If you use this code in your research and publications, please also put a reference to this paper. Thank you!
0 people have verified that this code works
0 people have not been able to make it work
0 people have verified that this code reproduces the results presented in the paper
0 people have not been able to reproduce the results
Submit your evaluation
Posted by Patrick on 28 Jun 2009 08:40
Leave a comment
Repository Staff Only: item control page