Reproducibility

Reproducibility Activity

Evaluating and comparing performance of wireless systems, like for any other scientific area, requires the ability to reproduce experimental results. For that, the ACM has introduced a new policy on result and artifact review and badging. The policy defines clearly the terminology to be used to assess results and artifacts but does not specify the review process. For the purpose of encouraging reproducibility in the community and providing recom- mendations on the artifact review process for conferences and journals of the SIGCOMM interest group, the ACM SIGCOMM 2017 Reproducibility Workshop (Reproducibility’17) was organized, and I served as one of its PC member. I also participated in a survey on artifacts from leading ACM computer networking confer- ences in 2017: CoNEXT, ICN, IMC, and SIGCOMM. In this survey, we tried to assess the state of reproducibility in the community. Based on the conclusion of the survey, I am organizing a reproducibility activity, where I manage a reproducibility review comity in our research team at INRIA that will badge one paper from the aforementioned papers. Knowing the challenges that arise when searching for reviewers willing to be in a reproducibility review comity, this experience will be the basis for a series of recommendation for authors and reviewers alike to simplify the review process.