Should We Preregister Experiments In Computer Vision? Call for Papers
ICCV 2019 WORKSHOP
Should We Preregister Experiments In Computer Vision?
CALL FOR PAPERS
We are trialling a new publication and peer review model, based on
preregistering experiments. We would like to invite authors to publish
and present their work at our IEEE International Conference on Computer
Vision (ICCV) 2019 Workshop. More information below.
WHAT IS PREREGISTRATION & HOW DOES IT IMPROVE PEER REVIEW?
Benchmarks have played a key role in the considerable measurable
progress that computer vision has made in the last few years. On the
other hand, reviewers can be tempted to prioritize incremental
improvements in benchmarks to the detriment of other scientific
criteria, destroying many good ideas in their infancy. Authors can also
feel obligated to make orthogonal improvements in order to “beat the
state-of-the-art”, making the main contribution hard to assess.
Preregistration changes the incentives by reviewing and accepting a
paper BEFORE any experiments are conducted. The emphasis of peer review
will be on whether the experiment plan can adequately prove or disprove
one (or more) hypotheses. Some results will be negative, and this is
welcomed. This way, good ideas that do not work will get published,
instead of filed away and wastefully replicated many times by different
groups. Finally, the clear separation between hypothesizing and
confirmation (which does not exist in the current model) raises the
statistical significance of the results significantly.
Preregistration deadline: 20th July (writing only, no experiments)
Experiments period: 10th August - 26th October
We are excited about this new model and invite you to participate. We
refer you to the FAQ on the website above for any questions, or to
contact the organizers.
Joao F. Henriques (University of Oxford)
Samuel Albanie (University of Oxford)
Luca Bertinetto (FiveAI Research)
Jack Valmadre (Google Research)