ACM S3 2010 Panel Abstract
The importance of real-world validation for computer science and
engineering research
Top conferences such as SIGCOMM, NSDI, MobiCom and MobiSys often
encourage the real-world implementation and evaluation of systems.
Yet, we often encounter work based on symplifying assumptions and
evaluated in simulation.
This panel will discuss the importance of real-world validation in
systems research.
We would like to hear from advocates of detailed simulations and
those who prefer collecting real-world data traces. We will also investigate
the drawbacks of using real-world data for solution evaluation, such
as undersampling, non-repeatability, impact of uncontrolled
environment and hardware/software glitches. On the flip side,
collecting and sharing data traces has the potential to increase
research impact, allow researcher to verify published results, and enable
more fair comparison between systems or protocols.