International Day Of Women in Statistics and Data Science: Full Schedule

Registration opens August 15th at www.idwsds.org.

You will be able to validate your attendance (Check-In to sessions) using your personal schedule.

arrow_back View All Dates

09:00 UTC

S206 - Beyond Simulation: Benchmarking for Method Comparison in Statistics and Machine Learning

Tuesday October 6, 2026 09:00 - 09:30 UTC

When initiating a statistical or machine learning analysis, one of the first and most consequential questions is: which method should be used for a given dataset? Traditional approaches for comparing methods include theoretical derivations and data simulation studies, which provide insight into method performance under controlled conditions. However, these approaches may not fully reflect the complexity of real-world data. Benchmarking—systematic comparison of methods across many real datasets—offers a complementary approach that can improve generalizability and provide practical guidance. Despite its common use in computer science, benchmarking remains underutilized in statistical methodology as new methods continue to emerge.

In this talk, I will discuss benchmarking in the context of statistical and machine learning research and contrast it with theory and simulation. I will outline key principles for conducting rigorous benchmarking studies and illustrate them using two case studies: benchmarking random forest variable selection methods for categorical and continuous outcomes and comparing methods for time-to-event data using the mlr3 framework. Together, these examples demonstrate how benchmarking can enhance scientific rigor, provide practical guidance for method selection, and support more transparent and reproducible methodological research.

Speakers

Jaime Speiser

Associate Professor of Biostatistics and Data Science, Wake Forest University School of Medicine

Dr. Speiser is a biostatistician focused on prediction modeling with applications in medicine. Her work involves developing novel machine learning methodology for prediction modeling, providing guidance on best practices for developing prediction models, and collaborating with medical... Read More →

Organizers

Jaime Speiser

Associate Professor of Biostatistics and Data Science, Wake Forest University School of Medicine

Tuesday October 6, 2026 09:00 - 09:30 UTC
Zoom Room #2

Plenary Talk, Technical Statistical Session

Country United States
Session ID 206

International Day Of Women in Statistics and Data Science

09:00 UTC

Jaime Speiser

Jaime Speiser

Get help with the event