Geoff Cumming - The New Statistics - Effect Sizes, Confidence Intervals & Meta-Analysis

Summary of Main Ideas

Geoff Cumming, a cognitive psychologist from La Trobe University, discusses the importance of effect sizes, Confidence Intervals, and Meta-Analysis in the context of modern statistics and scientific research. He emphasizes the need for better statistical practices to enhance the integrity and reproducibility of scientific findings.

Key Concepts and Lessons:

• Understanding Statistics:
  • Statistics is described as the "science of uncertainty" and variability, crucial for empirical science.
  • The normal distribution is highlighted as a fundamental aspect of statistical theory, reflecting how various independent influences in nature can lead to predictable patterns.
• The Role of Confidence Intervals:
  • Confidence Intervals provide a range of values that likely contain the true population parameter, offering a more intuitive understanding of uncertainty than P-Values.
  • Cumming introduces the "cat's-eye confidence interval" to visually explain how Confidence Intervals represent likelihood and uncertainty.
• Critique of P-Values:
  • The reliance on P-Values (especially the 0.05 threshold) is critiqued for oversimplifying the complexity of data and hiding uncertainty.
  • P-Values exhibit significant sampling variability, meaning that repeated experiments can yield vastly different P-Values, which can mislead researchers.
• Meta-Analysis:
  • Meta-Analysis is presented as a valuable tool for synthesizing results from multiple studies to provide a clearer picture of effect sizes and their Confidence Intervals.
  • It allows researchers to weigh the evidence from various studies to make informed decisions despite inherent uncertainties.
• Open Science and Best Practices:
  • Cumming advocates for the adoption of Open Science practices, including pre-registration of studies, to enhance transparency and replicability in research.
  • He calls for a shift away from P-Values towards estimation methods that focus on the magnitude of effects rather than binary significance.
• Future Directions in Statistics:
  • There is a need for developing intuitive teaching materials and methods for Bayesian approaches and other statistical techniques to better communicate statistical concepts to beginners and practitioners.

Methodology and Instructions:

• Adopt Confidence Intervals: Use Confidence Intervals to communicate uncertainty in research findings.
• Implement Meta-Analysis: Combine results from multiple studies using Meta-Analysis techniques to provide a comprehensive view of research questions.
• Shift from P-Values: Move away from the reliance on P-Values for determining significance, focusing instead on effect sizes and Confidence Intervals.
• Embrace Open Science: Commit to practices such as pre-registration of studies and sharing data to enhance the credibility and replicability of research.

Featured Speaker:

• Geoff Cumming - Cognitive psychologist and author from La Trobe University.