IPPCR 2019 Overview of Hypothesis Testing Part 2 of 5

Understanding Hypothesis Testing: P-Value and Bayesian Approach

Introduction to the Experiment

• Paul Wakim introduces himself as the chief of Biostatistics and Clinical Epidemiology Service at NIH, discussing the focus on p-values and Bayesian approaches in hypothesis testing.

• An experiment is set up involving two coins: one regular coin (heads/tails) and one with two heads. The audience is invited to choose a coin without knowing which is which.

Setting Up Hypotheses

• The chosen coin will be tossed, and results will be shared without revealing the true state of the coin, mimicking scientific research where data informs inference.

• The null hypothesis (H0) posits that the regular coin was selected, while the alternative hypothesis (H1) suggests it’s the two-headed coin. There’s an initial equipoise between these hypotheses.

Importance of Decision-Making

• Emphasizes that making a decision based on experimental results has significant public health implications; errors can lead to major consequences.

• Highlights urgency in decision-making due to potential life-and-death situations, along with financial costs associated with experiments.

Conducting Coin Tosses

• After tossing once and getting heads, participants are asked if they would reject H0. Most likely, they wouldn’t conclude it's a two-headed coin based on just one toss.

• As more tosses yield heads (up to seven), participants are prompted to consider how many heads would convince them that they have a two-headed coin.

Understanding P-values

• Introduces p-values as probabilities reflecting how likely observed outcomes would occur under H0. For example, getting one head from one toss has a p-value of 0.5.

• Discusses how low p-values indicate questioning H0; for instance, obtaining seven heads out of seven tosses yields a p-value of 0.008—suggesting strong evidence against H0.

Conclusion on Hypothesis Testing

• Concludes that if data shows very low probability under H0 assumptions (like 7 heads), researchers should reconsider their assumption about which coin was tossed.

Understanding P-Values and Bayesian Approach in Statistics

Misinterpretations of P-Values

• The speaker discusses the misconception that obtaining seven heads from a coin toss proves it is a two-headed coin, emphasizing that this result could still occur by chance with a regular coin.

• Clarifies that 0.008 represents the probability of getting seven heads if using a regular coin, not the likelihood of it being a regular or two-headed coin.

• Defines p-value formally as the probability of obtaining results as extreme or more extreme than observed, assuming the null hypothesis (H0) is true.

Introduction to Bayesian Thinking

• Introduces the Bayesian approach as intuitive, reflecting how people update their beliefs based on new observations.

• Describes the process of starting with a prior belief, gathering data, and forming an updated posterior belief through continuous learning and observation.

Updating Beliefs in Bayesian Framework

• Explains how posterior beliefs evolve into prior beliefs as new data is collected; this cycle continues indefinitely in life but concludes in clinical trials when sufficient evidence is gathered.

• Discusses applying this updating process at both individual patient levels and group levels during clinical trials.

Differences Between Bayesian and Frequentist Approaches

• Highlights that Bayesians view probabilities (like p for getting heads) as variables subject to change based on evidence, while Frequentists see them as fixed once a specific scenario (coin choice) is established.

• Illustrates that stating "it's a regular coin" equates to saying "the probability of heads is one half," showing how both concepts are interconnected.

Probability Distribution Updates

• Emphasizes that even after selecting a coin, uncertainty remains about its nature; thus, assigning probabilities reflects this uncertainty according to Bayesians.

• Reiterates that asking about chances related to either type of coin translates into questioning the respective probabilities associated with each outcome.

Practical Application of Bayesian Updates

• Starts with equal prior beliefs regarding both coins' likelihood before any tosses are made; initial assumptions set at p = 1/2 for regular and p = 1 for two-headed coins.

Understanding Bayesian Probability vs. P-Value

Key Differences Between Bayesian Probability and P-Value

• The Bayesian posterior probability of the regular coin is initially 0.333, decreasing to 0.008 after seven tosses, highlighting how evidence impacts belief in hypotheses.

• It is crucial to differentiate between the concepts of p-value and Bayesian probability; they are calculated differently and have distinct statistical definitions.

• After observing seven heads in seven tosses, the belief shifts significantly: there's only a 0.8% chance it's a regular coin versus a 99.2% chance it’s a two-headed coin.

Comparison of Tables for P-Value and Bayesian Approach

• Two tables illustrate different methodologies: one for p-values (number of tosses) and another for Bayesian updates (toss number), emphasizing their differing approaches to data interpretation.

• Despite being fundamentally different concepts, the numerical outcomes from both methods can appear similar, which may lead to confusion regarding their interpretations.

Misinterpretation of P-Values

• The p-value represents the probability of obtaining results as extreme or more extreme than observed under the null hypothesis; it does not indicate the likelihood that the null hypothesis is true.