Research Methods - Hypothesis Testing Pt1 - Hypothesis Testing

Overview of Statistical Hypothesis Testing

Introduction to Hypothesis Testing

• The video introduces the concept of statistical hypothesis testing, focusing on Null Hypothesis Significance Testing (NHST) as a common method.

• NHST begins with the assumption that there is no effect, difference, or relationship between variables, establishing a baseline for comparison.

Process of Hypothesis Testing

• The process involves calculating expected sample data under the null hypothesis and comparing it to real sample data collected. If the real sample deviates significantly from expectations, the null hypothesis may be rejected.

• A clear example is provided involving a pharmaceutical company testing a new drug called Memorex aimed at improving memory. The effectiveness of Memorex is evaluated against general population memory scores.

Setting Up Hypotheses

• In this scenario, past data indicates that the general population has an average score of 50 on a memory test with a standard deviation of 10. The study measures whether Memorex can raise this average score among participants who took it.

• Two hypotheses are established:

• H1 (Alternative Hypothesis): Memorex improves memory scores above the general population mean.

• H0 (Null Hypothesis): There is no improvement; Memorex does not work better than or could even worsen memory performance compared to the general population.

Understanding H1 and H0

• H1 posits that the mean score for those taking Memorex (denoted by μ) is greater than that of the general population's mean score (μ = 50). This reflects an expectation of improved performance due to drug efficacy.

• Conversely, H0 suggests that any observed differences in scores are due to random chance rather than any actual effect from Memorex; thus, it includes an equality sign indicating no effect or potential detriment from using Memorex.

Characteristics of Null and Alternative Hypotheses

• Both hypotheses must be mutually exclusive and exhaustive; they cannot overlap but should encompass all possible outcomes regarding their effects on memory scores. For instance:

• If H0 states equality or superiority in performance by the control group, then H1 must assert inferiority for clarity in testing outcomes.

• Ultimately, one hypothesis will hold true while the other will be rejected based on evidence gathered through statistical analysis during testing phases related to drug efficacy and its impact on cognitive function among users versus non-users in controlled settings.

Understanding Statistical Hypothesis Testing

The Null Hypothesis and Sample Means

• The process begins by assuming the null hypothesis is true, allowing for the calculation of the probability of obtaining a sample mean far from 50 through random sampling.

• By taking a sample that yields a mean of 55, we analyze the distribution of all possible sample means under the assumption that Memorex does not improve memory.

• If our observed sample is highly unlikely under the null hypothesis, we reject it; this process emphasizes starting with the assumption that the null hypothesis holds true.

Evaluating Sample Data

• We calculate expected samples when individuals on Memorex do not experience memory improvement to determine what average scores would look like.

• If encountering a mean of 55 is uncommon under the null hypothesis, we have grounds to reject it; otherwise, we retain it as plausible.

Rejecting vs. Accepting Hypotheses

• The essence of statistical hypothesis testing lies in rejecting the null if data appears unlikely under its premise; acceptance of an alternative hypothesis follows logically from this rejection.

• While rejecting the null suggests support for an alternative (e.g., Memorex improves memory), it's crucial to note that probability cannot definitively prove either hypothesis.

Language and Directionality in Hypotheses

• The terminology used in statistical testing avoids definitive proof; instead, hypotheses are supported or accepted based on evidence gathered from data analysis.

• A practical example illustrates how hypotheses can be directional or non-directional depending on prior expectations about outcomes.

Formulating Hypotheses: An Example with Eels

• In a hypothetical scenario involving American eels, Dr. Octavius posits that a newly discovered species may be shorter than known averages; this leads to formulating both null and alternative hypotheses.

• The alternative hypothesis (H1) states that this new species has a population mean less than 20 inches (the known average for American eels), indicating directionality in her suspicion.

Establishing Opposing Hypotheses

• Correspondingly, the null hypothesis (H0), which must include an equality sign, asserts that this new species' mean length is greater than or equal to 20 inches.

• This setup highlights how opposing hypotheses complement each other within statistical frameworks—if one suggests "less than," its counterpart must assert "greater than or equal."

Understanding Null Hypothesis Testing

The Basics of Null Hypothesis Testing

• The null hypothesis posits that the sample means are equal to or greater than 20, necessitating a statistical test on new eel samples to evaluate this claim.

• To determine if we can reject the null hypothesis, we temporarily assume it is true and calculate the probability of obtaining our observed sample mean under this assumption.

• A common convention in scientific research is to set an alpha level at 0.05 (5%), which defines what is considered "unlikely" for rejecting the null hypothesis.

Probability Distributions and Sample Means

• When assuming the null hypothesis is true, researchers create a distribution of all possible sample means based on their sample size.

• The most extreme 5% of these sample means (2.5% on each tail) represent values that would lead us to reject the null hypothesis if our observed mean falls within this range.

Determining Cut-Off Values

• If our calculated sample mean lies in the unlikely region defined by this distribution, we conclude that it is improbable under the null hypothesis and thus reject it.

• Calculating cut-off values for rejection can be done manually or through statistical software; these values stem from probability theory based on assumptions about the null hypothesis.

Alpha Levels and Their Implications

• While 0.05 is standard, researchers may choose different alpha levels before data collection to avoid bias; lower levels like 0.01 indicate stricter criteria for rejecting the null.

• Using an alpha level of 0.1 (10%) allows for more leniency but may reduce confidence in results due to broader definitions of "unlikely."

Directional vs Non-Directional Hypotheses

• A one-tailed hypothesis tests a specific direction (e.g., whether a group’s mean exceeds expectations), while a two-tailed hypothesis assesses differences without directional bias.

• In two-tailed tests, rejection regions are split between both tails; thus, each side receives half of the total alpha level (2.5% each for a total of 5%).

By structuring your understanding around these key concepts and insights related to null hypothesis testing, you can better grasp how statistical significance is determined in research contexts.

Understanding Hypothesis Testing in Statistics

The Concept of Rejection Regions

• In hypothesis testing, the rejection region is defined based on the alpha level (α). For a two-tailed test at α = 0.05, each tail receives 2.5%, which sets the cut-off points for rejecting the null hypothesis.

• The critical region or rejection region is where we reject the null hypothesis if our sample mean falls within this area, indicating that it is statistically significant.

Visualizing Sample Means

• A graph visualizes all possible sample means under the assumption that the null hypothesis is true. This helps to understand how extreme our observed sample mean is compared to what would be expected.

• If our actual sample mean appears unlikely under this distribution, we may reject the null hypothesis; otherwise, we do not.

Calculating P-values

• Statistical software can calculate how likely or unlikely a specific sample mean is if the null hypothesis holds true. This calculation generates a probability curve representing potential sample means.

• The P-value represents the probability of obtaining a sample mean as extreme as ours (M), assuming that the null hypothesis is true. It quantifies how far out our sample lies on this graph.

Interpreting P-values

• A low P-value (e.g., < 0.05) indicates that observing such an extreme sample mean under the null hypothesis is very unlikely, leading us to reject it.

• For instance, a P-value of 0.03 suggests only 3% of samples would yield such an extreme result if the null were true, reinforcing grounds for rejection.

Conclusion and Implications of Results

• If our P-value falls below our chosen alpha level (commonly set at 0.05), we conclude that it's improbable for our observed data to occur under the null hypothesis and thus reject it.

• Conversely, if our results fall within expected ranges (the normal distribution), we fail to reject the null hypothesis but do not accept it as true; rather, we acknowledge insufficient evidence against it.

• Failing to reject does not confirm H0's truth; instead, it leaves open possibilities for alternative hypotheses since they have not been ruled out by lack of evidence against them.

This structured approach provides clarity on key concepts in statistical inference and highlights important aspects of conducting and interpreting hypothesis tests effectively.

Understanding Statistical Hypothesis Testing

Null Hypothesis and Alpha Cutoff

• The alpha cutoff, often set at 0.05, is crucial in hypothesis testing. If the p-value falls below this threshold, one can reject the null hypothesis.

• Conversely, if the p-value exceeds the alpha cutoff, it indicates insufficient evidence to draw conclusions about the null hypothesis; thus, one would state they "fail to reject" it.

• This process emphasizes that statistical testing does not provide definitive proof but rather a probabilistic framework for decision-making.

Probabilistic Nature of Hypothesis Testing

• Regardless of how likely or unlikely a hypothesis appears based on data, language such as "prove" is avoided in statistics due to its inherent uncertainty.

• Even when accepting an alternative hypothesis after rejecting the null, statisticians frame their acceptance in terms of likelihood rather than certainty.