Lesson 10.3: Probability, Significance and Inferential Testing

Introduction

Welcome to Lesson 10.3 of Foundation Psychology! In this lesson, we will explore the fascinating world of probability and significance in inferential testing. By the end of this lesson, you (students) will be able to understand key concepts such as p-values, type I and type II errors, and the selection of appropriate statistical tests.

Learning Objectives:

Understand probability and the significance level of 0.05
Define and distinguish Type I and Type II errors
Comprehend the logic behind inferential tests
Differentiate between parametric and non-parametric tests
Select appropriate tests based on research design and data characteristics

Understanding Probability and Significance

Probability is a central concept in statistics that helps us understand how likely an event is to occur. For example, if you flip a fair coin, the probability of it landing on heads is 50%, or $P(Heads) = 0.5$.

In psychological research, we often use a significance level of 0.05. This means that we accept a 5% chance of making an error in our conclusions. In other words, if we find a result that has a p-value less than 0.05, we consider it statistically significant. A p-value represents the probability of observing our results (or something more extreme) under the null hypothesis, which states that there is no effect or difference.

Example:

Suppose that you want to test whether a new therapy is more effective than the traditional method. After running your experiment, you find a p-value of 0.03. Since this p-value is less than our significance level of 0.05, we conclude that the new therapy is statistically significantly better than the old one.

Type I and Type II Errors

When conducting inferential tests, researchers must be aware of the potential for errors. The two main types of errors are:

Type I Error (False Positive): This occurs when we reject the null hypothesis when it is actually true. Using our significance level of 0.05 implies that there is a 5% chance of making a Type I error.
Type II Error (False Negative): This occurs when we fail to reject the null hypothesis when it is false. The probability of making a Type II error is denoted by beta ($\beta$).

Trade-off Between Errors

Researchers often need to balance between Type I and Type II errors. Reducing the likelihood of a Type I error (making a false positive conclusion) may increase the likelihood of making a Type II error (missing a real effect). Understanding this balance is crucial for sound decision-making in research.

Logic of Inferential Testing

Inferential testing allows researchers to make conclusions about a population based on a sample. The basic logic involves comparing the observed results from the sample to what we would expect by chance under the null hypothesis.

Example:

Imagine you conduct an experiment to test if sleep deprivation affects memory. If your results indicate an improvement in memory performance among your sleep-deprived participants compared to a control group, you would perform a statistical test to determine if this result could occur by chance.

Parametric vs. Non-parametric Tests

Statistical tests can generally be divided into two categories: parametric and non-parametric tests.

Parametric tests assume that the data follows a normal distribution and meets other criteria, such as:
The level of measurement is interval or ratio.
The variances of the groups being compared are similar.

Examples of parametric tests include the t-test and ANOVA.

Non-parametric tests do not assume a specific data distribution and are often used when the above conditions are not met. Examples include the Mann-Whitney U test and Chi-square test.

Choosing the Right Test

Selecting an appropriate statistical test requires careful consideration of your research design and data characteristics, such as:

Design: Is it related or unrelated? (e.g., paired vs. independent samples)
Level of Measurement: Are the data nominal, ordinal, interval, or ratio?
Type of Analysis: Are you testing for a difference or a correlation?

Example:

If you want to compare the effects of two different teaching methods (A and B) on student performance using scores (which are interval data), you would likely use a t-test if the data meets parametric conditions. If the data does not meet these criteria, you may choose the Mann-Whitney U test instead.

Conclusion

In this lesson, you (students) explored the concepts of probability, significance, and the logic behind inferential testing. Understanding these concepts is crucial in psychological research, as it ensures that we draw valid conclusions from our data. Remember the importance of selecting the right statistical tests based on your research design and data characteristics.

Study Notes

Probability: likelihood of an event occurring
Significance level (0.05): threshold for determining statistical significance
p-value: probability of seeing results under the null hypothesis
Type I Error: false positive conclusion (null rejected when true)
Type II Error: false negative conclusion (null not rejected when false)
Parametric tests: require specific data conditions (normality, equal variance)
Non-parametric tests: used when parametric conditions are not met
Selecting tests depends on design (related/unrelated) and level of measurement