Lesson 1.5: Experimental Design and the Statistical Enquiry Cycle (Planning and Collection)

Introduction

In this lesson, students will explore the fundamental concepts of experimental design and the stages of the Statistical Enquiry Cycle (SEC) related to planning and data collection. Understanding these concepts is essential for conducting robust statistical investigations. This lesson aims to:

Discuss the key issues in experimental design such as experimental error, randomisation, replication, control and experimental groups, and the significance of blind and double-blind trials.
Explain the benefits of paired comparisons and blocking as strategies to improve the validity of experiments.
Outline the initial planning and data collection stages of the SEC, highlighting how to identify factors, define hypotheses, and avoid bias in data collection.
Describe how experimental designs, like completely randomised designs and randomised block designs, operate and explain the concept of blocking in reducing experimental error.

Experimental Design Issues

Experimental design is crucial in ensuring that the results of an experiment are valid and can be attributed to the factor being tested. Here are some key issues to consider:

1.1 Experimental Error

Experimental error refers to the variability in data that can arise from various sources during an experiment. This can be due to measurement inaccuracies, environmental changes, or individual differences among subjects. To illustrate this:

Example 1: Suppose you are measuring the effect of a new fertilizer on plant growth. If you notice different growth rates among plants not because of the fertilizer but due to varying sunlight exposure, this variability introduces experimental error. A common method to quantify this error is through standard deviation.

1.2 Randomisation

Randomisation is the process of assigning experimental units to different groups using random methods to minimize bias. This helps to ensure that any differences observed are due to the treatment effect and not other variables.

Example 2: Consider an experiment testing a new medication. Randomly assigning patients to either the treatment group (receiving the medication) or the control group (receiving a placebo) ensures that individual variances among patients do not skew the results.

1.3 Replication

Replication involves repeating an experiment multiple times under the same conditions to obtain more reliable data. This helps to confirm the results and identify any inconsistencies.

Example 3: In agricultural studies, planting multiple plots of crops treated with the same fertilizer across different fields can provide a clearer picture of its effectiveness compared to using only a single plot.

1.4 Control and Experimental Groups

The control group does not receive the treatment, while the experimental group does. Comparing results between these groups provides insight into the treatment's effectiveness.

Example 4: If you are testing a new diet plan, having one group follow the diet and another group not follow it (control) allows you to determine the diet's impact.

1.5 Blind and Double-Blind Trials

In blind trials, participants do not know which group they are in (control or treatment), while in double-blind trials, neither the participants nor the researchers know. This approach helps prevent bias.

Example 5: In a new drug study, if only the participants are unaware of whether they receive the active drug or a placebo, it could still introduce bias in how researchers gather data. Hence, employing double-blind methodology can mitigate this.

Benefits of Paired Comparisons and Blocking

Paired comparisons and blocking are techniques used to enhance the quality of experimental designs.

2.1 Paired Comparisons

This method involves pairing subjects based on certain characteristics, which helps control for variability among these characteristics.

Example 6: In an educational study, if you're measuring the effectiveness of two different teaching methods, you could pair students of similar ability and assign them randomly to different teaching methods. This way, you can compare outcomes more accurately between the methods.

2.2 Blocking

Blocking involves dividing subjects into blocks based on shared characteristics before randomisation. This helps to isolate the effect of the treatment from the variability caused by these characteristics.

Example 7: In a clinical trial for a new vaccine, you might block participants by age groups (children, adults, seniors) and then randomise treatment within those blocks. This minimizes the influence of age as a source of variability in response to the vaccine.

The Statistical Enquiry Cycle (SEC)

The SEC is a structured approach to conducting statistical investigations, divided into stages, A: Initial Planning and B: Data Collection.

A. Initial Planning Stage

This stage focuses on setting the groundwork for the investigation:

Identifying factors: Determine what variables will be included in the study.
Defining hypotheses: Develop careful statements that can be tested, typically a null hypothesis ($H_0$) and an alternative hypothesis ($H_a$).
Deciding what data to collect: Consider what measurements are needed to address the hypothesis.

Example 8: In studying the effects of a new teaching method, your factors might be student engagement and test scores. Your hypotheses could be that the new method leads to higher engagement ($H_0$: Engagement levels are equal; $H_a$: Engagement levels differ).

B. Data Collection Stage

During this phase, ensure that:

Avoiding bias: Design surveys or experimental methods that do not lead participants toward a specific response (leading questions).
Acknowledging sources: Clearly state the sources of data that will be used and any limitations involved.

Example 9: In a survey about educational technology preferences, instead of asking, “How much do you love our educational app?” rephrase to “What is your opinion on the educational app?” to avoid leading.

Types of Experimental Designs

Different designs can be used in experiments, particularly completely randomised designs and randomised block designs.

3.1 Completely Randomised Design

In this design, treatment is assigned completely at random to experimental units. This method is effective when there is minimal variability in the subjects.

Example 10: When testing a new type of fertilizer on crops, if you randomly assign plots without considering soil quality, this represents a completely randomised design.

3.2 Randomised Block Design

This design reduces variability by blocking subjects based on certain characteristics before randomisation occurs. The key advantage is that it accounts for potential confounding variables.

Example 11: In a study comparing two diets on weight loss, participants could first be grouped by gender (block), then randomise each group to receive either diet, thereby controlling for gender differences in metabolism.

Conclusion

Understanding the key aspects of experimental design and the stages of the SEC is vital for conducting effective and valid statistical investigations. students has learned about the concepts of experimental error, randomisation, replication, and the structures required for sound experimental design. By incorporating techniques such as paired comparisons and blocking, researchers can significantly reduce bias and variability in their data.

Study Notes

Experimental error can come from measurement inaccuracies, environmental variations, or individual differences.
Randomisation prevents bias by randomly assigning groups to treatments.
Replication increases reliability by repeating experiments.
Control and experimental groups allow for comparison of treatment effects.
Blind and double-blind trials minimize bias in results gathering.
Paired comparisons and blocking improve the quality of experimental designs.
The Statistical Enquiry Cycle includes identifying factors, defining hypotheses, deciding on data collection, and avoiding bias.
Randomised block designs can decrease experimental error compared to completely randomised designs.