Evaluating statistical claims

Official Digital SAT skill — Problem-Solving and Data Analysis domain.

What this question tests

This skill tests whether you can read a study description and choose the strongest conclusion that is justified by the design. On the Digital SAT, these items usually give a short summary of how data were collected (for example, a survey or an experiment), then ask you to determine what conclusion the study supports. The key move is to connect the method (observational study versus randomized experiment) to what kinds of claims are allowed (association versus causation). You also need to notice how the sample was chosen, because a random sample supports generalizing results to the broader population it represents, while a non-random sample limits generalization. This is tested because many real-world claims sound convincing, but the data collection method determines whether the claim is actually supported.

What to know

An association means two variables tend to vary together, which can be described by a relationship such as a difference in group means $\Delta=\bar{x}_1-\bar{x}_2$ or a correlation $r$, but association alone does not prove that one variable causes the other.
A causal conclusion (that changes in one variable produce changes in another) is generally justified only when the study is a randomized experiment, where participants are randomly assigned to treatments to reduce confounding.
An observational study measures variables without assigning treatments, so it can support statements like “is associated with” or “is linked to,” but it typically cannot rule out confounders that could explain the pattern.
Generalization depends on sampling: a simple random sample of a population supports extending findings to that population, while convenience samples or self-selected samples may not represent the population well.
Confounding occurs when an unmeasured variable influences both the explanatory and response variables, creating a spurious association; recognizing confounding is the main reason observational studies can’t support strong causal claims.

How to approach it

First, identify whether the study assigned treatments or merely observed existing behavior, because this single distinction determines whether causal language is allowed.
Next, check how the sample was obtained, because a randomly selected sample supports generalizing to the population it was drawn from, while a non-random sample limits the scope of the conclusion.
Then, translate the safest supported conclusion into precise wording, favoring phrases like “associated with” or “linked to” for observational studies, and reserving “causes” for randomized experiments.
After that, test each answer choice against the design: eliminate any option that claims causation when the design is observational, because confounding could be responsible for the observed difference.
Also eliminate answer choices that reverse the direction of explanation without evidence (for example, claiming the outcome causes the behavior) because the study usually doesn’t establish temporal order or mechanism.
Finally, if multiple choices sound cautious, pick the one that matches both the design and the sampling: it should be appropriately limited to association versus causation and to the correct population.

Common traps

Causation-from-correlation: students see a strong difference and assume a cause, but observational studies can’t rule out confounders; avoid this by checking whether random assignment occurred.
Reverse-causality distractor: an option may claim the outcome causes the predictor, which feels plausible, but the study often doesn’t establish direction; avoid it by noting that association does not specify which variable drives which.
Overgeneralization: a result from a non-random or narrow sample may be applied to a broad population; avoid it by asking whether the sample was randomly selected from the population of interest.
Unwarranted skepticism: some choices claim you cannot conclude anything, but a well-designed observational study can still support an association; avoid it by allowing the strongest justified claim, not the weakest possible claim.
Ignoring the population definition: a random sample supports generalization only to the population it represents; avoid mistakes by matching the conclusion’s population to the sampling frame described.

Tips & shortcuts

Scan for the words “randomly assigned” versus “randomly selected,” because assigned points to experiments (causation possible) while selected points to sampling (generalization possible).
If you don’t see random assignment, default to an association conclusion even if the pattern is large or intuitive.
If you don’t see random sampling, be cautious about generalizing beyond the sample, even if the study is otherwise careful.
When two answers both sound reasonable, choose the one with the tighter claim: association language and a correctly limited population are usually safer.

Worked example

A survey was sent to a randomly selected group of $900$ adults. Respondents reported whether they used a budgeting app and their self-rated financial stress. App users reported lower average stress than nonusers. The survey did not assign app use. Which conclusion is most strongly supported?

A. Using a budgeting app causes lower financial stress in the adult population sampled.
B. There is an association between budgeting app use and financial stress in the adult population sampled, but causation cannot be concluded. ✓ (correct answer)
C. Lower financial stress causes adults to start using a budgeting app.
D. Because participation was voluntary, no association can be generalized to the population sampled.

Why: The survey used a random selection of adults, so an observed relationship can be generalized as an association to the population sampled. Because app use was not assigned, the study is observational, so causation cannot be concluded. Therefore, the supported conclusion is that there is an association between budgeting app use and financial stress in the adult population sampled, but causation cannot be concluded. The correct answer is choice B.

Use the Practice Questions for this skill to drill it, then attempt a Timed Practice Test.