Statistical Claims: Studies & Experiments
Some of the trickiest SAT math questions don't ask you to compute anything — they ask whether a conclusion drawn from a study is valid. These problems test whether you understand the difference between observational studies and experiments, and whether you understand which conclusions each type can support. Once you learn the two key distinctions (random sampling and random assignment), every problem in this category becomes mechanical.
The two big ideas
Every SAT statistical claims question hinges on two distinct concepts that students often confuse:
This is HOW participants were selected from the broader population. Did the researchers pick people randomly, or did people self-select into the study?
- Random sampling means: every member of the population had an equal chance of being included.
- What it lets you conclude: the results can be generalized to the full population.
- Without it: results only apply to the specific group studied — you can't generalize.
This is HOW participants were placed into groups within the study. After people were already in the study, were they randomly assigned to a treatment group vs. a control group?
- Random assignment means: the researchers (not the participants) decided who got which treatment, by random chance.
- What it lets you conclude: a difference between the groups can be attributed to the treatment — i.e., causation can be inferred.
- Without it: any difference between groups might be caused by something else (a "lurking variable") — you can only conclude there's a correlation, not causation.
Random sampling → can generalize to the population.
Random assignment → can conclude causation.
The two concepts are independent. A study can have one, both, or neither. The conclusions you can draw depend on which it has.
Observational studies vs. experiments
The difference between an observational study and an experiment comes down to one thing: did the researcher actively manipulate something?
| Observational study | Experiment |
|---|---|
| Researchers OBSERVE participants without intervening. Variables are measured as they naturally occur. | Researchers ACTIVELY ASSIGN participants to different groups (treatment vs. control) and measure the outcome. |
| Example: "Researchers surveyed 500 adults about their coffee habits and exercise levels." | Example: "Researchers randomly assigned 200 patients to receive either Drug A or a placebo, then measured their blood pressure after 8 weeks." |
| Cannot establish causation, only correlation. | Can establish causation IF random assignment was used. |
Look for action words: "randomly assigned," "gave," "administered," "treated" → experiment. The researcher actively did something.
"Surveyed," "observed," "recorded," "measured existing" → observational study. The researcher just watched.
The 2×2 framework: what conclusions are valid
Every SAT study scenario falls into one of four cells based on whether random sampling and random assignment were used. The cell determines what conclusions are valid:
Every SAT statistical claims question can be answered by identifying which cell the study falls into.
Walking through each scenario
Researchers randomly select 200 adults from the U.S. population. They randomly assign half to take Vitamin X daily and half to take a placebo. After 6 months, the Vitamin X group has lower cholesterol on average.
Valid conclusion: Vitamin X causes lower cholesterol in U.S. adults (in general).
Researchers randomly select 1,000 high school students nationwide and survey their study habits and grades. Students who study more than 10 hours/week have higher GPAs on average.
Valid conclusion: Among high school students nationally, more studying is associated with higher GPAs. (Correlation.)
NOT valid: Studying more causes higher GPAs. There could be a lurking variable — students who are more motivated may study more AND get higher grades for independent reasons.
200 volunteers from a single neighborhood are randomly assigned to a new exercise program or a control group. The exercise program participants lose more weight, on average.
Valid conclusion: The exercise program caused weight loss in the volunteers from that neighborhood.
NOT valid: The exercise program will work for adults in general. The volunteers might be unusually motivated, of a particular age range, etc. — results may not generalize.
A gym surveys 50 of its members and finds that members who use the sauna have lower blood pressure than members who don't.
Valid conclusion: Among these 50 gym members, sauna use is associated with lower blood pressure.
NOT valid: Sauna use causes lower blood pressure. Sauna use causes lower blood pressure in the general population. Either claim overreaches the data.
The single-question framework
For any SAT statistical claims problem, ask two yes/no questions in order:
- Was the SAMPLE random? Were participants randomly chosen from the population? If yes → results generalize. If no → results only apply to the studied group.
- Was the ASSIGNMENT random? Were participants randomly placed into treatment vs. control groups? If yes → causation can be inferred. If no → only correlation can be claimed.
The combination of those two answers tells you exactly which conclusions are valid.
Sampling phrases:
- "Randomly selected from..." → random sampling ✓
- "Randomly chosen sample of..." → random sampling ✓
- "Volunteers signed up..." → NOT random sampling
- "Of the 200 students at one high school..." → NOT random sampling (single-location convenience sample)
Assignment phrases:
- "Randomly assigned to..." → random assignment ✓
- "Researchers gave half the participants..." → ambiguous, look for "randomly"
- "Participants chose which group..." → NOT random assignment
- "Compared people who already had/did..." → NOT random assignment (observational)
Sample SAT-style problems
A researcher randomly selects 500 adults from across the country and asks each one whether they regularly drink green tea, then measures their cholesterol levels. The researcher finds that adults who regularly drink green tea have lower average cholesterol than those who don't. Which conclusion is most appropriate?
- (A) Drinking green tea causes lower cholesterol in U.S. adults.
- (B) There is an association between drinking green tea and lower cholesterol in U.S. adults.
- (C) Drinking green tea causes lower cholesterol in the 500 adults studied.
- (D) There is no relationship between green tea and cholesterol.
- Random sampling? YES — 500 adults randomly selected from the country. Results generalize.
- Random assignment? NO — observational study, no intervention. Cannot infer causation.
- Valid conclusion: there's an association (correlation) in U.S. adults. Eliminate (A) — claims causation. Eliminate (C) — limits to the 500 only when generalization IS valid. Eliminate (D) — contradicts the data.
A research team tested a new study technique on 100 volunteer students from a single university. The team randomly assigned half the students to use the new technique and half to use their usual study methods, then measured their performance on a standardized test. Students using the new technique scored higher on average. Which conclusion is most appropriate?
- (A) The new technique improves test scores for all students nationally.
- (B) The new technique improves test scores for the volunteers in this study.
- (C) There is no relationship between the technique and test scores.
- (D) The new technique correlates with higher test scores, but causation cannot be inferred.
- Random sampling? NO — volunteers from a single university (convenience sample, not random). Results don't generalize broadly.
- Random assignment? YES — randomly assigned to treatment vs. control. Causation can be inferred for the participants.
- Valid conclusion: causation among the volunteers. Eliminate (A) — overgeneralizes. Eliminate (C) — contradicts data. Eliminate (D) — denies causation when it IS justified for this group.
A team randomly selected 1,000 patients with chronic headaches from hospitals across multiple states. They randomly assigned half to take a new medication and half to take a placebo. After 12 weeks, the medication group reported significantly fewer headaches. Which conclusion is most strongly supported?
- (A) The medication is associated with fewer headaches in the participants.
- (B) The medication causes fewer headaches in the 1,000 participants only.
- (C) The medication causes fewer headaches in chronic headache patients in general.
- (D) The medication only works for patients with very severe headaches.
- Random sampling? YES — random selection from across states. Generalizes.
- Random assignment? YES — randomly assigned medication vs. placebo. Causation valid.
- Valid conclusion: causation that generalizes. Eliminate (A) — only "associated," when causation IS justified. Eliminate (B) — limits to participants when generalization IS valid. Eliminate (D) — adds a claim not in the data.
The owner of a small bookstore noticed that customers who bought coffee also tended to buy more books than customers who didn't buy coffee. Based on this observation, which conclusion is most appropriate?
- (A) Buying coffee causes customers to buy more books.
- (B) Among customers at this bookstore, buying coffee is associated with buying more books.
- (C) Coffee buyers in general are likely to buy more books.
- (D) The bookstore should require all customers to buy coffee.
- Random sampling? NO — observed customers at a single bookstore. Doesn't generalize.
- Random assignment? NO — observational, not experimental. No causation.
- Valid conclusion: an association among observed customers only. Eliminate (A) — claims causation. Eliminate (C) — overgeneralizes. Eliminate (D) — recommends action not supported by data.
1. Causal language when only correlation is supported. "Causes," "leads to," "results in" — these all imply causation. Only valid if the study had random assignment.
2. Generalizing beyond the sampled population. If participants weren't randomly selected from the broader population, conclusions only apply to the studied group. Watch for sweeping claims about "all students," "all adults," etc.
3. Under-claiming a valid conclusion. The reverse trap: if a study DID use random sampling and random assignment, the conclusion "there's a correlation" is too weak. Pick the strongest conclusion the data supports — neither over- nor under-claim.