The old myth that women are less strong than men is based on something that seems intuitively obvious — in the same way that humans believed that it was obvious that the earth is flat. Most women you know ARE less strong than the men you know!
Unfortunately, that isn’t evidence for the conclusion. It’s a confusion of intuitively-derived observations with scientific evidence. To play on the old saying: common sense is fucking stupid. Or, to put it another way: the reason we came up with the scientific method is because our brains process information in the exact opposite way. We need it.
When sample sizes are small, as they nearly always are in our “normal” ways of thinking, we are even more likely to be led astray into believing conclusions that are false.
Today, I have a case in point. However, I want to state at the outset that I’m not intending to pick on anyone. It’s about the general problem — methods —, not any particular individuals. I just needed an example, and this will serve that purpose as good as any.
In this paper, by by Miller et al., from 1993, the authors make a couple of common errors, that lead to an erroneous conclusion.
I’ll give you the abstract, then I’ll make some notes:
Strength and muscle characteristics were examined in biceps brachii and vastus lateralis of eight men and eight women. Measurements included motor unit number, size and activation and voluntary strength of the elbow flexors and knee extensors. Fiber areas and type were determined from needle biopsies and muscle areas by computerized tomographical scanning. The women were approximately 52% and 66% as strong as the men in the upper and lower body respectively. The men were also stronger relative to lean body mass. A significant correlation was found between strength and muscle cross-sectional area (CSA; P≤0.05). The women had 45, 41, 30 and 25% smaller muscle CSAs for the biceps brachii, total elbow flexors, vastus lateralis and total knee extensors respectively. The men had significantly larger type I fiber areas (4597 vs 3483 μm2) and mean fiber areas (6632 vs 3963 μm2) than the women in biceps brachii and significantly larger type II fiber areas (7700 vs 4040 μm2) and mean fiber areas (7070 vs 4290 μm2) in vastus lateralis. No significant gender difference was found in the strength to CSA ratio for elbow flexion or knee extension, in biceps fiber number (180 620 in men vs 156 872 in women), muscle area to fiber area ratio in the vastus lateralis 451 468 vs 465 007) or any motor unit characteristics. Data suggest that the greater strength of the men was due primarily to larger fibers. The greater gender difference in upper body strength can probably be attributed to the fact that women tend to have a lower proportion of their lean tissue distributed in the upper body. It is difficult to determine the extent to which the larger fibers in men represent a true biological difference rather that a difference in physical activity, but these data suggest that it is largely an innate gender difference.
My problem is with the final conclusion:
“… but these data suggest that it is largely an innate gender difference.”
Actually, this study does NOT support that conclusion. Note the sentence I put in bold above: “Data suggest that the greater strength of the men was due primarily to larger fibers.” Where did these larger fibers come from? That’s a question to be looked into, not to be explained away.
Here are some key points:
- There were ONLY 8 males and 8 females. That is an abysmally small sample size. This should be your red-flag number one.
- There was no controlling for the MANY factors that can lead to a difference in fiber size.
- A study of this kind looks exclusively at correlation, not causation. That’s perfectly fine, but don’t confuse the two!
- The Men in this study appeared to be stronger relative to their lean body mass, which is interesting. (Exercise for the student: imagine all of the possible reasons this may be true for these 16 people. There are many possible answers.)
- They found a correlation between strength and cross-sectional area.
- There was effectively NO difference between men and women in the ratio of strength to cross-sectional area. (This point is the most important to keep in mind.)
- Hands down, the most important conclusion is that more cross-sectional area in a muscle correlates with more strength, in this sample.
- This means that any differences in strength may need to be explained by something else.
- They provided nowhere near enough data to come to a conclusion on what that cause might be.
To claim that the difference in strength is likely innate is simply dodging the question by positing a cause that can’t be inferred from the data.
Studies that exhibit issues like this — a discrepancy between what a conclusion claims vs what the paper actually showed — are the norm, not an exception, in the health and fitness sciences, as well as in many of the social sciences.
We like to blame the media for everything. But the truth, as always, is more complicated than that. Most writers have no way of knowing if the conclusion statement in a scientific paper, by a scientist, in a respectable journal, is right or wrong, or misleading, etc. They will usually default to believing the conclusion. It’s hard to blame media writers for reporting what scientists actually said.
Your homework? Stay skeptical.
Now go lift something heavy,
PPS. For more on small sample size, see The Specter of Small Sample Size Strikes Again.
PPPS. Photo Cred: Suneko.