Or, What to do When “Science Says”.
(Hold on to the last one. We’ll return to it. If you’re really interested, I’ll even link to a research study.)
Oh, research has numbers and formulas and an immoral number of letters that are supposed to represent numbers somehow and guaranteed to produce nightmares of figures with no insides forcing you to make up answers on badly-photocopied pages. But the numbers say something and we need to speak the language.
But here’s some badly-kept secrets: newspaper writers are looking for headlines, and generally don’t understand statistics. Also, science isn’t science unless it can be replicated. And much of what happens in social sciences can’t be replicated.
How should I interpret articles that say “research says”?
The easiest answer? Look to see if the study has been replicated. Replication is just a fancy word for “someone else tried the study again and, by golly, found out that the first study was right! (Or wrong.)”
I won’t go into all of the mathematics here, but I’ll say this: for a study to be what’s called “statistically significant” in social science researcher, there needs to be at least a 95% chance you’d get roughly the same results. That means, for some studies, there’s a 5% probability that you’d get different results just from the kind of people you picked for the study.1
Practically, this means that you need to ask yourself if the study has been replicated. If it’s been done a few times by a few different people with the same results? Feel good about it. Proclaim it from the mountains.2
Always look to see if the study has been replicated.
If not? Look at the study with the same skepticism you feel when your child says, “Don’t look in my room, but it’s clean.”
What about school start times, then?
Let’s take a look at a meta-analysis3. Since the meta-analysis analyzes similar studies of similar things, it ultimately is a discussion of replication. or whether the research replicates. Now, to be fair, true replication is more trustworthy than studies that are similar, but true replication isn’t common in the social sciences. (True replication completes an identical study at a different time with different people.)
This particular meta-analysis suggests4, based on a review of the literature, three things:
- Students’ grades do not necessarily seem to improve. However, students appear to sleep more, be more attentive in class, and be tardy to school less.
- Students may be associated with better behavioral health, i.e. less depression due to increased sleep.
- Later start times are associated with less frequent car accidents, i.e., more attentive driving. At least one study suggested that increased sleep time benefited adolescent drivers by increasing their reaction time, despite no significant difference in their self-reported tiredness.5
- Later start times may be beneficial on the basis of our knowledge of adolescent’s sleeping rhythms.
What is clear from the research that kids need more sleep.
As the meta-analysis notes, the American Pediatric Association recommended, in 2014, that schools aim for a start time of 8:30 and that earlier start times do not align with adolescent circadian rhythms.
We’re a school that’s interested in pursuing the practices of the best private schools. And that means following the data and admitting the strengths and limitations of the data.
Preparing students spiritually, socially, and academically means setting them up for success and helping them make healthy, wise decisions with their health. We believe, for the reasons suggested above and more, that we will be a happier, healthier school as a result of the later start time.
But we’ll continue to see what the data suggests!
1 The situation is even worse than that. But I don’t want to horrify you… too much.
2 Of course, there’s more too it. You need to consider effect size as well. Effect size is, basically, the impact that one thing has on another thing. A high effect size means that the thing really really is a big factor in making another thing happen. For example, legos on the carpet would have a huge effect on causing bare feet to hurt, although other factors (like calluses on the foot) would also be factors. A study can be statistically significant but have such a low effect size that it’s practically irrelevant.
3 A meta-analysis is an analysis of analysis. It’s sort of you, right now, thinking about yourself, thinking, except that’s meta-cognition or thinking about thinking.
A meta-analysis can give you a sense of the data or whether there’s sufficient research to back up the claims. It reviews similar studies and compares their conclusions. In short, it can be a review of replication.
4 Data never proves. Data only suggests. After all, more data from different subjects might suggest a different thing!
5 But it’s only one study. “Only one study?” you say. “Is this a test to see if I’ve been reading?” Why, yes. Yes, it is.