The Direct Approach
October 11th, 2007I recently read an excellent book about data analysis (no, really) by Ian Ayers called “Super Crunchers”, which is highly touted by Steven Levitt, a co-author of his that also co-wrote the bestseller “Freakonomics”. In fact, “Super Crunchers” was featured on the Freakonomics blog, and a giant Levitt quote adorns the book’s cover. (Which, of course, is how I found out about it; he’s over there in my sidebar on the right for a reason.) After I saw the book referenced again in The Economist, complete with a Black Swan reference, I knew I had to read it.
The basic premise of “Super Crunchers” is this: Computers that “super crunch” (i.e. analyze and make predictions on) large datasets can often make better decisions than people with our limited intuitions and obvious biases. Despite this somewhat counterintuitive claim, it squares well with me, especially considering my previous anti-theorizing post about Nassim Taleb and Black Swans.
The book has lots of great examples, ranging from Epagogix, the program that predicts how much a movie will gross based solely on its script, to Offermatica, a website that runs randomized trials for different online marketing test campaigns. Epagogix is also reported on here, by none other than Malcolm Gladwell. (The same names always seem to pop up in the stuff I read…) However, to keep things brief (hey, stop groaning, I’m really trying here), I’d like to focus on just one of those examples, namely Direct Instruction.
Direct Instruction (DI) is a relatively radical idea about how to teach elementary school kids (mostly). (There’s a short and glib Youtube video on DI here.) As you can see from that Youtube clip, Direct Instruction is also pretty straightforward. You work in groups of 5-10 kids (if possible) and constantly ask the kids questions as part of tightly scripted lessons. DI is not fancy or particularly clever, which is precisely why it’s such a radical idea. It’s just simple, rapid-fire question-and-answer. This goes against other educational ideals that pause more and use less interactive approaches. DI is more like high-energy coaching than teaching, with a close eye on anyone that’s falling behind. It’s so tightly scripted you might even think it sounds a little bit robotic. Nevertheless, that’s the basic idea. It was developed by an educational outsider named Siegfried Engelmann (who goes by Zig, a convention I will follow) to teach his 3rd grade fraternal twin sons.
As a twin, I might be a sucker for a story like that. And like Zig, I also got my Bachelor’s degree in philosophy, not to mention that I come from a long line of teachers. (I’m trying to practice Full Disclosure here, people, since I might be a *little* biased.) If you’re like me and want to read more about Zig (and how can you not love a name like that), he has a pretty nifty bio on his site. Unfortunately, it’s obvious he’s become somewhat bitter and jaded over the years. But there’s still plenty of interesting tidbits there.
Anyway, enough about me and my biases. Direct Instruction got its big break as part of Project Follow Through, a 600-million-dollar evidence-based study of different educational theories as part of Lyndon Johnson’s “Great Society” social program. DI was the only method not proposed by an education researcher. In fact, Zig didn’t have any formal training when he developed DI. He was an advertising guy. (Think it had anything to do with why he got chosen? Hmm…)
Nevertheless, as you can read from that Project Follow Through link, DI was by far the most successful of the 24 methods. As they mention, only 3 methods did better than the control group. (!) I wish I could give you direct links to the data, but as usual they are buried in some expensive journal that I don’t feel like paying for. The best links I could find (other than the first link) are here and here. Both of those links are from the same site and obviously not as unbiased as the data itself would be.
Which is fine because from my reading online, no one seems to be disputing the basic results of the study anyway. Direct Instruction works. From the sponsor findings link: “The Direct Instruction Model, the only one to show achievement above 3.0 grade level, is about one-half standard deviation above the mean of all other sponsors. It is nearly a quarter-standard deviation above the second-place model, Behavior Analysis.” Though I’m not a big fan of bell curve and standard deviation talk, at least the difference in standard deviations show how much better DI did compared to the other methods.
As Ayers mentions in “Super Crunchers”, DI has continued to do well over the years. Similar studies conducted more recently by the American Federation of Teachers (AFT) and the American Institutes for Research (AIR) have continued to show the effectiveness of DI. You can read about the AFT study here and about the AIR study here. (Although I’m not sure I’d recommend that - I’d say they fall just ahead of the phone book and this blog in terms of readability.) Both studies ranked DI highly, even if they’re a bit short on specifics in their reports.
So the big question you’re probably thinking is, if this method is so effective, why haven’t I heard more about it? After all, the data clearly backs DI, and apparently so does the Bush administration. (Yes, even the Bush administration does something right once in a while.) You might have seen this Youtube clip in the movie “Farenheit 9/11″ of George Bush watching DI in use in an actual classroom. The clip is normally used to make fun of George Bush, as he was intently focused on a elementary school classroom while the Twin Towers were falling. But, Bush jokes aside (it was a bit hard to find the undoctored clip!), it shows how concerned he was with education and DI in particular.
What’s the problem, then? As “Super Crunchers” mentions, a monkey can teach DI. There is “zero prep time”. You need small classrooms, highly motivated teachers, and a rigid methodology. Though it isn’t the most exciting style of teaching, it still demands a year or two of training to get it right. Not only that, but DI can seem dehumanizing. It knocks teachers off their high horse, as it were. You don’t see Robin Williams teaching DI in Dead Poets’ Society, and with good reason (though that involves older kids).
The same problem came up with the children’s show “Sesame Street”. Adults were deciding what was good for kids, and all they had to check their work was a primitive device known as “The Distracter” and a couple of studies, which of course were much better than nothing, but not enough. (Funny enough, one of the studies tried to distract kids with a floor full of toys to see if it would take their attention away from the show. Looking at my floor, I see some things never change.)
For example, adults made a “Sesame Street” episode about Big Bird changing his name, which is much too abstract for a five-year old to handle. And there were all these clever references to pop culture that only adults would understand, but would end up just confusing kids. Then “Blue’s Clues” came along and stood “Sesame Street” on its head. “Blue’s Clues” used the same highly interactive, simple, repetitive methodology as DI and it went over like gangbusters. Kids were distracted much less and learned much more. (If you’d like to read about the whole saga, it’s covered in Malcolm Gladwell’s bestseller “The Tipping Point”.)
The real problem though, as Nassim Taleb might say, is the real lack of experts in teaching, and unfortunately our intuition isn’t much help here either. As in so many other arenas, to be a good teacher you need to look at what works, not what you think will work. If the goal of teaching is to help students learn and increase their self esteem (DI, ironically, is better at this than other methods) then teachers should give a counterintuitive method like Direct Instruction a try. It might seem hard or dehumanizing at first, and in the long run, it might even lead to lower status and pay for teachers. But it will probably be a better deal for students, who ultimately matter the most.
Still, some teachers might try to resist making teaching into a commodity. (Except economics teachers, maybe.) History, however, is not on their side. Just look at the massive outsourcing and automation trends in the workforce. If the people who control the budget suddenly figure out they can do your job more cheaply and with less training, eventually they will probably do so. And what if we introduce vouchers and more educational competition? If those same budgeters can make a profit/bonus from implementing DI? The incentives would be clear.
Personally, the biggest concern I have about Direct Instruction is that I think it might be more generally useful. Right now DI is mainly used with elementary school kids and the learning impaired. However, I think you could take these principles and incorporate them more widely in other disciplines, though to a lesser extent than with DI. It goes along with my more general belief that I think education should be more routine and less of a creative process . (Dead Poet’s Society be damned!)
My own experience bears this out. When I teach people how to use different kinds of software at work, I find that a consistent, bare-bones, highly interactive approach works best. I don’t theorize about the best way to teach them or tailor the teaching specifically to their needs. All I do is explain things extremely simply and get them involved as quickly as possible. In fact, they usually don’t really learn much until they’re the ones behind the keyboard (while I watch). Any other approach I’ve tried has often taken more time, and I find they usually retain less too.
Direct Instruction may not be the most romantic idea in the world, but a lot of us could probably become better teachers by incorporating some of its lessons. In the process, we might even learn a thing or two ourselves.
Further Reading:
A really great page of various predictors that use “Super Crunching”
A Direct Instruction profile at Nifdi website (Nifdi is run by Zig’s twin son Kurt, one of the original DI “testers”)
A cheap link to “Theory of Instruction”, a book co-written by Zig about teaching methods
Other Online Articles About DI:
New Yorker
Washington Post
National Review
Ed Week
| | | del.icio.us |
October 12th, 2007 at 10:45 pm
How are the studies for testing the efficacy of this method done? I can see where it could be extremely useful in bursts i.e. spending 45 minutes a day to learn how to read or teaching a task but it seems to me that it would be hard (for both the teacher and students) to keep this up for the 7 hours that are in school. It also seems that there are certain subjects that you could not teach like this such as literature.
What grade is this method thought to be effective up to? You mentioned probably not in high school.
In low-income schools, they will often teach DI 7 out of 8 hours in the day. You are right, it’s exhausting and difficult for both students and teachers. It also takes a year or two of training for the teacher to even do a good job with DI. Still, the idea is most of the time you use DI in these schools.
DI is mostly meant for K-6 and special education. Once you go with older kids or more abstract subjects (like literature) it gets harder. I still feel like you can apply the lessons of DI in these other contexts, though, just to a lesser extent.
- Dave
November 29th, 2007 at 3:44 am
Hi, my name is disman-kl, i like your site and i ll be back
February 8th, 2008 at 12:32 am
[…] (which I originally found through the Freakonomics blog). Regular readers might remember that I blogged once previously about Ian Ayers’ book “Super Crunchers”, which discussed the power of automated […]