Do Teachers Really Come From The "Bottom Third" Of College Graduates?

This from the Shanker Blog:

The conventional wisdom among many education commentators is that U.S. public school teachers “come from the bottom third” of their classes. Most recently, New York City Mayor Michael Bloomberg took this talking point a step further, and asserted at a press conference last week that teachers are drawn from the bottom 20 percent of graduates.

All of this is supposed to imply that the U.S. has a serious problem with the “quality” of applicants to the profession.

Despite the ubiquity of the “bottom third” and similar arguments (which are sometimes phrased as massive generalizations, with no reference to actual proportions), it’s unclear how many of those who offer them know what specifically they refer to (e.g., GPA, SAT/ACT, college rank, etc.). This is especially important since so many of these measurable characteristics are not associated with future test-based effectiveness in the classroom, while those that are are only modestly so.

Still, given how often it is used, as well as the fact that it is always useful to understand and examine the characteristics of the teacher labor supply, it’s worth taking a quick look at where the “bottom third” claim comes from and what it might or might not mean.

Most people who put forth this assertion cite one of two sources, both from the McKinsey & Company consulting organization. The first is an influential 2007 report, (based on Condition of Education” (CoE) report, (2002 edition) which simply notes that “we are now recruiting our teachers from the bottom third of high school students going to college.” The authors fail to specify how “bottom third” is defined, or whether their data refer to graduates who planned to teach versus those who actually got a job (the latter method is, of course, far preferable)...

The second standard source for the “bottom third” claims is more clear and well-documented. It is a subsequent McKinsey report (2010), one which doesn’t rely on questionable interpretations from indirect sources, but rather its own analysis (based on SAT/ACT scores, specifically those of 1999 graduates whose first job was teaching). That report claims, “The U.S. attracts most of its teachers from the bottom two-thirds of college classes, with nearly half coming from the bottom third.” ...

Why the differences? Because these studies are looking at different groups of teachers. In the CoE data, it’s 1993 graduates who had taught by 1997 (four years later), while the data used in the second McKinsey include 1999 graduates who, in 2001 (two years later), said their first job was (or is) teaching. In other words, each set of results is based on two different cohorts of college graduates, who are also identified in different ways, at different points after graduation.

Neither sample is necessarily representative of the teacher workforce as a whole, or of prior and subsequent cohorts.

Overall, then, the blanket assertion that teachers are coming from the “bottom third” of graduates is, at best, an incomplete picture. It’s certainly true that, when the terciles are defined in terms of SAT/ACT scores, there is consistent evidence that new teachers are disproportionately represented in this group (see here and here for examples from the academic literature). But the differences are not always as large as is sometimes suggested. They vary by year and sample identification, as well as by other variables, such as school-level characteristics (e.g., poorer schools) and teacher characteristics such as race and gender. (And, by the way, their relative standing as graduates is based on tests that most took in high school.)

Finally, it’s very important to note that the “bottom-“ and “top third” may not be a particularly good conceptualization of either the problem or the solution. The connection between SAT/ACT scores and future (test-based) effectiveness, though it’s among the only associations strong enough to be discerned statistically, is still a highly imprecise predictor of “quality.” There’s some useful information there, but it’s unclear whether it merits the high-profile attention it receives.**

There’s no question that we should try and get as many highly-qualified applicants as possible into the classroom (though, again, how to do so is an open question). But we should also be very careful about oversimplifying the issue of aggregate teacher quality – and making sweeping statements about their qualifications based on a limited body of evidence – for the sake of intuitive, easy-to-understand talking points.