Monday, April 25, 2011

Evaluating Education, Evenhandedly

A story went around the interwebs a couple weeks ago that garnered many stories and more opinions. Unsurprisingly, it focused on a traditionally controversial subject which features prominently in politics at various levels, and a divisive figure. Given that that description could refer to any number of topics, I’ll end the suspense (if you didn't figure it out from the title): education. (Which I've talked about before, by the way).

You’ve probably heard about Michelle Rhee: wonderwoman, education radical, fearless reformer, draconian, unfeeling witch who wants to pump students out like so many mass-produced cans of tuna. By which I mean to say, there are a range of opinions regarding Ms. Rhee and her testing paradigm, which came under some scrutiny in the media after reports surfaced that may have indicated widespread cheating by the DC teachers during Rhee’s chancellorship of that public school system. All of the traditional divisions came up in full strength, with conservatives, anti-teacher unionites and Ms. Rhee herself defending the results as well as the program itself, and liberals, pro-unionites and teacher advocates leaping on the reports and declaring that not only had there been cheating, but that such cheating was an inevitable part of the overly pressurized, teach-to-the-test system imposed on DC schools.

Such a range of opinions does not exist when it comes to the state of public education in this country in general. It’s awful. We know. So what do we do about it? When faced with a problem this seemingly intractable, my first reaction might be to either throw up my hands in the air or go with the liberal opinion, given how often I tend to fall on that side. But this is actually a great example of an issue which is important enough to spend time researching and considering, complex enough to warrant some attention and distillation of the maelstrom of information and viewpoints that exist on the internet, difficult enough to be interesting and political enough that one needs to be cautious wading into the mix. Which means that it’s a great subject on which to practice rational, disinterested analysis, given in particular that it’s incredibly difficult, because children and the next generation and our dearly beloved public school teachers and all that are at stake. I say this with some sarcasm, but also self-deprecatingly, because this all worries me as well.

So, the basic story is that Michelle Rhee thinks that the problems with the school system are that the incentives aren’t in the right place, because teachers have tenure and aren’t held accountable for their skill at teaching or the results they produce. Her system gathers empirical data from standardized tests administered each year, ruthlessly fires bad teachers regardless of seniority and makes sure that kids keep doing better every year. It turns out that this system seems to have worked. A good, if perhaps overly generous summary is here. The problem is that there are suspicious patterns of erasures from wrong to right answers which might indicate cheating.

Here, Michelle Rhee defends herself. A friend sent me this article, and asked me to respond, which is really how this blog post got started (which also explains why this is written in a somewhat less formal tone). In the piece, she really doesn't say anything new or radical, she just defends her record against all the allegations of cheating that have been going on. I think the evidence is very murky. The evidence in her favor is certainly considerable, but much of it may not have been her at all. Testing district based on a few years of data is notoriously difficult, and some of the work, such as facilities upgrading, is done by a completely different agency than the DC Public Schools System. More can be read at the links above. The cheating is hard to prove, especially since the testing service only releases certain data.

But, as a broader issue, my brilliant rational analysis has led me to the following...I’m loath to say conclusions, since most of the work that I’ve done here is, I think, asking the right questions. That's almost always my approach, since if we can agree on the important values and ends we'd like achieved, the rest is just empirical. With that caveat, I'm pretty sure the main points are as follows:

1. What is education for? Whose interests are at stake, and which need to be protected?
2. How do we find out whether it's happening?
3. How do we fix it if it's not? What are the underlying causes of the difficulties in our educational system?

Obviously, this is political, social, sociological, economic and everything else you can think of, which is why it's so damn complicated. Preliminary answers, from a progressive concerns with outcomes rather than means:

What Education is For:
1. Education is for creating a certain type of citizen. It's often emphasized that we need to encourage math and science education, which is to say we are trying to create productive, economically fruitful citizens for the purpose of our national economy and for their own ability to sustain themselves. For example, I say this with some knowledge of the values of a system that includes vocational schools, and the arguments in favor of making an institutional distinction between those with working class skills and jobs and those with service class skills and jobs. It might be the case, for example, that that sort of division makes it easier for everyone to develop a minimum level of skills needed to succeed and become economically independent and to be a productive member of society.

Others argue, and I tend to agree with them, that what we need is broader than that. I still think it needs to be firmly defined and if at all possible, operationalized, so don't think I have some nebulous conception of an idealized Montessori-for-everyone system (as awesome as those schools are). I think we need to be cultivating critical thinking, logical reasoning, the ability to amass and process data in productive ways, argue, employ and critique rhetoric, understand abstract concepts. We should be instilling the valuing of knowledge, of empathy and understanding, of science and its capacity to understand and change the world. We should be teaching science as well as scientific thinking, math as well as formal logical thinking, history and an understanding of narratives and politics, economics, politics and how they work, and absolutely the ability to write well. These qualities are necessary for a functioning democracy, they serve national interests, and they are also vital for allowing people to self-actualize.

How we gain insight into our educational system:
2. Testing. Absolutely. I would never argue against the use of empirical data to understand whether or not important things are happening. I get fairly apoplectic when I see the phrase, “policies that over-rationalize teaching and learning.” Over-rationalize? As if rationality isn’t the way to go? The problems are: what are we testing, and what are the effects of testing? The answer to the second is clear: not always, but often, teaching to the test, overly standardized classrooms and teacher cheating. The reason for this, I think, is because of the answer to the first question, which is mostly multiple choice questions about specific knowledge bases. If we had broader questions, more essays and questions about critical thinking, (tests for which have been developed. Mostly, they're just incredibly expensive to proctor and grade), then it would be hard to teach to the test, and even if it happened, the kids would be being taught the right things. I'm still in support of specific knowledge, of course; people need to be mathematically and scientifically literate. But we absolutely need to test better. I happen to agree with Michelle Rhee that cheating is probably not that common, and anyway, it would be much harder to cheat on these tests.

What the problems are:
3. Poverty, race, marginalization, poor parent involvement, not enough money, not enough ideas, poor health and yes, in some part, bad teachers. So what do we do? A few things. We should absolutely be funding, probably on a federal level, extracurricular activities, after school programs, breakfasts, clubs, sports, medical facilities on campus for free, etc. These are proven to raise student performance and also increase the well-being of entire communities. See here, here and here. They allow parents to work throughout the day and make more money, make school a safe and healthy environment, keep kids off the streets and reduce the disparity between wealthy neighborhoods and poorer neighborhoods.

As for ideas, somehow we need to encourage alternative learning methods like plays and dance and projects for students who learn differently. We don't have to test them differently, but we should teach differently. We should definitely be launching pilot programs. Charter schools are great for this, when they're not funded by corporate interests who want all schools privatized. Longer school years would be great.

And sure, we should fire bad teachers. With good tests as well as a good understanding of pedagogy, we can see who those are. There's a metric, for example, called "value added", which measures how much teachers add to a student's ability in a year, regardless of the starting point. We know that bigger classes with better teachers are better than smaller classes with worse teachers. I'm ok with bonuses and performance pay mostly, I'm just worried about what that's going to incentivize, because it may not be creativity. What would be really helpful would be higher standards (they should have a degree in what they're teaching and we should pay for them to get masters degrees in education) and pay them all much more money to get the good ones. Unions can play a great role in this. We focus so much on teacher tenure and such, but unions do other things as well. I worry, for example, that this more dynamic, market-based system might cause more movement of teachers between schools, and that lack of continuity can be bad for children's learning and development.

These are my ideas, and they're probably overly idealistic and would cost a lot of money, but it might help us rethink how we approach education. At least we should be asking the right questions.

EDIT: I forgot to mention that many of the ideas I have here about the importance of alternative methods of learning and the construction of citizens I got from Martha Nussbaum's new book, Not for Profit. Although, I suppose I should point out that the ancient Greeks pretty much had those things figured out, with the balance between gymnastics, music and mathematics, and education as cultivation. Also pretty much every political philosopher ever had this opinion about education.


  1. Fourth time posting a comment. Really frustrating.

    1. Unions make it harder to fire all teachers. This isn't a problem with the good ones, but with the bad ones it makes it so that the process can take months if not years. (See:

    2. Unions tend to negotiate compensation that's deferred (ie, healthcare and pensions). This creates incentives for teachers not to rock the boat in order to wait it out to get their pensions and health care benefits. This means there are fewer bad teachers, but also fewer great teachers, as more teachers just cluster around mediocre than without these deferred benefits (and with just straight up higher salaries instead).

    3. Having fewer teachers but making them all of higher quality is a clear solution that's not just "throw more money." I mean, let's be honest: any problem can be trivially solved if there's more money in it, the trick is having a budget constraint.

    Lastly, more movement of teachers in a market based system isn't bad so long as principals are smart enough to realize that teachers should be fired/hired at the beginning/ends of school years rather than in the middle. Further, this would likely only be a problem if students were harmed by bad teachers leaving, which I doubt they are. (The harms of good teachers leaving already exist in the status quo, since good teachers are always offered better opportunities elsewhere).

  2. Chana,

    Thank you for the wonderful article. I have a follow up question and comment which I hopes illustrate some of the challenges we face.

    "As for ideas, somehow we need to encourage alternative learning methods like plays and dance and projects for students who learn differently."

    Since the mid-1960s we have had a wealth of ideas added to our commonly held pedagogy around education. Many of these ideas are very successful, yet marginalized due to being incongruent with the expectations of major stake holders in our current education system. Please see here or one of the best examples of radical and effective education ideas at work.

  3. @Alex Zhao: Sorry you had trouble commenting. Is it a problem with the site?

    1. I agree; in fact, I explicitly mentioned this as one of the potential problems. But in the article at the Huff Post written by Rhee herself, she defends unions and gives an account of how she worked with them to come to an understanding about the best solution. Tenure may be misapplied, but it's not necessarily inherently bad.

    2. That's an important analysis, to be sure. I think that higher salaries have a lot going for them, including as an incentive to enter teaching at all. The problem is that this is not just the teachers; the school systems love deferred payment because then they can, well, defer payment. Same thing with unions in general, where management won't budge on pay so they get better benefits. The thing with high salaries, too, is that we acknowledge as a society that people don't always make the right decisions, so we have social security instead of tax incentivized retirement plans like Ron Paul wants. Similarly, we want our teachers to be able to retire even if they didn't spend their high salaries wisely, especially because then when they're old and senile, they'll leave so we can higher a new, better, young one.

    3. I'm not sure any problem can be trivially solved with more money. Bill Gates has thrown a lot of money at this problem, to no avail. Obviously it helps, but it matters what you buy with that money. Yes, working on a budget would be great, and I think that solution is a start.

    4. Good point, I agree.

  4. @Robert:

    Thanks so much for reading and commenting! That looks like a really fantastic system, and I hope you have success with it. I'm sorry, though, I didn't catch the question...

  5. Well in general, as for the first point (both with regards to tenure and collective bargaining), the consistent abuse of that right has made me hesitant to continue to allow teachers the same broad discretion. Moreover, the realities of the economic facts make it very hard for me to believe that the benefits that unions love to negotiate for are feasible. Look at how the UAW's benefits were just not feasible because of the big three's collapse.

    2. While it's true that governments also like these kinds of benefits, one of the reasons they do is because of the difficulties of negotiating salaries with unions. If the government were allowed greater leeway in teacher hiring and firing, the fact that the salaries are higher (and the benefits are lower) would make them seem more like any other employer, likely deflecting the flack. I have no problem with retirement plans, however I think most people who are in the career of teaching are more than willing to save for retirement: moreover, the major problem isn't that you should mandate people save for the future (having opt out 401k's and the like do that), the problem is that relative to the amount of money they pay in, teachers get out a lot more in terms of benefits. Additionally, the structure of pensions often creates perverse incentives: most pensions actually pay out based on the last few years' average, rather than the career's average, which means teachers put in a lot of overtime late in their career in order to maximize their pensions. The fact that they can cash out pretty early relative to corporate work also makes pension plans problematic.

    As for three, I don't mean trivially in the sense that it'd be easy, I meant that more money always will produce better results, all other things equal.

  6. So, I wrote you a really long (like, pages long) comment on this. And then clicked "post comment" and it said there was an error. And the back button did not retrieve my beautiful comment! So let's sum up quickly:

    - HI! I read your blog now!

    - Teacher cheating is common (see Levitt's studies) - will more incentives really change that? How much can you cheat before you get caught?

    - Can we increase the prestige/sustainability of teaching without financial incentives (think Teach for America)?

    - Do we need education standards for teachers (master's degrees) or intelligence standards for teachers (aptitude and difficult, material-specific qualifying exams)? Would it make a significant difference?

    - Happy to talk education with you any time. :)

  7. thank you for your blog, it help me with букмекеры and was interesting to read