Instructional Incentive

Few issues in K-12 education are more fraught with controversy than that of linking teacher pay to student achievement. This doesn't mean 'performance pay' isn't an idea whose time has come.

By Anita Neal Harrison

When he talks about systems that prohibit performance pay for teachers, Michael Podgursky uses the dirtiest word in an economist's vocabulary.

"It's inefficient," says Podgursky, a professor of economics at MU. "That's the bottom line. We're not spending money in an efficient way to get the biggest bang per buck in terms of student achievement."

Podgursky is one of the nation's most visible researchers investigating plans to reform educator pay. He is the coauthor of Teacher Pay and Teacher Quality, a 1997 study that questioned the widely held assumption that boosting teachers' pay improves the quality of their teaching. He also serves on the advisory boards of several prominent organizations that urge school systems to explore the feasibility of performance pay plans.

His enthusiasm for this line of research and its potential role in school improvement belies decades of disagreement, much of it contentious, surrounding the use of performance incentives in educator compensation.

"This is the most hotly debated topic in education policy today," says Matthew Springer, a research assistant professor at Vanderbilt's Peabody College of Education and Human Development, director of the college's National Center on Performance Incentives, and research collaborator with Podgursky. "More and more states are building data systems that link student performance over time to schools and classrooms. So we have a better picture of how schools and teachers are performing. As we researchers explore this data, we're learning about what works and what doesn't work in compensation policy."

Ninety-six percent of the nation's public school districts -- districts employing some 3.1 million teachers -- report use of a "single-salary schedule" guaranteeing teachers will earn more money as they accumulate years of experience and advanced training. Such schedules mean that within a district, teachers with equal experience and education receive the same salary regardless of what they teach, where they teach, or how well they teach.

So far, Podgursky asserts, research has shown that the two variables for which teachers are typically rewarded, education and years of experience, show only a weak correlation with student achievement. He points, for example, to a 2002 study by University of Washington public affairs professor Dan Goldhaber, published in the journal Education Next, that found only about 3 percent of the contribution teachers made to student learning was associated with teacher experience, degree attained, certification status, and similar measures.

"Economics is about using resources efficiently: The basic idea comes down to costs and benefits. You try to estimate costs, you try to estimate benefits, and then you try to put your money where it has the highest ratio of benefits to cost," Podgursky says.

The problem, he explains, is that school districts are spending more and more per student with little to show for it.

"It isn't a question of cheapness. We are pouring resources into this sector, and still test scores have been fairly stagnant. The black-to-white achievement gap hasn't narrowed in a decade. So it's not just a question of throwing more money in. You've got to move resources to where they're having the biggest bang. You've got to take money from this thing that's not working and put it over here into this thing that is working, and 'this thing' could be a curriculum, it could be equipment, or it could be people. But it's the same old argument about cost efficiency."

School districts are getting more serious about maximizing resources in response to both national legislation, such as No Child Left Behind, and state legislation such as the Missouri School Improvement Act, that require schools to improve student achievement or face consequences.

"There's nothing in No Child Left Behind per se about teacher incentives, but there is about performance," Podgursky says. "So the point here is how do you get the performance? There are a variety of tools -- alternative teacher certification, charter schools, curriculum changes -- but this [performance-based pay] is one that really hasn't been used, and again, as an economist, we think incentives are important, so we think this is an area where there should be more attention, effort and experimentation. Because we think incentives matter."

It's more than a hunch. In a 2007 study published by the Journal of Policy Analysis and Management, Podgursky and Springer critique previously published studies on salary incentives. Their review includes only those studies with a rigorous study design, the authors say. Of the nine considered, seven studies associate incentives with positive student achievement effects. In every study, moreover, incentives raised the level of the variable tied to the incentive.

"The overall findings ... stand in rather sharp contrast to the mixed but generally negative findings of production function studies of the effect of teacher characteristics such as teacher certification, education, or class size," wrote Podgursky and Springer.

The studies examined the "motivation effect" of incentives, or how well incentives motivate teachers to perform better. In addition, Podgursky and Springer argue that incentives offer another key benefit: They tend to attract and retain people who are good at the activity for which there is an incentive.

Think of it this way, Podgursky says. If you tie an incentive to a specific outcome, say, test score gains, then you will not only motivate teachers already on the payroll to find ways to boost scores, but will also tend to attract and retain teachers who are skilled at raising student scores. In addition, such a system may encourage those who are not capable of raising scores to look for work elsewhere.

To illustrate this "selection effect," Podgursky draws a bell curve on the dry-erase board hanging in his office. "If all teachers were equally effective, then this might not have such a bang, but here's what we're observing," he says, pointing to the area of the curve representing the least effective teachers. "For every one of these who leaves, who gets replaced by an average teacher, you get a bump in performance."

Over time, he adds, this creates no small effect. Studies from the business world suggest about half of the gains from performance pay come from this selection effect. With some studies showing a positive motivation effect and others indicating a positive selection effect, Podgursky says, "there is a green light for districts to go out and begin experimenting here."

Podgursky concedes that his research does not go so far as to reveal a best set of practices -- for example, an optimal size of bonuses or a mix of individual versus group incentives. Districts, he says, will have to use "trial and error" to determine the best system for them.

And that has some educators concerned.

"The devil is in the details," says Carolyn Herrington, dean of the MU College of Education. "I think among educators in general, we'd be concerned if policymakers go down this road too quickly."

Those who argue against pay-for-performance voice three main concerns: It encourages "teaching to the test," evaluations are too subjective and it promotes unhealthy teacher competition.

"Teaching to the test" refers to the phenomenon of teachers and/or schools narrowing their curriculum to focus only on those subjects being tested. "There's a lot of argument out there that that's occurring under No Child Left Behind, that because we just test math and communication arts, that schools are dropping arts and so on," Podgursky says. "I haven't seen any systematic data on it, but there are enough anecdotes out there and commentary to suggest that there probably is some of that."

A fear of a narrowed curriculum is one reason the National Education Association, the nation's largest teachers union, opposes any program that ties incentives to test scores. "Any system that has us teaching to the test and narrowing the curriculum is not a good system," says Bill Raabe, director of the National Education Association's collective bargaining and member advocacy unit.

Herrington agrees, adding that not only do the untested subjects have value in themselves, but they also have value in keeping kids eager to learn. Many kids look forward to school because of music, art and P.E., she argues.

However, unlike Raabe, Herrington does not believe a narrowed curriculum is an unavoidable consequence of incentives. She, along with Podgursky and the American Federation of Teachers -- another powerful teacher's union -- believes it is possible to design a comprehensive incentive program that includes both principal and peer evaluations along with test scores in its measurement of teacher effectiveness. Not only would such evaluations discourage teaching to the test, but they would also allow districts to reward teachers who are doing good work that might not be reflected in results from student testing.

"So, for example," Podgursky says, "suppose we always assign the kids that act up to you because you are good at discipline. That will hurt you in terms of test score gains. But the principal will know that, so I'm not going to be stupid. I'm going to take that into account in looking at an overall evaluation for you."

The subjective nature of evaluations, however, brings up another concern. One of the key reasons school districts adopted the single-salary schedule was to prevent administrators from allowing personal bias to guide their compensation policies. Systems based on subjectivity, Raabe argues, create "too much opportunity for tomfoolery."

That's why the American Federation of Teachers says incentive plans must have easily understood standards for the rewards. "They have to be clear," says federation spokesman George Jackson. "Past merit pay programs have been very complex, very nebulous, to the point that people were suspicious of how they were being implemented."

Sometimes, Herrington says, the breakdown happens not because of dishonesty but confusion. "We need a lot more consensus about what good teaching looks like," she says. "School systems need to train principals and teachers how to observe teachers and how to accurately measure their performance. That, of course, has its own costs. These are not obstacles that cannot be overcome, but again, are districts willing to do it, and do it well?"

Though the National Federation of Teachers does not support compensation based on test scores or evaluations, it does endorse the use of evaluations and tests to measure performance. "The main purpose... is to improve practice so every child is getting the education he or she deserves," Raabe says. "It's not about pay; it's about practice."

Herrington agrees, at least to some extent. "That is probably the biggest bang you're going to get," she says. "Looking at student tests and these evaluations is what the teacher learns about how to improve his or her performance in the classroom."

A third criticism, that incentive systems promote unhealthy competition among teachers, stems from so-called "tournament" systems which put teachers in competition for a fixed number of merit rewards. Many educators oppose such systems because, they say, teachers must work together as a team: They need to brainstorm, strategize, and otherwise provide professional support, advice and expertise for one another. They fear -- with good reason, some research shows -- that a competitive pay system would undermine such collaboration.

Podgursky points out, however, that not all incentive plans pit individual teachers against each other. Some plans reward teams of teachers or whole schools, and some plans reward teachers based on their performance compared to some fixed standard so every participating teacher has the opportunity to earn the reward. But this, too, has a downside in that it can expose districts to financial pressures.

All this brings up a final problem for merit pay systems: If districts implement these plans, they may not be able to finish what they start.

"Teachers have learned there's a very good chance these systems won't last," says Herrington. "Something else comes along and takes the money." Adds Jackson: "You have to ask, 'Is this something we're going to be able to maintain for the long term?' If you can't, then it's not going to have any lasting effect on student achievement."

With so many variables to consider, policymakers, educators and administrators should not rush into an overhaul of the current system, Springer says. "We need to go slow and go small and evaluate big."

Herrington thinks so too, saying teacher compensation is only one piece, and perhaps not even a big piece, of what she calls the very large, very complex public education puzzle.

"It's dangerous to think one particular change will be strong enough to achieve the effect you're looking for," she says. "It's also dangerous to think that when you change one part of a system, it won't trigger another change that may dilute the desired change and introduce changes that are not desirable. We really don't know how these things work. I think policymakers should go slowly and carefully with them."

More experimentation is most certainly needed, Podgursky says, adding that there is already one groundbreaking experiment taking place in the Metropolitan Nashville Public Schools. Sponsored by the National Center on Performance Incentives, the Nashville study will evaluate, over a three-year period, the effects of a pay-for-performance trial involving 300 math teachers and their students.

Podgursky hopes more such investigations follow. With new longitudinal data sets from the states, he says, researchers will have the information they need to gain insight on improving achievement.

"But we need more school districts that are willing to innovate and experiment. There are some school districts that will take the lead. Many won't. For most, it will be business as usual. But some, if you give them the opportunity, they will push. Just like in any endeavor. Some will be the leaders. They'll break the ice and push forward."