Skip to Content

Vanderbilt News

Teacher performance pay alone does not raise student test scores

by | Posted on Tuesday, Sep. 21, 2010 — 12:45 PM

Rewarding teachers with bonus pay, in the absence of any other support programs, does not raise student test scores, according to a new study issued today by the National Center on Performance Incentives at Vanderbilt University’s Peabody College of education and human development in partnership with the RAND Corporation.

Watch highlights of the announcement.

Watch the full announcement with Q&A.

This and other findings from a three-year experiment – the first scientific study of performance pay ever conducted in the United States – were released at a conference on evaluating and rewarding educator effectiveness hosted by the National Center on Performance Incentives at Vanderbilt.

Paying teachers bonuses based on their performance has been a controversial issue nationwide since the 1950s, but until now the concept has never been scientifically researched.


“We tested the most basic and foundational question related to performance incentives — Does bonus pay alone improve student outcomes? – and we found that it does not,” Matthew Springer, executive director of the National Center on Performance Incentives, said. “These findings should raise the level of the debate to test more nuanced solutions, many of which are being implemented now across the country, to reform teacher compensation and improve student achievement.

The Project on Incentives in Teaching, called the POINT Experiment, took place over the 2007 – 2009 school years with participation by mathematics teachers in grades 5 through 8 in Metropolitan Nashville Public Schools. Nearly 300 teachers, approximately 70 percent of all middle-school math teachers in Nashville’s public schools, volunteered to participate. The complete study, including setup and analysis, began in 2005 and ended in 2010.

POINT tested no other types of incentives or systems of support for the teachers, such as professional development or guidance on instructional practices – many of which have evolved over the five years since POINT began.

“We designed POINT in this manner not because we believed that an incentive system of this type is the most effective way to improve teaching performance, but because the idea of rewarding teachers on the basis of students’ test scores has gained such currency,” Springer said. “We sought a clean test of the basic proposition: If teachers know they will be rewarded for an increase in their students’ test scores, will test scores go up? We found that the answer to that question is no. That by no means implies that some other incentive plan would not be successful.”

Here’s how the POINT experiment worked:

  • Following a year of detailed project design by a multi-disciplinary team from Vanderbilt and RAND, all middle-school math teachers in Nashville were invited to volunteer for the experiment. Approximately 70 percent of all middle-school math teachers in Nashville’s public schools stepped forward to participate.
  • Approximately half of the nearly 300 volunteers were randomly assigned to a “treatment” group, in which they were eligible for bonuses of up to $15,000 per year on the basis of their students’ test-score gains on the Tennessee Comprehensive Assessment Program (TCAP).
  • The other half were assigned to a “control” group not eligible for these bonuses. Teachers were evaluated based on an historical performance benchmark for MNPS teachers, not on competition with one another. All teachers in the treatment group had the chance to earn bonuses. (The names of participating teachers – and which group they were in – have been kept confidential by the research team.)
  • The annual bonus amounts were $5,000, $10,000 or $15,000. Over the course of the experiment, POINT paid out more than $1.27 million in bonuses. Overall, 33.6 percent of the original group received bonuses, with the average bonus being approximately $10,000.

Teacher attrition occurred during the experiment. About half of the 296 teachers who initially volunteered remained through the end of the third year. The teachers who left the study either left the school system, moved to other grades or stopped teaching mathematics. Only one participating teacher specifically asked to be removed from the experiment.

While there was no overall effect on student achievement across the entire treatment group, the researchers found a significant benefit for fifth graders in Year 2 and Year 3 of the experiment: fifth graders taught by teachers who earned bonuses did show gains in test scores. However, the effect did not carry over to sixth grade when students were tested the following year. Springer said this finding raises questions about what is different about fifth grade and what factors –student development, curriculum, teaching and classroom structure – may have played a role.

He also noted that implementation of POINT went smoothly, with no complaints from teachers about the calculation of bonuses, the payment of awards, bonuses they did or did not receive or the fairness of the process. This in itself is a significant finding, Springer said, because historically, teacher associations have opposed performance or merit pay plans, particularly if the pay plan awarded teachers solely on their individual value-added score.

Springer attributed this smooth implementation of the POINT experiment to a broad partnership involving the Metropolitan Nashville School Board and Metropolitan Nashville Public Schools administrators, the Mayor’s Office, the Metropolitan Nashville Education Association, the Nashville Alliance for Public Education, the Tennessee Education Association and the Tennessee Department of Education. The POINT experiment team received guidance and support from these organizations, as well as the participating teachers, throughout the project.

“We believe there is an important lesson here: Teachers are more likely to cooperate with a performance pay plan if its purpose is to determine whether the policy is a sound idea, than with plans being forced on them in the absence of such evidence and in the face of their skepticism and misgivings,” Springer said.

The full report is available at http://www.performanceincentives.org/. Archived video of the announcement will be available at that website Sept. 22.

The National Center on Performance Incentives at Vanderbilt University’s Peabody College of education and human development partnered with the RAND Corporation, a nonprofit public policy research institute based in Santa Monica, Calif., in 2006 to complete the study. The POINT experiment was funded by the U.S. Department of Education’s Institute of Education Sciences. Performance bonuses were funded by a private donor.

Springer is an assistant professor of leadership, policy and organizations at Peabody College. For more information about Peabody College, ranked the No. 1 education school in the nation by U.S. News & World Report for the past two years, visit http://peabody.vanderbilt.edu.

Contact:
Melanie Moran, (615) 322-NEWS
melanie.moran@vanderbilt.edu


  • AJ

    “He also noted that implementation of POINT went smoothly, with no complaints from teachers, with no complaints from teachers about the calculation of bonuses, the payment of awards, bonuses they did or did not receive or the fairness of the process.This in itself is a significant finding, Springer said, because historically, teacher associations have opposed performance or merit pay plans, particularly if the pay plan awarded teachers solely on their individual value-added score.”nnSpringer is misunderstanding. He insinuates that teachers unions oppose merit pay plans because they are “unfair” or inaccurately calculated. This is not true. Teachers unions oppose them because merit pay plans are largely ineffective and don’t solve the root problem of “how do we support teachers in their continuous growth cycle, so they can do their best to reach every student, every day?”

  • Kyliiee

    Thank the Lord for this website, I’m doing a research paper on this and I needed a negative side!

  • Dawn

    If the evidence is clear and there doesn’t seem to be a study in the nation that proves that merit pay works, why then are the fed sinking $440 million in to the same damn study right now?u00a0 I know because I am part of it…. a 3rd grade teacher in Colorado!u00a0 Nothing seems more like a complete waste of money especially when the research is already there!!!!!u00a0 Put this money into ending poverty in this country!u00a0 Only then will we see educational progress!!!

  • Dawn

    If the evidence is clear and there doesn’t seem to be a study in the nation that proves that merit pay works, why then are the fed sinking $440 million in to the same damn study right now?u00a0 I know because I am part of it…. a 3rd grade teacher in Colorado!u00a0 Nothing seems more like a complete waste of money especially when the research is already there!!!!!u00a0 Put this money into ending poverty in this country!u00a0 Only then will we see educational progress!!!

  • DRSherw

    It seems that this study, focused on “bonus” pay,u00a0missed the point.u00a0 It is not “bonus” pay that motivates.u00a0 It is performance measurement and subsequently differentiated pay increases,u00a0 resulting in base pay differentiation, determined by sustained performance, that matters.nnBonuses are one time events,u00a0possiblyu00a0repeatable, possibly not.u00a0 nnIf pay systems are based upon sustained categories of performance, measured annually, impacting the size and frequency ofu00a0increases in base pay, the greater the performance.u00a0 The measurement system must identify superior,u00a0above agerage, average, below average, and failingu00a0performers.u00a0 Pay increases should vary by category from generous to none.u00a0u00a0Below average andu00a0failing performers should receiveu00a0coaching and written definitions of what and how to improveu00a0performance.u00a0 Failing performers, eitehr improve are should be terminated.u00a0 When good teachers are appropriately rewarded and the underperforming teachers are separated, then overall performance will improve.u00a0u00a0Of course, this requires that “management” perform management functions, such as setting goals, measuring performance, recognizing excellent performance, coaching underperformers, and finally separating those that do not improve.nnPay and increases, alone cannot change behavior.u00a0 However, the lack of rewards for high performance inevitably leads to overall declining performance.u00a0 Good candidates will choose other fields for employment. u00a0Over time, the good performers will either leave or conclude that the effort to excell is unimportant and will likely cease their efforts to excell or improve.u00a0 Poor performers will cling to their jobs.u00a0 The result is an inevitable decline in overall performance.nnPay systems based upon length of service and degree credentials really provide incentive only to “hang in” and to continue padding the accumulation ofu00a0degrees.nnIu00a0believe this study to be too narrowly focused, thusu00a0providingu00a0limitedu00a0meaningful guidance for improving the education system.n



VU Tweets