The Grad Student Who Busted Reinhart & Rogoff Explains How Open They Really Were With Their Data

thomas herndon

This weekend, economists Kenneth Rogoff and Carmen Reinhart dropped a bomb on the economics world in the form of a big open letter to Paul Krugman, blasting him for his uncivil and nasty tone in the debate over recent years.

Remember, Rogoff and Reinhart were the economists whose work showed that growth for nations dropped off notably once debt to GDP crossed the 90% threshold. This result was shown to be flawed by UMass Amherst grad student Thomas Herndon, who discovered an Excel flaw in their work.

In their open letter, Rogoff and Reinhart dispute the popular notion that they were keeping their data secret.

They write:

The accusation in the New York Review of Books is a sloppy neglect on your part to check the facts before charging us with a serious academic ethical infraction.  You had already implicitly endorsed this from your perch at the New York Times by posting a link to a program that treated the misstatement as fact.

Fortunately, the “Wayback Machine” crawls the Internet and periodically makes wholesale copies of web pages. The debt/GDP database was first archived in October 2010 from Carmen’s University of Maryland webpage.  The data migrated to in March 2011.  There it sits with our other data, on inflation, crises dates, and exchange rates.  These data are regularly sought and found for those doing research who care to look. The greater disclosure of debt data from official institutions is testament to this.  The IMF began to construct historical public debt data only after we had provided a roadmap in the list of our detailed references in a 2009 book (and before that in a 2008 working paper) that explained how we had unearthed the data. 

Our interaction with scholars and practitioners working on real world questions in our field is ongoing, and our doors remain open. So to accuse us of not sharing our data is an unfounded attack on our academic and personal integrity.

Remember, part of the Thomas Herndon story is that he couldn’t replicate Rogoff and Reinhart’s result (that debt dropped off precipitously at 90% debt-to-GDP) and then discovered a flaw only once Carmen Reinhart sent him a copy of their Excel document, at which point he noticed a flaw.

Via email, we asked Herndon his precise take on what happened, and what he makes of their claim that their data was always made available.

Long-story short, Herndon agrees that they did post links to their data sources, but never the spreadsheet, and that it was the spreadsheet that allowed the flaw to be discovered.