Join

Enter Details

Comment on stories, receive email newsletters & alerts.

@
This is your permanent identity for Gizmodo, Kotaku, Lifehacker and Business Insider Australia
Your email must be valid for account activation
Minimum of 8 standard keyboard characters

Subscribe

Great discounts, competitions and special offers from our partners.
Email newsletters aren't ready to go out just yet, but will contain a brief summary of our top stories, news alerts, plus details of competitions and reader events.
Having issues creating your account? Contact Support

Forgotten Password

Enter Details


Back to log in

JOURNAL EDITOR: The Famous Reinhart-Rogoff Debt Paper Did Not Go Through The Normal Refereeing Process

WikipediaHow did Reinhart and Rogoff’s now-infamous (ok, in economics circles) Excel error make it through a peer-reviewed journal?

Answer: it wasn’t a peer-reviewed journal — or at least, the edition their work appeared in wasn’t.

If you’re new to this story, economic professors Carmen Reinhart and Ken Rogoff published a paper called “Growth in a Time of Debt” in the American Economic Review in 2010.

They argued that high debt levels lead to low growth for countries, and at 90 per cent debt levels, growth goes negative.

Everyone wrote this up at the time, and the study has been repeatedly cited by many deficit hawks.

But many had misgivings about their findings, because lots of people had trouble re-creating them using the data Reinhart and Rogoff cited. (There was also a lot of scepticism about whether the relationship between high debt and low growth was causal)

This week, a group of UMass economists published a paper saying they think they figured out, in part, why: the pair made an error in Excel that caused an entire subset of countries to be excluded from their data set. Finance blogger Mike Konczal was among the first to notice the new paper.

By Wednesday, Reinhart and Rogoff had fully owned up to the Excel oversight. 

Still, many are wondering how could it have gone unnoticed by the Review.

We spoke with Virginia economics professor William R. Johnson, who edited the edition of the Review in which the paper first appeared. 

This annual edition, “Papers and Proceedings,” differs from all others in that the papers come out of presentations made at the yearly meeting of the American Economic Association, he said.

The papers are personally selected by the AEA’s president-elect, in consultation with a  committee.

As a result of these unusual circumstances, Johnson said, the editing of “Proceeding” papers is less rigorous.

“Normal peer review doesn’t happen for these papers in the way of other issues of the AER.”

Here, is what he’s talking about. From the introductory section of the edition: 

AERThe data must still be replicable.

But author prestige also comes into play, Johnson said, adding that that was true for all AER papers, not just the ones that appear in “Proceedings.” 

And it’s not necessarily the referees’ responsibility to replicate the findings.

“A lot of this is based on trust and people’s reputations. The main mechanism for finding an error is after publication — someone tries to replicate it.”

It’s possible that if the paper had gone through the normal refereeing process it would have been caught, Johnson said, but he’s not confident.

“If it’s some internal data error, that’s very hard to detected without replication,” he said.

Reinhart and Rogoff are standing by their conclusions, despite the error.

But Arindrajit Dube (via Felix Salmon) has now made a convincing case that Reinhart and Rogoff have their causality reversed: rather than large debt driving slow growth, slow growth leads to large debts.

This is probably not over.  

Follow Business Insider Australia on Facebook, Twitter, and LinkedIn