There’s a cliche about how if you torture data long enough, you can get it to confess anything you want.
That’s pretty much true, and it’s rampant in Washington DC. In fact, it almost seems like a requirement for working there.
Fox News was recently the subject of a brutal expose about its abuse of statistics.
So we put together this very simple guide to doing it.
Gas prices are an excellent way to examine statistical tricks used to obscure the truth. During the campaign, politicians, interest groups and activists claimed that gas has doubled — or even tripled — under Obama.
Let’s dissect that and figure out why the statement — Gas has nearly tripled under Obama — is defensible, but ultimately obscures facts.
We’re looking at data from the Energy Information Administration about the spot price for a gallon of gasoline.
Here’s what we’re going to claim: Under Obama, gas prices have nearly tripled. When he was inaugurated, gasoline only cost $1.15. In September 2012, gas cost $3.27.
Here’s why this statement “Gas has nearly tripled in price under Obama” is a perfect example of manipulating statistics.
- It cherrypicks numbers amenable to the point I’m trying to make.
- It involves some favourable rounding.
- It ignores context.
- It argues correlation, not causation.
Here’s what that statement implies:
[credit provider=”Walter Hickey/BI Data from EIA”]
That seems like a very strong argument right there. It’s resonated with a lot of people and is used to support the point that Obama has been anti-gas, anti-consumer, and anti-environmentalist.
Let’s look a little deeper.
Here’s the real chart of the cost of gasoline by month from January 2009 to October 2012:
[credit provider=”Walter Hickey / BI Date EIA”]
So the first thing to notice is that our claim compares the very lowest month with the very highest month, even though I have additional data (October 2012, May 2012) that shows that September was a particularly high point. This is cherrypicking data.
The next thing you’ll note is that the statement “the price of gas has has tripled under Obama” entails a bit of rounding. The real value is not a tripling, or three times as much, it’s 2.85 times as much.
You may say that this isn’t a big deal. Think of it this way. If gas really had tripled in price since January 2009, the price would be $3.44 per gallon.
That’s 17 cents more expensive than the highest month (September 2012) and 47 cents more expensive than the 2012 average so far, $2.97.
The average yearly increase in the price of gas since 2000 is 18 cents annually. Just by rounding our multiple up, we’ve added one or two years’ worth of gas price increases to the estimate that didn’t actually happen under Obama. That’s why rounding is important.
But here’s the main, crucial point. This is data without context. Here’s the main chart that shows why this statement — gas has tripled under Obama — isn’t particularly fair.
[credit provider=”Walter Hickey / BI Data EIA”]
The price of gasoline crashed, just like the price of everything else in the market crashed, in the fall and winter of 2008 as the recession began. Obama had nothing to do with this, but gas prices are at an all time low in January 2009 — the month of the inauguration.
This is why context is crucial. The growth rate of the price gas looks relatively consistent between the pre-crash and post-crash charts.
Looking at this chart, it’s abundantly clear that the incumbent president didn’t have a lot to do with the growth of the price of gas.
Finally, take a look at this chart showing the price of oil month by month since June 1986:
[credit provider=”EIA, Chart Walter Hickey BI”]
We can look at this chart and learn a couple of things. First, it doesn’t seem like the party of the President really has a ton to do with the month by month price of gas. Gas remained stagnant, increased, and decreased during the terms of both Republicans and Democrats.
Next, It seems rather obvious that the price of gasoline has more to do with market forces than the incumbent president. For instance, compare this chart with the Dow Jones Industrial average from the same period.
Just because gas has increased in price over the course of the Obama presidency doesn’t mean that it’s a direct result of Obama’s policies, just as we can’t say that the growth in the DJIA is because of Obama’s policies.
This isn’t to slam one argument over any other. Each side does this to some degree.
It’s all about choosing a dataset that’s conducive to their argument.
A Democrat trying to make the argument that Obama is strong on gas prices might find the number of new permits issued to drill in the gulf.
[credit provider=”Gulf Permit Index, GNO Inc” url=”http://gnoinc.org/wp-content/uploads/GPI+-2012.08.24.pdf”]
And cherry pick the lowest month — right after the Deepwater Horizon well explosion — and a higher month — Let’s say a year later, June 2011 — and say that over the course of a year, the number of permits issued per month nearly quadrupled!
This cherrypicks data, involves favourable rounding (3.5 gets rounded up to 4), ignores context entirely (See: Horizon, Deepwater) and alleges correlation when the issuance of permits has more to do with demand for the permits than much else.
In short, you’re being lied to almost constantly by people who can make clever arguments.