Finding truth in the midst of truthiness

10-22-2008 - Edited to fix a stupid graph mistake!

Staying informed about politics during an election year is like trying to find a tiny nugget of gold hidden inside a marshmallow the size of a small house. You chew your way through the sweet gummy pablum, handfuls of substanceless garbage being shoveled into your mouth, and by the time you get to the gold nugget at the middle, you're just too sick to care.

Only it's even worse than that. In an election, propaganda is being slung about like dung in a monkey fight. The best you can do is to keep your head down and try not to get any on you.

But you can't just get out of the way, oh no. You need to pick the most nutritious marshmallow, the monkey with the sweetest-smelling poo. If you don't, the rest of the monkeys in the monkey house are liable to pick the gibbon with the most prominent nose, who will likely end up getting half of your brethren eaten when he decides to invade the crocodile habitat.

As a scientist, this is particularly frustrating. I like facts, numbers, and other silly things, like statements at least loosely tethered to reality. Frankly, it's important to me to know if politicians (all politicians, not just my politician of choice) are spouting bullshit. For this, I find websites like politifact.com to be a good resource.

For those that aren't familiar with it, politifact takes statements, comments, and ads made by the major candidates (sorry, Bob Barr) and fact-checks them, ranking them from truthful to "pants on fire" dishonest (for instance, Hillary's bosnian sniper fire). Additionally, the pages for each candidate have graphs on them! Graphs! With real numbers!

But not enough graphs. So I took a lot of the raw data on politifact and crunched the numbers for you, as a way of apologizing for not writing anything in a month.

First of all, it seems like politifact is fairly even-handed. They've analyzed 284 statements made by Democrats, 279 made by Republicans, 113 by Obama, and 111 by McCain.

Here's how they categorize those statements:

PolitiFact (pronounced puh-lit’-eh-fact) is bolder than previous journalistic fact-checking efforts because we’ll make a call, declaring whether a claim is True, Mostly True, Half True, Barely True or False. We even have a special category for the most ridiculous claims that we call “Pants on Fire.”

And some of the things they look at in addition to simple statements made by the candidates:

We'll also publish an “attack file” – a home for fact-checking the attacks candidates make against each other. We recognize that in a world of political bloggers and “independent” political action committees, attacks don’t just come from the candidates themselves. So we will also check out many claims that enter the public discourse via a talk show host, a blogger or even a fictional character in a YouTube video.

The first thing I decided to look at were the statements made by both McCain and Obama. (with all graphs, click for a bigger version).

The bars represent the statements made by both Obama and McCain as a percentage of their total statements.

The first thing that pops out from this graph (for me, at any rate) is the heavy weighting of Obama's statements towards the true end of the spectrum, as well as his lack of "pants on fire" outright falsehoods. McCain's graph shows that he's uttered only slightly fewer false statements than true ones.

Next up is the "attack pages," which include attacks made by each candidate, as well as attacks directed towards them. First, those attacks made by each candidate.

Whoa. As shown by the chart, attacks made by Obama have been overwhelmingly factual, with an enormous 41% being completely truthful. Remember, these are percentages! Politifact has cataloged a similar number of attacks for each candidate, with 49 for Obama and 54 for McCain. almost 70% of Obama's attack are at least half truths, compared with only 53% for McCain.

Perhaps unsurprisingly, the trend is reversed for attacks aimed at the candidates. Remember - this is not just attacks from the opposing candidate, but attacks coming from ANY source.

Only 41% of attacks made against Obama are half-truths or better, compared with 69% of those directed at McCain. This means that of the monkey dung that Obama receives, 60% of it is undeserved. Compare that with McCain - 70% of the monkey dung flung at him deserves to stick.

I know what you're saying at this point, though. "But Jaaaaaaacob, a half-truth isn't worth as much as a whole truth, and a "false" can't be nearly as bad as having my trousers aflame."

And you know what, I totally agree. I'm a big fan of partial credit, as some of my former students will attest.

So, I gave each of the attack columns a "score," by assigning each category a point value. True statements are worth one point, mostly true is worth .75, half truths half a point, barely true is worthless, false statements lose you a point, and telling a whopper loses you a whopping two whole points. Remember - the attacks on each candidate are attacks coming from any source at all.

You may draw your own conclusions from this graph.