The curious case of Tel Opinion Research

I have tried to avoid getting worked up over early polls — as opposed to getting worked up over people paying attention to early polls. The research on their (in)validity as long-term predictors of election success is pretty clear on the Democratic side, but I see people get worked up over a poll every day, so when I saw someone dressing down a poll a few days ago, I took a closer look.

What I found was bizarre.

The pollster, Tel Opinion Research (TOR), is one I hadn’t heard of, and while I’m not an expert on this stuff, I’ve been reading polls since before poblano revealed his identity, so I’m familiar with a number of outlets.

But not these guys.

Now, by itself, that fact necessarily means nothing. Anyone, using rigorous statistical and sociological methods, can put together a good poll that beats the people at Marist or Zogby or Monmouth or whoever. But as I scanned the demographics of the TOR poll — polls that don’t reveal demographics are of less statistical value — I found myself wondering just what in the hell these guys were doing.

Here’s why:

Now, a fair number of people have asserted, generally without evidence, that the folks in the age bracket below mine don’t vote, or barely vote. That’s not remotely the case — and where a gap in turnout exists, it’s true across generations, meaning that it’s not like my generation and the one below it just don’t care. People are more likely to vote the older they get. Many of the people who voted at a 69.7 percent clip in 2012 voted at a roughly 50.9 percent clip in 1968. So if we’re going to celebrate the people who vote a lot, we ought to also call attention to what they used to do.

But however little you think of the 18–29 bracket, the notion that only 3 percent of the New Hampshire primary electorate is going to be 18–29 is ludicrous. Yet that’s what this poll reflects.

Here, for comparison, is 2016:

2008:

2004:

That’s a giant pile of images to look at. Here’s a table that breaks things down nicely:

Now, in 2008, when Nate Silver outed himself as poblano, he wrote this:

“In politics, [using data badly] might mean cherry-picking a certain polling result or weaving together a narrative that isn’t supported by the demographic evidence.”

I quote that line because in my view, TOR is weaving a narrative that isn’t supported by the demographic evidence. You can’t more than double the 65+ vote while using ⅙ the 18–29 vote and call your poll credible. It’s a political fiction.

TOR also oversamples liberals. Here’s a chart for that:

If I were looking at trends for 2020 expected turnout, I might go 29 percent or even 30 percent very liberal, but 40 percent is not supported by these data. So I don’t know what TOR is doing, but I don’t see how it’s factual.

2020 might be 4 percent conservative, 22 percent moderate, 45 percent somewhat liberal and 30 percent very liberal. But the notion that a plurality of New Hampshire voters will be very liberal is hard to justify, to me.

So that’s one state where TOR’s sampling seems horribly off. Does that problem repeat itself in any other TOR polls?

You bet your bippy. Here’s the South Carolina poll’s age demographics versus previous years:

If you more than doubled the 65+ turnout from the best year for the elderly vote, you would still not be where TOR ended up.

And the political leaning demographics:

36 percent simply doesn’t fit if you’re modeling 2020 after the 2004–2016 trend. I might go 26, but 36? No. Not one of these numbers fits, in my view. I might go 9/32/33/25, but the notion that the plurality of South Carolina Democratic primary voters in 2020 will be very liberal — which hasn’t happened in a contested contest dating back to 2004 — seems ludicrous to me.

And finally, TOR’s Florida poll demographics versus previous years:

2008 has an asterisk, of course, because Florida and Michigan moved their primaries up in defiance of the DNC, meaning that no candidate fought hard for the delegates and Obama wasn’t even on the ballot. So the data from that year are for entertainment purposes only, but we don’t even need them. Florida’s Democratic Party primary voter base is simply not as old as TOR’s poll suggests.

And the political leaning demographics:

Moderate Democrats have been a plurality in Florida Democratic presidential primaries in each of the past three contested events. So the notion that they’re going to nosedive from high 30s to mid teens is, again, ludicrous. Florida Democrats are absolutely becoming more liberal — but not this much so.

Now, clearly these data are … not hewing close to a realistic mathematical model of the Democratic Party primaries’ history. But why?

1. Typographical errors

I am a professional copy editor. I’ve been in the field for more than a decade, and I’ve been finding flagrant errors in wire service copy for forever. So I’m pretty good at finding mistakes. And the South Carolina and New Hampshire reports have core formatting differences for which one of the following may be the case:

A) The proofreader flagged them, but nobody fixed them.

B) The proofreader did not flag them.

C) They were not flagged because nobody proofread the thing.

D) They are features, not bugs.

The nature of the differences — centering, header height, etc. — is such that an average person looking at those reports would just shrug them off. But a professional likely wouldn’t. And a professional would be looking for formatting as well as data errors. So it’s possible that if a skilled proofreader did not proofread this thing, but rather the duty was handed to someone who doesn’t do this, the job did not get done adequately — meaning that any errors such as number transposition (e.g. 27 and 44 getting switched because their cells are right next to each other) would be far less likely to be caught.

For that to happen once is understandable. But three times? That’s harder to believe. This brings me to the next option:

2) This is a Republican polling firm that is cooking the demographics to make Biden look good.

TOR tweeted for the first time May 23 with that New Hampshire poll. Within 12 hours, someone had responded with a screenshot of part of TOR’s client list. That client list page has since been removed from TOR’s site, but because it’s so hard to delete anything from the Internet, your humble researcher dug a little and found a nice, long list … of Republicans.

Here it is. I copied and pasted the (deleted) client list’s source code into a Google Document, then deleted the garbage from it and got that. It is overwhelmingly Republican.

Now, why would a Republican outfit prop up Joe Biden? Because he’s far more conservative than Bernie, which means good things for Republican policy goals, and because he’s far less likely to beat Trump (he has at least as many problems as Hillary, who lost).

Now, I’ve said repeatedly that early polls don’t matter. The closest they come to predicting anything is that the frontrunner the year before the election almost always loses. So you should not be taking any poll seriously, however rigorously it’s conducted.

But until TOR starts running polls with credible demographic mixes, I don’t see a reason to take any of its polls seriously.