Why The Polls Keep Getting It Wrong – Cheddar Explains


This pile of beans and this chart help explain this or why polls that are supposed to predict election results keep missing the mark. To really understand the problems with modern polling, we’ve got to start at the beginning. In 1936, the US presidential election pitted incumbent Democrat Franklin Roosevelt against Alf Landon, a Republican governor from Kansas. Now, up to this point, polls have been informal. Newspapers used to gauge voter sentiment by surveying their readers, but it wasn’t based on any formal survey methodology. In 1916, a magazine called the Literary Digest started conducting national polls. They sent out millions of postcards to their mailing list mostly drawn from phone books and vehicle registration records. The 1936 survey, was their biggest one yet. Meanwhile. From office of the Princeton, New Jersey a famous statistician, Dr George Gallup tells Washington from week to week what the nation is thinking. George Gallup had a background in consumer research. He accurately predicted that Franklin Roosevelt would defeat Alf Landon. The Digest had predicted a Landon victory. Gallup called that too. He knew the Digest would miss the answer because he saw the flaw in their method. Remember how they got their mailing list from phone and auto records, people with landlines and cars tended to be older more affluent and more conservative. Therefore, the Digest results undercounted FDR supporters, and overestimated support for Landon. Gallup got it right because he used a method from statistics called quota sampling. He said that sampling public opinion is like sampling soup. One spoonful can reflect the taste of the whole pot if the soup is well stirred. In other words, you don’t have to survey every single voter. You just need to survey the right subgroups of voters. Gallup used beans to explain his method. You’ve got a barrel of beans, millions of beans. Half of them are white beans and half of them are black beans. Now, count out 3,000 beans. Those beans are your sample. Write down what percentage of white beans you’ve got and what percentage of black beans you’ve got and repeat the process, 1,000 times. What you’ll find is that 997 out of those 1,000 times your sample will have the same make up of beans as the whole barrel. So, you’ll have 50 percent white beans 50 percent black beans. Well, roughly the same within three percent. So, In every Gallup poll, we include people from all walks of life, and in the right proportion. The farmer, skilled workers, white collar workers, business executive If you’re trying to figure out which presidential candidate Americans prefer, you want your sample to be the American voting public in miniature. The sample is the key to the whole thing. So, here’s where the trouble starts. Look at this chart. We just don’t use landlines as much as we used to. More than 60 percent of us primarily use our cell phones, and this demographic tends to be younger more diverse and more liberal leaning. The people who tend to still use landlines are older and more conservative. But if you don’t get in touch with the people who mostly use their cell phones, you’re going to throw off your sample in the same way the Digest did back in 1936. So, can’t we just call people on their cell phones? It’s not so easy. There’s a law against auto dialing cell phone numbers. It’s from 1991 and it was supposed to protect us from invasive telemarketing practices. But opinion surveys were designed to be conducted over the phone using automated dialing. Hello. This is a nationwide radio survey. Were listening to your radio just now? No I am not. Were you listening to your radio just now? Oh yes I am. It can be twice as expensive to reach poll respondents on cell phones because you have to hire and pay real live humans to dial the numbers. Whether we’re talking landline or cell phone, there’s a societal shift that’s endangered Gallup’s method. We’ve been trained by telemarketing and caller ID to screen our calls. We just don’t answer random numbers. That means low response rates. In the 1970s, a good response rate for a national poll was in the 80 percent range. Today it’s an abysmal eight percent. That’s bad news for polling because the smaller the sample size, the less accurate the poll. Pollsters can try to compensate for low response rates using weighting. But that’s another place where things can go off the rails. Let’s say you can’t get any Hispanic millennial men to answer the phone and respond to your survey. You finally get this one guy. But demographically, you should have three times as many Hispanic millennial men. So, you weight his answer three times. One guy standing in for the whole Hispanic millennial male population, and if he’s enough of an outlier that he actually answered the phone, he could be an outlier in other ways including his answers to the survey. If you’re just trying to measure preferences opinion surveys offer valuable insight From America which do you think it’s more important to do? Keep out of war ourselves or helping them win even at the risk of war. It’s when we try to predict human behavior that we get into trouble. While it’s easy to get people to tell you which candidate they prefer. It’s a lot harder to predict who’s actually going to get off the couch and go cast a ballot. Gallup himself had some notable misses. On election night 1948, Republican campaign leader Herbert Brownell and a legion of news men continued to echo public opinion polls that said Thomas E. Dewey would be the next president of the United States. A real poll taker polled in 1948 that the poll guardian valuable. But at the same time, we know that no better method has yet been found for measuring political trend. Ironically, he didn’t think predicting election outcomes was all that important. He only got into the election prediction game because he wanted to demonstrate the value of his main product. His opinion research surveys. What we’re seeing today is it’s harder to get a good sample thanks to changes in how he use the phone, and it’s just as hard as it ever was to predict human behavior. Thanks for watching. Hit the comments with your thoughts, questions, hot takes. Like, subscribe, and we’ll see you next time.

About the author

Comments

  1. They really should leave a message when they call so I can call back if I want to take a survey. I just never answer the phone because it’s usually telemarketers or they call while I sleep, but I’d be glad to take a political survey.

  2. I don't know if this is a theme, but it seems you taking issues most people just complain about and show the complexities involved, I love it! We need this in our society! Thank you again!

  3. Trump won in 2016 and will win again in 2020. Media bashed him so much they gave him victory on a silver platter. All the negativity has made the population more politically aware. Same thing will happen in 2020. 8 years of Greatness! MAGA

  4. Did I hear this right cell phones are Used by more liberals then conservatives? I thought near 100% of the population use cell phones

  5. Great video, but the only problem is that the polls were really not off. Your critiques of polling are valid, but your premise is false.

    Good reading on the topic: https://fivethirtyeight.com/features/the-polls-are-all-right/

  6. The fact is, even if someone has a 1% chance of winning, their winning doesn't make the polls wrong. Why is it people can't understand probability. Conversely, having a 99% chance of winning does not mean you will win, otherwise it would be 100%.

  7. The Obama team in 2008 worked directly on new media and new telecommunications arenas. Smartphones were new. He went around the main networks, but since he was a novelty they still covered him for free. He used international social media where possible. He was hailed as smart for this. His campaign directly worked with social media to twist a few realities but get people fired up. He also directly campaigned in every state, not just the ones where he was popular and got paid greatly. Mitt Romney and John McCain did not do this. They just let the traditional campaign firms run their campaigns like it was 1976. Romney even insulted half the population as lazy, welfare bums in a verbal quote.

    In 2016 Trump did the very same thing. New media, direct campaigning in every state, hiring NEW small analytic firms to tell him where the swing state electoral victories could be had. Lots of short videos released unedited by supporters. Mailings and content push directly to people that had taken polls like "Which Brady are You?". People that would vote for Trump where it mattered. Hillary Clinton just let her professional campaign firms run campaign 101, just like it was 1976. She didn't campaign much where she wasn't loved, as loving crowds gave money. Toward the last two months her rallies were attended by dozens where Trump's were attended by thousands. The main networks and social media firms did not want Trump, actively campaigned for Clinton, but Trump (like Sanders) was a novelty. So like any novelty, they covered him for free. Clinton then insulted half the voters by calling them Deplorables.

    Of course, when he did just what Obama did, but on a larger scale with MUCH more precision than was possible in 2008, "he must have cheated.". Obama admitted how Trump won in a Nov 2016 interview. He said Hillary Clinton just didn't campaign consistently or well.

    Party politics is funny. A does X and it's fine and dandy. B does X and B is a horrible bastard. It is like a perverted Categorical Imperative, from the sports world. Go Team!!! You Suck.

  8. Just to piggyback on the point about younger peole using cellphones and not having landlines. Statistics show that young people are the largest group of people who are eligible to vote but dont vote. Some polls do not ask you if you are registered, they ask you about issues. Such as are you pro life or pro choice. If you are polling people who are not going to vote that also will throw off your results.

  9. People are lying; and here is the solution: ask two questions, 1)Who are you gonna vote for? 2)No seriously, who are you gonna vote for?

  10. great video!
    Although i ain't no expert, but from what i learned from ap stats class back in 90s, these polls are usually subjects to multiple biases.

  11. I think coupled with what was mentioned in the article, there is a massive distrust of the media by the right. Therefore to many people that otherwise would respond are also not responding. So you go from the 8% number cited to a even smaller fraction. A fraction of 8% is obviously very close to useless.

  12. The answer is very simple, it's just guess work. IT'S NOT FACT. No one knows who will win and the media NEEDS to stop predicting who will win. Asking a few hundred people the most, can't predict who will win. Polls are useless, absolutely useless!!!

  13. But the phone theory would imply trump would poll higher? But it was millennials that seemingly made Hillary poll higher?

  14. You know what you genius pollsters should do? Go to political rallies.

    Hillary + Warren Buffet managed to get around 150 people for a rally of theirs.

    ALL Trump rallies are jam packed with thousands waiting outside.

    If the genius pollsters bothered to go to just 2 rallies, any rallies, they'd understand Trump was the easy choice.

  15. Orrrrr you could just use the 13 point method for predicting presidential elections. True or false, 7 is the winning number true for winning.

    1.Party Mandate: After the midterm elections, the incumbent party holds more seats in the U.S. House of Representatives than after the previous midterm elections.

    2.Contest: There is no serious contest for the incumbent party nomination.

    3.Incumbency: The incumbent party candidate is the sitting president.

    4.Third party: There is no significant third party or independent campaign.

    5.Short-term economy: The economy is not in recession during the election campaign.

    6.Long-term economy: Real per capita economic growth during the term equals or exceeds mean growth during the previous two terms.

    7.Policy change: The incumbent administration effects major changes in national policy.

    8.Social unrest: There is no sustained social unrest during the term.

    9.Scandal: The incumbent administration is untainted by major scandal.

    10.Foreign/military failure: The incumbent administration suffers no major failure in foreign or military affairs.

    11.Foreign/military success: The incumbent administration achieves a major success in foreign or military affairs.

    12.Incumbent charisma: The incumbent party candidate is charismatic or a national hero.

    13.Challenger charisma: The challenging party candidate is not charismatic or a national hero.

  16. No we understand it it's just that you're lying cheating and stealing people vote that's what they doing Russia is doing the same thing they did in 2016 to get Donald Trump elected again that way they can still secret papers out the White House they can record them anytime they want out the White House this man your President Donald Trump is disrespecting the White House disrespecting the flag committed treason on several different occasions and you disrespecting the flag right now as we speak

  17. The polls were actually not too far off in 2016, if you just look at the popular vote.
    But it's all kind of pointless if you're polling the popular vote, but the race is actually for the electoral college.

  18. The poles push the candidate or issue they favor . Plain and simple . Based on the premise that everyone loves a winner and NO ONE likes a loser . The simple psychology of human thinking .

  19. Polls are wrong because people don't want to admitt who they vote for because of fear of being labeled. No one was specifying Hillary would win the popular vote. Predictions said she would win the Presidency by a landslide and laughed about it. How sweet that was.

  20. Yes, it's illegal to robodial cell phones. And I get robocalls on my cell phone CONSTANTLY. As far as quality of life issues are concerned, I'd be highly motivated to work for, and vote for a candidate that would enforce the law regarding this, and to ban unsolicited solicitations by phone and email PERIOD.

  21. I don't remember subscribing and I am not sure if random boring knowledge is worth my time. Well, I'll stay for some more videos and decide…

  22. I remember when the polls said the blue wave was supposed to take over Senate. They are now saying there is an 80% change GOP will hold it. The House is still 80% in favor of Democrats. Let's see if that's correct.

  23. the polling is wrong because pollsters are intentionally under sampling different demographics to manipulate public opinion. They dont care , they want to generate phantom support that they will use to generate real support. 4chan figured this out when Trump was running when the gambling odd were 12-1

  24. There's also the issue of People interfering with samples. The majority of calls I received from pollsters hang up as soon as I give the location in which I live. If a poster knows the location of which a respondent lives in they could surmise the voting trend of that area. Polluting their results because they see a sound bike in media to be more important than accurate polling for the public. in the same respect most campaigns hire private polling firms that do not become aggregated in National polls.
    The takeaway is it there's too much money in polling and often some are repeating what customers want to hear.

  25. I acknowledge the producers of this video have talken the time , multiple times, to address a complaint I posted and did so in a courteous and amicable way. Regardless if I agree with the explanation the courtesy and tact they displayed in their explanation can be considered a far cry from what we have become accustomed to and that being the nasty rhetoric or complete denial of anothers reasoning and point of view. That at the very least garners my respect and appreciation for open informed dialog. A case point and lesson the entire left wing media could learn from unfortunately the public now understands that they are not about wanting to learn anything other than what they say. Soon to be gone are the days when Americans will tolerate such control and venom espoused by the mainstream media or any other platforms that seek to erode our American principals of honesty and fair play. They may and always will have the right to say it but will soon find it falling on deaf ears as We the People are not going to listen or entertain any and all such vile attacks upon the people and their values. We are going to simply remove those channels and no longer listen. Over time they will eliminate themselves. Isn't America Great. Lets restore it and keep it that way

  26. I saw polls in which they left out independent voters… but that's who decides elections. I think it was dishonesty though, instead of incompetence. The leftist media knows liberals/Democrats are more likely to suffer from voter apathy. If they think their side won't win, they're more inclined to give up thinking there's no point. So giving liberals/Dems a false sense of security increases turn out.

  27. Perhaps the identifying variables are outdated. Hispanic millennial men might have a lurking variable that causes misrepresentation even if they answer. I think there are ideological points that are more difficult to identify that party can't measure and are the reason polls have been wrong.

  28. This is wrong too. Trump voters don’t usually want to say they support Trump. They don’t want to be shamed or lose their jobs. Trump voters are the silent majority.

  29. Lots of polls (for elections, especially the 2016 election) were rigged in favor of whatever news source they wanted to win.

    For example, Detroit, if you want to make it look like Hillary would win the election by a landslide (which some polls did) they would call people specifically from areas of high black population (which statistically always vote Democrat which she is). What they did was call people that lived in places like Detroit, knowing more a lot people in Detroit like Hillary.

  30. The polls were not wrong, they accurately predicted the popular vote. The people reading the polls didn't break it down to the state by state bases, they only looked at the national average. Even 270 to win made this mistake if you broke it down by each state Trump had more a 45-48% chance to win the presidency. He just has to win the 3 rust belt state that he was only trailing by I believe 2 points in.

    Can polls be wrong? Yes but this comes more from underestimating unlikely voters then the system itself being bad.

  31. The trouble with polls is not the polls. It's the fact that most people don't understand statistics. Polls with 3 to 5% margin of error are projecting elections that are usually decided within 1%. Polls showed that the election was a coin toss – but people didn't understand what the polls were telling them.

  32. I personally find polls of no use, the public can think what they want I'm still voting who I think is best no matter how likely they are to win.
    There's only one poll that matters – the election.

  33. Polls are the laziest form of gauging, go and talk to the people directly. No robocalls or ads, townhalls, meet and greets matter, if you don’t meet with your constituents, get out of office, we’re not paying your salary to ignore us. This goes for both parties who are very guilty of this. Also no virtual townhalls you’re elected in your district, not an apartment in DC.

  34. When people get doxxed, beaten and lose their jobs for being MAGA, who in their right mind would tell a stranger on the phone who they support?

  35. It’s simple. The democrats use these “polls” as propaganda. They hope to discourage republican voters and encourage democratic voters, but it doesn’t appear to be working very well.. THANK GOD!!!!

  36. No however by your logic as landline users being old and conservative and easier to reach and cellphone users being young and liberal and harder to reach that would have meant they would have over estimated Trump which obviously did not happen.

  37. They keep missing the mark because they’re controlled by bias media that only questions people in large urban areas and not rural. Stop trying to push people away from the truth. Good pictographs and ahistorical facts doesn’t give credence to the idea that polls are wrong simply because they are out of touch with the people. Bias has corrupted everything. That’s why the democrats will lose, because their vein ideology has blinded them.

  38. this whole video is based on the presumption that polling is worse than it used to be. This perception is due to only two recent events: Trump's election and Brexit. And there's a huge misconception here: polls showed a very close race for Brexit, but there was a disconnect with how the pundits were presenting it. Pundits believed that Brexit was not going to pass despite polls showing that Brexit was a distinct possibility. Likewise, there's a big misconception with Trump: mainly, people were looking at national polls and not the electoral college. People do know that Clinton actually won the popular vote by about 2 percentage points right? And the polling showed her winning by an average of 3 points in the days leading up to the election, leading to an error of just 1 point (pretty small in the polling world). Meanwhile, actual research into polling errors shows that average polling errors from year to year have been pretty stable, despite the added difficulties of connecting to people via landline. This is probably due to more sophisticated statistics that help offsite the new challenges in contacting people. Point is, polling is definitely not perfect. But it has never been perfect, and evidence shows that it's actually not any worse now than it was before.

  39. The 2016 polls were only wrong in 4 states day of. Nevada was predicted for trump but went to Clinton, and Michigan, Pennsylvania and Wisconsin were predicted for Clinton but went to Trump. Correct me if I'm wrong

  40. One reason why polls failed in 2016 is because those supported Trump were very antiestablishment and antimedia, so when asked if they would take a poll they said no because they didnt want to be used as pawns.

  41. It doesn't make any sense?! How can you poll just 1300 people out of millions! And then say so and so is going to win?! 1300 people is nothing out of millions!

  42. There are 2 sorts of people–People who enjoy discussing their choices and those that don't–This causes a fundamental error. The poll is not truly random.

  43. The whole premise of this video is wrong. The polls are not getting it wrong. They remain roughly as accurate as they've ever been.

    Furthermore, the whole video is strictly anecdotal. How do you expect to be taken seriously if you don't back up your thesis with data?

  44. Because, in the days of Gallup people trusted the legacy media, they no longer do (not one bit) and will therefore no longer answer a poll.

  45. The USC/LA Times poll got 2016 right, and this was by polling against demography in proportion rather than polling against voter registrations from 2012/2008.

    Thats the real and short answer.

  46. If this is in reference to the 2016 elections and that Hillary won the polls. She won the election through the popular vote. Trump won the electoral college. You cannot poll the electoral college. In other words 3 million votes did not count. The polls were right.

  47. Then hire people and have the phone number calling identify itself as Gallop poll; see if people are interested in participating. I guess other assholes will identify as Gallop and mess this up, but that us the problem with assholes

  48. Personally I predict elections based on how energized one side is. I always knew Trump was going to win because his supporters are more loud and proud about liking Trump and I knew Moore would lose Alabama because people were so against him. Of coarse this can fail a lot, like with Sanders and many reactionary choices in west Europe, but in larger elections it seems to have done me well.

  49. Wow, my eyes have been opened. And a new robocall law was just passed. I'd imagine this could skew the polls even more. We'll see next year when the presidential election is in full swing. Since there are more and more people over the age of 60 or so living, it stands to reason there will be a lot of people who still have landlines.

    This is great news for Trump and his loving, closed-minded followers.

Leave a Reply

Your email address will not be published. Required fields are marked *