Public Policy Polling is a North Carolina-based Democratic polling firm that works for politicians and advocacy groups. Dean Debnam established the firm in 2001, and it has become a prominent public pollster despite both right-of-center and left-of-center observers criticizing the firm’s methodology.

Background

Public Policy Polling is a Democratic polling firm that conducts publicly released surveys routinely cited in mainstream media accounts. Dean Debnam founded the firm in 2001 in Raleigh, North Carolina. ¹

Debnam contends that campaigns and advocacy groups spend too much on polling that could be better spent on communicating with voters. He also said that for smaller organizations, public opinion surveys could be too cost prohibitive. Thus, PPP was established to offer more affordable polling. The firm has worked for politicians, political organizations, unions, consultants, and businesses. ²

PPP touts itself as “a progressive firm” that works for progressive clients. The website says, “we understand and support our clients’ goals, so when you hire PPP, you know you have a partner that is fully invested in your organization.” The firm charges between $2,000 and $3,000 to conduct a poll, which it says is “generally one-tenth the cost of a traditional poll.” ³

Polling Performance

Despite criticism of the pollster’s methodology and non-transparency, analysts find that Public Policy Polling polls are mid-pack for predictive accuracy. It is included in the Real Clear Politics, Pollster, and FiveThirtyEight public polling averages. ⁴

In its final national poll before the 2016 presidential election, the firm showed Democrat Hillary Clinton leading Republican Donald Trump by four points, 44 percent to 40 percent, with Libertarian, independent and Green party candidates snagging a total of 9 points. Debnam said: “We’ve consistently found Hillary Clinton with a national lead in the 3-6 point range since Donald Trump clinched the Republican nomination. She’s never been able to blow the race wide open and she probably never will, but her modest advantage has proven to be quite durable.” ⁵

Methodology

Public Policy Polling uses “robopolling,” or interactive voice response, polls. The organization was once criticized for only calling landline telephones; in recent years it has added internet and cellphone polling. PPP asserts the automated system allows it to complete a poll in two days and within 24 hours if necessary, a shorter time span than most of its competitors. ⁶

Criticism

Methodology

Prominent political analysts began to scrutinize Public Policy Polling’s methodology in what then-Guardian political analyst Harry Enten called the “Nerd Fight of 2013” between poll analysts and Public Policy Polling pollster Tom Jensen. ⁷ Then-New Republic political analyst Nate Cohn reported that as of 2013, PPP engaged in “an unusual process known as ‘random deletion,’” eliminating the responses of sometimes hundreds of respondents, in part to reflect their projected voter turnout models. ⁸

Public Policy Polling allegedly instructed nonvoters in the last election to hang up the phone. It also reportedly excluded the question of how many answered whether they voted in the last election. Cohn alleged, “Throughout its seemingly successful run, PPP used amateurish weighting techniques that distorted its samples—embracing a unique, ad hoc philosophy that, time and time again, seemed to save PPP from producing outlying results.” ⁹

Cohn further showed that PPP seemed to fudge the ethnic composition of its projected electorate to show consistent support for incumbent Democratic President Barack Obama over Republican challenger Mitt Romney, rather than the expected statistical fluctuations in a relatively ethnically consistent electorate. ¹⁰ Cohn wrote, “If PPP would have shown Obama doing too well, the white share of the electorate got higher; if they would have shown Obama doing worse, the white share of the electorate got lower.” ¹¹

Cohn alleged that the firm’s weighting procedures meant that analysts “really have no idea what PPP’s doing.” ¹²

After the 2014 elections, Nate Silver of FiveThirtyEight alleged that Public Policy Polling’s methodology was a form of “herding.” Herding is a term Silver defines as polling companies making their surveys match the results of other surveys in the field. Silver wrote that a “few pollsters are shameless about their herding. One of them is Public Policy Polling,” and later labeled PPP “the biggest herders in the business.” ¹³

Colorado Recall Poll Spike

In 2013, Public Policy Polling revealed that it had declined to release a poll in a Colorado State Senate recall election showing the Democratic incumbent being recalled by a wide margin, allegedly because the firm disbelieved the results. The incumbent was recalled by a comparable margin in the recall election. ¹⁴ FiveThirtyEight polling analyst Nate Silver criticized the decision, writing, “VERY bad and unscientific practice for [PPP] to suppress a polling result they didn’t believe/didn’t like. .  .  . I’m especially skeptical when a pollster puts its finger on the scale in a way that matches its partisan views.” ¹⁵

Troll Polls

Public Policy Polling routinely engages in so-called “troll polling” — asking questions designed not to determine valid public opinion on political issues but rather to make poll respondents, especially conservative respondents, look uninformed or stupid. The pollster has asked respondents their opinions on if the U.S. should bomb Agrabah, the fictional home of Disney character Aladdin;¹⁶ how the government should respond to the nonexistent “Bowling Green Massacre”;¹⁷ and whether Democratic presidential candidate Hillary Clinton was Satan. ¹⁸

Critics deride these “troll polls” for contributing to political polarization,¹⁹ deceiving well-meaning poll respondents,²⁰ and being irrelevant to meaningful political conversation. ²¹

1. About Us. Public Policy Polling. Accessed May 25, 2019. https://www.publicpolicypolling.com/about/
2. About Us. Public Policy Polling. Accessed May 25, 2019. https://www.publicpolicypolling.com/about/
3. Why Choose PPP. Public Policy Polling. Accessed May 25, 2019. https://www.publicpolicypolling.com/why-ppp/
4. Enten, Harry J. “The Public Policy Polling Controversy.” The Guardian. September 25, 2013. Accessed June 03, 2019. https://www.theguardian.com/commentisfree/2013/sep/25/public-policy-polling-controversy.
5. “Clinton Leads by 4 Nationally, Trump Hurt Himself in Debate.” Press Release. September 29, 2016. Access May 26, 2019. https://www.publicpolicypolling.com/wp-content/uploads/2017/09/PPP_Release_National_92916.pdf
6. Why Choose PPP. Public Policy Polling. Accessed May 25, 2019. https://www.publicpolicypolling.com/why-ppp/
7. Enten, Harry J. “The Public Policy Polling Controversy.” The Guardian. September 25, 2013. Accessed June 03, 2019. https://www.theguardian.com/commentisfree/2013/sep/25/public-policy-polling-controversy.
8. Cohn, Nate. “There’s Something Wrong With America’s Premier Liberal Pollster.” The New Republic. September 12, 2013. Accessed May 25, 2019. https://newrepublic.com/article/114682/ppp-polling-methodology-opaque-flawed
9. Cohn, Nate. “There’s Something Wrong With America’s Premier Liberal Pollster.” The New Republic. September 12, 2013. Accessed May 25, 2019. https://newrepublic.com/article/114682/ppp-polling-methodology-opaque-flawed
10. Cohn, Nate. “There’s Something Wrong With America’s Premier Liberal Pollster.” The New Republic. September 12, 2013. Accessed May 25, 2019. https://newrepublic.com/article/114682/ppp-polling-methodology-opaque-flawed
11. Cohn, Nate. “PPP’s Results Don’t Excuse Its Bad Methodology.” The New Republic. September 19, 2013. Accessed June 03, 2019. https://newrepublic.com/article/114769/ppp-methodology-results-arent-defense.
12. Cohn, Nate. “PPP’s Results Don’t Excuse Its Bad Methodology.” The New Republic. September 19, 2013. Accessed June 03, 2019. https://newrepublic.com/article/114769/ppp-methodology-results-arent-defense.
13. Silver, Nate. “Here’s Proof Some Pollsters are Putting a Thumb on the Scale.” FiveThirtyEight. November 14, 2014. Accessed May 25, 2019. https://fivethirtyeight.com/features/heres-proof-some-pollsters-are-putting-a-thumb-on-the-scale/
14. Kopan, Tal. “PPP Reveals It Held Colo. Poll.” POLITICO. September 11, 2013. Accessed June 03, 2019. https://www.politico.com/story/2013/09/ppp-colorado-poll-096628.
15. Quoted in The Scrapbook. “Rocky Mountain Surprise.” The Weekly Standard. September 13, 2013. Accessed June 03, 2019. https://www.weeklystandard.com/the-scrapbook/rocky-mountain-surprise.
16. Shapiro, Matthias. “Is Public Policy Polling a Reliable Source? No.” Paradox Project. December 23, 2015. Accessed June 03, 2019. https://theparadoxproject.org/2015/12/23/20151221is-public-policy-polling-a-reliable-source-no/.
17. Adams, Becket. “Polling Firm Releases Trash Survey with Trash Questions Meant to Make Respondents Look Stupid.” Washington Examiner. February 10, 2017. Accessed June 03, 2019. https://www.washingtonexaminer.com/polling-firm-releases-trash-survey-with-trash-questions-meant-to-make-respondents-look-stupid
18. Philbrick, Ian Prasad. “That Poll Asking Voters If Hillary Is the Devil Did Trump’s Dirty Work for Him.” Slate Magazine. August 11, 2016. Accessed June 03, 2019. https://slate.com/human-interest/2016/08/poll-asking-voters-if-hillary-clinton-is-the-devil-only-helps-trump.html.
19. Shapiro, Matthias. “Is Public Policy Polling a Reliable Source? No.” Paradox Project. December 23, 2015. Accessed June 03, 2019. https://theparadoxproject.org/2015/12/23/20151221is-public-policy-polling-a-reliable-source-no/.
20. McLaughlin, Dan. “PPP Polls Shows Why Issue Polling Is So Unreliable.” RedState. December 18, 2015. Accessed June 03, 2019. https://www.redstate.com/dan_mclaughlin/2015/12/18/ppp-polls-shows-issue-polling-unreliable/.
21. Philbrick, Ian Prasad. “That Poll Asking Voters If Hillary Is the Devil Did Trump’s Dirty Work for Him.” Slate Magazine. August 11, 2016. Accessed June 03, 2019. https://slate.com/human-interest/2016/08/poll-asking-voters-if-hillary-clinton-is-the-devil-only-helps-trump.html.