Results: Tables and Toplines from Premise’s survey, “Future of Colombia”
In anticipation of the highly contested Colombian presidential election on May 29th, Premise conducted a presidential poll pairing traditional polling strategy with non-traditional polling data collection carried out on our Premise smartphone app. The results were striking, with Premise predicting the outcome of the election more accurately than any other pre-election poll. In fact, Premise was one of only two organizations (along with CELAG) to correctly anticipate the two candidates who will proceed to the runoff on June 19th: former guerilla group member Gustavo Petro and populist Rodolfo Hernández. While Petro’s first-place finish was broadly anticipated, Hernández was widely expected to place third behind career politician Fico Gutiérrez. Yet, in line with the emerging media narrative in the weeks before the election, Premise successfully captured a late-stage surge by Hernández that vaulted him ahead of Gutiérrez and set the stage for a Hernández versus Petro runoff.
On the difference between Hernández and Gutiérrez, Premise came closer than any other public pollster. In fact, the majority of pollsters predicted third place Gutiérrez would beat second place Hernández by at least five points.
Equally important, Premise’s results tracked the actual margins between the major candidates significantly more closely than any other poll, despite having the smallest sample size. The results of Premise’s poll put Gustavo Petro’s first place lead over second place Rodolfo Hernández at 12 points; this 12 point margin fell within 0.2 points of the actual election results. As seen in the graph below, no other pollster came within four points of the final result of the election.
Speed and Reach Explain Premise’s Success
Premise’s success demonstrates the potential of app-based polling in settings where smartphone penetration is high and voters are difficult to reach through more traditional data collection practices. In many countries, smartphone penetration is greater than landline access; for example, as of 2020, 63% of Colombians have smartphones while only 14% have fixed landlines, limiting landline telephone interviews as a medium for data collection. In addition, polling through in-person interviews can be limited in Colombia by both security concerns and lack of transportation infrastructure, making certain areas inaccessible to interviewers. Unobstructed by these roadblocks, Premise can quickly survey voters who are less likely or unable to be reached by traditional polling methods.
Premise’s poll was fielded in just two and a half days between May 18th and 20th. The speed with which Premise launched and analyzed this poll allowed us to successfully capture real-time public opinion data, which can be especially elusive in the final days before an election. During the final days before the Colombian election, Rodolfo Hernández enjoyed a late rise in the polls. Despite this, every pollster other than Premise had Petro leading Hernández by at least 16 points. Of those pollsters, the most accurate one after Premise was CNC, which recorded a 16.7 point difference between Petro and Hernández. While CNC’s May 13-19 poll predicted some of Hernández’s upward momentum in the last few days of polling, it ultimately fell short because of its reliance on labor- and time-intensive in-person interviews and land-line telephone calls for data collection. Premise’s poll was executed in roughly one third the time of CNC’s week-long poll, positioning Premise to effectively capture voters’ intentions as close to election day as possible.
Premise’s poll surveyed voters in 233 municipalities across 29 Colombian departments. Our geographic reach far exceeded that of all top-rated Colombian pollsters. When comparing Premise’s geographic reach to CNC, Premise collected data from 129 more municipalities. Further, compared to the May 13-18 in-person home interview poll from Invamer—which esteemed Colombian political journal La Silla Vacía rates as the best pollster in the country—Premise received responses from voters in 6 more departments and 150 more municipalities.
This extended geographic reach helps explain the greater accuracy of Premise’s results. For instance, unlike Premise, Invamer incorrectly polled third place Fico Gutiérrez at 6.6 points ahead of second place Hernández. This was partly because five out of the six departments that Premise polled and Invamer did not—Caqueta, Guaviare, Vichada, Arauca, and Casanare—were carried by Hernández.
Premise’s Approach to Sampling
Between May 18th and May 20th, Premise obtained 1,023 survey responses from Colombian adults, 806 of whom indicated they would vote in the upcoming election. Through a combination of stratified random sampling from our network of app users and post-hoc weighting, we were able to construct a sample that reflected the demographics of the 2018 Colombian Census on geography, age and gender. We also found that Colombian users of the Premise app—and those in our final sample—are both socio-economically and politically diverse.
Premise’s stratified sampling approach targeted age, gender, and geography, and applied post-stratification weights across the same three variables. Although our raw age distribution somewhat over-represents adults between 30 and 49, the sample was balanced between men and women. And although most of Premise’s submissions came from the more populous Andean region of Colombia, we were successfully able to poll voters in the isolated Amazon regions to the east.
Premise also asked respondents to identify both their estrato—a socioeconomic status measure that assigns a number to neighborhoods based on the average income of its residents—and their current political views on a left-right scale. Premise’s network of contributors mirrored Colombia’s National Administrative Department of Statistics (DANE) report on estratos. Premise had a slight bias in submissions from estratos one and three at the expense of estratos four, five and six, but overall the distribution of Premise’s respondents mirrored the distribution of DANE1. Another pollster, Guarumo, was only able to obtain 13.5% of responses from estrato one, and because respondents from lower estratos were more likely to prefer Hernández to Gutiérrez, their results were biased from wealthier estratos. Premise, on the other hand, was able to get 21% of responses from estrato one, in which DANE estimates 20% of the population resides. Premise’s ability to reach Colombian adults from lower estratos allowed for Premise to identify Hernández’s late rise.
Additionally, Premise found respondents were evenly distributed across left and right political scales, suggesting Premise’s sampling strategy was effective at extracting opinions from persons in Colombia with mixed political beliefs.
Implication of Smartphone based data collection
In a time when smartphone penetration is far greater than the reach and speed of traditional data collection practices, polling by smartphone can quickly capture real changes in public opinion. Premise’s ability to reach voters on their smartphones granted us the opportunity to poll Colombians in areas of the country that are difficult to reach via in-person polling and that lack landline infrastructure for telephone polling. Coupled with that, Premise’s ability to reach out to voters with demographic profiles that are typically under-represented via these polling methods uncovers unique public opinion trends. The advantages of the Premise app and survey capabilities positioned Premise to successfully detect the rise in popularity of Rodolfo Hernández in the days leading up to the election. By launching a poll between May 18th-May 20th, our platform reached adults from various geographies, ages, political leanings and of every socioeconomic background and found that Rodolfo Hernández’s populist message resonated with Colombians, causing Hernández to surge past Gutiérrez in Premise’s poll and the first round of the election.
About Premise
Premise offers a unique ability to quickly source information from real people on the ground in hard-to-reach places. Over four million people in 140 countries are using the Premise app on their smartphones, enabling our customers to monitor a situation over time and employ a data-driven approach to timely decision-making. To learn more about Premise, watch our tech demo or get in touch with us.