The Art of Asking Questions: Creating Space for Ground Truth

The Art of Asking Questions: Creating Space for Ground Truth

By Sofia MacGregor-Oettler, Samantha Hubner | Survey Methodologist, Senior Platform Consultant

Developing the right questions to facilitate the information needed is nothing short of an art, and a major component of Premise’s success around the globe. When presented with any collection request, the Premise team relies on a unique blend of subject matter expertise and strict survey methodology standards to ensure optimal data delivery. 

In practice, Premise’s survey methodology aims to create a structure that allows contributors to answer carefully framed questions targeted to the population of interest. More specifically, our survey methodology’s goal is to increase data quality by maximizing our data collection’s integrity… in other words, how can we be sure the data says what we think it says? 

Data integrity is the fundamental measurement of how confident we are that our data is accurate, complete and reliable throughout its lifecycle. Data integrity, at its core, is the source of ground truth. 

To maximize data integrity, the Premise team works to identify and mitigate potential sources of error in questionnaire design and application. Some of the most common errors that the Premise team tackles include lack of clarity in phrasing, subject matter, metrics and/or overall task consistency. 

The process of refining Premise tasks is intensive but efficient. Team stakeholders in Premise’s Enterprise Solutions department work cross-functionally to identify any and all data quality issues as well as ensure uniform measurement in data collection. This is how we are able to confidently present the highest quality data available. 

The Measurement Aspect of Data Collection

The essential element of measurement is the questionnaire. This encompasses the wording of the questions, the answer choices provided, and the accessibility of the questionnaire topics. Multiple survey design aspects can increase measurement accuracy: appropriate question wording, concise questions, question order, as well as response options. Here are a few examples of how we establish response metrics in surveys to improve data collection.

Likert-type scales are crucial response option types when measuring sentiment. Likert scales allow us to measure attitudes, beliefs or behaviors by representing five ordered response categories. For example: 

Strongly agree, Agree, Neither agree nor disagree, Disagree, Strongly disagree 

These scales allow us to measure the strength or degree of opinion by presenting ordered answers that balance positive and negative options. This scale usually includes a neutral option in the middle to allow people with genuine neutral opinions to communicate their attitudes. These types of scales allow more granularity and nuance to people’s attitudes.

Our team works to provide mutually exclusive and exhaustive response options to increase data quality in close-ended surveys. Generally, the answer options for a question typically include an “other” option (with a free text field) so we can get direct insights into what the question could be missing.

Gathering Direct Insights 

A key component of Premise’s value is allowing the space to discover answers to the questions we may not even know to be asking. As a crowdsourced data and analytics platform, Premise relies on real people to provide direct insights that non-natives cannot. 

As such, there have been numerous instances in which Premise has been able to use its methodological prowess. This is how we are able to identify key findings to our clients’ most burning questions, as well as provide invaluable context as to what questions our clients ought to ask next.

One such example occurred when Premise was tasked with understanding the sale of cooking gas in Yemen. Despite tremendous efforts in refining task design, the team struggled to get valuable insights from Yemeni users as to the most reliable suppliers of cooking gas.

This finding led the team to revisit our methodological approach to allow users the space to provide carefully curated, less prompted insights that served as feedback on the initial tasks. In doing so, Premise learned that cooking gas was being provided by the aid community, and not at local markets. 

Whether we are locating diverted humanitarian aid or popular sentiment around COVID-19 safety measures, the art of asking questions remains deeply integral to Premise’s core operations. To learn more about our process and best practices, our task design, survey methodology, or to see what Premise can do for you, get in touch at!

About Sofia MacGregor-Oettler, Samantha Hubner

Sofia MacGregor is a Survey Methodologist at Premise Data. She helps to formulate surveys, design experiments and incorporate academic best practices into the work. Before joining Premise, she was in academia both as a researcher and graduate student. When she’s not working, she likes to explore trails and take pictures.

Samantha Hubner is the Senior Platform Consultant at Premise, under the Customer Success department’s Global Security team. She is responsible for training and assisting her colleagues as they become experts on how best to leverage the Premise platform to generate the highest quality of data. Currently based in Boston, Samantha is pursuing her Masters at the Fletcher School of Law and Diplomacy at Tufts University.