Finally Breaking Up with Traditional Field Data Collection?

Finally Breaking Up with Traditional Field Data Collection?

The global development community increasingly recognizes traditional methods can't keep pace in the real-time data world

By Chris Watson and Rebecca Abu Sharkh | Director of International Development, Solutions Consultant

In 2015, during the period of reflection that followed the Millennium Development Goals, UN Secretary General Ban Ki-Moon appointed a High Level Panel to advise on the global development agenda. They were among the first to call for a ‘data revolution’ in global development. This call made sense. Whole sectors of the global economy were being disrupted by new companies that leveraged real-time data and the increasing availability of smartphones and cheap sensors held the possibility of a new data source. 

Six years later and, from our vantage point in the technology sector, we don’t see a data revolution in global development. The bread and butter grant opportunities and requests for proposals that represent most funding in the sector make passing references to real-time data and “innovative use of technology.” But the deliverable requirements, the part people get paid for, are still focused on data collected through traditional approaches and written up in reports. 

The fault doesn’t necessarily lay with the professionals who continue to require deliverables based on traditional data collection. The proliferation of smartphones across the globe gave many citizens a real-time publishing platform through social media, but that data was never meant to answer the inquiries of a development professional. If your program needs 24 hour dietary recall data you can’t get it from social media—you’re forced to rely on traditional field data collection. 

The Problem with Traditional Field Data Collection

The global development community has been using panels and enumerators to provide data required by development programs for years. The problem is, it’s extremely slow and expensive. The community has sought to address this in two ways. 

  1. Take the biggest cross-cutting data requirements and turn them into independent programs. The best example of this is the National and Demographic Health (DHS) Survey, but it also includes initiatives like the Multiple Indicator Cluster Survey or Afrobarometer. 
  2. Standardize instruments so programs that require such data aren’t starting from scratch. The WHO Service Availability and Readiness Assessment (SARA) is one such example.

Next, you have the enumerators who collect such data do it on a smartphone or a tablet and voila, real time data. Not so fast… 

Most development programs still need data that is more specific to their goals and objectives than these established surveys and instruments. These programs still need to design a specific instrument and scope for their inquiry, tender that work, have the vendor construct a panel or train enumerators, then collect, clean, and analyze that data. In our experience, this process usually takes 3 – 12 months and the world inevitably changes during that time. The result is that the community recognizes that real-time data can help make their programs more adaptive but they’re reliant on traditional field data collection to answer their inquiries. 

Modernizing Field Data Collection

This is why we built Premise. We believe advancements in cloud computing and machine learning, coupled with increasing smartphone ownership, make it possible for ordinary citizens to conduct field research in the communities where they live and to the standard required by the international development community and market research professionals. The Premise Platform is a real-time data collection and analytics software product that provides our clients with access to a growing network of 2.4 million local citizens who get paid to answer surveys and do field data collection tasks in their communities. Our platform is used by more than 20 international development funding agencies and implementing partners to replace traditional field data collection with a real-time alternative that empowers citizens living in their intervention areas. 

We see five key factors driving our client’s use of the Premise Platform.

Persistent Access

Premise enables continuous access to the communities where development interventions are implemented. This is possible because citizens in local communities–and not researchers–provide the data, including in conflict or post-conflict zones. Data is recorded by trained and vetted locals directly in the Premise app, which is available to contributors around the clock and provides detailed instructions and specifications as to what the data collection task entails. This means Premise Contributors are not constrained by “field time” and can instead be tasked at any time, enabling more frequent data collection and resulting in greater responsiveness to current events. 

Faster Turnaround Time

Preparing for field research requires months of survey design and protracted processes of contracting and training enumerators. Research contracting is an onerous task as cost estimates in the face of imprecise research objectives and goals can draw out the bidding and negotiation process. 

Given Premise’s Software as a Service (SaaS) model, contracting is made easy as research scopes can evolve without contractual changes. Moreover, as Premise’s local data collection software is currently available in nearly 110 countries worldwide and operational in 34 languages, data campaigns start within days, with all tasks completed in under 45 days. This remains true for representative data even at the subnational level.


Traditional research is accompanied by a lengthy process to compile, digitize, visualize and analyze data. More often than not, these steps follow a strict order of operations: each self-contained step needs to conclude before work on the next step can begin. With Premise, data is stored, processed and visualized by the Premise Platform while being collected on the ground. Automatic and manual quality control, proprietary algorithms and our in-house data science team process data submissions as they come in, replacing discrete steps with continuous data processing and analysis cycles.  

Agile and Retaskable Data Collection 

Unlike traditional field data collection, in which the ability to adjust the research objective during data collection is generally prohibitive, Premise’s data collection tasks can be iterated upon at any time during the data collection life cycle. The power to make changes during an ongoing research project is crucial since research methods often cannot be precisely described in advance and work statements and end objectives change due to developments in the field. With Premise, however, once the desired changes are implemented, the tasks are deployed again immediately.

Similarly, while it may be hard for field researchers to re-observe the same places or reinterview the same people, Premise does so quickly, and in the recurrence schedules our clients require. Supplemental data collection is, therefore, quickly deployable on an ad hoc basis. 

One-Stop Solution 

Lastly, Premise integrates the people who collect the data, the technology for managing data collection remotely, and data analytics into one holistic solution. This integration staves off lapses in communication, gaps in management oversight, and breaks in information-sharing or collaboration. As a “whole product,” Premise’s platform, products and services achieve the desired result of research and data collection campaigns: actionable insights to improve impact.


While access to traditional field research has been yet another casualty of COVID-19, emerging technology help fills the gap. Field research remains critical in nearly every industry worldwide, yet the process of doing so is rapidly evolving.

To find out how you can adapt and increase your data collection in hard to reach areas of the world, email us at


Photo Credit: USAID Flickr

About Chris Watson and Rebecca Abu Sharkh

Chris Watson is an evangelist for data-driven adaptive management solutions that optimize the performance of international development and poverty programs. He currently leads Premise Data’s international development team, with a focus on product-market-fit, use case definition, sales, and customer success.

Rebecca Abu Sharkh is a Solutions Consultant based in San Francisco, California. With a background in global operations and government, at Premise, she serves as a technical and functional subject matter expert on Premise’s operations platform and smartphone app and conducts data collection and analysis.