Big Data Systematic Map

Published date: 10 June 2020
Last modified date: 01 December 2020

Overview

This systematic map is different from the traditional gap map as it maps data sources to development outcomes. In this map, we visualize the use of big data to evaluate development outcomes across the world with a special focus on fragile contexts. It aims to identify rigorous impact evaluations, systematic reviews and the studies that have innovatively used big data to measure development outcomes.

A total of 437 studies have been included in the map, which comprise of 48 impact evaluations, 381 measurement studies and eight systematic reviews. All the included studies were published during 2005-2020.

Given the scale of the map, we found areas where more disaggregation was needed to visualize the gaps at a more granular level. We created five submaps associated with the following outcomes economic development and livelihoods, health and well-being, governance and human rights, urban development and environmental sustainability.

Please click on these outcomes on the map to access the submaps.

Main Findings

  • Satellite images and mobile call detail records are the most used big data sources.
  • The development themes studied the most include:
    • Environmental sustainability
    • Economic development and livelihoods
    • Urban development
    • Health and well-being
    • Energy, industry and infrastructure provision
  • Interventions and outcomes that have spatial dimension are more likely to be measured using big data. Some of the lesser studied outcomes include agriculture, education, water, and so on.
  • Studies are evenly spread across the continents.
  • While there are a number of studies that have used big data for measuring various development outcomes, there are not many impact evaluations that have used these innovative big data-based outcome measures.
  • This indicates the opportunity for incorporating big data measures in impact evaluations to measure the impact at higher frequency and granularity.
  • Impact evaluations fare better than measurement studies in reporting on data quality issues and transparency, but less than 18 per cent of the have data publically available.

Implications for policy, programming, research investments

  • This map shows that big data can contribute to evidence base in development sectors where evaluations are not generally feasible due data deficiency.
  • Given the fast growing availability of big data and improving computation capacity, there is a great potential for using big data in the future impact evaluations.
  • There are several sources of pre-processed satellite data that could be used in evaluations directly without the evaluators having to process them using complex machine learning models themselves.
  • There is also an absolute gap in using mixed-methods jointly with big data and cost effectiveness. This should be prioritised by donors and researchers as a mix of quantitative big data analysis and qualitative field level analysis will help strengthen the validity of the results.
  • More efforts are required to set up best practices and ethical standards, and facilitating more interaction among remote sensing scientist, big data analysts and development evaluators

Original map publication date: 10.06.2020
Current map date of publication: 10.06.2020

Online map citation:

Rathinam, F, Khatua, S, Siddiqui, Z, Malik, M, Duggal, P, Watson, S, Vollenweider, X. 2020. Using big data for evaluating development outcomes: a systematic map [Online]. 3ie. Available at: https://gapmaps.3ieimpact.org/evidence-maps/big-data-systematic-map

Related paper:  Using big data for evaluating development outcomes: a systematic map

Other related versions of this map:

Links to the submaps:

This project was funded by the Centre of Excellence for Development Impact and Learning, supported by UK aid from the UK Government.