Small-n Approaches to Impact Evaluation
A Causal Inference Framework
A causal-inference framework for development
1 Small-n approaches to impact evaluation
When an evaluation has few intervention units, incomplete records, or no credible large comparison group, sample size does not decide what can be learned. The task is to define the causal claim, identify the available evidence, and state what the resulting conclusion can and cannot establish.
Effects
Did the intervention make a difference?
Configurations
Which combinations produced the outcome?
Mechanisms
How and why did change occur?
1.1 A way through the book
The book starts from the causal claim rather than a sample-size threshold. It then examines the methods that can support that claim, the assumptions that remain, and the practical work of commissioning and using the evidence.
01
Foundations
Small-n problems, causal claims, and inferential logics.
02
Methods
Counterfactual, configurational, process, contribution, realist, diagnostic, participatory, and simulation approaches.
03
Integration
Method selection, combination, quality, and transparent reporting.
04
Policy practice
Commissioning, translation, and evidence use under constraints.
05
Synthesis
Accumulating evidence across small-n studies.
1.2 Three causal claims, connected evidence
Each method chapter identifies the claim it addresses, the inferential operation it uses, and the evidence it requires. The categories overlap in applied work, but they should not be treated as interchangeable.
Difference-making
1.2.1 Effects
Did outcomes differ under intervention from what would plausibly have happened otherwise? Within-unit counterfactual designs are the primary home for this question.
Set relations
1.2.2 Configurations
Which conditions, alone or in combination, are necessary or sufficient for an outcome? Configurational approaches compare cases as combinations rather than as isolated variables.
Process evidence
1.2.3 Mechanisms
What process links an intervention to an outcome, and how do the available observations alter confidence in that account? Process tracing, contribution analysis, realist evaluation, and diagnostic updating contribute here.
1.3 What the book covers
Foundations
Defining small-n, separating causal claims, and mapping the logics that methods use.
Part IIMethods for causal inference
The main approaches for making and assessing causal claims from a small number of cases.
Part IIIIntegration and practice
Choosing, combining, assessing, and reporting methods with clear claim boundaries.
Part IVPolicy and development practice
Commissioning and using small-n evidence in policy processes and constrained settings.
Part VSynthesis and conclusion
Combining findings across studies and drawing the book’s argument together.
Reference tool
1.4 Select a method without starting from a threshold
The method-selection aid links the claim, unit of analysis, available variation, comparison, process evidence, rival explanations, ethical constraints, and intended decision.
1.5 How to use this book
Read from the beginning when the framework is new: later chapters rely on distinctions established in the foundations. Use the part cards to enter a particular method family, then return to the decision framework when several approaches are feasible. Commissioners can begin with the method-selection aid and the practice chapters, but should use the linked methods chapters to assess the claims a proposed design can support.
Preface
This book is for commissioners and senior evaluators who need to design or assess an impact evaluation when the number of intervention units is small. It is also relevant to methodologists, knowledge brokers, and advanced students. International development is the main applied setting, but the causal problems addressed here arise across public policy and practice.
Existing guidance on small-n evaluation is spread across method traditions. A commissioner may find a practical account of one design without help in deciding what causal claim the design can support, when another method is more suitable, or how different forms of evidence can be combined. This book addresses that problem. It begins with the claim to be made–about an effect, a configuration, or a mechanism–then examines the evidence and assumptions needed to support it.
The book does not offer a formula that chooses a method from a sample-size threshold. It shows how to assess the unit of analysis, available variation, comparison, process evidence, rival explanations, ethical constraints, and intended decision before a method is selected. The method chapters then set out what the main approaches require, what they can establish, and where their claims stop. The later chapters address commissioning, integration, reporting, evidence synthesis, and use in constrained settings.
The book is situated within the broader agenda of evidence-informed policymaking (EIPM). Small-n constraints can limit the designs available to an evaluation team, but they do not remove the need to define a causal claim or examine alternatives. A conceptual framework for EIPM (Sempé et al., 2025) identifies capabilities, relationships, structures, and evidence culture as pathways through which evidence may influence policy.
See Chapter 17 for the full mapping of this book to the EIPM framework. Part IV (Chapter 17 through Chapter 20) addresses the structures and processes pathway directly.
The book addresses the capabilities pathway by giving evaluation teams a way to match claims to feasible evidence and to state the limits of the resulting conclusion.
Parts of this book were drafted, revised, or restructured with the assistance of large language models (LLMs), including Claude (Anthropic) and GPT-5 (OpenAI). The authors used LLMs as writing aids for prose generation, bibliographic cross-checking, and structural editing. All substantive claims, methodological arguments, and analytical judgements reflect the authors’ expertise and were verified against primary sources. The bibliography was checked against original publications. The authors bear full responsibility for the content.