8 Process Tracing
Process tracing examines how an intervention contributed to an outcome within a case. The case may be a policy reform, organisation, programme site, or bounded implementation episode. Its purpose is not to estimate an average effect. It is to assess whether the events, decisions, and observations expected under a proposed causal process occurred, and whether rival accounts fit the evidence better (Beach & Pedersen, 2019).
This chapter draws on three sources for different purposes. Beach & Pedersen (2019) provide its main account of process-tracing designs and uses. Collier (2011) clarify the diagnostic vocabulary of hoop, smoking-gun, straw-in-the-wind, and doubly decisive tests. Scriven (2008) provides the general-elimination discipline used here to organise plausible alternatives. The chapter uses these sources to define inferential tasks, not to turn their labels into automatic ratings for documents or interviews.
The bibliometric map (Chapter 5) finds 4,482 title-and-abstract pairs naming process tracing, the second-largest core-method corpus after QCA. It concentrates in Political Science and International Relations (1,289 pairs) and Sociology and Political Science (1,117), with smaller shares in Economics and Econometrics (171), Strategy and Management (168), and Development (120): a discipline mix close to configurational approaches, which is consistent with the two families’ 0.62 per cent co-mention rate reported in Chapter 15. The United States (534 fractional works), the United Kingdom (223), and Germany (196) account for the largest institutional shares, alongside a substantial repository literature with no resolvable affiliation. Figure 5.4 shows the surface-mention trend for process tracing together with realist evaluation and general elimination methodology, the other members of the generative and mechanistic family; process tracing accounts for most of the family’s growth over the period.
The method is most useful when the causal question concerns a sequence: how a reform changed administrative practice, how an implementing organisation adapted an intervention, or why an expected outcome did not occur. It requires a theory of the process to be investigated, but it should not be reduced to a search for confirming material. The evaluator must state what would be observed if the proposed mechanism operated, what would be expected under plausible alternatives, and how the available records can distinguish between them.
8.1 Causal mechanisms and process hypotheses
A process-tracing hypothesis specifies the entities, activities, and sequence that are expected to connect an intervention to an outcome. It is more precise than the statement that an intervention improved capacity'' orchanged incentives’’. Each proposed link should imply observations that can be sought in records, interviews, decisions, or outputs.
Beach & Pedersen (2019) distinguish three broad uses of process tracing. Theory-testing process tracing starts with a specified mechanism and assesses whether it operated in a case. Theory-building process tracing works from observations towards a plausible mechanism. Explaining-outcome process tracing seeks a case-specific explanation and may combine several mechanisms. Evaluation commonly begins with theory testing because programmes usually have an explicit or implicit account of how their activities are meant to contribute to change. The theory remains revisable: evidence may show that one link did not operate, that an alternative pathway mattered, or that the mechanism worked only under conditions not recognised in the original account.
Mechanisms need not be linear. A process may depend on several actors, involve feedback, or operate differently at different points in the period under study. The task is not to draw every possible arrow in a theory of change. It is to identify the processes that matter for the causal question and to distinguish their observable implications. A diagram is useful when it clarifies this work; it is unhelpful when it lists activities and outcomes without explaining what each arrow represents.
Several mechanisms may operate together. The evaluator should consider whether they are alternatives, parts of a common sequence, or processes that could each contribute to the outcome. If two mechanisms are alternatives, evidence for one may reduce confidence in the other. If they are complementary, the failure of one may prevent the outcome even when the other operated. Evidence that one link occurred does not remove the need to investigate the remaining links in the proposed process.
The level at which a mechanism is specified also matters. A claim that an intervention “built capacity” is too broad to guide inquiry unless it identifies whose capacity changed, what they were then able to do, and how that action could affect the outcome. A claim that district officials used a monthly data review to reallocate staff is more specific. It directs the evaluator to review the data, meeting records, allocation decisions, staffing changes, and accounts of other factors that may have shaped the decision. Specificity does not mean claiming a level of detail that the evidence cannot bear. It means stating a process in terms that can be examined and potentially challenged.
The proposed mechanism should include enabling and constraining conditions where they affect the claim. An intervention may have reached staff but produced no change because the relevant authority remained elsewhere. A local organisation may have adopted a new practice but abandoned it when funding ended. These conditions are not simply background. They can explain why a mechanism operated in one case and failed in another. The distinction is useful when the evaluation is intended to inform adaptation or transfer: it directs attention to the conditions that need to be present, not only to the activities that were delivered.
8.2 Causal claims and the level of explanation
The causal claim should be settled before evidence is described. Some evaluations ask whether an intervention contributed to an immediate decision; others ask whether it changed organisational practice, service delivery, or a population outcome. These claims are related but have different evidential requirements. Evidence that a ministry adopted a guideline does not establish that facilities changed practice. Evidence of changed practice does not establish an improvement in health, learning, or income. The process trace must state the outcome to which its conclusion applies and avoid silently extending the claim to a more distant outcome.
The level of explanation also determines which rivals must be considered. A claim about adoption of a policy may need to address political priorities, other advocacy, budget pressures, or legal requirements. A claim about service delivery may need to address staffing, supplies, local management, demand, and concurrent reforms. The relevant alternatives come from knowledge of the case and causal process, not from a standard checklist alone. Standard validity threats can help organise inquiry, but they cannot substitute for an account of what else could plausibly have produced the observed change.
An evaluation may produce a mixed result across levels. It may find that an intervention influenced a policy decision but that implementation did not follow; that an implementing organisation changed routines but that the target population did not receive a different service; or that a programme contributed to an intermediate outcome while the final outcome remained uncertain. Such findings should be reported as they stand. They often identify the point at which a programme needs adaptation and prevent a policy or implementation achievement from being presented as evidence of a broader impact that was not examined.
8.3 Planning a process-tracing study
The unit of analysis must be fixed before evidence collection begins. A national policy may be the intervention of interest, while the process trace concerns one ministry, one policy decision, or one implementation episode within a defined period. The protocol should state the outcome or event to be explained, the start and end of the relevant process, the actors expected to matter, and the point at which an alternative explanation could have entered. Without these boundaries, a process trace can expand indefinitely and treat any later event as evidence for the preferred account.
The initial theory should be represented as a sequence of linked propositions. For each link, the evaluator should specify the expected activity or decision, the actors and records likely to bear on it, the timing, and at least one plausible rival. This produces an evidence plan rather than a general request for documents and interviews. It also identifies early where the evidence base may be inadequate. A mechanism that depends on confidential decisions cannot be assessed only through public reports written after the outcome; a mechanism that predicts a change in practice needs evidence about practice, not only evidence that a training event took place.
| Process link | Expected observation | Plausible rival | Evidence source and timing | Assessment |
|---|---|---|---|---|
| [Specify link in the process theory] | [Specify expected observation] | [Specify rival account] | [Specify source and when it was created] | [Record provenance, dependence, and diagnostic value] |
The protocol should also identify the point at which the theory will be revised. Revision is appropriate when evidence contradicts an expected link or reveals a previously unrecognised mechanism. It is not appropriate simply because the initial theory fails. The evaluator should retain the original hypothesis, document the evidence that prompted revision, and assess the revised account against the same standard of observable implications and alternatives. This prevents a study from treating every observation as support for a mechanism reconstructed after the outcome is known.
8.4 Evidence and diagnostic value
The observations used in process tracing are causal-process observations. They include programme documents, administrative records, meeting minutes, correspondence, decisions, interviews, direct observation, and contemporaneous accounts. Their value depends on the hypotheses they bear on. A record showing that a planned activity occurred establishes implementation; it does not by itself establish that the activity changed behaviour or produced the outcome. An interview may provide evidence about an actor’s reasoning, but its weight depends on timing, access to the decision, incentives to misreport, and corroboration from independent sources.
The evaluator should formulate evidence tests before reviewing the most revealing material. A test states what observation is expected if a hypothesis is true, what is expected if a rival is true, and how distinctive the observation would be. This makes it possible to assess both confirming and disconfirming evidence. Bayesian process tracing, covered in Chapter 11, formalises this comparison through likelihoods. A qualitative process trace still needs the same discipline: it should identify why one observation supports a hypothesis more than another rather than accumulating quotations that point in the same direction.
Source dependence is a recurring problem. Several documents may repeat a single official account; several interviewees may have learned about an event from the same manager. Apparent corroboration then adds little information. The evidence plan should identify independent sources where possible, preserve contradictory material, and distinguish a contemporaneous record from a retrospective explanation. Missing records also matter. The absence of an expected trace can weaken a hypothesis, but only if the evaluator has grounds to expect that the trace would have been produced and retained.
8.5 Evidence tests and inferential discipline
Process-tracing literature often distinguishes four idealised tests (Beach & Pedersen, 2019; Collier, 2011). A hoop test is one that a hypothesis must pass to remain plausible, but passing it does not establish the hypothesis. A smoking-gun test provides strong support if the predicted observation is found, but its absence does not necessarily eliminate the hypothesis. A straw-in-the-wind test shifts confidence only modestly. A doubly decisive test would both rule out a rival and strongly support one hypothesis, although such evidence is uncommon in evaluation settings. These labels are useful only when the evaluator explains why an observation has the claimed diagnostic value in the case.
The labels cannot be assigned from the source type alone. An internal memorandum is not automatically a smoking gun, and an interview is not automatically weak evidence. A memorandum may have been written for political cover; an interviewee may have direct knowledge of an unrecorded decision and no incentive to misrepresent it. The relevant question is comparative: how likely is this observation if the proposed mechanism operated, and how likely is it if a rival explanation operated? Chapter Chapter 11 develops the formal Bayesian version of that question. Here, the practical requirement is to state the comparison in words and preserve evidence that complicates the preferred account.
Evidence should be assessed in sequence as well as in isolation. A mechanism that predicts a chain of decisions requires the decisions to occur in an order compatible with the account. Temporal order is necessary but not sufficient: an intervention can precede an outcome while another process causes the change. Dated records, implementation logs, and contemporaneous accounts are especially useful because they constrain retrospective reconstruction. Where dates are uncertain or sources disagree, the report should show that uncertainty rather than forcing a single chronology.
Befani & Stedman-Bryce (2017) illustrate their contribution-tracing approach through an evaluation of a civil-society Universal Health Care campaign in Ghana. The campaign used the 2012 presidential election to advance health-policy priorities and involved more than 500 health-focused organisations. Its broader objective of legislation for free universal health care had not been achieved at the time of the evaluation. The worked example instead examined a more limited claim concerning the campaign’s challenge to the National Health Insurance Scheme’s reported coverage. The authors set out observable expectations and assess how evidence should change confidence in the contribution claim.
The case is useful because it does not treat the campaign’s visibility, stakeholder endorsement, or the existence of a report as proof of policy influence. Those observations have different probative value and need to be assessed against alternatives. It is a demonstration of process-tracing and Bayesian principles, not an independent estimate of the campaign’s effect on health policy.
Welle & Punton (2015) discuss an early process-tracing application concerning the Hunger and Nutrition Commitment Index and policy change in Tanzania. The study was presented alongside the Ghanaian Universal Health Care case as an example of using process-tracing tests in international-development impact evaluation. It is useful here as a contrast: the causal claim concerns policy influence through an advocacy and accountability instrument, rather than a programme’s direct delivery of a service. The source is a methodological practice paper, so the chapter uses it to document the application and its design, not as an independent estimate of the index’s policy effect.
Befani & Mayne (2014) demonstrate a combined contribution-analysis and process-tracing approach through an evaluation of a teaching programme intended to improve girls’ school performance. The article uses the case to show how an initial contribution story can be broken into process hypotheses and assessed against alternative explanations. Its claim is methodological as well as empirical: the combined design was intended to make the evidence behind the contribution account more explicit than either approach used alone. The paper does not supply a general effect estimate for the teaching programme. It illustrates how within-case evidence can qualify a programme-level contribution claim in an education setting.
Chemouni (2018) traces the political drivers behind Rwanda’s community-based health insurance scheme, which had achieved enrolment of more than three-quarters of the population by the time of the study. The single-authored account reconstructs the sequence of political and administrative decisions, drawing on an analytical framework built around political settlements and ideas, to explain how a very poor country reached comparatively high insurance coverage. The claim concerns why this particular reform sequence occurred in Rwanda, not an average effect of community-based insurance schemes generally.
The case illustrates process tracing applied to a single, closely observed policy trajectory rather than to an implementation chain within a delivery programme. It is a useful contrast with the multi-country statistical or configurational studies elsewhere in this book: the evidentiary base here is documentary and testimonial reconstruction of one country’s political process, and the resulting account is deliberately bounded to that case.
Hafner & Shiffman (2012) use process tracing to construct a case history of how health systems strengthening rose to prominence as a global health priority after 2005, following a period in which global health attention concentrated on disease-specific initiatives. The authors identify the actors, institutions, and events that shaped this shift in global attention and the factors that inhibited it earlier, drawing on documents and reconstructed chronology rather than a single organisation’s programme records.
The example extends process tracing beyond a national policy reform or a single advocacy campaign to the agenda-setting behaviour of a transnational field of organisations. It is a reminder that the unit being traced need not be a programme, a country, or even a single institution: it can be a shift in the priorities of a loosely coordinated set of global actors, provided the evaluator can specify the mechanism and assemble evidence bearing on it.
8.6 Collecting documents, interviews, and chronology
Evidence collection should begin with a source map, not a long unstructured request for documentation. The map identifies the organisations and actors involved at each link, the records they are likely to have created, and the period in which the records should exist. For an implementation mechanism, this may include delivery schedules, supervision records, routine data, staff communications, and records of decisions made when delivery departed from plan. For a policy mechanism, it may include drafts, meeting papers, correspondence, public statements, budget documents, and records from actors outside the lead institution. The evaluator should record who controls each source and whether its absence is itself informative or simply a consequence of ordinary record-keeping.
Document collection benefits from an initial chronological reconstruction. Dates can be entered into a provisional timeline with the source for each entry and an indication of whether the date is verified, reported by a respondent, or inferred. The purpose is not to create a polished story before the evidence has been assessed. It is to expose gaps, impossible sequences, and points at which competing accounts make different predictions. Records that place events in a different order should direct further inquiry rather than be resolved by selecting the version that fits the programme theory.
Interviews have a defined role in this process. They can identify unrecorded decisions, explain why a documented decision was taken, describe informal routines, and point to sources the evaluator would not otherwise know existed. They are weaker when used as a substitute for a record that ought to exist, or when respondents are asked to reconstruct a complex sequence long after the event. Interview guides should therefore ask about particular decisions, dates, participants, and alternatives rather than invite a general account of success. Accounts from people with different positions in the process are often more useful than repeated interviews with those who designed the intervention.
Sampling should follow the causal question. If a claim depends on a decision by district managers, interviews solely with programme staff and final beneficiaries leave an evidential gap. If a claimed mechanism concerns routine use of data, observation or records of the routine may be more diagnostic than interview accounts of its value. The study may still have to rely on partial access. In that case, it should explain which link has been examined with direct evidence, which rests on retrospective testimony, and which cannot be assessed. This is more useful than reporting a single number of documents reviewed or interviews completed.
8.7 Making and calibrating causal judgments
The final judgement needs to distinguish what the evidence establishes from what it makes more plausible. Evidence that an intervention was delivered can establish exposure. Evidence that an actor received the intervention and later changed practice can support a behavioural link, particularly where the timing and stated reasons fit the mechanism. The same evidence may leave open whether the change was sustained, whether it affected the intended outcome, or whether another intervention was decisive. These distinctions should appear in the findings, not only in a limitations paragraph at the end.
A useful reporting structure presents each causal link with four elements: the claim; the evidence bearing on it; the main alternative account; and the judgement warranted after considering both. Judgements such as supported, partly supported, not supported, and indeterminate are acceptable when the report specifies their meaning. They should not become a scoring system that gives every item of evidence equal weight. A single well-documented decision may matter more than several derivative documents, while a strong rival explanation can limit a conclusion even when most of the planned links have been observed.
The report should also separate a conclusion about a mechanism from a conclusion about the intervention’s overall effect. A trace may provide persuasive evidence that a programme contributed to a change through a particular pathway, yet still be unable to determine how much of the observed outcome would have occurred without the programme. Conversely, a counterfactual design may find a treatment contrast but leave the operative mechanism uncertain. Stating these different contributions is necessary when process tracing is part of a wider evaluation design.
8.8 A practical workflow
The work of a process trace can be organised in six linked stages. First, define the causal question, case boundary, outcome, and period. Second, set out the initial mechanism and principal rivals. Third, derive the observations that would be expected under each account and identify sources that could distinguish them. Fourth, collect and catalogue evidence without discarding material that complicates the initial theory. Fifth, assess the evidence link by link, including its timing, provenance, and dependence on other sources. Sixth, state the resulting explanation, unresolved alternatives, and scope of the conclusion.
The sequence is not linear in practice. Early document review may reveal that the intervention was implemented differently from the original account, requiring a revised hypothesis. An interview may identify a previously unrecognised rival that needs documentary investigation. What should remain stable is the audit trail: the evaluator should be able to show when the theory changed, why it changed, and which evidence was used to support the revised account. A process trace becomes less credible when the final mechanism appears fully formed but the reader cannot see how it emerged from the evidence.
The final analysis should separate the findings by causal link. It is possible for an evaluation to find strong evidence that an intervention reached the intended actors, moderate evidence that it changed practice, and weak evidence that the practice change contributed to the ultimate outcome. Combining these into a single affirmative narrative conceals the link at which the inference becomes uncertain. A link-by-link account also helps commissioners decide what to do next: strengthen implementation, collect more outcome data, test a competing explanation, or narrow the claim.
8.9 From evidence log to causal conclusion
An evidence log is the bridge between fieldwork and inference. It should record the observation, date, source, link or rival to which it relates, source dependence, and the reason it was judged to support, weaken, or leave a hypothesis unchanged. The log prevents two common errors. The first is treating a large volume of material as many independent tests when it derives from one account. The second is writing a final narrative that remembers the material supporting the preferred mechanism but not the evidence that required qualification.
| Evidence item | Link or rival addressed | Source dependence and timing | Expected under proposed mechanism | Expected under rival | Recorded assessment |
|---|---|---|---|---|---|
| [Record the observation and source] | [Specify the process link or rival] | [Record origin, timing, and dependence] | [State expectation] | [State expectation] | [Record the judgement and its basis] |
The conclusion should follow the log’s pattern of support and uncertainty. It may state that a process is well supported up to a particular link, that a rival remains plausible after that point, or that the available evidence does not distinguish two accounts. This is not a failure to complete the evaluation. It is the result the design can support. Stronger wording is justified only when the evidence plan, source record, and treatment of alternatives show why competing explanations have become less credible.
8.10 Rival explanations and general elimination
General elimination methodology provides a practical discipline for handling rivals (Scriven, 2008). The evaluator lists plausible alternatives for the observed outcome, including standard validity threats and alternatives arising from the setting: another programme, a change in leadership, a financial shock, selective participation, altered measurement, or a process unrelated to the intervention. Each rival should have observable implications. The investigation then seeks evidence that would support or weaken those implications.
The task is not to claim that every imaginable explanation has been eliminated. It is to show which alternatives were considered, why they were plausible, what evidence was examined, and which alternatives remain unresolved. A causal conclusion should reflect the strongest remaining rival, not only the evidence for the preferred mechanism. This is particularly important where programme staff participated in the theory of change or provided most of the evidence. Their knowledge may be essential, but it does not remove the need to test accounts that do not favour the programme.
8.11 Transparency, access, and ethics
Process tracing often relies on records and testimony that are politically sensitive. Access can be uneven: programme staff may retain documents that support their account, while dissatisfied participants or former officials may be difficult to reach. Confidentiality can prevent the release of the most diagnostic material. These constraints do not invalidate the method, but they alter what can be concluded. The report should describe material that could not be obtained, groups that could not be interviewed, and any reasons respondents had to shape their accounts.
The evaluation team should distinguish confidential evidence from unverifiable assertion. A confidential document may be reviewed under an agreed protocol and described in a way that protects identities while allowing readers to understand its relevance. An uncorroborated claim that cannot be inspected or compared with other evidence should carry less weight. Interview consent, data security, and the risk of reprisals require particular care when the proposed mechanism concerns political influence, corruption, exclusion, or conflict. Ethical protection and evidential transparency have to be planned together; neither is served by promising a level of disclosure that the study cannot safely provide.
8.12 Feasibility and common design failures
Process tracing is not justified by the presence of a theory of change alone. The study needs a bounded causal question, a plausible prospect of obtaining evidence about the relevant process, and enough time to investigate alternatives. When the available material consists only of an endline report and interviews with the programme team, a full causal trace may not be feasible. The evaluation can still document implementation and participants’ accounts, but it should not claim to have tested the mechanism. An early feasibility assessment should therefore ask what contemporaneous records exist, which actors can be approached independently, what evidence a serious rival would leave, and whether the study period includes the decisions that matter.
One frequent failure is to treat the programme theory as the explanation to be illustrated. Fieldwork then follows the boxes in the theory of change, interview questions ask whether each activity worked, and evidence outside the proposed pathway is recorded as context rather than as a possible rival. Another is to move too quickly from an activity to an outcome. Evidence of delivery and a later change in an indicator do not establish the intervening process. Breaking the account into links makes the resulting evidential gaps visible.
Retrospective coherence is a further risk. After an outcome is known, respondents and evaluators can assemble decisions into a sequence that appears inevitable. Contemporaneous documents do not remove this problem, because they may also reflect official interests, but they can show what was known or decided at the time. A protocol, evidence log, and explicit record of theory revisions reduce the scope for reconstructing the mechanism around a desired conclusion. They do not replace judgement; they make the basis for judgement available for scrutiny.
The work should be costed accordingly. A credible trace often requires iterative document review, follow-up interviews, translation or access arrangements, and time to investigate a rival raised late in fieldwork. It may be better to trace one or two decisive links carefully than to claim a complete account of a long national reform with only a short visit and a small set of interviews. Commissioners should decide this at inception, including whether the evaluation will be able to publish an evidence table or a sufficiently detailed account of source limitations. If those conditions cannot be met, a narrower descriptive or learning question may be more honest and more useful.
8.13 Case selection, comparison, and replication
Case selection follows the purpose of the process trace. A theory-testing study may select a case in which the intervention and outcome are both present because it asks whether a proposed mechanism operated there. An explaining-outcome study may select an unusual success, failure, or reversal because the outcome requires explanation. In either situation, the selection rule should be stated before the case is described as representative or revealing. A successful case cannot establish that the mechanism explains success generally, and a failure case cannot show that the intervention never works elsewhere.
Comparison across a small number of cases can strengthen a process-tracing programme when it is used to test a clear proposition. The evaluator might examine sites where the same intervention produced different outcomes, or sites where a contextual condition was present in some cases and absent in others. The comparison should not reduce the process trace to a list of anecdotes. Each case still requires a specified mechanism, evidence plan, and treatment of alternatives. The value of the comparison lies in identifying whether a proposed process recurs, fails at the same link, or depends on a contextual condition.
Replication can be direct or systematic. A direct replication asks whether the same process is supported with similar evidence in another case. A systematic replication examines whether the process operates after a deliberate change in setting, population, delivery organisation, or intervention form. Negative cases are informative when the study can show where the anticipated sequence failed and whether another mechanism operated. The resulting claim should name the cases and conditions in which the process has been examined; it should not treat two or three traces as a population estimate.
This form of comparison links process tracing to the other methods in the book without collapsing their claims. Configurational analysis can identify a pathway or a deviant case for process tracing. A counterfactual study can establish that an outcome differed under an intervention contrast and then use process tracing to investigate how that difference arose. Contribution analysis can use a focused trace to investigate a weak link in a wider programme theory. The design is strongest when the methods address different uncertainties and their evidence is not merely the same documents and interviews reorganised under different labels.
8.14 Reporting a process-tracing study
A report should allow a reader to reconstruct the causal question, the proposed mechanism, and the basis for the final judgement. It is not enough to provide a theory-of-change diagram followed by a narrative of activities and outcomes. The report needs to state the bounded case and period, the outcome to be explained, the original process hypothesis, and the alternatives regarded as plausible at the outset. Where the theory changed during the study, the report should show the original formulation and explain what evidence required revision. That information allows readers to distinguish learning from a mechanism that was retrospectively tailored to the case.
The findings are usually clearest when organised by causal link rather than by source or fieldwork activity. A source-based structure–a section on documents, then interviews, then observations–makes it difficult to see what each item of evidence means for the causal claim. A link-based structure asks whether the intervention reached the relevant actor, whether the predicted decision or practice occurred, whether that change was connected to the next link, and whether the proposed process contributed to the outcome. Within each section, the report can present evidence for and against the account, describe the relevant rival, and state the remaining uncertainty.
The evidence table should be selective but sufficiently specific to expose the reasoning. It can identify the type and date of a record, the causal link it bears on, whether the account is independent or derivative, and the judgement it informed. In sensitive work, the table may need to anonymise sources or group records. That is preferable to an assertion that confidential evidence settled a disputed question. If the reader cannot see the source itself, the report should at least state how the evaluator assessed its provenance, how it was corroborated, and what competing interpretation remains possible.
| Causal link | Main evidence | Alternative considered | Judgement and limitation |
|---|---|---|---|
| [Specify the causal link] | [Identify the evidence and its source] | [State the alternative account] | [Record the judgement, uncertainty, and limitation] |
Language in the conclusion should match the evidence. A finding may be that the available evidence supports the first two links but does not establish the final outcome link; that the proposed mechanism is more credible than a named rival but other explanations remain; or that the evidence is inadequate to judge a contested decision. These statements can be useful to commissioners because they identify what is known about implementation and where a programme theory requires further testing. They are weakened when converted into an overall verdict that exceeds the component judgements.
The report should make clear whether it is making a within-case causal claim, a claim about a programme’s contribution, or a claim about the recurrence of a mechanism across cases. The first concerns the specific trace. The second often requires a wider contribution analysis and evidence on the programme’s theory as a whole. The third requires comparison or replication. Using the same word, contribution, for each can conceal a substantial difference in the evidential task.
8.15 Analysis, challenge, and quality assurance
The analysis should not be left until the end of fieldwork. As evidence is collected, the team can update the chronology, record its judgement for each causal link, and note questions that require follow-up. This is not an invitation to close the case early. It makes the remaining evidential gaps visible while there is still an opportunity to address them. A team that waits until all interviews are complete may discover too late that it has no independent account of a decisive decision, or that it never asked about a rival that the documents made plausible.
Analysts should distinguish descriptions, interpretations, and inferential judgements in their notes. A description records what the source says or what was observed. An interpretation explains how the item may bear on a mechanism. An inferential judgement states the weight given to that interpretation after considering provenance and alternatives. Keeping these distinct is particularly useful when a source is ambiguous. It allows a later reviewer to disagree with the interpretation without disputing the underlying record, and it prevents an early conjecture from being repeated in the report as if it were evidence.
Internal challenge can be organised around the strongest rival rather than around general scepticism. One member of the team, or an external reviewer, can be asked to examine whether the evidence would look similar if another programme, a political decision, a change in resources, or selective participation explained the outcome. The task is not to generate implausible objections. It is to identify the alternatives that a well-informed reader would reasonably expect the study to address. The challenge should be documented where it leads to additional data collection, a revised chronology, or a more limited conclusion.
Disagreement between sources requires analysis, not a vote. Several staff may describe an intervention as influential while a dated record shows that the relevant decision had already been made. A manager may report a routine change that front-line workers did not observe. These accounts can reflect different positions, partial knowledge, incentive to defend a programme, or genuinely different processes across sites. The report should explain the basis for preferring one account, or retain the disagreement as a limit on the finding. Simply reporting that most interviewees agreed is usually inadequate when their accounts are dependent or when they share an institutional interest.
Quality assurance also concerns the treatment of negative evidence. An expected record may be missing because the action did not occur, because records were poorly kept, or because access was denied. These possibilities have different implications. The evaluator should ask whether the proposed mechanism would normally have left a trace, whether comparable records from the same institution survive, and whether another source can test the claim. A missing record can then be reported as a defined uncertainty rather than converted automatically into either disconfirmation or reassurance.
The final review should check that quotations, documents, and timeline entries are being used for the purpose they can support. A quotation showing enthusiasm for a programme may be relevant to mobilisation; it does not demonstrate altered practice or an outcome effect. A revised guideline may show a formal decision but not its implementation. An outcome trend may be consistent with the proposed process but not identify it as the cause. This discipline is repetitive by necessity. The same distinction between activity, mechanism, and outcome has to be maintained throughout the trace.
8.16 Using process tracing in an evaluation design
Process tracing can be commissioned as the principal method when the evaluation question is about a bounded episode and the available evidence can bear on its causal process. It is also often used as one component of a design. A counterfactual comparison may address whether outcomes differed between treatment conditions; a trace can examine whether the difference arose through the mechanism anticipated by programme theory. A configurational analysis can identify cases with an unexpected outcome pattern; tracing selected cases can investigate the processes behind a configuration. Neither combination automatically resolves the other’s limitations. The design needs to specify which claim each method supports and how disagreement between the results will be handled.
Timing affects this choice. A process trace during implementation can document decisions, adaptations, and constraints while records are still available and actors can describe events close to the time they occurred. It may be especially useful where a programme is testing an intervention model and needs to know why delivery differs across sites. A trace conducted only at the end of a long programme may still be worthwhile, but the evidence burden is greater: memories are less reliable, staff have moved on, and records may have been discarded. The protocol should reflect these differences rather than describe both designs as if they provided the same basis for inference.
The method also has a role in formative work, although formative evidence should not be confused with a completed causal explanation. Early tracing may show that a necessary implementation link has not occurred, that staff interpret an intervention differently from its designers, or that a predicted decision point does not exist in practice. These findings can support adaptation. They do not demonstrate that a later outcome will change. A later phase of evaluation may need to revisit the mechanism after the programme has stabilised.
Commissioners should agree in advance how findings will be used. If the likely decision is whether to scale an intervention, the study may need evidence about recurrence across sites or a complementary design that addresses outcomes under a credible comparison. If the decision concerns a blocked implementation process, a focused trace may provide the more relevant information. The question is practical: what uncertainty has to be reduced for the decision, and can the proposed sources distinguish the relevant explanations? A generic commitment to understanding ``how and why’’ is not a sufficient design brief.
8.17 Scope and limitations
Process tracing can produce a detailed account of a causal process in one case. It cannot, on its own, show how prevalent that mechanism is, estimate an intervention effect, or establish that the same process will operate elsewhere. Replication and comparison can examine whether the mechanism recurs in other cases. Configurational analysis can identify cross-case patterns that guide case selection; counterfactual designs can address a treatment contrast; and diagnostic updating can make evidential assumptions explicit. Each contribution should retain its own claim boundary.
Table 8.4 summarises the conditions under which each process-tracing design is applicable.
| Process-tracing use | Typical case number | Evidence base | Causal claim supported | Main limitation |
|---|---|---|---|---|
| Theory-testing | One or a few cases | Dated documents, records, interviews, and contemporaneous accounts | Whether a specified mechanism operated in the case | Cannot establish prevalence or effect size beyond the case |
| Theory-building | One case | The same evidence read inductively for a plausible mechanism | A candidate mechanism generated from the case | The mechanism is provisional and must be tested on new cases |
| Explaining-outcome | One case | The same evidence, often combining several mechanisms | A case-specific explanation of a particular outcome | The explanation is bounded to the case and may not transfer |
The method is labour-intensive and vulnerable to post-hoc reconstruction. Its credibility depends on the transparency of the process hypothesis, evidence record, alternative explanations, and inferential judgments. A process trace that presents a coherent narrative without these elements may be informative description, but it is not yet a well-supported causal explanation. The next chapter turns to contribution analysis, which examines a programme-level contribution story across its full theory of change.