10 Realist Evaluation
Realist evaluation examines how programme resources interact with context to produce outcomes. Its central question is commonly expressed as: what works, for whom, in what circumstances, and why? (Pawson & Tilley, 1997). The aim is to refine an explanation of the conditions under which a programme theory holds, not to estimate a single average effect across all settings.
Pawson and Tilley’s formulation supplies this chapter’s conceptual foundation (Pawson & Tilley, 1997). The later Alberta study is used as an application of a realist case-study design, not as a substitute for that conceptual source (Flynn et al., 2021). This distinction is useful throughout the book: the authors who establish a method and the studies that apply it should be cited for different claims.
The bibliometric map (Chapter 5) finds 3,524 title-and-abstract pairs naming realist evaluation, the third-largest core-method corpus after QCA and process tracing. Its disciplinary profile is distinct from the other two generative and mechanistic methods: it concentrates in General Health Professions (595 pairs), Management Science and Operations Research (246), Sociology and Political Science (193), and Public Health, Environmental and Occupational Health (145), reflecting the method’s origin and heaviest use in health-services and social-policy research rather than political science. The United Kingdom accounts for the largest institutional share by a wide margin (719 fractional works), ahead of the United States (260) and Australia (178), consistent with the method’s development at the UK’s Centre for Advancement of Realist Evaluation and Synthesis and its continued concentration in UK health-services journals. Figure 5.4 (Chapter 8) shows realist evaluation’s surface mentions alongside process tracing and general elimination methodology; its growth has been steadier and, in absolute terms, smaller than process tracing’s over the same period.
10.1 Context, mechanism, and outcome configurations
The unit of explanation is a context–mechanism–outcome configuration. Context refers to conditions that shape how programme resources are received and used. Mechanisms concern the reasoning, responses, or changes in behaviour triggered by those resources. Outcomes are the resulting changes, including intended and unintended consequences. The terms must be specified for the programme being studied. Labelling every programme activity a mechanism, or every setting characteristic context, obscures the explanatory work the configuration is meant to perform.
A realist programme theory should state how a resource is expected to alter an actor’s options, reasoning, or relationships in a particular setting, and what outcome should follow. It should also state circumstances in which the proposed mechanism is unlikely to operate. The evaluator then examines cases that can develop, test, and refine those propositions. This requires evidence on context and responses as well as outcomes. A site that did not achieve the intended outcome may be as informative as a site that did, if it tests a boundary condition in the programme theory.
10.2 Developing an initial programme theory
Realist evaluation normally begins with an initial programme theory. This is a set of propositions about the resources a programme provides, the responses those resources are expected to generate, the contexts in which those responses are likely, and the outcomes that may follow. The theory can be drawn from programme documents, prior research, interviews with designers and implementers, and knowledge of the setting. It should not be treated as an account to be confirmed. Its purpose is to make causal propositions explicit enough to be examined and revised.
Programme documents often state activities and outcomes but leave the mechanism implicit. The realist question is how a programme resource is expected to change what actors do. A usable programme theory identifies the resource, the proposed response, and the organisational conditions in which that response is expected. These propositions imply different evidence and may operate under different organisational conditions.
The initial theory should therefore separate programme resources from participants’ responses. A training course, grant, guideline, or mentoring visit is a resource or programme input. It becomes part of a mechanism only through what it enables, constrains, or prompts actors to do. The distinction prevents a common error in realist writing: renaming an activity as a mechanism and then treating its delivery as evidence that the mechanism operated. Evidence that a mentoring visit occurred does not establish that staff trusted the advice, altered their practice, or had the authority to act on it.
Context should also be specified rather than used as a residual category for everything outside the programme. Contextual conditions may include a staffing arrangement, an existing relationship between organisations, local authority over resources, prior experience with a programme, or a policy requirement. The relevant condition is one that changes whether, for whom, or how the proposed mechanism operates. A long inventory of demographic, institutional, and political features without a proposition about their causal role does not improve the explanation.
10.3 Mechanisms, resources, and responses
Mechanisms in realist evaluation are often discussed as if they can be directly observed. In practice, evaluators usually observe traces: decisions, accounts of reasoning, changes in practice, patterns of participation, and outcomes. The task is to develop a warranted interpretation of how a resource interacted with actors in context. Interview accounts can be valuable, particularly when they concern a specific decision close to the time it was made. They should be compared with records, observations, and accounts from other positions where possible.
The same programme resource can trigger different responses across cases. A realist proposition should capture the relevant difference without treating each case as a wholly separate mechanism. It should identify the resource, the response, the condition thought to shape that response, and the observation that could challenge the proposition. Refinement then follows from evidence about which elements were present, absent, or differently understood.
Outcomes also require careful definition. They can include intended changes, unintended consequences, and the absence of an expected change. A realist evaluation may find that a programme improves reporting but increases workload, or that it works for well-resourced organisations while widening the gap with less resourced ones. These observations belong in the programme theory if they are connected to the proposed resource, response, and context. They should not be treated as peripheral simply because they do not fit the programme’s stated objectives.
10.4 Designing a realist evaluation
Case selection is purposive. The evaluator selects settings that vary on contexts thought to affect the programme theory and collects information that can distinguish between competing explanations. Documents, interviews, observations, administrative data, and outcome records may all contribute. Their role should be stated. An interview may inform the reasoning of a participant; an administrative record may establish when a resource was provided; an outcome series may indicate whether the expected change occurred. No one source establishes a CMO configuration on its own.
Analysis proceeds through comparison and theory refinement. Evidence from one case may suggest that a proposed mechanism operated only where a particular organisational condition was present. Another case may show a different response to the same resource. The resulting CMO proposition should identify the cases and observations from which it was developed, the counterexamples considered, and the uncertainty that remains. It should not turn a post-hoc description of a successful site into a general claim about what the programme will do elsewhere.
Realist evaluation is often used alongside other methods. A counterfactual design can address whether an outcome differed under alternative intervention states; process tracing can investigate a particular causal sequence; configurational analysis can compare observed pathways across cases. Realist explanation can help formulate questions for those designs and interpret their findings. Combining methods does not make the realist claim stronger by itself. The contribution of each component and the independence of its evidence need to be clear.
10.5 Selecting cases and collecting evidence
Case selection should follow the propositions to be tested. If the initial theory predicts that a programme works where local managers have authority over resources, the study needs cases in which that authority varies or in which the expected outcome differs despite apparently similar authority. If the theory concerns how a particular group responds to a resource, the sample must include that group and evidence about its actual exposure. Selecting only sites known to have implemented the programme well limits what the study can say about failure, boundary conditions, or alternative processes.
The unit of analysis should be defined before data collection. It may be an organisation, a service site, a policy implementation episode, or a group of programme participants. A national programme can provide the wider setting while the realist cases are local delivery arrangements within it. The report should state this boundary and avoid moving between national claims and site-level observations without explanation. A proposition derived from three service sites may inform a theory about local delivery; it does not by itself establish that the programme worked nationally.
Data collection needs to cover each element of the proposition. Evidence on the programme resource may come from delivery records, programme materials, observations, or accounts from recipients. Evidence on responses may come from interviews, observed practice, decisions, or use of a resource. Evidence on context may come from organisational records, policy documents, staffing data, and accounts from actors outside the programme. Outcome evidence may be administrative, behavioural, reported by participants, or derived from other sources. The evaluation should specify what each source can establish and which parts of the CMO claim remain interpretive.
The chronology matters. A context can predate the programme, a resource can be provided at one point, a response may occur later, and an outcome may be recorded only after a delay. A timeline can reveal that the expected outcome preceded programme exposure, that a claimed mechanism could not have operated in the proposed order, or that another intervention entered at a critical point. Where dates are uncertain, the report should show that uncertainty rather than impose a sequence that favours the initial theory.
10.6 Testing and refining CMO propositions
Analysis asks how well the evidence supports a proposition compared with plausible alternatives. The claim becomes stronger when the evidence shows exposure to the resource, the proposed response, the relevant scope to act, and a sequence compatible with the theory. It is weakened when the outcome or response predates exposure, another process explains the change, or the same response is found in cases without the resource.
Refinement should be specific. A vague conclusion that context matters does not help a programme decide what to retain or change. The evaluation should identify the condition that mattered, the response it shaped, and the outcome to which the claim applies. The resulting propositions can guide further testing, adaptation, or case selection in a later evaluation.
Counterexamples are essential. A case in which the anticipated outcome did not occur may indicate that the mechanism was absent, that the relevant context was not present, that the resource was implemented differently, or that the outcome measure failed to capture change. A case in which the outcome occurred without the programme may identify an alternative mechanism. The analysis should examine these possibilities rather than assign every unsuccessful case to an unspecified contextual barrier. A CMO proposition that survives only because contradictory cases have been excluded is not well supported.
The team should retain an audit trail of theory revisions. For each revision, the record should show the initial proposition, the evidence that prompted change, the alternative interpretations considered, and the revised proposition. This is particularly important when fieldwork is iterative. It enables readers to see whether a theory was genuinely refined in response to evidence or simply reformulated after the outcome was known.
Flynn et al. (2021) used a qualitative realist case-study design to examine the sustainability of two provincially scaled interventions in Alberta: a delirium intervention in critical care and an intervention on appropriate use of antipsychotics in seniors’ health. The study developed an initial programme theory and used CMO configurations to investigate contextual factors and mechanisms reported to enable or hinder sustainability. Its findings concerned mechanisms such as sense-making, understanding the intervention’s value, empowerment, and motivation in the contexts studied. The study illustrates theory refinement across cases; it does not provide an independent estimate of either intervention’s effect on patient outcomes.
Beutel et al. (2023) used a realist-evaluation lens to examine an Australia Awards South and West Asia short course for educators from Nepal who were considered leaders in inclusive education. The study used data collected before, during, and after the course to examine how participants responded to the programme and to work backwards from reported outcomes. It identified the role of context in triggering mechanisms associated with participants’ transformative learning and inclusive-education practice. The article describes a realist approach to a short international-aid programme; it does not estimate the programme’s effect against an unexposed group (Beutel et al., 2023).
Doi et al. (2018) evaluated a refocused school-nurse programme at two Scottish sites using a three-phase mixed-method realist design. The programme was intended to improve pupils’ health and wellbeing through nine care pathways. Interviews and programme documents informed the initial theory; interviews with school nurses and six months of referral data were used to test and refine it. The study reported that the programme supported earlier identification of needs through streamlined referral, while nurses reported less preparedness to deliver some intended interventions. The result was a refined account of conditions and mechanisms for implementation before wider roll-out, not an estimate of the programme’s effect on pupil outcomes (Doi et al., 2018).
Jagosh et al. (2015) report a realist evaluation that followed up an earlier realist review of community-based participatory research (CBPR) partnerships. Interviews with academic and community partners retained from the original review were used to examine long-term effects, including partnership synergy and trust building, that the review had identified but not fully explained. The study used realist methodology to develop a fuller account of how these ripple effects arose from the CMO configurations proposed in the earlier work.
The example illustrates a realist evaluation built directly on a prior realist review, using follow-up interviews to test and extend propositions about mechanisms operating over a longer time horizon than the original synthesis could observe. It is a useful contrast with the single-programme examples above: the unit under study is a partnership model observed across several prior reviewed cases, rather than one programme delivered at defined sites.
Gilmore et al. (2019) document the data analysis and synthesis process used within two realist evaluations of community health interventions conducted across Uganda, Tanzania, and Kenya. Using case-study data analysed in NVivo, the authors describe in detail how coded material was synthesised to refine middle-range theories, and they discuss the strengths and weaknesses of the analytic choices made.
The article is a methods-transparency account rather than a report of substantive CMO findings, and it is included here for that reason: it shows what the analytic labour behind a realist evaluation looks like when the fieldwork spans several countries and multiple case studies, complementing the more outcome-focused examples elsewhere in this chapter.
10.7 Analytic procedures and team challenge
Realist analysis often moves between deductive and inductive work. The initial programme theory provides propositions and concepts that guide coding and case comparison. Fieldwork may reveal a response, contextual condition, or outcome that the initial theory did not anticipate. The evaluator can then develop a revised proposition and examine whether it is supported in other material or cases. The distinction between refinement and invention matters. A new proposition should emerge from identifiable evidence and remain open to challenge; it should not be added because it makes an unexpected outcome appear consistent with the programme.
Coding can be organised around the elements of each proposition: the resource, the actor or group, the contextual condition, the response, the outcome, and any rival process. This does not require every extract to be assigned mechanically to one C, M, or O category. The purpose is to preserve the relationship between observations and causal claims. A staff member’s account that a dashboard improved practice may be evidence about a response, but it may also reveal a contextual condition such as authority over staff allocation. The analyst should record the interpretation and the evidence on which it rests.
Team discussion is valuable when it tests competing readings of the same material. One analyst may regard a manager’s account as evidence of a mechanism; another may identify it as a post-hoc justification for a decision driven by a budget change. The disagreement should be resolved, where possible, by examining dates, records, accounts from other actors, and cases where the alleged mechanism did or did not operate. Where the evidence cannot distinguish the interpretations, the final proposition should retain that uncertainty.
An explicit challenge process can focus attention on counterexamples. The team can ask whether the same outcome occurred without the proposed programme resource, whether a different outcome occurred despite the resource and context being present, or whether another process better accounts for the pattern. It can also question whether the context has been defined too broadly. Saying that a mechanism operated because the organisational context was favourable is unhelpful if the relevant feature could be specified as staff discretion, local leadership, a protected budget, or a relationship with another institution.
The analysis should keep a clear distinction between an empirical regularity and a CMO explanation. A pattern in which sites with stronger local leadership also had better outcomes may be a useful observation. It does not establish that leadership activated a particular mechanism without evidence about the programme resource, actors’ responses, and alternatives. Conversely, a carefully supported CMO proposition may explain a pattern within a small number of cases but not show how prevalent it is. The report should state which type of inference it is making.
10.8 Reporting CMO propositions
CMO propositions are most useful when written as propositions that can be understood and assessed. A table of isolated terms–``context: leadership; mechanism: motivation; outcome: improved service’’–does not state a causal explanation. A proposition should identify how a resource is expected to change the reasoning or opportunity of a defined actor under a stated condition, and which outcome follows. It should also indicate the evidence that supports the proposition and the cases in which it was examined.
| Element | Reporting question |
|---|---|
| Programme resource | What did the programme provide or alter, and was it available in the case? |
| Context | Which condition is expected to shape the response, and how was it observed? |
| Response or mechanism | How did the relevant actors understand, use, resist, or adapt the resource? |
| Outcome | What changed, for whom, and in what period? |
| Evidence and alternative | What supports the proposition, and what rival or counterexample remains? |
The report should describe the initial theory and the revisions made during the evaluation. It should state which propositions were supported, which were revised, which were not supported, and which could not be assessed. This avoids a familiar form of overstatement in which every final CMO configuration is presented as a discovery, with no indication of the theory it replaced or the evidence that forced the change.
The conclusion should name the claim boundary. It may state that a resource appeared to generate a particular response among a defined group in specified organisational conditions; that the evidence suggests a mechanism but does not distinguish it from a rival; or that the programme’s adaptation altered the mechanism under examination. It should not infer an average programme effect, a population prevalence, or transferability to settings not examined. Those claims require different evidence.
10.9 Commissioning and using a realist evaluation
Realist evaluation is well suited to programmes that have a plausible theory of change but are expected to operate differently across settings. It can be useful when the principal decision concerns adaptation, implementation, or the conditions required for a programme model to function. It is less suited to a brief commission that asks for a single definitive answer about overall effectiveness but offers no time for theory development, case comparison, or investigation of alternatives.
The terms of reference should specify the programme component, outcomes, and contexts that the evaluation will examine. A large programme may have many possible CMO propositions. The team needs to select those linked to the intended decision, evidence gap, or observed variation. The evaluation should match its case selection and period to the claim being examined. The unexamined parts of the programme theory should not be described as covered by the realist design.
Timing matters. Early realist work can identify whether a resource is reaching actors and whether the anticipated responses are emerging. It can inform adaptation before outcomes are observable. A later study can examine whether the proposed responses were sustained and whether outcomes followed. These stages should be distinguished. Evidence that a programme is acceptable or used by staff may support an intermediate theory; it does not establish a later service or population outcome.
Programme staff can contribute practical knowledge and access to records, but their role requires boundaries. They may help articulate the initial theory and correct factual errors. They should not decide which counterexamples are reported or remove an adverse finding because it complicates the programme narrative. The evaluation should also seek evidence from actors affected by the programme, including groups for whom the expected response did not occur. This is necessary for testing the stated theory of difference across contexts.
10.10 Causal claims, comparison, and transfer
Realist evaluation makes an explanatory claim about how a programme resource is understood and used in particular circumstances. It does not, on its own, estimate the difference an intervention made relative to a credible alternative state. A CMO proposition may be well supported within the cases studied and still leave open whether the programme changed the outcome compared with what would otherwise have happened. This distinction should be explicit where realist findings are used in an impact-evaluation handbook.
Comparison remains valuable, but its purpose differs from a counterfactual design. A realist comparison may examine cases with different contextual conditions, different programme responses, or different outcome patterns in order to refine a proposition. It can show that a proposed response was found in sites with a particular organisational arrangement and absent in sites without it. It cannot establish a population-level effect of that arrangement without a design capable of addressing selection and other differences between the cases. The report should state whether a comparison is being used for theory refinement, effect estimation, or both.
The transfer of a proposition to a new setting also requires judgement. A claim that a resource prompted action where local managers had discretion may be relevant to another programme with a similar managerial structure. It should not be applied mechanically to a setting where authority, incentives, or the resource itself differ. The evaluator should specify which features of the original cases are thought to be causally relevant and what evidence would be needed to assess them elsewhere. Transfer is a reason to state contextual conditions precisely, not a basis for treating every finding as generalisable.
Realist propositions can support programme adaptation without settling an effect question. A finding that staff use a new tool only when they receive timely feedback and have authority to respond may guide changes to implementation. A later evaluation may then test whether the adapted programme produces a different outcome. The realist finding and the counterfactual finding address related but distinct uncertainties. Combining them is useful when the design keeps their claims separate.
10.11 Power, access, and the interpretation of context
Context is not politically neutral. Formal authority, control over budgets, professional hierarchy, gender relations, discrimination, and the position of an organisation in a policy system can shape who receives a programme resource and whose response is visible to evaluators. A theory that describes a programme as empowering participants should identify what power they previously lacked, which resource is expected to change their position, and whether the relevant actors can act on that change. It should not treat participants’ reported satisfaction as evidence of empowerment without examining the decisions available to them.
Access to evidence may reflect the same inequalities. Senior programme staff may be available for repeated interviews while temporary workers, participants who disengaged, or groups critical of the intervention are difficult to reach. Records may document formal decisions but not informal practices or contested implementation. The evaluation should describe these patterns and consider how they affect the CMO propositions. A mechanism inferred only from the accounts of programme designers carries a different weight from one supported by evidence from those expected to respond to the programme.
Ethical safeguards can limit the disclosure of sensitive information, especially where the programme concerns employment, social protection, political participation, violence, or access to services. The report may need to anonymise a site or source. It should still provide enough information for readers to understand the causal role attributed to the context and the basis for the judgement. A confidential account can inform a proposition, but it should not become an unchallengeable explanation. Where corroboration is impossible, that limitation belongs in the conclusion.
The language used to describe responses needs the same care. A participant may not adopt a programme resource because it is unsafe, impractical, or inconsistent with a legitimate priority. Labelling this a failure of motivation can obscure the relevant condition or attribute responsibility to the wrong actor. The analysis should distinguish refusal, constraint, adaptation, and unintended use where the evidence permits. These distinctions affect whether a programme should change its resource, its delivery arrangement, or the conditions it assumes it can influence.
10.12 Common weaknesses and limits
One weakness is to produce long lists of CMO configurations without showing how each was supported. The list can appear explanatory while the relationship between source material, interpretation, and causal proposition remains unclear. A smaller number of well-specified propositions, each linked to evidence and counterexamples, is more useful than a catalogue of labels.
A second weakness is to make context so broad that any result can be explained after the fact. If a programme succeeds, the context is described as enabling; if it fails, the context is described as constraining. The analysis needs to name the condition that mattered and show how it shaped a response to a specific resource. Where no such evidence is available, the conclusion should remain descriptive.
A third weakness is to equate participants’ accounts of a mechanism with proof that it caused the outcome. Accounts can be evidence of reasoning and experience. They may also reflect hindsight, loyalty to the programme, or partial knowledge of other processes. The evaluator should compare them with timing, records, observation, and accounts from other actors. The appropriate conclusion may be that a mechanism is plausible and warrants further testing, rather than that it has been established.
Realist evaluation also requires resources. Developing an initial theory, selecting informative cases, returning to field evidence as propositions are revised, and analysing contradictions take time. A short study can still use realist concepts to sharpen a process inquiry. It should not promise a fully developed theory-refinement exercise if it cannot examine cases and alternatives in sufficient depth.
10.13 A practical workflow
A realist evaluation can be organised in five linked stages. First, define the programme component, outcome pattern, cases, and decision question. Second, develop a limited set of initial CMO propositions from programme materials, prior evidence, and knowledge of the setting. Third, select cases and sources that can test the relevant resources, responses, contextual conditions, outcomes, and alternatives. Fourth, compare the evidence within and across cases, revising propositions where the evidence requires it. Fifth, report the refined propositions, the cases in which they were examined, the evidence on which they rest, and their limits.
The work is iterative, but it benefits from a clear evidence record. For each proposition, the team can record the initial formulation, the specific sources examined, relevant observations, competing interpretations, and the current judgement. The record should include negative cases and contradictory accounts. It helps prevent the final report from giving equal weight to every interview extract or treating a revised proposition as if it had guided the study from the beginning.
| Stage | Main task | Output |
|---|---|---|
| Initial theory | State a limited number of propositions about resources, responses, contexts, and outcomes | Initial programme theory and case-selection logic |
| Evidence plan | Identify sources and observations that can support or challenge each proposition | CMO evidence matrix and fieldwork protocol |
| Case inquiry | Collect evidence on delivery, responses, conditions, outcomes, and rivals | Dated evidence record, including contrary material |
| Cross-case analysis | Compare patterns and assess why a proposition holds, fails, or needs revision | Refined CMO propositions with stated claim boundaries |
| Reporting and use | Connect findings to programme decisions and specify remaining uncertainty | Findings that distinguish adaptation advice from effect claims |
The evidence plan should be proportionate to the proposition. It should establish whether the relevant actors received and used the resource, whether they could act on it, whether the proposed response occurred, and whether the ordering supports the account. The protocol can identify these requirements before the most revealing data are reviewed, reducing the risk that the team develops its explanation only from material that fits a final outcome.
The final discussion with commissioners should separate findings that can support an immediate programme change from those that require further evaluation. Evidence that an expected response did not occur because staff lacked access to a resource may justify a delivery change. Evidence that a revised resource was subsequently used may support a refined programme theory. It does not show that the adaptation improved the ultimate outcome unless the evaluation has evidence on that link. This distinction preserves the practical value of realist findings without turning adaptive learning into an untested impact claim.
10.14 Realist synthesis and scope
Realist synthesis applies related principles to evidence reviews. Rather than pooling effects, it examines how studies support, revise, or challenge programme theories across contexts. The resulting propositions remain dependent on the quality and relevance of the underlying studies. A realist synthesis does not establish a programme effect merely because several reports describe a similar mechanism.
A realist synthesis begins with a review question and an initial programme theory. The question may concern why a type of intervention is taken up, how it is sustained, or under what conditions it produces a specified outcome. The review then seeks evidence that can inform the propositions, including studies that describe implementation, outcomes, participant responses, and contextual conditions. Relevance is judged partly by whether a source can bear on the theory, not only by whether it reports a preferred outcome measure.
The use of diverse sources does not mean that every source has the same evidential weight. A detailed case study may offer information about a decision process while providing little basis for a claim about outcome patterns beyond that case. A trial may estimate an outcome contrast while offering limited evidence on the mechanism. A programme report may describe delivery but have an interest in presenting the intervention favourably. The synthesis should explain how source type, quality, setting, and relevance affect its interpretation of each proposition.
The search and selection process should be transparent. A review may begin broadly and then refine the search as the programme theory develops, but the report needs to state the sources considered, the basis for inclusion, and how the evidence was extracted and compared. Without this information, a realist synthesis can become a selective set of examples chosen to fit a preferred explanation. Negative cases, contrary findings, and studies from settings in which the programme theory did not hold are especially useful for testing the scope of propositions.
The final product is a set of refined, bounded propositions. It may conclude that a resource tends to prompt a response under certain institutional conditions, that the evidence on a proposed mechanism is mixed, or that the existing studies cannot distinguish competing accounts. It should not turn recurrence of a description across reports into proof of causation. The same caution applies when the review is used to develop a programme theory for a later realist evaluation: it provides propositions for inquiry, not findings about the programme that has not yet been studied.
10.15 Scope of the method
Realist evaluation is useful when the central uncertainty concerns variation in implementation, response, and outcome across settings. It is less useful when the question is limited to a well-defined effect contrast and a credible comparison can be constructed, or when the available evidence cannot speak to actors’ responses and contextual conditions. It can contribute to a wider design by explaining how an intervention may have operated in selected cases, identifying conditions for adaptation, and developing propositions for further testing.
The final claim should be no broader than the evidence permits. It may concern a mechanism among a specified group, in a particular organisational arrangement, over a defined implementation period. It should not imply that the mechanism is universal, that the programme produced an average effect, or that a finding will transfer to a new setting without assessing the relevant conditions. These limits are part of the method’s explanatory discipline, not a defect in the evaluation.
Realist evaluation is valuable where interventions are adapted across settings and programme outcomes are expected to depend on institutional, social, or organisational conditions. It is labour-intensive and open to post-hoc explanation if the initial programme theory, case selection, evidence record, and rival accounts are not made transparent. The causal-process observations that realist evaluation produces about context–mechanism–outcome propositions can themselves be assessed through diagnostic updating (Chapter 11), which formalises how evidence changes confidence in a causal hypothesis. The next chapter develops that logic.