This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Blamey, A. Articles by Mackenzie, M. Search for Related Content Social Bookmarking                         What's this?

Theories of Change and Realistic Evaluation

Peas in a Pod or Apples and Oranges?

NHS Health Scotland, UK, avril.blamey{at}health.scot.nhs.uk

Mhairi Mackenzie

University of Glasgow, Scotland, m.mackenzie{at}lbss.gla.ac.uk

Two proponents of theory-based approaches to evaluation that have found favour in the UK in recent years are Theories of Change and Realistic Evaluation. In this article we share our evolving views on the points of connection and digression between the approaches based on our reading of the theory-based evaluation literature and our practice experience. We provide a background to the two approaches that emphasizes the importance of programme context in understanding how complex programmes lead to changes in outcomes.We then explore some of the differences in how `theory' is conceptualized and used within the two approaches and consider how knowledge is generated and cumulated in subtly different ways depending on the approach that is taken. Finally, we offer our thoughts on what this means for evaluators on the ground seeking an appropriate framework for their practice.

Key Words: context • knowledge generation • realistic evaluation • theories of change • theory

References

• Auspos, P. and A. Kubisch (2004) Building Knowledge about Community Change: Moving beyond Evaluations. Washington, DC: The Aspen Institute.
• Barnes, M., H. Sullivan and E. Matka (2003) `Evidence, Understanding and Complexity: Evaluation in Non-Linear Systems', Evaluation 9: 263—82.
• Barnes, M., L. Bauld, M. Benzeval, K. Judge, M. Mackenzie and H. Sullivan (2005) Health Action Zones: Partnerships for Health Equity. London: Routledge.
• Bauld, L., K. Judge, M. Barnes, M. Benzeval, M. Mackenzie and H. Sullivan (2005) 'Promoting Social Change: The Experience of Health Action Zones in England', Journal of Social Policy 34(3): 427—45.[CrossRef]
• Bonner, L. (2003) `Using Theory-Based Evaluations to Build Evidence-Based Health and Social Care Policy and Practice', Critical Public Health 13(1): 77—92.[CrossRef]
• Chen, H.T. (1990) Theory-Driven Evaluation. London: SAGE.
• Connell, J. and A. Kubisch (1998) `Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects and Problems', in K. Fulbright-Anderson, A. Kubisch and J. Connell (eds) New Approaches to Evaluating Community Initiatives, vol. 2, Theory, Measurement, and Analysis. Washington, DC: Aspen Institute.
• Connell, J.P., A.C. Kubisch, L.B. Schorr and C.H. Weiss (1995) New Approaches to Evaluating Community Initiatives, vol. 1, Concepts, Methods and Contexts. Washington, DC: Aspen Institute.
• Dahler-Larsen, P. (2001) `From Programme Theory to Constructivism: On Tragic, Magic and Competing Theories', Evaluation 7: 331—49.[Abstract/Free Full Text]
• Davies, H. T. O., S. M. Nutley and P. C. Smith, eds ( 2000) What Works? Evidence Based Policy and Practice in Public Services. Bristol: Policy Press.
• Dowling, B., M. Powell and C. Glendenning (2004) `Conceptualising Successful Partnerships', Community 12(4): 309—17.
• Fulbright-Anderson, K., A. Kubisch and J. Connell, eds (1998) New Approaches to Evaluating Community Initiatives, vol. 2, Theory, Measurement, and Analysis. Washington, DC: Aspen Institute.
• Gambone, M. (1998) `Challenges of Measurement in Community Change Initiatives', in K. Fulbright-Anderson, A. Kubisch and J. Connell (eds) New Approaches to Evaluating Community Initiatives, vol. 2, Theory, Measurement, and Analysis. Washington, DC: Aspen Institute.
• Julnes, G., M.M. Mark and G. Henry (1998) `Promoting Realism in Evaluation: Realistic Evaluation and the Broader Context', Evaluation 4(4): 483—506.[Abstract/Free Full Text]
• Lindgren, L. (2001) `The Non Profit Sector Meets the Performance-Management Movement', Evaluation 7(3): 285—392.[Abstract/Free Full Text]
• Mackenzie, M. and A. Blamey (2005) `The Practice and the Theory: Lessons from the Application of a Theories of Change Approach', Evaluation 11(2): 151—68.[Abstract]
• Mackenzie, M., A. Blamey and P. Hanlon (2006) `Using and Generating Evidence: Policy Makers' Reflections on Commissioning and Learning from the Scottish Health Demonstration Projects', Evidence and Policy 2(2): 211—26.
• Martin, S. and I. Sanderson (1999) `Evaluating Public Policy Experiments: Measuring Outcomes, Monitoring Processes or Managing Pilots?', Evaluation 5: 245—58.[Abstract/Free Full Text]
• Pawson, R. (2002a) `Evidence-Based Policy: In Search of a Method ', Evaluation 8(2): 157—81.[Abstract/Free Full Text]
• Pawson, R. (2002b) `Evidence-Based Policy: The Promise of Realist Synthesis', Evaluation 8(3): 340—58.[Abstract/Free Full Text]
• Pawson, R. (2003) `Nothing as Practical as a Good Theory', Evaluation 9(4): 471—90.[CrossRef]
• Pawson, R. (2006) Evidence-Based Policy: A Realist Perspective. London: SAGE.
• Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: SAGE.
• Stame, N. (2004) `Theory-Based Evaluation and Varieties of Complexity ', Evaluation 10(1): 58—76.[Abstract]
• Sullivan, H. and M. Stewart (2006) `Who Owns the Theory of Change?', Evaluation 12(2): 179—99.[Abstract/Free Full Text]
• Sullivan, H., M. Barnes and E. Matka (2002) `Building Collaborative Capacity through "Theories of Change": Early Lessons from the Evaluation of Health Action Zones in England', Evaluation 8: 205—26.[Abstract/Free Full Text]
• Tilley, N. (2004) `Applying Theory-Driven Evaluation to the British Crime Reduction Programme', Criminal Justice 4(3): 255—76.[Abstract]
• Weiss, C.H. (1995) `Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community-Based Initiatives for Children and Families', in J. P. Connell, A. C. Kubisch, L. B. Schorr and C. H. Weiss (eds) New Approaches to Evaluating Community Initiatives, vol. 1, Concepts, Methods and Contexts. Washington, DC: Aspen Institute.
• Weiss, C.H. (1998) Evaluation: Methods for Studying Programs and Policies. Upper Saddle River, NJ: Prentice Hall.
• Weitzman, B.C., D. Silver and K.-N. Dillman (2002) `Integrating a Comparison Group Design into a Theory of Change Evaluation: The Case of the Urban Health Initiative', American Journal of Evaluation 23(4): 371—85.[Abstract]

Evaluation, Vol. 13, No. 4, 439-455 (2007)
DOI: 10.1177/1356389007082129


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Blamey, A. Articles by Mackenzie, M. Search for Related Content Social Bookmarking                         What's this?