|
Sign In to gain access to subscriptions and/or personal tools.
|
Developing Robust Approaches to Evaluating Social Programmes
Alice Sampson
University of East London, UK, A.Sampson{at}uel.ac.uk
Using `evidence' to falsify rather than verify patterns in data and searching for alternative explanations enables a better understanding of the circumstances that explain why and how a social programme works or does not work. An analysis of the extent to which a programme is meeting its aims and objectives to find out if it provides a solution to the policy problem, is more rigorous.The roles researchers adopt influence the quality of an evaluation; facilitating a better understanding of the theories embodied in programmes enhances an evaluation while being a `broker of compromise' can limit access to information. Researchers have a valuable role in promoting learning. A robust evaluation framework integrates strategies for generalizing at the outset and identifying mechanisms of change or causal mechanism is a way forward. Examples are taken from recent evaluations conducted by the author and colleagues to illustrate the arguments.
Key Words: evaluating solutions to social problems • generalizability • role of researchers • theory-driven evaluation • use of evidence
References
- Ahmad, E., R. Knight, A. Parmar, A. Sampson and S. Themelis (2003) `YIP Youth Work', proceedings from a seminar, 21 June 2003. Centre for Institutional Studies, University of East London.
- Brookes, N. (2002) `Monitoring Performance: A Survey of Satisfaction with Family Support Services amongst the Sure Start Target Population', Barts and The London Queen Mary's School of Medicine and Dentistry, unpublished manuscript.
- Brown, P. (1995) `The Role of the Evaluator in Comprehensive Community Initiatives', in J. Connell, A. Kubisch, L. Schorr and C. Weiss (eds) New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, pp. 201—25. Washington, DC: Aspen Institute.
- Brown, S. (1998) `Youth Works: An Evaluation', Faculty of Law, University of Sheffield, UK, unpublished manuscript.
- Burgess, T. (2000) `The Logic of Learning and its Implications for High Education', Higher Education Review 32(2): 53—65.
- Burgess, T. (2002) `Towards a Social Science: A Comment on Karl Popper's Twenty Seven Theses', paper presented at the Karl Popper Centenary Congress, University of Vienna, Austria, 3—7 July.
- Chen, H.-T. (1990) Theory-Driven Evaluations. London: SAGE.
- Chen, H.-T. and P. Rossi (1992) `Introduction: Integrating Theory into Evaluation Practice', in H.-T. Chen and P. Rossi (eds) Using Theory to Improve Program and Policy Evaluations, pp. 1—11. New York: Greenwood Press.
- Connell, J. and A. Kubisch (1999) Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects, and Problems. Available at http://aspenroundtable.org/vol2/connell/htm (accessed 15 June 2002).
- Connell, J., A. Kubisch, A. Schorr and C. Weiss (1995) New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts. Washington, DC: Aspen Institute.
- Cordis Bright Ltd (2003) `Sure Start Abbey: Community Survey 2002/03', unpublished manuscript.
- Fearnley, R. and N. McInroy (2000) `Agents of Change in East London: Reflections on Conducting Urban Regeneration Research', in T. Butler (ed.) Eastern Promise: Education and Social Renewal in London's Docklands, pp. 210—25. London: Lawrence & Wishart.
- Foot Whyte, W., ed. (1991) Participatory Action Research. London: SAGE.
- Griffith, J. and A. Sampson (2002) `Using and Developing the Popperian Approach to Evaluations', paper presented at Karl Popper Centenary Congress, University of Vienna, Austria, 3—7 July.
- Israel, B., S. Schurman and M. Hugentobler (1992) `Conducting Action Research: Relationships between Organisation Members and Researchers', Journal of Applied Behavioural Science 28(1): 74—101.[Abstract]
- Jarvie, I. (1972) Concepts and Society. London: Routledge & Kegan Paul.
- Judge, K. and L. Bauld (2001) `Strong Theory, Flexible Methods: Evaluating Complex Community-Based Initiatives', Critical Public Health 11(1): 19—38.[CrossRef]
- Kubisch, A., K. Fulbright-Anderson and J. Connell (1995) `Evaluating Community Initiatives: A Progress Report '. Available at: http://www.aspenroundtable.org/vol2/kubisch.htm (accessed June, 2002).
- Lincoln, L. (2001) `Engaging Sympathies: Relationships between Action Research and Social Constructivism', in P. Reason and H. Bradbury (eds) Handbook of Action Research, pp. 124—32. London: SAGE.
- Mackenzie, M. and A. Blamey (2005) `The Practice and the Theory: Lessons from the Application of a Theories of Change Approach', Evaluation 11(2): 151—68.[Abstract]
- McQuail, L. and V. Wigfall (2001) `Euston Sure Start: A Needs Survey'. Available at: http://www.ness.bbk.ac.uk (accessed 20 February 2003).
- Majone, G. (1980) `Policies as Theories', Omega: The International Journal of Management Science 8(2): 151—62.
- Marshall, J. (2001) `Self-Reflective Inquiry Practices', in P. Reason and H. Badbury (eds) Handbook of Action Research, pp. 433—9. London: SAGE.
- Morgan Harris Burrows (2003) `Evaluation of the Youth Inclusion Programme'. Available at http://www.youth-justice-board.gov.uk (accessed 30 November 2003).
- National Statistics Online (2003) ` Neighbourhood Statistics'. Available at: http://www.neighbourhood.statistics.gov.uk (accessed 3 February 2003).
- Nutley, S., I. Walter and H. Davies (2003) `From Knowing to Doing: A Framework for Understanding the Evidence-Into-Practice Agenda', Evaluation 9(2): 125—48.[Abstract]
- Oakley, A. (2000) Experiments in Knowing. Cambridge: Polity Press.
- Parmar, A. and A. Sampson (2007) `Evaluating Domestic Violence Initiatives: Transferring Practice Principles Not Projects', British Journal of Criminology 47(4): 671—91.[Abstract/Free Full Text]
- Parmar, A., A. Sampson and P. Jordan (2003) `Final Site Report of Croydon Domestic Violence Advocacy Project', submitted to the Home Office, unpublished manuscript.
- Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: SAGE.
- Popper, K. (1968) The Logic of Scientific Discovery, 2nd edn. London: Hutchinson.
- Popper, K. (1969) Conjectures and Refutations: The Growth of Scientific Knowledge, 3rd edn. London: Routledge & Kegan Paul.
- Reason, P. and H. Bradbury, eds (2001) Handbook of Action Research. London: SAGE.
- Sayer, A. (1992) Method in Social Science: A Realist Approach, 2nd edn. London: Routledge.
- Weiss, C. (1972) Evaluation Research. Englewood Cliffs, NJ: Prentice-Hall.
- Weiss, C. (1995) `Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families', in J. Connell, A. Kubisch, L. Schorr and C. Weiss (eds) New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, pp. 65—92. Washington, DC: Aspen Institute.
- Weiss, C. (1997a) `How can Theory-Based Evaluation Make Greater Headway?', Evaluation Review 21(4): 501—524.[Abstract/Free Full Text]
- Weiss, C. (1997b) `Theory-Based Evaluation: Past, Present and Future ', New Directions for Evaluation 76: 41—55.
Evaluation, Vol. 13, No. 4,
477-493 (2007)
DOI: 10.1177/1356389007082132
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
|
|
