Evaluation current issue

Editorial

Stern, E. — Thu, 01 Oct 2009 03:40:22 PDT

Evaluating Service Organization Models: The Relevance and Methodological Challenges of a Configurational Approach

Thu, 01 Oct 2009 03:40:22 PDT

Based on the example of the evaluation of service organization models, this article shows how a configurational approach overcomes the limits of traditional methods which for the most part have studied the individual components of various models considered independently of one another. These traditional methods have led to results (observed effects) that are difficult to interpret. The configurational approach, in contrast, is based on the hypothesis that effects are associated with a set of internally coherent model features that form various configurations. These configurations, like their effects, are context-dependent. We explore the theoretical basis of the configuration approach in order to emphasize its relevance, and discuss the methodological challenges inherent in the application of this approach through an in-depth analysis of the scientific literature. We also propose methodological solutions to these challenges. We illustrate from an example how a configurational approach has been used to evaluate primary care models. Finally, we begin a discussion on the implications of this new evaluation approach for the scientific and decision-making communities.

Synthesizing Legislative Evaluations: Putting the Pieces Together

Klein Haarhuis, C.M., Niemeijer, E. — Thu, 01 Oct 2009 03:40:22 PDT

Laws are important vehicles for policy. They are generally complex and involve various interventions. Despite their pervasive presence and numerous evaluation studies, laws have not been a topic of meta-analytical interest among social scientists. As a result, we lack an overall picture of the type(s) of interventions involved in laws, of the chains of events these are expected to set off and of the outcomes. Nor do we know much about the contents and quality of legislative evaluations. In a first attempt to address this gap, we developed a method to review legislative evaluations inspired by the sociology of law and by the realist synthesis approach. The article presents the results of a review of 75 legislative evaluation reports, which were completed between 1998 and 2005 in the Netherlands. These reports address laws in a variety of policy domains, from justice to health care and from social affairs to education. First, we subjected the reports to a methodological quality assessment procedure, after which 59 reports of acceptable quality remained for inclusion in a synthesis. Among other things, our method revealed that laws very often contain (new) public management interventions directed at executive bodies, rather than regulatory or economic interventions directed at citizens and businesses. The chains of events to be activated by these interventions remain vague. Nevertheless, we were able to come up with a preliminary typology of these programme ‘mechanisms’, which can be used as input for future research. Our method also demonstrated that contexts affected these chains of events in a variety of ways.

Stakeholder Mapping as an Assessment Framework for Policy Implementation

Mehrizi, M. H. R., Ghasemzadeh, F., Molas-Gallart, J. — Thu, 01 Oct 2009 03:40:22 PDT

In this article we develop a ‘Stakeholder Mapping’ approach to ex-ante policy evaluation. The proposed framework helps to assess policy implementation activities by applying several tests to evaluate the completeness, non-redundancy, proper assignment and internal consistency of a policy design. We illustrate the method by applying it to the analysis of e-commerce policy in Iran. We conclude by assessing the strengths and weaknesses of the approach.

Using the Objective Hermeneutics Method in Policy Evaluation

Mann, S., Schweiger, J. — Thu, 01 Oct 2009 03:40:22 PDT

At the outset of a policy evaluation, it is often necessary to gather data from the few personnel charged with responsibility for the policy in order to better understand the policy and its objectives. The objective hermeneutics method provides a tool for deriving the maximum benefit from several core text sequences of such interviews. Two examples of interviews with administrative officials are provided: the first on the subject of investment subsidies for winegrowers and the second concerning subsidies for regional marketing initiatives. These are used to illustrate that the objective hermeneutics method reveals important thought structures and patterns of justification underlying policy design, which may prove highly useful in subsequent stages of policy evaluation.

The European Union's Drive towards Public Policy Evaluation: The Case of Spain

Vinas, V. — Thu, 01 Oct 2009 03:40:22 PDT

Spain’s entry into the European Union (EU), in 1986, represented a great challenge in several areas, including the evaluation of public policy. In 1988 Structural Fund regulations were amended to include mandatory evaluation of European structural policies by both the Commission and the member states, and the 1999 reform strengthened evaluation requirements. This contribution discusses the major changes introduced in the Spanish evaluation landscape due to Spain’s entry into the EU. It provides a description and analysis of a case that has elicited great interest among countries that have entered the EU in the 21st century and are trying to learn from the experience of Spain and other countries.

Evaluation Networking in Romania

Mihalache, R. — Thu, 01 Oct 2009 03:40:22 PDT

In the context of a growing evaluation community in Romania, networking in evaluation tends to be not only a topic for analysis and research, but mainly a practical tool for reaching the generic community of evaluation practitioners. Furthermore, this is a process requiring resources (time, dedicated people, information and funding) which are scarce. Therefore the better documented the process is, the higher the chances of achieving results. The process of networking in evaluation is of course influenced by the context in which it is taking place, mainly characterized by confusion around what evaluation really is and how it works and by confusion around what would be the roles and the benefits of networking in evaluation. In spite of these challenges there are some opportunities to justify this analysis, and probably the most relevant one is the interest of the members and stakeholders who realize the potential for the ‘soft’ but highly relevant contribution of networking to professional development in evaluation. In this context, the central authorities in Romania wanted to better understand networking in evaluation in order to identify how this process might be stimulated in the future, and the how Romanian Evaluation Association — EvalRom — came about.

Book Review: Ann Crabbe and Pieter Leroy, The Handbook of Environmental Policy Evaluation: London and Sterling, VA: Earthscan, 2008

Vedung, E. — Thu, 01 Oct 2009 03:40:22 PDT

Cairo 2009: Conference on Impact Evaluation in International Development

Thu, 01 Oct 2009 03:40:22 PDT

French-language abstracts

Thu, 01 Oct 2009 03:40:22 PDT

Acknowledgement of Referees

Thu, 01 Oct 2009 03:40:22 PDT