The inspiration for this Special Issue of EuroVista was the work of the STREAM Project. STREAM was a major European research project, funded by the EU and involving criminal justice agencies, Ministries and universities in several European countries. It was led by the National Offender Management Service in England and Wales. The broad aim of the project was to support the development of effective practice across Europe in working with offenders in the community and to facilitate the sharing of evidence-based good practice.
One of the work packages in the project, led by Jean Hine at De Montfort University, Leicester, UK, tried to encourage and develop processes of evaluation. An earlier project (STARR) had found that there was a great deal of excellent practice taking place across Europe, but that evaluation was uneven: there has been substantial variation in both the extent and expectations of evaluation. Yet evaluation is necessary to show that projects and agencies are achieving their aims and to highlight how they may be able to improve their practice. So this component of STREAM was an attempt to develop guidance on evaluation to help member states learn from each other and to evaluate their own work. Among the questions to be addressed were: to what extent can agencies and organisations demonstrate the effectiveness of their work with offenders? how is evaluation undertaken? who funds and undertakes evaluations? how are findings used to enhance practice?
A great deal of probation research in recent years has attended to the question of what works? This has typically involved an analysis of the effects of various offending behaviour programmes. For example, how does the subsequent offending record of people who had completed a programme compare with a matched group of others who had not? But to undertake evaluative studies of this kind requires access to large sets of data and much of these data are not easy to obtain in many countries. Reconviction data (already no more than a proxy measure for reoffending) are difficult to obtain reliably, even in those countries that record diligently. We have wondered whether these demands have been daunting and sometimes discouraged probation practitioners from undertaking studies of their own and/or in partnership with a university. But there are other methods of evaluation that can and should be used in attempts to enhance practice. How evaluation is undertaken depends on what practitioners and researchers need to find out and it is this that ought to determine the methods they deploy.
Even if a programme can be shown to be effective, its achievements will only be replicated if the right conditions can be established. This depends not only on ‘programme integrity’, but on a range of organisational considerations, notably management, adequate staff training and resourcing to deliver the programmes. Indeed an authoritative review of what has been learnt since the introduction in the UK of what works places particular emphasis on ‘the importance of the broader service context in supporting effective intervention’ (Raynor and Robinson 2009: 109). Both practice and its evaluation, then, depend upon a context – of culture, organisation and infrastructure, of scholarship and research – and in these respects too there are wide variations across the continent that affect member states’ capabilities to implement and to test their work.
While the sharing of knowledge from evaluation can assist in the spread of best practice across Europe, ways of working identified as successful in one location cannot be exported/ imported indiscriminately because of these differences of judicial, social, organisational and economic context. The introduction of such imported practice needs to be evaluated to check its implementation and impacts, both intended and unintended. But with due regard to that caution, we are confident that we have a great deal to learn from one another.
Among the activities of the project was a seminar that took place in The Hague in September 2014. Several countries were represented and worked hard with members of the STREAM research team to consider how they might undertake evaluation and learn from one another. It was at this event that the idea was formed of compiling a special issue of EuroVista to offer a series of descriptive but also reflective accounts of specific research projects. Many of the papers in this collection were written by participants at that seminar.
The main ‘output’ of this component of STREAM has been a web-based ‘tool kit’ of evaluation methods, including examples of experience and good practice, which could be used to develop policy and practice across Europe. This special issue may be considered another output. The papers in this collection are intended to show that it is possible to undertake small-scale studies that illuminate the achievements of a programme or project and enable it to develop and improve. Authors were asked to include a detailed description of the aims of their evaluation; explain how these aims were translated into an evaluation design and then implemented; give some account of any difficulties in undertaking the evaluation and how they were addressed; outline how the results of the evaluation have been used; critically reflect on the evaluation in design and practice; and identify key learning points for anyone wishing to undertake a similar process. We hope that readers will be inspired by the contributions to this issue to undertake their own evaluations to enhance their work and to demonstrate their achievements to others.
The editors are extremely grateful to our contributors who have worked hard and responded to our requests and the comments of our reviewers with patience, courtesy and efficiency. We also wish to express our thanks to members of the STREAM research team: Jean Hine, Nick Flynn, Charlotte Knight, Jane Dominey, Joe Woods and Ross Little.
Raynor, P. and Robinson, G. (2009). Rehabilitation, Crime and Justice, Basingstoke: Palgrave Macmillan.
STARR Project – http://www.starr-probation.org/default.asp?page_id=101
STREAM Project – (Strategic Targeting of Recidivism through Evaluation And Monitoring [JUST/2011/JPEN/AG/2892]) http://www.stream-probation.eu/default.asp?page_id=101