The first thing that made me think, a lot, was this story from the BBC - "University study says city's 'Triple P' parenting scheme 'a failure'". Two things I find most interesting are firstly that the evaluation did find triple-p was a failure (more of that later) and secondly the reaction from the partners who commissioned the evaluation. The partners, particularly the Health Board, immediately dismissed the evaluation it seems. I would love to have been studying the whole process from the outside because I'm left with a number of questions:
- Were the partners ready and willing to hear results that were negative?
- How was the evaluation commissioned? And what project oversight was there as it ran? (why wasn't this foreseen?)
- How did the evaluation team work? Did they communicate interim findings?
- Do the partners have a learning culture where they are ready to reflect on failure, develop and move on?
- Why did they spend money commissioning an evaluation and then dismiss the results in this way? Is this an efficient use of limited resources?
Obviously, I've neither read the evaluation, or been part of this process, so these will have to be left as questions. However, I find it interesting that the Health Board are quoted as saying: "The result is an incomplete evaluation based on very limited data and conclusions that do not stand up to scrutiny" which, to me, does not suggest a learning culture at all; it suggests a culture where if you're criticised the first thing you attempt to do is undermine your critics.
I wonder if a weakness here was the lack of a developmental approach to evaluation overall by all parties. The project I'm leading has fallen into using developmental evaluation - it's essentially just a process of testing things through a process and then stepping back and reflecting on why the worked or did not, and moving forward. It aims to create a learning context within partnerships.
The second thing that made me reflect on evaluation was a bit of developmental evaluation happening in transport in Edinburgh. For a year, the Council have made George Street a one-way street for vehicles and made the other side of carriageway a two-way segregated cycle way with some huts outside restaurants for the smokers and people foolish enough to want to eat outside in Scottish weather.
I love the George Street cycle path. I can cycle at the speed I want to go, rather than the speed determined by being chased by a Wheeled-Black-Box-Of-Death (taxi). It has some teething issues - for practical reasons, as it's an experiment it swaps side at Frederick Street. You have to follow the path through, something which I've always found very easy. The traffic lights at Hanover Street take an age to change, but they do for everyone. The exit and egress could be a lot better - it's an island of very good cycling provision in and amongst a load of guff though, so at the moment there is nothing to connect it to. In it's pragmatism it reminds me a lot of the Danish and German cycling infrastructure that I've seen, rather than the Dutch. It's far from perfect, but it keeps you a lot safer and it's more pleasant than the road before.
But what I like most is it is an experiment - the Council are being very open about this. They are actively seeking responses from people who use it. Already they've put in some temporary bollards to stop idiots driving down it. So far, they're doing exactly what the Triple-P parenting folk weren't - listening to criticism and adjusting things as they go along, if they can. I encourage all critiques of it (and seemingly by a controversial twitter exchange there are many) to contact the Council. The last thing I want to happen is for the Council to conclude "well, everyone thinks this is rubbish, so we might as well give up". Let's celebrate some small successes where they've happened and we'll get more and it will be better. And long may Edinburgh's experimental spirit continue. If you want to feedback, email citycentre.vision at edinburgh.gov.uk.
Which doesn't bring me back to Triple-P, but I want to add this. I'm glad the evaluation of Triple-P was so critical. It's been very popular on the Scottish policy scene along with the Family-Nurse-Partnership (FNP). These are imported policy-initiatives that are all about "early intervention" which in Scotland is very much aligned to reducing the long term "burden" on the government from "failure". It's always troubled me that these initiatives can, and do, easily fall back on cultural explanations for poverty, without setting themselves in the broader context of a very economically unequal country, where the biggest cause of poverty is low income. Also, previous evaluations (as I've mentioned on this blog before) like the first incarnation of the FNP, Glasgow's "Starting Well" programme have been similarly critical. Often it is context that makes these interventions successful, as the core of realist evaluation make clear. Finally, as the evaluations of New Start (the inspiration for Sure Start) showed, and a finding repeated by a controversial recent study by the Institute for Education and Institute for Fiscal Studies on childcare showed, the benefits of these early intervention schemes get eroded fairly quickly by the daily grind of poverty and the children end up back where they started.
I don't want to dismiss these schemes completely out-of-hand. I'm currently reading the excellent Good Times, Bad Times by John Hills (BUY THIS BOOK) and he makes the point that the UK state does some of the greatest redistribution of any state in the world; the trouble is the society going into it is also the most unequal in the world, so the society we get out at the other end is very unequal. This suggests, to me, that there is something in Ed Milliband's "predistribution" - if we can increase skills, productivity and pay in the lower end of the labour market then we are probably going to achieve a lot more than relying on the state to redistribute income and wealth. And programmes such as Triple-P and a well-funded health visitor service (as the FNP should actually be) could be part of a programme of constant, tailored, service delivery to support the most vulnerable and poorest in society. Kind of what I argue for in neighbourhood policy.
No comments:
Post a Comment