Community Care Magazine reported recently that Children's Centres are being urged to focus on outcomes but concerns were raised regarding how success can be gauged.
Outcome (noun): A final product or end result; consequence; issue.
The dictionary explanation is simple enough. But measuring outcomes is a concept that has eluded children's services over the years. This is despite Every Child Matters distilling the entire system down to the achievement of five outcomes for every child, such as being healthy and staying safe.
The mistake made, according to Rob Hutchinson, a former director of children's services at Portsmouth and now an independent consultant, was that measures of achievement were in the form of either counting numbers (how many teenage parents there were) or efficiency measures (how many initial assessments had been completed in 10 days).
"These measures can be important but they mean nothing if you are not also measuring a third factor: are children better protected? says Hutchinson.
"Consequently we measured what was easy to measure but it filtered down as must-do targets for managers and workers at the expense of time spent with families."
The Munro Review of Child Protection called for a renewed focus on improving outcomes for each child. It added that the only factor that has been proven to do so is the relationship between a social worker and a family.
One of the few organisations to attempt to measure the outcomes from this relationship is Action for Children, which last year released an "impact" report on some of its early intervention and intensive intervention projects.
Kate Mulley, head of policy development and research for the charity, says the problem is that outcomes differ from child to child. "Trying to keep track of individual outcomes for each child and then aggregating them across the organisations proved incredibly difficult."
It is also easy to end up just collecting feedback from parents, she says. "You have to keep hold of the fact that we are trying to measure the impact on the child, particularly when it comes to areas such as neglect."
Action for Children's solution was to produce an outcomes toolkit containing tools associated with the most successful outcomes achieved for children on its programmes. For example, the Outcomes Star tool was correlated with an 82% increase in emotional well-being of children and young people while the Strengths and Difficulties Questionnaire was correlated with a 97% improvement to emotional well-being and a 90% improvement in ensuring protective measures were in place.
"I'm not advocating a return to form-filling but some of these researched assessment tools are incredibly useful in work with families. It's also better than simply counting the number of people on a programme. All the evidence shows that even very good programmes can make things worse if they are poorly executed or if the wrong people are being sent on them."
Hutchinson, who has helped devise an outcomes-based accountability toolkit, says measuring outcomes starts with the social worker asking themselves two simple questions:
● Have I treated you well?
● Have I made a difference? If so in what way?
Translating this into national indicators requires commissioners and local safeguarding children boards to ask similar key questions.
"If I'm commissioning services relating to child neglect how will I know they are being successful?"
In some areas commissioners have asked frontline social worker's and families this very question. It resulted in a series of measures of neglect including:
a) is the child properly clothed when it goes to school?
b) did the child have breakfast before starting school?
c) are they sleeping somewhere safe, clean and comfortable?
Senior interim children's services manager and consultant Nick Berbiers argues outcome measures need to be thought about and used across four levels.
● Individual: The extent to which good outcomes are being achieved for each child. This should influence changes to the plan and the level or type of services being provided.
● Organisational: annual outcome data (based on a national set of indicators) for all children and young people. This should inform service development in both councils and partner agencies as well as inspection reports.
● Transactional: Independent and third sector providers should be able to opt-in to an outcomes framework. This could then be used by local authorities to inform commissioning and purchasing
● Developmental: Longer term analysis of outcome measurement data to understand which practices, interventions and methodologies are the most successful in achieving good outcomes for children.
Sue White, professor of social work at Birmingham University and a spokeswoman for the College of Social Work, believes the role of inspection is vital and Ofsted needs to be taking a more imaginative approach.
"The simple answer is that we can't translate the relationship between a social worker and a family into 'measurable outcomes'.
"But there are some things we do know. We know families don't like numerous changes of social workers, we know there are poor outcomes when social workers do not properly talk to children. But having a crude indicator of 'did you speak to the child?' achieves nothing. You need to know how they are speaking with the child. Inspectors need to be spending the bulk of their time on the job with social workers and families, not looking through documents at head office."
This may be the only way in which outcomes can compete against the impact of cuts warns Catherine Witt, senior manager for social care at Cumbria Council - one of the areas piloting the scrapping of assessment deadlines as recommended by the Munro Review.
"We reassessed everything we did and realised we were wasting a lot of time and effort on 'feeding the machine' in order to meet previous targets," Witt says. "But we're finding that that time has simply been replaced by the time needed for reflective practice and good supervision. There's no doubt this will result in better outcomes but they may not appear for some time. Trying to quantify such long-term outcomes in hard numbers in a climate where councils are looking to make cost savings now has been a key difficulty for us."
Mulley points out that other flagship policies such as payment-by-results will also need to be considered when devising national outcome measurements.
Despite the difficulties, Berbiers says the sector cannot afford to shy away from outcome measures.
"Put starkly, we cannot design, deliver, commission or purchase services, or train and develop professionals if we cannot actually demonstrate and evidence what works best for vulnerable children and young people," he says.
OUTCOMES ACROSS THE NATIONS
England
The Munro Review included a draft set of national indicators for children's social care which included social worker caseloads, children's perceptions of their own safety, social worker perceptions of whether they have had enough time with families and whether their interventions have improved safety. The government accepted this dataset as a "good basis for further work on outcomes". The final dataset will be finalised by May 2012.
Wales
The Welsh Assembly is currently developing a National Outcomes Framework. Once completed it will be used alongside high level indicators which now include the number of children seen alone by social workers at assessment, the number of 16- and 17-year-olds with a pathway plan and the number of children classified as in-need.
Scotland
The Scottish government has adopted a national performance framework which contains 15 national outcomes including ensuring "our children have the best start in life" and that "we have improved the life chances for children, young people and families at risk". The government has set out a number of achievement indicators including the proportion of area child protection committees receiving positive reports.
Northern Ireland
In 2006 the government set out six outcomes for all children, including living in safety and with stability. Work is ongoing in identifying national indicators for each. The government is intending to ask children, young people and families to contribute to their outcomes monitoring report so that qualitative as well as quantitative information is collected.
Published in Community Care 20 October under the heading 'Getting the measure of outcomes'
So what do you think? what is your experience of trying to measure outcomes? some centres have made a really good effort at this in my opinion - despite the conflicting requests asked of them. Others seem permanently lost in the sea of outcome measuring confusion!
