As practitioners and researchers we need to think carefully about the types of measures we use with the people we support. The measures we use can cause pain and distress for participants, undermine trust and engagement, and produce unreliable data.
A few years ago, I was talking with some researchers and policymakers about measuring outcomes of family services. One of them was complaining that many family workers resisted using standardised outcome measures. I asked if it could partly be because many services were strengths-based and didn’t want to use measures that focused on problems and what was wrong. Her response was “tough!” For her the research and measurement were much more important than adopting a strengths-based approach.
The danger of this approach was driven home to me in a recent Alternatives to Violence Project workshop for parents. (You can read some background to the workshop here.) In order to help measure the impact we decided to use a survey to explore the parents’ parenting style. We were hesitant about using the survey because of some of the questions, but decided to give it a go. The survey had 30 statements (with a scale of strongly disagree to strongly agree): some of which were positive (e.g., “If we want children to respect us, we must first treat them with respect” and “I usually give my child clear instructions as to how I want something done”) but others were quite negative (e.g., “I have to threaten my child with punishment at least once a week” and “My child gets a spanking at least once a month”).
The parents reacted really badly to the negative focus of many of the questions. All the parents faced major challenges as parents and in their life generally, and felt that by using the survey we were judging them. They were also worried about how the information would be used (e.g., would it be passed on to child protection) and said it reminded some of them about negative experience as children.
Fortunately, before asking them to complete the survey, we had explained that we wanted their feedback about the survey and had taken steps to ensure confidentiality. After they had placed their surveys in an envelope (so we didn’t see them) we asked what they thought about the survey. When it became clear they had reacted quite negatively and we had discussed it for a while, we invited them to take their surveys back and to destroy them.
If we had not returned the surveys and not set it up as carefully as we had, the trust we had started to build would have been undermined and it would have been harder to engage them. As it was, the experience helped build trust and engagement because, by returning the surveys without having seen their responses, we clearly demonstrated that we had listened to them, trusted their judgement and valued their insights.
Such surveys can also produce quite unreliable data. Some years back a family worker told me about a researcher who had given a group of mums she was working with an anonymous survey to complete. The worker thought some of the questions were quite personal and intrusive so after the researcher left, she asked the mums whether they were worried about answering the questions. They replied, “No, we just lied.”
When we asked the parents in the AVP workshop to complete the strengths-based Parent Empowerment and Efficacy Measure (PEEM) they were happy to complete it and responded quite positively to it.
The measures we use need to be consistent with our approach (if we are strengths-based we need to find or develop strengths-based measures), be respectful and be appropriate to our audience. It is not OK for us to think that research and evaluation are more important than the people we work with.
If you liked this post please follow my blog, and you might like to look at:
- 10 things I’ve learnt about strengths-based community engagement
- What is the Strengths Perspective?
- Seven principles for a strengths-based approach to working with groups
- What is asset-based community-driven development (ABCD)?
- Bottom-up community development
- What are Alternatives to Violence Project (AVP) workshops?
If you find any problems with the blog, (e.g., broken links or typos) I’d love to hear about them. You can either add a comment below or contact me via the Contact page.