Planning and implementing evidence-based programs and practice in family services in rural and regional NSW

dorothee-graeme-and-deb-frsa-r

Dorothée, Me, Deb

The following is the text of a session that Dorothée Crawley (CentaCare Wilcannia Forbes), Deborah Hartman (Family Action Centre) and I presented at the Family and Relationship Services Australia (FRSA) conference

FRSAGraeme Stuart

We’d like to start by acknowledging the traditional custodians of the land we meet on and pay respect to their Elders past and present. We also pay respect to the knowledge and experience of Aboriginal and Torres Strait Islander families and practitioners throughout Australia.

We’re really pleased to be able to present today as we’re passionate about working with families and believe it’s important that our work is continually informed by a wide range of evidence.

I’m going to briefly introduce the project we’ve been involved in and discuss some of what we’ve learnt, Dorothee will outline her experience of using evidence in practice and then Deb will suggest some implications for policy and practice.

We were funded through the DSS Children and Families Expert Panel to support 9 services funded through the Children and Parenting Support program to enhance their ability to implement evidence-based programs and practice.

Location of servicesAs you can see the services ran programs, including parenting programs, playgroups, home visiting and information and referral, throughout large parts of rural and regional NSW.

While the focus of our presentation is on planning and implementing evidence-based practice, it may be worth noting that we provided support to the organisations through onsite visits and workshops, webinars, an online community of practice, brief summaries of research, and helping them to develop program logics.

Figure 1: Evidence-based practice (Source: Walsh, Rolls & Williams [2])

Evidence-based practice

Evidence-based practice is often described as a decision-making process that incorporates:

  1. The best research evidence
  2. The best clinical experience
  3. And ensures that it is consistent with family and client values [1].

Figure 3: Evidence-based practice. (Adapted from Walsh, Rolls Reutz, & Williams [10])

Evidence-based practice (valuing the experience and insights of families)

Because we come from a strengths-based, family-centred approach, we felt more comfortable describing it as incorporating family experience and insights rather than just being consistent with family values. This is a subtle, but important difference as it recognises that the experience of families is actually important evidence and not just something needing to be considered.

It’s also important to differentiate between evidence-based practice and evidence-based programs.

  1. Evidence-based practices (e.g., engaging parents) are built on theory and research but are not a complete, systematised program of intervention
  2. Evidence-based programs (e.g., Circle of Security) are standardised, systematised and rigorously evaluated

The practitioners we worked with largely relied on practitioner wisdom, generally had structures in place to reflect on their practice and would get feedback from other practitioners.

They also drew on data and feedback from the families they worked with but not many of them used standardised outcome measures, and this is an area many of them would like to explore further.

When it came to using research, they generally struggled to find the time to search for, and read, research, or to do their own research. Their focus was very much on service delivery.

With the exception of one organisation, which had a research team based in Sydney, none of them had anybody who was responsible for looking for research. As somebody said,

There’s no one we can flick off to: “Oh can you research this for us?”

Some of the managers were conscious of the importance of looking at research and literature, but it was largely because of a personal commitment rather than because they had time dedicated to it.

When they did look for research they often struggled to find relevant evidence about the families they worked with (e.g., Aboriginal families, families with complex needs) and even if they did find interesting articles they couldn’t down load them or the articles were too expensive.

Given these barriers to using research, and DSS requirements to use evidence-based programs, the main way they used research was through registries of evidence-based programs, especially the Australian Institute of Family Studies program profiles. Sometime they would find a program that looked interesting and then see if it was “on the list” and sometimes they looked through the list to see what they could find.

The main time they actively went looking for research evidence was when they were applying for funding and needed to demonstrate their proposal was evidence-based. Drawing on practitioner wisdom, they designed their proposal and then found “some clever sounding research” to back them up.

Having said that, some of the managers were conscious about looking for a range of evidence, and one organisation had a researcher specifically looking at evidence-based practice.

There were a number of challenges in using evidence-based programs. Many of the programs were either inappropriate for the families they worked with, or there was no evidence that they worked in their context. For example, increasingly services find they have parents attending parenting programs who don’t have custody of their children. Most parenting programs are not designed for these parents (e.g., they include homework to do with participants’ children between sessions), and there isn’t strong evidence that they work with non-custodial parents.

The costs involved in running evidence-based programs, especially for rural services, is a real problem. As well as some programs having licence fees, the cost of training staff was significant, particularly when travel, accommodation and meals were included. For example, it cost one organisation approximately $1500 to send two staff to “free” Triple P training and it cost another organisation $25 000 to have all their staff trained in Parents Under Pressure.

The need to have a variety of programs (because no single program can meet the needs of a wide range of families) meant that locations which were funded for only one or two part-time staff, found it really hard to meet the costs associated with  providing evidence-based programs.

Dorothée Crawley

The purpose of my part of this presentation is to share how CWF as an organisation puts evidence base into practice and why.

Just to set a little bit of the context of who we are:

  • 52% of NSW
  • 4 major branches
  • 14 offices
  • Hub style service delivery
  • Rural and remote communities
  • Aboriginal communities

centacare-wfAll our decisions, approaches and practices are underpinned by our mission. I have included it here to explain further why we commit to evidence practice and programs in the way we do.

To empower individuals, families and communities in Western NSW by enhancing social and emotional wellbeing and strengthening networks of support.

A little bit about our people:

  • 103 Staff
  • 22 Family Workers (funded by DSS, FaCS, Health)
  • 14 Female
  • 8 Male
  • 11 Aboriginal
  • 11 Non-Aboriginal

To appreciate why we invest so heavily in evidence based programs, it is important to understand where our people come from. This is a snapshot of our current team of family workers:

  • 4 Early Childhood
  • 4 Hospitality
  • 2 Studying (cert IV in community services)
  • 2 Without previous training or employment
  • 2 Health
  • 2 Employment
  • 1 Retail
  • 1 Beauty Therapy
  • 1 Legal Services
  • 1 Government Department
  • 1 Council
  • 1 Police

It is not uncommon for our family workers to come from an unrelated field. This is somewhat reflected in my very own story:

  • CentaCare’s first family worker
  • Based in Bourke (1000 km North-West of Sydney)
  • Closest regional centre: Dubbo (350 km South-East)
  • Population: 2500
  • 32% Aboriginal
  • Where did I come from?: Nutrition
  • My credentials: Attempting to raise five children in the ‘Back’o’Bourke

Again:  The context of where we are and who we are helps to understand why we invest so heavily in evidence based programs:

  • We are committed to recruiting locally.
  • The pool of applicants is limited and diverse.
  • We recruit attitude and build skills.
  • Over 70% of our family workers come from an unrelated field.
  • Almost 80% of our family workers have not worked in our sector before.
  • There is no single qualification for a family worker.

I will briefly touch on how we the put into practice the three elements of evidence base as described by Graeme but will I primarily focus on our approach to the delivery of evidence-based programs.

Figure 3: Evidence-based practice. (Adapted from Walsh, Rolls Reutz, & Williams [10])

Evidence -based practice

We recruit people who show capacity to build relationships. That is very important to us.

We look closely at their life skills, experiences and talents that will inform and assist their work.

During induction we conduct training needs’ analysis which inform the training that new and beginning team members require to be able to do the job.

We link experienced team members and new and beginning team members for peer support and learning.

We provide ongoing training and coaching.

We involve our teams in program development and evaluation by inviting them to be part of the program logic development process. This gives them the opportunity to name the needs they see in their communities, the change they want to create and how this can be done. It also gives them a sense of ownership.

We also regularly check in with our program participants in terms of what they see as the needs and how these can best addressed. This gives them also ownership of the program, even the service.

Through strength based service delivery models we explore families’ strengths and build on those.

We have chosen to train our family workers in a broad suite of evidence based programs:

  • 123 Magic – Engaging Adolescents
  • 123 Magic & Emotion Coaching
  • Bringing Up Great Kids
  • Circle of Security
  • Parents As Teachers
  • Parents Under Pressure (PuP)
  • Triple P (0-12)
  • Triple P (Indigenous Version)
  • Triple P (Teens)
  • Tuning into kids
  • Tuning into teens

Why do we do that? Our team members look for:

  • Skills needed for the job
  • Advice that has been tested
  • A toolkit of options
  • Confidence in service delivery

Our organisation looks for consistency in service delivery.

How do we make it work?

We train teams as a whole rather than individually as much as we can.  We bring training to our region rather than sending people away. Through whole of team training we create networks of support – people checking in with one another, co- delivering and – facilitating. Experienced team members coach newly trained team member through co-facilitation.

We allow team members as much as possible to adapt programs to make them work.

Bruce’s story

Bruce was one of our Aboriginal men’s worker, also a very passionate Dad of two little girls. An opportunity came about for him to be trained in Triple P. He agreed which meant he had to be away from home for a week.  He came back to his community on Friday and I called him on the Monday to see what he thought of the training. I expected a fair bit of feedback. I was a little bit anxious. Yet Bruce was excited. He said he loved it. I was surprised. I asked him if he had given thought to a trial run before his accreditation. He had already organized it. Triple P was to be delivered on Wednesday during his next men’s gathering on the Darling whilst cooking the fish they were going to catch. Bruce delivered the complete Triple P program on the river, entwined in storytelling and sharing. All content was covered, yet not a single slide was presented.

This may be seen as a breach of fidelity. The way we see it: it’s like hiding the vegetables in the meatloaf. You still get the goodness, you just do not know it’s there.

Do we see any downsides to investing in all this training? – Yes we do.

  • It is a great investment which represents a risk in the context of short term funding.
  • Highly trained people are much sought after. We have lost many team members after much investment into their training.

Yet…this still sits very comfortable with our Mission: Nothing is lost. Another individual has been empowered to do bigger and better things.

Deborah Hartman

Thanks Dorothee. Your story illustrates some of the key themes we identified together in this project. Support for organisations in the introduction of a reform such as this is vital. The organisations really appreciated the support and the fact that DSS was willing to fund this type of support, yet most organisations who want expert panel help have to find the funds from within their own resources. The organisations particularly valued the site visits from Graeme so the real context could be understood and explored. The opportunity to come together as a group at the start of the project in Sydney and the innovative methods of communication such as on-line communities of practice and webinars can help overcome the tyranny of distance by developing a strong network among services in similar contexts.

Adaptation is critical. All organisations were keen to implement and adapt programs and practices that they could be confident would meet the needs of the various families and clients they work with. The lack of programs with a strong evidence base for particular target groups (e.g., Aboriginal communities, fathers or parents without custody of their children) means that staff have to be confident in adapting programs to their particular context. At the start of the project, quite a few staff were under the impression that they should not adapt any part of a program, and were relieved to know that some adaptations are acceptable.

Ongoing training and follow up is critical to successful implementation. The level of awareness and capacity to implement evidence-based programs and practice before the project, varied greatly between organisations. It particularly helped the project when there was a strong, skilled advocate who understood evidence-based programs and practice. The level of awareness and capacity also varied greatly within organisations ranging from staff with no formal qualifications (but good skills in engaging and working with families) who worked largely from an intuitive base, to staff with post-graduate qualifications who felt confident dealing with research and evidence.

There is a need to focus not just on evidence-based programs but also evidence-based practice. Organisations appreciated the opportunity to reflect on evidence (including research and practitioner wisdom) relating to practice in particular areas (e.g., engaging “hard to reach” families and engaging fathers) and not just on how to select and implement evidence-based programs.

As Graeme mentioned, the cost of implementing evidence-based programs is considerable. As it is staff who are accredited, not organisations, staff turnover and extend leave (e.g., maternity and long service leave) make it particularly challenging and costly.

After training, staff need ongoing support in implementing programs to help them address problems and to prevent “program drift”. Funding for organisations to continue and update training in programs is important.

There is a need for more research around evidence-based practice, and research to practice resources. For example, as well as exploring the evidence about effectiveness of playgroups, there is a need for research which identifies practitioner skills and practices that make playgroups successful.

The expert panel has potential to be expanded so that services have easy access to people who can provide support in finding and interpreting research evidence and helping services collect their own evidence.

Organisations could be funded to have subscriptions to journal data bases so that practitioners can access journal articles freely through the AIFS website. At the same time, services could be encouraged to make greater use of university students’ projects and placements to undertake literature searches and review.

Support could be provided, again through the expert panel, to organisations needing to adapt evidence-based programs to particular contexts.

There is value in providing brief research to practice papers that summarise that latest research evidence in relation to working with families, particularly in terms of effective practice (not just programs). (We produced several short papers which we are happy to share).

Projects like this are successful at providing innovative ways of developing a network of support which could be further funded by DSS. All the organisations are now developing clear program logics. One organisation is now planning to submit a program to AIFS to be added to the Evidence Based Program List. Others  are intending to participate in trials to develop an evidence base for practices to support positive co-parenting relationships. One is developing a new program to work with Aboriginal families and is consciously incorporating evidence-based practice and measurement throughout.

We’d be happy to share resources we developed during this program and we’d like to honour the extremely important and difficult job that our partners in this project undertake every day with families in rural areas.

Reference

[1] Williams, K. E., Berthelsen, D., Nicholson, J. M., & Viviani, M. (2015). Systematic literature review: Research on supported playgroups. Brisbane: Queensland University of Technology. Available from http://eprints.qut.edu.au/91439/

If you liked this post please follow my blog, and you might like to look at:

  1. Posts from the expert panel work in evidence-based practice
  2. What is evidence-based practice?
  3. What are evidence-based programs?
  4. What is evidence-informed practice?
  5. Research evidence for family (and community) workers
  6. Finding literature on working with families

About Graeme Stuart

Lecturer (Family Action Centre, Newcastle Uni), blogger (Sustaining Community), environmentalist, Alternatives to Violence Project facilitator, father. Passionate about families, community development, peace & sustainability.
This entry was posted in Families & parenting and tagged , , . Bookmark the permalink.

I'd love to hear what you think!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s