Evaluating the impact of Alternatives to Violence Project (AVP) workshops on social capital

Participants from an AVP workshop in Sydney

The NSW Service for the Treatment and Rehabilitation of Torture and Trauma Survivors (STARTTS) have been supporting Alternatives to Violence Project (AVP) workshops in Sydney since 2012, and we are in the process of planning an evaluation of the impact of AVP on social capital.

For over 25 years, STARTTS has provided psychological and psychosocial treatment and support to people healing from the scars of torture and refugee trauma, and who are rebuilding their lives in Australia. In particular they support people and communities from refugee backgrounds, including asylum seekers, who were forced to leave their country due to persecution in the context of political conflict, organised violence and human rights violations. STARTTS also supports and resources service providers, educational institutions and volunteer groups to work more effectively with refugees.

AVP workshops are conducted in over 50 countries around the World, indicating that it can be culturally adapted to a range of settings, and they have been used in number of settings where there has been widespread experiences of torture and trauma1-5. AVP workshops are now regularly held with people who come to the workshops through STARTTS and from the broader community.

STARTTS supports the AVP workshops because they recognise that peace-building and reconciliation are important parts of trauma recovery, and see them as important components of their community development work. They also have a commitment to using research as a tool to explore, and share knowledge on, the psychological needs of traumatised refugees. (See for example their report on ‘The glue that binds’: Social capital in refugee communities settling in Australia6).

STARTTS are thus supporting an evaluation of AVP workshops that focuses on the impact of AVP on social capital, self-efficacy (the belief you can have an impact on what happens to you) and responding to conflict, particularly in the context of multi-cultural communities. I am involved in the evaluation not only because I am an AVP facilitator, but also because I hope to make nonviolence greater focus of my work again.

In order to measure the impact, we have identified five main objectives:

  1. Increased self-confidence, self-esteem and self-worth
  2. Increased sense of safety and stability
  3. Increased trust
  4. Increased harmony with family and other close social contacts
  5. Strengthened or expanded connections with people from outside one’s immediate community.

Each of these objectives, which are from a social capital evaluation tool developed by STARTTS  and UNSW7, relate to at least one of the AVP objectives identified in Tomlison’ literature review8 and have at least one associated indicator (see below).

Social Capital Objective AVP Objectives8 Indicators
Increased self-confidence, self-esteem and self-worth To develop participants’ self-esteem Increased ability to calmly handle difficult or culturally challenging situations

Increased sense of self efficacy

Increased sense of safety and stability To develop participant’s capacity for trust and cooperation

To manage and resolve conflict nonviolently

Increased understanding of how to address and manage conflict
Increased trust To develop participant’s capacity for trust and cooperation Increased willingness to engage with people from outside one’s immediate community
Increased harmony with family and other close social contacts To develop participants’ communication skills

To manage and resolve conflict nonviolently

Increased positive communication with family members and other close social contacts
Strengthened or expanded connections with people from outside one’s immediate community To develop participants’ communication skills

To develop participant’s capacity for trust and cooperation

Increased positive relationships with people outside one’s immediate community or locality

We will collect data through:

  1. A pre and post questionnaire
  2. Focus groups held during the workshops
  3. Analysis of data from 20 years of workshop feedback sheets.

The pre and post questionnaire

At the start and end of each workshop, we will ask participants to complete a questionnaire which includes the General Self Efficacy Scale and some of our own questions.

The General Self Efficacy Scale (GSES) is a 10 question, 4-point scale (1 = Not at all true; 2 = Hardly true; 3 = Moderately true; 4 = Exactly true) freely available in 32 languages. The questions are:

  1. I can always manage to solve difficult problems if I try hard enough
  2. If someone opposes me, I can find the means and ways to get what I want
  3. It is easy for me to stick to my aims and accomplish my goals
  4. I am confident that I could deal efficiently with unexpected events
  5. Thanks to my resourcefulness, I know how to handle unforeseen situations
  6. I can solve most problems if I invest the necessary effort
  7. I can remain calm when facing difficulties because I can rely on my coping abilities
  8. When I am confronted with a problem, I can usually find several solutions
  9. If I am in trouble, I can usually think of a solution
  10. I can usually handle whatever comes my way.

At the moment we have omitted Question 2 because it is not consistent with key themes of AVP. We might explore rewording it so that it is more consistent with our approach while still capturing the idea that it is possible to work around challenges.

We will also include eight additional questions, using the same 4-point scale, to explore the other indicators we have identified. The draft questions are:

  1. I know how to manage conflict
  2. I can stay calm when others are angry or upset
  3. I can support my friends, even when we disagree
  4. I can share my feelings calmly with my family
  5. I am confident to talk to people from different communities
  6. I am friendly with people from outside my community
  7. I am confident to manage disagreements with people from other cultures
  8. If cultural differences cause conflict, I can help to manage them.

There are also demographic questions (e.g., gender, age, ethnicity) and questions about how many AVP workshops they have completed.

The focus groups

The focus groups will be held towards the end of the Basic and Trauma Awareness workshops, and towards the start of the Advanced and Training for Facilitators (T4F) workshops. (The Trauma Awareness workshops have been created by AVP Sydney in response to the work they are doing with STARTTS). As much as possible we are building the focus groups into the structure of the workshop so that they are a part of the workshop process rather than being purely evaluation.

The Basic workshop is the first workshop people do, and so the focus group can be a way of reflecting on what they have discovered during the workshop. Similarly, the focus group for the Trauma Awareness workshop will allow people to reflect on the trauma-specific content of that workshop. In the Advanced and T4F workshops, the focus groups allow participants to reflect on what they discovered in previous workshops and what impact this had for them, and focus participants on the coming workshop. In the Advanced workshop, where participants decide the topic(s) to be explored during the workshop, the focus group will be held immediately after the decision is made, which will give the facilitators (who don’t participate in the focus groups) time to plan the rest of the workshop.

The questions for the Basic include:

  1. Has this workshop changed how you would respond to conflict or violence? (How? Why not?)
  2. Do you think this workshop will change how much you engage or connect with people from other communities?
  3. Sometimes we can face challenging situations that relate to cultural differences between people. For example, people may not understand or appreciate beliefs and practices from cultures other than their own. Do you think this workshop will change how you handle situations like this?
  4. Here is a copy of the questionnaire you did in the workshop. Do you want to make any comments about how your answers changed between the start and the end of the workshop?
  5. What have been the best or most useful aspects of AVP?
  6. What do you think we should do differently next time?

The questions will also provide the opportunity for participants to provide insights into their responses to the questionnaire. For example, before we had introduced the focus groups, there was a large drop in one participant’s overall score for the General Self Efficacy Scale. It is possible that the focus group would have provided some insights into what led to this drop, or it may have indicated that there was a problem in how they filled in the questionnaire.

In the focus groups for Advanced and T4F workshops, we will include a mapping exercise where participants are invited to think about important people in their lives, the sorts of interactions they have with them and what their relationships were like before their first workshop and what changes may have occurred in these relationships and interactions. Participants will be encouraged to create a map of these relationships, which will then form the basis for reflecting on any changes.

The workshop feedback sheets

Sydney has been collecting feedback sheets for around 20 years. While they are largely satisfaction surveys, they include eight open ended questions some of which explore the workshop impact.

  1. What was the highlight of the workshop for you?
  2. What changed you the most inside – and why?
  3. How did the workshop change how you will deal with violence in the future?
  4. What did you find the least helpful in the workshop?
  5. Do you have any suggestions about how we could have made this more helpful to you?
  6. Is there anything you would like to say about the facilitation of the workshop?
  7. Do you have any suggestions for future workshops?
  8. Is there anything else you would like to say about the workshop or the material presented?

While these are not a primary focus of the evaluation, we hope the surveys could form the basis of some student evaluation projects and provide some useful insights.

Some dilemmas involved in the evaluation

Planning the evaluation has involved a number of dilemmas.

Who should facilitate the focus groups?

In order to reduce bias and to encourage honesty, it is best if the focus groups are facilitated by someone besides the facilitators. If the focus groups are run by the facilitators, then participants may not feel as free to express criticisms or doubts about the workshop, and the facilitators may, unintentionally, influence the responses received.

But if the facilitators conducted the focus groups, they could specifically ask about issues raised during the workshop and, if there was a good relationship, they might be able to encourage participants to be open and honest. In addition, if the facilitators led the discussion, the focus groups could be more integrated into the workshop process. Particularly in Advanced and T4F workshops, where the focus groups will be towards the beginning of the workshop, it would be valuable for the facilitators to hear the discussion.

Essentially we had to make a choice between prioritising the evaluation and prioritising the workshop process. In making the evaluation the priority, we believe that the focus groups have the potential for a positive impact on the workshops even if the workshop facilitators do not take part, and that it is important that we gather as reliable information as possible. If the workshops do suffer or participants feel uncomfortable with the process, we will reconsider our decision. When I piloted some of the focus group questions in a workshop, the participants generally agreed it would be better if the focus groups were done by someone besides the facilitators.

What to leave out of the workshops to fit in focus groups?

While AVP workshops do not have set agendas, these is a great deal of material that can be covered in a workshop and so the workshops are quite full. In particular, the end of a workshop, which is when the focus group is being held in the Basic and Trauma Awareness workshops, can become quite full and even rushed if care is not taken.

Fitting in a focus group of around an hour, could be quite tricky. This is why we are trying to incorporate the focus group in to the workshop as much as possible, so that the focus groups add to the workshop process.

I think it is worth making room for the focus groups as they will add a reflective aspect to the workshop and support people to reflect on the implications of the workshops for their daily lives. While ensuring the focus groups are facilitated by people not involved in the workshop does have some disadvantages, it also has the advantage of encouraging participants to have a more critical reflection on their experience.

How to ensure confidentiality but still track participants?

We want to track participants from one workshop to another (so we can observe changes in their responses) but also need to protect their confidentiality and anonymity. I would have thought there were some widely-accepted ways to do this, but I haven’t found anything. (Any suggestions would be great!)

Initially the plan was to have participants create a code consisting of the year of their birth followed by their initials. This meant, however, it was quite easy to identify participants. We are now using a code consisting of the day and month of their birth followed by their initials. So if Yasmin Harb’s birthday was 4 November, her ID would be 0411YH.

It would be good to find something less obvious than initials but we have struggled to come up with a suitable alternative. One possibility was the initials of their mother, but then what happens if they don’t know their mother’s name or if they have a few people they call mother? It needs to be something that will be easy for them to remember, that doesn’t change and that will produce the same answer every time it is asked.

When we trialled the questionnaire the first time, quite a few people completed the code incorrectly, so now when facilitators give participants the questionnaire, they explain the code in more detail, which has increased the consistency in the use of the code.

In order to ensure confidentiality, when the questionnaires are completed, they are placed in an envelope and sealed without the facilitators seeing the responses. The forms are then sent to STARTTS who enter the data without knowing who has completed the workshop. When results are reported, the codes are not used.

This means that nobody sees both the code and the participants, which allows us to be more confident of people’s confidentiality.

Conclusion

Even though Alternatives to Violence Project was started in 1975, there is not a great deal of research demonstrating the effectiveness of the workshops. (For literature on AVP visit the list of resources we are creating.) We hope that this evaluation will contribute to building an evidence-base.

This blog is based on a presentation I am giving (online) to the AVP International Gathering being held in Nepal next week.

Thanks to the rest of the evaluation project team:  David Ajang (STARTTS), Helen Bibby (STARTTS), Katherine Smith (AVP), Malcolm Smith (AVP), Raphael Manirakiza (AVP & University of Sydney) and Wendy Lambourne (AVP & University of Sydney).

Thanks also to Ansuya Naguran, Jasmina Bajraktarevic, Jorge Aroche, Kedar Maharjan and Mariano Coello from STARTTS for their ongoing support of AVP and this evaluation.

If you liked this post please follow my blog, and you might like to look at:

  1. What are Alternatives to Violence Project (AVP) workshops?
  2. Strengths-based measurement
  3. Creating a collection of literature on the Alternatives to Violence Project (AVP)
  4. Principles of nonviolence
  5. What is social capital?
  6. 12 principles of a problem solving approach to conflict resolution

If you find any problems with the blog, (e.g., broken links or typos) I’d love to hear about them. You can either add a comment below or contact me via the Contact page.

References

  1. Kreitzer, L. M., & Jou, M. K. (2010). Social work with victims of genocide: The Alternatives to Violence Project (AVP) in Rwanda. International Social Work, 53(1), 73-86. doi: 10.1177/0020872809348954
  2. Niyongabo, A., & Yeomans, P. (2003). “I still believe there is good in all people”: An evaluation of the Alternatives to Violence Project in Rwanda. Kipkaren River: African Great Lakes Initiative of the Friends Peace Teams.
  3. Chico, L. S. (2007). I am my neighbour’s mirror: A community rebuilding after genocide. Kipkaren River: African Great Lakes Initiative of the Friends Peace Teams.
  4. Lambourne, W., & Manirakiza, R. (2017). Community-based psychosocial interventions for refugees living in Australia. Paper presented at the 1st Australia and New Zealand Refugee Trauma Recovery in Resettlement Conference, Sydney.
  5. Melaleuca Refugee Centre. (2014). AVP peace leadership training project: Report to NT Department of Children and Families. Darwin: Melaleuca Refugee Centre.
  6. Doney, G., Pittaway, E., Bartolomei, L., & Ward, K. (2013). ‘The glue that binds’: Social capital in refugee communities settling in Australia. Sydney: STARTTS.
  7. Bartolomei, L., Pittaway, E., & Ward, K. (2013). ‘The glue that binds’ the social capital evaluation tool: Objectives and indicators for social capital projects with refugee communities settling in Australia. Sydney: STARTTS.
  8. Tomlinson, K. (2007). A review of the literature concerning the Alternatives to Violence Project (AVP). Alternatives to Violence Project Britain.

About Graeme Stuart

Lecturer (Family Action Centre, Newcastle Uni), blogger (Sustaining Community), environmentalist, Alternatives to Violence Project facilitator, father. Passionate about families, community development, peace & sustainability.
This entry was posted in Alternatives to Violence Project (AVP), Being an academic and tagged , , , , , . Bookmark the permalink.

2 Responses to Evaluating the impact of Alternatives to Violence Project (AVP) workshops on social capital

  1. Vaughn John says:

    Hi Graeme

    I’ve found this post very useful for thinking about how AVP’s impact could be evaluated. The tools you are developing could be useful for other contexts too, perhaps with some revisions. Coincidentally, I heard about the evaluation from a talk Wendy gave in Durban, South Africa this week. I subsequently wrote to her to indicate my interest in wanting to learn more about STARTTS and the evaluation. So this post is most timely …

    Two thoughts …

    Could Q2 of the GSES be revised to something like:
    When someone disagrees with my view, I have the strength to articulate/defend my position?
    One drawback of such revisions is that your GSES scores may then not be comparable with GSES scores from other studies using the standardized scale.

    Could the initials of the adjective name provide part of the code you need. Perhaps these two initials and the persons birthday and month eg. VV1503 would be mine.
    If you provided more durable nametags (the conference badges with pins rather than sticky labels) participants could write their adjective name on the front and their code at the back. You could collect these at the end of the workshop and make them available to participants when they return for subsequent workshops. This would help them keep to the same code throughout.

    Regards and best wishes with the evaluation
    Vaughn

    Liked by 1 person

    • Thanks Vaughn – great that you caught up with Wendy.
      What to do with Question 2 is a bit tricky. By omitting it, we can’t really compare it to other studies either. Thanks for your suggestion – it is more in keeping with the AVP approach.
      Interesting idea re using the adjective name for the code. (For others, it is an activity where people choose a positive word that starts with the same letter, rhymes or is in some other way connected with their first name, and add it to their name.) At least in Newcastle we often encourage people to change their adjective name through the workshop. We do this for two main reasons.
      1. People sometimes take a “safe”, not very meaningful adjective name at the start of the workshop and later on are willing to try something more meaningful or more aspirational. (E.g., Gentle Graeme compared to Groovy Graeme)
      2. To me the adjective name isn’t about boxing or labelling ourselves. One day I might feel gentle, but another time I might feel growing.
      3. We sometimes encourage people to experiment with a name to see how it sits with them.
      That means the adjective name could change or people might not remember which name they used. Putting it on a name badge might make it easier to remember but means it wouldn’t be as anonymous.
      I’ve been surprised at how challenging it has been to come up with a solution!
      Thanks again for your thoughts and I’m glad you found it interesting.
      Regards
      Graeme

      Like

I'd love to hear what you think!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s