Change and Adaptation

03. 08. 2016

Evaluation

Goals

Change and Adaptation aims to contribute to improving health and wellbeing for communities and individuals by embedding arts practice in the delivery of services provided by the health and environment sectors. This involves the introduction of collaborative projects with artists that offer new and different ways for organisations to work. In the process it is hoped to demonstrate how the arts can support services to achieve their organisational goals and meet the needs of their community or clients.

The goals of the evaluation are therefore:

  • to collect and analyse feedback from staff in partner organisations that will assist artists to refine the ongoing development of projects
  • to identify and document best practice in partnerships between arts programs and other sectors
  • to build capacity in evaluating inter-sectoral projects among the lead artists.

Evaluation strategies

The goals are being achieved through:

  • individual and group meetings with managers of partner organisations
  • surveys and interviews (face-to-face and phone) with staff of ‘agencies’ (this refers to the service delivery units within the organisations)
  • regular meetings and workshops with lead artists and creative management team
  • support for lead artists in gathering feedback from their projects
  • design of data collection tools, analysis and reporting at each stage
  • development of tools for communicating the evaluation process across the program                                                                      (for example ‘Diagram of Evaluation Cycle at Project level’)

Examples of evaluation questions include:

Partnerships:

Q: How do they work? Are they sustainable? Is there mutual respect? Is there evidence of a growing commitment to inter-sectoral work with artists?

Sources: feedback from partner managers, C&A program staff

Organisations:

Q: What new or different ways of working are being introduced by the artists? Are the projects contributing to improved services? Are the organisational staff better able to partner with arts in future?

Sources: Feedback from agency staff & managers, C&A program staff

Artists:

Q: Are artists refining their skills in working across sectors? Are they more confident using research tools to gain participant feedback? Are they able to express the value of their practice effectively to other sectors?

Sources: Development of tools and data collected, feedback in individual and group meetings

Community:

Q: Are there evident or reported benefits for participants & the wider community from
the projects & the partnerships?

Sources: Feedback from agency staff & managers, program staff, artists,
individual participant and group feedback


 Evaluation Reporting

After the completion of the program the evaluation outcomes will be reported in two forms:

Main Report

– a summary of the overall impact of the program partnerships in relation to the goals.

Contents to include:

  • Report on evaluation/research process
  • Summary of outcomes against goals
  • Best practice insights:
    • Partnerships – key findings
    • Organisations – key findings
    • Artists – key findings
  • Case studies of selected projects to illustrate key findings.

Model of Practice

– a framework explaining the ‘unifying rationale’ for the program as an approach to integrating arts practice into other sectors.

Contents to include:

  • Introduction to a ‘Model of Practice’
  • Identifying the elements of the model
  • Strengths of this model
  • Barriers, constraints and operational Issues
  • Opportunities for further development
  • Research insights
  • Case studies of approaches to collaborative arts practice drawn from projects.

The final Artistic Report and Model of Practice completed by Ollie Black and Christine Putland in August 2015 can be viewed here Artistic Report and Model of Practice


Interim Summary of Evaluation activity (Jan – July 2014) and emerging findings

Management

Meetings with partner representatives have occurred on a needs basis when issues have arisen or feedback on a particular project is sought. Another round of face-to-face meetings commences in August to clarify and confirm the evaluation questions in the final phase of the program. The importance of this has become increasingly clear to ensure that the focus of the evaluation feedback aligns with the expected outcomes of respective partners. As partners progressively observe the emerging feedback and outcomes they are able to be more specific about the kinds of information they require in relation to their organisational goals. A tool for communicating how the evaluation relates to the program plan has been developed to assist in clarifying this process.

Organisations

At this stage in the program there is now a large number of staff in the partner organisations who have had interaction with artists and have been able to observe the project outcomes on the ground. A survey of these staff is being conducted and so far 24 responses have been received (about half of sample). A majority of these respondents scored their experience of the project and their communication with the artists very highly (that is, 8, 9, or 10 out of 10). Where members of the community have been involved, most staff indicated that they had engaged effectively, responded positively and gained benefits from participating. A good number said staff in their organisation had been introduced to new ways of learning, while a vast majority agreed that their organisation as a whole had benefited and they recognised the potential value of the arts. Another indicator of this positive response is where staff in some partner organisations have been actively seeking the artists’ involvement in ‘extension projects’ (for example, with different groups or in other services within the organisation).

Artists

The shift to the Art House, as a tangible presence within a health service site, signals a maturing of the program. While this has not been without its challenges, some strong relationships with health staff have been formed as a result. One of the key components of the evaluation plan was to facilitate the development of skills and confidence among the core group of artists to evaluate their own practice and projects. The main mechanism has been regular meetings and workshops supplemented by individual support to devise and implement data collection tools. This process has been hampered by some changes in the composition of the core group, the involvement of a wider group in one-off projects, and a hiatus in terms of regularity of meetings. At the same time those involved from the start have become more independent in developing their own tools and collecting data. A framework to help artists to report on this activity was developed and a summary of data collected for each project is being progressively updated for transparent monitoring of this aspect.

Projects

As data from completed projects or stages of a project become available these are written up in the form of case summaries to support program learning. Feedback collected from 68 participants in the ‘Long Lunch’, for example, provides not only a very vivid picture of the volunteers’ roles in protecting the local environment, but also shows how the arts-based event has enriched their experience and facilitated their reflections. A summary of the ‘Laughing Yourself Well’ project in Mobilong prison has informed core funding applications by the Murray Mallee Health Service, leading to clarification of partner expectations and refining how artists work alongside health workers. The summary of year 1 of ‘Drawing on Country’ has identified opportunities for improvement and helped to shape Year 2 in 2014.

Matthew Bennett


Leave a Reply

Your email address will not be published. Required fields are marked *

When

2011 - 2014

Type

Artist Residencies

Presenter/Partners

Country Arts SA in Partnership with Australia Council for the Arts, Alexandrina Council, Government of South Australia and Medicare Local

Location

Alexandrina Region

Country Arts SA recognises and respects that we are living and creating on Aboriginal Lands and we are committed to working together to honour their living cultures.