The Solva Care Toolkit

11 – Solva Care: research, monitoring and evaluation

It important to build in research and evaluation plans at the very start of your project, and if you are a grant holder, your funder may require you to do so at the application stage.  Given the importance of research and evaluation, it may seem strange that we are covering the topic at the end of our sequence of chapters, but we are covering it now because as Solva Care developed and changed, so did our thinking about how we should measure our success in the short and long term.  This is an ongoing challenge!

Monitoring

Monitoring involves collecting management data on your activities; a task often required by funders, as proof that you are spending your grant monies in line with your application.  Our Coordinator Lena has described our monitoring data more fully in an earlier chapter.  These data are reported quarterly to: our funders, to an agreed set of ‘indicators’, our Trustees, and to our wider community (see point 3 for examples of these indicators under the research and evaluation section in this chapter).  Initially we devised indicators for each grant because funding was for different aspects of our activities, but we are currently developing a set of ‘core’ indicators that cover all types of activities and extend to the impacts that we are aiming to achieve.

We have learned to make monitoring data useful for our purposes too, but they do not tell us whether we are meeting the current needs of our community, how effective we are in meeting our aims and objectives, nor do they convey anything about the quality of what we do.  For that, research and evaluation are needed.

Why we undertake research and evaluation

1. To find out what everyone in our community wants in care and wellbeing, and how to best work together to meet those needs.
2. To enable us to find out what works and what doesn’t, and to improve.
3. To make fundraising faster and easier.
4. To enable us to share our way of working with others

Research and evaluation

Solva Care is a small local care and wellbeing endeavour run largely by volunteers with a few paid workers.  Its strategy has grown and is now made up of many innovative components focusing on individuals, groups and the community as a whole.  The line between those that give and receive support is blurred, and some of our objectives are met by partnerships with other agencies.  All of these factors make Solva very complex and thus pose challenges for research and evaluation.  There is no template, as yet, that we can follow.

The following were our original research and evaluation questions, and we have added in italics how we – or others – set about answering them.

1. What are/were the barriers and enablers to the setting up and running of the full Solva Care Service?

We were one of the social enterprise cases researched by Cardiff University on start-ups.  Members of our Board of Trustees were interviewed, and meetings observed.

2. How is Solva Care resourced, and do the resources match requirements?

Regularly reviewed by Solva Care Finance Group.

3. What are the outputs of the voluntary service in terms of: coordination, the number of volunteers involved, the number of volunteers trained, type and amount of training received, number of hours used to help people, type of help provided, and number and type of referrals?

Monitoring data collected by our Coordinator.

4. Does Solva Care communicate effectively?

All informal suggestions by community are considered and Care Messenger technology trialled.

5. Are the needs of the users of the Voluntary Service being met effectively, compassionately and in a timely way?

We have researched needs using two community surveys – two years apart – by self-completed questionnaire.  Our activities are based, where available, on evidence of what works e.g. arts in enhancing wellbeing.  A focus group was run by Swansea University; feedback is sought from individuals by our Coordinator and volunteers, and; structured feedback by group discussion follows the last Friday Club session of each term.

6. Has the experience of the Coordinator and volunteers been positive, and in what way? Have there been gains for them on a personal level and, if so, what are those gains?

Structured reflection at meetings and an annual away day.

7. In terms of impact, has the voluntary service and full Solva Care service changed the number and types of visits to the GP surgery, and reduced hospital admissions and late discharge?
8. Is Solva Care cost effective?
9. Has Solva Care improved the self-reported wellbeing of those involved?

Aspects of 7, 8 and 9 explored, as part of feasibility study by Swansea University, on the measurement of impact using standardised questionnaires and a focus group discussion.  Solva Care has concluded from that study that qualitative methods are the most useful in generating meaningful data locally.  We are compiling anonymised case studies and are planning to use the Most Significant Change technique to look at the factors that help or hinder success and sustainability.  Question 7 frames success in relation to the agenda of other organisations and will require a joint, ethically approved study with our local health centre. 

10. Has Solva Care added value compared with other community initiatives, and if so, in what way(s)?
11. What are the characteristics of Solva’s community that has enabled it to progress the initiative (and now we are three years old, how can it be sustained)?
12. Does Solva Care network with key groups and organisations and what, if anything, has been the outcome of networking?

For 10, 11 and 12, grants have been sought with academic partners for a large-scale comparative research and evaluation study.

13. Was the business plan appropriate and effective?

The plan is developing and changing in accordance with needs, so is regularly reviewed internally.

Our website hosts research and evaluation reports that we, and other bodies, have produced.  The Toolkit Resources Page has tools that can be downloaded, such as our community questionnaire.

Some tips:

• Make sure your project objectives are amenable to research and evaluation.
• There is a lot of research out there that will enable you to plan your activities, with the likelihood that they will succeed; find and use such evidence.
• If you are basing your activities on existing evidence, a light-touch evaluation may suffice – e.g. did it meet expectations? Was it well run? How did it make people feel?
• Collect baseline data and build in ongoing evaluation into your project and your day-to-day activities, but avoid overkill.
• Research and evaluation by external people can be very useful as they are experts and are often seen as being independent and objective. Their findings, therefore, often carry weight.
• Beware! The lead-in time for securing dedicated research grants with academic partners can be very lengthy and the process very competitive.
• Involving independent people does not mean that your team should have no say or involvement. It is essential that you are involved in ALL stages of the research process.  Allow time for building trust and understanding.
• The community will be interested in participating and hearing about any research findings, but communication should always be tailored to the audience.
• Some members of your community will be keen to join a research working group. All will have a wealth of relevant experience and knowledge to contribute and if they haven’t the skills already, they will be keen to learn.  After all, it’s not rocket science!

The Solva Care RME Team