Thursday 19th September 2013
By Abi Delderfield, Macmillan Quality and Improvement Lead.
This year, for the first time, Macmillan’s Cancer Information Development team has started to evaluate the impact of the charity’s information for people affected by cancer.
Anna Leibowitz, Head of Cancer Information Development, explains why: ‘We know our patient information is popular and we have plenty of anecdotal evidence to show it really helps people. But we wanted to find out more about its actual impact, and to see what people did next after reading it.’
What we did
Working with Shared Intelligence, an evaluation agency, we devised a short survey about seven of our most popular titles: Help with the cost of cancer, Understanding chemotherapy, Coping with fatigue, Understanding breast cancer, Understanding lung cancer, Understanding colon cancer and Understanding early (localised) prostate cancer.
We asked people how they felt immediately after reading the information, and whether they would recommend it.
We used the same outcomes as those used for the new evaluation framework for Macmillan’s cancer information and support services (see below).
We wrapped copies of the survey around three months’ supply of each of the printed booklets (a total of 60,000 copies) and linked to an online survey from relevant web pages.
What we asked
After reading the information, we asked people to rate these outcomes on a scale of 1-5:
- I have all the information about the topic that I need at this time
- I have a better understanding of the topic
- I feel better able to make the right decisions for me
- I feel less anxious
- I know what to do next
- I feel more in control
Overall, we had a positive response, with 762 surveys completed. Although this is still a relatively small data set, Figure 1 shows the split of survey respondents.
Our next step was to ask people more detailed questions about the medium and longer-term outcomes of reading the information. We held 70 telephone interviews with respondents between one and three months after we received their initial feedback.
The average result across all outcomes was 4 out of 5, and we’ve included some of the highlights here.
Accessing the information
The majority of people accessed the information at the right time for them, but about a fifth received it too late. It was also interesting to see that people tend to access cancer-specific information online and more generic information in booklet form, and we want to explore this further.
Overall, people said they used our resources to find out general information about their situation, often to prepare for a first meeting with a cancer specialist.
Impact of the information
In the short term, the biggest impact of our information on the respondents affected by cancer was an improved understanding of their condition.
We found that, although our print and online content is the same, booklets generally had a greater impact across all outcomes than online information.
After reading information about benefits and financial help, 26% of respondents applied for benefits soon after, and a further 40% intended to apply in the future. See Figure 2.
We also found out that 21% of respondents saw a health or social care professional for more information after reading a resource, and 37% intended to do so.
The area where our booklets were shown to have the least impact was on people feeling less anxious and alone as a result of reading the information.
Although it is difficult for a piece of information to significantly reduce anxiety in many circumstances, it was interesting to see that these feelings were often substantially reduced in the longer-term analysis two-three months later.
The point of this evaluation is to identify ways we can increase the impact of our content. So, as the results show our information has a low impact on reducing anxiety, we will look at what more we can do within our information to reduce anxiety and increase a sense of control.
We’d like to build on this foundation and do further evaluation to find out more about how our information is used and to ensure we are producing useful resources. We’re now planning a second phase of evaluation later this year and you’ll see more information about this soon.
Please continue to encourage people to complete the surveys and to attend focus groups. If you’d like to read the full report, please email the Cancer Information Development team.
What do you think of Mac Voice online?
Please let us know your thoughts by taking part in this short survey.