Understanding the Charity Brand Evaluator

 

This page provides a guide to understand the data produced from our Charity Brand Evaluator. Linked to your online dashboard, it unpacks each measure and analytical tool to provide more context about our research and how it can be understood and applied.

If you have any questions about the data beyond the guide, please contact your Account Manager or email: cbe@nfpsynergy.net

Methodology

Sample - The sample for this research is 1,000 members of the public who are aware of the charity and feel confident in discussing it (750 for the pilot wave). We ask two questions to identify this group. We first ask Please indicate whether you have heard of the following charities or organisations.” If a respondent says ‘Yes’ to a particular charity they are then asked the next question

The second question is “You previously indicated that you were aware of each of the following charities. For each one, please choose the option that best applies to your knowledge of the organisation.”, with answer categories:

  • I’ve heard of it, but I don’t know anything about it
  • I know a little about it
  • I know quite a lot about it
  • I know it very well

We then filter out the respondents that say I’ve heard of it, but I don’t know anything about it’we do not ask any further questions about that particular charity to that respondent. We keep in those individuals who are aware of the charity but may have slightly less knowledge about the charity to ensure a range of respondents who have different levels of engagement with it which can impact brand perception.

Fieldwork - We carry out the research in Spring and Autumn of each year, using an online questionnaire that can be taken on a mobile, tablet or laptop/computer.

Brand Metrics 

Brand emotional responses

Donor decisions often tend to be quick, emotional choices, rather than deliberative decisions. We measure emotional responses in order to better understand this immediate instinctive reaction to your charity.

To understand how we could measure emotions we turned to the Circumplex Model of Feelings, an academic model developed by James Russell. This model suggests that emotions can be described by two axes – “arousal”, or how activating and energising an emotion is, and “valence” or how positive or negative an emotion is. Thus emotions can be positively activating (excitement, joy), positively deactivating (relaxation, calmness), negatively deactivating (depression, boredom) or negatively activating (anger, fear).

We used this lens to create two questions. The first question is: “We would like you to think about your feelings about [Insert charity]. Please click or drag up three options into a rank position you think best describes your emotion when thinking about it”. Respondents are able to rank up to three emotions of a choice of twelve – Frustrated, Angry, Afraid, Excited, Happy, Astonished, Sad, Bored, Depressed, Satisfied, Pleased and Relaxed’.

The second question is when the respondent is shown the image above illustrating the Circumplex model of emotion. We then ask “Please select any area (s) that you feel you identify with when thinking about [Charity name]”

We can then use these responses to create a visual representation of how respondents are reacting to your brand. These heatmaps are not currently "live" - they do not update when the data is filtered - and are therefore not currently available on your dashboard. This is a feature we are looking to implement soon, but in the meantime heatmaps are available in your presentation and on request for any different subgroups of interest.

Transparency

This measure aims to help you understand whether your charity is seen as clear and accountable as an organisation. Performing strongly in this metric can demonstrate a higher level of trust from the public in that work and conduct of that particular charity. 

This measure consists of three statements, asking the respondent “To what extent do you agree or disagree with the following statements". The three statements are:

  • “[Charity] is clear about how it spends its money”
  • “I believe [Charity] operates transparently”
  • “I believe [Charity] does what it promises to do”

Differentiation

This measure aims to help you understand whether your charity is seen by the public as unique in the work that it does or the sector it is in. Often charities can be confused between others that work within a similar topic or have similar names. These questions help understand whether this is common issue for a given charity. Performing strongly in this metric can demonstrate a charity brand’s ability to be seen as unique in the work it delivers and/or have a strong identity.

This measure consists of two statements, asking the respondent “To what extent do you agree or disagree with the following statements". The two statements are:

  • “I would never confuse [Charity] with any other organisation"
  • “No one else does what [Charity] does”

Momentum

This measure aims to help you understand the extent to which the public perceive you to be an active and dynamic organisation. It explores whether the public perceive you to be an energetic organisation, adding value and creating change within the work that you carry out.

This measure consists of three questions. The first two are statements, asking the respondent “To what extent do you agree or disagree with the following statements" with a choice of Strongly Agree, Agree, Neither agree or disagree, Disagree, Strongly Disagree. The two statements are:

  • “[Charity] is moving in the right direction”
  • “[Charity] is a dynamic/energetic organisation”

The third question is a dial from 0-100. We ask the public "Please use the scale below to indicate how dynamic/energetic you feel [Charity] is? The scale is 0 to 100. If a charity is 0, it is not dynamic/energetic at all, if it is 100, it is extremely dynamic/energetic." The respondent then moves the dial to where they feel the most appropriate number is. 

Authority

This measure explores how much your charity is seen as an authority in the charity sector, as well as the area that you focus on. This is about being an authoritative voice on the topic but also a reliable source of information for people to come to. We provide the option for you to choose the topic we prompt, whether it be something you are well known for or something that your charity is looking to move into/ expand.

We ask three questions for this measure. The first is a broader question about authority in the charity sector more widely. 

  • "To what extent do you agree that [Charity] is a well respected organisation?"

The second two questions refer to the charity's role in the specific area of work they have identified. They both have a scale of Strongly Agree, Agree, Neither agree nor disagree, Disagree, Strongly Disagree. They are

  • "[Charity] is the place to go when you need information on [Topic chosen by charity]"
  • "[Charity] is the authority on [Topic chosen by charity]"

Relevance

The relevance metric examines the extent to which you are seen as a relevant organisation by the respondent - both in terms of working on an important issue, and in terms of personal relevance to that individual. It is made up of three statements to which respondents can agree or disagree:

  • "[Charity] is an organisation that meets a genuine need"
  • "[Charity] is more relevant today than ever"
  • "[Charity] is relevant to me as an organisation"

Recommendation 

This measure explores the extent to which those who have heard of you engage with your work and are willing to share their knowledge and interest in you with their wider network. This information helps provide context about those who are aware of you and are confident in talking about you - does this awareness transfer to recommendation and a creation of wider interest with their connections?  As well as previous recommendations, these questions also cover likelihood in the future to provide recommendations for your charity.

There are two questions to this metric. They are: 

  • "Have you recommended [Charity] to family, friends or colleagues?" - Yes/No

  • "How likely is it that you would recommend supporting [Charity] to family, friends or colleagues in the future?" -Scale 0-10 with 0 labelled 'Not at all likely', 5 labelled 'Neutral' and 10 labelled 'Extremely likely'.

Visibility

This measure helps identify how often the sample group hear about you on a daily basis. This covers the different platform where your messages may be shared, whether that be on social media, printed media or word of mouth. It also addresses a more general perspective about how a growing presence that your charity may have in a respondent's world. 

This measure has three questions. The first two are statements, asking the respondent “To what extent do you agree or disagree with the following statements" with a choice of Strongly Agree, Agree, Neither agree or disagree, Disagree, Strongly Disagree. The two statements are:

  • "[Charity] is an organisation that I hear more and more about these days"
  • "I've had conversations about [Charity] recently with friends or family"

The third question is about frequency of hearing about your charity. It asks:

  • "How often do you hear about [Charity], whether through advertising, online or social media or from friends/colleagues?" 
    5 point scale with 1 at Never and 5 Very frequently 

Loyalty

This measure specifically drills down to those in the sample who self-identify as existing supporters of your charity. This provides a slightly smaller base but also an insight into the different types of supporter you have and how they see their relationship with the charity.

This measure has three questions. The first is:

  • "What type of supporter do you consider yourself to be:

1. A devoted life-long supporter – they are one of my favourite charities
2. An enthusiastic supporter – I will encourage anyone who’ll listen to support them as well
3. An occasional supporter – they are one of many charities I give to
4. A critical friend – I support them but I think they could do better

As well as providing data on the different types of supports your charity has, this question can also be used as a break in other questions - for example, understanding what attributes a 'devoted life-long supporter' would choose to describe your charity, compared to an 'occasional supporter'

The second questions are two statements, asking the respondent “To what extent do you agree or disagree with the following statements" with a choice of Strongly Agree, Agree, Neither agree or disagree, Disagree, Strongly Disagree. The two statements are:

  • “[Charity] would be the last charity I would stop supporting”
  • “I am proud to call myself a supporter of [Charity]”

Impact

This measure explores the extent your charity is perceived to have created impact in the work that it has done. Demonstrating impact is a critical way of engaging the public as often if the public feel that improvement has happened in the past, they are more likely to be confident that a charity can make an impact in the future. 

This measure consists of two questions, both statements, asking the respondent “To what extent do you agree or disagree with the following statements". The two statements are:

  • "The world would be a worse place without [Charity]"
  • "[Charity] makes a real difference in the world"
  • "[Charity] spends its money wisely"

Verbatim 

For one of the metrics you choose, you can also choose to ask an open ended verbatim question to respondents to explain their reasoning behind their answers. To do this, after a respondent have answered the standard questions for that measure, we then prompt then on their screen stating : These are your answers to the previous statements about [measure] for [Charity]. Why did you choose these answers?

The respondent is then provided with a open ended box to type in their answer in their own words.

Benchmarking

For each of the measures above, we create an index out of 100 that combines individual statements to give one figure for that metric. This allows you to easily benchmark your performance on, for instance Relevance, over time and against other organisations with just one figure. Although the exact method of calculating the benchmark indices varies according to the different statements or questions that feed into that metric, all indexes have a nominal maximum of 100.

Drivers of Support and Brand Analysis

Brand Attributes

  • We ask three questions to understand what different attributes are identified by the public to describe the ideal charity, ideal charity in different sectors and for specific charities.
  • These are:
     

1. “Listed below are a number of words that could be used to describe a charity or not for profit organisation. Please choose up to 7 words that you think describe your IDEAL charity.”

1. “Listed below are a number of words that could be used to describe a charity or not for profit organisation.Please choose up to 7 words that you think describe your IDEAL charity working in [sector] listed below.”

3. “Please choose up to 7 words that you think best describe [charity name].”

Brand Preference

The Brand Preference measure is one of overall affinity towards a brand. We ask the public:

“Please indicate how much you like each of the following charities on the scale below”

On a scale from 0-10, respondents are asked to identify where their preference is positioned. 0 = I strongly dislike this charity, 5= I am not particularly interested in this charity, 10 = This is one of my favourite charities.

This measures overall warmth towards a brand and allows you too see how favourably or unfavourably you are seen, both overall, and among specific subgroups. It is also used for the purposes of driver analysis to identify the most important metrics and attributes for your brand.

Driver Analysis

Driver analysis is a statistical technique which explores the relationship between a set of potential driving factors and an outcome variable.

In this case, we the relationship between what attributes a respondent associates with a brand, as well as how they rate the brand on different metrics, on respondent’s preference towards that brand.

The meaning of ‘% of the variance of “Charity”s brand preference among this is group’ is the proportion of a charity’s brand preference can be explained by this driver analysis. Other factors besides these attributes can also of course explain why a person prefers one brand over another - demographics, personal experience, or a whole range of other potential reasons can account for the rest of the variance.