Versions Compared

Version Old Version 32 New Version Current
Changes made by

Nicole Mc Neilly

Nicole Mc Neilly

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Panel
bgColor#C0B6F2

Intended Learning outcomes

This page is designed to help you:

  • Understand in detail when to apply certain methods

  • Understand the pros and cons of commonly-used methods

  • Explore new ways of collecting data

...

Panel
bgColor#DEEBFF

Primary and secondary data

Is the data collected by your organisation for this purpose only? Then it’s primary data. Secondary data describes data that has been collected by another organisation or for a different purpose. 

Don’t rule out looking at secondary data. It can be a great starting point for your research as the hard work has already been done by others. These data can work as a guide for you in many ways. For example, you can use the data but also learn from their approach. What does their baseline say? Can these data act as a baseline? Can you collect data in the same way, and then compare?

The methods described below are qualitative.

1. Interviews

Expand
titleExpand for more information

What: Face-to-face, phone or online conversations to gather deep insights, first-hand experiences or expert opinions

When: If you are interested in gaining more context and understanding of your stakeholders (e.g. to better interpret quantitative data)

Advantages

  • In-depth and often rich answers

  • Also open to people with low literacy skills

  • Opportunity to explore unknown or sensitive issues in more depth

  • Generally people are willing to participate

  • Flexibility in the questions you pose

Disadvantages

  • Time consuming to schedule, conduct and transcribe

  • Need for trained and confident interviewers

  • Possible interviewer bias (learn about interview bias and how to avoid this)

  • Sensitive issues can be challenging for the interviewer and interviewee

  • Difficult to compare between interviewees

  • Small samples

  • Challenge of interviewing those who speak different languages

See our tips on interview etiquette.

Interview etiquette

...

Expand
titleExpand for more information

Read our case study on Outcome Harvesting and how we used it in an impact assessment of EuropeanaTech and the implementation of IIIF.

Phase two case study: Outcome Harvesting, EuropeanaTech and IIIF

The methods described below use mixed methods.

1. Questionnaires

Expand
titleExpand for more information

What: An online or paper form that allows you to ask the same questions to a large number of people

When: If you want to ask a lot of people the same easy-to-answer questions to effectively compare experiences and get a good high-level overview of what change occurred for them

Advantages

  • Low cost or free

  • Large samples are possible

  • Wide reach, can be easily shared

  • Can be anonymous

  • Easy to analyse

  • Can be multilingual

  • Can be used to identify and recruit interview participants

  • No transcription needed

  • Can be conducted on different platforms and at different scales, e.g. during registration for an event, social media, pop-ups on your website or digital exhibitions, at events (e.g. in online polls or interactive slides), online, etc

Disadvantages

  • Bias due to self-reporting

  • Low response rate

  • Inflexible

  • Response can lack context

  • Responses may not be complete or valid

  • Can be very long

  • Time consuming to analyse large amounts of open text question responses

See our tips on writing a good questionnaire.

Writing a good questionnaire

...

Expand
titleExpand for more information

What: analysing engagement with your social media over time to understand responses to your digital offer (promoted or displayed on social media) or how our online communities are engaging with each other. This could include using the comments tool on Facebook or analysing Twitter or Instagram replies. Some tools help you scrape these comments and the text can be analysed manually or automatically (e.g. using sentiment analysis)

When: If you have a digital or other campaign with the aim to increase awareness of your organisation, a certain topic or event, or if you want to understand your audience’s reaction with an online or physical project or activity

Advantages

  • Easy to assess if a baseline is collected

  • Useful when used in triangulation with other data sources

  • Opportunity to analyse qualitative data

  • ‘Natural’ setting where participants do not feel like they are being observed for a specific purpose

  • Tools like sentiment analysis can give rudimentary insight into qualitative responses

  • Tools exist to help scrape social media comments from websites into more workable formats e.g. Excel spreadsheets

Disadvantages

  • Likely to contribute outputs rather than outcomes if numbers alone are used

  • Need to be clear about ethics and data protection, including anonymising reporting what you have learned

  • Qualitative analysis of comments, etc, takes time

The methods described below are quantitative.

1. Digital user statistics

...

Expand
titleExpand for more information

We have a dedicated page to ways of assessing your environmental impact.

Assessing environmental impact
Tip

Checklist - have you chosen the right method(s) based on:

  • The scale of your research and available resources and time?

  • The sample you want?

  • The moments (data points) at which you interact with the person or group you are surveying (data subject)?

  • Your own experience and confidence in analysing the data?

  • How you are going to use and learn from the data?

  • What the ultimate reader of the findings is going to want or need?

Panel
bgColor#B3F5FF

How do you choose the right method? Some resources.

  • Check out Social Impact Navigator’s guide to data collection methods, with a table showing which method works best in different circumstances.

  • You can also use BetterEvaluation’s rainbow checklist. This takes you through a number of steps to identify appropriate methods for your case.

...

Finally - go back to your data collection plan

It’s now time for you to complete your data collection plan! You’ve identified the methods you’ll want to use. Go back to the page that outlines the data collection plan format and ensure that you’ve ticked everything off the list.

...

Next steps