| Panel |
|---|
| panelIconId | 1f3eb |
|---|
panelIcon |
|---|
| :school: | | panelIconText | 🏫 |
|---|
| bgColor | #C0B6F2 |
|---|
|
Intended Learning outcomes This page is designed to help you: Understand in detail when to apply certain methods Understand the pros and cons of commonly-used methods Explore new ways of collecting data
|
...
Below we share a number of commonly-used data collection methods. At the bottom of the page, we also link to separate pages that show methods used to measuring environmental and economic impact.
| Panel |
|---|
| panelIconId | 2139 |
|---|
| panelIcon | :information_source: |
|---|
panelIconText | ℹ️ |
|---|
|
Primary and secondary data Is the data collected by your organisation for this purpose only? Then it’s primary data. Secondary data describes data that has been collected by another organisation or for a different purpose. Don’t rule out looking at secondary data. It can be a great starting point for your research as the hard work has already been done by others. These data can work as a guide for you in many ways. For example, you can use the data but also learn from their approach. What does their baseline say? Can these data act as a baseline? Can you collect data in the same way, and then compare? |
The methods described below are qualitative.
1. Interviews
| Expand |
|---|
| title | Expand for more information |
|---|
|
What: Face-to-face, phone or online conversations to gather deep insights, first-hand experiences or expert opinions When: If you are interested in gaining more context and understanding of your stakeholders (e.g. to better interpret quantitative data) Advantages In-depth and often rich answers Also open to people with low literacy skills Opportunity to explore unknown or sensitive issues in more depth Generally people are willing to participate Flexibility in the questions you pose
| Disadvantages Time consuming to schedule, conduct and transcribe Need for trained and confident interviewers Possible interviewer bias (learn about interview bias and how to avoid this) Sensitive issues can be challenging for the interviewer and interviewee Difficult to compare between interviewees Small samples Challenge of interviewing those who speak different languages
|
See our tips on interview etiquette. Interview etiquette |
...
| Expand |
|---|
| title | Expand for more information |
|---|
|
What: arts-based research methods are generally seen as part of the field of creative methods, where artistic practices are used to help answer research or evaluation questions, or where more commonly-used methods are combined with artistic practices. The artist is likely to be involved at both the research delivery and the analysis stage, and the participants too, including: Story-stems - giving participants the beginning of a story and asking them to complete the story (using words, drawings, creating characters using craft materials, puppets, etc). Writing - documenting through writing, which could include poetry, graphic novels, cartoons, short stories and more. Visualisation - visualising an experience or result in some way including through, for example, drawing, collage, sculpture, collaborative film-making and photography. Performance - making a performance (e.g. through writing or improvising music or a play). Photovoice- an established technique combining photography and accompanying words or text. Creative mapping - asking participants to draw their journey in the form of a map, which can be helpful to understand a user journey and to think about how an experience or emotion has changed over time. This could also be combined with emotion mapping. Crafting - making something practically, e.g. with textiles, knitting, or other types of physical materials. Digital story-telling - a participant (or group of participants working together) shapes a narrative about their experience, often combining words (e.g. through sub-titles and structures), visuals (e.g. through videos or photography) and sound (e.g. spoken description). Creative drawing or writing in response to a prompt - creating an art-work in response to a prompt, and then analysing that art work to understand the meaning(s) behind it, either with or separately from the participant. Scrapbooking or journalling - asking participants to keep a visual or written diary of their experience, which they can later share or report on. Video-journaling - creating vlogs or short videos describing an experience.
When: many situations, for example, when you want to understand people’s physical and emotional interaction in a physical or digital space; when the project results in a creative output Advantages Suits visual thinkers and communicators A creative and fun method that engages people’s memories and sense of experience Valuable with younger participants, and fun for others A fun data collection method for older participants Useful when used in triangulation with other data sources Visual way to tell an impact story Can be more intuitively embedded into (digital) exhibition mediation or guides, educational materials, etc
| Disadvantages Software may be costly or not available Few set or agreed methodological approaches May not be an accepted methodological approach in some scientific or research circles Communities of practice are only just emerging, e.g. the Arts & Evaluation Community of Practice in Canada
|
See our case study from St Fagans Museum of National History, Wales, UK, for more insights into how creative mapping methods were used and combined with more conventional methods like interviews. Phase two case study: storytelling and mapping - creative digital methods |
5. Outcome Harvesting
| Expand |
|---|
| title | Expand for more information |
|---|
|
Read our case study on Outcome Harvesting and how we used it in an impact assessment of EuropeanaTech and the implementation of IIIF. Phase two case study: Outcome Harvesting, EuropeanaTech and IIIF |
The methods described below use mixed methods.
1. Questionnaires
| Expand |
|---|
| title | Expand for more information |
|---|
|
What: An online or paper form that allows you to ask the same questions to a large number of people When: If you want to ask a lot of people the same easy-to-answer questions to effectively compare experiences and get a good high-level overview of what change occurred for them Advantages Low cost or free Large samples are possible Wide reach, can be easily shared Can be anonymous Easy to analyse Can be multilingual Can be used to identify and recruit interview participants No transcription needed Can be conducted on different platforms and at different scales, e.g. during registration for an event, social media, pop-ups on your website or digital exhibitions, at events (e.g. in online polls or interactive slides), online, etc
| Disadvantages Bias due to self-reporting Low response rate Inflexible Response can lack context Responses may not be complete or valid Can be very long Time consuming to analyse large amounts of open text question responses
|
See our tips on writing a good questionnaire. Writing a good questionnaire |
...
| Expand |
|---|
| title | Expand for more information |
|---|
|
What: analysing engagement with your social media over time to understand responses to your digital offer (promoted or displayed on social media) or how our online communities are engaging with each other. This could include using the comments tool on Facebook or analysing Twitter or Instagram replies. Some tools help you scrape these comments and the text can be analysed manually or automatically (e.g. using sentiment analysis) When: If you have a digital or other campaign with the aim to increase awareness of your organisation, a certain topic or event, or if you want to understand your audience’s reaction with an online or physical project or activity Advantages Easy to assess if a baseline is collected Useful when used in triangulation with other data sources Opportunity to analyse qualitative data ‘Natural’ setting where participants do not feel like they are being observed for a specific purpose Tools like sentiment analysis can give rudimentary insight into qualitative responses Tools exist to help scrape social media comments from websites into more workable formats e.g. Excel spreadsheets
| Disadvantages Likely to contribute outputs rather than outcomes if numbers alone are used Need to be clear about ethics and data protection, including anonymising reporting what you have learned Qualitative analysis of comments, etc, takes time
|
|
The methods described below are quantitative.
1. Digital user statistics
...
| Expand |
|---|
| title | Expand for more information |
|---|
|
We have a dedicated page to ways of assessing your environmental impact. Assessing environmental impact |
...
| Tip |
|---|
Checklist - have you chosen the right method(s) based on: The scale of your research and available resources and time? The sample you want? The moments (data points) at which you interact with the person or group you are surveying (data subject)? Your own experience and confidence in analysing the data? How you are going to use and learn from the data? What the ultimate reader of the findings is going to want or need?
|
| Panel |
|---|
|
How do you choose the right method? Some resources. Check out Social Impact Navigator’s guide to data collection methods, with a table showing which method works best in different circumstances. You can also use BetterEvaluation’s rainbow checklist. This takes you through a number of steps to identify appropriate methods for your case.
|
...
Finally - go back to your data collection plan
It’s now time for you to complete your data collection plan! You’ve identified the methods you’ll want to use. Go back to the page that outlines the data collection plan format and ensure that you’ve ticked everything off the list.
...
Next steps
Methods quiz
Step 1. Preparation - get familiar with the The data collection plan
Explore the methodologies in Europeana’s impact assessments - Europeana has published a number of our own impact assessments. In each, we set out our methodology and we’re open about what we learned and the limitations.