Step 1. The data collection plan
Intended Learning outcomes
This page is designed to help you:
Understand what the components of a data collection plan are and how it will help you collect the data you need
In Phase one you will have emerged with a completed change pathway.
You might have used Word or Excel (or something else) to map out each change pathway for your stakeholder, like this.
Or this example, which uses a slide deck.
It’s now time to extend your change pathway to document the indicators that you will use to measure change, how you will measure them, and when and from whom you’ll collect the data. This will give you a data collection plan.
Create a data collection plan
The steps below relate to the development of your data collection plan and not phase two more generally.
You’ll complete different parts of your data collection plan over the course of Phase two!
1. Agree the scale of your impact assessment
“If your budget is small, or you’re still at the very beginning of the project, it’s best to start with small, straightforward measures that you can expand little by little.”
It’s good to be ambitious (you are - you’re already here!) but ensure that what you invest in the data collection is balanced with the quality of data you’ll get in return and the scale of the project. You can always build up to a more in-depth approach at a later point. How much money and time you have is also important. How much of both are you able to invest? How much time do your team or colleagues have to support you? This will change and influence your approach. You might also want to do a quick pilot or test before launching anything large scale.
2. Create your indicators
See the dedicated page on how to work with indicators.
3. Set your ideal sample size and response rate
The data that you collect should be representative and robust. You need to know how many responses you need to draw valid conclusions about those whom you are surveying.
Europeana Network Association impact assessments - sample size
In 2020 and 2021 we shared impact assessment and satisfaction questionnaires with the Europeana Network Association membership. The questionnaire was sent to the whole group (over 3,000). We wanted a minimum of a 10% response rate to give us a valid insight into Network membership, and we achieved this in both 2020 and 2021. We know, however, that the bigger the sample size, the stronger the findings.
We assessed the responding sample against other data (e.g. what we know about length of membership, experience in the sector, location) to understand to what extent this was reflective of the full group of members. Read the impact assessments on Europeana Pro.
The sample size you need will influence how you collect data. You might not expect a high response rate for a survey, so you could instead interview a smaller sample of stakeholders. This might also give you richer insights, but it might not fulfil the desires of your Board, for example, if they want easily shareable infographics about their impact.
4. Think about your baseline
If you know how you will collect your data you now need to think about the design of your data collection.
Can you collect and then compare a pre- (baseline) and post-situation (endline)?
Do you want to continually monitor the effects of your work (and thus collect data on a continuous basis)?
Are you doing retrospective research (looking back on what happened) focused on a certain time period?
Will you collect data at multiple points after the activity has finished?
5. Think about short-term vs long-term
Asking people questions about an event that took place a year ago may not give you very reliable information if you don’t have data from closer to the time to compare it to. At the same time, asking people questions too soon after an intervention may lead to you missing out on relevant changes that are still to occur. Therefore, you could collect data from stakeholders in the short-term and long-term, if resources allow.
Long-term outcomes can take a long time to appear. You might not have the resources to continue an impact assessment to the point where you can begin to evidence these outcomes. The goal of the Impact Playbook is to empower you to show that you’ve helped create shorter term outcomes that can help lead to this longer-term change and impact, and to give you something that you can learn and improve from.
When we sent longitudinal questionnaires to attendees of our national workshops and Europeana 2019, there was a very low response rate. This might be because the participant thinks that the activity under question is no longer relevant, or that they have other priorities to reflect on than an activity that was so long ago. The risk of low response rates should be borne in mind.
6. Agree what methods you will use
Before going into research for your methods, be aware that it can a time-consuming procedure if you have no experience. Keep in mind that external advice might be good to get at this point.
Take a look at:
Our advice on qualitative, quantitative and mixed methods approaches.
Our overview of different types of commonly-used methods.
7. Find or create your ‘data points’
The best data collection method for your research will depend on when and where you can collect data. How can you reach out to your stakeholders so that you can collect data from them? When and where can you collect this data? Consider:
Where your stakeholders are located
How you can access or contact them
Whether a group or subgroup can be representative of your stakeholder, and what size this group should be
How can you collect data in an unobtrusive way
8. Set realistic timelines
By now you should have a feeling of what is a realistic timescale to set for your data collection and analysis. Some methods are more time-consuming than others, depending on the scope and rigour you have decided on. For example, you may want to focus in more depth on a smaller sample of stakeholders, but that will require resources over a long time period. Thinking through what is a realistic timescale for you and your team will help you set a realistic timeline. It’s also important to be flexible in the longer-run, as things might change or not run according to plan (e.g. a project might be extended or reduced in size).
9. Ensure that you are confident about ethics, data protection and privacy approach and GDPR compliance
Before going any further, ensure that you are up to date with what data protection laws govern your data collection and how you should ethically approach data collection. Read our short guidance by clicking the link below.
You can also read in more detail the section on data protection in Europeana’s Standardised Question Bank.
You’re now ready to create your data collection plan!
We’ve developed a simple ‘extension’ to the change pathway of Phase one to help you do this. For each outcome (short, medium or long-term), you will:
Map the indicator(s) that will help you measure change
Identify the source of where the data comes from
Identify what sample you need to make it valid
Outline how you’ll collect the data (when/where/how)
outcome (you’ll already have these) | indicator(s) of change | source and sample | data collection method and plan |
the changes that have occurred in (or for) your stakeholders that can be directly attributed to your activities (see Phase I for a recap!) | what are the things that show us that the outcome has been achieved? | where can we get the data from (existing data sources or new data needs to be collected)
| surveys, interviews, analysing existing data sources...consider the pros and cons
when and how can we collect this data? |
e.g. pupils engage more in heritage as a result of going to Transcribathon event | e.g. number of pupils who use more digital heritage in homework assignments in the four months after the Transcribathon event | e.g. at least 10% of educators need to participate | e.g. follow-up interviews with educators after Transcribathon event. Challenge: small data sample, need a native speaker trained in qualitative methods |
Download the extension Excel template
Download the extension Word template
