Before you choose the method that you’ll use to collect data, you need to think about the size of population you are studying (e.g. the group of visitors to your museum or online exhibition), the characteristics of that population and the amount of data (the sample) that will be representative enough so you can report what you learn with confidence.
...
1. Define the population.
Population = the whole group at the centre of the research question.
Who are you trying to learn from? This is your population. You’ve already done this in Phase one - you should know who your key stakeholder(s) are. Ask yourself: can - or should - you survey the whole population?
If you are interested in feedback from the general public who might visit your website, this is a very big population. For example, we can include all Europeana Network Association (ENA) members in our population, We don’t include the wider heritage workforce in Europe who are not members, for example.
However, we’re only interested in a segment of the Network membership - educators - this becomes our target population. This is a much smaller population.
What are the characteristics of the group? Is your target population homogenous (similar) or heterogenous (different)? You can ahead and ask yourself whether you will collect and segment your data based on characteristics like agenda, gender or location. If you have a diverse and heterogenous group, this has implications for your sample.
...
4. Calculate your sample.
Sample = a small part of the whole population intended to show what the group is like or experiences
Sampling = gathering information or data from a subset of a larger population, rather than from everyone
You now need to work out how you will sample the (target) population/stakeholder. This is based on an understanding that you can’t hear from everyone, so you have to try to get a sample that is representative enough so that you can report confidently on what you have learned. How can the sample be representative of the whole population?
Normally people say that 10% is a good sample, as long as the 10% is a homogeneous population (that is to say, a group made of people with the same backgrounds, experiencesexperience, age, for example). This is likely to be the case if you focus on a target population, but less likely if you focus on a general population. With more heterogeneous groups you need a bigger sample size.
The sample you need will define what method you use, and each method has different considerations for agreeing your sample. Below we think about how you can work out the sample you need based on two of the most commonly used data collection methods.
Why might we sample a population instead of surveying the whole group/population?
Cost efficiency: collecting data from an entire population can be expensive and time-consuming. Sampling helps you to collect enough information with fewer resources.
Time efficiency: it may be impractical or impossible to collect data from a whole population, especially when the population is large or constantly changing. Sampling allows researchers to obtain results more quickly.
Feasibility: in some cases, it may be impossible to study an entire population due to logistical constraints
Accuracy: good sampling can provide accurate estimates of population experiences. Statistical methods are used to make inferences from the sample to the population, and these methods are well-established and reliable. (See more below)
Fewer errors: sampling reduces the chances of errors that can occur when trying to collect data from every member of a (large) population.
Ethical considerations: in some situations, it may be ethically or practically inappropriate to collect data from every member of a population.
Generalisability: if a sample is chosen correctly and represents the population well, the results from the sample can be generalised to the entire population, meaning that researchers can draw conclusions about a larger group based on the sampled data.
Variability management: knowing more about the characteristics of a sample and the variables that may effect their experiences helps to you manage the variability that may exist within a population.
| Panel | ||
|---|---|---|
| ||
Questionnaires 10% is the minimum you should aim for for a representative survey sample. This is the case when you are collecting data up to 1,000 responses. For example, if you only have 600 visitors, try to collect at least 60 responses. After you collect 1,000 responses, no matter how big your population size, you should normally have a good sample. For example, if you have 60,000 visitors, you don’t need to collect 6,000 responses - 1000 should normally give you representative perspective (see more in https://tools4dev.org/resources/how-to-choose-a-sample-size/). |
| Panel | ||
|---|---|---|
| ||
Interviews Interviews collect Sample sizes in qualitative research Qualitative methods like interviews result are likely to result in rich, qualitative data. Numbers of responses, or people that you interview, matter less in this context. For a small population size, you might interview everyone involved. This will give you a very complete insight into the perspectives of the population. For a big population size, it’s unlikely that you will have time or the money to interview 10% of the population. Rather, you should aim for a smaller sample that is representative of the overall group. If your overall group is very diverse, think about whose perspectives you need most (e.g. educators) and how many you would need to interview to get their perspectives. You should also think about how many people it would take to interview before you start seeing the same patterns or trends - this is called saturation. After this point, there is less value in interviewing more people because you do not learn anything new. The sample therefore depends on how homogenous (how much it is the same) or heterogenous (how much it is different) your population is. If you have a very heterogenous or diverse population, you need to interview more people. If your population is homogenous, you might need to interview a small group. Therefore there is no fixed right or wrong with interview sample sizes, though some sources suggest that between 10 - 30 interviews are usually sufficient. The sample will also depend of course on how much time you have available to conduct interviews. You have to be realistic and consider what you want to know and what you can do to get this perspective. If you need more data than you think you can get through interviews, consider combining them with another method, like a survey. Tips:
|
...
Select members of the population at a regular interval that you agree in advance. For example, we would survey every 10th person in a list of ENA members arranged alphabetically.
...