Before you choose the method that you’ll use to collect data, you need to think about the size of population you are studying (e.g. the group of visitors to your museum or online exhibition), the characteristics of that population and the amount of data (the sample) that will be representative enough so you can report what you learn with confidence.

Where do I start?

While this might seem like a challenging part of Phase two - and it’s not easy - it can be broken into several key components.

1. Define the population.

Population = the whole group at the centre of the research question.

Who are you trying to learn from? This is your population. You’ve already done this in Phase one - you should know who your key stakeholder(s) are. Ask yourself: can - or should - you survey the whole population?

If you are interested in feedback from the general public who might visit your website, this is a very big population. For example, we can include all Europeana Network Association (ENA) members in our population, We don’t include the wider heritage workforce in Europe who are not members, for example.

However, we’re only interested in a segment of the Network membership - educators - this becomes our target population. This is a much smaller population.

What are the characteristics of the group? Is your target population homogenous (similar) or heterogenous (different)? You can ahead and ask yourself whether you will collect and segment your data based on characteristics like agenda, gender or location. If you have a diverse and heterogenous group, this has implications for your sample.

2. Think about validity.

There are two types of validity. They raise different questions that help you determine your sample, chose your method and shape your data collection plan.

Internal validity - to what extent are the outcomes you are observing actually the result of your activity, rather than other external influences (variables)? You can think about things like causality and attribution. We can increase our internal validity by controlling other variables. For example, if we were to survey the Europeana Network Association (ENA) membership, we might consider focussing our study on those who are not involved in European-projects where Europeana is a partner. This might explain why this person counts themselves as ‘active’ in the Network, though there may be other factors too.

External validity - to what extent can your findings can be applied to the real world and to other settings (e.g. if you would repeat the study)? This is the issue of generalisability, or ‘transferability’ in qualitative studies. You can generalise results normally for other groups (or samples) who share the same characteristics. For example, when surveying ENA members, we might try to compare results across different clusters to see to what extent the results are generalisable (i.e. that it is true for all groups and not just one specific sample).

See more in thishttps://www.scribbr.com/methodology/experimental-design/.

3. Agree your confidence interval and confidence level.

Looking back at external validity above, it’s important now to think about your confidence interval. If you were to repeat the same study, what scale of difference would there be if you compared the results? For example, would they be within five above or five below? Ten? This is your confidence interval. Five below would indicate a 95% confidence level. Ten below would indicate a 10% confidence level.

You should ask yourself what confidence interval you would expect and/or find acceptable. You can use this both to report on your findings and, even more importantly, plan your sampling approach. If you are happy that the results are not easily generalisable, then you can proceed with a lower confidence level.

95% is a fairly common and accepted confidence level.

4. Calculate your sample.

Sample = a small part of the whole population intended to show what the group is like or experiences

Sampling = gathering information or data from a subset of a larger population, rather than from everyone

You now need to work out how you will sample the (target) population/stakeholder. This is based on an understanding that you can’t hear from everyone, so you have to try to get a sample that is representative enough so that you can report confidently on what you have learned. How can the sample be representative of the whole population?

Normally people say that 10% is a good sample, as long as the 10% is a homogeneous population (that is to say, a group made of people with the same backgrounds, experience, for example). This is likely to be the case if you focus on a target population, but less likely if you focus on a general population. With more heterogeneous groups you need a bigger sample size.

The sample you need will define what method you use, and each method has different considerations for agreeing your sample. Below we think about how you can work out the sample you need based on two of the most commonly used data collection methods.

Why might we sample a population instead of surveying the whole group/population?

• Cost efficiency: collecting data from an entire population can be expensive and time-consuming. Sampling helps you to collect enough information with fewer resources.

• Time efficiency: it may be impractical or impossible to collect data from a whole population, especially when the population is large or constantly changing. Sampling allows researchers to obtain results more quickly.

• Feasibility: in some cases, it may be impossible to study an entire population due to logistical constraints

• Accuracy: good sampling can provide accurate estimates of population experiences. Statistical methods are used to make inferences from the sample to the population, and these methods are well-established and reliable. (See more below)

• Fewer errors: sampling reduces the chances of errors that can occur when trying to collect data from every member of a (large) population.

• Ethical considerations: in some situations, it may be ethically or practically inappropriate to collect data from every member of a population.

• Generalisability: if a sample is chosen correctly and represents the population well, the results from the sample can be generalised to the entire population, meaning that researchers can draw conclusions about a larger group based on the sampled data.

• Variability management: knowing more about the characteristics of a sample and the variables that may effect their experiences helps to you manage the variability that may exist within a population.

Types of sampling

There are four main types of probability sampling used in quantitative research.

1. Simple random sampling

Randomly generate a list of the people who you will survey out of a bigger population. For example, out of 4,000 Europeana Network Association (ENA) members, we might use a random sampling tool to decide who we will survey instead of surveying the whole membership.

2. Stratified sampling

Different groups with the same characteristics in one population are divided into separate groups or ‘strata’ (the target population). Then these groups are randomly sampled. For example, we add all educators in ENA into one group and then randomly select members to survey, and do the same for researchers.

3. Cluster sampling

Somewhat the same as stratified sampling, but the whole population is broken up into clusters (randomly selected). All, or a certain number, of these groups are then randomly sampled and you can compare the results. For example, we group all ENA members into ten clusters, and we randomly sample members in five of these ten groups.

4. Systematic sampling

Select members of the population at a regular interval that you agree in advance. For example, we would survey every 10th person in a list of ENA members arranged alphabetically.

Next step

• Step 3. Choose your methods