Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Before you choose the method that you’ll use to collect data, you need to think about the size of population you are studying (e.g. the group of visitors to your museum or online exhibition), the characteristics of that population and the amount of data (the sample) that will be representative enough so you can report what you learn with confidence.

...

1. Define the population.

Population = the whole group at the centre of the research question.

Who are you trying to learn from? This is your population. You’ve already done this in Phase one - you should know who your key stakeholder(s) are. Ask yourself: can - or should - you survey the whole population?

...

3. Agree your confidence interval and confidence level.

Looking back at external validity above, it’s important now to think about your confidence intervalConfidence intervals, confidence levels and margins of error: how do we use them?

  • Primarily quantitative research (numbers)

  • Share your perceptions about the representativeness of the sample and robustness of the findings

  • Help others to interpret the data

  • Guide you to use a tool to determine the sample for your research

Confidence means probability. If you were to repeat the same study, what scale of difference would there be if you compared the results? For example, would they be within five above or five below? Ten? This is your confidence interval. Five below would indicate a 95% confidence level. Ten below would indicate a 10% 90% confidence level.

What level of risk or uncertainty am I willing to accept?

You should ask yourself what confidence interval you would expect and/or find acceptable. You can use this both to report on your findings and, even more importantly, plan your sampling approach. If you are happy that the results are not easily generalisable, then you can proceed with a lower confidence level. 95% is a fairly common and accepted confidence level. E.g. you are 95% sure that if you repeated the survey, the results would lie within these parameters.

Higher confidence interval demands a bigger sample. Resource limitations and some degree of uncertainty means that a lower confidence level may be appropriate.

What is a margin of error?

  • Helps you ascertain the sample size

  • Looks similar to the confidence level but different

  • Expressed as

    • ⧮ (or similar)

    • Percentage (e.g. 9%)

    • ± 9

4. Calculate your sample.

Sample = a small part of the whole population intended to show what the group is like or experiences

Sampling = gathering information or data from a subset of a larger population, rather than from everyone

You now need to work out how you will sample the (target) population/stakeholder. This is based on an understanding that you can’t hear from everyone, so you have to try to get a sample that is representative enough so that you can report confidently on what you have learned. How can the sample be representative of the whole population?

...

Panel
bgColor#E3FCEF

Sample sizes in qualitative research

Qualitative methods like interviews result are likely to result in rich, qualitative data. Numbers of responses, or people that you interview, matter less in this context.

For a small population size, you might interview everyone involved. This will give you a very complete insight into the perspectives of the population.

For a big population size, it’s unlikely that you will have time or the money to interview 10% of the population. Rather, you should aim for a smaller sample that is representative of the overall group. If your overall group is very diverse, think about whose perspectives you need most (e.g. educators) and how many you would need to interview to get their perspectives. You should also think about how many people it would take to interview before you start seeing the same patterns or trends - this is called saturation. After this point, there is less value in interviewing more people because you do not learn anything new.

The sample therefore depends on how homogenous (how much it is the same) or heterogenous (how much it is different) your population is. If you have a very heterogenous or diverse population, you need to interview more people. If your population is homogenous, you might need to interview a small group. Therefore there is no fixed right or wrong with interview sample sizes, though some sources suggest that between 10 - 30 interviews are usually sufficient.

The sample will also depend of course on how much time you have available to conduct interviews. You have to be realistic and consider what you want to know and what you can do to get this perspective. If you need more data than you think you can get through interviews, consider combining them with another method, like a survey.

Tips:

  • Be clear on the diversity of the population and your target population.

  • Define clear criteria for who should be interviewed.

  • Be realistic about the time it takes to schedule, prepare for, deliver and transcribe an interview. This might have to inform your sample size.

  • If the sample is big, consider first sending a questionnaire then following up with interviews.

  • Acknowledge the time demands of data collection and analysis.

Tip.

Look
Panel
bgColor#DEEBFF

Tips:

  • Don’t let the complexity stop you from tackling quantitative data analysis

  • Use the principles to define your ideal sample size

  • Use margin of error or sample size calculators - look at different sample size calculators to help you determine what sample size will work for you.

Such tools ask you to consider: 

  • The population size (e.g. everyone who participated in an event); 

  • The sample size that you were able to survey (in terms of numbers or the percentage of respondents); and 

  • Your confidence interval, namely, how confident you are (up to 100%) that the sample that you surveyed has the same attitudes or perspectives as the overall sample (see).

The calculator then works out your margin of error, which you should ideally report with your findings. Based on observation in the non-academic cultural sector, such margins of error are rarely reported.

Examples

...

Expand
titleThere are four main types of sampling used in quantitative research.
  1. Simple random sampling - randomly generate a list of the people who you will survey out of a bigger population.

  2. Stratified sampling - different groups with the same characteristics in one population are divided into separate groups or ‘strata’ (the target population). Then these groups are randomly sampled. 

  3. Cluster sampling - the whole population is broken up into a number of clusters and the results are compared.

  4. Systematic sampling - sampling at a regular interval.

Expand
titleThere are at least four types of sampling used in qualitative research.
  1. Maximum variation - surveying a diversity of the population

  2. Theory-based - defining who you want to sample as new theories emerge from your research

  3. Criterion - selecting people based on a particular criteria relevant to the research

  4. Snowball - those who you survey recommend others

...

Next step