Measuring economic impact

Owned by Nicole Mc Neilly
Last updated: Aug 17, 2023
8 min read

There are a number of methodologies through we can assess the economic impact of activities relating to (digital) cultural heritage.

Intended Learning outcomes

This page is designed to help you:

  • Understand the different types of economic impact assessment methodologies available, as well as their pros and cons

  • Understand how economic impact assessment can be meaningfully applied in the cultural sector

Research from the UK in 2020 has shown that economic impact assessment is most likely to inform external decision-making in the cultural sector, as opposed to internal decision-making. This highlights its importance in advocacy for sustained funding as well as broader reflections on the impact being created through the arts and culture (see for example, Dave O’Brien’s 2012 report on how the cultural sector should use standard economic methods, set out in the UK government’s main policy evaluation guide, to assess their value).

This page is not designed to be used as a guide to conducting economic research. Rather, it outlines pros and cons of the many types of methodological approaches that can be taken. It is up to you to evaluate if you have the skills and experience, as well as the right context, to apply them, or if you need to choose other methods or get outside help.

What is economic impact?

‘A state in which the activities deliver economic benefits to society, stakeholders or to the organisation’. Europeana Impact Playbook.

Measuring economic impact in the cultural sector

Assessing the economic impact of cultural activity has its opportunities and challenges, and consequently, its supporters and detractors. As rhetoric has shifted from the funding or subsidy of culture to ‘investment’, so too has the perspective on its economic impact.

There are few more deeply contested relationships in cultural policy than that between economics and cultural value…failing to adequately value cultural benefits risks an underappreciation of the social value of cultural investments. The challenge is, therefore, to capture the non-market benefits of culture in a way that takes into account the range of values it offers both to ‘users’ and the public as a whole.

Measuring Economic Value in Cultural Institutions. Hasan Bakhshi, Daniel Fujiwara, Ricky Lawton, Susana Mourato and Paul Dolan (2015)

Economic impact occurs when our activities create a positive economic effect, or an economic return on investment, or when an effect can be measured (through the application of a range of methodologies) in monetary terms. In many cases, economic impact is not the main driver of cultural activity. That does not mean that it is not relevant to consider. Economic impact is one of the Impact Playbook’s strategic perspectives.

the cultural sector has generally been hesitant towards traditional economic approaches to valuation. There are valid questions about whether monetary values should ever be applied to culture. However, as discussed, the benefit of attaching monetary value to non-market goods, like culture, is that it increases the likelihood that these values will be considered in economic decision-making.

Measuring Economic Value in Cultural Institutions. Hasan Bakhshi, Daniel Fujiwara, Ricky Lawton, Susana Mourato and Paul Dolan (2015)

Opportunities

  • Economic / statistical information is considered to be easily understood by policy-makers and the general public

  • Methods such as social return on investment (see above) can give a more socially-oriented perspective to economic impact

  • Economic impact can include cost savings (efficiency) as well as income generated

  • Economic impact assessments can build on qualitative, case study approaches (like in the case of the Natural History Museum in the UK - see our Europeana Pro interview below)

Challenges

  • Most cultural activity is publicly-funded for the public good, with economic impact often not being the core driver

  • Economic impact assessment is challenging methodologically and requires investment into expertise

  • Economic impact can’t give a holistic perspective of impact created. What about the other strategic perspectives of organisational, social and educational impact?

Inspiration

https://pro.europeana.eu/post/understanding-the-economic-impact-of-digitisation-at-the-natural-history-museum

Economic impact methods: a quick overview

Below we list some methods that are commonly used in the cultural sector. Each has its plusses and minuses, which are outlined in brief. Before deciding to use the methodology, we recommend that you find more information. Remember, you may also have to get the support of an external expert.

Economic impact assessment (EIA)

What: the assessment of the effect of an activity or organisation on a specific area, e.g. local, regional or national

When: you want to evaluate the effect of your activity or organisation in terms of cost-savings, wages or new jobs, generated revenue or donations

Advantages

  • Can be used for both long-term and short-term interventions or activities

  • Comparable to other EIAs if strict methodologies are followed

  • Widely accepted form of impact assessment

  • Plenty of resources have been published and many economic impact reports are available for the cultural and heritage sector

Disadvantages

  • May require expert assistence to go into the detail of the methodology (e.g. direct, indirect or induced effects, gross-value added, etc)

  • Does not give insight into non-economic (e.g. social or wellbeing) outcomes, which might be very important, so may need to be combined with another methodology

  • Economic impact might emerge in the short-term but not be sustained in the long-term, as suggested, for example, in the case of European Capitals of Culture (Policy and Evidence Centre, UK, 2021)

 

Read our impact assessment - Europeana 2019

The positive economic impact for Europeana 2019, held in Lisbon, was calculated by assessing additional ‘tourism’ nights accommodation and additional daily spend, as well as the hotel and daily spend directly associated with the conference. The data show that 39% of guests stayed for additional tourism days. During these days, they also spent more money than usual.

With a confidence level of 95%, we calculate that between €82,000 and €98,000 was invested into the Lisbon economy as a result of Europeana 2019 conference catering, hotel spend and additional tourism hotel and daily spend. Read more on Europeana Pro

https://pro.europeana.eu/post/impact-assessment-report-europeana-2019

See more:

https://www.espon.eu/working-paper-cultural-heritage
https://pro.europeana.eu/post/measuring-the-multidimensional-impacts-of-digital-heritage
https://www.artscouncil.org.uk/research-and-data/value-libraries
https://www.nationalmuseums.org.uk/what-we-do/encouraging_investment/economic-impact/
https://pro.europeana.eu/post/how-impact-assessment-helps-us-to-understand-the-value-of-built-cultural-heritage

Social return on investment (SROI)

What: A holistic, stakeholder-focused methodology that helps attribute value to the positive changes being created by an activity or organisation, expressing the value in monetary terms, but accounting for changes that might not traditionally be captured in an economic impact analysis.

When: Your project involves close cooperation or engagement with stakeholders; the outcomes created might be expressed directly or through a proxy in an economic way; a monetary result might be helpful in advocacy to funders or reporting your impact.

Advantages

  • Can be built into other methodologies

  • Can be an interesting way to understand value created from the perspective of your stakeholders

  • Results are easy to share and memorable

  • Results expressed monetarily are easy to expand

  • Requires transparency

  • Based on clear principles (see Social Value UK)

Disadvantages

  • Can be labour intensive

  • May need coaching or expertise of an external expert

  • May result in over-claiming and thus claims might not be believed or taken seriously

  • You may need to conduct other research to find out more information that will be valuable for your assessment

See more:

https://www.betterevaluation.org/methods-approaches/approaches/social-return-investment
https://socialvalueuk.org/resources/sroi-guide/

Return on investment (ROI)

What: an assessment of the amount of return of a project, organisation or investment compared to the cost invested

When: you want to measure and communicate the value or efficiency of a service in a simple monetary way

Advantages

  • Simple result expressed as a ratio or percentage

  • Result is easily understood

  • Effective in decision-making (when used in advance or when advocating for investment) and in evaluation (looking back to evaluate the use of resources)

  • The return of the investment can be calculated in terms of outcomes including, for example, research, like in the Natural History Museum below

Disadvantages

  • It ignores other non-monetary outcomes (and this is where Social Return on Investment is more valuable)

  • Cultural projects or activities rarely have financial return or investment as the primary driver

https://pro.europeana.eu/post/understanding-the-economic-impact-of-digitisation-at-the-natural-history-museum

Cost-benefit analysis (CBA)

What: comparing the cost of an activity with the value of the benefits created for society (strengths vs weaknesses), expressed in monetary terms (e.g. Euros). The methods used may include many of those listed on this page, rather than one specific method

When: Often used at the start of a project to evaluate possible courses of action and make decisions based on cost. It can also be used retrospectively to evaluate an activity

Advantages

  • Good for situations like policy development or action

  • Aims to support the most impactful allocation of financial and time resources

  • Can avail of different methodologies

Disadvantages

  • More difficult than it sounds

  • Requires a robust list of all different outcomes and costs that could occur

  • Not useful in all situations

  • Ignores that some actions may benefit some and disadvantage or not benefit others (ignoring diversity in stakeholders)

  • Other benefits might occur that are non-monetary

See more:

https://www.betterevaluation.org/methods-approaches/methods/cost-benefit-analysis

Contingent valuation (CV)

Contingent valuation (CV) methods make it possible to calculate the use value (including the option to use something) as well as non-use. Two methods, willingness to pay and willingness to accept, are often used in the cultural sector.

Willingness to pay (WTP)

What: what the maximum is that someone would be willing to pay to access a service. It is a type of contingent valuation/stated preference technique that asks people to say how they might behave in a certain scenario

When: You might use this to ascertain pricing for an activity, or to give an idea of value created for the beneficiary

Advantages

  • Can be conceptualised as a range

  • Metric is expressed in monetary terms (e.g. euros)

  • You could assess willingness to pay for repairs or research on top of a ticket price, for example

Disadvantages

  • Context specific, e.g. in terms of the person being asked and income, as well as the situation

  • Can be complex and expensive to calculate

  • When something is offered as a public good, e.g. culture, or where it is normally free, e.g. voluntary services, it may be disconcerting for the audience to have to think about paying for the service. This may have a negative impact on the relationship with the audience

  • It is hypothetical and may be overstated (e.g. if the service is highly valued, it may be estimated as very high, but the audience may be unlikely to be able to pay for it in reality)

See more:

https://www.gov.uk/government/publications/the-economic-value-of-culture-a-benefit-transfer-study
https://www.nesta.org.uk/report/economic-value-heritage/

Willingness to accept (WTA)

See more

For more about using contingent evaluation in the cultural sector, this report might be valuable:

https://www.researchgate.net/publication/303118306_Measuring_Economic_Value_in_Cultural_Institutions

Next steps