For the first time we know of in the cultural or cultural heritage sector, in 2020 Europeana piloted the use of Outcome Harvesting, a method increasingly applied in developmental contexts, in an impact assessment of EuropeanaTech.

Outcome Harvesting is a methodology developed by Ricardo Wilson-Grau and colleagues to help understand and verify change in complex and often long-term situations where it may be difficult to define the specific goals of an intervention (e.g. because they hadn’t undertaken a Change Pathway exercise, like in Phase one of the Impact Playbook), or where actions and activities have evolved organically over time. With this in mind, we felt that it was a suitable methodology to pilot in our Impact Assessment of Europeana and EuropeanaTech’s contribution to the implementation of IIIF. 

We sought advice and coaching on the methodology from an external expert to help us. It was a time-intensive methodology and we relied on the participation of  partners involved in or connected to EuropeanaTech and IIIF. Read the EuropeanaTech impact assessment report in full on Europeana Pro.

What is Outcome Harvesting? 

Outcome Harvesting does not measure progress towards predetermined objectives or outcomes, but rather, collects evidence of what has changed and, then, working backwards, determines whether and how an intervention contributed to these changes. The outcome(s) can be positive or negative, intended or unintended, direct or indirect, but the connection between the intervention and the outcomes should be plausible. BetterEvaluation.

Outcome Harvesting is a six-step evaluation methodology that can be used in more complex situations ‘where relations of cause and effect are not fully understood’. Information about outcomes (explained below) are ‘harvested’ and analysed across different steps.  

What is an outcome in Outcome Harvesting? 

Unlike the simpler descriptive outcomes that identify behavioural or attitudinal change (e.g. increased likelihood to visit an exhibition), an outcome in Outcome Harvesting is rich and descriptive, and it can range anywhere from a sentence to several paragraphs long. It should cover the who, what, why, where and how. The ‘who’ is particularly important - they have to be identified in order for the outcome to be verified later in the methodology. 

When should Outcome Harvesting be used? 

We learned from external expertise and from the BetterEvaluation website that Outcome Harvesting can be used in the following situations: 

• You are focussed more on outcomes (effectiveness) than activities (efficiency) and when, for example, activities might develop organically over time. 

• You want to know more about processes of change. 

• The relationship between cause and effect is complex, e.g. in innovation contexts. 

What is the Outcome Harvesting methodology?

1. Map the users of the Outcome Harvest and identify stakeholders

2. Define the research question

3. Map the outcomes (document review, consultation with partners)

4. Substantiate the outcomes with the ‘who’ in the outcome

5. Interpret and analyse your results

6. Use the findings

Read more on Europeana Pro

Next steps

• Step 3. Choose your methods

• Data collection tips

• Step 4. Analyse the data