| Panel |
|---|
| About the Europeana XX Phase four workshop: we conducted a debrief after the final deliverables had been submitted but before the project review meeting (something that happens at the end of every European-funded project). What we did: we adapted the template below to look at the three main stages of the Europeana Impact Playbook approach followed. We used Miro to do this. Who was at the meeting: the lead on the impact approach and two other project partners involved in various aspects of the impact assessment (as well as the Europeana Impact Advisor who helped as the facilitator). |
What worked well Impact was built in from the proposal writing stage (using Phase one design). The kick-off meetings then also took the impact thinking one step further, which embedded an impact approach from the outset and validated the impact plan and helped better map relevant stakeholders. The funding proposal received a good mark on the impact section, thanks to the impact approach taken. Stakeholders were defined early, and activities were connected to the stakeholders for whom they would deliver benefit. It was helpful that one member of the team had significant experience with the Europeana impact approach. Improvements were made iteratively to the data collection plan because user engagement events happened throughout the project period and because feedback loops and evaluation were built in.
What didn’t work so well Impact design The original impact assessment plan had to be adjusted because of the COVID-19 pandemic and a change in the project activities. Some outputs/activities happened or were delivered late in the project period, making it difficult to understand if they were delivering value for stakeholders.
Impact assessment could be a stronger part of other parts of the project, e.g. the qualitative impact of the editorial could have been captured to provide an impact narrative that goes beyond visitor statistics and thus steer the future editorial plan. The team collecting data around editorial engagement could have shared this faster with the team who could then make improvements and adjustments. It is challenging to track standardised data across very different activities (in particular, in physical events) coming from many sources (project partners) and some data can’t be compared. It was difficult to coordinate data collection across so many activities (because of a lack of capacity). It is still challenging to measure shifts in mindset stimulated by the project activity, also because of the impact assessment limited to the project’s duration.
Ideas for improvements - Europeana XX recommendations for future Put impact on the agenda for review meetings and build in an impact ‘sprint’ after the project’s end but before the project review (relevant for funded projects only). Ensure that impact assessment is part of all relevant project elements. Future research could investigate how to articulate the impact of the enrichment of Europeana data and metadata (relevant also for other generic services). Bring together more team members with different expertise and skills to evaluate the impact assessment, as this provides a deeper experience. Provide indicators that could help the measurement of long-term outcomes in the final activity report or deliverable and consider what your forecast could be. Consider how longitudinal evaluation could be built in after the end of the project. Set out a clear approach to agile evaluation and improvements throughout the project processes. Shape the review process using the impact narrative to make the final results as powerful as possible. Collect baseline data on partner expectations at the beginning of the project and use this in the evaluation at the end of the project.
What changes or actions did we take afterwards? This evaluation meeting inspired the project lead to structure the subsequent final project review meeting based on the findings of the impact assessment. Having a dedicated moment, or moments, to reflect on the impact assessment process, to identify its strengths and areas with a room for improvement is very relevant for ongoing work but can also help better structure and plan impact assessments in future projects. It is important to be smart about choosing evaluation moments. On reflection, Phase four doesn’t necessarily have to come as the last phase. It can be used throughout the assessment, supporting agile adjustments in the project based on the gathered data. |