Country pages validation (Member States)
- Julie Van Oyen
Table of Contents:
Links:
___________________________________________________________________________________________
Alessandra Luciano (Luxembourg), Mon 4 Nov
Link to recording
Key takeaways (UX)
She could see using it as a dashboard as well as a presentation/advocacy tool.
To advocate for more funding.
To inspire a bit of competition in the data providers!
User expected to be able to navigate to her country page via the map/country list on the main dashboard - she tried to do this twice!
She did not find ‘Country Pages’ tab until prompted.
She had difficulty finding Luxembourg in the Country Pages dropdown.*
The top section of the country page (3D/HQ/All cards) are very useful; they clearly show the current and target states of the data.
The user navigated back to the main dashboard several times, and did not understand the difference between the two without explanation.
She requested an easier-to-find explanation of data tiers, as she could not recall what they meant from memory.
Data Progression table is too complex*; she avoided interacting with it until prompted. She preferred comparing countries on the main dashboard page. She didn’t realise how to get the hover effect on the line chart until prompted. Once she realised, she found it quite useful to compare countries.
The function of the squiggly lines to ‘zoom-in’ on the Content Type bar chart near the bottom of the country page was not clear to her.*
User requested a high-level entry landing page/section to her own country page; she doesn’t think it makes sense to have the main dashboard is the entry point for member states - they want to see their own country first.
*The styling of components could be clearer if they used more common patterns (e.g. https://styleguide.europeana.eu/#/Style/Bootstrap%20Vue?id=dropdown )
Key takeaways (data)
?
Graph in the middle of the country page doesn’t show 2021, it only shows in the [hidden] table
<<RAW NOTES BELOW>>
Discovery questions: about her role and day-to-day
She coordinates digital strategy for cultural heritage at Luxembourg’s Ministry of Culture,
works with 8 chi’s national level: library, archives, architects, museum, history museum;
helps them with dig transition ambitions, their collections and collections systems and emerging technologies.
to house everything under national govt to not duplicate resources or systems;
‘bootstrapping initiative’ to support the first years of the transitions (money resources; over the years they should become more independent).
Tracking on Europeana - They don't have a national aggregator, so there's no built in reporting back to the ministry;
the EC recommendations are one of the big driving forces behind her department (started in 2016/17); before that there was very little digital ambition.
She needs data to inform their practices and policies; also for her dept to continue to exist and for more understanding about what they do. She is interested in how they could attach funding to contributing to Europeana?
She can use the dashboard to write the first criteria for that; for example, the dashboard will help show that her budget is not increasing proportionally to how much more data is needed.
She hasn't used the statistics dashboard yet.
Browser: Firefox (like most of her team) - doesn't know anyone who uses Chrome or Edge
Device: Desktop or laptop usually
Usability test
The dashboard didn’t load on Firefox or Chrome.
She thought it could be her network protecting it, but didn’t get a message to that effect? - ministry of culture - gov it blocks any page that may be considered a threat or or something - she tried via ethernet and wifi. (Julie to notify devs - DONE)
She tried to navigate to Luxembourg via the map, which doesn’t lead to the country page; had to be instructed to go to the top left of the screen (quite separate from the rest of the dashboard); also didn’t find Lux. in the dropdown.
The numbers for Lux. are different than she expected; but then she looked at the providers, and then it “made sense”.
She can use this data and UI to make her case for the funding and resources that she needs, because the targets are very clear.
She navigated back to the rest of the dashboard to compare, not directly in the Lux. one using that table as I think is intended - the divide between these are too much, it’s easy to leave the country page and not find her way back again….or maybe it was intentional? Because her team is so small, she’s looking to compare/or come up with examples or targets?
She’s not super familiar with metadata tiers
For advocacy type work, she wants to take a ‘snapshot’ - what info does she need for that work?
Cabinet level - Work off the recommendation that was accepted contribute x amount of data towards Europeana, as you can see we are close/far from the target (those cards are very useful for presentation)
Create competition between institutions is interesting - compare who’s contributing the most, vs where there is room for improvement, is there any way I can help you close the gap, this is screenshot of x year, since then you can see we’ve grown this much; likes to compare growth year-to-year (check that the dashboard does that in the table/line chart)
She looked at different countries to see who had the most of what kind of data.
INTERESTING the UK has the most CCBY? Not the country I would have expected
“Lux. is so far down” in the countries table on the main dashboard
Found the date picker easily’ interacted with the filters easily on the main dashboard
She interacted more with the main dashboard, and the filters - is it because the data there, or the view, is easier to interact with?
“So this is the landing page” about the main dashboard
<Fiona directed her to get back to her country page>
Using the map to navigate to Lux. is hard because it’s so small, but clicking there doesn't lead her to the dashboard; she clicked on the map and country list AGAIN to navigate there - “how do I get to my country?” - she mentioned that’s how she expect to navigate to her country via that list and map
I would use this more [if there was more data for lux.?]
Understanding what the definitions of tiers would be helpful - that’s complex and jargon
“I don’t know what tier 1 really means - is that high or less quality” - could use an explanatory pop-up -”I assume it’s lowest common denoimianotr because it’s the most we have”
“What is this?” the UI feature that ‘zoom’s in to a section on the bar chart is confusing and doesnt match what she’d expect to see for that kind of UI pattern
She did eventually find the tier explanation but it took a while - “it is there, but maybe not as prominent as it could be”
To me the landing page doesn't feel ‘clean’ - you don’t get an overview effect; the tiers’ print is really small; a higher-level entry point would be really helpful
How do people normally do it? They’d want to start with their own country - I wouldn't care about any other data until I need to compare to my data; so the country page would be my entry point, to track my progress and use it as an advocacy tool; can I do comparative? Can I look at 3 countries next to each other? Can I select three countries to compare? France, belgium, germany; malta is more comparable to us
(she hadnt seen this graph yet - she hadnt interacted with it at all, which shows that something about how it looks isnt useful or comfortable; when she uses it, she does think it’s interesting; she would like to see the table have info on hover, not just when the table below is open; “why do I have germany” - she couldnt see how the pin was ‘holding it there’
I would also be interested in seeing some things that are useful to me, not just for the recommendations - who’s contributing the most and what kind of data are they contributing
Question - what other types of data do they want from the data space, to motivate their institutions?
Their institutions need to feel like the process is easy - having to reconfigure their whole data to fit the data space is a barrier; they want to be ae to use whatever they’re already doing; more of a drag and rop experience would be ideal, and they’d be far more likely to share more; 5-10 year long progression curve so they can see their efforts would be great; but the non-ease of it is the major challenge - the aggregation process is not the smoothest for them, and that’s where most of the barriers lie; otherwise they’re happy to comply;
They work through domain aggregators; interested in finding other ways of aggregating; they dont like how it has to be ‘dumbed down’ to fit into europeana
She needs to explain what the data space is to both her cabinet and her data providers; how do we aggregate, and what is that process like, is there a new way we can do it and can they be a pilot for it? Because they are small and they are willing
Being discoverable on Europeana is not interesting enough on it’s own for CHIs,
they have their own platforms and networks, that are keeping them busy;
more interested in being part of the community around testing tools and technologies like IIIF projects where they codevelop or codesign new data schemes that would ofc work better for them;
levelling up competencies is really what it’s about ;
the activities are alway an excuse for the bottom line which is building competencies and future-proofing - that’s the philosophy is being structured
Tonny Skovgård Jensen (Denmark), Thu 7 Nov
Key takeaways (UX)
Key takeaways (data)
National Library and Royal Library (x2) are all the same institution now - Fiona to flag with Henning after the meeting
<<RAW NOTES>>
Discovery questions:
Royal Danish Library; used to be the national aggregator in Denmark; national and university library, quite big (800 people);
He is the Chief Consultant; he works with cultural heritage and data translation projects; CDCHE group Denmark representative
They aren’t very engaged with Europeana and the DS; but open to it
Shadi: Hopefully, the dashboard can be a motivation for them
Though he was surprised by how many items have come form Denmark, especially from SMK
He’s not directly engaged with tracking statistics; occasionally studies them
If Denmark decides to become more involved, then he could see them being interested in the dashboard
Device: Laptop
Browser: Chrome
USABILITY TEST
Main dashboard
He asked: does High Quality mean when it was ingested, or no? Fiona: it’s showing now (within last few weeks)
Tried to find Denmark in the country map and country list
The country bar chart doesn't show all countries next to the bars
Country page
Took a few seconds to find Denmark in the country dropdown
The top charts were good, easy to understand
Data Progression chart (middle of page)
The line chart is confusing
The target can’t be so low? He was only looking at the 3D one, which is why it was so low
Also switching it on and off is not easy to find and do - needing to click on the year PER COLUMN - he kept forgetting and not understanding how; “I guess it’s meaningful once you get it”; the other coloured lines look like they’re above the target ( better line labelling is needed)
He compares with Belgium - they’re dotted lines
“How do I interact with that?”
The chart is really convoluted when you select multiple data, and multiple countries, and their targets etc
“I guess if you’re really focused on the targets it’s useful, but we have no ambitions to reach those so its less relevant; it’s more relevant to see the development (the angle of the lines going up or down over time)
“I wonder if you can change the # of years you can see on the x-axis” - I would like to see the progress into earlier years
Clicks ‘Show Data’ on lower right - that shows when the data points came from, started in 2021 when the recommendations came in; that’s when we started measuring
According to the columns “in show data” the HQ data is decreasing, but the line chart doesn't show that because it doesn't show 2021 (it should show 2021, “I would like to see a broader timespan”)
The drop in the items - broken links go to tier 0 after an update; or depublished items with broken links;
Image type table
“I was curious about…” in Europeana Sounds they contributed a lot of audio files, some video files;
The x-axis is inconsistently labelled with or without units?
If you click the link in the image type table it’ll take you to Europeana - COOL, NOT OBVIOUS
Data provider table
National Library and Royal Library (x2) are all the same institution now - Fiona to flag with Henning after the meeting
The don’t have the infrastructure to update their metadata for us, they have new links for all of the content but they don’t have the infrastructure to keep track of what they’ve sent to us, and what should be updated, etc.
Danish Agency for Culture is castles and sites, which are very in focus for the recommendations (as opposed to the royal library, which of course don’t have many sites)
They (and the ministry) are not so interested in 3D scanning - they do it for architects, but that never makes its way to us? Why not? The barrier could be the data model, also “lack of infrastructure” as he says
Discussion
Shadi: Why not interested in DS?
We see a value in participating, but Europeana is more a ‘nice-to-have' than a need-to-have
The big institutions that merged, and since they’ve been trying to consolidate all of those different IT systems; every project cxreated their own infrastructure and their own principles, so they’re struggling with all of their own many different systems; still consolidating all of that; that’s why looking outwards to europana is not their top priority
Many changes in the ministry of culture and their agencies; they work in a different way now; not many people in the ministry/agency that are professionally from cultural heritage, unlike many other countries (as he’s seen from the expert group);
Denmark’s big institutions are all focused on digital; it’s just not centrally coordinated
He thinks the recoemmendations are confusing, in tat they pretend that being on Europeana is the most important aspect of digital CH in europe. It can’t be “the” most importat from the ocuntry perspective, since the vast majority of their users are national users; especially when given that Danish is a language that few people outside of Denmark speak. So their local/national users will always be the priority.
Other ppl at Danish institutions that Europeana has played a key role in creating awareness around standards that they in turn use, etc.; but Europeana website itself, not that much of an effect
What would make Europeana not just a nice to have but a need? If some institutions would really invest in 3D it could be;
from the library the work from the concept of collections as data;
a lot of researchers are using their collections as data; not just one picture or one text, but getting millions of webpages collected over 20 years and they analyse that as a set; text and data mining; machine learning/ai techniques to analyse and use their content.
and Europeana expanding into a data space could then be relevant; if danish researchers got to use the new cloud for cultural heritage and got the libraries content through that channel the most easily, that could change things;
but he still has an unclear understanding about which new kinds of data will be in the data space?
the EPF with metadata specifications they have to adapt to; “collections as data” is always connected to a lot of legal work since most of what they have is in copyright, but is sometimes available for specific purposes if they have special contracts - the data space can support people managing as data owners who gets access to which data;
that would be easier if it was through the data space and not the library, which would make the data space more interesting;
if the ds offer possibility to upload them an ai-produced keywords that they could also use in their own system; if they know it’s in europe and in a secure platform, it’s much easier for them to use
These could be new arguments for more people to participate in the data space; and convince ministries to give more funding to do so
They are already acting as an aggregator within denmark, within their own systems; 20,000 broadcasts every month for example; adding to europeana with the specific rights requirements we have is an extra step they just can’t do
35 million newspapers digitised; the OCR quality is a few years old so it could be done much better today; 6-8 million pages of them are public so they could go on Europeana;
Fiona: Is an international audience interesting to you?
Most text-based items (which is most of their collection) are probably only most interesting for Danes
Katarzyna (Kasia) Waletko and Marta Ćwiek (Poland), Fri 15 Nov
Recording can be found here.
<<RAW NOTES>>
Discovery questions:
Ministry of Culture and National Heritage; digital culture team;
works with polish cultural institutions on the subject of digitisation; digitisation of cultural heritage on Poland is her team’s focus, encouraging the institutions to do more digitisation;
capacity building and policy;
super computing institute that they collaborate with helps them track the number of items on europeana, and which institutions are cooperating with europeana
She has used the statistics dashboard, needs this kind of data when they talk about europeana with high level dialogue with heads of uni, ministry, etc; they use the stats that are already published there
She’s used it 5x in her two years there
Desktop / Chrome
Usability test
3D Data charts
Gave her the direct link to the Poland country page https://metis-statistics.europeana.eu/en/country/Poland
Quite obvious what’s it about, but maybe for people who don’t know the exact context, maybe they won’t understand - what is the target, what does it mean - contextualising that it’s about the EC recommendation, link to that, etc.
What does high quality mean? Not everyone might know (and it’s not obvious where to find the definition - the “?' info icon is not easy to find, and is too small (not accessible, hidden until you get closer with your mouse (don’t hide it)
Numbers are what she’d expect, but the target for 991,127 3D items by 2030 is impossible, “we can’t do it” - that number in the recommendation - why is it so high? Who made this number? Was it our idea or somebody else’s? How did they count it?
Data progression chart
Scrolled past?
By media type chart - “I have to focus on that; it’s not so clear for me what is here” - “you need some time” - squinting and reading, Fiona offered explanation for the pills at the top showing quality tiers; a lot of info and numbers
She interacted with the dropdown at the top, and was confused at first when it showed Netherlands (because it was the highest) instead of Poland
“I can see Europe here” the label of Europe as a country on the y-axis is not fully accurate
Flag button in top right of that card doesn't work
Fiona had to prompt her how to go back to her country page, via the link in the top left
(a lot of prompts required for navigation)
By Content Tier - “top tier” as “1” doesn’t make intuitive sense; it’s the most common tier, not the best tier
The squiggly line to zoom in is not intuitive - 'maybe you can switch it, make it not visible when there are small numbers only when there are big numbers(???)
Not obvious that you can click on them to see queries on europeana.eu showing those tiers
Export data - is interesting; “is it on every page” - would also want on the country page itself
Main dashboard
Would like to click to access her country from the map on the main page
She managed to click a “hidden” icon next to the list of country names and navigate to the country page that way, unlike the others
Data progression chart
“What is that” the pin - clicked it and lost Poland (was not clear that it was pinning the country)
Fiona had to explain that she can compare to other countries here
Are you interested in comparing to other countries? “It is interesting to see how others are managing, if we are comparable” “It’s great for analysing which parts of Europe have a bigger concentration of 3D digitisation processes
She would be interested to ask other countries who have more 3D content, to ask how they did it
Czech republic and Slovakia - we compare a lot with them, it’s nice to see their numbers here too
Data provider chart
This is very useful to know which providers are the most active; we like to discuss with them how they manage, why are they doing well/not well
Everything is clear here, in this chart it’s quite obvious who is the first one
Rights category
Also really interesting; for now our PD mark is quite high, but 3D is coming so that might change; we are discussing a lot about that matter in Poland right now
We are interested to see what rights categories will emerge in 3D
Thank you for the report [on 3D?] that you sent us, because it’s very interesting material
Had to be prompted to use filters on the far left, to filter for 3D media type in this chart
Provider chart
At first, didn't see the difference between this and data provider (terminology is too similar)
Everything is clear, but is less interesting than the data provider and rights statement charts (aggregators are less interesting
General
It’s really useful
Give more description that it’s about European Commission recommendations
Have you thought maybe about the perspective of users - I would be really interested to see how the end-users work with this data
In other words, she’s interested in the usage stats dashboard - wants to see which data is popular with end-users
She thinks it would make sense to be displayed here in the same dashboard
Georgia Angelaki and Elena Lagoudi (Greece), Thu 28 Nov
Notes
Chris de Loof (Belgium), Thu 28 Nov
(in person)
Notes
Antonio Davide Madonna (Italy), Fri 29 Nov
Recording can be found here.
<<RAW NOTES>>
Discovery Questions