Search Filters: Usability testing (Aggregators)
- Julie Van Oyen
Recording timestamps = [green]
Links in page:
Usability test with Marco on 9 Feb 2023
Recording can be found here.
DISCOVERY QUESTIONS
What is your organisation and role?
Managing Director of European Fashion Heritage Association - an Aggregator of fashion institutions for Europeana
Jewish Heritage Association
EUscreen (audiovisual aggregator)
Europeana Aggregators Forum (was chair from 2018-2021)
Brief overview of your day-to-day
Manages a team of 9 people
Focus on aggregation and promotion of fashion heritage, mostly but not only for their member institutions
Also: capacity building, involved in several Generic Service Projects, Horizon/Erasmus + project;
Work in different contexts: Education & Reuse, in general.
How does this intersect with Europeana.eu?
His team produces a lot of editorial content for europeana.eu: Blogs, virtual exhibitions, galleries
Team members (e.g. fashion curators) use the site to search for and select items to review and reuse for editorial
How often do you use the website?
About every other day
How often do you use the search/filters?
50% of the time (the other 50% is more general browse of editorial content or items)
SIDE NOTE: A bit disappointed in how editorial content is not as easy to access now, as with the previous homepage (more clicks) [3:50], [6:30]
How do you access the website?
Desktop / laptop/ mobile / other?
Laptop mostly
Chrome / Safari /IE / Firefox / other?
Chrome
USABILITY TEST
[Starts at 6:40 in the recording]
Key takeaways
Primary user flow is Search for everything → Aggregator filter → Institution filter → additional filters
Most commonly used filters:
Aggregator
Institution
Type of media
Rights statement
Image orientation & Image size (e.g. when choosing images to add into a blog post)
Copyright is very important because Fashion items in particular are often in copyright (up to 90%)
Did not use the Theme filter, because he knows which Aggregator or Institution he’s looking at and starts there [15:29]; and the Themes list does not seem complete, almost feel arbitrary or like “false information” [17:25]
Assumed that the Theme filter was “for a more general user” [16:08]
Because of this, he did not use them or realise that the Fashion Theme filter gives additional options [26:00]
He really likes the additional Fashion filters [28:05]
He suggests that by making the Themes seem more complete/comprehensive, and therefore more trustworthy to an experienced user with a specific use case, they may be more likely to try the themes [27:03], [28:37]
But in his case, he didn’t think they applied to him because (for example) we have Fashion but not Film or Audiovisual heritage
If there were an Audiovisual heritage Theme, additional filters could be Duration and Colour vs. Black & white
Potential areas for improvement:
Only the English-translated name is shown in the Institution filter options [6:48], [12:02]
Institution is not searchable by acronym [7:10], [31:50]
“Can I use this?” filter is a bit too vague (e.g. Maybe, seek permission option); he thinks the Rights statement filter is better for this, and could be placed higher in the flow [7:58]
note: this may be due to the options that he saw (none were “Yes” - which sounds like often the case with fashion items)
Film heritage or Audiovisual heritage are not listed in the Themes [16:35]
Language filter: For audiovisual content, users may not be sure if it’s the language of the metadata vs. language in the actual audio/video [19:18]
The Colour filter could be useful but needs work [22:54]
Search by date/date range is the number one filter he’s missing [24:26]
It’s easy to miss a lower-down filter that’s applied, when you want to update your search [32:41]
Usability test with Effie, Henk & Kate on 9 Feb 2023
Recording sound did not work properly. See notes below for session details.
DISCOVERY QUESTIONS
What is your organisation and role?
CARARE - Aggregator primarily concerned with archaeology and architectural heritage
Kate = operations manager; Henk = IT management (?); Effie = content & engagement
Brief overview of your day-to-day
Effie - uses europeana.eu the most of the three
How does this intersect with europeana.eu?
How often do you use the website?
Multiple times per week
How often do you use the search/filters?
this makes up most of the user flow
How do you access the website?
Desktop / laptop/ mobile / other?
Laptop (Henk); Desktop & laptop (Kate)
Chrome / Safari /IE / Firefox / other?
Chrome (Henk); Chrome & Firefox (Kate)
USABILITY TEST
Key takeaways
This group will interact with the Search itself (typed in search terms “Beeldbank”; “
Use case 1: checking how CARARE’s items look on the site, if they are working properly, meet publishing criteria, etc.
Effie & Henk’s primary user flow: Search Term (e.g. “Beeldbank”) → Aggregator filter (CARARE)
Kate’s primary user flow: Search Term (e.g. “Amsterdam”) → Aggregator filter (CARARE)
They also did not interact with the Themes filter immediately; but did express interest in the Archaeology, Art and Industrial Heritage themes.
They are interested in being able to build queries, e.g. “Amsterdam” + “archaeology”(which would display the results for “archaeology” WITHIN “Amsterdam”), instead of it becoming “Amsterdam archaeology”
Other filters of interest:
3D
Use case 2: Finding items for newsletter
User flow: Search Term (e.g. “Snow”) → Type of Media filter (Image) → Rights Statement filter (which didn’t show them what they needed to know in this case) → moved on to Can I use this? filter and selected “Yes” & “Yes, with conditions” → these results gave some images with children in bright red snowsuits; interested, the users tried to find more by interacting with the Colours filter
The Colours filter gave too many options for Red & results are not reliable
Use case 3: Finding items from a certain time period and place
These filters would be very helpful! Simply typing in the search term “windmills 17th century“ gave only 4 results, which doesn’t seem right
Kate suggested we try with the entity collection, under the entity “17th century”
Searching for “windmill” within the 17th century entity collection gave 3 results; and one was a plant (windmill palm)
If the number of the results is lower than expected, they will toggle on “Not meeting publishing criteria”
Potential areas for improvement:
Effie expressed that even when using a combination of search terms and filter, she feels that there are some items missing from the results; so she will go page by page through the results [after only applying aggregator filter?], to look at everything “or as much as she can” to get a better idea of what is actually there
Having Institution Filter language consistent with Org pages would be nice (similar to what Marco said)
Having an exclusion/NOT filter would be very helpful (Kate would like to exclude natural history items, for example - she says they crowd her results often
she suggests a dropdown with select/deselect like in her gmail inbox
Simple text search falls short sometimes - windmill palm example
subject terms vs. free text search (ask devs)
Time Period and Place filters would be great
Use a smaller list of colours for our Colour filter - the colours we currently have are confusing, there are too many options, and many of them just return monochrome or greyscale images
Usability test with Cosmina on 10 Feb 2023
Recording can be found here.
DISCOVERY QUESTIONS
What is your organisation and role?
Deutsche Digitale Bibliothek
Previously: involved mostly in metadata and data quality
Project management: Frontend redesign project and a new portal that is called Collections from Colonial Contexts
Brief overview of your day-to-day
Uses Jira as a PM tool; test systems; works with design & development teams
How does this intersect with europeana.eu ?
Wants to see how the data they’ve aggregated looks like on the portal
If everything is working/showing correctly, “are there any mistakes, what can we do better”
Uses our portal to interact with their own content, since we have some filters they don’t!
Likes to look at the blogs,
How often do you use the website?
About every other day
How often do you use the search/filters?
“I use the filters a lot”
Doesn’t use the search box a lot
How do you access the website?
Desktop / laptop/ mobile / other?
Desktop
Chrome / Safari /IE / Firefox / other?
Chrome
USABILITY TEST
[Starts at 6:32 in the recording]
Key takeaways
Use case 1: Aggregator perspective - checking the quality of the data they’ve provided
Her main user flow is: Europeana.eu → “Search for everything” to get to the filters (sometimes she browses here a bit “to see what’s new”; always uses grid view) → “Aggregator” filter (e.g. “German Digital Library”) → “Institution” filter (she knows which has had data added in the last month or week) → she checks how this new data looks, how many records there are, if there are any problems, etc. → “Rights Statement” filter (some confusion with their naming here, what exactly the different categories mean, so she needs to find it); the “Can I use this?” filter is easier for this [6:30 - 9:03]
She will then look in the “Type of Media” filter (she calls them facets) to see that the numbers in brackets add up to what she knows they have uploaded [9:08]
If there is a mismatch, she tries to identify what is missing - image files? text files? She will compare the data on their site with ours.
She will ask the colleague who uploaded them to check and see in Metis if there were mistakes
She will toggle on “Not meeting publishing criteria” as well, and then try to troubleshoot what’s showing up there (e.g. check links to missing pdfs, etc) [11:00]
Use case 2: Aggregator perspective - using our filters (“facets”) to interact with their own data
She will test combinations of the filters we have to see how we’re dividing them (e.g. File Format vs. Type of Media)
“We do not have this feature, so I would…” Select the DDB -> “and use your facets to analyse my content” [13:05]
For example, she will filter large (i.e. “very nice” or high-quality) images, to see which data providers they are coming from - the Institution filter shows how many high-quality images each institution sent)
They will sometimes reuse these nice images for editorial (blogs/articles; give to colleagues) [14:08]
She describes the Themes filter as “very clean” which isn’t always useful, because they are clearly highly curated [14:45]
also - she found an object under the sports theme that didn’t belong [15:10]
due to there being no disambiguation between the name of an author of this book and a famous footballer with the same name
Instead of Theme filter, she’d like more of a category of some kind, for example “Postcards” [16:10] - (would this be more like our Topics?)
“Type of object” filter would be useful [16:50]
“Your facets are very clean” - very specific categories, which is good in some cases (e.g. country, etc.) but a bit extreme in some other cases (e.g. Sports theme) [17:15]
Newspapers theme - she would potentially use this one; [18:40]
for the date picker, she first tries only the year (adding date & month are extra steps that don’t seem necessary)
The Themes filter showing under the Dates filters makes her think she can choose or add a theme within her current selection of Newspapers [19:37]
Then she uses the Newspapers theme to see which German institutions are contributing them [20:39] - she has never done this before but finds it interesting
She likes the colour filter, but has a hard time with all of the options [22.27]
“I’m not very convinced that it works very well” [23:10]
“I would use it for fun”
Use case 3: Creative hobbyist - “I’m making collages with pictures from Europeana”
I ask her about this and we discuss it after [23:40], refer to the recording to hear more (out of scope for Aggregator use case)
Looking for images of certain size, the proper rights, certain colours, etc.; uses search terms such as “flower”
Potential areas for improvement:
More info next to the different Rights Statement codes would save her some time/clicks [8:20]
“Type of object” filter would be useful [16:50]
Layout hierarchy could be improved when Newspapers theme is selected [19:37]
Colour filter could use a more limited/clear set