The Europeana Data Publishing Services team (DPS) is the main point of contact for all data partners wishing to publish data in the Europeana website. Since January 2019, the Europeana DPS team adopted scrum to manage the work of the team and since July 2019, we use Jira as a ticket tracking system to organise the work of the DPS team. 

Why Jira?

Working with Jira makes the work of the DPS team and the data publication tasks more transparent. It also helps to populate the backlog and prioritise the work on data publication.

Both DPS and data partners can follow the progress of individual datasets, each represented by a ticket, and exchange feedback quickly and easily.

Colleagues and partners that regularly work with Jira among our data partners have reported to us that it has become easier to monitor the progress of data processing work compared to communicating via emails or Google docs. 

Getting access to Jira

For every data partner contact person that needs to be involved in the data publication work, access to Jira is needed. For this to happen, EF needs to be informed about who requires access, and needs an email address from the person that will be added to Jira. This will trigger the person to receive an invite to join Jira and to create a Jira account.

Open

Email invitation to join Jira

We aim to have one to three main contacts per data partner. It is also possible to register an aggregator specific email address, like a generic contact email account that several people have access to. This way it also improves flexibility to e.g. have other people that only temporarily need access to Jira to work on a ticket.

Open

Login screen for creating an account in Jira

The Data Partner Epic 

Every data partner has a high-level ticket (called epic) that is named after the aggregator.

There is one unique URL for every data partner epic that unites all other tickets relevant for this data partner. The list of epic URLs is in a separate sheet.

The main contact person/s for the data partner alone will have access to their specific epic, so you will not be able to see and interact with the data processing work of other aggregators. This is important with regard to data protection regulations as the DPS Jira project contains some personal data which cannot be publicly exposed. 

The dataset as a ticket

Every dataset that was processed and published since we work with Jira has a ticket in Jira that is linked to the data partner epic.

Its name consists of the Europeana dataset ID plus the name of the data provider. While the epic itself refers to the data partner as a whole, the dataset ticket will collect specific information about the dataset, such as recommendations for quality improvements and links to the set in the preview environment as well as a link to the published data in the Europeana website. 

Data ingestion specialists will comment on the ticket with feedback on the specific ingestion round and tag the data partner when the work is complete or if advice is required. Email notifications will be sent to the data partner with every comment or status change. The ticket can also be checked at anytime regardless of status and the data partner can respond when necessary. This feedback exchange between data partner and EF is replacing email communication for most of the data specific communication on datasets.

It is also possible to upload attachments to Jira (e.g. a sample dataset). The default maximum file size for these attachments is 10Mb.

Requesting an update of an existing dataset

If you would like to request that the DPS team considers your dataset for processing, then you should add a comment to the relevant ticket requesting an update.

Please add any relevant information, such as details of additional records, quality improvements etc.

The DPS team will be notified and we can include the request in our planning.

Creating a ticket for a new or missing dataset

For new and missing datasets that do not yet have a Jira ticket, data partners can create one by entering all relevant details for the dataset via the DPS service desk.

This way we can establish the basic information for every dataset, create a ticket and then move it into our workflow to be linked with the appropriate data partner epic.

The link to the service desk is also in your epic, so it will be available to you when you need it.

The Jira Guide Epic

In addition to the data partner epic and the dataset tickets, we have created one central epic that all data partners have access to. In this epic we provide this guide alongside general updates relevant for all of you. In this epic we will also flag periods when Metis may be down for maintenance and we cannot publish. Like with other tickets in Jira, data partners will get email notification once a new comment or update was added to the ticket.

Timeframe for accepting requests

Europeana DPS is working in sprint cycles of two weeks and our sprint starts on Tuesdays. 

The preparation for every sprint starts on the Monday before the Tuesday. Therefore in order for us to consider your request for data processing in the next two week period then please ensure it has been received on the Monday before the next sprint.

Once your ticket is pulled into the sprint the DPS team will aim to complete the work within this two week period. In case of any unforeseen issues that might affect this progress, the data partner will be informed in the comments of the specific ticket.

Prioritisation of work

At the start of the sprint we have a planning meeting where our work for the next two weeks is defined. This is the last chance for us to discuss and accept recent dataset update requests for the upcoming sprint. Accepting new tickets after the sprint has started will change the scope of the sprint and so we can only consider introducing more tickets where these are deemed to be a high priority by the Product Owner (Henning Scholz) or in a quiet period where many other tickets are already complete.

As a rule, every dataset update request submitted by a data partner will be added to the bottom of the backlog.

Europeana DPS is working on such requests on a first come first serve basis, however we will give datasets a higher priority where important deadlines are involved, if there is a high demand (e.g. from users) or if the data is of demonstrably high quality.

Notifications and the ticket status

As soon as a dataset is prioritised and planned into the upcoming sprint data partners will receive notifications.

If you wish to reopen a closed ticket then you simply have to comment on it stating your request.

Configuring notifications in Jira

Should you wish to configure notifications in Jira then please read below

Importing data from Jira into Google Sheets

With the Jira Cloud for Sheets add-on we now have the option to import data from Jira into a spreadsheet using JQL (Jira Query Language). This means we can generate various different reports, for example:

1. Overview of all tickets linked to the data partner epic

2. Overview of all tickets that were updated within a specific time frame (e.g. last sprint, last month etc.)

3. Overview of all tickets that were completed (marked as “Done”) within a specific time frame

JQL allows users to select filters for importing data - this means that spreadsheets will only contain columns specified in the filter. DPS team supports the following filters:

1. Aggregator name: data partner’s name

2. Key: unique identifier of the ticket

3. Assignee: name of the DPS team member responsible for the ticket

4. Sprint: name of the sprint(s) in which ticket was worked on

5. Updated: date when ticket was last updated

6. Data provider: name of the institution that provided data to the aggregator

7. Summary: title of the ticket. For datasets this is the Europeana dataset ID and the name of the data provider

8. Status: allows you to track progress of the ticket. The four possible values are:

To Do, In Progress, In Acceptance, Done

1. Fix version: all datasets that have value "Data publications - name of the DPS team member" were processed up to the preview and are waiting to be published

2. Description: here you can find additional information about the set (e.g. link to the set in the preview environment as well as a link to the published dataset, recommendations for quality improvements etc.)

3. Flagged: if there are any internal blockers preventing DPS team from processing data, the dataset will be marked with a flag and value "Impediment" will appear in the spreadsheet

4. Data Submission Details:  OAI endpoint, setSpec, metadata format, number of records, dataset ID etc.

If you would like to know more about the imports generated with Jira Cloud for Sheets, please contact the DPS team.