This project aims to make AI technology widely applicable within media archives based on interactive learning interfaces that enable users to explore the data visually. This requires research into combining the different types of data sources within archives, both for analysis and for displaying and visualising in the interface.
The research for this is done in two research lines:
- The first research line 'Multimedia Analytics' investigates new methods and techniques for interactive and learning exploration of media archive data at scale.
- The second research line 'Computer Vision' focuses on the development of flexible algorithms for extracting information from visual archive data with a focus on generic and non-task-specific applications.
To give the research a practical shape within the media archive context, an interactive learning module is being developed. Through this module, users can organise, categorise and interpret data themselves in order to find answers to research and information questions. The point of departure is that such a module must be able to be linked to data infrastructures and archives, and thus be in line with the applicable standards and best practices for the large-scale and computationally intensive processing of audiovisual data, including (secure) storage and retrieval of data.
The module's source code is made available open-source to enable collaboration with other parties within the creative industries, for example in the context of AI field labs, and to promote application for a wide range of CD-related questions within the creative industries.
Plan of approach
The project is set up iteratively, with a focus on increasingly developing knowledge about how Computer Vision and Multimedia Analytics techniques can be used within media archives. Based on an agile working method, we start with the current state-of-the-art, with the MediaTable system developed by the UvA MultiX lab (https://multix.io/) and a selection of media archive data.
In addition, the project will build on the DANE infrastructure developed by UvA and Sound and Vision. This infrastructure developed for CLARIAH makes it possible to perform large-scale CD analyses on media archives with the aim of making them accessible for applications such as interactive exploration of the dates. By building on this starting point in iterations, work is being done on a prototype in which the principles developed in the research can be linked to the media archives in the Media Suite.
Project budget €558,000 with €397,000. PPS surcharge being used.