From pixels into text
The Dutch technology used to power the site has won numerous international competitions over the past five years, and has now been made available to the general public for the first time. The search engine translates pixels into text and can automatically label on-screen objects, such as ‘guitarist' and 'drummer'. The engine uses a large number of examples to learn the correlation between a visual object and thousands of shape, colour and structure characteristics.
Users can navigate to various labelled objects in the concert video in the blink of an eye using a specially developed video player that displays these objects on a timeline. In addition to live concerts at Pinkpop, users can also search a database of interviews with Dutch artists using automatic speech recognition.
The website challenges users to actively provide feedback on the automatic search results. For example, a user can replace the label 'drummer' with 'keyboard player'. This user input can then be applied to further refine the automatic recognition process. Users can also add text comments in order to further improve the searchability of the Pinkpop database.
The www.hollandsglorieoppinkpop.nl website was developed as part of the national MultimediaN research programme and the Images for the Future (Beelden voor de Toekomst) digitisation project. MultimediaN represents an alliance of: University of Amsterdam (image recognition), Twente University, (speech recognition), Video Dock (video players) and the Netherlands Institute for Sound and Vision.
The video search engine will be online until the end of February 2010.