Amsterdam video search engine tested as the best
The U.S National Institute of Standards and Technology (NIST) tested the video search engine developed by UvA researchers as the best during an evaluation that this institute conducts yearly.
A team of computer scientists at the University of Amsterdam (UvA)and its
spin-off Euvision Technologies, led by Cees Snoek, has developed this video
search engine that achieved highest scores among systems participating in 2014
search effectiveness tests.
In a comparison of more than twenty video search engines from renowned
international universities and research labs, the UvA search engine scored the
highest average precision against the 2014 test data. Due to constant
innovation, the Amsterdam video search engine was also a top scorer in 2008,
2009, 2010, and 2013.
Challenge
To measure the precision, the search engines are asked to find video clips in a collection of hundreds of hours of Internet clips in which, for example, a lecture hall, a baby or a goal is present. The scientific challenge is to automatically interpret the image content so that the search engine can determine what happens when and where in the digital video content.
Team member Koen van de Sande explains: 'retrieving videos on YouTube goes well as long as the content is described in text by someone in advance, but finding fragments from described or undescribed video is impossible, requiring sophisticated image recognition technology. Our software looks at the pixels, understands the content, and translates the pixels into descriptive text.' NIST determined for each retrieved fragment whether it met the search question.
Deep learning
The team relied on a technique known as 'deep learning' to make the image description as precisely as possible. Team member Daniel Fontijne taught the video search engine what different objects look like on the basis of many labeled examples, instead of specifying their appearance in advance. The search engine further makes use of efficient programmed neural networks, advanced graphic cards and national computer facilities. The computer scientists will present the video search engine during a workshop organized by NIST in November.
More information
Read here more about the test-evaluation of NIST on the concept search task (semantic indexing). http://trecvid.nist.gov. Results do not constitute endorsement of any particular product by the U.S. Government.