Leading by example
Cees Snoek is Professor of Intelligent Sensory Information Systems and Pascal Mettes is Assistant Professor at the Institute of Computer Science.
What are you working on?
Pascal Mettes: ‘We ensure that artificial intelligence is able to recognise objects and behaviour in videos. What or whom do you see in the video, with what object, what are they doing?’
Cees Snoek: ‘If AI can interpret videos correctly, there are all kinds of applications that can be imagined. For example, AI can assess medical scans. But you can also think of cameras at Schiphol that monitor what happens to your suitcase. And we are working with TomTom, for example. There are TomTom cars driving around capturing the area. With those images, we create advanced maps for self-driving vehicles.’
What makes this field challenging?
CS: ‘AI works on the basis of machine learning. If you show a learning system what a tree looks like a hundred times, the software can learn to recognise trees. We try to encapsulate that learning process in algorithms.’
PM: ‘The old strategy was to impose rules on a system. Suppose you have a camera system in a nursing home that monitors if someone has fallen. Previously, you would tell that system: if someone is lying on the floor in his or her room, they have fallen, and you have to warn the staff. But practice is more complex than that. Hard rules won’t get you anywhere. What we do now is to give a computer examples of images and their correct interpretation. The machine itself learns to take the step from image to comprehension.’
CS: ‘This is how we ensure that AI can assign meaning to pixels.’
Why do young talents choose this field?
PM: ‘When I started seven years ago, it was a field of ideals. We wanted to do a great deal, but we weren’t there yet. Over the past few years, development has been moving at breakneck speed. The example of that tree? Seven years ago, we barely knew how to do it. Now that’s an easy job. It’s nice to work in a field that’s changing so fast.
Is it hard to find people for this research?
CS: The industry attracts many people, so we run into difficulties when talents choose the academic world. People like Pascal are hard to find. But the more you can attract such talents, the more other young people you attract. They see the great opportunities that lie here.’
Maarten de Rijke is University Professor of AI and Information Retrieval and Harrie Oosterhuis completed his PhD in the Information and Language Processing Systems Research Group, both at the University of Amsterdam.
What is the core of your work?
Maarten de Rijke: ‘We research how, for example, search systems and recommendation systems can learn from user behaviour.’
Harrie Oosterhuis: ‘These systems search large collections of, for example, websites, films or products. They have to show a selection that meets the needs of the user. By keeping track of what people click on, what questions they ask, and how quickly they make choices, such a system can learn to do that better and better.’
What makes the task of a search engine complex?
MdR: ‘Each action by the user provides the system with additional information. So, it’s a constantly changing issue. Moreover, users have different needs at different times. Suppose someone always reads the same genre of books. But one day that user is searching for a gift for someone else. A search system must be able to deal with unexpected signals like that.’
HO: ‘Another challenge is if several people use the same account. You watched Stranger Things on Netflix; your kid watched Peppa Pig. If the system doesn’t understand what’s going on there, it’s going to offer very strange suggestions. A smart interactive system understands the situation and doesn’t suggest children’s programmes after bedtime.’
Why does this require artificial intelligence?
HO: ‘In his lectures, Maarten compares a search engine to a librarian. On the one hand, they know which books are available, and, on the other hand, they understand what you are looking for. Even if you don’t know it yet. In such a role, you have to make decisions based on countless uncertain factors. That’s too complex for programmes that follow simple rules. That takes AI.’
MdR: ‘What is also crucial is that AI learns from new information through user interactions and can improve a search engine.
How does a machine like that learn?
MdR: ‘Recommendation systems sometimes add strange suggestions to the results. This is deliberate: the system is testing to see whether you are interested in something other than your usual choices. That’s how it gets to know you better.’
HO: ‘Say, someone is looking for recipes. If the system always suggests Italian food, it would seem that this person thinks pasta is the perfect food. But if the system had suggested stew, it might have discovered that the user likes that even more.’
How do you get your work out into the world?
MdR: ‘We work with companies that use search engines and recommendation systems, both with large organisations such as Google, bol.com, and Albert Heijn, and with small parties such as grocery stores. That makes this field interesting. As a researcher, you have one leg in the academic world and one leg outside of it. That gives you inspiration and requires a special kind of talent.’