PhD candidate in Reinforcement Learning

Faculty of Science – Informatics Institute

Publication date
17 April 2018
Level of education
Master's degree
Salary indication
€2,222 to €2,840 gross per month
Closing date
1 June 2018
38 hours per week
Vacancy number

Applications are invited for a PhD candidate to work on the development and application of a novel framework for personalised health interventions that combines elements of contextual bandits, causal prediction, and online learning. The successful candidate will be based in the Amsterdam Machine Learning Lab (AMLab) led by prof. Max Welling within the Informatics Institute of the Faculty of Science.

The research will be supervised by dr Joris Mooij and dr Danielle Belgrave of Microsoft Research Cambridge.

AMLab conducts research in the area of large scale modelling of complex data sources. This includes the development of new methods for probabilistic graphical models and nonparametric Bayesian models, the development of faster (approximate) inference and learning methods, deep learning, causal inference, reinforcement learning and multi-agent systems and the application of all of the above to large scale data domains in science and industry ('Big Data problems').

The Informatics Institute is one of the largest research institutes within the faculty, with a focus on complex information systems divided in two broad themes: 'Computational Systems' and 'Intelligent Systems.' The institute has a prominent international standing and is active in a dynamic scientific area, with a strong innovative character and an extensive portfolio of externally funded projects.

The Faculty of Science holds a leading position internationally in its fields of research and participates in a large number of cooperative programs with universities, research institutes and businesses. The faculty has a student body of around 6,000 and 1,500 members of staff, spread over eight research institutes and a number of faculty-wide support services. A considerable part of the research is made possible by external funding from Dutch and international organizations and the private sector. The Faculty of Science offers thirteen Bachelor's degree programs and eighteen Master's degree programs in the fields of the exact sciences, computer science and information studies, and life and earth sciences.

Since September 2010, the whole faculty has been housed in a brand new building at the Science Park in Amsterdam. The installment of the faculty has made the Science Park one of the largest centers of academic research in the Netherlands.

Project description

Contextual bandits are often used to personalise user experience for optimising web search results, content displayed to a user and ranking advertisements.    In the health care domain, there are many scenarios where we can use a similar approach to personalise the patient’s treatment experience in order to develop the most effective health intervention strategies. However, due to the additional complexity of treatments affecting various outcomes which may underlie a causal relationship and variations in pre- and post-intervention clinical features, it may be more appropriate to extend the contextual bandit framework to incorporate a causal modelling framework. Such an extension can have high impact on the personalisation and effectiveness of health intervention strategies.

The goal of this project is to develop a framework for personalised health interventions by combining elements of contextual bandits, causal prediction, and online learning in a novel way. We motivate the setting using scenarios from personalised medicine, but it applies more generally to other domains where interventions directly target features that are causally related to the features we want to control.

The developed framework will be refined and applied within the context of an online therapeutic health intervention to support and promote positive behaviour change and mental wellbeing. The student will use information available from an app developed for mental well-being to identify whether we can optimise treatment strategies to improve mental health outcomes for heterogeneous groups of patients with mental health conditions using the framework described in the causal contextual bandit strategy. During the application phase of the PhD, the student will engage with the clinical team in order to understand the problem domain and how to refine the causal contextual bandit framework to the mental health domain.

You will be employed full-time at the University of Amsterdam (UvA) working with dr Joris Mooij (promotor) and prof. Max Welling (co-promoter) in AMLAB. You will conduct research in close collaboration with dr Danielle Belgrave (co-supervisor) of Microsoft Research Cambridge. You would visit a Microsoft Research Cambridge during your research. You are expected to develop this research line, and to assist in teaching and in supervising Master’s students.


We are looking for highly motivated and creative individuals who enjoy working in a multidisciplinary research environment.

If you are the successful applicant we are looking for, you need:

  • a Master's degree in artificial intelligence, computer science, mathematics, statistics, or closely related area;
  • a solid understanding and working knowledge of reinforcement learning;
  • excellent mathematical skills (especially in probability theory and statistics, calculus, and linear algebra);
  • excellent programming skills (preferably in at least one of the following languages: C++, Python, R);
  • strong communication, presentation and writing skills and excellent command of English;
  • commitment and a cooperative attitude;

Hands-on experience with causal discovery methods or contextual bandits will be a plus.

Further information

You may send informal inquiries by email to:


Preferred starting date: between September 2018 and January 2019.

The appointment will be full-time (38 hours a week) on a temporary basis for a period of 4 years (initial employment is 18 months). Periodic evaluations will be held after 9 and 14 months, and upon positive evaluation, the appointment will be extended to a total of 48 months. The appointment must lead to a dissertation (PhD thesis). An educational plan that includes attendance of courses, summer and/or winter schools, and national and international meetings will be drafted for the PhD candidate. We also expect the PhD candidate to assist in teaching of undergraduate students.

The salary is in accordance with the university regulations for academic personnel and will range from €2.222 (first year) up to a maximum of €2.840 (last year) before tax per month (scale P) based on a full-time appointment. There are also secondary benefits, such as 8% holiday allowance per year and the end of year allowance of 8.3%. The Collective Labour Agreement for Dutch Universities (CAO) is applicable.

Among other things, we offer:

  • competitive pay and good benefits;
  • top-50 university worldwide;
  • very friendly, interactive and international working environment;
  • excellent computing facilities;
  • new building located near the city center (10 minutes by bicycle) of one of Europe's most beautiful and lively cities.

English is the working language within the Informatics Institute. Since Amsterdam is a very international city where almost everybody speaks and understands English, you need not be afraid of the language barrier.

Job application

The University of Amsterdam is striving for a better balance in its staff whereby, by equal suitability, the appointment of a female candidate will have our preference. Women are, therefore, strongly encouraged to apply.

You may submit your application to To process your application immediately, please quote vacancy number 18-197 and the position you are applying for in the subject-line.

Your application should include:

  • a curriculum vitae;
  • a motivation letter that explains why you have chosen to apply for this specific position;
  • a copy of your Master’s thesis;
  • a complete record of Bachelor and Master courses (including grade transcripts and the explanation of the grading system), and
  • the names and contact information of two academic references (please do not include any recommendation letters).

Please group all these in a single PDF attachment.

Please do not send or copy your application to dr Joris Mooij or prof. Max Welling – they will get lost.

We will accept applications until 1 June 2018. The committee does not guarantee that late or incomplete applications will be considered.

We will not process applications not mentioning the vacancy number and the title of the position you are applying for in the subject-line. #LI-DNP

No agencies please

Published by  University of Amsterdam