You will likely come into contact with AI when dealing with healthcare. ‘When you call an ambulance, software calculates where the ambulance will go first.  We’re not yet at the stage where robots are replacing doctors, but that could change in the future’, says Van Kolfschooten. Her research focused on patients' rights in the EU in terms of technological developments. 

What are patients’ rights anyway?

‘The most recognisable example is that you must give permission before anything can be done to your body. For example, if a doctor gives you an injection, you must first be told the consequences and agree to them. Another example is privacy: who is in the treatment room, and who can see your medical records?’

How will AI change patients' rights?

‘Much is written about how AI is changing society, but healthcare is the only sector where AI is already widely used. It's problematic that it's unclear to doctors and patients how exactly these algorithms work. That touches on the right to “informed consent”. If the doctor does not understand how AI works, conveying that information to a patient becomes difficult. Another point is that personal data is needed to develop suitable algorithms, while patients have the right to privacy.’

How is AI used in healthcare?

'It's often used in diagnostics. In a brain scan, AI can filter out certain areas if something suspicious is seen. Someone may also be called in sooner for an appointment if AI recognises certain risky signal words in the description of a complaint. The UMC Utrecht recently developed an algorithm to predict the cognitive development in infants by tracking the growth of 300 babies. You can then respond to this in the child's upbringing. But it's somewhat controversial because it means you can start treating children differently. We also see a lot of AI in elderly care. Intelligent camera systems register if someone fell down in their room. The risk of depression can also be estimated by looking at social activities and eating and sleeping patterns.’ 

Do you see these as positive developments?

‘I take a cautiously critical stance. When you open the newspaper, it is all positive and wonderful and what can be done. That is why I also want to clarify that there is another side. If AI is used properly, it has many advantages and can save time. But then you have to have the patients’ rights in order first. A lot of money is being poured into AI because hospitals think it is “cool” to use it. But I see problems in the implementation. Too much is being done from the perspective of tech developers and too little from the perspective of patients. In home care, all employees were given an iPad because it was part of the company's new image. But the employees and patients did not understand the new way of working, which actually resulted in more time being lost. AI is often imposed from above, while we know that autonomy is very important for trust between doctor and patient. Such insights sometimes seem to be missing in the hype. AI should improve patient care, not just enhance a hospital’s reputation.'

Can't we help fewer people without using AI in healthcare?

‘No, that is the image that is being portrayed in politics. It is as if technology is the solution to everything, and that image is also being pushed by the business community. Technology costs a lot of money to develop and a lot of energy to use. If we invested that money in staffing and salaries, we could address many of the challenges facing healthcare. It is not a case of: “either we use AI, or people will die”.’

Is there a lot of regulation in the EU?

‘The EU has introduced the AI Act, which also applies to healthcare. However, it remains broad and abstract. In the EU, there are already a lot of healthcare regulations on product safety, but there are still few regarding patient rights. In my dissertation, I propose creating a charter for European patient rights precisely because they are under pressure from AI.’

What is the danger of that?

'Europe will soon be able to determine what products are on the market because the market will be regulated. But the use of products will be left up to the member states. There is a risk that human rights will diverge in the member states as a result. Healthcare crosses borders, so as a patient, you may suddenly find yourself dealing with countries that share your medical records with companies. It is important for the patient-doctor relationship that trust in healthcare is not damaged. If that trust is no longer there, people will avoid healthcare. We know that from research. If we do not set safeguards for the use of AI in healthcare, there is a real risk that patients will lose trust in the healthcare system. Another risk is that if something goes wrong with software that everyone uses, it will immediately go wrong everywhere.’

Is that already a real risk?

'I think we are already there. A lot of AI not available on the free market is already being used in hospitals. The focus is on efficiency in healthcare, and you can express that in euros. But it is also important to consider the patient. Healthcare should not be about maximising profit but about protecting people who are already vulnerable and in need of care. The focus no longer seems to be on the patient if you start seeing AI as the solution to everything.'

What do you want your dissertation to achieve?

'I have spoken with people from the European Commission and Parliament. I noticed that they were not aware of the effect of AI on healthcare. There is a lot of talk about ChatGPT and self-driving cars, but not much about this. I want to see more awareness and better consideration of how to protect human rights in this area. The EU wants to regulate AI products but must not forget the rights of the people who use these products.’