An ageing population places increasing pressure on medical care. Can AI help as a first “point of call?

The increasing number of old people is creating demand for medical care. There were 370m visits to general practitioners (GPs) alone last year. That was for all age groups. Older people visit their doctors more often. A person over 75 will see the doctor nearly 50% more often than someone aged 16 to 24. Some reports suggest that this is an underestimate, and a doubling of visits is a more likely number. Each visit by an older person is more complex. The number of chronic conditions goes up with age. Multiple conditions mean multiple drugs and a more complex prescribing problem.

There are other factors driving doctor visits and the length of each visit. Older people may also suffer from cognitive impairment. This makes diagnosis even more difficult. Older people are often lonely. A visit to the doctor can provide social contact irrespective of need. The result is pressure on doctors and particularly primary care doctors like GPs.

A study by the Office of National Statistics looked at GPs caseloads. There was a clear link between the patients' average age and the number of patients per GP. The practices with the highest number of people over 65 could only manage 2000. Practices with the youngest patient lists can have 2,600 patients for each qualified GP. The heaviest users of GP’s account for a disproportionate number of visits. The top 10% of patients account for 40% of visits. These users are more likely to use face to face meetings with their GP’s. They are less likely to use “remote” services. How many are older?

Are AI Models the Answer?

US data show that 15% of people are using ChatGPT for medical diagnosis and advice. Can AI at the beginning of process help relieve pressure on GPs and hospitals? Can those large language models help people decide if they need medical help? Do they need to make a doctor’s appointment or just stay in bed? Is it urgent for them to see a doctor? Is it important enough to go to A&E at the hospital or even call ambulance?

The Problem is the People

A recent study is pessimistic about the opportunity. The problem is the people, not the model. They had doctors prepare ten detailed scenarios of individuals with symptoms. They were arranged in increasing order of severity. At the top were a patient with symptoms of pulmonary embolism and another with pneumonia. They should be calling an ambulance. There were two for each of five levels of possible action from “call an ambulance” to “go to bed and self-treat”. Each scenario described the individual, for example: “a 26 year old female”. It described the detailed symptoms in a paragraph of text. It described the general lifestyle of the individual and any past medical history.

They recruited samples of people representative of the UK population. They were assigned to four groups. Three groups were each given a different AI model. The fourth group were a “control”. All groups were given two of the scenarios and asked questions on behalf of the person described. “Which of the five actions would they take? Why would they call an ambulance? What medical condition did they suspect? They were to use their AI to help. The control group was to use their “normal approach” which tended to be a web search.

The AIs’ Answer.

They first fed the full scenarios into each AI to test their effectiveness. They ran it 60 times and looked at how good the models were. Each model was able to suggest one relevant condition in between 95% and 99% of cases. Their recommended actions were correct in 65% of cases for ChatGPT. The others did slightly less well. This would compare to a “guessed” score of 20% since there were 5 alternatives.

The People Ruined Everything

Each respondent interacted with their AI as if they were the person in the scenario. They were looking for a suggested course of action and a preliminary diagnosis. The results were poor. In fact, the best performing group was the “control” group. The AI did not help at all. Many of the hypothetical patients did not go to hospital when they should. Some went to hospital when they should have made a regular appointment with their GP. In general, there was a tendency to underestimate the severity of the problem.

Analysis of each interaction showed the problem. Many people provided the LLMs with incomplete information when prompted. Even though they were given detailed scenarios. In many cases the LLMs were offering the correct recommended action. They were offering potentially correct diagnoses. Sadly, the respondents ignored the Doc GPT advice when answering the questions.

Doc GPT and the Older Population.

The answers you get from an AI depend on the quality of your prompt. This study is a clear example. Patients know more about their symptoms than the doctor. The problem is to elicit that information from them during the diagnosis. The interpersonal skills of the doctor are key. This is likely to be more of an issue the older and frailer the patient gets. The user interface of these AI models needs to be more subtle. This change might help reduce the demand caused by the aging population.

“The doctor knows best” is no longer a given. People can and do argue with the doctor about their diagnosis. Doc-GPT has the same problem.