Veronique Cayol, a Parisian gynecologist, is experiencing ChatGPT-4 for the first time.

She explains that, whether she sees patients over video conferencing or examines them in person, an AI program called Copilot that she is currently testing can transcribe her dialogue with the patient word-for-word, then condense and arrange it into symptoms, past medical history, diagnosis outcome, and prescription contents by column. Once she confirms the contents, it is uploaded into the digital medical records system.

Copilot’s ability to condense medical histories is derived from the large-scale linguistic model, ChatGPT-4.

Cayol observes that, due to a manpower shortage of physicians in France, patient waiting times are lengthy, sometimes even forcing them to go to other doctors. Consequently, more and more patients are asking doctors to provide them with their medical records to facilitate other physicians taking over their care. The busy Doctor Cayol thus turned to Copilot to compile medical records.

“The best part is that I can concentrate on seeing the patient, freeing me from having to type out a record on the computer,” she says.

Interestingly, the concept for this hot medical software program, recently featured in a New York Times article, was born in southern Taiwan, at National Chung Cheng University in Chiayi.

Chung Cheng University's sole foreign student

Alexandre Lebrun (Source: Alexandre Lebrun)

Alexandre Lebrun, co-founder of Paris-based startup Nabla, Copilot’s mother company, holds a Master’s degree from Télécom Paris.

A specialist in recognition science and software engineering, he is also a student of Chinese, and was National Chung Cheng University’s only foreign student when he first arrived in 2000.

Over the past 23 years, all of Lebrun’s three business ventures have been about creating virtual assistants in different fields.

This idea was first planted in his mind during the six months Lebrun was studying in Chiayi. On his own in Taiwan, the lonely student mused about how great it could be if machines could converse with humans. After returning to France, he immediately set about making this concept into reality.

His first venture was VirtuOz, a commercial version of Apple’s Siri, which ended up getting absorbed by Microsoft. The second time, the AI natural language interface company he built was acquired by Facebook. He then entered Facebook’s AI research department, where he worked with head AI scientist Yann LeCun. This leading master of AI subsequently became a consultant and investor in Lubrun’s third company, Nabla, focusing on AI applications in the medical field. Most of the team members previously worked for Facebook.

“The issue we want to resolve is a lack of doctor manpower,” explains Lebrun. As societies age, the demand for chronic illness care has followed. He wants to help doctors save time from writing patient medical histories and insurance claims each day.

French regulations require first seeing a family doctor before going to someone else for medical care. Early this year, French President Macron mentioned in a speech that there were over 600,000 chronically ill people in France who cannot wait for an appointment with a family practitioner.

Since Copilot went online, several hundred doctors in the United States and France have put it to use, and Europe’s largest private medical group is also planning to adopt it. Pricing is currently a US$119 monthly subscription for individual practitioners, and US$0.50 per patient usage for large hospitals.

Writing medical histories is just one of the popular medical applications that ChatGPT-4 is being tasked to handle.

Is there a way for Taiwan to share in the commercial opportunities that Lubrun sees in medical records?

The most obvious challenge is that Taiwanese doctors regularly use Mandarin and Taiwanese when conversing with patients, while notating their medical histories in English. Dr. Chang-Fu Kuo, director of the Center for Artificial Intelligence in Medicine at Chang Gung Memorial Hospital, recognizes that this language gap is a major issue for AI models, but that “it will eventually be resolved.”

Dr. Chang-Fu Kuo (Source: Chien-Tong Wang)

Dr. Kuo observes that research emerged as early as 2020 about leveraging AI to help produce medical histories. In the U.S. many doctors do not write down medical histories, turning to specialized medical transcriptionists who transcribe from recordings. Accordingly, experiments with using AI in their place “have long been attempted; it’s just a question of how good the results are.”

“Results” refers to accuracy. GPT is most capable of impacting the quality of medical care, causing “illusions” by generating erroneous information, as there are numerous cases of data seemingly generated out of thin air. “The future application of GPT-4 should include mechanisms to address illusory, missing, and erroneous data.”

This is currently the greatest flaw with generative AI technology, and Lebrun is keenly aware of it.

He explains that, at first, the large-scale language model used by Copilot did not include GPT. However, he found that GPT’s attention mechanism resolved the issues faced at the time.

Put simply, the attention mechanism assigns data input different weighting, making it easier to read context and match up key information.

Lebrun gives an example whereby the physician asks the patient, “Are you allergic to honey?” The patient immediately answers no, but then after some thought recalls a sensitivity to honey. Copilot is able to successfully recognize a conversation structure such as this, thanks to this mechanism.

Cayol corroborates this, saying that even after a nearly 20-minute doctor-patient conversation, “it’s incredible that Copilot doesn’t include smalltalk about shopping and children in the medical record.” And that the final highlighted contents “rarely” require correction.

In order to help GPT accurately read medical terminology, with both doctors’ and patients’ permission, Nabla collected recordings of 30,000 private clinical consultation dialogues as training materials. Each time a physician gave feedback and corrected the contents, the Nabla team would make adjustments accordingly, raising accuracy to 99 percent.

Taiwanese version ready to go online

Dr. Kai-Cheng Hsu, Director of the Artificial Intelligence Center (AIC) for Medical Diagnosis at the China Medical University Hospital (CMUH), explains that the “Hsiao Nan Nursing Audio Assistant” compiled over 1,000 Chinese and English audio datasets, as well as actual nurse shift changeover audio records as training data to present to Whisper AI to conduct micro adjustments, resulting in an error rate of approximately seven percent.

He and his team are currently using Whisper AI in tandem with GPT-4 to arrange medical history records, and anticipate going online in another month.

(Source: Kai-Cheng Hsu)

Also a practicing neurologist, Dr. Hsu relates that initial patient consultations take at least 20 minutes. However, if they use GPT first, then a significant amount of time can be saved.

“Many doctor-patient relationships are established during conversation,” relates Hsu. With the time saved from GPT’s assistance, the doctor can engage in casual conversation with the patient without having to constantly gaze at the computer monitor. 

Even so, for GPT to formally become integrated into clinical procedures as doctors’ assistant for patient consultations, numerous reservations remain.

Article 12 of the Physicians’ Act states that practicing physicians shall keep medical records including the main complaint, diagnosis, medication, “and keep signed or sealed medical records stating year, month and date.”

(Source: Chien-Tong Huang)

Care is taken to sign off on medical records so as to better assign legal responsibility in the event of a medical dispute. Lebrun stresses that this is why, when designing Copilot, the physician must read the entire contents before confirming and uploading.

An even bigger issue is whether, in contrast to French physicians facing manpower shortages, there is sufficient incentive given Taiwan’s medical environment for Taiwanese doctors and hospitals to incorporate GPT, and if GPT, which is susceptible to illusions, can truly help physicians ease their burden and improve doctor-patient relationships. These answers will hopefully become clear in the future.

---

Have you read?

• Apps, musicians, language tutors: Taiwan’s AI visionaries 
• The story behind Taiwan’s ChatGPT 
• ChatGPT servers powered by Taiwan's Lite-On

Translated by David Toman
Edited by TC Lin
Uploaded by Ian Huang