A study led by the University of Cambridge has found that the clinical knowledge and reasoning skills of GPT-4, a large language model, are approaching the level of specialist eye doctors. GPT-4 was tested against doctors at different stages in their careers and scored significantly better than unspecialised junior doctors. It also gained similar scores to trainee and expert eye doctors, although the top-performing doctors scored higher. The researchers believe that large language models like GPT-4 could be useful in providing eye-related advice, diagnosis, and management suggestions in well-controlled contexts where access to specialist healthcare professionals is limited.
The researchers suggest that large language models like GPT-4 could be deployed in triaging patients with eye issues to determine which cases are emergencies requiring immediate specialist attention, which can be seen by a general practitioner, and which do not need treatment. This has the potential to improve healthcare and reduce waiting times for eye care. Large volumes of clinical text are needed to fine-tune and develop these models, and work is ongoing around the world to facilitate this. The study compared the abilities of AI models to practicing doctors to provide a fair comparison and characterise the capabilities and limitations of commercially available models.
The study included questions about a wide range of eye problems taken from a textbook used to test trainee eye doctors, which is not freely available on the internet. The results of the study are published in the journal PLOS Digital Health. The researchers emphasize that doctors will continue to be in charge of patient care, but it is important to empower patients to decide whether they want to involve computer systems in their care. GPT-4, along with GPT-3.5 and other large language models, is trained on datasets containing hundreds of billions of words from articles, books, and internet sources. These models have the potential to provide more accurate responses than human doctors in certain scenarios.
GPT-4 powers the online chatbot ChatGPT, which has attracted significant attention in the medical field for its passing level performance in medical school examinations and its accurate and empathetic responses to patient queries. Since the study was conducted, more advanced models have been released, which may be even closer to the level of expert eye doctors. The researchers believe that large language models like GPT-4 could be valuable in providing eye-related advice, diagnosis, and management suggestions in a range of healthcare settings where access to specialist healthcare professionals is limited.
In conclusion, the study demonstrates that GPT-4 and other large language models have the potential to improve healthcare by providing accurate advice and diagnoses for a range of medical conditions. While doctors will continue to play a crucial role in patient care, the integration of AI models like GPT-4 can enhance the clinical workflow and improve patient outcomes. As the field of artificially intelligent large language models continues to evolve rapidly, it is important to consider how these technologies can be effectively integrated into healthcare systems to benefit patients and healthcare professionals.
