Domestic neural network passed the doctor’s exam

GigaChat passed the exam at the medical institute after six months of study

One of the first artificial intelligences to pass an exam at an institute, and not just a simple one, but a medical one, was Sber’s GigaChat neural network model.

To pass the exam of a higher medical institution in the field of general medicine, which is necessary to obtain the qualification of a general practitioner, GigaChat was trained for six months. Specialists from the Sberbank Health Industry Center, National Medical Research Center named after. V. A. Almazova and the GigaChat development teams uploaded 42 GB of specialized information into the AI, including educational materials recommended for teaching students at medical universities in Russia.

After this, the AI, like any student, passed an oral exam on the ticket and testing. The oral exam consisted of three situational tasks – in therapy, surgery, obstetrics and gynecology – and 3-5 questions for them (“indicate the expected diagnosis”, “make a treatment plan”, “prescribe additional examinations”, and so on). And the testing included 100 questions. AI scored 82% with a passing threshold of 70%. The final grade for the tests passed is 4. The exam was taken by a commission of professors of therapy, surgery, obstetrics and gynecology from the Institute of Medical Education of the National Medical Research Center named after. V. A. Almazova.

“Our GigaChat neural network model is developing very quickly, mastering new areas of knowledge. We and our partners across the country will continue to develop digital solutions and technologies for medicine and health. Today I would like to acknowledge the role and thank the employees of the Almazov Center, who provide control over the training of the model and its validation. The first stage – passing the doctor’s exam – has been completed. There are new tasks ahead and the discovery of new applied solutions based on existing capabilities,” said Sergei Zhdanov, director of the Sberbank Health Industry Center.

Zhdanov added that in the future, the model can become the basis for creating a doctor and patient assistant, provide new conditions for care and knowledge about their health for each person and become a significant help for the clinician in his daily practice. The use of large language models and their successors will be one of the key technologies for the development of human-centric healthcare. However, the model is not a real doctor; in any case, recommendations received from her must be approved by the attending physician.

“The project to teach the large language model GigaChat medical knowledge at the graduate level of a medical university has become a great challenge for the Almazov Center. Several hundred teachers and researchers are involved in the project. Residents and students actively joined in the work. We are satisfied with the current results and model training will continue. Already, together with Sber, we have planned a whole line of application solutions for medical institutions, patients and doctors based on GigaChat, the development of which will begin this year. I express my sincere gratitude and gratitude to the management of Sberbank for the trust that was placed in our Center when choosing us as a partner in such an ambitious and socially significant project! Russian Cardiological Society Evgeniy Shlyakhto.

Photo: Sber press service