INTERNATIONAL SCIENTIFIC RESEARCH AND INNOVATION CONGRESS -II
RELIABILITY OF CHATGPT IN PREPARING FOR THE RESIDENCY TRAINING PROGRESS AND COMPETENCY EXAMS IN ORTHOPEDICS AND TRAUMATOLOGY
Objectives: In this study, “‘Residency Training Progress Examination (UEGS)” and “Competency Exam” exam questions of different years prepared by the Turkish Society of Orthopedics and Traumatology (TOTBİD) were asked to the ChatGPT 3.5 model; according to the answers given, it was questioned whether these artificial intelligence models can be trusted in preparation for these exams. Methods: The study was conducted with the UEGS exam questions between 2012 and 2023 (except 2020), which were accessed from the official website of TOTBID, and the Competency Exam questions between 2014 and 2023, which were also accessed from the official website of TOTBID. The answers given by ChatGPT and the answers given by the residents and specialists were statistically evaluated according to subcategories. Results: While ChatGPT gave the most correct answers in the General Orthopedics category with 61% in UEGS, this rate was the lowest in the Adult Reconstructive Surgery category with 39%. It was observed that ChatGPT gave significantly more correct answers than the average of the participants only in the Spine Surgery category in UEGS exams. In the Specialist Competency Exam, ChatGPT gave the most correct answers in the Pediatric Orthopedics category with 68% and the least correct answers in the Infection and Tumors category with 46%. In 4 out of the 10 exams, ChatGPT passed the exam with 60 or more correct answers in 100 questions, while in the other 6 exams, ChatGPT could not reach the required number of correct answers to pass the exam with less than 60 correct answers. Conclusion: The use of ChatGPT offers several advantages for both students and educators. However, it should not be forgotten that these systems have not reached perfection and their evolutionary process continues. These systems still have inadequacies and deficiencies in terms of Orthopedics and Traumatology theoretical education.
