AI chatbots evaluated as adjuncts for endodontic education in simulated oral-board format

Introduction / background

A study reported in the Journal of Endodontics examined whether large language model (LLM) chatbots can support clinical reasoning and treatment‑planning skills in endodontics. The investigation, led by Dr Poorya Jalali (clinical associate professor and director of graduate endodontics, Texas A&M College of Dentistry), assessed two AI systems—GPT‑4o and Gemini 2.5 Pro—in a simulated American Board of Endodontics (ABE) oral examination format. The Dental Tribune International report summarising the work was published on 5 May 2026.

What was studied

• Two LLM chatbots (GPT‑4o and Gemini 2.5 Pro) were tested using a simulated ABE oral exam prepared by two board‑certified endodontists.
• The exam comprised three distinct clinical cases; for each case the chatbots received a detailed patient profile and descriptions of radiographic findings, then were asked 20 open‑ended questions.
• Responses were independently evaluated by the same two examiners for clinical validity and for the accuracy and relevance of any supporting references. Each answer also received an overall performance score on a 0–3 scale.
• The study title is “Artificial intelligence chatbots taking American Board of Endodontics simulated oral board examination,” and it was published online 26 February 2026 in the Journal of Endodontics.

Key findings

• Both chatbots produced responses that were largely rated acceptable to excellent.
• On the 0–3 overall performance scale, Gemini 2.5 Pro achieved a mean score of 2.83 and GPT‑4o a mean score of 2.73.
• After statistical analysis accounting for variation by case, individual questions and examiners, there was no significant difference between the two models in clinical validity or overall performance.
• The investigators observed model‑specific behaviour: GPT‑4o’s performance appeared to vary more by case type, whereas Gemini 2.5 Pro was more consistent across the three scenarios.
• As Dr Jalali summarised: “This suggests that current models can demonstrate structured clinical reasoning and answer oral board‑style questions at a level comparable to advanced trainees.”

Relevance for dental practice

The authors and commentators in the report framed the chatbots as potential educational adjuncts rather than clinical decision‑makers. Possible applications identified include:

• Supplementing interactive learning and self‑directed practice of board‑style oral questioning for students and residents;
• Providing a comparative reference for trainees to test clinical reasoning and receive model answers; and
• Supporting exam development—Dr Jalali noted ongoing, promising pilot work on whether LLMs can assist in designing high‑quality examination questions.

Dr Jalali emphasised that, when used appropriately and under human supervision, LLMs could help students prepare for examinations and encourage more critical engagement with clinical scenarios.

Limitations and context

• The chatbots were supplied with written prompts and radiographic descriptions; they did not perform live, timed interactions, independent radiographic interpretation, or clinical examinations—components of a real ABE oral examination.
• The systems cannot perform clinical tests, physically examine patients or independently interpret radiographs in a real clinical setting; their outputs depend on the completeness and accuracy of the information provided to them.
• The authors cautioned against over‑interpreting the results: good performance in a simulated, written scenario does not equate to reliably passing an actual oral board exam or replacing human expertise.

SOURCE

https://www.dental-tribune.com/news/ai-chatbots-show-promise-as-educational-adjuncts-in-endodontics/