eP744 - Examining the Validity of Training Simulator for a Novel Miniaturized Robotic Assisted Surgery System

Purpose/Background: Improving technical skill is critical in surgical education and training programs for robotic assisted surgical systems. Robotic simulation training allows surgeons to improve technical skills and expand robotic experience without subjecting patients to unnecessary risk. The aim of this study is to examine the face (realism to the user), content (usefulness as a training tool), and construct (ability to differentiate levels of user experience) validations of a simulator developed for a novel miniaturized robotic assisted surgical device (mRASD).Methods/Interventions: Thirty-six participants were enrolled in this prospective study. They were classified as Novices (subjects with a college degree or currently enrolled in a higher education program with no previous robotic experience or training) or Experts (general surgeons with >100 robotic cases). All participants received approximately 1 hour of training involving a system overview followed by hands-on use of the equipment. Participants were allowed to complete one practice session on the simulator. After a 24-hour training decay, participants completed 5 rounds of 5 basic exercises on the simulator in a single session. Exercises included: (1) Camera Positioning, (2) Basic Movement: Jacks Pick and Place, (3) Multi-Quadrant: Rings Peg Board, (4) Coordinated Tool Motion: Rope Walk, and (5) Basic Cautery Use: Energy Application. Performance metrics, captured by the simulator software for each exercise, were used to compare the outcomes of each group and evaluate the construct validation. The metrics analyzed were: (1) time to complete, (2) economy of motion, (3) instrument collisions, (4) excessive instrument force, (5) instruments out of view, (6) master workspace, (7) drops, and (8) misapplied energy time. To assess face and content validity, each participant completed a questionnaire (using a 5-point Likert Scale).Results/Outcomes: There were 30 Novices (14 females and 16 males), and 93% of Novices had some video gaming experience. There were 6 males in the Expert group with robotic surgery experience. The mean age was 25.8 ± 10.4 years for Novices and 55.2 ± 8.6 for Experts. All participants were right-handed. The face and content validity were rated high: 4.44/5 realism by the Novice group and 4.83 by the Expert group, and 100% of the Experts found the simulator to be highly relevant (5/5). Significant differences were observed in favor of the Expert group in 3 of the 8 metrics (time to complete, economy of motion, and drops). For Pass/Fail proficiency levels, where the proficiency baseline was defined as the Mean Expert Score + 2*SD, 43% to 67% of Novices reached proficiency levels in the safety performance metrics (Instrument Collisions and Drops respectively).Conclusion/

Discussion: The study identified several relevant performance metrics for five exercises that could be used to assess robotic surgical skills and supports the face, content, and construct validities of using the mRASD simulator for training.