Benjamin W. Cooper, MMS [1], Nicholas A. Jaeger, BS [1], Maureen A. Hirthler, MD, MFA [1], and Cathy J. King, DNP, RN, CNE [2]
1 -Lake Erie College of Osteopathic Medicine
2 -State College of Florida, Manatee-Sarasota
Correspondence should be addressed to B.C. (benjamin.william.cooper@gmail.com)
Clay illustration by Lily Offit; Photographed by Ben Denzer
ABSTRACT
Introduction: Simulated patient encounters in preclinical medical education are being implemented at various medical schools throughout the United States. In this study, we aimed to evaluate the perceived effectiveness of early preclinical simulations on medical students’ education.
Methods: Two classes of osteopathic medical students, class of 2019 and class of 2020, participated in two simulated clinical experiences using high-fidelity patient mannequins and standardized family members during their second year of medical school. These same students, who are now completing their clinical rotations, completed a survey to assess if the experience with a simulated patient/family
Results: A total of 379 students were emailed a survey. Out of 379 total students, 122 surveys were completed (59 from the Class of 2019, 63 from the Class of 2020). The overall response rate was 32%. A majority of the respondents responded positively, with 63% of students stating in their free responses that the simulated patient encounters were experiences that improved their ability to interact professionally and appropriately with patients and family members. P-values between the classes for each survey question was greater than 0.05, indicating that there were no statistically significant differences between the classes of 2019 and 2020 in their survey responses.
Conclusion: This study provides evidence that early simulated patient encounters provide a self-reported experience that improves empathy and communication for the students during their clinical rotations.
INTRODUCTION
Simulated patient encounters in preclinical medical education are being implemented at various medical schools throughout the United States [1]. Osteopathic Medical school education is largely driven by standards created from the National Board of Osteopathic Medical Examiners. A portion of the preclinical standards are based on the humanistic domain of medicine, which includes communication, interpersonal skills, and professionalism [2]. All osteopathic medical students are expected to pass COMLEX-USA Level 2 PE during their third year of medical school. This standardized test assesses whether graduating medical students are proficient in their clinical decision-making skills, and in their ability to communicate effectively with patients and their families as well as the entire healthcare team. According to multiple studies, school-based performance measures during preclinical years are a reliable tool for predicting performance on COMLEX-USA Level 2 PE and rotations [3, 4]. These studies show that experiences during the preclinical years have a notable correlation with future performance.
Medical school curricula are constantly being examined to ensure students can be both successful on these examinations and in their future career as practicing physicians. It has been well established that, “assessments drive teaching and learning.” Patient and family surveys routinely find that patients want better communication from their doctors, and simulations are one way to foster this skill [5, 6].
Simulation encounters occurring during the preclinical years of medical school provide skills that assist in the success of students during their clinical years, such as communication and teamwork [7]. These skills should continue to assist them as resident physicians. One study concluded that resident communication skills training that focused on family system theory resulted in increased confidence and skills for residents, acting as participants, in communicating difficult information to their patients and family members [8]. Family system theory, applied to medicine, is the idea that medical professionals are not only interacting with patients to deliver difficult news, but they also must interact with patients' family members in a family-centered approach.
In the first phase of this study, second year medical students from the Lake Erie College of Osteopathic Medicine in Bradenton, Fl. partnered with RN-BSN nursing students from the State College of Florida, Manatee-Sarasota (SCF) to participate in scenarios involving high-fidelity simulators and standardized patient families. The purpose of these encounters was to foster collaboration and communication skills between members of healthcare staff, as well as communication with patients and their families.
This study was performed to investigate the benefits perceived by clinical students of the early preclinical mannequin simulations with standardized patients as families. The goal of this experience was to prepare students for interactions with actual patients, staff, and family members during their clinical years [9].
To our knowledge, this is the first study that attempts to analyze how third- and fourth-year medical students compare clinical medicine in their clerkship years with the simulations they experienced in their preclinical years. This study expanded upon our previously published work [7].
METHODS
Participants: This retrospective, non-experimental, mixed-methods study surveyed 196 students in the LECOM-B Class of 2019 (3rd year students) and 183 students in the class of 2020 (4th year students). All these students participated in two simulated clinical experiences with RN-BSN students, human patient simulators, and standardized patients as family members. The students were scheduled for these experiences once during the fall semester and another during the spring semester of their second year of medical school. The students in the Class of 2019 and 2020 engaged in these encounters during the 2016-2017 and 2017-2018 years, respectively.
Procedure: Students completed the simulated clinical experiences in the Simulator Center at the SCF. The Simulation Center is set up to reflect an intensive care unit, with nine patient rooms positioned around a large nursing station. The patient rooms contained a hospital bed, cardiac monitor, IV pole with medications hanging and a medical supply cart. A high-fidelity human patient simulator was used. Nursing faculty controlled the mannequin from a remote location. Alterations in vital signs and voice responses were used in the simulation scenario.
Two medical students were paired with one RN-BSN nursing student. The medical students were given a brief bedside report from the nurse on the patient and were then asked to complete a full medical examination of the standardized patient. Upon completion of the physical assessment and huddling with the nurse, they were directed to discuss the status of the patient with two of the patient’s family members, who were standardized role-players. The actors were specifically scripted to act out a stressful scenario. Students were expected to follow the SPIKES 10 protocol in relaying difficult news to the patient’s portrayed family members.
At the end of the encounter, video debriefing occurred with everyone involved with the medical students, the RN-BSN nursing student, and LECOM-B clinical professors. Throughout the debriefing sessions, students were provided constructive feedback of the encounter. Students were asked specific questions to evaluate areas for improvement and the medical students were asked to reflect on their overall experience. SCF, LECOM-B faculty, and the simulated family members provided written evaluations assessing each student on their patient bedside physical examination, family member encounter, and professionalism. These evaluations were not analyzed for this study, as it was primarily to provide students with immediate feedback and show them areas for improvement. All students were exposed to the same level of acuity in scenarios and the same group of patient families throughout all simulated clinical experiences.
Students in their clinical years, who had previously participated in the simulations, were then surveyed regarding their experiences. A 10- item survey was created using SurveyMonkey®, which was sent to the Classes of 2019 and 2020 on November 1st, 2018. The Class of 2019 submitted responses for the simulation experience, which occurred about 18 to 22 months prior to being surveyed. The Class of 2020 had their simulation approximately 7 to 11 months prior to being surveyed. We had an interest in evaluating differences, if any, between these two classes.
Reminder emails were sent on November 15, 2018, and January 7th, 2019 to encourage completion of the survey. All responses were anonymous and confidential. Seven of the survey items used a 5-point Likert-type scale ranging from five points to one point. Two of the survey questions allowed for written responses. The responses to these open-ended questions were then manually tallied. The study was approved by the Lake Erie College of Osteopathic Medicine’s Institutional Review Board, which waived the need for informed consent.
Statistical Analysis: SPSS software (IBM) was used to analyze the data. Responses were compared between the two medical student cohorts using a Mann-Whitney U Test. A non-parametric test was chosen because the data was assumed to not have a normal distribution. The sample means came from the same population. A 2-tailed test was confirmed. Statistical significance was defined as a P<0.05.
RESULTS
A total of 379 students were emailed for this study. Out of 379 total students, 122 surveys were completed (59 from the Class of 2019, 63 from the Class of 2020). The overall response rate was 32%. The responses between the two cohorts to all the questions were not statistically significant.
A majority of the respondents from both cohorts (Class of 2019 and Class of 2020) agreed with the survey responses that scored their responses somewhat valuable, valuable, or extremely valuable. The only survey item that didn’t receive over 50% positive responses was whether the students would want to repeat the simulated patient encounters during 3rd year while on rotations. The classes of 2019 and 2020 scored the simulation as being most valuable for their Internal Medicine rotation, with 40.38% and 48.21% responding in the affirmative, respectively.
There were a total of 56 free responses detailing the students' perception of the simulation experience and its impact on their clinical rotations. The overall tone of the response was evaluated. The free responses for both classes were overall positive; however, there were negative free responses (negative responses made up 37% of the free responses). There were 13 negative free responses for the Class of 2019 and there were 8 negatives for the Class of 2020.
One student in the Class of 2020 commented “Very poor translation between the simulation and real-world experiences. Difficult to act “natural” when being recorded and judged on every statement.” Some of the students’ comments seemed to be contradictory. Another student, who was from the Class of 2019, said “I feel the pattern of two experiences was somewhat contrived, but re-watching video of my own body language was the most helpful.” One student noted, “I thought it was a great experience during second year. I feel that during third year I would much rather have the true patient experiences than be taken away from my rotation for a simulation, especially being so far away. I am very happy I had the chance to experience the simulation during 2nd year and would recommend it.” A few students agreed that these experiences were helpful, but still not realistic, with comments such as “It's really hard to simulate the same emotions in real life. The actors were good, but it is just not the same. Plus, these situations need to be practiced over and over. I'm not sure it's worth it to do it just one time.”
However, other students felt the simulated encounters were not realistic at all, and that these skills must be learned through real life scenarios, making remarks such as “not worth the time” and “practicing on plastic does not help”, indicating a minority of the students felt this experience was not valuable to them as future clinicians.
Figure 1: Inter-professional students acting as physician and nurse
Table 1: Raw dataset showing numbers and percentages for each of the surveyed questions
DISCUSSION
The studies mentioned throughout this paper show that establishing relationships, displaying empathy, and communicating effectively are necessary in order to provide the best care possible. Other studies have shown that clinical performance in the 4th year of medical school affects performance in residency [11]. Therefore, methods to instill and foster these skills should be implemented earlier in medical schools across the country. The NBOME has taken note of the importance of such skills, and therefore now includes a humanistic domain in the COMLEX-USA Level 2 PE which measures skills in doctor-patient communication, interpersonal skills, and professionalism [2]. Our results show that students find simulated encounters to be beneficial during their clinical rotations. According to the survey responses in this study, simulated encounters were more beneficial for some rotations, like Internal Medicine, rather than other rotations. When asked which rotation benefited the most from these encounters, 44/85 of students selected IM rotations. In reviewing the written responses, many students commented that the simulated encounters were overall helpful to their education, however, the simulations will never be able to replace real clinical encounters. Since the simulated encounters take place during the second year of medical school before clinical encounters, these encounters may help build the foundation for third year medical students and beyond.
The Interprofessional Education Collaborative identifies four core competencies for effective collaboration including: 1) values/ethics, 2) roles/responsibilities, 3) interprofessional communication, and 4) interprofessional teamwork and team-based care 12. The objectives of implementing the simulated experience were to allow students to improve in each of these four areas. Also, integrating nursing students with medical students earlier in their careers could allow for improved interprofessional healthcare collaboration throughout residency and their careers. Multiple studies have shown the benefits of physicians being able to connect with patients and their families. Duffy et al. demonstrated that “professional conversation between patients and doctors shapes diagnosis, initiates therapy, and establishes a caring relationship” [5]. Karkowsky and Chazotte showed that effective communication in a medical setting can improve patient adherence, improve patient outcomes, and lower risk of litigation [9]. Also, Zachariae et al. showed that communication ratings correlated with reduced emotional distress in cancer patients [13].
Throughout the high-fidelity simulations, the medical students were recorded so that they would be able to review their ability to effectively communicate in difficult scenarios. They were then able to share their opinions of the simulation through a free response question in our survey. The idea of not being able to act “natural” when being video recorded and evaluated was mentioned in several written responses. These comments raise concern that the recordings of students during the simulations could have had an effect on how they delivered the news to the families. However, rewatching video playback of the student’ simulation encounter has been shown to be beneficial in clinical research. As noted in M. Bussard’s research on nursing students, who were video recorded using high-fidelity mannequins to assess their preclinical judgement, the study found that students can improve their confidence, communication and decision making, and can improve their clinical practice using video recordings [11]. One student in our study voiced that it is hard to treat simulations like real life, and that it may take some time to get used to simulations. Comments like these highlight the need for future studies to evaluate whether simulated encounters improve students’ performance in the hospital using objective measures.
The results of this study were based on responses from the students, which may be ineffective at measuring the usefulness of the simulated encounters. Since this survey was voluntary, this study only included self-selected students, rather than all students that participated in the simulated encounters. Students may have answered the survey negatively because the encounter was difficult for them, even though it was useful. For the Class of 2020, at the time of the survey, the students only had about 6 months of rotation experience. Because of this, some of the students may not have had the opportunity or as many opportunities due to the configuration of their rotation schedules (for example, not every student may have had an internal medicine rotation at the time of the survey, which, according to the survey data, was the in which the simulated encounters were most helpful). Additionally, the students’ responses to the survey were collected more than 6 months after the simulated encounters, which may lead to inaccurate recall of the utility of the simulated encounters. The students had limited participation in only two simulated encounters with mannequins (one during each semester in their second year of medical school). Lastly, the response rate was 32% (122/379), which may not fully represent the opinions of all students, however, the results between the two classes were consistent.
Our study measured how the medical students reacted to the training using mannequins and simulated encounters (Kirkpatrick Level 1). Future studies should aim to assess the impact of the simulated encounters by measuring changes in behavior (Kirkpatrick level 3) [14]. One possible scenario would be to survey the patients and patients’ families of the medical students who did have simulated mannequin training, as well as the patients and patients’ families of medical students who did not have this training. We were unable to do this since all LECOM-Bradenton students receive the same simulated encounters using mannequins. Also, future studies could increase the number of encounters to allow the students to become more familiar with the mannequin technology. It might be beneficial to not have video recorded encounters to better simulate in-hospital rotation experience and allow the students to act more naturally. A longer follow-up study to assess these students’ perceptions once they are practicing residents might also be useful information. Future studies should also validate that 3rd and 4th years students that are being surveyed have had the opportunity to implement the skills that are emphasized during the simulation training during first and second year.
REFERENCES
Underman K. Playing doctor: Simulation in medical school as affective practice. Soc Sci Med. 2015; 136-137:180-188.
Scoring Principles. https://www.nbome.org/exams-assessments/comlex-usa/comlex-usa-level-2-pe/scores-transcripts/scoring-principles/. Accessed October 10, 2018.
Baker HH, Cope MK, Adelman MD, Schuler S, Foster RW, Gimpel JR. Relationships between scores on the COMLEX-USA Level 2-Performance Evaluation and selected school-based performance measures. J Am Osteopath Assoc. 2006; 106(5):290-295.
Cope MK, Baker HH, Foster RW, Boisvert CS. Relationships between clinical rotation subscores, COMLEX-USA examination results, and school-based performance measures. J Am Osteopath Assoc. 2007; 107(11):502-510.
Duffy FD, Gordon GH, Whelan G, et al. Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Acad Med. 2004; 79(6):495-507.
Williams-Reade J, Lobo E, Whittemore AA, Parra L, Baerg J. Enhancing residents' compassionate communication to family members: A family systems breaking bad news simulation. Fam Syst Health. 2018; 36(4):523-527.
Carpenter AM, Hirthler MA, King CJ. Interprofessional Collaborative Practice: Use of Simulated Clinical Experiences in Medical Education. J Am Osteopath Assoc. 2018;118(4):235-242.
Williams-Reade J, Lobo E, Whittemore AA, Parra L, Baerg J. Enhancing residents' compassionate communication to family members: A family systems breaking bad news simulation. Fam Syst Health. 2018; 36(4):523-527.
Karkowsky CE, Chazotte C. Simulation: improving communication with patients. Semin Perinatol. 2013;37(3):157-160.
Rosenzweig MQ. Breaking bad news: a guide for effective and empathetic communication. Nurse Pract. 2012; 37(2):1-4.
Bussard ME. Self-Reflection of Video-Recorded High-Fidelity Simulations and Development of Clinical Judgment. J Nurs Educ. 2016; 55(9):522-527.
Schmitt M, Blue A, Aschenbrener CA, Viggiano TR. Core competencies for interprofessional collaborative practice: reforming health care by transforming health professionals' education. Acad Med. 2011; 86(11):1351.
Zachariae R, Pedersen CG, Jensen AB, Ehrnrooth E, Rossen PB, von der Maase H. Association of perceived physician communication style with patient satisfaction, distress, cancer-related self-efficacy, and perceived control over the disease. Br J Cancer. 2003; 88(5):658-665.
Kurt, S. (2018, September 06). Kirkpatrick Model: Four Levels of Learning Evaluation. Retrieved October 10, 2018, from https://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/