The last decade has witnessed exponential growth in the field of endovascular interventions, although only in the last few years has there been a widespread interest in the carotid artery stent (CAS). Endovascular physicians with different medical backgrounds such as interventional cardiologists, radiologists, and vascular surgeons all recognize the importance of this changing technology.1 This procedure is almost unique, as the risks to the patient (stroke and death) as a result of the physician’s learning curve are unacceptably high. This has been summarized by editorials written after the publication of the Carotid and Vertebral Artery Angioplasty Study (CAVATAS)2–4 and the Endarterectomy versus Angioplasty in Patients with Symptomatic Severe Carotid Stenosis trial (EVA-3S).5–7 Recent publications of the rates of medical errors and adverse events within healthcare8 have drawn the spotlight toward methods of establishing credentials for physicians preparing to perform complex procedures. In order to improve patient safety, operators must have the appropriate cognitive and technical skills and experience of CAS. Furthermore, physicians should have previously achieved a high level of proficiency in other catheter-based interventions and completed dedicated training in CAS.9,10
Traditional methods have focused on meeting a minimum number of procedures and on the duration of training to ensure competence, inappropriately correlating experience with competence.11,12 What is needed, given the complexity and risk of CAS plus the conflicting skill sets of physicians across several subspecialties, is a standardized and objective method for assessing procedural performance. Within aviation and in other medical fields such as laparoscopy, virtual reality (VR) simulation is able to train and objectively assess technical performance, and subsequently to define the benchmark level of skill.13–15 Therefore, the three specialties involved in the treatment of CAS have joined forces and launched the European Virtual Reality Endovascular Research Team (EVEREST). The ultimate goal of this group is to improve the training of current and future endovascular therapists.
In this article, an overview of different endovascular VR simulators and validation studies will be given, highlighting the role of the EVEREST members. Additionally, the possible benefits of VR simulation in the endovascular field, integration of VR simulation within a proficiency-based endovascular curriculum, and future applications will be explained.
Virtual Reality Simulator Overview
Dawson defines a simulator as a physical object that reproduces, to a greater or lesser degree of realism, a procedure that must be learned and which incorporates a system of metrics that allows progress and learning to be recorded.16 In 2000, his article describing the first advanced vascular simulator was published.17 Now, there are at least four commercially endovascular simulators available.
The Procedicus Vascular Intervention Simulation Trainer (VIST™, Mentice, Gothenburg, Sweden) comprises a mechanical unit housed within a mannequin cover, a high-performance desktop computer, and two display screens. Modified instruments are inserted through the access port using a haptic interface device. The term haptic relates to tactile feedback, which is created by a series of motorized carts that lock onto the inserted instrument in realtime with force-feedback (i.e. mechanical simulation of the sense of touch). The physician is able to select appropriate endovascular tools and perform interventional procedures using the simulated fluoroscopic screen. The performance is measured using assessment parameters such as contrast fluid used, total procedure time, fluoroscopy time, clinical parameters (endovascular tools used, stent placement accuracy, etc.), and errors. A procedure report is provided automatically for each session. Simulation modules include atherosclerotic stenotic diseases in coronary and peripheral vessels (carotid, renal, iliac, superficial femoral artery [SFA]), over-the-wire lead placement for biventricular pacing, closure of patent foramen ovale, neuro-interventions, and insertion and retrieval of a caval filter.
The Angio Mentor™ family (Simbionix, Ohio, US) has a similar range of simulation modules in the peripheral and coronary arteries and allows implantation of cardiac pacemaker leads, the management of cardiac rhythm diseases, and the performance of neuro-interventions. Furthermore, this VR simulator has incorporated patient monitoring, drug administration, and response to physiological disturbances during the endovascular procedure and the occurrence of complications. Two cheaper and more portable editions are available: the Angio Mentor Express and the Angio Mentor Mini (see Figure 1). The SimSuite® (Medical Simulation Corporation, Colorado, US) is the largest simulator, with up to six interactive screens to facilitate multidisciplinary team training. Similarly to the Angio Mentor, adverse events and response to physiology feature substantially in the simulation. Additionally, the full process of patient care, patient history, physical assessment, and other diagnostic tests is presented, allowing the clinician to formulate a diagnosis and treatment plan. The treatment of SFA occlusions and placement of cardiac valves are now simulated by this system in addition to the existing modules.
The CathLabVR™ Surgical Simulator (formerly the Endovascular AccuTouch® System, Immersion Medical, Maryland, US) not only boasts peripheral and coronary simulation modules with metrically based assessments, but also allows you to manage cardiac rhythm diseases, replace cardiac valves, and implant cardiac pacemaker leads.
Validation of the Virtual Reality Simulators
The first flight simulator, invented in 1929 by Edward Link, was not fully adopted until nearly 15 years after it was developed. During the last 75 years, substantial amounts of money have been invested, ensuring that it became as effective and sophisticated as it is today. No controlled trials were required to prove the effectiveness of these simulators. Pioneered in the 1980s, surgical simulation is still not widely accepted, principally because of scepticism within the medical community and the lack of validation studies. Prior to assessment or training of physicians, the validity of these simulators needs to be demonstrated. It is incorrect to assume that a realistic simulation (face validity) equates to an effective training or assessment model.18 In addition to developments in the realism of VR simulation, a demonstration of reliability, feasibility, and validity is necessary (see Table 1).
So far, most papers have sought to demonstrate the face and construct validity of renal and carotid modules of the VIST simulator (see Table 2). The assessment studies carried out by the EVEREST members differed from other studies as they did not include medical students but only physicians with the appropriate medical background in treating these lesions.19,20 Furthermore, in the CAS assessment study, only experienced interventionalists were included in order to respect the guidelines of international societies who stated that CAS should be performed only by physicians who have at least acquired basic endovascular skills. Nevertheless, our study, similar to other papers, has shown that more experienced interventionalists perform a CAS procedure more quickly, press the fluoroscopy pedal less often, and perform fewer angiograms to complete the intervention.19–24 Thus, the VIST simulator can objectively differentiate between levels of CAS experience in experienced interventionalists based on these automatically recorded quantitative assessment parameters.
The difficulty is that a therapist who performs a procedure quickly and uses little radiation is not always a good interventionalist.25,26 To overcome this criticism, simulator companies assess technical performance not only using quantitative assessment parameters, but also using more clinically relevant parameters (endovascular tools, residual stenosis, stent/vessel ratio, etc.) and error scoring (movement of embolic protection device after deployment). To our knowledge, only the study from the EVEREST members has attempted to prove the construct validity of these metrics of quality of performance.20
Although most would agree that efficiency, precision, and avoidance of errors are qualities that reflect technical skill in the interventional suite, few have attempted to develop reliable and valid measurements of these attributes. Follow-up studies by the EVEREST members will focus on task analysis. The aim is to identify the task or step of the CAS procedure during which most errors are made, and to define and weight those different errors. The outcomes of both the task analysis and weighting of the errors should enhance the metrics currently available and may assist designers of simulators to implement metrics efficiently.27 Other simulator companies have incorporated similar assessment parameters to the VIST (although the definition of the assessment parameters might vary), but have also included metrics such as patient selection errors, drug administration and physiology reporting, overall management, and complications (e.g. dissections and perforations). The validity of these assessment methods are under evaluation.
Although further work is required to validate the different simulators, highly experienced interventionalists in CAS (>50 CAS) agreed that the simulated CAS procedure (using the VIST simulator) is a realistic interpretation of the actual procedure and provides good force-feedback, and that endovascular therapists should all train on this model prior to performing CAS in real patients.20 Subjective opinions from both inexperienced and experienced groups regarding the realism and usefulness of the simulator for training (Angio Mentor, SimSuite, and VIST) were also positive.20,23,26,28–33 Basic wire and catheter handling skills have previously been acquired by performing diagnostic catheterization studies. The increased use of non-invasive imaging techniques, budgetary constraints in the interventional room, and implementation of the European Working Time Directive (EWTD) will restrict such training opportunities. Additionally, patients have also become more demanding and less tolerant toward errors, and expect their primary operator to be proficient.
The term ‘learning curve’ used in the context of skills training refers to the time taken and/or the number of procedures an average practitioner needs to be able to perform an intervention independently with an acceptable outcome. Lin et al. have studied sequential groups of patients undergoing CAS and demonstrated that improved outcomes correlated with decreased procedure-related complications, fluoroscopic time, and contrast volume used with increased physician experience.34 In 2004, the US Food and Drug Administration (FDA) reported that simulation might be beneficial as part of a training package prior to allowing a physician to perform a CAS procedure on a real patient. Simulation-based training may allow this early part of the learning curve to take place without exposing patients to unnecessary risks. However, simulators need to be effective (learning objectives are met) and efficient (minimization of costs and time taken to achieve proficiency).35 Training studies examining the potential use for VR systems in endovascular skills training have analyzed the learning curves of both novice and experienced subjects. The EVEREST group and others have demonstrated that the performance of experienced endovascular physicians (inexperienced in CAS) improved during a virtual CAS procedure after a two-day course including supervised training on the Angio Mentor. The intervention post-course was not only carried out more quickly using less radiation, but, more importantly, catheter handling errors and spasms of the internal carotid artery occurred less frequently.23,26,33 Novices improved their simulated performance following a minimum of two hours of supervised training on a carotid VIST module.21 Similar improvements in simulator performance following training have been reported for iliac and renal angioplasty.19,29,30,36
The procedure reports provided by the simulators permit supervisors to follow the learning curve of an individual, allowing training to be tailored to a pre-defined benchmark level of skill. However, prior to adoption of VR simulation into the endovascular curriculum, it is necessary to demonstrate the transfer of endovascular skill to real procedures and to show that that these skills are maintained over time. Recent evidence of skills transfer using VR simulation suggests that this can be achieved.29,37 The first randomized trial examining skill transfer to the human model was carried out by Chaer et al. The simulator-trained group improved significantly during two supervised iliofemoral procedures compared with the control group, using a procedure-specific checklist and general rating scale to assess performance. The benefit of simulation-based practice is that subjects gain core endovascular skills that become automated by the time that they perform interventions on real patients.
The Limitations of Virtual Reality Simulators
Currently, simulators are not only expensive, but also their stability remains a problem, in particular following rough handling by inexperienced subjects. Keeping the simulators up and running remains a real challenge. There is a noteworthy requirement for regular maintenance and calibration to ensure optimal levels of force-feedback. The majority of these calibration and maintenance tasks are carried out by research fellows at our institution (following manufacturer training), but skilled technical support is required for heavy usage periods, especially those involving more challenging endovascular cases such as CAS.38 Hence, although it has been suggested that VR training is less expensive than live animal training, it is unlikely that many institutions will be able to afford such expensive resources. However, to be useful these simulators should be available to all trainees in specialist surgical skills centers.39,40 Furthermore, current simulators require the presence of an expert mentor to ensure correct learning and to provide immediate feedback on errors, since the current assessment parameters need to be enhanced.
Proficiency-based Stepwise Curriculum Incorporating Virtual Reality Simulation
While the focus of training in endovascular skills has been on the new opportunities presented by simulators, it is obvious that the key is not the simulator; rather, it is the curriculum that incorporates a simulator.41 As simulator training alone is not sufficient for a physician to be certified as competent to perform interventional care,16 the overall approach to training endovascular skills should be graded and provided within a stepwise structured proficiency-based training curriculum rather than over an unpredictable and often short training period. Aggarwal et al. have developed a framework for this type of systematic training and assessment of technical skills (STATS).42 The curriculum needs to consist of teaching the cognitive component, including error identification, followed by a test before allowing a subject to start simulator-based training of psychomotor skills. The procedure needs to be deconstructed into tasks and steps; key tasks need to be identified and used to enhance the assessment parameters of the current VR simulators. Subsequently, trainees can learn these endovascular skills in a safe environment on standardized models, which can then transfer to improved performance in the real environment.
Evidence-based training curricula, which define which simulated endovascular tasks, how often, and in which order they should be performed, are currently under development and need to be validated prior to widespread use in endovascular training programs. These will allow trainees to benefit from a flexible training curriculum including VR simulation that is tailored to their pace, learning comprehension, and schedule. They will have the opportunity to practice interventional skills and procedures in order to meet objective standards of proficiency prior to performing high-risk procedures such as CAS in real patients.
Proficiency-based curricula aim to train future endovascular therapists and have the potential to bring the different professions involved in CAS together. Vascular surgeons are familiar with patient selection and post-procedural care, but need more catheter skills training, radiologists are often less familiar with overall patient care, and cardiologists probably need training in the novel anatomical territory of the carotid, despite having endovascular skills. VR simulation is not only a good way to train technical skills, it also allows the entire interventional team to learn how to work together. The anesthetist, radiographers, theater nurses, and angiography suite nurses can acquire both the technical and non-technical skills (team working, leadership, situation awareness, decision-making, task management, and communication) that are mandatory to be able to perform CAS. The interventional team can be acquainted with rare complications and learn how to manage crisis situations in a simulated environment. These team training sessions can take place in a simulated interventional suite, allowing feedback by knowledgeable instructors.43
The EVEREST members hope to be able to encourage the academic centers, different professional societies, and medical device companies who already provide structured training programs to work together and develop a standardized approach to endovascular training, including the simulation training described above.
The Future of Virtual Reality Simulation
Interventional specialties may become early users of VR simulation for board examination. They can use simulation to transform an oral examination from a verbal description of how a procedure is performed to actual observation of how a candidate performs a procedure.16 Physicians and interventional teams working in low-volume centers can refresh and maintain old skills using VR simulation.
They can learn a new procedure that was invented after the physician’s post-graduate training or familiarize themselves with new devices. Furthermore, the endovascular therapist and the interventional team can be exposed to complex and life-threatening events and learn how to manage crisis situations in a simulated environment without exposing patients to risk. Physicians can not only ‘warm up’ on a simulator before beginning interventions, but VR simulation now allows endovascular therapists to practice complex endovascular procedures before performing them in vivo using the PROcedure Rehearsal Studio™ (Angio Mentor) or Mission Rehearsal (VIST). The endovascular therapist and his or her team can plan the approach, choose the endovascular tools, and address potential complications before a procedure. Furthermore, these types of rehearsals might influence decision-making in the treatment of symptomatic carotid artery lesions.
Studies in the US and Europe (EVEREST) are currently investigating the feasibility of the PROcedure Rehearsal Studio for CAS and the implications from an economic point of view. VR simulation provides an opportunity for training and assessment of endovascular skills prior to real life experiences. To avoid the isolation of acquisition of technical endovascular skills from cognitive and clinical skills, simulation training needs to be integrated into an appropriate curriculum. Simulation-based training is unlikely to replace real-life experience, although it may become an adjunct to teaching and maintaining basic and advanced endovascular skills, with the hope of shortening and flattening the learning curve our patients are subjected to.