Lessons learnt from previous trials

Participants discussed the two major halted randomised controlled trials (RCTs) of DBS for depression: the Reclaim trial targeting the VC/VS region29 and the BROADEN trial targeting the SGC region. While the results of the Reclaim trial had already been published29 at the time of this workshop, the BROADEN data were still under review at that time and therefore were not fully available to workshop participants. BROADEN results were published during the review of this manuscript34 and are included here for the sake of completeness and for the relevance they bear to this discussion.

Reclaim was a randomised, double-blind, sham-controlled, multisite trial of VC/VS DBS for depression sponsored by Medtronic. All subjects were implanted and then randomised (in double-blind fashion) after 4 weeks to either receive stimulation (‘active’) or no stimulation (‘sham’) for 16 weeks. During this blinded phase, medications were kept constant, and stimulation parameter adjustments were tightly regulated. Following the 16-week blinded phase, all patients received active, open-label stimulation. The primary outcome variable was response rate, defined ≥50% Montgomery–Åsberg Depression Rating Scale (MADRS) score reduction.

The trial was designed to enrol 208 patients, based on a power analysis assuming a response rate of 15% in the sham group (placebo effect) and 35% in the active group. After the first 30 patients were implanted (29 of whom completed the blinded phase), the sponsor broke the blind for an interim analysis. At this point, the response rate was 2/14 (14.3%) in the sham group, which was close to the expected rate, but only 3/15 (20%) in the active group, which was less than the expected rate.29 During the subsequent open-label phase, the response rate was 6/30 (20%) at 12 months and 7/30 (23.3%) at 24 months. The trial was not powered to detect a difference between active and sham in 30 subjects, but given these interim results, the sponsor terminated the trial.

BROADEN was a randomised, double-blind, sham-controlled, multisite, trial of SGC DBS for depression sponsored by St. Jude Medical (now Abbott).34 All subjects were implanted and then randomised (double blind, 2:1 active:sham) after 2 weeks to active or sham stimulation for 6 months. During the blinded phase, medication changes were not permitted, and stimulation parameter adjustments were closely controlled using a prescribed algorithm. Following the 6-month blinded phase, all patients received active, open-label stimulation. The primary outcome variable was again response rate, defined this time as a ≥40% reduction in MADRS.

The trial was designed to enroll 201 subjects, based on a power analysis assuming a response rate of 18.5% in the sham group and 40% in the active group. An interim analysis was performed after 90 patients had completed the 6-month blinded phase. At this point, the response rate was 5/30 (17%) in the sham group, which was close to the expected rate, but only 12/60 (20%) in the active group, which was less than the expected rate. An FDA-mandated futility analysis demonstrated a 17% chance of achieving a significant difference between the sham and active groups if the study were to be continued. The a priori cut-off for terminating the study was a futility analysis result of <10% chance of success. Nevertheless, the sponsor decided to terminate the study following the interim analysis. Long-term open-label follow-up results demonstrated a response rate of 29% at 12 months, 53% at 18 months and 49% at 24 months.35

The following were proposed during the workshop as speculative reasons for the results observed in these trials.

1. True lack of efficacy of DBS in the respective target. This possibility must of course be considered first in any treatment study that does not demonstrate a difference between active and sham treatment. It is possible that DBS is ineffective in the management of treatment-resistant depression. Workshop participants could not rule out this possibility but considered it unlikely given observed improvement in several patients and other observations such as acute deterioration after unexpected DBS battery depletion. The latter essentially represents a blinded, randomised discontinuation of therapy. Symptomatic decline in such a situation, as has been observed,36 suggests that at least some patients do derive efficacy from the therapy. The recent DBS for depression study from the Netherlands mentioned above37 intentionally used a blinded discontinuation design strategy and demonstrated efficacy of the therapy.

2. Placebo response. This effect is clearly present in DBS for depression, which is not surprising given its prevalence in psychiatric treatment studies.38 Sham response rates in Reclaim and BROADEN trials were 14% and 17%, respectively. In addition to a pure placebo response, there may have been an additional beneficial effect from trial-related interactions. The intensive follow-up plan for these studies resulted in frequent and positive interactions between enrolled subjects and psychiatrists as well as engaged and empathetic study coordinators. Thus, subjects in the sham arm may have derived therapeutic benefit simply from this increased exposure to caregivers, in addition to placebo effect itself.

3. Trial design. A number of aspects of trial design were scrutinised by the attendees. As described in detail above, both Reclaim and BROADEN subjects were randomised soon after implantation. Placebo effect from the surgical procedure would certainly still be relevant at the onset of the blinded phase. Furthermore, these blinded phases were short (4–6 months) relative to the 12–24 month period over which responses were observed in previous open-label studies. In addition, both trials placed strict limitations on medication changes and stimulation adjustments. For example, Reclaim did not permit monopolar stimulation29 and BROADEN did not permit adjustment of frequency or pulse width.34 The higher response rate in the open-label follow-up period in BROADEN, during which stimulation adjustments were unconstrained, suggests that unaccessed regions of the parameter space may have been important. These factors could certainly have contributed to both high sham response and low active response, thereby decreasing the chance of seeing a difference. Further aspects of trial design are discussed in section 4.2 below.

4. Inappropriate or inadequate measurement tools. It is possible that the employed rating scales did not capture subtle improvements. In a number of instances, subjective improvements were noted by family members or even patients themselves—improvements that were not reflected in scores on the chosen symptom scale. The studies may thus have been asking the wrong questions. As an analogy, if patients undergoing DBS for Parkinson’s disease were judged based on response to gait freezing and dysphagia rather than bradykinesia and rigidity, then that therapy would also have been considered a failure. Another complicating factor is that unlike in movement disorder surgery, stimulation effects are not immediate for psychiatric procedures. When patients are fitted with hearing aids, for instance, hearing gains are not immediate; it takes weeks for the brain to adapt to the newly amplified auditory stimuli.39 Given the complexity of the brain’s response to electrical stimulation, it is not surprising that response to DBS surgery is correspondingly delayed. Thus, the question of appropriate biomarkers is not just a matter of ‘what’ to look for but also ‘when’ to measure it, and a sophisticated and nuanced approach needs to be adapted to DBS for psychiatric disorders in planning these trials and analysing outcomes. This possibility emphasises the need for more sensitive or appropriate diagnostic scales and objective biomarkers of response.

5. Patient selection. Depression (for example) is a heterogeneous disorder. The diagnosis requires the presence of five of nine possible major criteria40; thus, two patients may carry the same diagnosis with only one overlapping symptom. Compounding the difficulty, it may be the case that stimulation in different targets has different antidepressant effects. Thus, some patients may be better candidates for SGC stimulation, while others may benefit more from VC/VS DBS. While no head-to-head comparison has been done, it was suggested that patients responding to SGC stimulation are more likely to suffer from psychomotor slowing, while those responding to VC/VS DBS may have primary anhedonia. These impressions reflected expert opinion, as hard data to this regard are not available. A related issue is the lack of clinical ‘biomarkers’ that can predict response. For example, response to levodopa treatment is a predictor of response to subthalamic nucleus (STN) DBS for Parkinson’s disease. Patients who do not show such a response are not candidates for the therapy. For psychiatric disorders, we have no such criteria. Contributions of other psychiatric comorbidities are also common and variable. The importance of this heterogeneity is now better appreciated than it was previously and will need to be considered for future inclusion criteria design.

6. Finally, anatomo-functional targeting may have been suboptimal. This possibility is particularly relevant in the BROADEN study. While the study was ongoing, the Emory group fine-tuned their targeting approach to this region, using DTI to define critical white matter tracts, as described in prior publications.41–43 Because the trial was already underway, this accumulating knowledge could not be incorporated to refine targeting within the trial. In a very recent open-label study using this method for prospective targeting, this group showed an improvement from a 41% responder rate with anatomical targeting to a 73% responder rate at 6 months and 82% rate at 12 months using the DTI-based method.44 The same approach may be applied to other targets that are essentially white matter bundles, including the VC/VS and sl-MFB.

Participants also commented generally that pressure to take results from the initial relatively small open-label studies to a large pivotal RCT resulted in premature initiation of the larger trials. Many of the considerations listed above were known or suspected prior to starting the trials. Future work is clearly needed to understand underlying neurocircuitry, how it differs between patients and how best to measure effects of its modulation before embarking on further time-consuming, expensive trials.

Lessons learnt from lesion surgery for OCD and depression

Stereotactic lesions have been used since the 1950s as treatment for refractory neuropsychiatric disorders. Classically, four distinct lesions have been described: anterior capsulotomy, cingulotomy, subcaudate tractotomy and limbic leucotomy.5 The contemporary indications for lesion surgery are as broad as those studied using DBS, but most commonly include OCD and depression. Because these procedures simply use US FDA-approved ‘tools’ (eg, radiofrequency electrode/generator, radiosurgery unit, laser ablation process and high-intensity focused ultrasound unit) to create intracranial lesions, they are typically not subject to the ‘investigational’ categorisation that labels DBS for any indication outside the approved four (tremor and Parkinson’s disease as fully approved, and dystonia and OCD as approved on HDE). However, these procedures are only performed at a relatively small number of centres in the USA and typically under the supervision of local institutional review boards (IRBs).

There are no head-to-head comparisons of lesions and DBS. Retrospective, predominantly uncontrolled, open-label studies report clinical outcomes of lesion procedures that are comparable with similar open-label DBS studies. A recent report described a 47% full and 22% partial response to cingulotomy and limbic leucotomy for severe OCD.45 Another comparison of anterior capsulotomy to cingulotomy found a comparable response rate of 54% and 41%, respectively.46 A recent retrospective comparison of capsulotomy to VC/VS DBS (thus comparing interventions in the same region) found a significant advantage in Yale-Brown Obsessive Compulsive Scale (YBOCS) reduction (51% vs 40%) and remission rate favouring capsulotomy.47

Lesions offer a number of potential advantages over DBS. Patients do not have to contend with device-related worries (possibility of infection or breakage and requirement for battery replacements), do not have to be ‘tethered’ for their whole lives to a geographical location near an institution familiar with device programming and do not have to make frequent, indefinite visits to the programming site. They also avoid the possibly stigmatising feeling of having a permanent brain implant and therefore ‘being a cyborg’. In some cases, lesion surgery may be the only alternative for a patient with contraindications to DBS, such as skin picking, thin skin due to comorbid eating disorders and others.

However, lesion surgery is irreversible and less forgiving in terms of target accuracy. Once efficacy and side effect magnitude has reached its steady-state level, little can be done to alter it. New DBS technology is increasingly available, including segmented DBS leads with current steering capability and closed-loop devices capable of adjusting stimulation parameters based on real-time measurements of physiological biomarkers. These capabilities further highlight the difference between the immutability of lesions and adjustability of DBS. Patients, families, caregivers and payers need to be educated about the advantages and disadvantages of lesions and DBS.

The beneficial clinical effect following lesion creation is often not immediate. For example, clinical improvement may not manifest for several months, and often continues to evolve with continued improvement reported years after the initial surgery.22 45 Similarly, gradual improvements are also commonly seen in DBS studies.48 49 Therefore, characterisation of the time-course of response is instrumental in the design of future clinical trials. For example, the primary outcome of a recent double-blind, sham-controlled RCT of Gamma Knife (GK) capsulotomy for OCD measured at 12 months (the end of the blinded phase) showed 2/8 responders in the active treatment group and 0/8 in the sham group, a difference that was not statistically significant (P=0.11).22 Two months into the open-label phase (month 14), however, a third patient who had had active treatment became a responder. It is impossible to know whether placebo effect after breaking the blind at 12 months contributed to that subject’s further improvement and threshold crossing into responder status. A fourth patient achieved full response at month 18. It is certainly possible, therefore, that had blinding been maintained and the primary outcome measured a few months later, the difference may have been significant. Thus, in addition to the short-term effects from lesion generation, long-term social and environmental interactions with the underlying neurobiology may be integral in achieving maximal efficacy.

Lesions can be created using a variety of techniques. While traditional methods include radiofrequency ablation and radiosurgery, laser interstitial thermal therapy (LITT)50 and high-intensity focused ultrasound (HIFU) have gained popularity in recent years.51 Like GK, HIFU is an incisionless procedure. Ongoing and pending clinical trials aim to study the use of HIFU in the treatment of severe OCD.52 While the goal of HIFU in these trials is to create a permanent lesion, in theory HIFU can also be used for non-lesional neuromodulation. While lesions produced by all of these methods may look radiographically similar on MRI, there may be underlying microscopic physiological differences in mechanism of action, a topic worthy of further investigation.

On the flip side, DBS should not be thought of as a functional lesion. The mechanisms underlying DBS are complex, incompletely understood and may include both stimulation and inhibition of neuronal activity.53 54 While both lesions and DBS appear to have comparable clinical efficacy, underlying differences in mechanism of action may account for differences in time course and qualitative clinical results.

Structural and functional neuroimaging will play a greater role in identifying the most efficacious lesion location in the treatment of psychiatric disorders.5 Indeed, neuroimaging has revealed a high degree of individual anatomical variability in currently used targets.55 Unlike DBS, there are no restrictions placed on the postoperative MR imaging of patients with lesions, a fact that should be leveraged to characterise the relationship between lesion location relative to patient-specific structural and/or functional anatomy, as in a recent report.55 This methodology could then be used to prospectively plan individualised lesions. In addition, neuroimaging may be used for response prediction and in selection of patients who are most likely to respond to surgery.56 57

Closed-loop approach to neurostimulation for psychiatric disease

Participants discussed combining two relatively new approaches: closed-loop DBS and a novel research platform for targeting functional domains in a transdiagnostic fashion, known as Research Domain Criteria (RDoC), in order to improve clinical outcomes.

Unlike other areas of medicine, psychiatry is unique in that it still classifies and treats disorders based on broad symptom categories, and frequently these symptoms are subjectively reported rather than objectively measured. As a counterexample, cancer treatment has evolved to take into account specific genetic and other biological markers, in order to provide more objective patient-specific and disease-specific targeted therapy. Thus, no oncologist would offer chemotherapy based purely on symptoms alone, let alone subjective self-reports. Over the past several years, the National Institute of Mental Health (NIMH) has developed a framework known as the RDoC project, which applies a multidimensional approach to psychiatric disorders based on more granular behavioural and thought patterns, neurobiology and genetics.58 Rather than avoiding comorbidity and heterogeneity, this approach is designed to incorporate them and is particularly important given how hard it is to find pure Axis 1 disorders like major depressive disorder and PTSD. The RDoC paradigm may facilitate objective symptom measurement across functional domains, such as avoidant behaviour, excessively rigid behaviour and so on, that are linked to known circuitry that in turn can be targeted by neuromodulation. We can then plot these symptoms on a multidimensional axis and use specifically designed behavioural tasks to assess the impact of neuromodulation on function in each domain.

In closed-loop DBS, signals are recorded from relevant brain structures and used to control stimulation patterns.59 In the scheme proposed during the workshop, neuronal signals (spikes, local field potentials or combinations) would be combined with behavioural data obtained using the RDoC classification to create a biomarker (eg, local field potential (LFP) signal or network activity) versus RDoC matrix, which could then be decoded using machine-learning algorithms to generate the appropriate control signals.60 The use of computational methods including machine-learning algorithms may have a special role in identifying underlying patterns that are not obvious in the data. These signals would then be paired with stimulation of relevant structures to treat RDoC-classified symptoms.

Such approaches will be extremely challenging and require multidisciplinary effort. Although DBS for movement disorders was first introduced 30 years ago, closed-loop approaches are just beginning to show initial promise. Furthermore, our understanding of the circuitry underlying movement disorders is more developed than that for psychiatric disorders. Nevertheless, most participants agreed that this is a high-priority area and that progress in one field will likely help in a synergistic fashion with others. In fact, a circuit design was proposed for a device that could potentially record from and stimulate several brain regions, with on-board logic controls and transcutaneous telemetry capability, all within a form factor that could fit within a 14 mm burr hole.61 As proof of principle, the first example of closed-loop brain stimulation, responsive neurostimulation for epilepsy, is already in clinical use.62 63 Also discussed was an approach using epilepsy monitoring methods to study neuropsychiatric circuitry in detail with intracranial recordings, in order to accelerate our understanding and development of new therapeutic approaches. There was enthusiasm from NIH representation for grant submissions to the BRAIN Initiative using next-generation approaches such as these.

Regulatory considerations

A representative of the US FDA discussed the current approach of the FDA towards implantable devices. Of note, a paper has recently been published detailing this process.64 He explained that a key regulatory consideration in bringing novel neurosurgical interventions for psychiatric disorders to market is communication with the FDA. The cost of bringing a new medical device to market can be substantial. An advanced understanding of the current FDA process and the potential to rationally adapt the process to psychiatric neurosurgery will result in decreased wasted effort, and eventually, better outcomes for patients.

A major goal of the FDA is to provide access to high-quality, safe and effective medical devices to the public. The FDA defines a medical device as any device that diagnoses, treats or prevents a medical condition and whose main mechanism of action does not involve chemical interaction. Examples of medical devices include tongue depressors, clot retrieval devices and DBS electrodes. The FDA classifies devices based on the level of control necessary to assure the safety and effectiveness of the device.65 The three classes of devices, and the requirements that apply to them, are Class I, II or III, on a scale of increasing regulatory control and potential for risk to the patient or user. In general, Class I devices are exempt from regulatory control (eg, tongue depressors). Class II devices require general and special controls, and Class III devices require additional premarket approval (PMA). A reclassification process is available to apply the appropriate level of regulatory controls66 for a device type based on the most current information regarding its safety and effectiveness.67 In general, DBS systems are Class III devices. One regulatory pathway to bring a medical device to market is the 510(k) pathway. A 510(k) is a premarket submission made to FDA to demonstrate that the device to be marketed is at least as safe and effective, in other words, substantially equivalent, to a legally marketed device that is not subject to PMA (21 CFR 807.92(a)(3)).68

Principal investigators who would like to study a medical device may do so under an investigational device exemption (IDE). An IDE allows the investigational device to be used in a clinical study in order to collect safety and effectiveness data required to support a PMA application or a premarket notification 510(k) submission to the FDA. Clinical studies with devices of significant risk must be approved by FDA and by an IRB before the study can begin. Studies with devices posing a non-significant risk must be approved by an IRB alone before the study can begin.69 While IDE approval time previously used to take over 400 days, over the past few years, the median time to full IDE approval has been reduced to 30 days. Reviews can focus on a number of issues, such as the appropriateness of the proposed scientific endpoints.

The FDA’s determination of safety and efficacy is based on valid scientific evidence.70 Valid scientific evidence can range from well-controlled investigations to partially controlled trials, well-documented case histories by qualified experts and postmarket data. An important consideration in any trial design is assessing safety. To aid with submission of protocols, the FDA also offers guidance in the form of a free presubmission pathway, which provides the opportunity to obtain FDA feedback prior to IDE or marketing submission. By this and other means, investigators are encouraged to engage and contact the FDA early in the process. An understanding of the FDA regulatory process by investigators involved in psychiatric neurosurgery will result in matching of agency and investigator expectations, resulting in a more efficient working relationship.

NIMH priorities relevant to neuromodulation in psychiatry

A representative of the NIMH briefly discussed how the NIMH is approaching neuromodulation for psychiatric indications. She discussed four key aspects of the current thinking: (1) impact on public health, (2) overall treatment goals, (3) challenges and (4) current opportunities.

Mental illness results in a tremendous economic and social burden.71 It is the single largest risk factor for suicide.72 As is true of many other neuropsychiatric conditions, refractoriness to pharmacotherapy is common. The Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study showed that less than 50% of patients respond to the initial medication, with diminishing returns from additional medications.73 Despite the availability of an increasing number of FDA-approved medications for depression in recent decades, the suicide rates for men and women in the USA increased from 1999 to 2014, with the largest increase being in middle-aged Caucasian males.74 The limited effectiveness of current medical management provides an opportunity for DBS or ablative surgery as possible therapeutic alternatives.

The ultimate promise of neuromodulation is a relatively rapid and effective treatment response. We know from the ECT literature that patients can respond rapidly and robustly. For example, following ECT, the majority of patients have complete resolution of suicidal ideation by their second treatment.75 This potential for rapid antidepressant response has also been reported with other interventions, including ketamine,76 sleep deprivation77 and DBS.21

A number of questions and challenges must be addressed prior to embracing DBS for psychiatric disorders. These include: (1) which patients should we target?; (2) what components of the illness and domains of function are most likely to respond?; (3) What are the specifics of the methodology?; and (4) how do we measure successful response to treatment? The answers to these questions will inform how we devise future trials.

As discussed previously, the classification and diagnosis of psychiatric disease is still rooted in a symptom-based approach rather than more objective neurobiological criteria. This is a problem for treatment development because the diagnostic system is not aligned with the neurobiological basis of the illness. In contrast, and perhaps ironically, preclinical animal models of psychiatric illness often use more objective biological criteria. For instance, we model depression in rodents with tail suspension and the forced swim test, using objectively measurable behavioural responses as the outcomes. Yet, in clinical trials, we focus on asking patients about their thoughts and feelings, with little emphasis on neurobiological biomarkers. Not only is it unclear that the animal models represent valid surrogates of the disease, but leaping from objective measurement to subjective self-reports is a massive epistemological gap.

Another observation made by the working group was that the presence of comorbid mental illness is the norm, rather than the exception, in patients with severe refractory disorders such as those undergoing surgical procedures. Recent clinical trials have sought to attain highly homogenous groups, lacking comorbidity and ideally off medications. However, in reality, these patients are highly heterogeneous, typically take multiple medications and often possess multiple comorbidities. This mismatch motivated the previously mentioned RDoC approach,58 which seeks to define tractable, objectively measureable behaviours that can be measured and modulated. The RDoC approach focuses on domains of function, such as valence, arousal and cognition, described at increasing levels of organisation, from genes, molecules, circuits, all the way to behaviour and finally self-reported subjective symptoms. RDoC is not a diagnostic categorisation system. Instead, it is a research framework for studying psychopathology based on dimensions of observable behaviour and neurobiological measures to inform target identification and patient selection.

Fortunately, we have made major strides in both behavioural quantification and in understanding of neurobiological mechanisms. Approaches using resting state functional MRI, direct neuronal recordings, even semantic analysis of language using graph-theoretical methods have shed light on mechanisms that may be targetable.78 As mobile and wearable technology advances, the hope is to put both brain data and behaviour into computer algorithms to link brain activity with behaviour at the speed of thought as in the BRAIN Initiative projects described earlier.

Another major challenge is the identification of the ideal target and stimulation parameters. Target identification is part of a larger experimental medicine approach that the NIMH is using to fund trials going forward. In the past, we have had an issue with uninformative trials falling into the ‘Valley of Death’. The point of the experimental medicine paradigm is to avoid these failures, by achieving a few key goals: (1) ensure target identification, (2) establish the optimal stimulation parameters, (3) show that the intervention causes a quantifiable change in a relevant brain activity or mental process and (4) correlate this change to a mechanism of action. NIMH reviewers will not accept grants proposing therapeutic trials that do not conform to this approach. All four goals have to be met. The point is to design trials that are either successful or fast to fail; these trials will therefore be informative about the proposed mechanism, whether the results are positive or negative. Of note, neither of the large DBS depression trials used an experimental medicine approach, thus limiting information that they can provide regarding target engagement validity.

Finally, we discussed the specific opportunity for partnership with industry in the context of funding. Recognising the critical role of industry partners, several NIH BRAIN Initiative funding mechanisms specifically request proposals styled as public–private partnerships. These mechanisms encourage collaborations between academia and industry in tackling these challenging problems.

Role of industry

Representatives from Medtronic, St. Jude (now Abbott) and Boston Scientific were present and discussed industry considerations in these trials. There was a consensus that the pivotal trials were initiated too quickly. The commercial value of being ‘first to market’ for a new indication, especially one as prevalent as depression or OCD, is extremely powerful. The rapid progress to pivotal trial was also due to the success of initial open-label trials, as well as enthusiasm stemming from the success of DBS for movement disorders. In the latter diseases, the treatment effect was so clear that significance may have been achieved, despite shortcomings of trial design. Furthermore, the endpoints of DBS for movement disorders are highly focused and selective neurological criteria with objective measures such as tremor and rigidity, with the explicit awareness that surgery will not slow down or cure disease progression.

A regulatory difference between drug trials and device trials has been pointed out: drug trials explicitly require a phase 2 dose-finding study. If this step were required of the devices, it is possible that target refinement and endpoint development might have been conducted in such a way that pivotal trials would have been positive. As it is, we are left to consider next steps that require continued buy-in from industry, after the fact. However, a great deal has been learnt from negative trials, both in terms of disease pathophysiology and the possible mechanisms of treatment effect. Given advances in the understanding of the underlying biology of treatment responses to stimulation-based therapies, it should be possible to design more informative trials. Importantly, these advances should allow us to customise disease-specific approaches to target, endpoint and treatment delivery. With these considerations in mind, trials can be conducted that lead to rapid FDA approval, with engagement from insurance payers and a possibly sceptical public.

Medtronic representatives proposed a short framework for the kind of pilot data that would encourage industry funding of new trials. Significant progress in any one or two of these domains would likely cause industry to be willing to fund new trials. First, is there a diagnostic test that could reasonably predict response to DBS? For example, we know that levodopa response predicts response to STN stimulation. An analogous test for mood or other psychiatric disorder surgery would be greatly valuable. Second, is there an acute intraoperative effect of stimulation? In OCD, for example, a ‘mirth response’ can indicate engagement of the VC/VS target. A similar effect in mood disorders would help localise the most appropriate target region. Third, rapid onset of therapeutic effect would also confirm efficacy. It is understood that effects can be seen 1–2 years after surgery, but in the present economic environment, we need to consider whether late effects will be considered cost-effective by payers. Fourth, the treatment effect needs to show durability over time. We need to be able to prove that the treatment is ameliorating diseases that are typically characterised by a fluctuating course. Participants acknowledged that prevention of relapse and durability of treatment is a laudable goal. This goal may be accomplished by increasing our knowledge of the underlying neurobiology and searching for more objective clinical measures of response.

The case for registries

Maintaining detailed records of patients’ responses to treatment in nationwide databases or registries, appropriately curated for accuracy and consistency, is an alternative means to clinical trials for evaluating treatment outcomes. Participants explored the relative strengths and weaknesses of clinical trials versus patient registries.

Although clinical trials are preferable to data registries from a purely statistical perspective, they are limited by several factors, many of which are unique to psychiatric neurosurgery. These constraints include ethical considerations, proper blinding, the inherent difficulty of creating sham treatments, patient heterogeneity and achieving sufficient sample size. While DBS does provide the ability to switch between ‘on’ and ‘off’ stimulation states during a trial, an advantage for a within-subjects design, trial design in psychiatric neurosurgery remains challenging.

Due to these limitations of trials, some investigators are advocating for the pooling of data into national or global patient registries. Currently, only a tiny fraction of patients in the USA with movement disorders are enrolled in clinical trials, with the rest of the data stored either in local patient registries or not documented at all. In contrast to trials, registries provide a continuous and exhaustive data repository that can be used for multiple studies and is well suited for exploratory analysis. Challenges with registries include localised data collection, the logistics of sharing data across centres, maintaining Health Insurance Portability and Accountability Act (HIPAA) and other regulatory compliance, long-term management and funding. An ideal registry would be funded by the NIH and would have at least part-time staff to maintain it. Registry participants would be encouraged to use Common Data Elements in study design, as the NIH has provided for studies of traumatic brain injury (TBI) and epilepsy.

An example of a registry programme currently in use is the CranialCloud project out of Vanderbilt. The CranialCloud project uses a distributed cloud-based architecture to support crossinstitutional collaboration on movement disorders surgery. CranialCloud includes a HIPAA-compliant pipeline with several distinct steps: (1) data upload, (2) normalisation to a standard reference space, (3) standard preprocessing and (4) advanced visualisation. There are more than 3000 patients represented in CranialCloud, and it allows for collaboration and use of a vast clinical toolbox by practitioners. A similar pipeline may be helpful in psychiatric neurosurgery. Of note, a German group has created similar tools for reconstruction of electrodes and local functional connectivity, called Lead-DBS and Lead-Connectome, respectively.79–81 As the field progresses, more of these tools will become available.

Novel approaches to trial design

Participants discussed a number of novel concepts in trial design. The gold standard paradigm is the randomised, double-blind, sham-controlled trial. One of the features of DBS, the ability to turn off stimulation, lends itself well to randomisation. This feature has traditionally been used with blinded, crossover designs. However, RCTs were borrowed from the pharmaceutical world, and it may be that attempts to shoehorn surgical treatments into the constraints imposed by traditional RCTs are fraught with unique challenges. Other problems with the conventional RCT include: (1) difficulties with individualised programming, as most trials called for a strict algorithm that prevented the typical parameter exploration with which clinicians are familiar, and thus perhaps impeded identification of optimal settings, (2) delayed onset of treatment response and (3) behavioural responses/immediate responses such as mirth that unblinded patients to their treatment arm. As discussed in section 2 above, these issues may have contributed to the negative results of larger trials.

Participants discussed a variety of alternative trial designs. These included randomised withdrawal designs,82 waitlist designs83 and stepped-wedge designs.84 Randomised withdrawal designs include a significant period (months to a year) of open-label treatment, allowing time to optimise programming. This period is followed by a withdrawal phase in which patients are assigned to cease or taper stimulation. Timing of cessation may be randomised within a certain narrow window of time and can occur in a double-blind fashion. Symptomatic regression after cessation of therapy provides high-level evidence of efficacy.

Some objections were noted, including interpretational problems if patients have to be censored for ill effects with treatment discontinuation. In addition, abrupt discontinuation may produce effects noticeable enough to the subject to unblind him/her, thus affecting the integrity of the double-blind process. For example, equipment or battery failures in the VC/VS target have been shown to cause worsening of symptoms,36 48 and dystonia battery failures can lead to marked clinical deterioration.85 A blinded discontinuation phase was described in one experience with SGC DBS, which did correlate with decrease in mood.86 Gradual tapering of stimulation may not be noticeable to the subject, however, and may therefore allay the concern of unblinding. In fact, a recent trial from the Netherlands used this approach to successfully demonstrate efficacy of VC/VS DBS for depression.15 This design may thus be an appropriate model for smaller pilot studies.

Larger trials could benefit from waitlist designs or stepped-wedge designs. The advantage of waitlist-type designs (in which patients get surgery at different times, depending on position on a waitlist), or stepped-wedge designs (in which subjects receive a treatment at different times), is that between-subjects analysis is possible. This is also the case in a theoretical efficacy study in which large groups of people are randomly assigned to active or sham stimulation for long periods of time, and then a between-subjects analysis is conducted (which has never been done in DBS). As the placebo effect in psychiatric neurosurgery is a major factor, it is particularly important that a sham procedures and double blinding be maintained.87

A final consideration is the fact that observational data (especially about treatment safety and efficacy) can provide very meaningful information. Unlike drug trials, neuromodulation trials that involve permanent device implantation provide the opportunity for long-term follow-up. As several institutions accrue long-term follow-up data on an increasing number of patients, evidence will accumulate regarding treatment efficacy. Discussions with the FDA regarding national registries that closely monitor long-term data on patients outside of formal trials could be very productive in this field. It is conceivable that the FDA may accept such observational studies as sufficient to judge the safety and efficacy of device-based interventions. Without doubt, however, early discussion with the FDA is strongly encouraged for any such development plans.