Neurotech regulations: Why the recent FDA leapfrog guidance on medical BCIs matters to you — Part 2 of 2

Over the two-part series, I will explain why the FDA Leap Frog guidance is relevant and provide a practical summary of what’s inside. Part 1 found here.

(This article is a deep dive into recent FDA guidance on BCI regulations, read part 1 for an introduction to regulations and why they matter to you).

Until recently, understanding BCI (Brain Computer Interface) regulations was no small task. Designing a BCI can bring together a diverse range of fields from implantable materials through to safety critical software. Each field has had its own regulatory requirements which must be met for a product to be released. To clear things up, the FDA recently consolidated requirements from the many fields relevant to BCIs to release what they have called ‘Leapfrog Guidance’ on the topic.

In this article, I’ll be breaking down the document to provide a summary of the FDA Leapfrog Guidance on BCIs. To simplify things, I’ve broken the content into 9 areas covering all the document’s key content:

1. Device description — what is being designed and made
2. Managing risk — measures to ensure patient safety
3. Software development — creating software that is safe and effective
4. Potential misuse — avoiding unexpected outcomes
5. Biological harm — testing for safety at the cellular level
6. Electromagnetic safety and concerns — functionality in the real world
7. Bench testing — does the system perform as expected?
8. Non-clinical Animal testing — safety and reliability testing ‘in-vivo’
9. Clinical trial design — measuring the impact on patients’ lives

Before we begin, you might have seen that the full Leapfrog Guidance is titled ‘Implanted Brain-Computer Interface (BCI) Devices for Patients with Paralysis or Amputation — Non-clinical Testing and Clinical Considerations’ and wondered how relevant the content is to neurotech more generally. Whilst the FDA has chosen to focus specifically on implanted BCIs for patients with mobility issues, the safety and technical concerns raised are universal across all applications. As such, the Leapfrog Guidance provides valuable insights for all working in the field of BCIs as it reveals how the solutions to these challenges are regulated.

So without further ado, here is a summary of the FDA leapfrog guidance on brain computer interfaces.

1. Device description

The bulk of the FDA’s guidance is on what to submit to when seeking approval for early human trials. For structure, this advice is broken down into sections with the first discussing what to include when describing your device to the FDA.

Clear communication of what your device is and how it’s used is vital. This is called the ‘device description’, and the FDA list the broad range of information required for the description to be complete. Beyond a text description, the FDA expects documents outlining product details from components and system diagrams to details of the software algorithms and a lists of safety features.

To tailor the advice to BCIs, the FDA get into specific modules and features which they expected in a BCI system, describing what level of detail should be provided when specifying everything from electrodes to battery packs. For stimulatory devices they even provide a template for full characterization of stimulation characteristics, covering factors from the number of channels to the electrical features of any waveforms used.

Beyond a description of the device itself, the FDA are also looking for a full account of any other devices or tools the product is used in conjunction with, which could potentially alter the recipient’s experience:

• Details of the programming/control unit, which could become complicated if compatibility with interchangeable consumer grade phones and wearables is planned
• A description of peripheral devices (such as leads, electrodes, connectors, receivers, prosthetic end effectors, etc)
• A ‘thorough’ description (ideally including diagrams) of all interactions between the system’s components, the user and patient, and the environment
• All complimentary surgical tools and procedures
• The device packaging and sterilization procedure
• The test tools, procedures and limits used for factory release testing

In effect, only once a company can provide a comprehensive account of exactly what the device is and how it is used will they be ready to submit their ‘device description’ to the FDA in support of running an early human trial.

High-level break down of a Brain computer interface (BCI)

2. Managing risk

Now the FDA knows what your proposed device is, they want to see how you’ve assessed the risk it poses to patients and users. This is vital for avoiding unnecessary harm to patients due to reasonably forceable circumstances such as loss of power. The Leapfrog Guidance recommends manufacturers follow the internationally recognized risk management process, ISO 14971. For those unfamiliar, the ISO 14971 process has the following steps. Records of each step must be kept for submission to the FDA:

• Identifying the hazards (i.e. dangerous situations) which the system could present, e.g. leaching of battery contents, excessive electric current, etc. and rating each hazard based on the harm (i.e. injury) they might cause — this is called a hazard analysis
• Next, each device failure mode (i.e. ways in which a system can malfunction) is analyzed and assigned a score based on it’s probability and the hazard it might cause. These scores are typically assessed against a predetermined criterion for acceptability (with higher probability failure modes causing hazards that result in more serious harms being unacceptable) — this is typically called an FMEA (Failure Modes and Effects Analysis)
• Unacceptable risks must then be controlled, typically by some additional safety features (such as system redundancy, internal monitoring or re-design) to fail into a safe state when the hazard occurs. The changes implemented then become specifications which the device must be verified against
• With the risk mitigating features implemented, the residual risk is then evaluated and further controls implemented until the overall risk is deemed acceptable
• Finally, a tabular report is issued detailing all of the above, along with a rationale for residual risks that were deemed acceptable.

Not only does risk management need to be in place for the device itself but typically needs to extend to all peripheral aspects of the device production, installation and use.

Once a device is in use, a further system should be put in place for ongoing monitoring of device performance.

3. Software development

Software incorporated into the device should be documented, a process that varies in intensity based on the ‘level of concern’ assigned to each software feature. Note that though similar, this level of concern is different from the IEC 62304 framework which is the functional safety standard covering software design and maintenance most often used in medical devices. For reference, the levels of concern are as follows. Documentation of what is required for each level of concern when submitting a request for an EMA can be found in “Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices” :

Major concern

We believe the level of concern is major if a failure or latent flaw could directly result in death or serious injury to the patient or operator. The level of concern is also Major if a failure or latent flaw could indirectly result in death or serious injury of the patient or operator through incorrect or delayed information or through the action of a care provider.

Example: Software for driving a custom invasive brain surgery robot to implant the BCI

Moderate concern

We believe the level of concern is moderate if a failure or latent design flaw could directly result in minor injury to the patient or operator. The level of concern is also Moderate if a failure or latent flaw could indirectly result in minor injury to the patient or operator through incorrect or delayed information or through the action of a care provider.

Example: Software for a trans cranial electrical current stimulation device with hardware limited maximum current.

Minor Concern

We believe the level of concern is minor if failures or latent design flaws are unlikely to cause any injury to the patient or operator.

Example: An app for displaying the battery level of a prosthetic limb, which itself has secondary power indicator LEDs

For any software incorporated into early feasibility studies, there should be evidence that adequate performance testing has occurred to provide assurance that the system operates within safe parameters, as determined by documented risk management activities.

For cybersecurity there is a recommendation that potential issues be assessed, addressed and documented in steps similar to ISO 14971 described in the ‘Managing Risk’ section above.

4. Potential for misuse

Not only does the FDA need to see that the device is designed safely, but they also want to see that there’s been consideration of potential misuse (intentional or otherwise). This analysis of use related hazards is called ‘human factors’ and typically requires designers go beyond their labs to put devices in the hands of real users and observe the weird and wonderful things they do. This process often throws up surprising failure modes for the simplest devices, so is a key part of developing something as highly novel and deeply integrated with daily activities as a BCI. These tests take time! Whilst the FDA understand the process won’t be completed for an early device trial, they recommend strongly that the process of identifying and addressing ‘human factors’ related risks is started as early as possible. Once a baseline understanding of usability concerns is established, use can further be monitored during early trials, allowing time for redesign prior to release.

5. Biological harm

Anything implanted into, or in constant contact with, the body has the potential to harm the user unless constructed out of safe materials and cleaned properly. Although risks are reduced significantly for non-implantable devices, there’s a possibility of harm, especially if the device is in extended contact with skin that might be compromised due to a rashes, injuries or other problems.

As such, biohazard risks are a big focus of the leapfrog guidance. To cover all bases, they request information on three separate areas: Biological safety (material related hazards), Sterility (pathogen related hazards) and Pyrogenicity (anything causing a fever).

• Biological safety — This concerns the risks associated with the device’s materials. The requirements for evaluating materials are divided up into 4 categories, dependent on the level of contact with the body. The most comprehensive biocompatibility evaluations are required for implants in permanent contact with internal tissues (category 1) and the least for surface devices which only contact intact skin (category 4). For each category, a list is provided of the factors to be considered in the biosafety evaluation.
• Sterility — For any implanted device, adequate sterilization is required to avoid pathogens entering patients and causing infection. The FDA require details of both the sterilization chamber and facility, as well as a justification for residue limits in the case of a chemical process. They also require details of the sterility assurance level, i.e. the probability that a unit that has undergone the sterilization process is in fact non-sterile. The is typically stated as an order of magnitude.
• Pyrogenicity — Pyrogenicity concerns the presence of materials which may cause a fever, typically bacterial endotoxins which remain after the bacteria are neutralized during sterilization. A traditional method of testing for these endotoxins uses the bright blue blood of horseshoe crabs, which contain special enzymes that cause the blood to clot when exposed to endotoxins. Thankfully for the crabs, synthetic alternatives have been approved for use.

For all of the above factors related to biological harm, there also needs to be a demonstration that the BCI will remain safe and effective through to the end of its shelf life and throughout the intended implantation period. Where a long implantation time is demonstrated to be safe via accelerated aging tests, how representative the test is of real world aging should be considered and evidence provided.

6. Electromagnetic safety and concerns

Any brain computer interface will, by necessity, have an electrical component for the computer side of the interface. This poses several risks to the user, some directly related to the use of electrical energy, others related to failures due to electromagnetic interference from the environment. Because of this, the FDA requires testing (ideally following applicable ISO standards) to demonstrate devices perform correctly in the intended environment.

For devices used in the presence of an MRI scanner, additional risks arise. Beyond MRI fields causing electrical malfunctions, devices may also heat up, move or vibrate. To prevent harm, devices must be labeled clearly with the level of MRI compatibility they are tested for.

For any wireless technology, the timely and accurate transmission of medical data is key. To help achieve this the FDA provide a standard for assessing and documenting the risks associated with wireless data transfer. They also provide a standard for wireless coexistence to help designers maintain a robust connection in busy areas, so that hopefully no one suddenly loses control of their prosthetic limbs function unexpectedly.

7. Bench testing

The need for bench testing prior to the first use on a human subject is a clear focus of the FDA guidance. Testing recommendations are made for each likely aspect of a BCI system with specific context and guidance for each sub system from electrodes through to the system housing, however the general sequence for this section is as follows:

1. Test that the real parts and sub-assemblies are within the tolerances (pre-defined limits for variability) set during the design phase.
2. Perform testing to characterize attributes such as resistance, insertion forces, tensile strengths etc. and confirm they are within the agreed specification
3. Confirm that minimum performance criteria are met, not just once, but continuously throughout the intended lifetime of the medical BCI.
4. Verify that, when all components of a system are combined, the system as a whole meets its specification and doesn’t present new risks. Where this type of full integration testing isn’t possible, an account of why it could not be completed and what has been done to mitigate risks should be provided.

As for all safety critical engineering, the testing doesn’t just need to verify that the correct result was achieved, but that the test tools and methods used were appropriate too.

On the topic of full system testing, the FDA supports ‘mix and match’ compatibility through collaboration across multiple manufacturers (e.g. one company’s data acquisition system is compatible with multiple assistive technologies and visa-versa). Integrating other manufacturers subcomponents/modules into a new device is also supported so that each company need not reinvent the wheel. To further avoid duplication of work, testing of these submodules can be substituted with references to pre-existing submissions if relevant performance was demonstrated.

8. Non-clinical animal testing

We’ve all seen videos of Neura-link’s monkey playing Pong, but this isn’t just amusement for an eccentric billionaire, animal testing is a critical step in developing implanted medical devices.

On this topic, the FDA support adoption of the ‘3Rs’ principle for animal testing (reduce, refine and replace). To aid manufacturers in following the ‘3Rs’, they suggest making a direct request for feedback on any animal study design. Where manufacturers are seeking alternatives to animal testing, the FDA will also consider assessing their proposed method for equivalency too.

Beyond promoting the 3Rs principle, the FDA also provides a list of factors to consider with respect to animal, or ‘in vivo’ testing:

• Study purpose — although defining a test’s purpose is up to the manufacturer, the FDA recommend the main focus should be on safety and aspects of performance that cannot easily be assessed on a lab bench (e.g. in vivo reliability). On the other hand, the Leapfrog Guidance suggests animal studies are not typically suited to evaluating effectiveness, especially where cognitive evaluations are required for results.
• Study scope — The study should build on what’s already known to answer important new questions relating to device functionality. The FDA prompts BCI manufacturers to consider past work and literature on similar devices. A case can then be presented, alongside study results, in support of steps such as reducing the number of animals tested, switching to an animal which can’t take a full size implant, or reducing a study’s duration.
• Study analysis — When discussing study analysis, the FDA makes clear that quantitative data is vital. This applies to data on both BCI reliability and safety. For safety, it is proposed that, at the end of the study, quantitative measures are used to assess both macroscopic and microscopic tissue impact (what’s typically called histology). The analysis should be performed by an independent veterinary pathologist and assess measurements such as quantity of necrotic tissue at the site of the implant. Proposals of appropriate negative controls (such as animals with an inactive device, or no implant at all) are recommended for comparison. For stimulation devices, both normal use and edge cases such as up to 24 hours of maximal stimulation should be investigated (animals may be sedated to relieve suffering in such tests).
• To assess reliability, several forms of analysis are discussed. These include i) reviewing ‘explanted’ devices (devices removed after use) for any failures, including microscopic ones such as insulation degradation or electrode corrosion and ii) comparing signal quality at the start and end of the study using metrics such as recorded spike amplitude or signal-to-noise ratio.
• Surgical techniques — Not only does the device need testing, but the surgical approach for implantation and explantation (removal) requires assessment too. To this end, the FDA requests a detailed description of the surgical approach for use in animal trials, along with a justification for the target site and any deviations from the planned approach for human subjects.

9. Clinical trial design

Despite all the above, the FDA still has a checklist of criteria to be met before clinical studies begin. Their focus is twofold: first, to demonstrate that each element of the trial design is scientifically supported and justified, and second, to ensure that the potential benefits to the patients outweigh any possible exposure to risks.

To balance potential benefits with risks, manufactures should combine selecting patients with a suitably severe condition with measures to enhance patient safety. Measures discussed by the FDA range from providing trained caregivers for at home use to a list of suggested exclusion criteria such as a history of seizures, or specific autoimmune issues. They also make clear that safety endpoints (outcomes used to assess success) should be clearly identified, along with a contingency plan for adverse events. Safety related triggers for halting the trial should also be identified as part of the trial design.

To back up the study design, the trial needs to have a clear pre-determined goal expressed in clinical endpoints, their meaning, and how they will be validated. This must be sufficiently supported by the weight of evidence provided by the number of patients enrolled. Often BCIs intend to treat symptoms common to many conditions, such as a paralysis, amputation etc. Therefore, the FDA will accept pooling of patients with different conditions provided there is sufficient justification. The set of accepted conditions is then be expressed a participant inclusion criteria which compliment the exclusion criteria mentioned earlier. If the resulting patient group is not representative of general US demographics, then this discrepancy may need to be accounted for with justifications.

After deciding the study purpose and participant profiles, the treatment patients will undergo must be clearly defined. Beyond a ‘Device Description’ (discussed in section 1 of this article), the FDA requires information on the post treatment period for recovery, and any training regime on device usage provided to patients. Lastly, there must be a justification for the overall time the study will run prior to assessment of endpoints. The FDA suggests studies last at-least 1 year as BCIs are a new technology with relatively little pre-existing data on safety, reliability or durability in long term use.

Given that a significant amount of resource and effort will have been expended to reach the stage of clinical trials, the FDA encourages the use of adaptive trial design. Rather than defining a rigid protocol and waiting on the results, a list of pre-specified modifications to trial procedures or statistical analysis should be prepared ahead of the trial commencing. These modifications can later be implemented given certain triggers, to increase the chance of a statistically significant result.

To conclude, it may seem that there is an overwhelming amount of work to be done reading all 9 sections above. Fear not!

The FDA are keen to make the process as efficient as possible, and provide a voluntary mechanism for companies seeking support. This mechanism, called the ‘Q-submission process’ allows companies to request feedback or even a meeting to guide their studies or pathway to regulatory approval.

The FDA’s clear communication of expectations and offers of support might seem uncharacteristically friendly from regulators who are often feared in the world of medical device development. However, providing such help fits into the FDA’s broader mission. Beyond preventing release of unsafe products, the FDA is also responsible for:

“advancing the public health by helping to speed innovations that make medicines more effective, safer, and more affordable.”

It appears the FDA have earmarked brain computer interfaces as an innovative field ripe with the potential to advance public health. Through the Leapfrog Guidance and their eagerness to engage with, and provide feedback to, companies in the field, it is evident that the FDA are investing a significant effort in paving the way for these technologies to reach the market and aid real world patients at scale as soon as possible.

Written by Jonathan Casey, edited by Avinash Royyuru, Emily Dinh, and Chelsea Lord, with artwork also by Chelsea Lord.

Jonathan’s work applies principles from engineering and product development to broaden access to technological innovations. He has worked extensively on the development of new medical devices and scientific instrumentation with a special interest in multidisciplinary projects where biological and artificial systems are combined to solve a genuine need.

Avinash is a freelance product manager writing a SciFi novel about merging brains.

Emily is a clinical trials researcher and data scientist. She is currently attending the University of San Diego for her MS in Artificial Intelligence.

Chelsea works in clinical operations and is currently working for a company that is a leader in brain-computer interfaces. She studied neuroscience and is interested in the AI and Machine-Learning side of neurotechnology.