By Jorn van Binsbergen, Consultant
Introduction
Medical device software (MDSW) can differ quite a lot from a regular medical device. A catheter is something completely different than a hospital information system and a hip implant is not at all comparable to imaging software. Algorithms are not tangible, they do not need to be sterile, they don’t have sharp edges and they do not cause allergic reactions. However, despite these obvious differences there are no special requirements just for software included in the European Medical Devices Regulation (EU-MDR). Article 61(1) of the EU-MDR specifies that confirmation of the conformity of a medical device with the General Safety and Performance Requirements (GSPRs) shall be based on clinical data providing sufficient clinical evidence. The level of clinical evidence that is necessary for that demonstration is at the manufacturer’s own discretion with the condition that it needs to be appropriate in view of the characteristics of the device and its intended purpose. Thus, clinical evaluation for MDSW is no different than the “ordinary” clinical evaluation when we purely look at the requirements. Luckily, the Medical Device Coordination Group (MDCG) published specific guidance for the clinical evaluation of MDSW which offers manufacturers a framework on how to apply the EU-MDR requirements specifically for MDSW. In this blog I will highlight the key aspects for clinical evaluation of MDSW under the EU-MDR. This also means that software for in-vitro diagnostic purposes (i.e., MDSW under IVDR) is left out of scope. Please also consider that MDSW is a definition from the EU-MDR. The International Medical Device Regulators Forum (IMDRF) and other markets may adopt software as medical device (SaMD), which has slight differences in concept and expectations, depending on the jurisdiction.
Is my Software Application a Medical Device?
Before we dive in further into the intricacies of MDSW clinical evaluation we need to take a step back. Because what does it actually mean when we talk about software with a specific intended medical purpose? For answering this, we need to take another MDCG guidance document into consideration namely MDCG 2019-11 “Qualification and classification of software – Regulation (EU) 2017/745 and Regulation (EU) 2017/746.”MDCG 2019-11 offers a step-by-step approach to characterize the MDSW and to verify that the proposed device is indeed a medical device. The most important questions in that sense are:
- Does the software perform an action on data beyond storage, archival, communication, simple search or lossless compression?
- Is that action for the benefit of individual patients?
- Is the software intended for a purpose that falls within the definition of a medical device as described in article 2 (1) of the EU-MDR?
If the answer is yes to all, we can fairly assume that the proposed device is indeed a MDSW, and we can take a further look into the clinical evaluation of such devices.
Principles of Clinical Evaluation of MDSW
MDCG 2020-1 “Guidance on clinical evaluation (MDR) / Performance Evaluation (IVDR) for medical device software” was published in March 2020 and distinguishes two different concepts of MDSW. There is the concept of software with a specific intended medical purpose leading to a clinical benefit, and there is the concept of software intended to drive or influence another medical device without having a medical intended purpose of its own. For the last category, the clinical evidence should relate to the device that’s been driven or influenced by the software, and therefore the standard clinical evaluation practices apply which we will not further take into account in this blog. For the first category, on the other hand, the clinical evaluation is focused on providing evidence for conformity of the software itself with the applicable GSPRs. MDCG 2020-1 proposes a framework for performing a clinical evaluation of MDSW that exists out of three different phases. The first phase is to establish a valid clinical association between the output of the software and the targeted clinical condition. The second phase is to provide evidence for the technical performance of the device. Finally, the third phase complies validation of the clinical performance by demonstrating that the software’s output is clinically relevant. In the following sections of this blog, we will dive deeper into these three phases and compare them with the “ordinary” clinical evaluation. Is it that much different?
Establishing a Valid Clinical Association
The concept of establishing a valid clinical association is unique for MDSW. The EU-MDR does not mention this concept and nor does the MEDDEV 2.7/1 rev. 4. The goal is to verify that the output of the MDSW (e.g., calculations, audio data, DICOM images etc.) is generally accepted and relevant from a clinical point of view. Although the concept of establishing a valid clinical association is new, the means to achieve this are not. A systematic literature review and review of guidelines, product standards and other public sources of clinical data will provide you with the information you are looking for in the majority of cases. And since a thorough and comprehensive review of the state-of-the-art is required anyway, these activities would be performed regardless of the need for establishing the valid clinical association. Thus, although the concept of establishing a valid clinical association between the software’s output and the targeted clinical condition is new, the sources of data that need to be consulted for this are already on the clinical evaluation menu, which means that this can be easily integrated in the regular state-of-the-art review process.
Validation of Technical Performance
Providing evidence for the technical performance of MDSW is achieved through verification and validation activities. The goal is to demonstrate that the software can generate its output accurately, reliably and consistently in line with the intended purpose in real-world usage. For a standard medical device clinical evaluation, pre-clinical data may be used to substantiate certain performance and/or safety aspects.Think of biocompatibility, usability or sterilization validation, for example. The focus for MDSW lies on different device aspects (e.g., accuracy, data quality, cybersecurity), but the concept of collecting and analyzing pre-clinical data in order to support the base of clinical evidence is the same. Noteworthy is that MDCG 2020-1 uses the terminology “real-world usage.” It means that technical performance characteristics may also be substantiated by using real-world data by analyzing existing, previously collected patient data. If this data is not sourced from publicly available databases but collected by a manufacturer in a previous stage, it’s important to consider whether patients have consented with this type of use of their data. Thus, in comparison with the standard clinical evaluation, the pre-clinical verification and validation for MDSW does not differ conceptually. However, other types of data may be used and there may be a stronger focus on this part of the MDSW evaluation.
Validation of Clinical Performance
The final step is the validation of the clinical performance of the device. This step is not different from a “normal” clinical evaluation. The goal is to substantiate all clinical safety and performance claims with clinical data in order to provide evidence of a sufficient amount and quality to confirm that the device is safe, that it performs in line with its intended purpose and that the clinical benefit is achieved and outweighs the risks associated with the use of the device. Where it can be challenging to formulate and substantiate the clinical benefit for a “regular” medical device, it can be even more challenging to do so for MDSW. Because of the differences in interaction between the device and patient, which for MDSW is often more indirect of nature, it’s not always clear how the patient benefits and how this benefit can be quantified.
Clinical Benefit
By definition of EU-MDR article 2 (48), the clinical benefit of a medical device is:
“the positive impact of a device on the health of an individual, expressed in terms of a meaningful, measurable, patient-relevant clinical outcome(s), including outcome(s) related to diagnosis, or a positive impact on patient management or public health.”
Therefore, when we read this definition there are basically two options for a clinical benefit:
- Positive impact on the health of an individual, or
- Positive impact on patient management or public health.
For a lot of MDSW, the second option seems to be more fitting. For example, an imaging software application that is developed to analyze images to automatically detect anatomical structures does not directly impact the health of an individual; but it does exert an indirect effect by assisting the physician, and therefore it clearly has a positive impact on patient management. However, this second part of the definition of a clinical benefit does not seem to have made it into MDCG 2019-11, which clearly states that the actions performed need to be for the benefit of individual patients. In contrast, MDCG 2020-1 broadens the understanding of the concept of a clinical benefit by stating that a clinical benefit for MDSW may deviate from that which applies to other medical devices, since the benefit may lie in providing accurate medical information whereas the final clinical outcome may still depend on further diagnostic or therapeutic options. Thus, the MDCG guidance documents do not seem to be fully aligned on the concept of a clinical benefit of MDSW, where MDCG 2019-11 seems to narrow its definition and where MDCG 2020-1 seems to broaden it. When considering the EU-MDR definition as presented above, it makes sense to adapt the broader understanding of a clinical benefit that is presented in MDCG 2020-1.
Concluding Remarks
In this blog I have led you through the different phases of clinical evaluation of MDSW, highlighting both the obvious and the more subtle differences in regard to the regular clinical evaluation. Conceptually, there is not a lot of difference, which makes sense as the requirements are not different for MDSW. However, other types and sources of data may come into play when evaluating MDSW and outcome parameters may differ. In addition, the state-of-the-art review may be designed with a different focus and probably requires more regular updates, as progress in the realm of software generally occurs in a higher pace and the concept of the clinical benefit may be interpreted in a somewhat different way. Another obvious difference is that MDSW is more prone to change. Some medical devices may exist for decades on the market with unchanged design. However, software is updated frequently and with each update the question arises whether that affects the results of the clinical evaluation. Therefore, in conclusion, clinical evaluation for MDSW is not fundamentally different than for any other medical device. At the same time, a different and wider perspective is required to effectively apply the principles of clinical evaluation for MDSW to meet the general requirements as laid down by the EU-MDR.
Do you have additional questions? Reach out to us for more information or to discuss your specific needs. Our team is here to answer any questions or concerns you may have.