The risk evaluation refers to the moment when each manufacturer must determine if the residual risks posed by their medical device are within the acceptable limits when compared against the criteria described in the risk management plan (RMP). The risk evaluation can be performed before the design completion and/or after the implementation of all risk controls.
According to ISO 14971, the acceptability of residual risks should be assessed for individual risks (each hazard and hazardous situation) and overall risks. The appropriate method for risk evaluation can be selected by the manufacturer, as well as whether the criteria used for the assessment of individual risks will be the same as those used for the evaluation of the overall residual risks. However, manufacturers must remember that the chosen method of risk estimation would determine the evaluation method. For example:
- Risk evaluation for qualitative method – Risks are stratified into relative rankings from high to low. Here, manufacturers can use product safety standards and state-of-the-art to determine risk acceptance.
- Risk evaluation for semiquantitative method – To indicate the risk acceptance, manufacturers can use relevant standards, state-of-the-art data, or refer to benefit-risk analyses. This method allows the numerical comparison between the probability of occurrence of harm and the state-of-the-art.
- Risk evaluation for quantitative method – All mentioned above ways of determining the risk acceptance are applicable. This method allows a straight forward comparison of the risks with the state-of-the-art. Here, manufacturers can compute the residual risk for a hazard, hazardous situation or overall in the following five severity classes: catastrophic, critical, serious, minor and negligible. Afterwards, if the computed residual risk for any severity class is bigger than the acceptable risk limits (stated in the RMP), the risk is determined as unacceptable.
How to determine the risk acceptance criteria?
The risk acceptance criteria are stated in the product safety standards applicable to the medical device. The medical device compliance with such standards is considered as an indication of risk acceptance for the particular presented hazard(s). When identifying relevant product safety standards, the harmonised standards should be sought first, and if there is none available, then other national or international recognised standards or publications should be considered.
Except referring to standards, manufacturers can establish the acceptable risk levels by comparing the risk posed by the examined medical device with the state-of-the-art. State-of-the-art refers to “what is currently and generally accepted as good practice”. In such cases, the risk of the medical device must be less than or equal to the state-of-the-art to be defined as acceptable. To determine the-state-of-the-art, a couple of ways can be used:
- Review on the current literature – The aim is to find relevant published scientific papers on the same or similar devices. Criteria such as sound inclusion and exclusion are very important as they may be subjected to inspection later on.
- Evaluation of relevant field data – It is applicable only in cases when the medical device is a new iteration of an existing approved device produced by the same manufacturer.
- BXM method – It uses quantitative methods for risk evaluation which bring the need to have quantitative risk acceptability criteria.
State-of-the-art data cannot be obtained for the following two cases:
- New and novel medical devices are produced
- A significant iteration on existing medical equipment is made to deliver novel new therapies.
In such cases, the risk acceptance criteria can be derived from:
- Evaluating clinical study data
- Formal benefit-risk analyses.
In addition to all said above, it’s good to note that the stakeholders’ concerns should also be taken into consideration during risk evaluation. This means that the perception of risk that a wide cross-section of stakeholders has is of importance when deciding what risk is acceptable. And, to meet the public’s expectations, manufacturers might need to give additional weighting to some risks.