11 Types of Medical Device Design Testing

First, what does testing mean? The word “testing” can have several meanings, some of which are:

• Establishing confidence that the medical device does what it is supposed to do;

• Detecting specification errors and deviations from the device specification;

• Verifying that a medical device system fulfils its specified requirements;

• Identifying differences between expected and actual results;

• Operating a medical device or a component of it under specified conditions and observing and recording the results, as well as evaluating some aspects of the system or the specific component.

As it becomes clear from the various definitions above, the focus can be different (e.g. assessing the quality of the product or achieving specific goals). Given that, there are 11 types of testing that you as a medical device manufacturer are advised to perform in order to assess the design quality of your medical device.

1. Verification

The verification testing relates to undertaking any procedures that could determine that the product of each stage of the development process is an implementation of a previous stage. The testing objective of each verification procedure is to detect as many errors as possible.

2. Validation

The validation is the type of testing that should help you evaluate a system or a component of a system during or at the end of the development process to determine whether it fulfils the indicated requirements.

3. Black box

To perform the black box test, you don’t need any knowledge of the internal structure of the medical device, because the main goal here is to verify that all of the end-user requirements are met, looking from the end user’s standpoint.

Visually, the black box testing represents a black box, illustrating the medical device, with a set of inputs coming into it and a set of outputs coming out of it. This type of testing is a data-driven testing scheme, where the tester is only concerned with finding circumstances in which the medical device doesn’t behave according to its specification. The internal behaviour and structure of the device are not taken into any consideration.

4. White box

Here, you or your testing team need to have knowledge of the internal structure of the medical device, because the testing is done from the developer’s point of view. This is a logic-driven testing scheme. Simply said, the white box testing is the opposite of the black box testing. Therefore, the tester is only interested in examining the internal structure of the medical device and extracting test data from the performed examination.

5. Hardware testing

Hardware testing is not just a singular test, but it includes various types of tests depending on the intended use of the device. Those tests that can occur during almost every product development cycle are, as follows:

• Vendor evaluation;
• Component variation;
• Safety evaluation;
• Standards evaluation;
• Environmental testing;
• Shipping tests;
• Reliability demonstration;
• Product use/misuse.

Note that the hardware testing associated with the calculation of reliability parameters often is performed twice during the product development process:

• Immediately after the design phase, evaluating the design’s robustness and reliability;
• And, after the customer units’ production begins, evaluating the robustness and reliability of the product’s manufacturing process.

6. Software testing

Software testing includes several levels of evaluation, as follows:

• Module testing: this testing relates to evaluating and stress testing of the individual modules of the software program. It consists of validating the design and implementation of the specification at the program’s smallest component. The module testing involves running each module independently to assure it works, and then inserting errors, possibly through the use of an emulator.
• Integration testing: It occurs after each one of the modules has been tested successfully. During this testing, the various modules are integrated with each other and tested to assure that they work together.
• System testing: Here, the software is merged with the hardware to assure that both will work as a system. This type of testing consists of verifying the external software interfaces, reassuring that the system requirements are met, and assuring that the system, as a whole, is operational.
• Acceptance testing: This is the concluding testing, and it includes the final review of all the requirements indicated for the system and assuring that both hardware and software address them.

7. Functional testing

The purpose of this testing is to verify that all the functional requirements have been met. It is a termed success-oriented testing because it is expected the tests to produce successful results. This type of testing consists of the exercising of the operational modes and the events that allow a transition between the various software operational states. You should perform functional testing to verify that:

• Proper mode transitions are executed and appropriate outputs are produced given the correct inputs;
• The software generates the anticipated output given the expected user input.

During this type of testing, you should use a communication test tool to test:

• the proper operation of the remote communications protocol;
• the functionality of the communications software placed in the examined product.

Moreover, you should also perform timing and battery tests. The timing tests relate to testing the system’s critical functions (the system critical time and the operational window). Battery tests are those that should be performed whenever there has been a software change made to the software that monitors the battery levels.

8. Robustness testing

The purpose of performing this testing is to determine how the medical device software performs given unpredictable inputs. Robustness testing helps you determine whether the software will:

• recover from an unexpected input;
• lock the system in an indeterminate state;
• or, continue to operate in an unpredictable manner.

Unlike the functional testing, this one is a termed failure-oriented the reason for which is that the test inputs are designed in a way to cause the product to fail given foreseeable and reasonably unforeseeable misuse of the medical device. As a part of this testing:

• algorithms are tested for overflow and underflow;
• the user interface is tested by entering unanticipated values and sequences;
• routines, tasks, or processes which are time constrained are altered to introduce reasonable delays to determine the reaction of the medical device;
• unexpected commands and data are given to the communication software, which are then transmitted to the remote communications handler. 

9. Stress testing

With stress testing, you can determine how the medical device will react to a condition in which the amount or rate of data surpasses the amount or rate expected. This type of testing is good for determining the margin of safety existing in the medical device. It consists of overnight and weekend runs that gain the optimal benefit of the allocated test time. As part of the stress testing:

• global buffers and data structures should be tested under loaded and overflow conditions to determine the software’s response;
• remote communications load tests should be carried out to verify that the remote communications interface transfer rate at the maximum transfer rate under worst-case and maximum load conditions.

10. Safety testing

Safety testing is meant to verify that the medical device performs in a safe manner and that its safety design has been assessed completely. It utilizes the hazard analysis in relation to failures. The fail-safe tests can help you verify the fail-safe provisions of the medical device software design. However, these tests do not address warnings or alarms because they only cover the error conditions.

An acceptable level of design safety is ensured by performing an analysis of the error handling routines and data corruption tests. Safety aspects that you must also address are, as follows:

• the protection of critical parameters;
• the events that lead to a loss of safety-critical indicators.

Additionally, internal product safety self-tests and validation safety tests, performed on past products, should be collected, executed, and compared against the new examined product.

11. Regression testing

This type of test should be performed whenever there is a software or hardware change that affects your medical device software. Generally, regression testing is used for validating that:

• the change generates the desired effect on the altered element;
• no other element that relies on the altered part is adversely affected.

When performing regression testing, you must follow the steps listed below to guarantee its proper execution:

• compare the new software to the existing baseline with a version difference tool;
• generate a cross-reference list to assess the changes and ensure that no unexpected side effects are introduced;
• assess the number of changes and their criticality;
• estimate the level of effort needed to perform the regression and assess the risk;
• perform tests on the alterations;
• execute the compiled list of core tests to establish that there have been no new unexpected changes.   

Source:

1. Dhillon, B.S. (2000). “Medical device reliability and associated areas”, CRC Press LLC

2. Fries, R. (2006). “Reliable design of medical devices”, 2^nd, CRC Press