Test, Validation, and Verification
Test & Evaluation (T&E) are crucial to validation and verification ensuring that AS meet operational requirements and perform safely, effectively, and provide interoperability in military operations, while also building trust in their autonomy.
V&V challenges are largely software centric due to the reliance on AI, and like other military systems, AS must undergo specific qualification and certification before battlefield use.
As AS evolve with higher autonomy, traditional T&E procedures – covering all possible scenarios – become unfeasible as these AS adapt and learn with each interaction, making it nondeterministic for legacy V&V approaches based on a test case situation. AS, being the test subject, create dynamic test environments while being tested. Their expanded autonomy increases the difficulty of evaluating risks and validating and verifying correct behaviour under diverse conditions.
Traditional V&V means of compliance are apparently inadequate for testing AS and present non-evaluated environments. New T&E processes, infrastructure (test ranges, labs), and tools are essential to encompass complexities of autonomy relevant for evaluation.
1424c13fa4d264cfa776ff000087ef0f.jpeg "Test validation verification")
Member State Test Centres represented at the Defence Test and Evaluation Base (DTEB) and the Test Centres Networks of Excellence could facilitate more inclusive coordination of testing activities aimed at contributing to the definition of developing harmonized T&E procedures and respective means on emerging (non-legacy) technologies in support of future uptakes with built-in interoperability and exchangeability.
Certification
As AS become more advanced and operate with greater independence, their complexity and unpredictable behaviour pose significant challenges for certification and regulatory authorities. The need to update traditional V&V methods, combined with the lack of universal certification standards for AS, results in inconsistent criteria across different regions. This inconsistency makes it particularly difficult to obtain approvals in multiple jurisdictions, especially within the diverse landscape of the European Union.
Certification processes for AS can encompass design, production, and ongoing maintenance, which, in turn, encourages valuable discussions about the inclusion of “learning” and “continuous surveillance” in the certification framework. Certification plays a vital role in demonstrating that AS are safe and reliable according to defined requirements. It is essential to ensure that certification supports development, fostering a favourable environment for innovation in a rapidly evolving global landscape. Exploring risk based approaches can provide effective alternatives for specific applications and environments. While looking forward to the establishment of regional or internationally accepted regulatory frameworks in the future, the current focus should be on developing best practices that enhance safety and promote progress in the field of AS.
Finding experts capable of integrating diverse disciplines – data, AI, control algorithms, mechanics, and electronics – is another challenge. As the development of AS accelerates, it is essential to strengthen the interconnection between several disciplines beyond today’s frontiers in the context of AS certification. Safety and trust are paramount, and a major hurdle is the explainability of AI.
Certifying collaborative unmanned and/or AS with varying sensors and domains adds complexity, posing limitations to the certification process and timelines. Lastly, it’s crucial that certification evolves side by side with technology following a step-by-step approach.
The DTEB
Member State Test Centres represented at the Defence Test and Evaluation Base (DTEB) and the Test Centres Networks of Excellence could facilitate more inclusive coordination of testing activities aimed at ensuring cross-certification of similar equipment across platforms in support of future uptakes with built-in interoperability and exchangeability. Additionally, liaising with certification entities/organizations to align the development of test, validation, and verification processes is also another important topic that needs to be considered in the coming years.
To know more about the Defence Test and Evaluation Base please visit our dedicated portal: DTEB (europa.eu)”