A test that costs less might cost you more
The scope of a presentation attack detection test programme determines the range of threats it evaluates a system against. Understanding the variables – the sophistication of attack species, the diversity of instruments, and the number of platforms tested – is the starting point for any meaningful conversation about cost and coverage.
What a test programme actually consists of
PAD testing scope is determined by two primary variables.
Attack species sophistication Presentation attack species are classified across three sophistication levels, characterised in alignment with the NIST Strength of Function for Authenticators for Biometrics (SOFA-B) framework.
Level A species require minimal expertise and widely available materials.
Level B species involve more refined techniques and a degree of specialist knowledge.
Level C species require significant expertise and access to advanced tooling, including three-dimensional face masks and deepfake video sequences.
The range of sophistication levels included in a test programme determines how thoroughly the full threat spectrum is exercised.
Presentation attack instruments Presentation attack instruments (PAIs) are the physical artefacts presented to the biometric sensor during testing. The number and diversity of PAI species, and the volume of instruments per species, determines how comprehensively the system’s detection capability is challenged. PAIs are developed from source imagery derived from a diverse population representing a broad range of ethnicities, ages, and genders.
Scope also includes the number of devices and platforms tested. A system that performs well against presentation attacks on one platform may not perform equally well across all environments in which it is deployed.
The combination of these variables determines both the coverage of a test programme and its cost.
From compliance baseline to advanced assurance
Ingenium’s PAD evaluation programme is structured in five tiers. Each tier builds on the last, expanding the range and sophistication of attack species and the volume of presentations to provide progressively more comprehensive coverage. Every tier is evaluated using the two core metrics defined in ISO/IEC 30107: Attack Presentation Classification Error Rate (APCER) and Bona Fide Presentation Classification Error Rate (BPCER).
Level 1
An entry-level evaluation covering Level A and Level B attack species. Level 1 does not include any Level C attack species. A well-structured starting point for organisations beginning the process of understanding their system’s resilience, or where baseline assurance against low-to-medium-sophistication attacks is the primary requirement.
Total transactions: 1,100
Level 2
Building on Level 1, this tier expands the volume of Level B attack instruments. Level 2 does not include Level C attack species. It provides a more rigorous assessment of performance against medium-sophistication attacks and a greater statistical basis for the results.
Total transactions: 1,600
Level 3
Level 3 introduces Level C attack species for the first time, including three-dimensional face masks and deepfake video sequences. It evaluates resilience across the full spectrum of low, medium, and high-sophistication attacks. Designed for organisations that require testing to go beyond the medium-sophistication threshold and reflect a more realistic threat picture.
Total transactions: 1,900
Level 4
Broadens the range and volume of Level B and C attack species. The number of PAIs per species increases, and the bona fide subject cohort is expanded. This tier provides a more demanding evaluation of performance across the full sophistication range and is designed for systems where a high standard of security assurance is required.
Total transactions: 2,300
Level 5
Ingenium’s most comprehensive PAD evaluation. Level 5 incorporates the widest range of attack species and the largest volume of presentations across all sophistication levels. The bona fide subject cohort is expanded to maximise statistical confidence. Designed for systems that operate in high-risk environments or where the highest available level of independent assurance is required.
Total transactions: 2,500
Choosing the right level of assurance
The scope of a PAD evaluation directly influences the level of assurance it provides. Evaluations covering fewer presentation attack species, lower attack sophistication or a limited range of devices require less time and fewer resources. They also provide assurance against a narrower set of threats.
That is not necessarily a compromise. For systems operating in lower-risk environments or protecting low-value assets, a more limited evaluation may be entirely appropriate. The objective should always be to align the depth of testing with the system’s intended use and expected threat landscape.
However, as the value of the protected asset increases, so does the capability and motivation of potential attackers. Systems used in financial services, digital identity, government or border security are more likely to be targeted by well-resourced adversaries employing sophisticated presentation attacks. In these environments, a limited evaluation may not provide sufficient evidence that the system can withstand the attacks it is most likely to encounter.
Compliance alone does not answer that question. A report can demonstrate that a system has been tested, but the level of assurance depends on how it was tested—what attacks were included, how challenging they were, and whether the evaluation reflected realistic threat scenarios.
The most important question is not, “What is the minimum level of testing required?” It is, “What level of assurance does this system need, given the risks it faces?”
How the scope of an evaluation is determined
Every PAD evaluation is tailored to the system being assessed. While our five-tier framework provides a consistent basis for defining coverage, the final scope of an engagement depends on the deployment environment, the level of assurance required and the complexity of the system under test.
The following factors influence the scope of an evaluation and, consequently, the time and resources required to complete it.
Devices and platforms
PAD performance can vary between devices and deployment environments. Evaluating additional mobile devices, browsers or dedicated hardware increases confidence that results are representative of real-world performance, while also increasing the volume of testing, analysis and reporting required.
Presentation attack instruments
The presentation attack instruments (PAIs) included in an evaluation are selected during the scoping process to reflect the client's threat model, deployment context and user population. Broader coverage, particularly across higher levels of attack sophistication, requires the development, execution and analysis of a larger and more diverse set of attack scenarios.
System complexity
The complexity of the biometric system also influences the evaluation. Systems incorporating multiple capture modes, configurable security settings, active PAD mechanisms or bespoke deployment architectures may require additional planning, environment configuration and test execution to ensure the evaluation is both comprehensive and representative.
Project timelines
Evaluation programmes are planned to allow sufficient time for preparation, execution, analysis and reporting. Where an accelerated delivery is required, additional laboratory resources or parallel testing may be needed to meet compressed timescales. This can increase the overall cost of the engagement.
Access and operational requirements
Practical considerations such as secure remote access, on-site testing, network restrictions and client-specific security or data handling requirements can affect the logistics and duration of an engagement.
Reporting and assurance requirements
Every evaluation includes a structured technical report. Some organisations also require additional reporting to support regulatory submissions, procurement exercises, internal governance or certification activities. These requirements are agreed during project scoping and reflected in the overall engagement.
Understand your testing requirements
Every biometric system is different, and the right level of PAD testing depends on how and where it will be deployed. Before defining a detailed evaluation programme, it is often helpful to discuss your platform, deployment environment, threat profile and any regulatory or procurement requirements.
An initial consultation allows us to provide an indicative view of the appropriate evaluation scope, likely level of assurance and the effort involved. Whether you are planning a new deployment, comparing testing providers or preparing for certification, we’ll help you understand the options before you commit to a full evaluation.
Not all PAD testing is created equal. Make sure yours covers what it needs to.
Understanding the right level of testing for your systems starts with a conversation. Our team will help you identify the appropriate scope for your risk profile, regulatory context, and budget.