- Email: [email protected]
Vulnerability assessment
FEATURE
Vulnerability assessment Ted Dunstone, Geoff Poulton, Biometrics Institute In the popular portrayal of biometrics, from Minority Report to the recent Get Smart movie, a common theme is the defeat of biometric systems using faked (or spoofed) biometrics. While such fears make for good stories, the issues relating to the vulnerability of biometric systems are all too real. It is these issues that the Biometrics Institute, Australia, is addressing by introducing a methodology for the identification and assessment of vulnerabilities – the Biometrics Vulnerability Assessment methodology (BVA). Many biometric products now claim some degree of spoof detection, but questions such as how this vulnerability resistance was tested and what kinds of attack might have a likelihood of success are mostly left unanswered. If you think this issue is of mainly academic concern try a search for ‘fake fingerprint’ in YouTube. This turns up some quite detailed information on the creation of artificial fingerprints. In this context, the Biometrics Institute, with partial funding from the Australian Government, has developed a framework to assess the vulnerabilities in biometric systems and to give scientifically credible results for the likelihood of an attack succeeding. Since 2007 the methodology has been applied to a face, finger, voice and iris biometrics and in each case it has demonstrated vulnerabilities, often previously unknown, in commercially available systems. The findings from many of these studies are subject to publication restrictions for security reasons, however this article will discuss both the methodology and how it was applied to assess a fingerprint and iris system.
Vulnerability versus accuracy The vulnerability of a biometric system should not be confused with its accuracy. It is possible to have a system that is extremely accurate at distinguishing the correct person under normal conditions, but which is highly vulnerable to simple methods used to circumvent the security. This might be by mimicking physical biological characteristics, or by bypassing or altering the information used as part of the matching process. By way of analogy consider your front door lock. Performance testing would tell you how likely it is that any randomly selected key will unlock the door, vulnerability testing will tell you what is the likelihood that a thief might successfully pick your lock.
May 2011
“It is possible to have a system that is extremely accurate at distinguishing the correct person under normal conditions, but which is highly vulnerable to simple methods used to circumvent the security” Significant effort goes into improving the accuracy of the biometric algorithms for the task of distinguishing between different people, but much less is generally known about how the systems will stand up to determined attack. Most large-scale testing results on biometrics measure false match and false non-match rates, the former focusing on the chance that a random person has biometric characteristics that are sufficiently similar to pass as that of another individual. Vulnerability testing on the other hand needs to focus on identifying likely threats, optimising the attack methods and the measuring their success.
BVA methodology The main aim of the Biometric Institute’s vulnerability assessment methodology is to provide a reliable level of assurance against a spoofing attack for a particular threat on a particular technology. Other aims are to ensure the assessment is timely and cost effective, to incorporate
the flexibility to adjust to different biometric modalities in an evolving threat landscape, and to allow it to be part of a wider security assessment and threat mitigation process. The threat landscape is complex, as a wide variety of factors need to be considered during the evaluation. Each biometric product will have different levels of vulnerability to a range of threats, and each threat is dependent on the attributes of an attacker. The fundamental building block of this evaluation methodology is that the spoofing risk can be broken in two factors: the Exploitation Potential, which relies on the biometric system’s properties, and the Attack Potential, which is related to the capabilities of the attacker. The spoofing risk can then be defined as: Spoofing Risk = Attack Potential x Exploitation Potential Where the Attack Potential is a number between 0 and 1 that gives the relative strength of the attacker. This in turn depends on the attacker’s knowledge about the system, his or her access to it and the particular method of attack to be used. In the simplest implementation AP is estimated on the basis of experience. The Exploitation Potential is the likelihood that a particular vulnerability exists, and that it can be exploited to breach the system. EP is obtained from experimental measurements. For the particular attack method chosen, the likely strength of an attacker with reasonable resources is used to estimate the Attack Potential. Measurement and analysis is then carried out in two phases. Phase 1 is Exploration. An experimental program whose object is to determine the optimum parameters for the attack – those parameters most likely to succeed. For example, parameters for an Artefact Attack on a face biometric might include the identity of the enrolled subject and the size, resolution and presentation method of a photo artefact. This
Vulnerability Assurance Levels VAL4
Fewer than 1 in 100 attacks are likely to succeed
VAL3
Fewer than 1 in 30 attacks are likely to succeed
VAL2
Fewer than 1 in 10 attacks are likely to succeed
VAL1
Fewer than 1 in 3 attacks are likely to succeed
VAL0
Cannot guarantee at least VAL 1, A system whose level for a particular attack is VAL0 essentially has no protection from that attack.
Biometric Technology Today
5
FEATURE “The two attack methods were the construction of false fingers (artefacts), an attack on the input and algorithmic aspects of the system and the enrolment of an artificial fingerprint into the database”
A
B
Artefact attack
C
D
E
F
Full set of 36 artefacts made for Phase 1 testing. (a) Silicone Putty, 1mm; enhanced, (b) same, processed, (c) Pinksil, 1mm, enhanced, (d) same, processed; (e) ProSkin, 0.5mm, enhanced, (f) same, processed.
phase usually comprises a number of repeated attacks to test the parameter space. Phase 2 is evaluation. Using the optimum parameters from Phase 1, the system is attacked many times and the results recorded. Appropriate statistical calculations are then used to determine the Exploitation Potential. This, together with Attack Potential, is then used to find the spoofing risk, giving a direct measure of the degree of vulnerability assurance. A key part of the Biometrics Institute methodology is to measure vulnerability in terms of Vulnerability Assurance Levels. The Vulnerability Assurance Level (VAL) is a logarithmic measure indicating the assurance that there will be no more than a given percentage of successes from repeated single attacks on the system. The first five levels are shown in the table Vulnerability Assurance Levels and are computed with a confidence interval of 95%. Higher levels are possible. 6
Biometric Technology Today
Fingerprint system In February 2009, the Biometrics Institute commissioned an independent study that applied the methodology to the evaluation of two spoofing attack methods on a commercial fingerprint biometric system. The fingerprint system tested was a DigitalPersona Pro U.are.U 4000B scanner with Neurometrics Verifinger software. The two attack methods were the construction of false fingers (artefacts), an attack on the input and algorithmic aspects of the system and the enrolment of an artificial fingerprint into the database, designed to allow entry by multiple attackers (known as a ‘lamb’ template attack). This is an attack on the database and the enrolment process. The effect of larger attacker numbers was also investigated.
A small database of fingerprint images was first constructed, comprising four instances each of 20 attacker fingers. The methodology first optimises parameters of the attack, and then repeatedly attacks the system with the optimised parameters. For the Artefact Attack, six of the possible 20 identities were preselected on print quality considerations. Two types of image processing were used to enhance the prints, which were then used to make moulds by etching copper-clad printed circuit board. Three types of silicone rubber were used to make a total of 36 artificial fingerpads, which were tested on the system and the optimum found. The optimum pad was then used to attack the system and the vulnerability calculated. The system proved to be extremely vulnerable, with a Vulnerability Assurance Level (VAL) of VAL0 for most conditions. This means that a spoofing success rate less than one in three cannot be guaranteed. A ProSkin artefact covering a finger pad without adhesive is pictured on this page. If adhesive were used, the fit to the finger would be even tighter and harder to detect.
Lambtemplate attack A ‘lamb’ template is a database template capable of being matched by several attackers. Several approaches are possible for creating a Lambtemplate, mainly:
ProSkin artefact covering a finger pad (without adhesive).
May 2011
FEATURE • Template with Dominant Features: One or more strong features dominate an enrolled template image, to the extent of affecting the features extracted by the system and used for matching. Examples might be thick, dark moustache and beard (face recognition) or distinct scars or tattoos (fingerprints). A problem with this method is that each attacker has to change his physical biometric to accentuate the added features. It is also likely to be ineffective for the fingerprint biometric since most systems are minutiaebased, and adept at ignoring surface features such as scars and discoloration • ‘Hillclimb’: This is the name of a general optimisation process which, given a good algorithm and sufficient time, may be able to produce a synthetic print capable of being matched by several attackers. This method is expected to work for image-based features but will have more difficulty if minutiaebased features are used. • Composite Template: This is perhaps the simplest method and is the one chosen for the present study. A composite template is formed from parts of fingerprint images drawn from a number of attackers. This method is ideally suited for minutia-based systems. A disadvantage is that the more attackers are included, the less information in the composite relates to any one attacker, reducing the overall likelihood of spoofing the system. The template may have insufficient information to enable some attackers to spoof the system successfully. Whatever method is used, once the template has been constructed it must be inserted into the database. This may be accomplished by collusion with an operator (or a database entry officer). Without this level of access it may be possible to enrol a composite identity using an artefact. However, this requires either unsupervised enrolment or a composite that could be mistaken as a real fingerprint.
region, due to misalignment or other factors, did lead to successful spoofs.
Conclusion
Composite print from four attackers, no overlap.
Each quadrant may in fact comprise more than a quarter of the source images, and this ‘overlap’ may enhance performance by including more minutiae. The degree of overlap is a potential variable. For the Lambtemplate Attack, 12 composite templates were constructed, using different groupings of four attackers and a print ‘overlap’ parameter. Again, testing was used to provide the optimum template, which was then used to repeatedly attack the system, using the four associated attackers. The system once again proved to be highly vulnerable to this attack, with VAL0 applying for most conditions.
Iris system In 2010 a commercial iris system with liveness turned on was tested using BVA. The tested system was shown to be resistant to simple 2D artefacts including laser printed and inkjet printed pictures and an image shown on an iPhone screen. Wearing the fake contact lenses was also not successful in spoofing the system, however this may have been due to difficulties in maintaining alignment of the pupil and the contact lens.
"In 2010 a commercial iris "Whatever method is used, system with liveness turned once the template has been on was tested using BVA. constructed it must be The tested system was inserted into the database. shown to be resistant to This may be accomplished simple 2D artefacts including by collusion with an operator laser printed and inkjet or a database entry officer" printed pictures and an image shown on an iPhone The third choice outlined above was choscreen" sen, with composite images made from partial fingerprint images from a number of attackers, each partial print being one quadrant of the appropriate attacker.
May 2011
Subsequent tests using a glass eye with a contact lens and blacked-out pupil demonstrated that the removal of visible artefacts in the pupil
Several projects have been successfully completed, yielding statistically significant assessments of the vulnerability of various systems under a variety of types of attack. Formal vulnerability testing for biometric systems has in the past been less effective than desired. The methodology developed by the Biometrics Institute can be used to assess biometric systems and rate their vulnerability performance. It has been designed to ensure that the evaluation is cost effective and leads to a practical outputs that can be used as part of the risk analysis process for any system using biometrics. It is also hoped that it will spur developments in techniques and procedures that can be used to mitigate biometric vulnerabilities.
Expert Group The Biometrics Institute now offers a Biometric Vulnerability Assessment Service (BVAS). In 2010, it established a Biometrics Vulnerability Assessment Expert Group (BVAEG) including representatives from Australia, Germany, Spain and the UK to look at closer collaboration on vulnerability assessments in regards to raising awareness and developing common standards.
About the authors Dr Ted Dunstone is the founder and currently also the Chair of the Technical Committee of the Biometrics Institute Ltd. He is also CEO of Biometix, provider of biometric data mining and border control. Over the past 15 years he has provided consultancy services analysis tools. He is a recognized expert in biometric risk assessment, biometric data to many government agencies and is a frequent contributor to Australian government debate on the application of biometrics. Geoff Poulton is a member of the Technical Committee of the Biometrics Institute. If your organisation is interested in having a system tested or to find out more about the BVAEG please contact the Biometrics Institute
References 1 ‘Biometric Vulnerability: A Principled Assessment Methodology’, Dunstone T, PoultonG. A white paper produced for the Biometrics Institute, September 2008.
Biometric Technology Today
7
Vulnerability assessment Ted Dunstone, Geoff Poulton, Biometrics Institute In the popular portrayal of biometrics, from Minority Report to the recent Get Smart movie, a common theme is the defeat of biometric systems using faked (or spoofed) biometrics. While such fears make for good stories, the issues relating to the vulnerability of biometric systems are all too real. It is these issues that the Biometrics Institute, Australia, is addressing by introducing a methodology for the identification and assessment of vulnerabilities – the Biometrics Vulnerability Assessment methodology (BVA). Many biometric products now claim some degree of spoof detection, but questions such as how this vulnerability resistance was tested and what kinds of attack might have a likelihood of success are mostly left unanswered. If you think this issue is of mainly academic concern try a search for ‘fake fingerprint’ in YouTube. This turns up some quite detailed information on the creation of artificial fingerprints. In this context, the Biometrics Institute, with partial funding from the Australian Government, has developed a framework to assess the vulnerabilities in biometric systems and to give scientifically credible results for the likelihood of an attack succeeding. Since 2007 the methodology has been applied to a face, finger, voice and iris biometrics and in each case it has demonstrated vulnerabilities, often previously unknown, in commercially available systems. The findings from many of these studies are subject to publication restrictions for security reasons, however this article will discuss both the methodology and how it was applied to assess a fingerprint and iris system.
Vulnerability versus accuracy The vulnerability of a biometric system should not be confused with its accuracy. It is possible to have a system that is extremely accurate at distinguishing the correct person under normal conditions, but which is highly vulnerable to simple methods used to circumvent the security. This might be by mimicking physical biological characteristics, or by bypassing or altering the information used as part of the matching process. By way of analogy consider your front door lock. Performance testing would tell you how likely it is that any randomly selected key will unlock the door, vulnerability testing will tell you what is the likelihood that a thief might successfully pick your lock.
May 2011
“It is possible to have a system that is extremely accurate at distinguishing the correct person under normal conditions, but which is highly vulnerable to simple methods used to circumvent the security” Significant effort goes into improving the accuracy of the biometric algorithms for the task of distinguishing between different people, but much less is generally known about how the systems will stand up to determined attack. Most large-scale testing results on biometrics measure false match and false non-match rates, the former focusing on the chance that a random person has biometric characteristics that are sufficiently similar to pass as that of another individual. Vulnerability testing on the other hand needs to focus on identifying likely threats, optimising the attack methods and the measuring their success.
BVA methodology The main aim of the Biometric Institute’s vulnerability assessment methodology is to provide a reliable level of assurance against a spoofing attack for a particular threat on a particular technology. Other aims are to ensure the assessment is timely and cost effective, to incorporate
the flexibility to adjust to different biometric modalities in an evolving threat landscape, and to allow it to be part of a wider security assessment and threat mitigation process. The threat landscape is complex, as a wide variety of factors need to be considered during the evaluation. Each biometric product will have different levels of vulnerability to a range of threats, and each threat is dependent on the attributes of an attacker. The fundamental building block of this evaluation methodology is that the spoofing risk can be broken in two factors: the Exploitation Potential, which relies on the biometric system’s properties, and the Attack Potential, which is related to the capabilities of the attacker. The spoofing risk can then be defined as: Spoofing Risk = Attack Potential x Exploitation Potential Where the Attack Potential is a number between 0 and 1 that gives the relative strength of the attacker. This in turn depends on the attacker’s knowledge about the system, his or her access to it and the particular method of attack to be used. In the simplest implementation AP is estimated on the basis of experience. The Exploitation Potential is the likelihood that a particular vulnerability exists, and that it can be exploited to breach the system. EP is obtained from experimental measurements. For the particular attack method chosen, the likely strength of an attacker with reasonable resources is used to estimate the Attack Potential. Measurement and analysis is then carried out in two phases. Phase 1 is Exploration. An experimental program whose object is to determine the optimum parameters for the attack – those parameters most likely to succeed. For example, parameters for an Artefact Attack on a face biometric might include the identity of the enrolled subject and the size, resolution and presentation method of a photo artefact. This
Vulnerability Assurance Levels VAL4
Fewer than 1 in 100 attacks are likely to succeed
VAL3
Fewer than 1 in 30 attacks are likely to succeed
VAL2
Fewer than 1 in 10 attacks are likely to succeed
VAL1
Fewer than 1 in 3 attacks are likely to succeed
VAL0
Cannot guarantee at least VAL 1, A system whose level for a particular attack is VAL0 essentially has no protection from that attack.
Biometric Technology Today
5
FEATURE “The two attack methods were the construction of false fingers (artefacts), an attack on the input and algorithmic aspects of the system and the enrolment of an artificial fingerprint into the database”
A
B
Artefact attack
C
D
E
F
Full set of 36 artefacts made for Phase 1 testing. (a) Silicone Putty, 1mm; enhanced, (b) same, processed, (c) Pinksil, 1mm, enhanced, (d) same, processed; (e) ProSkin, 0.5mm, enhanced, (f) same, processed.
phase usually comprises a number of repeated attacks to test the parameter space. Phase 2 is evaluation. Using the optimum parameters from Phase 1, the system is attacked many times and the results recorded. Appropriate statistical calculations are then used to determine the Exploitation Potential. This, together with Attack Potential, is then used to find the spoofing risk, giving a direct measure of the degree of vulnerability assurance. A key part of the Biometrics Institute methodology is to measure vulnerability in terms of Vulnerability Assurance Levels. The Vulnerability Assurance Level (VAL) is a logarithmic measure indicating the assurance that there will be no more than a given percentage of successes from repeated single attacks on the system. The first five levels are shown in the table Vulnerability Assurance Levels and are computed with a confidence interval of 95%. Higher levels are possible. 6
Biometric Technology Today
Fingerprint system In February 2009, the Biometrics Institute commissioned an independent study that applied the methodology to the evaluation of two spoofing attack methods on a commercial fingerprint biometric system. The fingerprint system tested was a DigitalPersona Pro U.are.U 4000B scanner with Neurometrics Verifinger software. The two attack methods were the construction of false fingers (artefacts), an attack on the input and algorithmic aspects of the system and the enrolment of an artificial fingerprint into the database, designed to allow entry by multiple attackers (known as a ‘lamb’ template attack). This is an attack on the database and the enrolment process. The effect of larger attacker numbers was also investigated.
A small database of fingerprint images was first constructed, comprising four instances each of 20 attacker fingers. The methodology first optimises parameters of the attack, and then repeatedly attacks the system with the optimised parameters. For the Artefact Attack, six of the possible 20 identities were preselected on print quality considerations. Two types of image processing were used to enhance the prints, which were then used to make moulds by etching copper-clad printed circuit board. Three types of silicone rubber were used to make a total of 36 artificial fingerpads, which were tested on the system and the optimum found. The optimum pad was then used to attack the system and the vulnerability calculated. The system proved to be extremely vulnerable, with a Vulnerability Assurance Level (VAL) of VAL0 for most conditions. This means that a spoofing success rate less than one in three cannot be guaranteed. A ProSkin artefact covering a finger pad without adhesive is pictured on this page. If adhesive were used, the fit to the finger would be even tighter and harder to detect.
Lambtemplate attack A ‘lamb’ template is a database template capable of being matched by several attackers. Several approaches are possible for creating a Lambtemplate, mainly:
ProSkin artefact covering a finger pad (without adhesive).
May 2011
FEATURE • Template with Dominant Features: One or more strong features dominate an enrolled template image, to the extent of affecting the features extracted by the system and used for matching. Examples might be thick, dark moustache and beard (face recognition) or distinct scars or tattoos (fingerprints). A problem with this method is that each attacker has to change his physical biometric to accentuate the added features. It is also likely to be ineffective for the fingerprint biometric since most systems are minutiaebased, and adept at ignoring surface features such as scars and discoloration • ‘Hillclimb’: This is the name of a general optimisation process which, given a good algorithm and sufficient time, may be able to produce a synthetic print capable of being matched by several attackers. This method is expected to work for image-based features but will have more difficulty if minutiaebased features are used. • Composite Template: This is perhaps the simplest method and is the one chosen for the present study. A composite template is formed from parts of fingerprint images drawn from a number of attackers. This method is ideally suited for minutia-based systems. A disadvantage is that the more attackers are included, the less information in the composite relates to any one attacker, reducing the overall likelihood of spoofing the system. The template may have insufficient information to enable some attackers to spoof the system successfully. Whatever method is used, once the template has been constructed it must be inserted into the database. This may be accomplished by collusion with an operator (or a database entry officer). Without this level of access it may be possible to enrol a composite identity using an artefact. However, this requires either unsupervised enrolment or a composite that could be mistaken as a real fingerprint.
region, due to misalignment or other factors, did lead to successful spoofs.
Conclusion
Composite print from four attackers, no overlap.
Each quadrant may in fact comprise more than a quarter of the source images, and this ‘overlap’ may enhance performance by including more minutiae. The degree of overlap is a potential variable. For the Lambtemplate Attack, 12 composite templates were constructed, using different groupings of four attackers and a print ‘overlap’ parameter. Again, testing was used to provide the optimum template, which was then used to repeatedly attack the system, using the four associated attackers. The system once again proved to be highly vulnerable to this attack, with VAL0 applying for most conditions.
Iris system In 2010 a commercial iris system with liveness turned on was tested using BVA. The tested system was shown to be resistant to simple 2D artefacts including laser printed and inkjet printed pictures and an image shown on an iPhone screen. Wearing the fake contact lenses was also not successful in spoofing the system, however this may have been due to difficulties in maintaining alignment of the pupil and the contact lens.
"In 2010 a commercial iris "Whatever method is used, system with liveness turned once the template has been on was tested using BVA. constructed it must be The tested system was inserted into the database. shown to be resistant to This may be accomplished simple 2D artefacts including by collusion with an operator laser printed and inkjet or a database entry officer" printed pictures and an image shown on an iPhone The third choice outlined above was choscreen" sen, with composite images made from partial fingerprint images from a number of attackers, each partial print being one quadrant of the appropriate attacker.
May 2011
Subsequent tests using a glass eye with a contact lens and blacked-out pupil demonstrated that the removal of visible artefacts in the pupil
Several projects have been successfully completed, yielding statistically significant assessments of the vulnerability of various systems under a variety of types of attack. Formal vulnerability testing for biometric systems has in the past been less effective than desired. The methodology developed by the Biometrics Institute can be used to assess biometric systems and rate their vulnerability performance. It has been designed to ensure that the evaluation is cost effective and leads to a practical outputs that can be used as part of the risk analysis process for any system using biometrics. It is also hoped that it will spur developments in techniques and procedures that can be used to mitigate biometric vulnerabilities.
Expert Group The Biometrics Institute now offers a Biometric Vulnerability Assessment Service (BVAS). In 2010, it established a Biometrics Vulnerability Assessment Expert Group (BVAEG) including representatives from Australia, Germany, Spain and the UK to look at closer collaboration on vulnerability assessments in regards to raising awareness and developing common standards.
About the authors Dr Ted Dunstone is the founder and currently also the Chair of the Technical Committee of the Biometrics Institute Ltd. He is also CEO of Biometix, provider of biometric data mining and border control. Over the past 15 years he has provided consultancy services analysis tools. He is a recognized expert in biometric risk assessment, biometric data to many government agencies and is a frequent contributor to Australian government debate on the application of biometrics. Geoff Poulton is a member of the Technical Committee of the Biometrics Institute. If your organisation is interested in having a system tested or to find out more about the BVAEG please contact the Biometrics Institute
References 1 ‘Biometric Vulnerability: A Principled Assessment Methodology’, Dunstone T, PoultonG. A white paper produced for the Biometrics Institute, September 2008.
Biometric Technology Today
7