Automatic Software-Based Self Test Generation for Embedded Processors

Proceedings, 15th IFAC Conference on Proceedings, IFAC and Conference on Systems Programmable15th Devices Embedded Proceedings, 15th IFAC Conference on Available Proceedings, 15th IFAC Conference Programmable Devices Embedded Systemsonline at www.sciencedirect.com Proceedings, 15th IFAC and Conference on Ostrava, Czech Republic, May 23-25,on 2018 Programmable Devices and Embedded Systems Programmable Devices and Embedded Systems Ostrava, Czech Republic, May 23-25,on 2018 Proceedings, 15th IFAC Conference Programmable Devices and Embedded Systems Ostrava, Czech Republic, May 23-25, 2018 Ostrava, Republic, May 23-25, Programmable and Embedded Systems Ostrava, Czech Czech Devices Republic, May 23-25, 2018 2018 Ostrava, Czech Republic, May 23-25, 2018

ScienceDirect

IFAC PapersOnLine 51-6 (2018) 125–130

Automatic Software-Based Self Test Generation for Embedded Processors Automatic Software-Based Self Test Generation for Embedded Processors Automatic Software-Based Self Test Generation for Embedded Automatic Software-Based Software-Based Self Self Test Test Generation Generation for for Embedded Embedded Processors Processors Automatic Processors JánGeneration Hudec* Automatic Software-Based Self Test for Embedded Processors Ján Hudec*

 Ján Ján Hudec* Hudec* Ján  Hudec*and Applied Informatics *Institute of Computer Engineering Ján Hudec*and Applied Informatics *Institute of Computer Engineering *Institute of Computer Engineering and Applied Informatics Faculty of Informatics and Information Technologies Slovak University of Technology  *Institute of Engineering and Applied Informatics Faculty of Informatics and Information Technologies Slovak University of Technology *Institute of Computer Computer Engineering and Applied Informatics Faculty of Informatics and Information Technologies Slovak University of 84216 Bratislava, Slovakia, (e-mail: [email protected]) *Institute of Computer Engineering and Applied Informatics Faculty of of Informatics Informatics and Information Technologies Slovak University University of Technology Technology 84216 Bratislava, Slovakia, (e-mail: [email protected]) Faculty and Information Technologies Slovak of Technology 84216 Slovakia, (e-mail: Faculty of Informatics and Information Technologies Slovak University of Technology 84216 Bratislava, Bratislava, Slovakia, (e-mail: [email protected]) [email protected]) 84216 Bratislava, Slovakia, (e-mail: [email protected]) 84216 Bratislava, Slovakia, (e-mail: [email protected]) Abstract: The paper deals with automatic software-based test generation for processors. Processors are Abstract: The paper deals with automatic automatic software-based software-based test test generation generation for for processors. processors. Processors Processors are are Abstract: The deals with basic blocks ofpaper current complex systems software-based on chip and embedded systems. Processors testing can are be Abstract: The paper deals with automatic test generation for processors. Processors basic blocks of current complex systems on chip and embedded systems. Processors testing can be Abstract: The paper deals with automatic software-based test generation for processors. Processors are basic blocks of current complex systems on chip and embedded systems. Processors testing can be extended by functional tests or using various application programs. Such types of tests are serving as Abstract: The paper deals with automatic software-based test generation for processors. Processors are basic blocks of current complex systems on chip and embedded systems. Processors testing can be extended by functional tests or using various application programs. Such types of tests are serving as basic blocks of current complex systems on chip and embedded systems. Processors testing can be extended by functional tests or using various application programs. Such types of tests are serving as additional tests to structural testing or as tests used in verification. Functional tests (programs) are basic blocks of to current complex systems on chip andinembedded systems. Processors testing can are be extended by functional tests or using various application programs. Such types of tests are serving as additional tests structural testing or as tests used verification. Functional tests (programs) extended by functional tests or using various application programs. Such types of tests are serving as additional tests to structural testing or as tests used in verification. Functional tests (programs) are generated over an instruction set architecture and processor model Such description. A metric for quality extended by functional tests or using various application programs. types of tests are serving as additional tests to structural testing or as tests used in verification. Functional tests (programs) are generated over an instruction set architecture and processor model description. A metric for quality additional tests to structural testing or as tests used in verification. Functional tests (programs) are generated over instruction set architecture and processor description. metric for quality evaluation of thean software-based tests is as obviously provided bymodel code coverage of aA processor model. A additional tests to structural testing or tests used in verification. Functional tests (programs) are generated over an instruction set architecture and processor model description. A metric for quality evaluation of the theansoftware-based software-based tests is obviously obviously provided bymodel code coverage coverage of aaAprocessor processor model. A generated over instructionmethod settests architecture and processor description. metric for quality evaluation of is provided by code of model. A new functional test generation is based on VHDL model of processors and genetic algorithms with generated over ansoftware-based instructionmethod settests architecture and processor model description. metric for quality evaluation of the is obviously provided by code coverage of aaAprocessor model. A new functional test generation is based on VHDL model of processors and genetic algorithms with evaluation of the software-based tests is obviously provided by code coverage of processor model. A new functional test generation method is on model processors and algorithms with using various strategies. contribution to the SBST methods based on usingmodel. the latest evaluation of evolutionary the testsThe obviously provided byof coverage a GAs processor A new functional testsoftware-based generation method isisbased based on VHDL VHDL model ofcode processors andofgenetic genetic algorithms with using various evolutionary strategies. The contribution to the SBST methods based on GAs using the latest new functional test generation method is based on VHDL model of processors and genetic algorithms with using various evolutionary strategies. The contribution to the SBST methods based on GAs using the latest defined ES was identified. Functionality effectiveness of SBST the of developed the new functional test generation methodThe is and based on VHDL model processors and genetic algorithms with using various evolutionary strategies. The contribution to the the methodsmethods based onwere GAsevaluated using the the in latest defined ES was identified. Functionality and effectiveness of the developed methods were evaluated in the using various evolutionary strategies. contribution to SBST methods based on GAs using latest defined ES was identified. Functionality effectiveness of the developed the implemented system AGenMIX over two types of RISCtoprocessor. using various evolutionary strategies. Theand contribution the methodsmethods based onwere GAsevaluated using the in latest defined ES identified. Functionality and effectiveness of the methods were evaluated in the implemented system AGenMIX over two two types of RISC RISC processor. processor. defined ES was was identified. Functionality and effectiveness of SBST the developed developed methods were evaluated in the implemented system AGenMIX over types of defined ES was identified. Functionality and effectiveness of the developed methods were evaluated in the implemented system AGenMIX over two types of RISC processor. © 2018, IFACProcessor, (International Federation of Automatic Control) Hosting by Elsevier All rights reserved. implemented system AGenMIX overtesting, two types of RISC processor. Keywords: functional software-based self test, geneticLtd. algorithms, evolutionary Keywords: Processor, functional testing, software-based self test, genetic algorithms, evolutionary implemented system AGenMIX over two types of RISC processor. Keywords: Processor, functional testing, software-based self test, genetic algorithms, evolutionary strategies, VHDL code coverage, fitness function Keywords: Processor, functional testing, software-based strategies, VHDL VHDL code coverage, coverage, fitness function Keywords: Processor, functionalfitness testing, software-based self self test, test, genetic genetic algorithms, algorithms, evolutionary evolutionary strategies, code function Keywords: Processor, functionalfitness testing, software-based self test, genetic algorithms, evolutionary strategies, code function strategies, VHDL VHDL code coverage, coverage, fitness function  strategies, VHDL code coverage, fitness function  experimental results (if the code coverage for a SBST tests is  1. INTRODUCTION experimental results (if the code coverage for aa SBST tests is experimental the code coverage tests is more than 90results % thus(if sufficientfor stuck-at fault 1. INTRODUCTION  experimental results (ifthe thetests codeis coverage forfor a SBST SBST tests is 1. INTRODUCTION more than 90 % thus the tests is sufficient for stuck-at fault experimental results (if the code coverage for a SBST tests is 1. INTRODUCTION more than 90 % thus the tests is sufficient for stuck-at fault coverage). 1. INTRODUCTION Embedded processors are often used in systems that are experimental results (ifthe thetests codeis coverage forfor a SBST tests is more than 90 % thus sufficient stuck-at fault coverage). more than 90 % thus the tests is sufficient for stuck-at fault Embedded processors are often used in systems that are 1. INTRODUCTION coverage). Embedded processors are often used in systems that are safety-critical and long-living. Reliability of nowadays system more than 90 % thus the tests is sufficient for stuck-at fault coverage). Embedded processors are used in systems that are coverage). The paper is organized as follows. The next section brings a safety-critical and long-living. Reliability nowadays system Embedded processors are often often used has inof systems that safety-critical and long-living. Reliability of nowadays system on chips (SoCs) and embedded systems to be solved at are the coverage). The paper is organized as follows. The next section brings a Embedded processors are often used in systems that are safety-critical and long-living. Reliability of nowadays system The paper is organized as follows. section brings short state-of-the-art related to The the next software-based testaa on chips (SoCs) and embedded systems has to be solved at the safety-critical and long-living. Reliability oftonowadays system The paper is organized as follows. The next section brings on chips (SoCs) and embedded systems has be solved at the design level including high quality of their verification and short state-of-the-art related to the software-based test The paper is organized as follows. The next section brings a safety-critical and long-living. Reliability of nowadays system on chipslevel (SoCs) and embedded embedded systems has to verification be solved solved at atand the generation short state-of-the-art related to the software-based test processors. Section 3 describes design including high quality of their on chips (SoCs) and systems has to be the The paper ismethods organizedfor as follows. The next section brings a short state-of-the-art related to the software-based test design level including high quality of their verification and life-time testing. Processors are basic blocks of each SoC generation methods for processors. Section 3 describes short state-of-the-art related to the software-based test on chips (SoCs) and embedded systems has to be solved at the design level including high are quality ofblocks their of verification and generation methods for processors. Section 3 describes principles of the new adaptive test generation method. life-time testing. Processors basic each SoC design level including high quality of their verification and short state-of-the-art related to the software-based test generation methods for processors. Section 3 describes life-time testing. Processors are basic blocks of each SoC and embedded systems therefore they need testing in the same principles of the new test generation method. generation methods for adaptive processors. Section 3 describes design including high are quality ofblocks their verification and life-time testing. Processors basic of each SoC principles of the test generation method. andnew selected experimental results over the embedded systems therefore they need testing in the same life-timelevel testing. are basic blocks of each SoC and Implementation generation methods for adaptive processors. Section 3 describes principles of the new adaptive test generation method. embedded systems therefore they testing in the same frequency as realProcessors performance andneed more and more to drive Implementation and selected experimental results over the principles of the new adaptive test generation method. life-time testing. Processors are basic blocks of each SoC and embedded systems therefore they need testing in the same Implementation and selected experimental results over the RISC type processors DP32 and DLX are presented in section frequency as real performance and more and more to drive embedded systems therefore they need testing in the same principles of the new adaptive test generation method. Implementation and selected selected experimental results inover over the frequency as real performance and more and more to drive testing similar to functional modes. Functional and softwareRISC type processors DP32 and DLX are presented section Implementation and experimental results the embedded systems therefore they need testing in the same frequency as real performance and more and more to drive RISC type processors DP32 and DLX are presented in section Conclusion is the section of the paper. testing similar to functional modes. and softwarefrequency asare real performance and Functional more and more to drive 4. Implementation andlast selected results in the RISC type processors processors DP32 andexperimental DLX are presented presented inover section testing similar to functional modes. Functional and softwarebased tests important for design verification, functional 4. Conclusion is the last section of the paper. RISC type DP32 and DLX are section frequency asare real performance and Functional more and more to drive 4. Conclusion is the last section of the paper. testing similar to functional modes. and softwarebased tests important for design verification, functional testing similar to functional modes. Functional and softwareRISC type processors DP32 and DLX are presented in section 4. Conclusion is the last section of the paper. based tests are important for design verification, simulation, enriched manufacturing testing processors but also 4. Conclusion is the last section of the paper. testing similar functional Functional and functional softwarebased tests are important for design verification, functional RELATED simulation, enriched manufacturing testing processors but also based tests aretoSoCs important formodes. design verification, functional 4. Conclusion is the2. last section ofWORKS the paper. simulation, enriched manufacturing testing processors but also testing during life-time. The test pattern generation 2. RELATED WORKS based tests are important for design verification, functional simulation, enriched manufacturing testing processors but also 2. RELATED WORKS testing during SoCs life-time. The test pattern generation simulation, enriched manufacturing testing processors but also 2. RELATED WORKS testing during SoCs life-time. The test pattern generation (TPG) targeted to software-based and self-testing 2. RELATED WORKS on functional testing processors and simulation, enriched manufacturing but also There is a long history testing methods during SoCs life-time. Thetesting test processors pattern generation (TPG) methods targeted to software-based and self-testing testing during SoCs life-time. The test pattern generation There is aa long history on functional testing processors and 2. RELATED WORKS (TPG) methods targeted to software-based and self-testing (SBST) processors over their instruction set architecture (ISA) There is long history on functional testing processors microprocessors. Fault coverage is low in application of and the testing during SoCs life-time. The test pattern generation (TPG) methods methods targeted to instruction software-based and self-testing self-testing (SBST) processors over their set architecture (ISA) There is aa long history on functional testing processors and (TPG) targeted to software-based and microprocessors. Fault coverage is low in application of the There is long history on functional testing processors and (SBST) processors over their instruction set architecture (ISA) have been developed and published. The tests are applied at microprocessors. Fault coverage is low in application of the earlier developed TPG methods for functional testing. They (TPG) methods targeted to software-based and self-testing (SBST) processors over their instruction set architecture (ISA) There isdeveloped a long history on functional testing processors and have been developed and published. The tests are applied at microprocessors. Fault coverage is low in application of the (SBST) processors over their instruction setminimal architecture (ISA) earlier TPG methods for functional testing. They microprocessors. Fault coverage is low in application of the have been developed and published. The tests are applied at normal operating processor frequency and or even no earlier developed TPG methods for functional testing. They have started more from behavioural level and not from (SBST) processors over their instruction set architecture (ISA) have been developed and published. The tests are applied at microprocessors. Fault coverage is low in application of the normal operating processor frequency and minimal or even no earlier developed TPG methods for functional testing. They have been developed and published. The tests are applied at have started more from behavioural level and not from earlier developed TPG methods for functional testing. They normal operating processor frequency and minimal or even no circuit modifications are needed for running SBST tests have started more from behavioural level from transfer level (RTL) or structural leveland of not processor have been developed and published. The tests areSBST applied at register normal operating processor frequency and minimal or even no earlier developed TPG methods for functional testing. They circuit modifications are needed for running tests have started more from behavioural level and not from normal operating processor frequency and minimal or even no register transfer level (RTL) or structural level of processor have started more from behavioural level and not from circuit modifications are needed for running SBST tests performed in normal operating mode. Various methods have transfer level (RTL) or level of processor models. Recently, some research activities been normal operating processor frequency minimal or even no register circuit modifications are for running SBST tests have started more behavioural level not from performed in normal operating mode. Various methods have register transfer levelfrom (RTL) or structural structural leveland ofhave processor circuit modifications are needed needed forand running SBST tests models. Recently, some research activities have been register transfer level (RTL) or structural level of processor performed in normal operating mode. Various methods have been presented for the functional TPG based on different fault models. Recently, some research activities have been initialized towards functional testing using mainly RTL. The circuit modifications are needed for running SBST tests performed in normal operating mode. Various methods have register transfer level (RTL) or structural level of processor been presented for the functional TPG based on different fault models. Recently, some research activities have been performed in normal operating mode. Various methods have initialized towards functional testing using mainly RTL. The models. Recently, some research activities have been been presented for the functional TPG based on different fault models (Bernardi et.al. 2011, Schölzel et.al. 2013, Psarakis initialized towards functional testing using mainly RTL. The processor applies functional test programs named test mixes performed in normal operating mode. Various methods have been presented for the functional TPG based on different fault models. Recently, some research activities have been models (Bernardi et.al. 2011, Schölzel et.al. 2013, Psarakis initialized towards functional testing using mainly RTL. The been presented for the functional TPG based on different fault processor applies functional test programs named test mixes initialized towards functional testing using mainly RTL. The models (Bernardi et.al. 2011, Schölzel et.al. 2013, Psarakis et.al. 2010, Hudec, 2013). Some of them use genetic processor applies functional test programs named test mixes in the paper (this notation is defined and presented in section been presented for the functional TPG based on different fault models2010, (Bernardi et.al.2013). 2011, Some Schölzelof et.al. et.al. 2013, Psarakis initialized towards functional testing using mainly RTL. The et.al. Hudec, them use genetic processor applies functional test programs named test mixes models (Bernardi et.al. 2011, Schölzel 2013, Psarakis in the paper (this notation is defined and presented in section processor applies functional test programs named test mixes et.al. 2010, Hudec, 2013). Some of them use genetic algorithms (GAs) with selected evolutionary strategy (ES) in the paper (this notation is defined and in section four) using its native ISA. The test presented mixes run atmixes the models2010, (Bernardi et.al.one 2011, Schölzel et.al. 2010, Hudec, 2013). Some of et.al. them2013, use Psarakis genetic processor applies functional test programs named test algorithms (GAs) with one selected evolutionary strategy (ES) in the paper (this notation is defined and presented in section et.al. Hudec, 2013). Some of them use genetic four) using its native ISA. The test mixes run at the in the paper (this notation isfrequency. defined and presented in section algorithms (GAs) with one evolutionary strategy (ES) and randomly generated aselected first population (Bernardi et.al. four) using its native ISA. The test mixes run at the processor´s actual operating et.al. 2010, Hudec, 2013). Some of them use genetic algorithms (GAs) with one selected evolutionary strategy (ES) in the paper (this notation isfrequency. defined and presented in section and randomly generated aaselected first population (Bernardi et.al. four) using its native ISA. The test mixes run at the algorithms (GAs) with one evolutionary strategy (ES) processor´s actual operating four) using its native ISA. The test mixes run at the and randomly generated first population (Bernardi et.al. 2006, Corno et.al. 2004, Sanchez et.al. 2006). Quality of fault processor´s actual operating frequency. algorithms (GAs) with one selected evolutionary strategy (ES) and randomly generated a first population (Bernardi et.al. four) using its native ISA. The test mixes run at the 2006, Corno et.al. 2004, Sanchez et.al. 2006). Quality of fault processor´s actual operating frequency. and randomly generated a first population (Bernardi et.al. processor´s actualhave operating frequency. Various methods been presented for the functional TPG 2006, Corno et.al. 2004, Sanchez et.al. 2006). Quality of fault coverage is basically evaluated by code coverage calculation and randomly generated a firstbyet.al. population (Bernardi et.al. 2006, Corno et.al. 2004, Sanchez 2006). Quality of fault Various methods have been presented for the functional TPG processor´s actual operating frequency. coverage is basically evaluated code coverage calculation 2006, Corno et.al. 2004, Sanchez et.al. 2006). Quality of fault Various methods have been presented for the functional TPG on different fault models (Bernardi et.al. 2011, coverage is basically evaluated code coverage calculation using professional simulator. Various methods have been for functional TPG 2006, aaCorno et.al. 2004, Sanchezby Quality of fault based coverage is evaluated by code coverage calculation based different fault models (Bernardi et.al. 2011, Variouson methods haveSchölzel been presented presented for the theThe functional TPG using professional simulator. coverage is basically basically evaluated byet.al. code2006). coverage calculation based on different fault models (Bernardi et.al. 2011, Psarakis et.al. 2010, et.al. 2013). use genetic using a professional simulator. Various methods have been presented for the functional TPG based on different fault models (Bernardi et.al. 2011, coverage is basically evaluated by code coverage calculation using a professional simulator. Psarakis et.al. 2010, Schölzel et.al. 2013). The use genetic based on different fault models (Bernardi et.al. 2011, using a professional simulator. The evaluation metric was defined in (Corno et.al. 2003). algorithms Psarakis et.al. 2010, Schölzel et.al. 2013). The use genetic (GAs) with one selected evolutionary strategy (ES) based on different fault models (Bernardi et.al. 2011, The evaluation metric was defined in (Corno et.al. 2003). Psarakis et.al. et.al. 2010, Schölzel et.al.evolutionary 2013). The Thestrategy use genetic genetic using a professional simulator. (GAs) with one selected (ES) Psarakis 2010, Schölzel et.al. 2013). use The evaluation metric was defined in (Corno et.al. 2003). Relationship to structural fault coverage can be evaluated by algorithms algorithms (GAs) with one selected evolutionary strategy (ES) and randomly generated a first population (starting test The evaluation metric was defined in (Corno et.al. 2003). Psarakis et.al. 2010, Schölzel et.al. 2013). The use genetic Relationship to structural fault coverage can be evaluated by algorithms (GAs) with one selected evolutionary strategy (ES) The evaluation metric was defined in (Corno et.al. 2003). and randomly generated a first population (starting test algorithms (GAs) with one selected evolutionary strategy (ES) Relationship to structural fault coverage can be evaluated by decoding the SBST tests to binary vectors and using a and randomly generated a first population (starting test The evaluation metric was defined in (Corno et.al. 2003). (Bernardi et.al. (starting 2006, Corno Relationship toSBST structural fault coverage can be be and evaluated bya algorithms programs/test mixes) used in (GAs) withis one selected evolutionary strategy (ES) decoding the tests to binary vectors using and randomly generated a first population test Relationship to structural fault coverage can evaluated by and randomly generated a first population (starting test ( Bernardi et.al. 2006, Corno programs/test mixes) is used in decoding the SBST tests to binary vectors and using a professional fault simulator, e.g. for stuck-at faults. A ( Bernardi et.al. 2006, Corno Relationship to structural fault coverage can be evaluated by programs/test mixes) is used in decoding the fault SBSTsimulator, tests to to binary binary vectors andfaults. using Aaa and et.al. 2004, Sanchez et.al. 2006, Hudec, 1994). Fault coverage randomly generated a first population (starting test professional e.g. for stuck-at decoding the SBST tests vectors and using (Bernardi Bernardi et.al.Fault 2006, Corno programs/test mixes)et.al. is used used in Hudec, professional fault simulator, e.g. for stuck-at faults. A et.al. 2004, Sanchez 2006, 1994). coverage ( et.al. 2006, Corno programs/test mixes) is in correlation between defined code metrics and structural stuckdecoding the SBSTdefined tests code to binary vectors andfaults. using a programs/test et.al. 2004, Sanchez et.al. 2006, Hudec, 1994). Fault coverage professional fault simulator, e.g. for stuck-at A quality is basically evaluated by code coverage calculation correlation between metrics and structural stuckprofessional fault simulator, e.g. for stuck-at faults. A ( Bernardi et.al. 2006, Corno mixes) is used in et.al. 2004, 2004, Sanchez et.al. et.al. 2006,by Hudec, 1994). Fault coverage correlation between defined code metrics and structural stuckis basically evaluated code coverage calculation et.al. 2006, Hudec, Fault coverage at fault coverage presented ine.g. (Corno 2006,faults. Sanchez professional faultis simulator, for et.al. stuck-at A quality quality is basically evaluated by code coverage calculation correlation between defined code metrics and structural stuckand aSanchez professional simulator or1994). a professional fault at fault coverage is presented in (Corno et.al. 2006, Sanchez correlation between defined code metrics and structural stucket.al.using 2004, Sanchez et.al. 2006, Hudec, 1994). Fault coverage quality is basically evaluated by code coverage calculation at fault coverage is presented in (Corno et.al. 2006, Sanchez and using a professional simulator or a professional fault quality is basically evaluated by code coverage calculation et.al. 2006, Hudec and Gramatova 2015), and it is based on correlation between defined code metrics and structural stuckand using a professional simulator or a professional fault at fault coverage is presented in (Corno et.al. 2006, Sanchez simulator based on stuck-at fault models (stuck_at 0, stuck-at et.al. 2006, Hudec Gramatova 2015), and it is based on at fault coverage is and presented in (Corno et.al. 2006, Sanchez quality is based basically evaluated bymodels code coverage calculation and using aa professional simulator or aa(stuck_at professional fault et.al. 2006, Hudec and Gramatova 2015), and it is based on simulator on stuck-at fault 0, stuck-at and using professional simulator or professional fault at fault coverage is presented in (Corno et.al. 2006, Sanchez simulator based on stuck-at fault models (stuck_at 0, stuck-at et.al. 2006, Hudec and Gramatova 2015), and it is based on et.al. 2006, Hudec and Gramatova 2015), and it is based on and using a professional simulator or a professional fault simulator based on stuck-at fault models (stuck_at 0, stuck-at simulator based on stuck-at fault models (stuck_at 0, stuck-at et.al. 2006, Hudec and(International Gramatova Federation 2015), and it is basedControl) on 125Hosting 2405-8963 © IFAC of Automatic by Elsevier Ltd. All rights reserved. Copyright © 2018, 2018 IFAC simulator based on stuck-at fault models (stuck_at 0, stuck-at Peer review©under of International Federation of Automatic Copyright 2018 responsibility IFAC 125Control. Copyright © 2018 IFAC 125 10.1016/j.ifacol.2018.07.141 Copyright © 2018 2018 IFAC IFAC 125 Copyright © 125 Copyright © 2018 IFAC 125

2018 IFAC PDES 126 Ostrava, Czech Republic, May 23-25, 2018

Jan Hudec / IFAC PapersOnLine 51-6 (2018) 125–130

1). The functional TPGs are mostly evaluated by covering activation of registers during data transfer based on the processor RTL model. It was published that if tests for SBST show high code coverage then also can achieve high coverage of stuck-at 0, stuck-at 1 faults.

All SBST and functional test generation methods using GAs generate test programs by using one type of GAs and one applied evolutionary strategy ES(µ+λ). The first population is generated randomly in all published SBST TPG methods. Obviously the main drawback of these methods is high computing complexity and therefore there is open space for further research.

Automation of the developed SBST methods is essential aspect in testing processors.

3. AN ADAPTIVE SBST TEST GENERATION METHOD

One of the first automatic functional test generation method is based on a system graph representing registers (nodes) and data transfers (arcs) and they are evaluated by mnemonic code of instructions (Thatte and Abraham, 1978). Evaluation of the proposed method was later realised over the processor core DP32 (the 32-bit RISC type processor) and by an in-house structural fault simulator based on stuck-at fault coverage (96 % fault coverage has been achieved by the described method) (Belkin and Sharshunov, 2006). The other functional test generation methods use abstraction models of processors like RTL models and their ISA. The methods are based on analysis of data dependencies of available SBST programs and some parameters of the tested processors. These methods are targeted to pipeline processor architectures. Experiments shown remarkable results up to 95 % in stuck-at fault coverage for two RISC processors MiniMIPS and OpenRISC 1200 (Gizopoulos et.al. 2008). Similar ideas were presented with experiments over the Sun OpenSPARC microprocessor (Apostolakis et.al. 2009).

The key idea of the development of a new functional test generation method for SBST was based on GAs with adaptation of various evolutionary strategies. A new automatic test generation method for SBST has been developed using different types of genetic algorithms (GAs) with specific generated first population and changing evolutionary strategies (ESs) (Schwefel and Back, 1995). Besides of basic evolutionary strategies ES(µ+λ) and ES(µ, λ) it should be useful to use more sophisticated ES defined recently. This evolutionary strategy increases usability of other specified parameters as life-time of population (k) and probability of mutation (p) with the goal to use changing the mentioned strategies. The ES changing is based on feedback quality of generated tests (meaningful sequences of instructions) evaluated by defined fitness function. The major features of the new test generation method are adaptability, flexibility and lower computational time for finding optimal tests with accepted functional fault coverage. The metric used for functional fault coverage in SBST is mostly based on code coverage of a modelled processor. Some rules and strategies have been be defined for developing the effective and flexible automatic TPG system for generation of SBST tests. The new method is based on processor description in language VHDL and known processor ISA.

The next methods for automatic test programs generation use a selected GA and ESs application with feedback information targeted to generation of new populations using simulators (Bernardi et.al. 2006, Corno et.al. 2004, Sanchez et.al. 2006, Squillero, 2005). The idea presented in the papers has been transformed into implementation of a new automatic TPG system, named µGP. The µGP system composes of a test generator at RTL, a fault simulator for stuck-at faults, ISA library and processor division into modules. The evolutionary strategy ES(µ+) was used in the generator where µ is parents population and λ is new individuals for generating a new population, (new test programs/new test mixes). The new individuals are achieved by means of tournament selection for their reproduction (Squillero, 2005). The µGP system was also used for automatic test programs generation for 32-bit processor Leon2 with SPARC V8 architecture (Corno et.al. 2004). The simulator ModelSim was used for simulating the processor design and calculating the statement coverage. The fitness function was defined as the direct measure of the test program´s coverage.

The latest evolutionary strategy ES(μ, k, , p) is mentioned in (Schwefel and Bäck, 1995) and it hasn’t been used in practical GAs applications and not in automatic TPG methods till now. Thus, it was main motivation for new GAs application in functional test generation. The strategy ES(μ, k, , p) uses:  Pseudorandom selection of r individuals (r ≥ λ ≥ µ) from µ parents.  Executing operations by genetic operators.  Making an order of the obtained individuals based on fitness selection of  individuals to the new created population with the best fitness in accordance of selected value of parameter k (k ≥ 1) – lifetime of individuals (test mixes) in particular generations.

Some results for µGP and the DLX/pII processor core are presented in (Sanchez et.al. 2006); the ModelSim simulator was used as the external evaluator with different metrics. The obtained results were shown up to 90 % in statement and branch coverage and above 80 % in condition and toggle coverage. Another case study and results with µGP have been described in (Bernardi et.al. 2006) and (Sanchez et.al. 2007). The µGP effectiveness were tested also over the Intel i8051 processor core with 93,6 % structural fault coverage using on the academic fault simulator FENICE (Squillero, 2005).

The ES changing should allow finding maximum faster than using only one basic ES. Thus adaptability, flexibility and optimal test generation for SBST can be improved. The goals of the design were to develop and implement the new SBST generation method suitable for its automatic implementation based on this new ES and to develop a new method for generation of the first population more deterministically than randomly.

126

2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018

Jan Hudec / IFAC PapersOnLine 51-6 (2018) 125–130

Three strategies have been defined in the above context described below.

3.1 Basic definitions and notations Basic terms used in the proposed generation method for SBST are the following.

3.2 Adaptive SBST method

A test mix is a valid sequence of finite and ordered instructions that can be run in normal execution instruction mechanism (similar to functional programs). Then a test for processor consists of a set of test mixes. Test mix length is number of instructions in a test mix.

The new method for generation SBST tests is based on the next defined strategies and defined fitness function. Strategy 1. Deterministic or pseudo-random generation of an initial started population (test mixes) with higher code coverage can increase quality of the final functional test consisting several test mixes.

Code coverage is defined as coverage of different types of VHDL statements in processor model description. The basic code coverage types are: statement coverage, branch coverage, states or toggle coverage, condition coverage.

Strategy 2. Application of the advanced ES in which all the basic ES´s can be combined using different genetic operators for selection and combinations of crossover and mutation can improve finding optimal final solution for test mixes and thus for functional testing processors.

The advanced evolutionary strategy ES(µ, k, λ, p) is used in GAs application. Parameters µ, λ are parents and new population of generated test mixes, k is the population lifetime and p is the mutation probability.

Strategy 3. The fitness function designed more sophistically in GA contributes to better feedbacks in GA run.

If k = 1/k = ∞ in evolutionary strategy ES(µ, k, λ, p) then ES(µ, λ)/ES(µ+λ) is predetermined.

Other important parameters for generating test mixes for SBST have to be specified and they are:  Number of test mixes or ending their generation based on specified value of the code coverage.  ES alternation, operators, a first population.  Weights and the code coverage specification for fitness calculation.  Test mix length depending of ISA complexity;  Dependability of instructions sequence in a test mix,  Operands selection for instructions.

Diversity means a measure for characterizing existed population of test mixes and indicates how many different types of them are involved in new population. Evenness is defined as a quantitative measure of type representations in existing population which quantifies how equal is the population of testing mixes numerically. The Shannon-Wiener index (Schwefel and Bäck, 1995) and/or standard deviation have been defined for diversity and evenness computation. Both types have to be specified for SBST method in application to a VHDL processor model for receiving an optimal test with sufficient (good) code coverage.

Inputs to test mixes generation is VHDL model of a tested processor, parameters for test mixes quality and for GA application, described in previous section, and databases of instructions and operands for instructions involved in the test mixes. The specific part is initial test mixes generation as a starting set of test mixes (starting population in GA) based on using one of the implemented method (random, priority, grouping, etc.). Users can select one of them for a tested processor.

Fitness function Fj for one test mix Mj, is defined by (1) with condition described by (2) where j = 1, 2, ... M and M is number of test mixes. Fj = ws.sj + wb.bj + wc.cj + wt.tj,

(1)

where ws, wc, wb and wt are weights of code coverage types (statement, branch, condition, toggle) and sj, bj, cj, tj are relevant values of code coverage for the test mix Mj received by simulation. The weights ws, wb, wc, wt are values from interval <0, 1> and they have to satisfy condition ws + wb + wc + wt = 1.

127

(2)

The fitness function was defined as the direct measure of the test program´s coverage. First population (starting individuals) for running GA in SBST generation is a set of test mixes. Besides of using the new ES, it is important to find the best first population. It can be generated randomly as it is used in the published SBST method till now. The idea to create this population as best as possible means that GA application can find the best test mixes in shorter time (Hudec, 2013).

Fig. 1. Comparison of the methods for generating of initial starting population for GA A lot of experiments with the initial generation methods (random, priority, grouping with various modifications, etc.) 127

2018 IFAC PDES 128 Ostrava, Czech Republic, May 23-25, 2018

Jan Hudec / IFAC PapersOnLine 51-6 (2018) 125–130

shown the best efficiency of the method grouping (Fig. 1). Then the final test mixes can be generated in shorter time with higher code coverage. (Note: All used simulation code statement coverage for fitness evaluation were obtained with using each initial method as average values from 10 experiments; the length of test mix was 10, 20 and 30 instructions) (Hudec and Gramatova, 2015).

influence to searching progress of new individuals targeted to the best solution. There is a lot of craftsmanship in definition and assessment of fitness function and the GA parameters. In the presented SBST method there is combination of various constraints and code coverage for estimating the parameters of the fitness function: evaluation statement coverage, branch coverage, condition coverage and toggle coverage. The evolutionary scheme in used GA is very comprehensive and is based on changing advanced strategy ES (μ, k, , p).

The main principle of the method grouping is following: All instructions from ISA of a tested processor are divided into several groups based on their characteristics, e.g. instructions with zero, one or two operands, branch instructions, arithmetic operations, logic operations etc. Then a test mix is generated by adding one instruction from each group based on a specified test mix length. Each instruction has the same probability of its occurrence. There is possible also to use instruction prioritization, it means to involve different probabilities of the used instructions of the tested processor.

4. IMPLEMENTATION AND EXPERIMENTAL RESULTS The SBST generation method was implemented and integrated in a new software environment (Fig. 3), named as AGenMIX (Automatic Generation of test MIXes). All methods are implemented in C# language using Microsoft Visual Studio and Microsoft .NET Framework on the platform Microsoft Windows. All inputs/outputs, except for the evaluating the individuals use the XML format. XML allows usability of standard tools such as browsers, constraints inspection, data library and options for populations and configuration. The VHDL language is used for each tested processor for which also ISA library, database of operands, specification of parameters and ESs, they have to be created. The high-end simulator ModelSim from Mentor Graphics is integrated to the system for simulation. The system AGenMIX is adaptable to various ES application and can be extended by other blocks or functionalities, if necessary. A final test consists of the best test mixes in optimal length with sufficient code coverage.

The main part of the new SBST method is automatic test mixes generation based on the advanced ES, feedbacks from the fitness function evaluation and using different types of GA operators (Fig. 2).

TPG Processor instructions library (ISA)

GAs

Fig. 2. Basic scheme of the adaptive SBST method of test generation with using of GAs.

ModelSim®

Graphic user interface

(GUI)

The major and new developed blocks are:  Generation of the initial population.  Fitness evaluation.  Parameters adaptation. The fitness evaluation is designed according the section 3.1 and affords the opportunity to select sufficient universal formulation for the specific needs of the tested processor. The used GA operators are known and therefore only the consecutive two blocks are explained. Generation of the initial population block is based on selecting the started test mix as best as possible. The specific technique has been developed for creation of a started population based preferably on instruction grouping, but also the random and priority methods are available. Parameters adaptation block consists of changing ES, operators in test mixes generation based on diversity and evenness measures during application of one GA. It has been proven, that ES(μ+) is very positive in selection of optimal solution and ES(μ,) is better in overcoming of local extremes. Loosing parents population can give positive

Control unit module

Adaptive module

User parameters setting

Processor VHDL description RAM test template

Fig. 3. Implementation block scheme of TPG in AGenMIX. Inputs to test generation are VHDL model of a tested processor, parameters for test mixes and for GA application described in previous section, and databases of instructions and operands for instructions involved in test mixes. Then GA with genetic operators (selection / reproduction, mutation, crossover) is performed on the first population of test mixes, and with reference to the fitness value of each mix calculated by ModelSim. The parents for the next population of test mixes are selected by changing parameters and stated ES. Some constraints such as elitism of individuals with changing parameter k in ES (μ, k, , p), tournament and roulette selection are also applicable. If the feedback values in 128

2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018

Jan Hudec / IFAC PapersOnLine 51-6 (2018) 125–130

the GA progress indicate the small value of diversity or high value of evenness then alternation of parameters in GA have to be adopted. Thus continual adaptation of input parameters k (life-time of test mix/elitism) and p (probability of mutation) and the selection methods (roulette, tournament , stochastic universal sampling, etc.) are applied until the higher fitness values of generated test mixes are achieved. This procedure is repeating until the best code coverage of test mixes or the maximal number of populations (generations) is achieved.

achieved code coverage are sufficient for functional test for processor in comparison with published results till now. 5. CONCLUSIONS Functional test generation is a long-standing open challenge, which is an important problem to be solved for design verification, manufacturing testing and periodic testing of processors during their lifetime as well. One key to develop a practical functional test generation approach is to avoid the exponential growth of the test generation complexity in terms of the growth with respect to the design size. The SBST approaches and achieved results over them till now have showed that such functional tests for various processors reached significant level of maturity. They are used not only in academic designs but also in practical SoC and embedded system applications.

Functionality and effectiveness of the new implemented SBST method have been tested on the RISC type processors DP32 and DLX and their ISA. Described experiments use statement coverage for fitness evaluation. DP32 is a RISC type 32-bit processor core (Ashenden, 1990). Its ISA contains 20 types of instructions with the length of 32 or 64 bits that are frequently used in programs. It is a typical representative of the ARM (Advanced RISC Machine) processor architecture. The processor core is described as a synthesizable VHDL model for academic and research purposes. Specification and architecture of DP32 is open, portable, non-proprietary and scalable to embedded processors, all sharing the same core (non-privileged) instruction set. It can be implemented in programmable logic such as FPGA or as soft IP cores.

The main contribution of this continuing research is development of the new complex and adaptive functional test generation system based on GAs and the advanced evolutionary strategy defined as ES(μ, k, , p). The developed methods, algorithms have been implemented in the software tool AGenMIX for automatic software-based test generation for RISC type processors modelled by VHDL. The new system is useful for verification and testing processors. The test mixes (test programmes) for processors can be generated in shorter time than using similar methods with GA. And one specified basic ES.

The DLX is a RISC processor, essentially simplified MIPS with simple 32-bit load/store architecture (ISA contains 66 types of instructions), widely used in university-level computer architecture courses to get knowledge about the RISC processor in terms of instruction set and operation, instruction encoding and decoding, pipelined operations and functions of each processor component (Patterson and Hennessy, 2014). VHDL model of DLX processor consists of various functional units which were created as components and follows classic Harvard architecture of separate instruction and data memories to allow simultaneous instruction fetching and data memory transactions.

The implemented system AGenMIX is flexible to specified parameters, scalable to processor models and their ISA and allows the users generate test mixes with interactions. Theoretical contributions of this research are in application of the advanced ES with their parameters (diversity and evenness) in software-based test generation for processors and variety of first population generation with the goal to receive the final test mixes in shorter time with sufficient high code coverage.

Table 1. SBST test generation, parameters and results Parameters Number of instructions in test mix Number of individuals in population Number of generation Starting population based on Selection operator (initial) Method GA with Parameter p Parameter k Code coverage metric Maximum code coverage (fitness)

DP32

DLX

30 16 256 grouping tournament feedback 0.1 3 statement

30 16 256 grouping tournament feedback 0.1 3 statement

95,89 %

94,72 %

129

In practice, the developed methods, algorithms and the system AGenMIX contribute to using different operators applied in GAs and varying evolutionary strategies. The new automatic generation system is adaptive, flexible and interactive for SBST test generation. The generated test mixes (test programs) are useful mainly for verification of processors in their design process and testing processors during their lifetime. A test (a set of test mixes) for testing processors based on ISA can be generated automatically by the system AGenMIX in shorter time than previous published SBST methods with one ES and a random starting first population. Experimental results have been confirmed the effectiveness of various evolutionary strategies combination and more genetic operators utilized for receiving an optimal set of test mixes on qualitative higher level for a tested processor. Based on obtained results, the proposed method can be suggested as a very promising method for the automatic design and generation functional tests of processors and embedded cores in SoC.

Various experiments have shown influence of changing evolutionary strategies and adaptation of GA application in text mixes generation. After complex experiments we achieved maximum for DP32 processor 95,89 % and for DLX processor 94,72 % in the statement coverage (Table 1). The

129

2018 IFAC PDES 130 Ostrava, Czech Republic, May 23-25, 2018

Jan Hudec / IFAC PapersOnLine 51-6 (2018) 125–130

Usability, functionality and effectiveness of the new implemented SBST method have been verified on the DP32 RISC type processor (Hudec, 2011, Hudec, 2013, Hudec and Gramatova, 2015) and DLX processor.

Gizopoulos, D., Psarakis, M., Hatzimihail, M., Maniatakos, M., Paschalis, A., Raghunathan, A. and Ravi, S. (2008). Systematic software-based self-test for pipelined processors. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 16, No.11, 2008, pp. 1441-1453. Hudec, J. (1994). VLSI system test design: The methods, problems and experience in microprocessor testing using AFTG. Proc. of 16th Int. Conference Information Technology Interfaces ITI´94, Pula, June 14-17, 1994. Eds. V. Ceric - V.H. Dobric. Zagreb, University Comp. Centre. pp. 191-193. Hudec, J. (2011). Processor functional test generation - some results with using of genetic algorithms. Proc. 2nd Eastern European Regional Conference on the Engineering of Computer Based Systems ECBS-EERC 2011, Bratislava, Slovakia, Sept. 5-6, 2011, ed.by V. Vranic IEEE Comp. Press, Los Alamitos: IEEE Comp. Society., pp. 159-160. Hudec, J. (2013). Some results in automatic functional test design for processors. Sobh, T, Elleithy, K. (Eds.), Emerging Trends in Computing, Informatics, Systems Sciences, and Engineering, Lecture Notes in Electrical Engineering, Vol. 151, New York: Springer Science + Business Media B.V., pp. 965-972. Hudec, J. and Gramatová, E. (2015). An efficient functional test generation method for processors using genetic algorithms. Journal of Electrical Engineering, vol. 66, no. 4, pp. 185-193. Patterson, D. A. and Hennessy, J. L. (2014). Computer organization and design. Morgan Kaufmann Publishers, N. York, in 5th edition. Psarakis, M., Gizopoulos, D., Sanchez, E. and Reorda, M. S. (2010). Microprocessor software-based self-testing. IEEE Design & Test of Computers, Vol.27, No. 3, May/June 2010, pp. 4-19. Sanchez, E., Reorda, M. S. and Squillero, G. (2006). Efficient techniques for automatic verification-oriented test set optimization. International Journal of Parallel Programming, Vol. 34, No. 1, Springer Science + Business Media, pp. 93-109. Sanchez, E., Schillaci, M., Squillero, G. and Reorda, M. S. (2007). An enhanced technique for the automatic generation of effective diagnosis-oriented test programs for processor. Proc. of Design, Automation & Test in Europe Conference & Exhibition 2007, DATE 07, Nice Acropolis, France, April 16-20, 2007, pp. 1-6. Schölzel, M., Koal, T., Röder, S. and Vierhaus, H. T. (2013). Towards an automatic generation of diagnosis in field SBST for processor components. Proc. 14th IEEE LATW 2013, Cordoba, Argentina, April 3-5,2013, pp.1-6. Schwefel, H. P. and Bäck, T. (1995). Evolution strategies. J. Périaux, G. Winter, (Eds), Genetic Algorithms in Engineering and Computer Science. John Wiley & Sons Ltd, Chichester.

The novelty of the proposed method is in progressive process used for generating first population of GA, in advanced evolutionary strategy ES(µ, k, λ, p) used in GA application and in adaptability, flexibility and lower computational time. Future works are targeted to experiments with other type of RISC processors and experiments with using more code coverage parameters and also how to specify the relevant weights for fitness function. ACKNOWLEDGEMENT This work has been supported by the Research and Development Operational Programme for the project ITMS 26240220084 University Science Park STU Bratislava, cofunded by the European Regional Development Fund, Slovak National Research Grant Agency under VEGA 1/1008/12 project Optimization of low-power design of digital and mixed integrated systems and VEGA 1/0836/16 project Methods and algorithms for improving efficiency and multimedia content delivery in IP networks. REFERENCES Apostolakis, A., Psarakis, M., Gizopoulos, D., Paschalis, A. and Parulkar, I. (2009). Exploiting thread-level parallelism in functional self-testing of CMT processors. Proc. of 14th IEEE European Test Symposium ETS 2009, Seville, Spain, May 25-29, 2009, pp. 33-38. Ashenden, P. J. (1990). The VHDL cookbook. Department Computer Science University of Adelaide, Australia, in first edition, 111 p. Belkin, V. V. and Sharshunov, S. G. (2006). ISA based functional test generation with application to self-test of RISC processors. Proceedings of the 9th IEEE Workshop on Design & Diagnostics of Electronic Circuits & Systems (DDECS 2006), Prague, Czech Republic, April 18-21, 2006, pp. 73-74. Bernardi, P., Grosso, M., Sanchez, E. and Reorda, M. S. (2011). The software-based self-test of embedded microprocessors. Design and Test Technology for Dependable Systems-on-Chip, chapter 15, R.Ubar, J.Raik, H.T. Vierhaus,(Eds), IGI Global, New York, pp. 339-359. Bernardi, P., Sanchez, E., Schillaci, M., Squillero G. and Reorda, M. S. (2006). An evolutionary methodology to enhance processor software-based diagnosis. Proc. 2006 IEEE Congr. on Evolutionary Computation, Vancouver BC, July 16-21, pp. 859-864. Corno, F., Cumani, G., Reorda, M. S. and Squillero, G. (2003). Automatic test program generation for pipelined processors. Proc. ACM Symposium Applied Computing SAC 03, March 9-12, 2003, Melbourne, Florida, California (USA), ACM Press, pp.736-740. Corno, F., Sanchez, E., Reorda, M. S. and Squillero, G. (2004). Automatic test program generation: A case study. IEEE Design & Test of Computers., Vol.21, No. 2, pp. 102-109. 130