An Exchange Protocol for Continuous Speech Recognition and Synthesis System

An Exchange Protocol for Continuous Speech Recognition and Synthesis System

Copyriglll II-":\C Indian ;l . l 'S:\ I ~ll'I.:.! C()lll)IU(,1 \"led 1)( ·'I ,\.!1i AN EXCHANGE PROTOCOL FOR CONTINUOUS SPEECH RECOGNITION AND SYN...

1MB Sizes 3 Downloads 147 Views

Copyriglll II-":\C Indian ;l . l 'S:\ I ~ll'I.:.!

C()lll)IU(,1

\"led

1)( ·'I ,\.!1i

AN EXCHANGE PROTOCOL FOR CONTINUOUS SPEECH RECOGNITION AND SYNTHESIS SYSTEM G.Osman Laboratoire d'Automatique de Grenoble, E.N.S.l.E.G., B.? 46, 38402 Saint-Martin-d'Heres, France

Abstract. ARIAL 11 is a project currently under development at Toulouse University to study the connected speech understanding problem. The Speech Recognition and Synthesis System (SRSS) developed in the context of ARIAL~II has been based on a desire to exploit multiprocessor with multi-task architecture for efficient implementation, and a desire to handle more complex speech task domains with an optimal processing time (Perennou, 1980), (Osman, 1979,1981), The SRSS is organized on three modular sub-systems: 1) Recognition System (RS); 2) Acquisition and Synthesis System (ASS) ; 3) Exchanges and Control Module (ECM). We describe in this paper the ECM functionning and the exchanges protocol between modules and between sub-systems. The functional structure of the SRSS is based on the parallelism of procedures between the different modules on a multi-task hierarchical architecture (Anceau, 1974), (C.G. Bell and all, 1971,1973), (D.R. Reddy and all, 1970). The RS mainfram computer is coupled with the ASS by means of the ECM communication network. control and processing tasks are distributed between the mainfram computer and th e multi-microprocessor (ASS-ECM) system according its nature and complexity. The data transfer between ASS and RS is performed with a 1 M bytes Is speed, by means of a direct memory access controler (DMAC). Keywords. Computer interfaces; Communications control application; Hierarchical systems ; Microprocessors ; Signal processing ; Speech recognition. OVERVIEW OF SRSS

break sentenses up into gramma ti cal phases, and guide procedures for both phone classification and higher level linguistic analysis.

SRSS is a system for acquisition, acousticphonetic analysis, recognition and synthesis of spoken French sentenses.

Three levels of the acoustic analysis are considered (Cael en, 1979) :

It is a modular organization which allows for graceful modification of existing modules and algorithms or addition of new ones. The following is a brief description of the SRSS subsystems.

1) The acoustical, phonetical analysis l eve l

consists of signal processing and spectral parameters extraction. Segmentation and phonetic labels are derived. 2) The lexical network decoding decodes phonetic strings into lexical strings.

RS Implementation Overview.

3) The syntactic semantic analysis consists of giving, for each string of words, the predictive constraints for the next word.

RS is a system for acoustic-phonetic analysis and recognition of continuous speech. It is being developed to serve as part of an automatic speech understanding system.

A modular recognition algorithm has been implemented in the biprocessor mainfram computer "IRIS-80".

The design of RS is centered around a global data base and a prosodicaly guided search (Baker, 1974), (Erman and al, 1973). The global data base (blackbord) is structured as a multilevel, uniforme, interconnected data structure. The strategy for recognition and several details of the acoustic and linguistic analysis procedures are distinctive in that prosodic features (formants, energy functions and certain timing information) are used to

*

ASS Implementation Overview. ASS is a modular system built around two Motorola M 6800 microprocessors. The structure of this system is based on the parallelism of procedures in a hierarchical multi-task architecture. (AND RE ,1978). I t is organi7ed of two independant modules as follows :

ARIAL : Analyse et Reconnaissance d'Information Acoustique et Linguistique.

Acquisition modules for acoustic signal. The acquisition, the sampling, the AID conversion 285

G. Osman

286

and the storage in common memory of the vocal signal are the principal functions of this module. These above functions and the communications with the user and the transfer module are controled by a monitor program written in M 6800 Assembler. The acquisition task starts with the exchange, in an interrupt aknowledge mode, of status bytes with the transfer module . The acquisition microprocessor is initialised by the transfer module and it also executes its commands. It communicates its status to the user when it is ready to receive the first sample of the vocal signal. The sampling frequency is selected from 10, 12,5 or 15 KHz. The storage in common memory blocks and the transfer of data from these memory blocks can be made simultaneously .

To overcome this constraint, the ECM functioning has been based on three independent phases : Initialization, Transfer and Termination phases. The control of the ECM working is performed by a moni tor program written in M 6800 assembler language. The program assumes the dialogue with RS, acquisition module, synthesis module and the user. The dialogue consists in establishing the exchange protocol, to initialize the programmable elements, to start the transfer procedure and to settle the final state of the exchange . Initialization.

Synthesis module for acoustic signal . The principal roles of this module are the restitution of the vocal signal after the treatment, the D/A conversion and the vocal output throught a loud- speaker. These functions and the communications tasks are conducted by a monitor program written in M 6800 Assembler . The control of the vocal signal output is performed in reference with the status byte sended by the transfer module. The transfer of data to or from common memory blocks is performed by an interconnection network.

One of the two partners (initiator), RS or ECM, requices a data transfer in setting up its INITIATE signal. The initiator sends simultaneously its request and an "order" byte in which the transfer sens is specified. The rejoinder (ECM or RS which receive INITIATE) can accept the transfer request if it is ready and then it returns its ACKNOWLEDGE signal that launches the transfer phase. It can refuse the transfer request if it is not ready and it returns an interruption signal (TERMI NATE). In this case, the exchange passes straightly on the termination phase. The case of a simultaneous transfer requests is determined by attributing respectively a hardware priority (high and low priority to each partner). The initialization phase is conducted by the ECM microprocessor in an interruptible mode through a PIA interface.

ECM Implementation Overview.

Transfer.

The major focus of the design of ECM was the development of an exchange protoco l for effi cient multi-task implementation (Osman, 1981), (Swan, 1977). The ECM func t iona 1 structure is based on the monitoring of the main tasks. The ECM M 6800 microprocessor dispatches the different tasks to be performed, activates the initialization of DMA and PIA interfaces, as sumes the synchronization of the exchange process and starts the execution of the transfer and dialogue software. The ECM monitor program controls the following tasks :

Transfer phase is started by the acknowledge ment of the rejoinder. The data transfer is considered by the first or second channel of the DMA controller according to the transfer sens. The programmation of the DMA controller is made during the initialization phase. Acting of the obstruction of the RS IUC-7650 channel, the transfer speed can reach until I M bytes/second if we use 2 MH z clock frequency. The utilisation of the DMA controller in TSC-STEAL mode, enables a good compromise between the availability of the ECM microprocessor and the transfer speed (500 K bytes/s) . During the transfer, the DMA controller steals three cycles about four. This mode allows the ECM microprocessor to control simultaneously a data transfer and other tasks in the system.

An interconnection network insures the access control and the rotation of memory blocks, (DOURS, 1979) .

- Management of acquisition and synthesis modules . - Management of the memory blocks in input and output. - Control of data transfer between RS and the acquisition and synthesis memory blocks. - Communication with the user. A "TSC-STEAL" modes alows to the ECM microprocessor the possibility to execute simultaneously data transfer and control tasks. ECM FUNCTIONING A primary goal in the design of ECM is to exploit the potential parallelism of the SRSS organization as fully as possible. In order to adapt its functioning to the global system design, an exchange protocol must be provided to support the logic of RS and ASS interfaces.

Termination . The termination phase corresponds to the sending of an interruption signal (TERMINATE) with a final byte in which is brought back the status of each partner (command error, improper length, abnormal end). The "state" byte points out an abnormal sequence if the rejoinder cannot assume the data transfer (data not available, no place in memory . ... ). It points out an error of order if the transfer sense is the same for the two partners. In the case where the rejoinder has refused the data transfer, the initiator returns his TERMINATE signal instead of ACKNOWLEDGE signal.

An Exchange Protocol for Continuous Speech Recognition Transfer modes. Two transfer modes could be envisaged in the ECM functioning : - HALT-BURST mode Ln which the DMA controller desalables the ECM microprocessor (Halt position), controls the address and data bus access and drives a data tran~fer by block of bytes. The transfer speed is 1 M bytes/so - TSC-STEAL mode in which the DMA controller will set bus and drivers in three state control position, and starts a data transfer by stealling one clock cycle about four. The second clock cycle is used for synchronization (DUMMY). The third cycle is used to system control and the fourth cycle is DUMMY. The transfer speed is 500 K bytes/so ECM COMMUNICATION CONTROL Two half-duplex connections with RS and ASS are controled by ECM. The RS computer commu~i­ cates with ECM throught its controller "IUC 7650". Data bus, address bus and control lines permit the connection to ASS through its input/output board. The following sections describe the ECM exchanges with his environment. Exchange with RS. This exchange is performed by the "IUC 7650" controler according to the following characteristics : - Parallel and bidirectionnal interface. - Connection with RS memory is established by a 4-bytes data bus. - Connection with the peripheric is established by "order" bytes in the two unidirectionnal channels with half-duplex. ECM has to respect the logic of control and data signals of the "IUC 7650" controller. Consequently, some limitations design are imposed. It is restrained to generate signals that will be similar to "IUC 7650" ones. adapt its input components to the logic of these signals. Interface lines. These are control, data and address signals. The sending of these signals in a clearly defined order allows to set out the sequences of the normal functioning of the exchange. The sending of these signals in a different order conducts to an abnormal end of transfer or to an "error of order" that is signalized to the user. The signification of these signals and its roles are shown in the following : 8 b~ data b~ : It is connected to the RS 32-bits bus throughoidirectionnal buffers.The buffers access is stired up in one direction or in another according to the nature of the t rans fe r. 76-b~ ad~~~ b~ : It is connected to RS address bus through bidirectionnal buffers that are exclusively controlled by the RS mi-

287

croprocessor.

STROBE : Signal of validation of data and of status bytes. This line can be used as a synchronization signal between the interconnection components and the system.

PROCEED: General validation signal. It can be used by the receiver to synchronize the data transfer. INITIATE: Signal of the transfer initialization request. It is accompagnied with an "Order byte" th.:!tshows the nature of the request transfer. ACKNOWLEDGE : It is an answer in the affirmative to the request of an initialization by INITIATE for a transfer.

TERMINATE: Signal of the transfer end. It is accompagnied with a "terminate byte" vThich caracterises the transfer. Exchange with ASS. The exchange with ASS is made under the control of the ECM microprocessor through a specific input/output connection. The address bus and data bus are controlled according to the following procedure : 1) When the DMA controller initializes a data transfer from the acquisition module memory blocks to RS memory (in reading position) or from the RS memory to the synthesis module memory blocks (in writing position), address bus and data bus are controlled by the DMA controller. 2) When the ECM microprocessor initializes his interface organs, it controls the two address and data bus. The address bus is used like this : - the first five lines (AO-A4) are used to select the internal registers of DMA controller. - The 11 last lines (A5-A15) are used to select the ECM organs. The access to the acquisition memo ry blocks and the synthesis memory blocks is performed by an interconnection network according to the CROSS-BAR technique. (Access simultaneously to two different blocks) . Interfaces lines. There are two classes of signals which perform the ECM control. The first class consists of signal artificially generated to answer to the control and handling of the system as the "interface signals" with RS that were described before. The second class consists of signals given by the system as for example the control signals of its different organs (PIA, DMAC, Microprocessor .... ). All these signals are useful to define the protocol of exchange between RS and ECM. They carry the orders for initialization of the ECM different components (DMAC, PIA, Flip-Flop, ... ), the request and modes of transfer sent by the DMAC, the consideration of these requests and so on .... They also allow to stop the microprocessor clock and then to fill up this one in order to give the control at DMAC and to

288

G. Osman

make a transfer in "Halt-Burst" mode. Control of Exchanges. The ECM microprocessor supervises the intermodule communication and initiates the different stages of exchange through its peripherics. The commands of "state" and "order" are started from a PIA (Peripheral Interface Adapter). The orders of transfer are achieved by the DMAC (Direct Memory Access controller). In the following, we present the DMAC and PIA main functions in the control . I) Contro l of transfer: The DMAC is acting as manager for transfer. When it achieves a transfer, it fills up the ECM microprocessor and contro ls the data bus and address bus access. DMAC makes then a direct connec tion between the ASS memory blocks and the RS memory space. 2) Management of control signals : the PIA is programmed by the ECM microprocessor to generate or to receive the "order " signals in the transfer initialization and the "state" signals in the transfer termination. The management of these signals is formed as it is following up : Four lines of data in input to receive the control signals (except STROBE). - Four lines of data in input to receive the significant bits of the "state" byte or the " order" byte. - Four lines of data in output to generate the control signals (except STROBE). - Four lines of data in output to generate the significant bits of "state" byte or of "order" byte. Two lines of control in interruption input for the re ception of INITIATE and TERMINATE signals. - One line of control in pulse output during the "state" byte or " order" byte emiSSion. CONCLUSION This paper has presented an organization for a multi-task system. An intermodule communication is involved by a multi-l evel exchange protocol performed by ECM. ECM allows SRSS to have very interesting functioning possibilities specially at two levels : the high speed data transfer and the multi-t ask treatment : - Data transfer speed riches I M bytes/s (clock frequency = 2 M Hz ) when the DMA controller is programmed in Halt-Burst mode. - ECM can execute simultaneously a control task and a data transfer. That is possible when the DMA control ler is programmed in TSC-STEAL mode. With this transfer mode, we arrive to a compromise between the transfer speed (500 K bytes Is) and the microprocessor availability .

REFERENCES

Anceau, F. (1974). Contribution a l'etude des systemes hierarchises de res sources dans 1 'architecture des machines informatiques" These Doctorat. Es-Sciences, Grenoble. Perennou, G. (1980). ARIAL 11 : Stst€m for speech recognition. I.E.E.E. 5 h International conference on Pattern Recognition. Caelen, J. (1979). Un modele d'oreille. Analyse de la parole continue . Reconnaissance phonetique. State thesis, Toulouse University. Victor, R. et al . (1975). Organization of the Hearsay 11 speech understanding system. I.E.E.E. Trans. on ASSP, vol. 23, nOI. Be 11, C;. G., R. C. Chen eta 1. (1973). The a rchitecture and application of computer modules : a set of components for digital systems design. COMPCON 73 , San Francisco, Calif. pp. 177- 180. Reddy, D.R., L.D. Erman and R.B. Neely.(1970) . The C-MU speech recognition project. I.E.E. E. Conf. System Sciences and Cybernetics, Pi tts burgh , Pa. Baker, J. (1974). The GRAGON system - an overview. Proc. I.E.E.E. Symp. Speech Recognition, Carnegie-Mellon Univ . Pittsburgh, Pa. pp. 22-26. also this issue, pp. 24-29. Wesley W. et al. (1980). Task allocation in distributed data processing. I.E.E.E. Trans. on Computer. Nov . 1980. Dours, D., R. Facca. (1979). Systeme multimicroprocesseur pour l'acquisition et la synthese du signal vocal. Congres A.F.C . E . T Toulouse. Erman, L.D., R.D. Feuwell and V.R. Lesser. (1973). System organization for speech understanding : implications of network and multiprocessor computer architectures for AI. Proc . 3 rd Int. Joint. Conf. Artificial Intelligence, Stanford, Calif. Osman, G. (1981). Transfer and excha nge control between a computer and a multi-microprocessor system. ISMM Inter. Symp. on "Mini and Microcomputer in Control and Measurement", San Francisco . Osman, G. (1979). Etude et realisation d'un coupleur de liaison entre IRIS 80 et un systeme multimicroprocesseur. Rapport D.E.A. d'Informatique, Universite de Toulouse.

=:=:=:=:=::::::=:=