- Email: [email protected]
Computational intelligence and bioinspired systems
ARTICLE IN PRESS
Neurocomputing 70 (2007) 2701–2703 www.elsevier.com/locate/neucom
Editorial
Computational intelligence and bioinspired systems In this special issue of Neurocomputing we present 20 extended versions of selected papers from the 8th edition of the International Work-Conference on Artificial Neural Networks (IWANN 2005). This is a biennial meeting which focuses on the foundations, theory, models and applications of systems, which are inspired by nature (e.g. neural networks, fuzzy logic and evolutionary systems). The IWANN’2005 edition was organized in Spain by the Universitat Polite`cnica de Catalunya, UPC, with the strong cooperation of the Universidad de Granada and Universidad de Ma´laga. Sponsorship was obtained from the organizing university UPC, the Spanish Ministerio de Educacio´n y Ciencia, and the City Council of Vilanova i la Geltru´, where the conference took place from 8th to 10th June 2005. Since the first meeting of IWANN, held in Granada (1991), the artificial neural network (ANN) community, and the domain itself, have matured and evolved. Under the ANN banner we find a number of heterogeneous scenarios but all having the main interest and objective of providing a better understanding of nature and living beings for the correct development of theories, models and new algorithms. This is a very good way for scientists, engineers and professionals working in the area to develop solid and competitive applications. We are facing a real revolution with the emergence of embedded intelligence being applied into many artificial systems (that is, systems covering diverse fields: industry, home automation, healthcarey). So, we are convinced that an enormous amount of work must still be, and could be, done. Many pieces of the puzzle must be built and placed in their proper positions, offering us new and solid theories and models (necessary tools) for the application and praxis of these today’s paradigms. In IWANN 2005, after a careful review process of more than 240 submissions, 150 papers were accepted for publication including the contributions of the three invited speakers. This special issue reveals how IWANN deals with a wide variety of topics in neural computing and neuroscience connecting theoretical models with their applications in real life; this is the main reason for the sub-title of this special issue: Computational Intelligence and Bioinspired Systems. A number of authors were invited to submit an extended version of their conference paper to be considered for 0925-2312/$ - see front matter r 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2006.06.009
publication in this special issue of Neurocomputing. These authors were selected based on the recommendation of the reviews of the conference papers and the chairs of the different sessions. The extended versions were meticulously reviewed again by at least two independent and anonymous experts and the papers accepted, after this new review process, are presented in this volume. The papers can be grouped into the following four categories: (1) Neurocomputation formulations, considering the neurocomputation concept as a step toward the understanding of how the brain works in order to obtain valid models to build information processing systems; (2) Conventional models, including both theoretical and practical aspects of simple or hybrid models, (3) Neuroengineering and hardware implementations, and (4) Applications.
1. Neurocomputational formulations In the paper by Angelo di Garbo et al., the functional role of electrical synapses in a network of inhibitory interneurons is investigated by using a single compartment biophysical model of a Fast Spiking cell. In particular, the parameter values, which lead to the emergence of synchronous regimes in a network of Fast Spiking interneurons coupled by chemical and electrical synapses in the weak coupling limit, were determined theoretically. The paper by E.E. Claverol-Tinture´ et al. describes the progress in the polymer-on-multielectrode (PoM) technology, specifically towards the development of vertebrate neuronal cultures devoid of glial layers compatible with PoM devices, the connection of pairs of invertebrate neurons threading microchannels, and the recordings of synaptic and spike-like activity. The model authored by Hugges Berry and Oiliver Temam is based on a structure of biological neural networks that allows generating surrogate networks with realistic biological structure, as would be needed for complex information processing/computing tasks. This structure reproduces most of the graph properties exhibited by Caenorhabditis elegans, including its small-world structure. The development of this model contributes to the realization of chip–neuron interfaces using real biological neurons as long-term alternatives to silicon transistors.
ARTICLE IN PRESS 2702
Editorial / Neurocomputing 70 (2007) 2701–2703
2. Single and hybrid conventional models Enrique Romero and Rene´ Alque´zar obtain heuristically a suboptimal solution for the selection of weights of the new hidden units in sequential feed-forward neural networks. This process, that usually involves a non-linear optimization problem, cannot be solved analytically in the general case. The obtained results indicate that the orthogonalization of the output vectors of the hidden units outperforms the usual strategy of matching the residue, both for approximation and generalization purposes. The paper presented by R. Salas and co-workers introduces robustness and flexibility to the Hierarchical Self Organizing Maps model (RoFlex-HSOM) that has the plasticity to find the structure that best fit the data, gradually forgetting (but no catastrophically) previous learned patterns. The proposed algorithm is robust to the presence of outliers and preserves the topology to represent the hierarchical relation of the data in non-stationary environments. In the work developed by Ralf Eickhoff and Ulrich Ru¨ckert, the robustness of radial basis function networks in presence of malfunctioning elements and noise in its inputs and parameters is analyzed in order to operate in noisy and unreliable environments, as it happens in nanoelectronic devices and circuits. The work by Andre´s Berzal and Pedro Zufı´ ria addresses a comparative study with a deterministic discrete-time (DDT) formulation of the Sanger Hebbian ANN model that characterizes the average evolution of the net, preserving the discrete-time form of the original network and gathering a more realistic behavior of the learning gain. The results thoroughly characterize the relationship between the learning gain and the eigenvalue structure of the correlation matrix. The paper by Jero´nimo Arenas-Garcı´ a and co-workers addresses a general technique to reduce the computational burden associated to the operational phase of most neural networks, such as multi-net or radial basis function networks, which calculate their output as a weighted sum of terms. Sung-Kwun et al. introduce a new topology of fuzzyneural networks—fuzzy polynomial neural networks (FPNN)—that is based on a genetically optimized multilayer perceptron with fuzzy set-based polynomial neurons (FSPNs), and develop a comprehensive design methodology involving mechanisms of genetic optimization and information granulation. The performance of the networks is quantified through experimentation where they use two modeling benchmarks commonly employed within the area of fuzzy or neurofuzzy modeling. Ana Porto and co-workers develop a new hybrid learning method based on recent studies that have confirmed that the modulation of synaptic efficacy affects emergent behavior of brain cells assemblies. In order to find the best solution to a given problem, the new method combines the use of genetic algorithms with particular changes to connection weights based on this behavior.
The work by Miguel Rocha et al. presents two hybrid evolutionary computation/ANN algorithms: the first evolves neural topologies while the latter performs simultaneous optimization of architectures and weights. Sixteen real-world tasks were used to test these strategies. Competitive results were achieved when these algorithms were compared with a heuristic model selection and other data mining algorithms. 3. Neuroingeneering and hardware implementations The paper by Christian Morillas and co-workers shows the development of a set of software and hardware tools to interface with neural tissue, in order to transmit visual information encoded into a bioinspired neural-like form. The set is composed of a retina-like encoder, implemented both in software and on FPGA chips, and a platform to optimize the electrical stimulation parameters for a multielectrode implant. The main objective is to progress towards a functional visual prosthesis for the blind. Miguel Atencia et al. present a FPGA implementation of a hardware module that performs parametric identification of dynamical systems based upon the methodology of optimization with Hopfield neural networks, leading to an adapted version of these network models. This implementation achieves modularity and flexibility due to the usage of parametric VHDL to describe the network. The natural parallelism of neural networks is preserved at a limited cost in terms of circuitry cost and processing time. 4. Applications Contrary to the conventional point of view based on the modeling of non-linear systems, the paper by K. Madani and L. Thiaw proposes to deem the multi-modeling identification concept as building a modular architecture, inspired from ANN operation mode, where each neuron (module), represented by one of the local models, realizes some higher level transfer function in the non-linear system’s behavior identification and prediction context. Alberto Guille´n et al. present a new algorithm that applies fuzzy logic to a previous clustering technique for the initialization of the centres of the radial basis functions used in a neural network that approximates a function. The results achieved by the new algorithm improve significantly the results obtained by its ancestor and by other algorithms in the literature developed for similar purposes. A global methodology, which combines direct prediction strategy and sophisticated input selection criteria for the long-term prediction of time series, is proposed by Antti Sorjamaa et al. This methodology is successfully applied to a real life benchmark: the Poland Electricity Load dataset. Luis Javier Herrera et al. paper introduces a general technique that upgrades the performance of one-stepahead prediction models, when they are used recursively for long-term time series prediction; in order to show the operation of this new general technique, the work has used
ARTICLE IN PRESS Editorial / Neurocomputing 70 (2007) 2701–2703
two different methodologies as one-step ahead models to perform long-term recursive prediction: the TaSe Fuzzy TSK model and the least-squares SVMs. Juan Jose´ Murillo et al. propose a new method to solve the problem of blind robust watermarking of digital images. The method is based on the application of the independent component analysis (ICA) to compute some statistically independent transform coefficients where the watermark is embedded. With this approach, the ICA transformation is not unique, so each user can define its own as his private-key, and some of these transform coefficients have white noise-like spectral properties. In the paper by Cemil Oz and Ming C. Leu, two different American Sign Language (ASL) word recognition systems are proposed using ANN to translate the ASL words into English. These methods were compared with each other showing accuracy of recognition within 92% and 95%, respectively. Note that the sign language, which is a highly visual-spatial, linguistically complete, and natural language, are the main modes of communication among deaf people. Jesu´s Fraile-Ardanuy and co-workers develop a novel adaptive power system stabilizer (PSS). It is based on the online tuning of conventional PSS parameters for a singlemachine infinite bus system, using adaptive network-based fuzzy inference systems. The training data for these fuzzy neural networks are automatically generated based on the
2703
optimization of a fitness function using genetic algorithms. The proposed stabilizer has been tested by performing simulations of an overall non-linear system. The guest editors would like to express their gratitude to all authors for their submissions and the anonymous reviewers for their comments and useful suggestions in order to improve the papers. We would also like to thank Dr. Tom Heskes and the Neurocomputing editorial board for giving us the opportunity to publish this special issue, and Elsevier as the publisher for the very efficient and seamless management of the publication process. It is a pleasure for us to invite all authors and interested readers of this issue to future IWANN conferences, which are announced at http://www.iwann-conference.org.
Alberto Prieto Universidad de Granada, Spain E-mail address: [email protected] Joan Cabestany Universitat Polite`cnica de Catalunya, Barcelona, Spain Francisco Sandoval Universidad de Ma´laga, Spain
Neurocomputing 70 (2007) 2701–2703 www.elsevier.com/locate/neucom
Editorial
Computational intelligence and bioinspired systems In this special issue of Neurocomputing we present 20 extended versions of selected papers from the 8th edition of the International Work-Conference on Artificial Neural Networks (IWANN 2005). This is a biennial meeting which focuses on the foundations, theory, models and applications of systems, which are inspired by nature (e.g. neural networks, fuzzy logic and evolutionary systems). The IWANN’2005 edition was organized in Spain by the Universitat Polite`cnica de Catalunya, UPC, with the strong cooperation of the Universidad de Granada and Universidad de Ma´laga. Sponsorship was obtained from the organizing university UPC, the Spanish Ministerio de Educacio´n y Ciencia, and the City Council of Vilanova i la Geltru´, where the conference took place from 8th to 10th June 2005. Since the first meeting of IWANN, held in Granada (1991), the artificial neural network (ANN) community, and the domain itself, have matured and evolved. Under the ANN banner we find a number of heterogeneous scenarios but all having the main interest and objective of providing a better understanding of nature and living beings for the correct development of theories, models and new algorithms. This is a very good way for scientists, engineers and professionals working in the area to develop solid and competitive applications. We are facing a real revolution with the emergence of embedded intelligence being applied into many artificial systems (that is, systems covering diverse fields: industry, home automation, healthcarey). So, we are convinced that an enormous amount of work must still be, and could be, done. Many pieces of the puzzle must be built and placed in their proper positions, offering us new and solid theories and models (necessary tools) for the application and praxis of these today’s paradigms. In IWANN 2005, after a careful review process of more than 240 submissions, 150 papers were accepted for publication including the contributions of the three invited speakers. This special issue reveals how IWANN deals with a wide variety of topics in neural computing and neuroscience connecting theoretical models with their applications in real life; this is the main reason for the sub-title of this special issue: Computational Intelligence and Bioinspired Systems. A number of authors were invited to submit an extended version of their conference paper to be considered for 0925-2312/$ - see front matter r 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2006.06.009
publication in this special issue of Neurocomputing. These authors were selected based on the recommendation of the reviews of the conference papers and the chairs of the different sessions. The extended versions were meticulously reviewed again by at least two independent and anonymous experts and the papers accepted, after this new review process, are presented in this volume. The papers can be grouped into the following four categories: (1) Neurocomputation formulations, considering the neurocomputation concept as a step toward the understanding of how the brain works in order to obtain valid models to build information processing systems; (2) Conventional models, including both theoretical and practical aspects of simple or hybrid models, (3) Neuroengineering and hardware implementations, and (4) Applications.
1. Neurocomputational formulations In the paper by Angelo di Garbo et al., the functional role of electrical synapses in a network of inhibitory interneurons is investigated by using a single compartment biophysical model of a Fast Spiking cell. In particular, the parameter values, which lead to the emergence of synchronous regimes in a network of Fast Spiking interneurons coupled by chemical and electrical synapses in the weak coupling limit, were determined theoretically. The paper by E.E. Claverol-Tinture´ et al. describes the progress in the polymer-on-multielectrode (PoM) technology, specifically towards the development of vertebrate neuronal cultures devoid of glial layers compatible with PoM devices, the connection of pairs of invertebrate neurons threading microchannels, and the recordings of synaptic and spike-like activity. The model authored by Hugges Berry and Oiliver Temam is based on a structure of biological neural networks that allows generating surrogate networks with realistic biological structure, as would be needed for complex information processing/computing tasks. This structure reproduces most of the graph properties exhibited by Caenorhabditis elegans, including its small-world structure. The development of this model contributes to the realization of chip–neuron interfaces using real biological neurons as long-term alternatives to silicon transistors.
ARTICLE IN PRESS 2702
Editorial / Neurocomputing 70 (2007) 2701–2703
2. Single and hybrid conventional models Enrique Romero and Rene´ Alque´zar obtain heuristically a suboptimal solution for the selection of weights of the new hidden units in sequential feed-forward neural networks. This process, that usually involves a non-linear optimization problem, cannot be solved analytically in the general case. The obtained results indicate that the orthogonalization of the output vectors of the hidden units outperforms the usual strategy of matching the residue, both for approximation and generalization purposes. The paper presented by R. Salas and co-workers introduces robustness and flexibility to the Hierarchical Self Organizing Maps model (RoFlex-HSOM) that has the plasticity to find the structure that best fit the data, gradually forgetting (but no catastrophically) previous learned patterns. The proposed algorithm is robust to the presence of outliers and preserves the topology to represent the hierarchical relation of the data in non-stationary environments. In the work developed by Ralf Eickhoff and Ulrich Ru¨ckert, the robustness of radial basis function networks in presence of malfunctioning elements and noise in its inputs and parameters is analyzed in order to operate in noisy and unreliable environments, as it happens in nanoelectronic devices and circuits. The work by Andre´s Berzal and Pedro Zufı´ ria addresses a comparative study with a deterministic discrete-time (DDT) formulation of the Sanger Hebbian ANN model that characterizes the average evolution of the net, preserving the discrete-time form of the original network and gathering a more realistic behavior of the learning gain. The results thoroughly characterize the relationship between the learning gain and the eigenvalue structure of the correlation matrix. The paper by Jero´nimo Arenas-Garcı´ a and co-workers addresses a general technique to reduce the computational burden associated to the operational phase of most neural networks, such as multi-net or radial basis function networks, which calculate their output as a weighted sum of terms. Sung-Kwun et al. introduce a new topology of fuzzyneural networks—fuzzy polynomial neural networks (FPNN)—that is based on a genetically optimized multilayer perceptron with fuzzy set-based polynomial neurons (FSPNs), and develop a comprehensive design methodology involving mechanisms of genetic optimization and information granulation. The performance of the networks is quantified through experimentation where they use two modeling benchmarks commonly employed within the area of fuzzy or neurofuzzy modeling. Ana Porto and co-workers develop a new hybrid learning method based on recent studies that have confirmed that the modulation of synaptic efficacy affects emergent behavior of brain cells assemblies. In order to find the best solution to a given problem, the new method combines the use of genetic algorithms with particular changes to connection weights based on this behavior.
The work by Miguel Rocha et al. presents two hybrid evolutionary computation/ANN algorithms: the first evolves neural topologies while the latter performs simultaneous optimization of architectures and weights. Sixteen real-world tasks were used to test these strategies. Competitive results were achieved when these algorithms were compared with a heuristic model selection and other data mining algorithms. 3. Neuroingeneering and hardware implementations The paper by Christian Morillas and co-workers shows the development of a set of software and hardware tools to interface with neural tissue, in order to transmit visual information encoded into a bioinspired neural-like form. The set is composed of a retina-like encoder, implemented both in software and on FPGA chips, and a platform to optimize the electrical stimulation parameters for a multielectrode implant. The main objective is to progress towards a functional visual prosthesis for the blind. Miguel Atencia et al. present a FPGA implementation of a hardware module that performs parametric identification of dynamical systems based upon the methodology of optimization with Hopfield neural networks, leading to an adapted version of these network models. This implementation achieves modularity and flexibility due to the usage of parametric VHDL to describe the network. The natural parallelism of neural networks is preserved at a limited cost in terms of circuitry cost and processing time. 4. Applications Contrary to the conventional point of view based on the modeling of non-linear systems, the paper by K. Madani and L. Thiaw proposes to deem the multi-modeling identification concept as building a modular architecture, inspired from ANN operation mode, where each neuron (module), represented by one of the local models, realizes some higher level transfer function in the non-linear system’s behavior identification and prediction context. Alberto Guille´n et al. present a new algorithm that applies fuzzy logic to a previous clustering technique for the initialization of the centres of the radial basis functions used in a neural network that approximates a function. The results achieved by the new algorithm improve significantly the results obtained by its ancestor and by other algorithms in the literature developed for similar purposes. A global methodology, which combines direct prediction strategy and sophisticated input selection criteria for the long-term prediction of time series, is proposed by Antti Sorjamaa et al. This methodology is successfully applied to a real life benchmark: the Poland Electricity Load dataset. Luis Javier Herrera et al. paper introduces a general technique that upgrades the performance of one-stepahead prediction models, when they are used recursively for long-term time series prediction; in order to show the operation of this new general technique, the work has used
ARTICLE IN PRESS Editorial / Neurocomputing 70 (2007) 2701–2703
two different methodologies as one-step ahead models to perform long-term recursive prediction: the TaSe Fuzzy TSK model and the least-squares SVMs. Juan Jose´ Murillo et al. propose a new method to solve the problem of blind robust watermarking of digital images. The method is based on the application of the independent component analysis (ICA) to compute some statistically independent transform coefficients where the watermark is embedded. With this approach, the ICA transformation is not unique, so each user can define its own as his private-key, and some of these transform coefficients have white noise-like spectral properties. In the paper by Cemil Oz and Ming C. Leu, two different American Sign Language (ASL) word recognition systems are proposed using ANN to translate the ASL words into English. These methods were compared with each other showing accuracy of recognition within 92% and 95%, respectively. Note that the sign language, which is a highly visual-spatial, linguistically complete, and natural language, are the main modes of communication among deaf people. Jesu´s Fraile-Ardanuy and co-workers develop a novel adaptive power system stabilizer (PSS). It is based on the online tuning of conventional PSS parameters for a singlemachine infinite bus system, using adaptive network-based fuzzy inference systems. The training data for these fuzzy neural networks are automatically generated based on the
2703
optimization of a fitness function using genetic algorithms. The proposed stabilizer has been tested by performing simulations of an overall non-linear system. The guest editors would like to express their gratitude to all authors for their submissions and the anonymous reviewers for their comments and useful suggestions in order to improve the papers. We would also like to thank Dr. Tom Heskes and the Neurocomputing editorial board for giving us the opportunity to publish this special issue, and Elsevier as the publisher for the very efficient and seamless management of the publication process. It is a pleasure for us to invite all authors and interested readers of this issue to future IWANN conferences, which are announced at http://www.iwann-conference.org.
Alberto Prieto Universidad de Granada, Spain E-mail address: [email protected] Joan Cabestany Universitat Polite`cnica de Catalunya, Barcelona, Spain Francisco Sandoval Universidad de Ma´laga, Spain