- Email: [email protected]
Neural-symbolic networks for cognitive capacities
Biologically Inspired Cognitive Architectures (2014) 9, iii– iv
Available at www.sciencedirect.com
ScienceDirect journal homepage: www.elsevier.com/locate/bica
EDITORIAL
Neural-symbolic networks for cognitive capacities Over the last years, intelligent technologies and cognitively-inspired systems have gained popularity and widespread acceptance, be it as part of natural-language or gesture-based interfaces to smart phones and computing devices, in the form of cyber-physical systems deployed as part of modern workplaces and manufacturing lines, as assistance and support units in ambient-assisted living spaces and smart homes, or in the form of assistance and control systems for drivers and pilots in cars and airplanes. Still, in the less application-driven and product-oriented domain of human-level artificial intelligence and cognitive systems modelling, researchers continue to face fundamental challenges in their quest to develop biologically and cognitively plausible models and implementations of cognitive capacities and intelligence. One of the methodological core issues is the question of the integration between subsymbolic and symbolic approaches to knowledge representation, learning and reasoning in cognitively-inspired models. Whilst the past has seen many heated debates about whether neural or symbolic approaches are more suitable to provide a general framework for intelligent processing, in recent years movement towards consensus has emerged agreeing on the need for integrated neural-symbolic processing. The motivation for this insight comes from different sources:
From the perspective of cognitive and computational neuroscience, a symbolic interpretation of an artificial neural network architecture is desirable, since the brain has a neuronal structure and the capability to perform symbolic processing. Network-based approaches very often enable flexible tools which can discover and process the internal structure of (possibly large) data sets. They promise to give rise to efficient signal-processing models http://dx.doi.org/10.1016/S2212-683X(14)00061-9
which are biologically plausible and optimally suited for a wide range of applications, whilst possibly also offering an explanation of cognitive phenomena of the human brain. On the other hand, from the perspective of symbolic knowledge-based processing, neural-symbolic representations seem to offer a chance for integration of several complementary properties. Whilst symbolic representations tend to be superior in terms of their interpretability, the possibilities of direct control and coding, and the extraction of knowledge, neural representations clearly possess a higher degree of biological motivation, and outmatch symbol-based approaches in terms of learning capacities, robust fault-tolerant processing, and generalization to similar input. Since these advantages are mutually complementary, the hope in developing symbolic connectionist architectures is to combine the respective strong points whilst mutually mitigating the weaknesses, with the resulting neural-symbolic systems possibly offering the key to finally unlocking answers to the intelligence puzzle. But although research on integrated neural-symbolic systems has made significant progress over the last two decades, the extraction of high-level explicit (i.e. symbolic) knowledge from distributed low-level representations thus far has to be considered a mostly unsolved problem. In recent years, network-based models have seen significant advancement in the wake of the development of the new deep learning family of approaches to machine learning. Due to the hierarchically structured nature of the underlying models, these developments have also reinvigorated efforts in overcoming the neural-symbolic divide. The aim of this special issue on ‘‘Neural-Symbolic Networks for Cognitive Capacities’’ is to bring together recent work developed in the field of network-based information
iv processing in a cognitive context, which bridges the gap between different levels of description and paradigms and which sheds light onto canonical solutions or principled approaches occurring in the context of neural-symbolic integration to modelling or implementing cognitive capacities. In doing so, rather than providing an exhaustive overview of the entire field or advocating one particular approach as the one and only solution to the problem of neural-symbolic integration, this volume wants to highlight the rich variety of paradigms and scenarios considered by different researchers working towards the unification of symbolic and connectionist computation in cognitive modelling and human-level artificial intelligence. Also, opening up the topic and tying into the wider context of general neuralsymbolic integration, two contributions on metacognition have been included in the special issue. Models of metacognition, representing the upper end of the scale of symbolic processing in cognitive modelling, are of relevance for neural-symbolic integration in addressing the challenge of integrating analog and symbolic representations into one framework. Zhang and Liu propose a salient object detection framework that surpasses the bottom-up versus top-down processing dichotomy in directing attentional behavior, instead relying on a selection history, the current goal and physical salience as determining criteria. Rutledge-Taylor, Kelly, West and Pyke describe Dynamically Structured Holographic Memory as a lifetime-scalable model of human memory using high dimensional vectors for representing items in memory, whilst Emruli, Sandin and Delsing use a binary vector symbolic architecture in combination with a sparse distributed memory component for modelling context-dependent prediction by learning from examples in interoperable systems. Glodek, Geier, Biundo and Palm present a layered architecture for probabilistic complex pattern recognition, using different temporal granularities for inferring complex patterns of user preferences, and Biswas, Sinha, Purakayashta and Marbaniang report on a hybrid expert system combining case-based reasoning with neural networks for classification, trying to overcome the feature weighting problem in case-based reasoning. Achler introduces a symbolic neural network using feedforwardfeedback connections similar to auto-associative networks but with the symmetrical feedback connections being inhibitory, promising better performance in terms of symbolic recall than most previous approaches. Caro, Josyula, Cox
Editorial and Jimenez propose and validate a metacognition metamodel for intelligent systems based on a literature analysis of existing approaches. Finally, Samsonovich give an account of goal reasoning as general form of metacognition in biologically inspired cognitive architectures and present and evaluate a general model of goal reasoning. In summary, the field of neural-symbolic integration has seen progress over the last decades. Today, symbolic connectionist cognitive systems and models are being studied and deployed in many different contexts. Still, a great number of challenges remain as of yet unresolved: Many questions in the extraction of symbolic knowledge from connectionist representations, in the integration between symbolic and sub-symbolic forms of computation and processing, and in the suitability, the strengths and weaknesses, as well as in the modes of implementation of different paradigms for specific classes of application scenarios are unanswered and merit further study and reinforced research efforts. A fairly general answer to any of these questions undoubtedly would constitute a significant step towards solving the problem of neural-symbolic integration as one of the most fundamental current questions in cognitive modeling and human-level artificial intelligence.
Tarek R. Besold Institute of Cognitive Science, ¨ck, 49069 Osnabru ¨ck, University of Osnabru Germany Artur d’Avila Garcez Department of Computer Science, School of Informatics, City University, London EC1V OHB, UK Kai-Uwe Ku ¨hnberger Institute of Cognitive Science, ¨ck, 49069 Osnabru ¨ck, University of Osnabru Germany Terrence C. Stewart Computational Neuroscience Research Group, 200 University Avenue West, Waterloo, ON N2L 3G1, Canada
Available at www.sciencedirect.com
ScienceDirect journal homepage: www.elsevier.com/locate/bica
EDITORIAL
Neural-symbolic networks for cognitive capacities Over the last years, intelligent technologies and cognitively-inspired systems have gained popularity and widespread acceptance, be it as part of natural-language or gesture-based interfaces to smart phones and computing devices, in the form of cyber-physical systems deployed as part of modern workplaces and manufacturing lines, as assistance and support units in ambient-assisted living spaces and smart homes, or in the form of assistance and control systems for drivers and pilots in cars and airplanes. Still, in the less application-driven and product-oriented domain of human-level artificial intelligence and cognitive systems modelling, researchers continue to face fundamental challenges in their quest to develop biologically and cognitively plausible models and implementations of cognitive capacities and intelligence. One of the methodological core issues is the question of the integration between subsymbolic and symbolic approaches to knowledge representation, learning and reasoning in cognitively-inspired models. Whilst the past has seen many heated debates about whether neural or symbolic approaches are more suitable to provide a general framework for intelligent processing, in recent years movement towards consensus has emerged agreeing on the need for integrated neural-symbolic processing. The motivation for this insight comes from different sources:
From the perspective of cognitive and computational neuroscience, a symbolic interpretation of an artificial neural network architecture is desirable, since the brain has a neuronal structure and the capability to perform symbolic processing. Network-based approaches very often enable flexible tools which can discover and process the internal structure of (possibly large) data sets. They promise to give rise to efficient signal-processing models http://dx.doi.org/10.1016/S2212-683X(14)00061-9
which are biologically plausible and optimally suited for a wide range of applications, whilst possibly also offering an explanation of cognitive phenomena of the human brain. On the other hand, from the perspective of symbolic knowledge-based processing, neural-symbolic representations seem to offer a chance for integration of several complementary properties. Whilst symbolic representations tend to be superior in terms of their interpretability, the possibilities of direct control and coding, and the extraction of knowledge, neural representations clearly possess a higher degree of biological motivation, and outmatch symbol-based approaches in terms of learning capacities, robust fault-tolerant processing, and generalization to similar input. Since these advantages are mutually complementary, the hope in developing symbolic connectionist architectures is to combine the respective strong points whilst mutually mitigating the weaknesses, with the resulting neural-symbolic systems possibly offering the key to finally unlocking answers to the intelligence puzzle. But although research on integrated neural-symbolic systems has made significant progress over the last two decades, the extraction of high-level explicit (i.e. symbolic) knowledge from distributed low-level representations thus far has to be considered a mostly unsolved problem. In recent years, network-based models have seen significant advancement in the wake of the development of the new deep learning family of approaches to machine learning. Due to the hierarchically structured nature of the underlying models, these developments have also reinvigorated efforts in overcoming the neural-symbolic divide. The aim of this special issue on ‘‘Neural-Symbolic Networks for Cognitive Capacities’’ is to bring together recent work developed in the field of network-based information
iv processing in a cognitive context, which bridges the gap between different levels of description and paradigms and which sheds light onto canonical solutions or principled approaches occurring in the context of neural-symbolic integration to modelling or implementing cognitive capacities. In doing so, rather than providing an exhaustive overview of the entire field or advocating one particular approach as the one and only solution to the problem of neural-symbolic integration, this volume wants to highlight the rich variety of paradigms and scenarios considered by different researchers working towards the unification of symbolic and connectionist computation in cognitive modelling and human-level artificial intelligence. Also, opening up the topic and tying into the wider context of general neuralsymbolic integration, two contributions on metacognition have been included in the special issue. Models of metacognition, representing the upper end of the scale of symbolic processing in cognitive modelling, are of relevance for neural-symbolic integration in addressing the challenge of integrating analog and symbolic representations into one framework. Zhang and Liu propose a salient object detection framework that surpasses the bottom-up versus top-down processing dichotomy in directing attentional behavior, instead relying on a selection history, the current goal and physical salience as determining criteria. Rutledge-Taylor, Kelly, West and Pyke describe Dynamically Structured Holographic Memory as a lifetime-scalable model of human memory using high dimensional vectors for representing items in memory, whilst Emruli, Sandin and Delsing use a binary vector symbolic architecture in combination with a sparse distributed memory component for modelling context-dependent prediction by learning from examples in interoperable systems. Glodek, Geier, Biundo and Palm present a layered architecture for probabilistic complex pattern recognition, using different temporal granularities for inferring complex patterns of user preferences, and Biswas, Sinha, Purakayashta and Marbaniang report on a hybrid expert system combining case-based reasoning with neural networks for classification, trying to overcome the feature weighting problem in case-based reasoning. Achler introduces a symbolic neural network using feedforwardfeedback connections similar to auto-associative networks but with the symmetrical feedback connections being inhibitory, promising better performance in terms of symbolic recall than most previous approaches. Caro, Josyula, Cox
Editorial and Jimenez propose and validate a metacognition metamodel for intelligent systems based on a literature analysis of existing approaches. Finally, Samsonovich give an account of goal reasoning as general form of metacognition in biologically inspired cognitive architectures and present and evaluate a general model of goal reasoning. In summary, the field of neural-symbolic integration has seen progress over the last decades. Today, symbolic connectionist cognitive systems and models are being studied and deployed in many different contexts. Still, a great number of challenges remain as of yet unresolved: Many questions in the extraction of symbolic knowledge from connectionist representations, in the integration between symbolic and sub-symbolic forms of computation and processing, and in the suitability, the strengths and weaknesses, as well as in the modes of implementation of different paradigms for specific classes of application scenarios are unanswered and merit further study and reinforced research efforts. A fairly general answer to any of these questions undoubtedly would constitute a significant step towards solving the problem of neural-symbolic integration as one of the most fundamental current questions in cognitive modeling and human-level artificial intelligence.
Tarek R. Besold Institute of Cognitive Science, ¨ck, 49069 Osnabru ¨ck, University of Osnabru Germany Artur d’Avila Garcez Department of Computer Science, School of Informatics, City University, London EC1V OHB, UK Kai-Uwe Ku ¨hnberger Institute of Cognitive Science, ¨ck, 49069 Osnabru ¨ck, University of Osnabru Germany Terrence C. Stewart Computational Neuroscience Research Group, 200 University Avenue West, Waterloo, ON N2L 3G1, Canada