A Novel Robust Metric for Comparing the Intelligence of Two Cooperative Multiagent Systems

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 96 (2016) 637 – 644

20th International Conference on Knowledge Based and Intelligent Information and Engineering Systems, KES2016, 5-7 September 2016, York, United Kingdom

A novel robust metric for comparing the intelligence of two cooperative multiagent systems Laszlo Barna Iantovicsa*, Corina Rotarb, Elena Nechitac b

a *Petru Maior University of Tirgu Mures, Romania, ”1 Decembrie University” of Alba Iulia, Romania , Vasile Alecsandri University of Bacau, Romania

Abstract Cooperative multiagent systems are used for solving many computational hard problems. In the scientific literature, the intelligence of cooperative multiagent systems is considered at the systems’ level and is based on the “intelligent problem solving” consideration (highly efficient and flexible problems solving; difficult problem solving, with missing or erroneous data; efficient solving of NP – hard problems). In this paper, we propose a novel accurate metric called MetrIntComp (Metric for Cooperative Multiagent Systems Intelligence Comparison) for a robust comparison of two cooperative multiagent system’s intelligence, effective even in the case of small differences in intelligence between the considered systems. For proving the effectiveness of the metric we considered an illustrative case study for two cooperative multiagent systems composed of simple agents, in that the intelligence emerge at the systems’ level, each of them specialized on solving the same type of computational difficult, NP-hard problem. The conclusion of the case study was that the metric is able to make a differentiation between the two multiagent systems even the numerical difference between the measured intelligence is small. Based on this fact, the two multiagent systems could not be considered that belong to the same class of intelligence.

© 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license © 2016 The Authors. Published by Elsevier B.V. (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under under responsibility Peer-review responsibilityofKES of KESInternational. International Keywords: cooperative multiagent system, intelligent system, computational intelligence, Traveling Salesman Problem, ant algorithms

1. Introduction Most of the developed intelligent systems are agent-based systems [2, 3, 10, 16]. We call agent-based system generally agents and multiagent systems. Most of the developed multiagent systems are cooperative. In a cooperative multiagent system, the agents cooperatively solve the undertaken problems. Not all the agent-based systems are intelligent. It is not a mandatory property for an agent or a multiagent system to be intelligent. In our approach, we will refer to agent-based systems, cooperative multiagent systems composed of two or more agents that cooperatively solve problems. The members of such a system are not necessarily intelligent but at the system’s level emerge an increased intelligence. The study of those aspects related to computing systems intelligence is important in order to develop computing systems that are able to solve hard problems with different types of difficulties. Many times the most feasible solution

* Corresponding author. Tel.: +0-000-000-0000 ; fax: +0-000-000-0000 . E-mail address:[email protected]

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of KES International doi:10.1016/j.procs.2016.08.245

638

Laszlo Barna Iantovics et al. / Procedia Computer Science 96 (2016) 637 – 644

of difficult problems solving consists in the intelligent agent-based systems approaches. Cooperative multiagent systems are used for many problems solving [2, 3]. Many times, in the scientific literature a cooperative multiagent system is considered intelligent based on the simple consideration that the efficient and flexible cooperation between the agents emerge in intelligence at the systems’ level. Intelligent multiagent systems could be composed even of simple agents, who very efficiently solve problems. The existence of some properties that could be associated with intelligence does not allow a quantitative evaluation, just proves its existence. We consider that the evaluation of a system’s intelligence should be made by some metrics that measure the “quantity of intelligence”. In this paper, we propose an accurate metric called MetrIntComp (Metric for Cooperative Multiagent Systems Intelligence Comparison) for the robust comparison of two cooperative multiagent systems’ intelligence. For demonstrating the effectiveness of the metric we designed an illustrative case study of two cooperative multiagent systems, specialized in solving the TSP (Traveling Salesman Problem) that is an NP-hard problem. The metric is able to treat the variability in the intelligence of the multiagent systems. Depending on different situations, a cooperative multiagent system sometimes behaves with lower intelligence, other times behaves with higher intelligence. The upcoming part of the paper is organized as follows: Section 2 presents different considerations related to intelligent agentbased systems and metrics for measuring the intelligence; in Section 3 is presented our proposal, in Section 3.1. is presented the proposed metric, Section 3.2 shows a case study, Section 3.3 includes a discussion on the proposed metric and the results of the case study; In Section 4 the conclusions of the research are presented. 2. Intelligent Agent-based Systems. Measuring Agent-Based System’s Intelligence In this section, we present some considerations related to intelligent agent-based systems, application of agent -based systems and some elaborated metrics for measuring the intelligence. There are numerous researches presented in the scientific literature, that are focused on the development of agent-based systems able to intelligently solve different computational hard problems [2, 3, 16]. There are many interesting applications of intelligent agent-based systems, like regulation of the buyers’ distribution in management systems based on simultaneous auctions [17], methods for the management of distributed electricity networks using market mechanisms [18], multiple energy carrier optimizations [19], users expect intelligent virtual agents to recall and forget personal conversational content [20], discovering Semantic Web services using SPARQL and intelligent agents [21], Intelligent Transportation Systems modeling with combinatorial auctions [22], web services and intelligent agents-based negotiation system for B2B eCommerce [23], tasks scheduling and communications management in a critical care telemonitoring system [24], and so on. There is no unanimous definition of the agents’ intelligence [2]. The intelligence estimation of the agent-based systems is realized based on different considerations. Many times the intelligence of an agent-based system is considered based on capabilities, like [3]: learning, self-adaptation, and evolution. To illustrate the impossibility to define the agents’ intelligence, let us consider the differences in intelligence between static software agents vs. of mobile software agents. Generally, mobile agents are more limited in intelligence than the static agents. There are very few developed mobile agents that could be considered intelligent. Limitations in the mobile agents’ endowment with intelligence are based on some practical reasons, like: network overloading (the transmission of a large number of intelligent mobile agents - many times knowledge-based, with increased size, may overload the network with data transmission), hosts overloading (a large number of intelligent mobile agents, that execute complex computations, at a host may overload that host), limited communication possibility (the mobile agents migrate during their operation in the network, based on this fact it is difficult to estimate where a mobile agent is at a moment of time). The chapter [16] argues that intelligent agents must know more than just the task they are performing during collaboration with humans. Intelligent agents should be able of managing engagement with the humans. The implementation of the affect is important in pertaining an efficient collaboration. The agents should develop a collaboration relation with the humans. There are some researches [4, 9] focused on the study of decision making in the frame of cooperative coalitions. Decisions taken in the frame of coalitions, often outperform the decisions of individuals that operate in isolation. In many cooperative multiagent systems, the intelligence could be considered at the level of the whole system [2, 3]. The intelligence in these systems is higher than the individual agents’ intelligence [2, 3]. Yang, Galis, Guo and Liu [8] present an intelligent mobile multiagent system composed of simple reactive agents. The mobile agents are specialized in a computer network administration. They are endowed with knowledge retained as a set of rules. The multiagent system could be considered intelligent based on the fact that it simulates the behavior of a network administrator. There are different metrics developed for making different measurements in intelligent systems. Such metrics not always are developed for measuring the whole system’s intelligence, but only for measuring some aspects that represent interest. The paper [44] analyses the fault tolerant systems. A fault tolerant system is able to diagnose and recover from faults. Sometimes this property of the systems is associated with the intelligence. The authors propose a useful metric for evaluation that is the effectiveness measure of fault- tolerance. The paper [45] presents a study realized by the US National Institute of Standards and Technology (NIST) related to the creating standard measures for intelligent systems. The researchers outline the question related with how precisely intelligent

Laszlo Barna Iantovics et al. / Procedia Computer Science 96 (2016) 637 – 644

systems are defined and how to measure and compare the capabilities that intelligent systems should provide. NIST's initial approach to establishing performance metrics attempts to address different pragmatic and theoretical aspects. In the paper [29] the agent-based systems’ intelligence is considered based on the ability to compare alternatives with different complexity. In their research it is considered an agent-based distributed sensor network system. For measuring the intelligence a specific approach is applied. The proposal was tested by comparing the MIQ of different agent-based scenarios. Hernández-Orallo and Dowe [30] propose the idea of a universal anytime intelligence test. Based on the authors’ consideration such a test should be able to measure the intelligence level of any biological or artificial system. It should be able to measure very low and very high intelligence level. The proposed approach is based on C-tests and compression-enhanced Turing tests developed in the late 1990s. The authors discuss different tests highlight their limitations. They introduce some new ideas that they consider necessary for the development of a “universal intelligence test”. Hibbard, B. [38] proposes a measure of intelligence based on a hierarchy of sets of increasingly difficult environments. An agent’s intelligence is measured as the ordinal of the most difficult set of environments that it can pass. The measure is defined in finite state machine models of computing. The measure includes the number of time steps required to pass the test. Winklerová [39] presents a solution to measuring the collective intelligence of particle swarm systems using a model called Maturity Model. The model is derived from the Maturity Model of C2 (Command and Control) operational space and the model of Collaborating Software. The objective of the research was to gain explanation on the emergence of intelligence in the particle swarm system. 3. A novel metric for comparison of two multiagent systems’ intelligence 3.1. The proposed robust algorithm for the intelligence comparison In the following, we propose an accurate metric for the comparison of two multiagent systems intelligence composed of simple reactive agents. The agents interact with each other based on some simple rules. We consider two multiagent systems, denoted with MGA and MGB. MGA={MGA1,MGA2,…..,MGAn}. |MGA|, |MGA|=n denotes the cardinality (number of agents) of MGA. MGB={MGB1, MGB2,…..MGBm}. |MGB|, |MGB|=m denotes the cardinality of MGB. Algorithm “Multiagent Systems Intelligence Comparison” abbreviated as MetrIntComp presents our proposed metric for the intelligence comparison. INTA={INTA1, INTA2,….., INTAr} denotes the measured intelligence indicators obtained during the intelligence evaluation of different problems solving for the evaluation of the problem’s solving intelligence of the MGA system. |INTA| (where |INTA|=r) denotes the intelligence indicators sample size. INTB={INTB1, INTB2,….., INTBk} denotes the measured intelligence indicators obtained during the intelligence evaluation in different simulated scenarios of the MGB system. |INTB| (where |INTB|=k) denotes the intelligence indicators sample size. MetrIntComp: Algorithm Multiagent Systems Intelligence Comparison IN: INTA={INTA1, INTA2,….., INTAr}; INTB={INTB1, INTB2,….., INTBk}; Out: IntelligenceComparisonDecision; Step1. verification of IntA and IntB data normality //Finding the answer to the question if IntA is sampled from a Gaussian population @ verifies if IntA is normally distributed //Finding the answer to the question if IntB is sampled from a Gaussian population @ verifies if IntB is normally distributed If (both IntA and IntB are normally distributed) then @Suggests to humans that could be elaborated a more appropriate metric. @set the human approval to YES or NO. EndIf Step2. Verification of intelligence equality //it was not necessary the approval if the data were not normally distributed If (approval == YES or “was not necessary the approval”) then //formulate the Null Hypothesis @formulate H0 //formulate the Alternative Hypothesis @formulate H1 @Apply the Mann-Whitney test for Two Unpaired Samples. @Obtain the P-value. Step 3. Interpretation of the intelligence evaluation result If (P-value>) then Begin

639

640

Laszlo Barna Iantovics et al. / Procedia Computer Science 96 (2016) 637 – 644

//CANNOT be realized a differentiation between the intelligence of MGA and MGB @Accept H0. @MGA intelligence is statistically equal to the MGB intelligence. End If (P-valued) then Begin //CAN be realized a differentiation between the intelligence of MGA and MGB @accept H1 If (CentrIndA >CentrIndB) then @MGA is more intelligent than the MGB. Else @MGA is less intelligent than the MGB. End EndComparisonMultiagentSystemsIntelligence For the estimation of INTA and INTB sample sizes we have realized a calculus, based on the established;  (alpha); E (beta); power=1-E; Effect size.  is a parameter of the algorithm, representing the probability of rejecting the null hypothesis when it is true, to make a type I error. E represents the type II error. A type II error is the failure to reject a false null hypothesis. Broadly speaking, a type I error is detecting an effect that is not present, while a type II error is failing to detect an effect that is present. An effect size is a quantitative measure of the strength of a phenomenon. The proposed intelligence comparison is based on a specific mathematical calculus using the indicators obtained as results of some simulations in the case of MGA and MGB, retained in INTA={INTA1, INTA2,….., INTAr} corresponding to MGA and INTB={INTB1, INTB2,…..,INTBk} corresponding to MGB. The Comparison Multiagent Systems Intelligence algorithm was proposed as a solution for a robust intelligence evaluation. In Step 1 of the algorithm it is verified the Data Normality in the case of both samples INTA and INTB, if they are normally distributed, having the significance that INTA and INTB are sampled from a Gaussian population. If both samples do not pass the normality test than it is recommended the application of the proposed metric, elsewhere it is recommended the elaboration of a more appropriate metric, adapted to the properties of data. The previous affirmation is a general one (indication that could be elaborated a more appropriate metric), which is based on the consideration that if about a problem/task is available additional information/knowledge than it could be possible to elaborate a more accurate problem/task solving method that takes into consideration the available auxiliary information/knowledge. The presented metric for the intelligence comparison is developed as a way to be robust. Based on this fact, it can be considered that it is possible to be designed a less robust, but more accurate metric that takes into consideration the data properties, and works for less available data than the actual one. An example As a very simple illustrative example, we consider the problem to search for a number in a set of numbers. If about the set of numbers there are not available additional information, like that they are ordered, than can be applied the sequential search method. Elsewhere, if about the numbers there are available another information that the numbers are ascending ordered, than it can be applied a more efficient search method adapted based on the ordering properties of the numbers. Another information that could be helpful in the development of a more efficient algorithm consists in the existence of the number of searches of each number (how many times was realized search for each number), which can give use the idea that it could be implemented a more appropriate search method that takes into consideration this available information. The main property of the proposed metric consists in the robustness, and for accurate evaluation, it is recommended to be used for larger samples. Applying the Multiagent Systems’ Intelligence Comparison algorithm the MGA and MGB intelligence can be compared. We call, Null Hypothesis denoted as H0, the statement that the CentrIndA of MGA is equal from the statistical point of view with the CentrIndB of MGB. We call Alternative Hypothesis and denote it with H1 the hypothesis that the CentrIndA of MGA is statistically different from CentrIndB of MGB. The testing of H0 and H1 is realized with the significance level denoted . We considered for , the value =0.05. As central intelligence indicator of both measured intelligence samples INTA and INTB it can be considered the mean or the median. If in a data set a value is changed significantly than the mean it could change more than the median. Based on this aspect, we consider that the median is more robust than the mean. In our algorithm, we considered as a central intelligence indicator the median based on the robustness consideration of the proposed metric. In the presented Multiagent Systems’ Intelligence Comparison algorithm, for the H0 testing, we considered as the most appropriate the application of the nonparametric Mann-Whitney test for Two Unpaired Samples [5, 42, 43]. If the H0 is verified then can be concluded that the two swarm systems’ intelligence is equal from the statistical point of view. The numerical difference is the consequence of variability within samples. We consider that making the measurements in different experimental conditions it is possible to obtaine slightly different results.

Laszlo Barna Iantovics et al. / Procedia Computer Science 96 (2016) 637 – 644

If H1 is accepted, and CentrIndA CentrIndB then can be concluded that MGB is less intelligent than the MGA. 3.2. Case study for comparison of two swarm systems’ intelligence For proving the effectiveness of the proposed metric we realized a case study. There was considered the TSP (Traveling Salesman Problem) solving, a very well known NP-hard problem in combinatorial optimization [6, 15]. The TSP has many applications [25, 26, 46]. The TSP problem can be announced as follows: a map is given it includes a list of cities; there are distances between each pairs of cities. The TSP problem addresses the question what is the shortest possible route that visits each city once and returns to the origin city? Ant Colony Optimization techniques represent a meta-heuristic for hard combinatorial optimizing problems solving. They simulate the behavior of ant colonies to determine the shortest path to food. Individual ants are very simple, but emerge a complex behavior at the colony level. Each ant deposits pheromone on the trail while walking. Ants follow the pheromone trails with some probability which is proportioned to the density of the pheromone. More ants walk on a trail, the more pheromone is deposited on it and more and more ants follow the trail. Ant Colony Optimization algorithms consist on more ants that imitate the behavior of real ants. The Ant Colony-based problem solving was proposed by Marco Dorigo [27]. The general problem-solving idea can be explained as following. At the first step of the problem solving, each ant is placed on some randomly chosen city. An ant denoted with the number k that is at city i chooses to move to a city j by applying a transition rule (1).

(1) In formula (1) there are used the following notations: dij is the distance between city i and j; Kij is the heuristic visibility of edge (i, j); generally Kij=1/dij. Jk(i) is a set of cities which remain to be visited when the ant is at city i. The positive parameters  and  control the relative weights of the pheromone trail and of the heuristic visibility. There could be a trade-off between edge length and pheromone intensity. After each ant completes its tour, the pheromone amount on each path will be adjusted according to (2).

(2) In the equations (2) we have used the following notations; Q is an arbitrary constant. m is the number of ants. 1-p (0
641

642

Laszlo Barna Iantovics et al. / Procedia Computer Science 96 (2016) 637 – 644

Table 1.Simulation results, in order to establish the two Multiagent Systems Intelligence in problems solving Best-Worst Ant. Colony System/ INTA

Ant Colony System/ INTB

7.172; 6.864; 7.691; 6.12; 6.572; 6.612; 7.413; 5.786; 6.262; 6.626; 6.135; 6.217; 6.586; 6.467; 8.084; 7.313; 7.295; 6.473; 6.516; 6.657; 7.009; 8.297; 7.714; 6.729; 7.177; 6.887; 6.612; 5.99; 7.007; 7.333; 5.78; 6.585; 7.257; 7.225; 8.005; 6.592; 6.741; 6.37; 5.944; 6.573; 6.911; 6.513; 7.447; 7.066; 7.277; 6.924; 7.343; 6.204

5.342; 4.868; 6.134; 5.251; 4.85; 5.307; 5.605; 5.624; 6.053; 4.788; ; 5.055; 5.153; 5.779; 4.87; 5.339; 4.992; 5.322; 5.363; 5.493; 5.004; 5.261; 5.476; 5.469; 7.278; 5.53; 5.084; 5.337; 5.595; 5.352; 4.631; 5.068; 4.911; 4.831; 5.431; 4.933; 4.987; 5.092; 5.377; 5.589; 5.623; 5.563; 5.195; 5.926; 5.136; 5.325

Figure 1. Graphical representation of Intelligence Indicators Table 2.Results obtained by applying the Multiagent Systems Intelligence Comparison Algorithm Best-Worst Ant Colony System/MGA

Ant Colony System/ MGB

Median

6.735 (InvCentrIndA)

5.325 (InvCentrIndB)

Sample size

48 (IntA)

45 (IntB)

Minimum

5.780

4.631

Maximum

8.297

7.278

Mean

6.8411041667

5.3376

According to Step 1 of the algorithm, we verified if both samples pass the normality test. For the normality testing, we applied the Kolmogorov-Smirnov Goodness-of-Fit Test [28]. If both samples does not pass the normality test and based on this consideration the decision was to use of Mann-Whitney test for Two Unpaired Samples. Mann-Whitney is a nonparametric test. We have decided to use the Two-tail test. Applying the Mann-Whitney test, we have obtained that the two-tailed P-value is <0.0001, which indicate a significant difference between the medians of the MGA and MGB for the TSP problem solving having 35 cities placed on the map. Other calculation details were: Mann-Whitney U-Statistic=48; U’=2122; Sum of ranks in Best-Worst Ant Colony System=3288; Sum of ranks in Ant Colony System =1083 The results show that the Best-Worst Ant Colony System intelligence is lower than the intelligence of the Ant Colony System (InvCentrIndA>InvCentrIndB and H1 is true). It is based on the fact that in both MGA and MGB it was considered as intelligence indicator the best to date travel distance. However, it should be considered more intelligent those MAS that obtained the shortest path. 3.3. Discussions Many definitions of the cooperative multiagent systems’ intelligence are based on different considerations, most of them biologically inspired, like the intelligence of humans [31, 32], the intelligence of plants [33, 34, 35], the intelligence of swarms of insects [36], the intelligence of animals like the dogs [37]. The life on the earth is a result of a very long time evolution, thus resulting in very high complexity that many times emerges in intelligence. We consider that is impossible to give an unanimous definition of intelligence, based on the large variety of cooperative multiagent systems. We considered that an important research direction lesser explored in the scientific literature, is the elaboration of metrics for the evaluation of the intelligence that permit also the effective comparison of the intelligence level of more multiagent systems. In most of the papers that report intelligence measurements, these measurements are realized based on different considerations related to the cooperative multiagent systems. They are relatively particular (could be applied to particular multiagent systems) and mostly theory oriented. Some are very theoretical and does not have applicability (for instance, those article, which report the elaboration of the universal measures that could evaluate the intelligence of any artificial or biological system).

Laszlo Barna Iantovics et al. / Procedia Computer Science 96 (2016) 637 – 644

In this paper, we proposed an accurate metric for a robust comparison of two cooperative multiagent systems intelligence with the same type of intelligence. The multiagent systems should operate in a similar environment and should solve the same type/class of problems. The metric is based on experimental evaluations of the intelligence for different problems solving. It implies the identification in case of the multiagent systems of an intelligence indicator (it should indicate how intelligently the multiagent solves a problem). The metric makes a comparison of the two multiagent systems at the level of whole system not at the agent level. Our metric is appropriate for MAS, where the intelligence indicator for a problem solving is expressed as a single value. If necessary, this value can be calculated as a weighted sum of some other values that measure different aspects of the system’s intelligence. The proposed metric takes into consideration the variability in the intelligence of the compared MAS. A cooperative multiagent system could have different reactions with different degree of intelligence in different situations. We have chosen for validation purposes two multiagent systems with a heuristic behavior. As central intelligence indicator of a cooperative multiagent system, we have considered the median of measured intelligence indicators. Mostly, in evaluations the mean is considered as the most important central indicator, but the mean is less robust than the median. A newly included value modifies less the median than the mean. An outlier value could modify the mean in a higher degree. The choosing of the median was based on the fact that we have analyzed the intelligence data normality, that not passed. By working with not normal intelligence data samples offers robustness. Our metric could be applied in the case of normal data also, but we consider in this case that it could be developed a more accurate metric based on the specific property of the data. In the Algorithm Multiagent Systems Intelligence Comparison algorithm it is proposed the use of the Mann-Whitney test for Two Unpaired Samples [5, 42, 43]. It is known that is a nonparametric test, which is not sensitive to the data normality. In the case of multiagent systems that have a less heuristic behavior, we suggest that is more appropriate the t-Test for Two Independent Samples [5, 40, 41]. For the validation of proposed metric, we realized a case study. In the case study, we considered the simulation of two cooperative multiagent systems composed of simple reactive agents that operate in a physical environment, solving the TSP problem. The multiagent systems operate as a well known Best-Worst Ant Colony System [1, 11, 12] and Ant Colony System [7, 13]. Both multiagent systems with heuristic behavior were formed by simple agents that cooperatively solve the TSP problem. The agents’ communication is realized via signs. It is a similar communication with the biological ants realized via pheromones. 4. Conclusion In this paper, a novel accurate metric for robust comparison of two cooperative multiagent systems’ intelligence was proposed. The metric takes into consideration the variability in the intelligence of the compared multiagent systems (more or less intelligence manifested in different situations, different problems solving). Our proposal is able also to determine the necessary sample sizes of the two central intelligence indicators of the two multiagent systems. For the validation purposes, we realized a case study. We compared the intelligence of two cooperating multiagent systems composed of very simple agents able to solve the TSP problem that is an NP-hard problem. The multiagent systems operated similarly as the Best-Worst Ant Colony System and the Ant Colony System. The result of the intelligence comparison shows that the difference in the concretely measured intelligence of the two systems is true and it is not a consequence of the variability. By, repeating the intelligence evaluation similar result was obtained. Based on a comprehensive study of the scientific literature we consider that our proposed metric is innovative and will represent the basis for intelligence evaluation in many future researches. Acknowledgment The authors acknowledge the support of the project “Bacu and Lugano - Teaching Informatics for a Sustainable Society”, cofinanced by a grant from Switzerland through the Swiss Contribution to the enlarged European Union. References 1. Cordon O., de Viana I.F., Herrera F. Analysis of the Best-Worst Ant System and Its Variants on the QAP, M. Dorigo et al. (Eds.), LNCS 2002; 2463: 228–234. 2. Iantovics L.B. A New Intelligent Mobile Multiagent System. Proc.of the IEEE-SOFA 2005. Szeged-Hungary and Arad-Romania, IEEE CS Press. 2005; 153– 159 3. Iantovics L.B., Zamfirescu C.-B. ERMS:An Evolutionary Reorganizing Multiagent System, Innovative Computing, Information and Control, 2013; 9(3), 1171-1188. 4. Zamfirescu C-B, Duta L., Iantovics L.B. On investigating the cognitive complexity of designing the group decision process, Studies in Informatics and Control, 2010, 19(3), 263–270. 5. Marusteri M., Bacarea V. Comparing groups for statistical differences: how to choose the right statistical test? Biochemia Medica, 2010; 20(1):15-32 6. Leeuwen, Jan van, Ed. Handbook of Theoretical Computer Science. Vol. A, Algorithms and complexity. Amsterdam: Elsevier. 1998 7. Prakasam A., Savarimuthu N. Metaheuristic algorithms and probabilistic behaviour: a comprehensive analysis of Ant Colony Optimization and its variants, Artificial Intelligence Review, 2016; 45(1):97-130 8. Yang K., Galis A., Guo X., Liu D., Rule-Driven Mobile Intelligent Agents for Real-Time Configuration of IP Networks, Knowledge-Based Intelligent Information and Engineering Systems, Lecture Notes in Computer Science, 2003; 2773: 921-928.

643

644

Laszlo Barna Iantovics et al. / Procedia Computer Science 96 (2016) 637 – 644

9. West D., Dellana S. Diversity of ability and cognitive style for group decision processes, Information Sciences, 2009, 179: 542–55. 10. Guillaud A., Troadec H., Benzinou A., Bihan J.L., Rodin V.A. Multiagent System for Edge Detection and Continuity Perception on Fish Otolith Images, EURASIP Journal on Applied Signal Processing 2002; 7:746–753 11. Lutuksin T., Pongcharoen P. Best-Worst Ant Colony System Parameter Investigation by Using Experimental Design and Analysis for Course Timetabling Problem, Second International Conference on Computer and Network Technology (ICCNT), IEEE CS, 2010; 467 – 471. 12. Wei X. Improvement and Implementation of Best-worst Ant Colony Algorithm, Research Journal of Applied Sciences, Engineering and Technology 2013; 5(21): 4971-4976. 13. Dorigo M., Gambardella L.M. Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem, IEEE Transactions on Evolutionary Computation, 1997; 1(1): 53-66. 14. TSP world tour problem which has already been solved to within 0.05% of the optimal solution http://www.math.uwaterloo.ca/tsp/world/ 15. Woeginger, G.J. Exact Algorithms for NP-Hard Problems: A Survey, Combinatorial Optimization – Eureka, You Shrink! Lecture Notes in Computer Science, 2003; 2570:185–207. 16. Sidner C.L. Engagement, Emotions, and Relationships: On Building Intelligent Agents, Emotions and Technology, 2016; 273–294 17. Lopez-Rodriguez I., Hernandez-Tejera M., Hernandez-Cabrera J. Regulation of the buyers’ distribution in management systems based on simultaneous auctions and intelligent agents, Expert Systems with Applications, 2015; 42(21): 8014-8026. 18. Lopez-Rodriguez I., Hernandez-Tejera M., Lopez A.L. Methods for the management of distributed electricity networks using software agents and market mechanisms: A survey, Electric Power Systems Research, 2016; 136:362–369. 19. Skarvelis-Kazakosa S., Papadopoulosb P., Undac I.G., Gormana T., Belaidia A., Zigana S. Multiple energy carrier optimization with intelligent agents, Applied Energy, 2016; 167:323–335. 20. Richards D., Bransky K. ForgetMeNot: What and how users expect intelligent virtual agents to recall and forget personal conversational content, International Journal of Human-Computer Studies, 2014; 72(5): 460–476. 21. Sbodio M.L., Martin D., Moulin C. Discovering Semantic Web services using SPARQL and intelligent agents, Web Semantics: Science, Services and Agents on the World Wide Web, User Interaction in Semantic Web research, 2010; 8(4): 310–328. 22. Satunin S., Babkin E. A multi-agent approach to Intelligent Transportation Systems modeling with combinatorial auctions, Expert Systems with Applications, 2014; 41(15): 6622-6633. 23. Lau R.Y.K. Towards a web services and intelligent agents-based negotiation system for B2B eCommerce, Electronic Commerce Research and Applications, 2007; 6(3): 260-273 24. Vázquez J.C.D., Martínez A.C., Gómez A., Varela B.A. Intelligent agents technology applied to tasks scheduling and communications management in a critical care telemonitoring system, Computers in Biology and Medicine, 2007; 37(6): 760-773 25. Doppstadt C., Koberstein A., Vigo D., The Hybrid Electric Vehicle – Traveling Salesman Problem, European Journal of Operational Research, In Press, 2016 26. Murray C.C., Chu A.G. The flying sidekick traveling salesman problem: Optimization of drone-assisted parcel delivery, Transportation Research Part C: Emerging Technologies, 2015; 54: 86-109. 27. Dorigo M. Optimization, Learning and Natural Algorithms. Ph.D. thesis, Dipartimento di Elettronica, Politecnico di Milano, Italy. 1992. 28. Chakravarti I.M., Laha R.G., Roy J. Handbook of Methods of Applied Statistics, Volume I, John Wiley and Sons, 1967; 392-394. 29. Anthony A., Jannett T.C. Measuring machine intelligence of an agent-based distributed sensor network system. In Advances and Innovations in Systems, Computing Sciences and Software Engineering, K. Elleithy, editor. Springer. 2007; p. 531-535 30. Hernández-Orallo J., Dowe D.L. Measuring universal intelligence: Towards an anytime intelligence test, Artificial Intelligence, 2010; 174(18):1508–1539. 31. Weng J. Task Muddiness, Intelligence Metrics, and the Necessity of Autonomous Mental Development, Minds and Machines, 2009; 19(1): 93-115. 32. Neisser U. Rising Scores on Intelligence Tests. American Scientist, 1997; 85: 440–447. 33. Bhalla US; Iyengar R. Emergent properties of networks of biological signaling pathways. Science, 1999; 283(5400): 381–7. 34. Trewavas A. Green plants as intelligent organisms. Trends in Plant Science, 2005; 10(9):413–419. 35. Bailey N.W., Fowler-Finn K.D., Rebar D., Rodriguez R.L. Green symphonies or wind in the willows? Testing acoustic communication in plants. Behavioral Ecology, 2013; 24(4):797–798. 36. Sharkey AJC. Robots, insects and swarm intelligence. Artificial Intelligence Review, 2006; 26(4):255–268. 37. Coren S. The Intelligence of Dogs: A Guide To The Thoughts, Emotions, And Inner Lives Of Our Canine Companions. New York: Bantam Books, 1995. 38. Hibbard B. Measuring Agent Intelligence via Hierarchies of Environments, Artificial General Intelligence, Lecture Notes in Computer Science, 2011; 6830: 303-308. 39.Winklerová Z. Maturity of the Particle Swarm as a Metric for Measuring the Collective Intelligence of the Swarm, Advances in Swarm Intelligence, Lecture Notes in Computer Science, 2013; 7928:40-54. 40. Mankiewicz R. The Story of Mathematics (Paperback ed.). Princeton, NJ: Princeton University Press. 2004. p.158. 41. Fisher Box, J. Guinness, Gosset, Fisher, and Small Samples. Statistical Science, 1987; 2 (1):45–52. 42. Mann, H.B.; Whitney, D.R. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. Annals of Mathematical Statistics, 1947; 18(1):50–60. 43. Fay M.P. Proschan M.A. Wilcoxon–Mann–Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Statistics Surveys, 2010; 4:1–39. 44. Kannan B., Parker L.E. Metrics for quantifying system performance in intelligent, fault-tolerant multi-robot teams, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007; 951-958 45. Schreiner K. Measuring IS: toward a US standard, IEEE Intelligent Systems and their Applications, 2000; 15(5):19-21. 46. Crisan G.C., Pintea, C.M., Palade V. Emergency Management Using Geographic Information Systems. Application to the first Romanian Traveling Salesman Problem Instance. Knowledge and Information Systems. An International Journal, 2016, 1-21