A study on the convergence of genetic algorithms

Computers ind. Engng Vol. 33, Nos 3--4, pp. 581-588, 1997 © 1997 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0360-8352/97 $17.00 + 0.00

Pergamon PII: S0360-8352(97)00198-8

A S t u d y on the C o n v e r g e n c e o f G e n e t i c B.M. Kim

Algorithms

Y.B. Kim

Dept. o f Industrial

C.H. Oh

The Research i n s t i t u t e o f

Engineering,

Industrial Technology

University. of UI-san

Myon9 Ji Univ.

Dept of Industrial Engineering, The College of Su-won

Abstract This paper extends genetic algorithms to achieve f a s t solutions to d i f f i c u l t To accomplish this,

we present empirical

and the functionized model of mutation

problem.

r e s u l t s on the terminated condition by bias rate

in genetic algerithms.

The terminated

condition by bias enable to reducing computation tima(CPU time) according to l imitted and pre-astimated number o f generations. The functionized model o f mutation operator reducing computation time and improving sol ut i on should be accomplished by applying quite

low mutation rate on the continuing generation with remaining 95 percentage o f

bias.

© 1997 E l s e v i e r S c i e n c e L t d

Keywords : Bias, Generation, Mutation, Population

I. A

Introduction successful

dynamic

the

effect

of

environments

adaptive solutions. a optimal dimensions, uncertain fields.

system

usual ly

It

expected

number o f

in case o f pseudo-boolean

under

the

search of s o l u t i o n

the

function

to get

I~ck[2]

1/l accelarated

rate

demands

is so d i f f i c u l t

schemeta.

su9gested mutation which

the

of

evaluation

function

is

mul t imodal.

s o l u t i o n using noise, m u l t i f u l a various reaction which are

Meanwhile, genetic algorithma comes to an end

in the various related a p p l i c a t i o n

generation upon convergence stage o f s o l u t i o n

On these f i e l d s ,

a successful

when

result

it

arrives

at

the

predetermined

or i f the increasing range o f f i t n e s s function

can be obtained by using the a p p l i c a t i o n o f

is

genetic algorithms for the global optimum. But

terminations

t h i s a p p l i c a t i o n o f genetic algorithms to have

spending massive computation time and memory

a 91obal

optimum may be d i f f e r e n t i a t e d

f i t n e s s function, rate,

initial

crossover

rate.

less

by

according

population, mutation

Moreover,

The r e s u l t s

are

also

than

s<:~te

I imi ts(~:) [3] [7].

to

the

number

the operation

is

on

of

increase on the some level.

Such

above

or

degree of

parameters

have

convergence.

been

used

of

generation.

continuous while

the value of f i t n e s s function increases f a s t l y

changed depending on the number o f generation the evolution

This

techinque have the some problem

as

the

increase

initial for

stage, fitness

slow

or

immaterial

Such a rate o f

function

becomes too

various value under the given environments in

small on the f a l l i n g below the some level.

the

Hence, the total s o l u t i o n time may be reduced

genetic

algorithms.

guaranteed that optimal.

It

would

be

not

these parameters are always

and

Many problems such as CPU time and

the

quality

preserved well

space o f storage would be influenced by the

of

solution

may be

also

i f we terminate the algorithm

on the proper time which the increasing time

parameters. Eventhough the adaptive parameters

rate o f the f i t n e s s function value is too slow

were setted up to the given circumstances,

or unnecessary.

In t h i s aspect termination o f

condition

computation

proper

combination

on

the

parameters

a is

required to get more improved solution. For

instance

Goldber9[5]

researched

for

reflecting

the

c h a r a c t e r i s t i c s o f parameter is demanded. This the

technique o f computation f o r the r e l a t i o n s h i p

study suggests the terminated condition o f algorithm reducing the computation time which

between population size and computation time

applies terminated condition by using bias in

used to convergence o f population as applying

the

581

solving

the

problem

as

a

genetic

582

Proceedings of 1996 ICC&IC algorithms and p a r t i c u l a r l y

in the case o f / <~

The terminated conditions applied to genetic

n, methods e l e v a t i n g the q u a l i t y f o r s o l u t i o n as improving the f i t n e s s function using the

algorithms

character ist i cs o f murat ion operator.

the

thus

far

is

that

continuously executed u n t i l determined

algorithm

is

arrived

at

it's

maximum generation

or

the

increasement size o f f i t n e s s function is less 2. The c h a r a c t e r i s t i c s

o f p a r m m t e r s using

by bias

then some l imits(~). condithion

have

However, this terminated

an

important

effect

upon

population size and mutation operator. This

chapter

9ives

characteristic

priority

to

grasp

a

point o f genetic system using

To

search

the

solution

wi thin

I imi ted

genration might be e f f e c t e d since

the small

the l imitted generation s u i t i n g the subject o f

popu lat ion

problem. As grasping a c h a r a c t e r i s t i c s for the

smaller generation then the large population.

various

parameters

analysis

reducement

of

such

a

system,

comparat ivel y

converged

the

Naximum generation arranged by population size or the terminated condition by the increasing

for

real

size o f f i t n e s s function gives r i s e to trouble

cost which

to CPU time and storage space because o f the

pre-estimate the genetic algorithms

the number o f generation regards as a cost.

r e s t r i c t i o n by c ~ u t a t i o n

Bias

the rate o f mutation operator

distributes

variously

upon

time and

to computation

problem is enable to reduce total

is

between

50/, and

time. Meanwhile, i f is

large,

the

100/. as the s t r i n g measures a rate of gene

quality

within population.

improved but the more number o f generation is

if all

Since such a bias is 100"/.

s t r i n g s in population

all

the some

of

solution

on

initial

stage

is

increased,

the more i t

impedes in convergence

level on each loci. a solution is not improved

st at e

population

because

any more as a l l s t r i n g s are coincident.

operator at the convergence stage of s o l u t i o n

Let's

consider

loci

is

are

bit

a

on

each

chromosome that the population size is 20,

of

the

mutation

i n t e r f e r e s with being a gene o f string.

And

if

therefore, we are enable to ear l y confirm the

15 units have the value o f b i t i and the odd 5

approached st at e to optimal s o l u t i o n according

units have the value o f b i t

to apply the terminated condition by bias.

which loci

is 9,

0 on chromosome

this bias becomes 75Y., and

Now we apply the method by bias as follows :

t h i s bias applies the larger one out o f the rate of

0 or

1. The s t r i n g

bias

to use to

terminated condition of s o l u t i o n in t h i s study

Aoplying Stage Stage 1. Using the applying stage for e x i s t i n g

is as following ; i f each s t r i n g bias is the mean value o f I units o f b i t biases. From such a bit

bias,

using s t r i n g

bias

genetic algorithms, Stage 2.

In case that over,

to grasp the

not

convergence state o f the whole population, and the

a I 9or i thm,

convergence Operator

rate

is

of

is

and

if

increasing one percent or more even a f t e r

o p e r a t i i n g from n time to 2n times.

genetic

cont i nuous

95"/. and

the algorithm

then the present bias,

we make i t the termination condition. Applying

the bias

terminating

At

these above the method f o r

operating

from

properly reduced according to the subject o f

the

present

generation, gradually

generation

increasenent

to

rate

and on some level

gets

the

next

increases

times

for

termination,

converges to 100/. bias while getting connected

algorithm

could

be

problem.

gradually

slow or immaterial, Hence, each chromosomes of

3. Practical experiment

population which has the bias above 95~ are very s i m i l a r

and converged to

the s p e c i f i e d

string.

The purpose of t h i s experiment is to grasp the problem point for the reduction o f computation

After computing the measurement o f convergence

time,

rate l i k e as bias, we should determine whether

quality

to make a pre-estimation and to improve

i t is the termination o f algorithm or not. And

condition.

i t continuously comes into operation by stages

Pentium

i f algorithm is not terminated.

Vet 2.2.3.

of

solution

using

The using computer

the

terrniated

is

IB4 PC586

100 N-Iz, the language is Nathematica The subject o f experiment

is

the

Proceedings of 1996 ICC&IC unconstraint

condition

quadratic

and

the

fucnt ion,

problem

of

discontinuity,

population.

In such case, the i n c r e ~ t

the

convergence

function f,

this

state

of

geno

the

-5.12~x~, xz~5.12

increesement

of

population

to

premature convergence makes the This stage o f experiment suggests, applied

of

parameter

from chapter

keep the q u a l i t y

2,

of

the

for

is understood to recluse the

poss i b i I i ty for premature convergence. Hence,

f(xl, xz) = Integer(x, z + Xzz)

character i st i cs

of

population size has an e f f e c t to considerably loose

unimClelity, nonconvexity as fol lowing :

583

selecting

us i n8

B i as

technique

that

s o l u t i o n on time point

avoid

convergence

rate o f sol ut i on slow. To confirm such an above assumption, Figure 2 appreared mutation

rate

P=--O,01,

population

size 30, 50, 100 applied algorithm performance

above rate for 95 percent a f t e r reviewing and

for function f.

comparing

each

o f n=30 as c~mpared with population size n=50,

generation

and

individual solve

genes

a

for

problera

each

cause

to

reduc i n9 c ~ p u t a t i on time.

n=lO00

convergence

limitted is,

3. I Influences by population size

And we can f i n d out,

rection

rate

could

9ene

within

is remarkably reduced.

the problem for

9erie

of

in case

be

That

premature convergene o f

effectively

reduced

as

increasing the population size. One of

the

genetic

algorithms

common problems is

in

performing

how many influences

algorithm is given to population size. o f small population size,

it

In case

is very easy to

In

the

situation

al lowed,

the

various

population size to maximum value f" and mean calue

j

of f i t n e s s function to measure the

be prematurely converged cause occurrence o f

fitness of

chromosome is

shows the maximum f i t n e s s f o r function f based

search large

the

too deminent or

solution

population

space.

size,

recessive

And

in

in

case o f

some troubles

are

on

function f was applied.

population

size.

Convergence

Figure 3 rate

of

algorithm is able to be quickly got from n=30

e x i s t e d to perform the algorithm cause to a l l

as compared with population size n=50, n=iO0

search

but the r e s u l t o f premature convergence which

chromosome on

every

searching the s o l u t i o n space.

generation

generation to be used limitted, are be

some problems

is

not

optimal

of

convergene population

to

established

the is

convergent

adaptively

course required

to

state

is

appeared.

This

presents that a better algorithm even though

consequently the population size to

accurred

solution,

in

I f the number o f

rate

of

so I u t i on

is slower

than small

for

Iarge

population.

Meenwhile, Figure 4 shows average f i t n e s s for

perform genetic algorithms e f f e c t i v e l y to improve the computation time for computer and

function f based upon population size.

the q u a l i t y o f solution.

3.2 The effectiveness for mutation rate

Fortunately, most o f

the experiment r e s u l t s are presented that 30 units

population

problem,

but

size

this

established

suit

case

initial

the is

subject

influenced

population.

of

Mutation operator

by

decisive

Goldberg

recently proceeds an experiment using smaller population suggests an establishment the

simulation

population

about

the

to e f f e c t i v e l y

size

1,

if

adaptive

operate algorithm,

not to a f f e c t the r e l i a b i l i t y On Figure

of

through

the number of

also one o f [11].

Mutation operator

one b i t

the

14=35. Most

of

loss

population

means

for

convergence

to

and

apply

in previous research

and

effects

however, of

to

to get

in recent researches

experiment

appl i cat i on

of

about

genetic

one

mutation

size

operator for every one thousand b i t s has been

this

is about

useless to improve the q u a l i t y o f sol ut i on in

in

a whole

case o f smaller Population size than length o f

population

gene

operator

for every one thousand b i t s

simulation

applying

genetic

to be

algorithms

about genetic algorithms applied mutation

generation of I00 percents convergence rate by in

the

genetic

random walk method by the space o f s t r i n g [ I ]

algorithms,

n=50 and mutation rate PaFO.OI,

is genetic operator

applying

better e f f e c t ,

o f solution.

we consider

in

state

of

string[5][7]. apply

rate

Particularly. of

mutation

in the e f f e c t

to

operator,

the

584

Proceedings of 1996 ICC&IC discontinuous case for the subject o f problem

according as generation is proceeded.

has

Figure 7 shows by a diagram for convergence

been

proved

efficient

than

to

be

the

more

relatively

continuous

case

in

effectiveness of s o l u t i o n for average f i t n e s s

improving q u a l i t y o f s o l u t i o n [4].

degree

Losed gone can be restored as increasing the

mutation operator when genetic algorithms

rate o f mutation operator since the degree of

appl led

based on to

function

convergence is reduced according as rate for

increasement

mutation operator is increased.

operator

But,

when the rate o f mutation operator

increased,

is

apparently suggested research for

Table subject

gene by

premature

improved

as

existed,

the

reduced

convergence can be only

increasing

rate

of

mutation

random

rate

And

for

also

rate

for

is the

genetic

l i k e mutation reduce a performeance

degree o f genetic

algorithm

acarcely

of

f.

of

algorithms could be found

out.

being influence to the performance, degree o f is

increasement

1 presented the of

problem

to

result find

to

solve

the

the computation

time and precision of sol ut i on for terminated condition by bias. By experiment data(30 uni t s

operator. And an increasement o f the rate for

for population size 10, 50, 100),

mutation operator increases the number o f gene

time o f average 9W. was saved as compared with

which

samplin9

can

be

taken,

this

no-9ood influence upon algorithm

exerts

performance

degree. For

generations which

of is

the

Figure 5 shows the e f f e c t

to

=0.001,

0.01,

0.03,

0.1.

solution

quality

typical

the present,

condition o f algorithm,

instance,

apply population size n=50, mutation rate Pe

of

genetic terminated

and i t was maintained 92"/, from

the

problem

knowing optimal solution.

And Fi9ure 5 shows

average convergence rate o f gene based on rate of

1000

algorithms

computation

the various mutation operator.

more rate o f mutation operator the more convergence rate o f

4. E f f e c t i v e convergence o f s o l u t i o n

Since the

is

increased.

9ene get

4. I Improvement o f s o l u t i o n based on change

low,

of bias

performance degree for algorithm is given some troublers.

In case that premature convergence

One o f

for

is

solved

occurred in searchin9 s o l u t i o n o f the subject

the rate o f mutation

o f problem based on genetic algorithms is that

9ene

problems as operator but

accurred, increasing

it

could

be

convergence e f f e c t i v e n e s s based

the

problem points

the p o s s i b i l i t y

of

appeared

on

surely considered.

premature

convergence

Figure

6

is

showing

the

performance

effectiveness

degree

of

for

genetic

performin9 is

algorithm. variously

generation.

Especially,

quality

decreased is the case of />n,

is increased.

solutions

be

verified

maximum f i t n e s s

algorithm

is

multation

operation

rate

for

defree

of

the best one when the rate o f is

mutation

effectiveness fitness

i t can't

to

function

1/[.

Increasement o f

operator

improve upon

have

the

fitness

initial

stage

the

event

This

occurred

that

the

smaller than s t r i n 9 no

solution that

is

is, most

would be prematurely converged in

technique

to

size

of

length.

population

is

However, there is

confirm

in

advance

the

premature convergence state as the method o f

an

the

of

s o l u t i o n can be get,

when

be

depending on value o f p r o b a b i l i t y or number o f

algorithms when the rate o f mutation operator In this Figure 6, as B~ck presented,

could

premature convergence is

on performance degree o f algorithm should be

maximum

which

present

genetic

convergence state

algorithm.

Improved

no entering a premature

as making

population

size

algorithm is performed, but t h i s is considered

large,

to

much cause by r e s t r i c t i o n of computation time

be

due

improvement

to

rather

of

solution

cause

troubles

after

that

for the

but the expected effectiveness is not

and problem of

memory. And

the

change

of

generation is proceeded on a c e r t a i n degree.

crossover rate can be considered, however, new

In case that mutation operator o f low rate was

chromosomes consisting o f next generation are

applied,

very herd to be expected due to the occurrance

premature

f i t n e s s o f f i t n e s s function cause by convergence

of

gene

is

reduced

of s i m i l a r

chromosomes based on increasemant

Proceedings of 1996 ICC&IC for

the

number of

generation

in

genetic

q u a t i t y o f solution.

algorithms.

P= =

The change of mutation rate play a decisive role

for

performance

generation

is

predetermined

of

proceeded. mutation

is

of

as

the

Since

rate

performance o f algorithm value u n t i l

algorithm

the

under

the

is applied constant

the termination for algorithm,

no

use

in

confirming

it

premature

convergence. Hence a new technique to improve the

quality

of

solution,

not

state

entering

premature

convergence

algorithm.

This section presents the q u a l i t y

on

performing

o f s o l u t i o n based on increasing the mutation rate

on

stage

eighty

percent

[examPle;

population size 10, raise mutation rate up to 0.3]

that

the

possibility

for

new

gene

occurrence is decreased. To

identify

Figure

10 showed the for

from Figure

technique

premature

to

8

to

improve

convergence

as

r a i s i n g up the rate o f mutation operation on stage o f bias 80"/. in the population size n=10 which

has

high

convergence. case

of

problem, solution solution, 91% as

the

possibility

of

premature

From the experiment

the

particularly,

small

population

results, size

a

n=10,

on the discontinuous subject of

From this,

a and

~ are not exceeded 0.1,

which is mutation rate l i k e as tentave search method in the i n i t i a l

stage for algorithm, and

we apply the rate in so small that i t couldn't be interrupted to improve sol ut i on at a point of

time

maintained

as

bias

above 95% on

9enerat ion process. Since

the

st at e

above 95% acts

crossover

operation a

in

rate

the

of

bias

mutation

operator, and from a point of time maintained as bias above 95%,

it

maintain

of

the

rate

may be e f f i c i e n t mutation

to

operation

exerting a influence to q u a l i t y o f s o l u t i o n in so small.

An applying the case o f n=lO which

p o s s i b i l i t y of premature convergence is high,

correctness

degree

for

total

is also high and crossover rate 0.6,

and the

experiment using terminated condition that

is

previously suggested was presented by draw on result

of

experiment to apply a mutation rate for

Figure

]l

and

the

function form,

Figure

12.

The

the q u a l i t y o f s o l u t i o n doesn't

have some change as compared with the present techniques but computation time

is come down

to 94%. This appears an effectiveness to save the computation time.

is appeared to about 95% of optimal and computation time compared

with

the

is reduced to

typical

In

4.2 An appl ication of functionized mutation operator for iBproving solution Parameter

exerting

a

direct

influence

to

algorithm performance is a population size and the rate of mutation operator. Especially the rate o f operator act as a d i r e c t improve

solution

perform

the

in

final

algorithm.

factor

stage

to

when we

Accordingly,

this

section suggests that the large mutation rate is applied to diminish the stage o f premature convergence in the i n i t i a l

stage o f algorithm

to improve the q u a l i t y of solution,

and that

the small mutation rate is applied to improve in the convergent stage for optimal

s o l u t i o n on generation process. Let us simply express the rate o f mutation operator o f

5. Conclusion

genetic

algor i thins techniques.

solution

a exp(-~tN), a < 0.1, /t ~ 0.1

n=50 which the improvement degree of algorithm

above case,

possibility

585

functionize

form to

improve

the

this

paper,

genetic

a

terminated

a 19or i tl'lns

time(CPU

tim)

condition

reducing

of

computaion

by using bias

to

given problem has been proposed.

solve

the

And also,

functionized mutation operator was u t i l i z e d to improve

fitness

method

for

function,

iraDroving

as a result,

premature

has

the been

studied. The

results

for

various

numerical

experimentations to apply terminated condition by bias

to

the discontinuous

been proved to total

9et

an optimal

function solution

have in

experimentation even though there is a

some d i f f e r e n c e according as population rate, mutation rate and crossover operator rate. Accordingly,

this study set up the computation

time within

number o f generation enabling to

p e r d i c t i o n and c h a r a c t e r i s t i c for parameter to improve the q u a l i t y of sol ut i on as using the similarity

relationship

a~ong genes.

It

was

Proceedings of 1996 ICC&IC

586 confirmed this

through

technique,

the

experimentation

as compared with

genetic

algorithms,

quality

of

didn't

solution

that

the typical

come down

and

enabled

to

the

Proceedings

of

the

fifth

conference on Genetic

international

Algorithm,

Morgan

Kaufmann Publishers, Sen Mateo, CA, 1993.

make

computation time reduced above 91~. iJ=l

Additionally, mutation

we obtian that the functionized

rate

should

be

decreased

during

convergence.

The assumptions made for

explanation

regarding

the

this

frequency

of

absorption end the losses due to mutation rate can in p r i n c i p l e This

be v e r i f i e d experimentally.

approach to determine

mutation

rate

experimental

the

functionized

seems fundamental

verification

of

and

an

the assumptions Figure 1. Bias o f f i t n e s s function f for

used here is planned.

n=5O, P,=O,O. 001, O. 01, O. 05, O. 1 References

L~

1. Yong-Beom

Kim,

convergence

of

"A

Study

Genetic

on

the

Algorithms',

PH.D | =;

thesis, University of Myeong-Ji,1996. 2. B&ck,

T.,

"The

Mutation

Rate,

Self-Adaptation Algorithms',

,o

Interaction

of

Selection, within

a

In Reinhard,

and Genetic

M. and Bernard,

M., pp 85 - 94,1992.

3. De

Jon9

K.

A.,

Behaviour o f Systems",

"An

Analysis

a Class o f

Ph.D.

y

of

the

Figure 2.

University

of

E.,"Genetic

Algorithms"

in

fitness

function

]

n = 30,50,100

Genetic Adaptive

thesis,

Bias of

Michgan, 1975. ~o

4. Goldber9,

D.

Search,

Optimization

and

i '°

Learning, Addison-Wesiey, Reading, MA, 1989. 5. Hesser,

J.,

Optimal

l~rmner,

Mutation

Algorithms',

In

R.,

"Towards

Probability Schwefel,

ts

Machine

for

an

Genetic

H.

and M~nner,

"Genetic

Algorithms

20

R., pp 23 - 32,1990. 6. Michalewicz,

Z.,

+Data Structures=EvolutionPrograms ",

Figure 3. Evaluate o f maximum f i t n e s s

Spring

function ] for

- Verlag, 1993. 7. Nicholas, Study

J.

R.,

Felicity,

in Set Recombination',

A.

W.,

n=30,50,100

"A

In Stephanie,

F., pp 23 -30,1993. 8. Reinhard, Problem of

the

M.,

Bernard,

Solving

from

Second

M.,'Parallel

Conference

on

|

Problem Solving from Nature',

1st Workshop,

PPSN1, Springer-Verlag, 1990. F.,'Genetic

iJ

Parall el

Problem Solving , North-Holland, 1992. 9. Scheefel, H., Minner, R.,"Parallel

lO. Stephanie,

o~

Nature',Proceedings

-LLo.

'~

'°°

Figure 4. Evaluate of average f i t n e s s AI9orithms',

function ] for

n=30,50,100

for

Proceedings of 1996 ICC&IC

II'~l,~/t J

587

sam

I

ml

Ii

,! j

u

Jol

==l

SO

~

ISO

~

Figure 5. Bias of fitness function J for

P.=O.O01,O.Ol,O.03,0.1

Figure function

9.

Evaluate

of

maxirnum fitness

J by mutation rate at the state of

bias 80/, for

n=]O

*= ¢11

M

ul

I"

I"

Jo =l

~s

=*

=o

Figure 6. Evaluate of maximum fitness function ./ for

P,=O.O01,O.Ol,O.03.0.1

Figure function

10.

Evaluate

of

average

fitness

j by mutation rate at the state of

bias 80"/. for

n=lO tD

sB

l" ==

Figure 7. Evaluate of average fitness function J for

P.=0.001,0.01,0.03,0.1

Figure function

11.

Evaluate

of

maximum fitness

] by functional mutation rate

for

n=lO

if/

,=

i"

.[

Figure 8. Bias of J by mutation rate at the state of bias 80"/. for

n--lO

Figure function

n = 50

12.

Evaluate

of

maximum fitness

] by functional mutation rate for

Proceedings of 19961CC&lC

588

Table 1. Comparative results between t r a d i t i o n a l G.A, method of bias, method of Bias 80~ by mutation varity, and functionized mutation model Terminate method Pop_size Terminae generation Quality of solution(%) Computing time(sec)

~ition~ 10

~

G.A

Method of Bias

Method of Bias 80% by mutation varity

1~

10

50

100

10

50

100

10

56

100

60

108

25

68

123

17

49

96

97.0

97.5

~.0

~.3

~.3

1~

1000

1~

21

~.0

2.8

97.4

82.0

96.8

97.7

~.7

9101

15.0

234.6

932.6

18.9

5~.1

~.8

Functional mutation Model

267.7 1067.1

12.2

156.6 759.0