Distributed Processing System by History for Load Balancing

Copyright © IFAC Control in Natural Disasters, Tokyo, Japan, 1998

Distributed Processing System by History for Load Balancing Keizou YASUMATSU Yuji TAKEDA Hideki MURAKOSHI Noboru FUNAKUBO

Elettric S~-stcms S~'stcms Enginccring Dept. of Electric Tokyo :\Ictropolitan Institutc of Tcchnology _\sahigaoka. Hino. Hino, Tokyo 6-6 .\sahigaoka. Tok~'o 191 Japan

s~'stpm is distrihutpd distrihntpd procpssing s~-stf'1ll S~·stPIll h~b~' Histor~' Histor~' for Abstract: This proposal s~-stPm s~'stem is considered the load halancing load balancing. This s~-stem balancing and communication datahase of load st ates aYf'l"age to time of all hosts. and learned cost. History is a database ~Iap . Capsuled objects of class df'srribed dpsrrihed in object with Self-Organization Feature ~Iap. orien ted programming language are distrihuted. dist ribu ted. Remote Relllote Class Controller COil troller (RCC) (R CC ) oriented RC... are sent to the and RemMe Class (RC) are generated from the source class. RC... ser\'('r;; suggested by History. and. and, by linking RCCs. a main program can commuserwrs resnlt. the load was balanced and the cOllllllunication cOllllllllnication cost nicate with RCs. As a result. becaIlle low. Copyright © 19981FAC in executing became

network, Local area networks. Keywords: Distribution network. Object oriented programming prograIllming function. Processes. Ohject

1.

I:'\TRODl~ CTIO:'\ I:.\TRODl~CTKJ:\

Prohahilit~· Prohahilit~·

distribution

net\\'ork inclill(' inclill!" to a part of IlPtwork. on the net,,,ork network. And ser\'('r which is distributed the load ha.<; alif a serwr rpad~' had Illuch much load. load . the selTer seITer can rare execute expcn tp read~' Thpll the servl'r ser\"l'r might be do\\·n. dO\\'ll . it fa.<;t. Then

yeas . the net\\'ork makes remarkable In recent yeas. :"Iany hosts are connected. hut each host growth. :\Iany withont cooperation in general. works for itself without On network. load is not alwa~-s alwa~'s balanced. balanced, likf' likp that SOllle hosts works hard and the ot hers are t hat some idlp. If one seryer cooperatps with the others and idle. selTer cooperates Illany load, it is clear that the many seryers share the load. load of the ser\'('r is distributpd loarl serwr distributed and anrl the network works entirely. many entirel~'. So distributed processing on lllany focusahle than processing on one servers is more focusable seITer. :\Iany distrilmted systellls are selTer. rlistri]mted processing systems proposed, but there is no standard one. proposed.

distribntpd system which On the other hand. d.istributed proposed . But Bnt the considers the load balancing is proposed. system t hp load of seryers for systelll which is sampled the e\"pr~' request to distril>11te disrril>11te the process has many eYer~' net\\"Ork. traffic on the net\\·ork. Thpn in this pappr. the system considered the Then balancing and ("ollllllunication load halancing cOlllmunication cost is propospd. This s~'stPlll posed. s~-stPl1l distributes programs deorielIted programming language srrilwd in object oriellted histot·~· of load states and replies to (OOPL) h~' histOl'~network . A capsulpd capsuled cla.ss cla.<;s is one nnit hosts on network. unit distribntion. Because Becanse in cla.ss. cl a..;; s , the diffprence difference of distribution. ]wt\\'pen instiUlCP and pl1 blic instance pnhlic instancp is IlPt"'pen private pri\"ate installce

distributed system without considering the On distrihuterl load halancing. balancing. programmer must select SOllle some servers which are distributed. TIlt·refore. Tllt'refore. the load

33

rT

1

,

1

11~1 ~I

I

I I

Hi~lruy Hi~lru~·

OK",,'" DKmo'"

Ili3-!:$-I-e-1 i~ I

I

,

,I A iiPProcessor rocessorA

I

I

I

s:\'stem. t\\'O cla.:;ses cia..:;ses arf are needed net"dpd to to expxIn this systf'm. ecute program. Compiling a pro?;ram. program. fxecu execntahle fCU te pro?;ram. tablf eypr\, distributed distribu ted class are generated. generarp(l. At files for eyery Rpmote Cia"" Cla.'
Processor B

Processor C

I !

, Fig.l:

11('1

\l'ork \\'OI'k

dear. are fixfd. fixed. _-\nd _\nd it iT can clfar. their communications arf he sfnding sending just onf one mfssagf message to assign a.'isign T ht" rflarelalw t hf function s hecausf hecause mfthods methods for t hf hp dass tional functions class are Mf colleCTed. Thfrfforf. Thpreforp. thf the communication communicaTion cost 1wcollfetfd. lwcomes comfS low. neT\vork. stall of network npt\\'ork is so wasted wastpd and On nfT\mrk. almOST happen happpn hy oyer network accidfnts. a('('idents. it almost OYfr load. nftwork these prohlt"m. halancing is thf the To solw thfsf prohlem. load halaIlCing kp:-·. So. load statfs statps should 1w sampled to halkfY. Iw samplfd ance load. But load on nem'ork nt"twork lwconws hecomes higher anCf hos t samplfs samples load statf statp fwry ewry rfquiring requiring to disif host tribute. In this systfnl. system. load aYfragf awragp is samplfd sampled trihutf. time. It's callfd called Histor:-' does not COl11to timf. Histor~' History dOfs ('Ommunicatt"s with othfr other hosts in distrihuting and it municatfs samplps load awragfs awrages of tlw the otlwr ot Ilt"r hosts' host s' at e\'samplfs fYery long tfml. term. Thfrffore. Therefore. Histor~' Histor:-' ayoid to haw fry pxtrpmt" load. extrfmf Thpse informations arf are lfamfd lparned "'it \\'ithh SelfThfsf Fpature ~lap (SOFl\I). (SOFl\I ). which is one Organization Featurf npural nftwork. nptwork. By using nfural neural nftwork. nptwork. Hisof nfural storpd onf onp if tory can collect informations with storfd states changes rapidly. lwcausf because of lfaming. Iparning. load statfs neural nftwork. network . amount of calculation ordfr ordpr In nfural SOFl\I is less the othfr other nfnral nenral nN\mrks. nf't\\·orks. of SOFl\1 Ifss than thf Becausp its calculation hand band is narrow. Becausf result. load is balancfd balanced wit wi thou indining As rfsult. hou t inclining one host. And c0I11111unication cost hfcomes beCOI1lt"S to onf _-\nd communication lower. 10WfL

2.

B:,' COIlst!'llCTOr of RC, RC. some somp Ohjfcts objects are arp B~' calling constructor gpneratpd on tthhfe Sfrn'L "e r\'('!'. U 11co11strl1('(or callt"d gfnfratfd nconstructor is called thosp ohjt"CT" expcution. After Aftt"r fillfinto 1n'pak break those objf'ct" after execution. progra lll <'xfcution. (';(ecution. RCC requfsts rpquest s to ishing main program frpe tthp procpss of RC to the thp serwr. (Fi?;0) (FigS) free hf process

3. HISTORY .).1. S. l. Sdi

OTgnnZZ1.ny O Tga.nizU)'f FUI.t'l/.TP. Fla.f.'II.TP.

Mnp Mop

The ha.:;ic ha..:;ic st1'11ctnrf' stl'1lct1\l'P of rlIf rIl l" Sflf Splf Orgallizing F patnrp Or?;anizin?, Featnre ~Iap (S OFl\I ) is ShOWll sho\\'1I in Fig.G. The TIlt" neT\mrk nptwork con~lap (SOn\I) t\\'o la.wr" la:"t"r" called calkd input and output. The TIlt" sists of T\\'O \'eCTor x(t) x(t ) = [./1 (t )..r2(t) .. 1'2(t) ......r,,(t)]T ... .. .1'" (t)]T E IR" R" input yectm [.1'1 (t) connp('(ed to all output celb cplls ((111.112 11 1.112 ..... ..... 11"') is connected 11111) with tllf> difff'rfnt dijfpl'f'llt weight \\,pight Yfctms \'ectors Wj(t) W j( t ) \\'ith the

SYSTE~lSTRCCTURE SYSTE~ISTRCCTURE

Fig1 oyeryifw. History Histor~' dafmon Figl shows this systfm system own'iew. daemon the sflfction selection of SfITfrs selTers and communicamanages thf dapmon always alwa\'s mns tion with hosts. History daemon runs on each host. it has ha.<; databasf databasp of load statfs statps of all fach hosts. History dafmon daemon eXfcutfS t"xecutes supplifd supplipd OhjfctS ohjeCT s hosts. ont" prOCfSS. proct"ss. History Histor\, is leamed It''arned with SOF~I. SOF~l. as onf states of thf thp host on "'hich \\'hich History runs A load statfs arp samplfd sampled fyery eyery short tfnll term (minutes). (minutps ). and the are other hosts' ewr~' long tfrm hosts ' arf are done at ewr:-' term (days. (da\'s. state is wt"pk). \wfk). As communication sampling load statf not always. its cost is lo\\'er. lowfr. statps of all hosts. Histor:-' Rpfering to load states ReffIing Histor~' daeclient selects sejt"cts SfrVfrs. servers, These Thpsp SerYfrS spl'\'ers is semon on difnt lected stochastically. stocha.<;tically. It giws gives prior to SfITer serwr hayha\,lectfd ing lower load state to be selected. splpcted, Histor:-' Histor~' keeps load halancing balancing on network.

I. Ii

!i ,

!. !' I

!

Ii

I I '

I!

t

Remote Class Controller

~c~trol 9c-:trol -+---+~, -+----r-~,

Remote Class

~Ol I

Fi. !!. .·.!: Fi.!!..'2:

34

I

cOlIll'ol cOlIll'o!

Class

I

learned with SOFnl SOFnI at ewry short term (minhroadra..<;ted at every f'Yf'ry long term utes). and it is hroadcasted (days. week). wf'ek). Histor~' daemon daf'mon of hosts which rt>rf'updatf's load states for others. ceived it updatt>s \"ector x In learning with SOF1L the input wctor (0.0 S x S 1.0) is load state of the host. and the numher number of tht> thf' output cells rf'lls 11/I. is ;Y N which is tht> thf' numbf'r of short terms for OlW Thf' numbt>r one long tf'rm term.. The w'" = (WkO.Wk1.···.WkS-1). Yf'ctor is Wk weight vector (WkO' Wk1.···. WkS-1)' reprf'sents the load state at host numand Wb W/;i represents ber I.: at time timf' i . T.,. where \\'hen~ T" T., is time of short her term. W s. and load awrage average can Also. Wo is equal to W;\'. Iw Vahlf' of he shown in Figi. In this system. initial Yalut> mf'ans all hosts haw Wki is 0.0 (O.OV<). and this means ha.w time t for onf' tf'rm. no load. At learning at timt> ont> long term. w hirh is suffix of willner cell is 1.1- and W k is c which updated. updated .

C:UU.'i

ClJSS Conlmller Cunlrnllcr Remote (JJSS (RCO (ReC)

mui" pro[:Ttlm mu;" pro~rtlm fclllSl) le/ass)

RelTlole RCltlOIC Class Clas!'t (RC) (RC)

~ ue(:llrahl~ main pro~ram t.uC:lltahl~ pro[:rtIm

Fig.:3: rornpi I" Fig.;3:

[Il'jd [I/'jd t ). II.'j2(t) II'j2(t ). .... U' lL'j,,(t)]T j,,(t)f E R" (1 S j S m). At first. all Wj(O)'s art> arf' initialized to sonlt' somf' small valUf'S at random. Tht>n Thf'n an inputx(t) inputx(t ) is presented values nf'twork and hy dt>finition df'finition of eq.(l). \w find to the nt>twork a number numher c. which is a suffix of /I. 11. called raIled aa. winner winnf'r cell:

IIx(t) w,,(t)11 II x (t) - w,.(t)1I

Ilx(t) = min m~n IIx(t) J

Wj(t)1I

r;, r;,

,'1.:;. Sdat'ing Se[ating of Sert'I-r S f: rl'f( :J.:J. daf'mon selects a serwr which should Histor~' daemon be assigned when History daemon is called hy he rf'ff'rs History learned If'arned with RCC. Histor~' daemoll reft>rs SOFM and receiwd rereiwd from other hosts. and selects self'cts a..<; choosing prior to a Sf'rver haying storha..<;tirally stochasticall~' as st>rver having lower load state. At first. i = is giwn as a..<; tIlt' tlw element numw~. at C11lT~nt r11lT~nt time t. and the thf' load state of hf'r of Wk. her given as follows (Fig8): nlllllher I.. is giWll host nlllnher

(1)

\Ve also define the neighhorhood S 'Ye .Y,.(. around the cellu ,.(. that is shown in Fig.6. All W jj's 's arf' art> Illodimodifif'd fit>d by byeq.(2) eq.(2)::

r;.1 ;.1

Wj(t) +c11(t){X(t) - Wj(t)}

w;(t+ 1) = w;(f+

1

if j E S(.(t) .Y,.(t)

I k·~.

W~' (i+11 -T, = Wk'(i+11 T.,

Wki(

T') ., ' I .,'

t-

( .) + W k (.) I

(3)

otherwise S-l S-1

r

(2) wlwre 0 ratf' and should decrease df'creasf' with time. ltf'ratIteratstf'ps. tht> the network will ht> he spatially ing the abow steps. acrording to the statt> state of vector wrtor x. organized according

a..<; Pk JIk = ((11 - Id and 2: i=O Pi. the And. i\." = Li=O self'rted Sf'rYf'r numl)!>!' I.. is following condition: selected server num!wr

3.2. SOFM for History

'Yht>re \rhf'rf' n (0.0 S n S 1.0) is random numher. sern-!' is chosen rhosen at ratio of Pk' p~. / So. the a..<;signed i\.<;signed serwr Finally. the thf' address of selt>cted selerted serwr is rt>turned rf'turned to t.o the RCC.

.\'-1

2:i=O Pi Ji i Li=O

r

<" <

" "S S-l l

L.J i- O Pi L.Ji-O

(4)

r

r.

Histor~' daemon has ha..<; a datahase dataha..<;e of load states for History state of itsf'lf all host on network. The Tht> load statt> itself is

I I

IHi, ·,,_ 1 IH.......,-I

Client Clit'nt

...... ".,," I, I, I,

Remote Class Controller

c§~

.t,""

eques t brea

Sen'u'

I

I--!,--f-~I I ~reak ~~eak

ii

I

1 J

break

Object R<:

Fig.·1:

Fi.~ Fi .~ .. ..-" ->:

('X(,('11 t<. ('x('('ul<'

35

hr('"k hr('ak

II

Remote Remote Class

I

Output

,------------, r-------------------~

..",,=r- N c(t 1)

(tl
Fi.e;.G: I\o hor", " 's ' flod,,1

Fi,g.8: lo,,,l st al.(· at I

nE:-IOTE CLASS 4. fiE:'·IOTE Remot.e da.<;s. The TIll' class da.<;s Remote Class is generated from class. to RC ou executahlE' file. fil e. links t.o 011 sernr. sernl". hecomes executahle given ID lI1Imher. ulllllher. RCC All method in class cla.<;s is ginn has same method names as SOllrce da.%. class. and ha.<; a.<; t.he SOlllTe the source. But. But in the tlH' case it can he t.reated as a.<; t.he ca.<;e of referriug puhlic instances. instances . the methods returning returnin g referring needed . So all accesses for RC lllUSt them are needed. must he be implemeuted hy met hods. hods . implemented c:entral controller of distribution. distrihutiou. links RCC is c:ent.ral comlllunicates with History to main program. communicat.es daemon and controls RC on servers. RCC acmessages controlling districepts and supplies all messa/?;es executahle file hy compilhu lion. tion. RC hecomes execut.able ing. RC manages the t he somce source class assignf'd da.<;s on a.<;signed serwr. server which be a.<;signed server. The Thf' Sf'rVf'r whic:h should shonld \w assignf'd RC clif'ut. and the RC exef'xeis df'cidf'd decided hy History on dient. filf' is sent seut to the thf' server hy thf' RCC. Hiscutahle file by the receiYes the file .aun ruu the file as tory daemon receives .ann run a.<; \wt.ween RCC and RC a process. Communication Communicat.ion lH't.ween c:onnect.ed. and the RC is waiting for messages is connect.ed. from RCC. execu t.ing class. Fig9 shows execut.ing da.<;s. At first. main proeXf'cutf'd on dient. a.met.hI is called. gram is executed thf' length of call message messagf' and call c:all RCC sends the Illf'Ssagf' to RC on seITer. serw~r. And RCC waits for remessage t.o

tmn from RC. RC . The Th f' call message includes iucllldf's turn message .from mpthod mUllher uumber anel aud method aroffset address. ml'thod guments . Accepting Acceptin g it. RC on Sf'rYf'r guments. seITer executes fiuishill !!, the execution. RC Sf'IHls met.h 1. After finishin;!, senels RCC . The return retllrI! message messagf' inreturn Illf'ssage message to RCC. Yalnf'. and a lld reference rf'ff'rf'nce valnes. cllldes return value. cludes values.

3.

I:\IrLET\lE::"JTATIO~ A~D L\IrLE:vIE~TATIO~

DISCLI SSIO~S DISCLTSSIO~S

f'yaluation. (listrihut.ing (\istrihnting Spf'N\ objf'cts For the evaluation. speed of objects load states of all hosts an' discussf'd. and loael discussed. and aIld following t,,·o ca..,es cases are iU'f' compareel: compared: 1) applying to otlwl' otlwr hosts for their thf'ir load state. 1) and a.<;signing assigning t IH' ha\'ing lowest lowf'st load. hp host having 2 ) and using th!' thf' proposal History. 2) Histor~·.

I1n ....·\ :doJn........ lf 'h.."I'hll"\:1

1IJ.:lhtltlarj:UIIIl'''''-

""'", GJ ( ".-7 "--7

111 ll1l " ' ::

J

, .• .• vv •

Fig. l'a rll<'d Load Av('rag<' Fip;. T: T: LL"arru'd Lo"d t\ v"r"p;"

36

.

.

••••"" ....•. . .. ... ••• "'," , ·•••. . "

,

Yamai.N. P.f. 0.1(1994). df'sign Yamai.~. \\"akaba~·a"hi.S. \Yakaba~-a....hi.S. P.t n.l( 1994). A design Implf'mf'ntation of Loa Balan,ing Balancing 11f',hanism 1If'chanisIll and Implf'mentation Lt'\'f'l on 011 C'HX. Tmns.Dat Command LE>wl C::\IX. IEICE Trans.D1.J77.pp-!83--!92.

rf'plif's of liYing liying In 1). I). it is IH'f'df'd nf'f'ded to wait for thf' replif's hosts. TIH' em'ironmenr f'll\-irOnIllf'nt is that all hosts are arf' SC~ SF~'~ Tlw Sparc Station 10 and LL L\N7 is the thf' Ethernet Ethf'rIlf't of Spar, Basf'-T. COIllIllnnications. C so,kf't sockf't and 10 BaseT. For 'ommlmi'ations. TCP l IP are arf' USf'd. In this implementation. illlplf'Illentation . C++ TCPfIP langnagf' as OOPL is nSf'd df'snihf'd in langnage uSf'd.. RCG is df'scrihed languagf'. and output ontpnt RC and RCS are arf' also C++ language. Thf' Sf'rwr rf'cf'iw RCs RC s and ,an can lw exf'Xin C++. The serwr re,eiw rnnning f'cn te t heir hf'ir direct dirf'ct ly hf'causf' hostss are arf' running be,ause all host ecn sallle operating opf'rating Systf'lll. saIlle system. f'yalllating. the program. which thf' For eYaluating. \\'hi,h finds the key nsed nSf'd to encode f'ncode by giying the thf' string and the thf' kf'y string en,oded f'ncodf'd it with one onf' key. kf'Y. is exe,nted f'xf'C1ltf'd in this RCC' . and ;;omf' ohem·ironnH'nt. enYironment. This has one RCC'. some objects are arf' constr11ctf'd rf'snlr. ,onstructed from the RCC. As a resnlt. 2) fa.o:;tf'r than 1) 1wcansf' nf'f'df'd to 2) was fa..:;ter becanse it is not lleened gf'tting the thf' load statf'. wait for all hosts for getting state. Also. statf'S while whilf' eXf',nting eXf'cnting the program are arf' all load states halan,ed. halancf'd.

CO:-JCLCSIO:,( 6. CO::\CLCSIO\, distrihnrf'd proThis proposal I1H'thodology methodology is disrrihnted load balan,ing balancing and systf'm ,onsidered considf'rf'd ttIlt> he loan cessing system communication ,ost. cost. For loan load balan'ing. balancing . History ,oll1mnni,ation Hisror~' has a database daraba..,f' for di"rribllis proposed. Histor~' di"triblltlH' data dat a is load states statf'» awrages awragf'S learned If'anH'd tion. and the Ff'atnrf' :'IIap. d,wsn 't ",,,it 'ithh Self Sf'lf Organizing Feature :\lap. And it d,wsn't need nef'd to sample samplf' the thf' load states statf'S of othf'rs. ,ommncommnni,ation moment is not ,harged. nication cost in IllOmf'nt chilrgf'd . B~' usdf'snilwd in OOPL as nnit capsulf'd class described ing capsuled unit of commnnication ,ost cost in f'xe,nting f'xf'cnting is distribntion. ,ommuni,ation low. implf'mf'ntation. it is known By implementation. B" knO\\'l1 proposal HisBur History !w\Omes bf'COlllf'S larger largf'r tory balances load. But sizf' of network. nf'twork. and the thf' data \\'as \\"11,S proportional to size needf'd to lw comprf'ssf'd on large largf' network nf'twork in the thf' be ,ompressed needed ure. In implf'mf'n tation t ation the t hf' program is dedf'fu tr me. III this implemen Blit by nsing JAYA. this system systf'm scri lwd in C++. Bm using J:\YA. scribed will not depend on platform.

REFERENCES REFEREl\CES I~itaori.I\:. 1\1 urakoshi.H. and Funaknbo.?\'. Funakubo.}\". (1990). (19% ). I-:itaori.I-:. ~lurakoshi.H. l\f'''' Approa,h Approach to Solw the Trawling Sales Salf's A l\e\\' Prohlf'Ill by hy Csing L~s ing the thf' Improwd Kohonen's hohonf'n's SelfSf'lfProblem Organizing Feature Featurf' :'lap. l\Iap. Proc.IECON·95. PToc.IECON·95.

Eohonf'n.T.(1993 SelJ-OTI)n.nizinl) a.nd A8socinAssocia.Eohonen.T. (1993)) Sel/-Orgnnizing t· i l'l'. M pmoTy. SpringerSpringf'r- Verlag.Berlin. Vf'rlag.Bf'rlin. t-iN'. Mp.m-ory. Stf'wns.W.R. (1990). (1990 ). UNIX NETWORK PROStewns.W.R. Englf'wood Cliffs. Nf'w GRAMMING. Englewood New Jf'rsf'Y. Jf'rsf'y. Tanf'nhaum.A.S. ( 1992). Modp.Tn ModpTn Opp.mting Oppmtinl) SysTanenbaum.A.S.(1992). tp-m. Englf'wood Cliffs. ::\ f'\\' Jersf'Y. J f'rsf'y. tp.m. Englewood :\"f'W

37