Copyright © IFAC Control of Power Systems and Power Plants, Beijing, China, 1997
MUIUAL
BACKUP BEIWEEN SCADA SYSTEM ELECI'RIC POWER COMPANY
T.Tsuruma
T.Tatekouji
Y.Kudo
IN
HOKKAIDO
T.Inoue
Toshiba Corp.
Hokkaido Electric Power Co.
Abstract : The Sapporo NlShi Regional Control Center ofHokkaido Flectric Power Co. was put in operation in March 1 The system is designed to provide mutual b!K:kup between this Center and the Sapporo Regional Control Center, much is to be repla:ed The paper outlines the background for the bactrup operation between these two systems, and describes the mutual b!K:kup system, system configuration and fimctiom. Copyright © 1998IFAC
m.
Keywords: Power system controL Bock-up systems, TelecontroL Disttiblned computer control systems, Supervisay control
1.
suhitations were controlled by these systems. Accordingly, the control area foc Sapporo ha<; been divided into two, \Wich are controlled by two SCADA systeIm on equal terms, one of \o\i!ich is controlled by the Sapporo N1Shi Regional Control Center was built in ploce of the SunagaWcl Control Center.
IN1RODUCTION
Hokkaido Bectric Power Co. supplies electric power to the noohem area of Japan. The total generating capocity of its power system is 5430MW. Five SCADA systeIm perform power system control opel3Iiom in Hokkaido Bectric Power Co. as shown in Fig. 1. As electric power systems were becoming in::remingly complex along with the rapid growth in power demaOOs, these SCADA systems were no longer 00equate to ~ the requirerrents. Hokkaido Bectric Power Co. ha<; been replaEg its existing systeIm sin:e 1994 in view of the operntionallimits of these systems. Of the five regional control centers, the Sappcro Regional Control Center that controls Sapporo was a key system of Hokkaido Bectric Power Co. became the load was coo:entrated in Sapporo and was expected to grow, and the impact of a blockout due to a failure in the Sapporo Regional Control Center would be serious. There are 117 substatiom in this area at present and the number is likely to increase. The Sapporo Regional Control Center controlled them in coocert with the SunagaWcl Control Center which was uOOer the command of it And it would result in an excessively large burden for operators, if all
Fig. 1. Areas controlled by SCADA systeIm in Hokkaido Flectric Power Co.
515
Since Sapporo is the most important city in Hokkaido, one half of Sapporo would be seri~ly affected by an interruption in a SCADA system even though Sapporo is divided into two systems that are supervised and controlled separately. Therefore, it \WS necessary to redoce the influence of a system interruption. As a series of big ear1lxjuakes have recently taken ploce in Hokkaido that could have damaged the SCADA system, treaSUreS \\ere taken to maintain normal supelVisoIy and control of power systems in the event of a failure in a SCADA system by using the 0Iher SCADA system.
system without moving from one system to the 0Iher. A lOO-Mbps IDMA (T"nre Division Multiple A!x:ess) Ring l.AN connects the two SCADA systems. Sapporo Nishi SCADA System
2. CONCEPT OFMUIUAL BACKUP The concept of dividing Sapporo area into two and supervising and controlling them by two SCADA systems and opemting a dual reduOOancy system with mutual backup in the event of an intem.Jption is as follows: 1) Backup ofwhole system 2) Backup ofsupervisory and controlfonction The deIails of these are described below. Figure 2 shows how Sappcro area is divided into two and supervisory and control of them by two SCADA systems during normal operation.
Fig. 2. Operntion dming normal operation Sapporo SCADA System
1) Backup ofwhole system
This is applicable to a case where the building is damaged by a fire or the system is severely destroyed by an eartlxJuake. The normal SCADA system perfonns supervisOI:y and control fi.mctions for the whole of Sapporo in soch a case. The system is shown in detail in Fig. 3.
Fig. 3. Backup of whole system As a SCADA system cannot qJel3te in the above case, the substations in the area of this SCADA system are supervised and controlled using the tele-rontrol nmter unit for backup installed on another system \Wich is operating nonnaIly. The ~ of damaged SCADA system moves to the normal SCADA system and uses the opemting console and Mimic Board provided for simulation. 2) Backup ofsupervisory and controlfonction This is applicable to a case where the human inte:rfiw:e equiprm:lt of the client is nonnal, but the server computer that perfonns online q>eration is ~ The system is shown in detail in Fig. 4.
Fig. 4.Backup ofsupelVisoIy and control fi.n:tion In the above case, mutual backup is established by intercormection of the system of the normally operating SCADA system to the client of the partner SCADA system upon receiving a trigger to start the backup q>erntion. The q:Jel3tOr can thus supervise and control using his own control console through the server of the partner SCADA
This corx:ept assures the reliability foc supervisay and
C()ntrol of the Sapporo area equivalent to that of qwdruple system. This \WS anained by the client-server system configuration with an qleIl architecIure. And when substalions are added to the Sapporo area, the Sapporo area
516
Mutual backup is attained between two SCADA systeIffi using this mechanism The Sapporo NJShi Regional Control Center was put in operation in March 1997, arxi the rephrerrent of the existing Sapporo Network Control Station is scheduled to be completed in March 2001, thus providing complete mutual backup. A tempoony backup scheme will be med for the period between Marclll997 aM March 2001.
can be further divided by incorporating these new SCADA systeIffi into the Ring LAN. The Il11lIllal backup between the two SCADA systeIffi takes into account the following:
a. Database Fa:h SCADA system has to have a database of the entire Sapporo area in oo:ier to backup partner SCADA system Discrimination marks are provided within the database for distioction of substaIio~ under its own control from those under the control of partner SCADA system. All information taken into a system is processed but only information that is pertinent to a SCADA system is communicated to the cp:ratoc of the system arxi information related to another SCADA system is simply recorded arxi displayed This provision reduces burden to ~ arxi assures continuity of data in an event of backup.
3. SYSTEM CONF1GURATIOJN Figure 6 shows the system configuration. Jndeperx:Ient SCADA systems are employed by the Sapporo NJShi arxi Sapporo regional control centers. Bodl systems have a clientserver system aM q;JeIl distributed architecnre in oo:ier to ensure flexibility aM expandability. Ea:h SCADA system is reliable against various disIlJrbances a'> they are duplex systems in v.tUch the pincipal components are duplicated.
The databases maintained in the two SCADA systeIm must match ea;h other in order to assure Il11lIllal backup. The data is kept identical by managerrent of the source of the database at a SCADA system arxi is downloOOed to the sezvers aM clients ofbolh systems. Identical progrnms are imIaIled on the two SCADA systeIffi to allow backups of the server to be made.
b. Tele-Control A duplicated configuration is wed for substations under the control of a SCADA system a'> shown in Fig. 5. A single configuration is med for substations under the control of the partner SCADA system The line switch (l.SW) is automaricaIIy qJelated so that upward data is transmitted to boIh SCADA systems in a hytrid rrode, ~ the data related to COIlIrOI is tIansmitted from the SCADA system that controls them Sapporo Nisbi SCADA System
Fig. 6 System Configuration of Mutual BlK:kup 3.1 System-link
Sapporo SCADA System
EIhemet (1O~) is employed fortheLAN of elK:h system. Fa:h component is interconnected to one another through the industrial standard TCP/IP (fransmission Control ProtocollIntemet Protocol). Eight LAN lines of 5 types are provided aM connected with the single m:xIe qltical Ring LAN through RNU (ring network units), v.tUch lK:t as the data system The transmission rate over the qJtical Ring LAN is 100 ~ aM multiplexed transmission in units of 10 ~ is canied out between elK:h RNU by IDMA This anangement allows the two systems to transmit aM receive data at 10 ~ over the entire LAN system
LSW
Substation
Sapporo Control Area
Fig.5 Trarumission diagram
517
4. SYSIEM RJNCTIONS
server or front en:! prc:>resSOr (FEP) in a single system is interrupted.
This type of ba:kup system was put into operation at the Hakodate Regional Control Cent.er in 1995, and the same system was established at the Sapporo NIShi Regional Control Cent.er with little modification.
d Backup o/whole system The ba:kup function of the whole system is initiated \\hen any one system receives catastrophic damage by an earthIuake or the like, and ~n the operation of the hmnan interface clients and FEPs as well as the supervisory and control servers are intenupted.
The new nunuaI bockup function between SCADA systems developed for this system is described below.
4.1 DetaiJs ofbackupfwu:tiiJn between SCIDA systems
After confinning the damage by telephone, the hockup fimction is initiated by the ~ of the system that is still nmning nonnally. Operation of all supervisory and control servers or FEPs which remain online are stopped.
a. NomuJi operation Each system indepeOOently supervises and controls substations belonging to its own system during normal operation, and does not control substations belonging to the partner SCADA system which is to ba:k up, but can monitor infoonation of all substatiom. The exchange of unnecessary data between these two systems is prevented by gateways CGW) installed on the LAN which mainly transmit supervisory and control infonnation (system LAN and process lAN). They continuously monitor, however, the operation status of the oIher system \Wich they are to bock up through the communicarion LAN.
The supervisory and control server and PEP on the noonaIly operating system ptrl()l'Iffi the hockup of all the online infoonation to supervise and control all subslalions belonging to the two systems. Data are exchanged with the supervisory and control units by the tele-control nmt.er tmit fa- hockup (!'CM-BD) in the noonal system. Sin:e the noonal system am damaged system are operated in the control room am the training room, respectively, iOOication and aIarrn outputs to the mimic board and hmnan interface clients are dislIibuted to the control room and training room a::cording to the system to which they belong to.
b. Classification ofbackupfwu:tiiJns Ba::kup is perlooned according to the c~ sclmle shown in the following Table 1 \\hen any fault occurs in the noonal operation of one of these syst.erm.
e. Backup ofsupervisory and controlfunction A system starts the ba:kup operation \\hID the supervisory and control servers or the FEPs that execute the online jobs are tempoolrily lost from the system.
Table 1 cmsification SctlerD! of Mutual Bockyp Fault level Supervisory and control server
Ught
Middle
After confuming the damage. the ba:kup function is started by the ~ of the system that is nmning noonaIly. The supervisory am control server am the PEP on the normal system supervise and control all of the subsIaIions belonging to the two systems. Data must be transmitted, however, between the supervisory and control server and the FEP on the normal system am the mimic board and hmnan intelfuce client on the damaged system in CIder to ~ the damaged system in the control room of the damaged system. Therefore, gateways on the system LAN am process LAN open the gate to enable data to be exchanged between the systems upon receiving the badaJp request
Heavy
Any device
Any device
Any device
interrupted in
interrupted in
interrupted in
single system
both system
both system
Front End
processor (FEP) Human interface
Normal
Normal
Inoperable
Tele-control master unit (reM)
Nomal
Nomal
Inoperable
Backup by Backup stablS
duplex
Backup of supervisory and control function
Backup of whole system
The system can be switcIm into badaJp IOOde even \\hen the two system are nmning nonnally in CIder to perfonn training or testing of the two systems using the ofiline system In this case, the switching is done on the condition that no computer exists in the same operation mode as the IOOde to be hocked up.
c. Backup by duplex operation oftwo systems As two-system duplex operation is carried out in ea::h system in \\hlch the main components are duplicated, ea::h system independently ba:ks up and mutual hockup operation is not started when the supervisory and control
518
Although full-fledged operation of the mutual backup will not be started tmtil2001 when the Sapporo Regional Control Center will go into operation, all of the functions have been verified and the electric power system will be highly reliable.
f Backup oftele-control channel The tele
If faults occur on the same channel of both TCM-A and TCM-B of a system, data is exchanged with the remote supervisory and control units of substations using the TCM-BU of the other system. This ~on is done regardless of the operation mode of a, c, d and e.
Fig.7 External view of Control Room The technique improves the SCADA system as follows: 1) In an event of reorganization of regional control centers due to an increase in the number of substations, a new SCADA system can easily be incorporated into this System-Link. 2) The human-interface client can be installed separately from the supervisory and control server, which greatly improves the maintainability and expandability as well as the reliability.
4.2 Identification ofcontrol system The system which controls data of the components to be supelVised and controlled is identified by defining the system to which ea:h subslation belongs in the database. During nonnal ~on, ea::h of the two systems processes only infonnation of the system. All of the information belonging to both systems are processed during the backup of the whole system and the backup of the supervisory and control fi.Joction.
REFERENCES
4.3 Human interface
T. Tsunnna, T. Tatekouj~ F. MasOOa and R HayaW (1995): "New Scada System On Open-Distributing An::hitecture," CIGRE 1995 Symposium", Helsinki
The human interface client and the mimic board process infonnalion belonging to either system defined for ea::h component during the backup operation in a similar way to that dming normal operation. Therefore, the information to be roonitored or the range to be operated is not iIx:reased by the mutual backup, which allows the operator to work during the backup operation in the same way as during normal operation.
,6(X).]J
4.4 Maintenance offaciJiJy Infonnalion on the ta:ili.ties of both systems is processed altogether by a relational database for parallel maintenance in ea:h system.
5. mNQ.,USION
The paper describes mutual backup between SCADA systems, which is a Wlique technique based on the open architecture. The technique was started for the Sapporo NIShi Regional Control Center in March 1997 as shown in Fig. 7.
519