- Email: [email protected]
Bi-level Demand Response Game with Information Sharing among Consumers*
11th IFAC Symposium on Dynamics and Control of 11th IFAC Symposium on Dynamics and Control of 11th IFACSystems, Symposium on Dynamics and Control of Process including Biosystems 11th IFACSystems, Symposium on Dynamics and Control of Process including Biosystems Process Systems, including Biosystems June 6-8, 2016. NTNU, Trondheim, Norway Available online at www.sciencedirect.com Process including Biosystems June 6-8,Systems, 2016. NTNU, Trondheim, Norway June 6-8, 2016. NTNU, Trondheim, Norway June 6-8, 2016. NTNU, Trondheim, Norway
ScienceDirect
IFAC-PapersOnLine 49-7 (2016) 663–668
Bi-level Demand Response Game with Bi-level Demand Response Game with Bi-level Demand Response Game with Bi-level Demand Response Game with Information Sharing among Consumers Information Sharing among Consumers Information Information Sharing Sharing among among Consumers Consumers
∗ ∗∗ ∗ ∗ Zhaohui Zhang ∗ Ruilong Deng ∗∗ Tao Yuan ∗ S. Joe Qin ∗ Zhaohui Zhang Deng ∗∗ ∗ ∗ Ruilong ∗∗ Tao Yuan ∗ ∗ S. Joe Qin ∗ ∗ Zhaohui Zhang ∗ Ruilong Deng ∗∗ Tao Yuan ∗ S. Joe Qin ∗ Zhaohui Zhang Ruilong Deng Tao Yuan S. Joe Qin ∗ ∗ Viterbi School of Engineering, University of Southern California, Los of Southern California, Los ∗ ∗ Viterbi School of Engineering, University University of ∗ Viterbi School of Angeles, CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) Viterbi School of Engineering, Engineering, University of Southern Southern California, California, Los Los CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) ∗∗Angeles, Angeles, CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) Department of Electrical and Computer Engineering, University of ∗∗Angeles, CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) Department of Electrical and Computer Engineering, University of ∗∗ ∗∗ ∗∗ Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada (email: [email protected]) Department of Electrical and Computer Engineering, University Alberta, Edmonton, AB, Canada (email: [email protected]) of Alberta, Edmonton, AB, Canada (email: [email protected]) Alberta, Edmonton, AB, Canada (email: [email protected]) Abstract: In we demand response problem grid Abstract: In In this this paper, paper, we we formulate formulate the the demand demand response response problem problem in in smart smart grid grid as as aa bi-level bi-level Abstract: game: a consumer-level game and aaresponse one-leader-one-follower game Abstract: In this this paper, paper,noncooperative we formulate formulate the the demand problem in in smart smartStackelberg grid as as aa bi-level bi-level game: a consumer-level noncooperative game and one-leader-one-follower Stackelberg game game: a consumer-level noncooperative game and a one-leader-one-follower Stackelberg game between the provider-level and the consumer-level. We prove the existence of a Nash Equilibrium game: a the consumer-level noncooperative game andWe a one-leader-one-follower Stackelberg game between provider-level and the consumer-level. prove the existence of a Nash Equilibrium between the and the consumer-level. We existence of Equilibrium for game Stackelberg for the game, focus between the provider-level provider-level andand thea We prove prove the the of aa Nash Nash Equilibrium for the the noncooperative noncooperative game and aconsumer-level. Stackelberg Equilibrium Equilibrium for existence the Stackelberg Stackelberg game, focus on on for the noncooperative game and a Stackelberg Equilibrium for the Stackelberg game, focus on the case with information sharing among all consumers, and design distributed algorithms for the noncooperative game and a Stackelberg Equilibrium for the Stackelberg game, focus on the case with information sharing among all consumers, and design distributed algorithms for for the the the case with information sharing among all consumers, and distributed algorithms for supply and demand side as well as the results are provided the caseside with information sharing among allinformation consumers, platform. and design designNumerical distributed algorithms for the the supply side and demand side side as well well as the the information platform. Numerical results are provided provided supply side and demand as as information platform. Numerical results are to illustrate the performance of the proposed algorithms and the effectiveness of information supply side and demand side asofwell asproposed the information platform. Numerical results are provided to illustrate the performance the algorithms and the effectiveness of information to illustrate the of algorithms and the effectiveness of information sharing for each payoff. to illustrate the performance performance of the the proposed proposed sharing for improving improving each consumer’s consumer’s payoff. algorithms and the effectiveness of information sharing for improving each consumer’s payoff. sharing for improving eachFederation consumer’s payoff. Control) Hosting by Elsevier Ltd. All rights reserved. © 2016, IFAC (International of Automatic Keywords: electric power systems, demand Keywords: electric power power systems, demand demand response, response, game game theory, theory, information information integration integration Keywords: Keywords: electric electric power systems, systems, demand response, response, game game theory, theory, information information integration integration
1. INTRODUCTION 1. INTRODUCTION INTRODUCTION 1. 1. INTRODUCTION Nowadays, taking advantages of widely deployed smart Nowadays, taking taking advantages advantages of of widely widely deployed deployed smart smart Nowadays, meters and two-way communication facilities of smart Nowadays, taking advantages of widely deployed smart meters and two-way communication facilities of meters and two-way communication facilities of smart grid, demand response, aiming to change the electricity meters and two-way communication facilities of smart grid, demand demand response, response, aiming aiming to to change change the the electricity electricity grid, usage patterns of end-users by the pricing strategies of grid, aiming electricity usage demand patternsresponse, of end-users end-users by to thechange pricingthe strategies of usage patterns of by the pricing strategies of the supply side, can lead to a significant improvement of usage patterns ofcan end-users bysignificant the pricing strategies of the supply side, lead to a improvement the supply side, can lead significant improvement of gird reliability (2014)]. Among all the side,and can efficiency lead to to a a [Siano significant improvement of gird supply reliability and efficiency [Siano (2014)]. Among all all gird reliability and efficiency [Siano (2014)]. Among the pricing programs, real-time pricing (RTP) has been gird reliability and efficiency [Siano (2014)]. Among all the pricing pricing programs, programs, real-time real-time pricing pricing (RTP) (RTP) has has been the widely considered to be the most efficient, the the pricing programs, real-time pricing (RTP)where has been been widely considered to be be the most most efficient, where the widely considered to the efficient, where the electricity price varies with every time interval (usually widely considered to be theevery mosttime efficient, where the electricity price varies with interval (usually electricity price varies with time interval (usually each hour or each 15 minutes) [Deng al. (2015)]. In electricity varies with every every time et (usually each hour hour price or each each 15 minutes) minutes) [Deng etinterval al. (2015)]. (2015)]. In each or 15 [Deng et al. In such a system, game theory can be leveraged to model the each hour or each 15 minutes) [Deng et al. (2015)]. In such a system, game theory can be leveraged to model the such aa system, game be to the interactive decision-making process of the power provider such system, game theory theory can can be leveraged leveraged to model model the interactive decision-making process of the the power power provider 1 interactive decision-making process of provider [Saad et al. (2012)]. and consumer 1 interactive decision-making process of the power provider [Saad et al. (2012)]. and consumer 1 1 and consumer 1 [Saad et al. (2012)]. and consumer [Saad et al.researches (2012)]. on various demand There have been extensive There have been extensive researches on various demand There have been extensive researches on various demand response game models that characterize behaviors of There have been extensive researches on the various demand response game models that characterize the behaviors of response game models that characterize the behaviors of the power providers and consumers. For example, Ibars response game models that characterize the behaviors of the power providers and consumers. For example, Ibars the power providers and consumers. For example, Ibars et al. (2010) and Nguyen et al. (2012) studied the demand the power providers and etconsumers. For example, Ibars et al. (2010) and Nguyen al. (2012) studied the demand et al. and Nguyen et al. studied the response game cost minimization and et al. (2010) (2010) andfocusing Nguyenon et energy al. (2012) (2012) studied the demand demand response game focusing on energy cost minimization and response game focusing on energy cost minimization peak-to-average ratio minimization. Mohsenian-Rad et al. response game focusing on energy cost minimizationetand and peak-to-average ratio minimization. Mohsenian-Rad al. peak-to-average ratio minimization. Mohsenian-Rad et al. (2010) and Deng et al. (2014) studied the noncooperative peak-to-average ratio minimization. Mohsenian-Rad et al. (2010) and Deng et al. (2014) studied the noncooperative (2010) and Deng et al. (2014) studied the noncooperative game among residential consumers. Chen et al. (2010) (2010) and Deng et al. (2014) studied Chen the noncooperative game among residential consumers. et al. (2010) game among residential consumers. Chen et al. studied the electricity market from the social game among residential consumers. Chen etperspective, al. (2010) (2010) studied the electricity market from the social perspective, studied the electricity market from the social perspective, aiming at optimization of the social welfare of studied the electricity market from the social perspective, aiming at optimization of the social welfare of the the entire entire aiming at optimization of the welfare of system. Wu et on and aiming optimization the social social welfare charging of the the entire entire system. at Wu et al. al. (2012) (2012)offocused focused on PHEV PHEV charging and system. Wu et focused on PHEV charging and discharging Maharjan (2013) and Chai system. Wu scenarios. et al. al. (2012) (2012) focused et on al. PHEV charging and discharging scenarios. Maharjan et al. (2013) and Chai discharging scenarios. Maharjan et al. (2013) and Chai et al. (2014) studied the scenario where multiple power discharging scenarios. Maharjan etwhere al. (2013) andpower Chai et al. (2014) studied the scenario multiple et al. studied the scenario where providers interact with multiple consumers. et al. (2014) (2014) studied scenario where multiple multiple power power providers interact withthe multiple consumers. providers interact with multiple consumers. providers interact with multiple consumers. In spite of the aforementioned works on different demand In spite of the aforementioned works on different demand In spite of the aforementioned works on different demand response game models under different scenarios, there is In spite of the aforementioned works on different demand response game models under different scenarios, there is response game models under different scenarios, there is lack of a unified hierarchical game structure that characresponse game models under different scenarios, there is lack of a unified hierarchical game structure that characlack of a unified hierarchical game structure that characlack of a unified hierarchical game structure that charac The work is supported in part by Alberta Innovates Technology
The work is supported in part by Alberta Innovates Technology The work is supported in part by Alberta Innovates Technology Futures (AITF) postdoctoral fellowship. The work is supported in part by Alberta Innovates Technology Futures (AITF) postdoctoral fellowship. 1 Futures (AITF)the postdoctoral fellowship. Throughout text, the terms “consumer” and “user” are inter1 Futures (AITF)the postdoctoral fellowship. Throughout text, the terms “consumer” and “user” are inter1 1 Throughout the text, the terms “consumer” and “user” are interchangeably used. 1 Throughout the text, the terms “consumer” and “user” are interchangeably used. changeably used. changeably used.
terizes the high-level leader-follower relationship between terizes the high-level leader-follower relationship between terizes the high-level leader-follower relationship between the supply side and demand side, and has flexibility of terizes the high-level leader-follower relationship between the supply side and demand side, and has flexibility of the supply side and demand side, and has flexibility of incorporating different game models within each side. Bethe supply side and demand side, and has flexibility of incorporating different game models within each side. Beincorporating different game models within each side. Besides, there is lack of study on comparing different inforincorporating different game models within each side. Besides, there is lack of study on comparing different inforsides, there is of on comparing different information sharing mechanisms game, while information sides, there is lack lack of study study in onthe comparing different information sharing mechanisms in the game, while information mation sharing mechanisms in the game, while information symmetry and sharing are crucial in game theory. mation sharing mechanisms in the game, while information symmetry and sharing are crucial in game theory. symmetry and sharing are crucial in theory. symmetry and sharing crucialpoint in game game theory. This paper serves as aaare starting for addressing the This paper serves as starting point for addressing the This paper serves as a starting point for addressing the above-mentioned problems, in which we focus on a microThis paper servesproblems, as a starting pointwefor addressing the above-mentioned in which focus on a microabove-mentioned problems, in which we focus on a microgrid one multiple conabove-mentioned which weand focus on a microgrid system system with with problems, one power powerin provider provider and multiple congrid system with power provider and multiple consumers, and the overall demand response game by grid system with one one multiple sumers, and model model the power overallprovider demand and response gameconby and model the overall demand response game asumers, bi-level model, comprising (1) a consumer-level noncoopsumers, and model the overall demand response noncoopgame by by aa bi-level model, comprising (1) aa consumer-level bi-level model, comprising (1) consumer-level noncooperative game and (2) aa one-leader-one-follower Stackelberg aerative bi-level model, comprising (1) a consumer-level noncoopgame and (2) one-leader-one-follower Stackelberg erative game (2) aa one-leader-one-follower Stackelberg game the provider-level and The erative game and and one-leader-one-follower Stackelberg game between between the(2) provider-level and consumer-level. consumer-level. The game between the provider-level and consumer-level. The structure has good extendability in terms of any game between the provider-level consumer-level. structure has good extendabilityand in terms of adding adding The any structure has good extendability in terms of adding any game on the supply side or changing any game on the structure has supply good extendability in terms ofgame adding any game on the side or changing any on the game on the supply side or changing any game on the demand side, as long as the game satisfies similar good game on side, the supply side or changing any game ongood the demand as long as the game satisfies similar demand side, as long as the game satisfies similar good properties. In addition, have designed an information demand side, long aswe game satisfies good properties. In as addition, wethe have designed ansimilar information properties. In we designed an information sharing for studied the case properties. In addition, addition, we have haveand designed sharing platform platform for consumers consumers and studiedan theinformation case when when sharing platform for consumers and studied the case all consumers share their demand information. Numerical sharing platform for their consumers andinformation. studied the Numerical case when when all consumers share demand all consumers share demand information. Numerical results are to win-win all consumers share their their demandthe information. Numerical results are provided provided to illustrate illustrate the win-win effectiveness effectiveness results are to illustrate the of sharing consumers. results are provided provided to among illustrate the win-win win-win effectiveness effectiveness of information information sharing among consumers. of information sharing among consumers. of information sharing among consumers. The rest of the paper is organized as follows. The system The rest of the paper is organized as follows. The system The rest of the paper is organized as follows. The system model is described in Section 2. In Section 3, we formulate The rest of the paper is organized as follows. The system model is described in Section 2. In Section 3, we formulate model is described in Section 2. In Section 3, we formulate the demand response problem as a bi-level game and model is described in Section 2. InasSection 3, wegame formulate the demand response problem a bi-level and the demand response problem as a bi-level game and prove the existence of equilibrium. In Section 4, we study the demand response problem as Ina Section bi-level 4,game and prove the existence of equilibrium. we study prove the existence of equilibrium. In Section 4, we study the information sharing design distributed prove thewith existence of equilibrium. In Section we study the case case with information sharing and and design 4, distributed the case information sharing and design distributed algorithms. Numerical results are provided Section the case with with information sharing designin algorithms. Numerical results are and provided indistributed Section 5 5 algorithms. Numerical results are provided in and conclusions are drawn in Section 6. algorithms. Numerical results are provided in Section Section 5 5 and conclusions are drawn in Section 6. and conclusions are drawn in Section 6. and conclusions are drawn in Section 6. 2. SYSTEM MODEL 2. SYSTEM MODEL 2. SYSTEM 2. SYSTEM MODEL MODEL 2.1 2.1 Two-way Two-way Communication Communication Infrastructure Infrastructure 2.1 2.1 Two-way Two-way Communication Communication Infrastructure Infrastructure Consider a microgrid system of Consider a microgrid system of one one power power provider provider and and Consider aa microgrid system of one power provider N consumers. As shown in Fig. 1, the infrastructure Consider microgrid system of one power provider and and N consumers. As shown in Fig. 1, the infrastructure N consumers. As in Fig. the infrastructure of the system contains two On the power layer, N consumers. As shown shown inlayers. Fig. 1, 1, the infrastructure of the system contains two layers. On the power layer, of of the the system system contains contains two two layers. layers. On On the the power power layer, layer,
Copyright © 2016, 2016 IFAC 663 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2016 IFAC 663 Copyright ©under 2016 responsibility IFAC 663Control. Peer review of International Federation of Automatic Copyright © 2016 IFAC 663 10.1016/j.ifacol.2016.07.252
IFAC DYCOPS-CAB, 2016 664 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
each user connects to the power provider via the power line. On the communication layer, users connect with each other and the power provider via the local area network (LAN). Through the LAN, the real-time twoway communication between the power provider and each user becomes feasible. In addition, users can share their information with each other via the LAN conveniently.
Air conditioner
User 1 Refrigerator …
Smart Meter
Washer
Local Area Network (LAN)
Power Provider
Power Line
Smart Meter User 2
Smart Meter
…
In this paper, we consider a quadratic gain function [Samadi et al. (2010)]: αj 2 ωj ωj dj − 2 dj , 0 ≤ dj < α j (4) Gj (dj ) = ωj ωj 2 , dj ≥ . 2αj αj where ωj > 0 and αj > 0 are pre-determined parameters, which vary among consumers, and also vary for the same consumer during different times of a day. We can see this gain function corresponds to a linear decreasing marginal benefit: ∂ 2 Gj (dj ) < 0, (5) ∂ 2 dj ωj when 0 ≤ dj < αj . 3. BI-LEVEL GAME FORMULATION
User N
3.1 Supply Side: Leader-level Fig. 1. Two-way communication infrastructure. In smart grid, the computation and optimization is implemented by programmed computers and smart meters. On the supply side, usually the pre-programmed computers are responsible for computing and implementing the realtime pricing strategies. On the demand side, each residential consumer is equipped with a smart meter, which can be pre-programmed to do computation in response to the real-time price, and to take automatic control of all household appliances based on the computation results. 2.2 Cost Function for Power Provider The cost function C(s) models the expense of supplying s unit of energy by the power provider. Demand response accommodates any form of cost functions as long as they satisfy the following two properties [Deng et al. (2015)]. Property 1. Increasing: the cost always increases when the supply amount increases. C(s1 ) < C(s2 ), ∀s1 < s2 . (1) Property 2. Strictly convex: the marginal cost always increases when the supply amount increases. ∂C(s2 ) ∂C(s1 ) < , ∀s1 < s2 . (2) s1 s2 The piece-wise linear function and quadratic function are two common choices. In this paper, we consider a quadratic cost function [Samadi et al. (2010)]: C(s) = as2 + bs + c, (3) where a > 0, b ≥ 0, and c ≥ 0 are three pre-determined parameters.
Let l denote the total energy demand from all consumers, i.e., dj , (6) l= j∈M
where M denote the set of all consumers. For the power provider, the profit from supplying s unit of electricity at the price of p is calculated as the revenue minus the cost function, i.e., Pp (s, p) = pl − C(s). (7) The local optimization problem at the supply side is: max Pp (s, p) s,p s≥l s.t. p ≥ 0.
(8)
It can be seen from (7) that once the total demand l is fixed, the increasing of the supply amount s will result in increasing of the cost and thus decreasing of the profit. Therefore, the provider tends to stay at the minimum supply amount that matches exactly with the demand s = l. (9) Substituting (9) into (7), we have (10) Pp (s, p) = ps − C(s). The price is chosen to maximize (10): ∂C(s) ∂C(s) ∂Pp (s, p) =p− =0⇒p= . (11) ∂s ∂s ∂s s=l Substituting (3) into (11), we have p(l) = 2al + b. (12) The price described in (12) is called the market-clearing price [Chen et al. (2010)], which is the local optimal choice of the power provider under the market-clearing condition.
2.3 Gain Function for Power Consumer 3.2 Demand Side: Follower-level The gain function Gj (dj ) models the power consumer j’s satisfaction degree obtained by consuming dj unit of energy. The gain function should be nondecreasing and concave, i.e., it is increasing before the energy consumption reaches a desired level, and gradually gets saturated when the desired level is satisfied [Deng et al. (2015)]. 664
Each consumer can respond to the power provider based on only the price information. As shown in Fig. 2, each user only communicates with the power provider to exchange the price and demand information. There is no communication or information sharing among consumers.
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
2 ∂ Pj (d) (14a) ∂d ∂d ≥ 0, ∀i = j ∈ M j i ∂ 2 Pj (d) ≤ 0, ∀i = j ∈ M. (14b) ∂dj ∂di Lemma 1. For the S-modular game, when NE exists and is unique, best response can be used to drive the solution converging to NE [Altman and Altman (2003)]. Theorem 1. NE of the noncooperative consumer game we formulated in this paper exists and is unique. Proof 1. Due to the space limitation, we omit the proof. Theorem 2. NE of the noncooperative consumer game we formulated in this paper can be achieved by best response. Proof 2. Due to the space limitation, we omit the proof.
Power Provider price
demand
demand
price
…
User 1
price
demand
…
User j
User N
Fig. 2. Without information sharing among consumers.
Power Provider price
demand
demand
price dj
User 1
N
price
demand
d1
d…
dN
User j
dj
…d
665
User N
Best Response Best response means that at each iteration, each consumer adapts his strategy to the strategies of others to maximize his own payoff. Mathematically, each consumer aims at solving (15) d∗j = arg max Pj (dj , d−j ),
1
Fig. 3. With information sharing among consumers. In reality, consumers can choose whether to share their demand information with each other or not. In this paper, we focus on the scenario where all consumers share their demand information and all shared information are authentic, as shown in Fig. 3. More complicated situations such as partial population participation or cheating will be considered as potential directions in our future work. Note that although consumers share demand information, the objective of each individual remains selfish. Each user still aims at maximizing his own gain while minimizing his payment. With the rational and selfish assumption, it is still a noncooperative game among consumers. We can utilize noncooperative game theory for building the model. Noncooperative Consumer Game The noncooperative game G = M, D, {Pj (·)}j∈M among consumers consists of three components: (1) Players: M = {1, 2, · · · , N } is a finite set of players. Each consumer in the system is a player. (2) Strategies: D = ×M j Dj is the strategy space of all players in the game. Each player j ∈ M chooses a demand strategy dj from his strategy set Dj . Let d−j = (d1 , · · · , dj−1 , dj+1 , · · · , dN ) denote the demand strategies of all consumers but j. We can write (dj , d−j ) for the overall demand profile d. (3) Payoff functions: the player j’s payoff is determined by the demand profile d. Each selfish and rational player j ∈ M chooses dj according to the other players’ strategies d−j to maximize Pj (dj , d−j ). For each consumer, the payoff from consuming dj unit of energy at the price of p is calculated as the gain function minus the payment, i.e., Pj (dj , d−j ) = Gj (dj ) − p(l)dj . (13) Definition 1. Nash Equilibrium (NE) and Ti [Fudenberg role (1991)]: A strategy profile d∗ = d∗j , d∗−j is called NE if and only if Pj (d∗ ) ≥ Pj dj , d∗−j , ∀j ∈ M, ∀dj ∈ Dj . Definition 2. An S-modular game restricts the payoff functions {Pj (·)} such that for ∀j ∈ M either of the following is satisfied [Neel et al. (2004)]: 665
dj ∈Dj
where Pj (·) is defined in (13). 3.3 Leader-Follower Interaction: Stackelberg Game In fact, the whole electricity market is a one-leader multifollower game, where the power provider leads the game by moving first, i.e., defining the electricity price, and then all consumers make their moves following the price afterwards, i.e., deciding the energy demand, and meanwhile consumers may share information with each other on the follower level. The main difference of a leader-follower game from a normal game lies in the situation of asymmetry of information. The leader chooses his strategy in advance, and then the follower makes the move accordingly. Therefore, many concepts and strategies in normal games are no longer suitable. For example, the equilibrium in leader-follower games may not satisfy NE conditions. In this paper, the overall game between the power provider and all consumers is now modeled by a bi-level game, comprising (1) a follower-level noncooperative game and (2) a one-leader-one-follower Stackelberg game. This model is due to the special structure of the game. On the leader side, the power provider does not care about the detail of how each consumer behaves. The provider’s supply and price are only affected by the aggregate behavior of all consumers, as shown in (9) and (12). Therefore, we can represent the response of the whole follower level by one single follower with strategy l = j∈M dj .
Another reason for this model formulation is due to the existing research on Stackelberg games. Stackelberg games are sourced from and have been extensively studied on the one-leader-one-follower case. For multi-leader-multifollower cases, there exists no special focus on the interactions among followers. Since as long as NE exists in the leader-level game, the game structures and equilibrium theories are well-formed. For more complicated cases where followers interact/cooperate/share information with each other, even for the single leader case, the equilibrium theories are not trivial and have not been well studied yet.
IFAC DYCOPS-CAB, 2016 666 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
One-Leader-One-Follower Stackelberg The Stack Game elberg demand response game Θ = Ω, S, ΦP (·), ΦL (·) is modeled as follows: (1) Players: Ω = {P, L}. The player set has two players: the player P is the leader, representing the power provider, while the player L is the follower, which is a virtual player representing all consumers. (2) Strategies: two sets of strategies P and L. The player P ’s strategy is the price p ∈ P, and the player L’s strategy is the aggregate demand l ∈ L. (3) Payoff functions: two sets of payoff functions ΦP (·): P × L → R and ΦL (·): P × L → RN , where ΦP (·) is described in (10), ΦL (·) = [P1 (·), · · · , PN (·)] is the payoff vector of all consumers. Definition 3. Best Response Set RL (p): The best response l∗ (p) ∈ RL (p) of the follower after observing the leader’s strategy p is l∗ (p) = j∈M d∗j (p).
Definition 4. Stackelberg Equilibrium (SE): A pair of strategies (pS , lS ) ∈ P × L is called SE if lS ∈ RL (pS ) and ΦP (p, l) ≤ ΦP (pS , lS ) for every pair (p, l) with l ∈ RL (p) [Bressan (2010)]. Lemma 2. If the sets P and L are compact metric spaces, and the payoff functions ΦP (·) and ΦL (·) are continuous, SE always exists [Bressan (2010)]. Theorem 3. SE of the one-leader-one-follower demand response Stackelberg game we formulated in this paper exists. Proof 3. Due to the space limitation, we omit the proof. 4. DISTRIBUTED ALGORITHMS WITH INFORMATION SHARING In this section, we focus on the scenario where all consumers share their demand information with each other. However, in practice, it is obvious that no one would like to share his information if he can obtain the information of others without sharing his own. Therefore the equilibrium is no one obtains any other information. This is the reason why information sharing is not the usual case. Power Provider price
demand
User 1 retrieve aggregate demand
…
User j
…
Specifically, we calculate the best response of the virtual follower in the Stackelberg game by achieving NE of the noncooperative game among consumers, and let the leader choose his optimal action accordingly. NE of the noncooperative consumer game is calculated by deriving the best response strategy of each consumer. The above calculation processes repeat until the overall demand response game converges. The convergence condition is that the power provider and all consumers do not revise their strategies any more. We begin with solving the best response problem for each consumer as formulated in (15), which could be directly solved by letting the first-order derivative equal zero, i.e., ∂Pj (dj , d−j ) = Gj (dj ) − 2a(l + dj ) − b = 0. ∂dj However, this method may not be applicable in practice since both a and b are the power provider’s cost function parameters. It is not realistic that the consumers know them. Due to the strict concavity of the payoff function, a gradient projection method [Wang and Xiu (2000)] is designed to solve (15), which does not require the functional parameters of the power provider: Dj ∂Pj dkj , d−j k+1 k dj = dj + γ ∂dkj Dj ∆p k dj . (16) = dkj + γ Gj (dkj ) − pk − ∆l On receiving each consumer’s demand strategy, the information sharing platform computes the aggregate demand by (6) and shares it with all consumers as well as the power provider. Based on the aggregate demand, the power provider updates his supply and price by (9) and (12) respectively, and broadcasts the updated price to all consumers. On receiving the updated price from the power provider and the aggregate demand from the information sharing platform, each consumer updates his demand strategy by (16). The iteration processes repeat until the game converges, i.e., no one revises his strategy any more. The algorithms for the supply side and demand side as well as the information platform are summarized as Algorithm 1, 2, and 3, respectively. Algorithm 1 : Executed by the power provider. 1: Initialization. 2: repeat 3: Receive demand dkj from all consumers j ∈ M. 4: Update supply sk by (9). 5: Update price pk by (12). 6: Broadcast updated price pk to all consumers. 7: until price does not change
User N
contribute demand
Demand Information Sharing Platform
Fig. 4. With demand information sharing platform. To avoid the above-mentioned situation, an information sharing platform can be set up, as shown in Fig. 4, which forces each user to contribute his own demand strategy in order to retrieve the aggregate demand information of others. We derive the game solution for the situation that (1) all consumers participate and (2) everyone contributes his real demand strategy, and design distributed algorithms for the supply side and demand side. 666
5. NUMERICAL RESULTS In this section, we provide numerical examples to evaluate the performance of the proposed bi-level demand response game with information sharing among consumers. For ease of illustration, we consider a simple case for a microgrid system with one power provider and three consumers. It can be extended to more users, with similar results.
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
3.5
Payoff1(share) Payoff2(share) Payoff3(share) Payoff1(no) Payoff2(no) Payoff3(no)
3 2.5 2 Payoff
3 Demand
3.5
Demand1(share) Demand2(share) Demand3(share) Demand1(no) Demand2(no) Demand3(no)
2.5
1.5 1
2
14
10 8 6 4
0.5
1.5 1
2
0
0
2
4
6
8
−0.5
10
ProviderProfit(share) AvUserPayoff(share) SociallWelfare(share) ProviderProfit(no) AvUserPayoff(no) SociallWelfare(no)
12
Payoff/Profit/Welfare
4
667
0
2
4
6
8
0
10
0
2
4
Iteration
Iteration
(a) Consumer demand
6
8
10
Iteration
(b) Consumer payoff
(c) Power provider profit
Fig. 5. Performance comparison between solution 1 (no) and solution 2 (share). 10 Social Welfare
10 8 Provider Profit
6 Demand3 Demand2
4
Payoff3
Demand1
Payoff2
2
Payoff1 1
2
3
4
7 6
Provider Profit
5 Demand3
4 3 2
Payoff2 Payoff1
5
6
7
0
8
1
2
3
4
5
6
7
4
Demand2 Payoff3 Payoff2 Payoff1
2 Demand1 1
2
3
4
5
6
7
(d) Fix α and vary ω
Demand3
3
7 6
Provider Profit
5 4 3
Demand3 Demand2
Payoff3
Payoff2 Payoff1
Demand1
2
8
0
Payoff3
Demand2
Payoff2
2 Demand1
Payoff1
1
2
3
4
5
6
7
8
14 ω=3.5; α1=0.6, α2=0.5, α3=0.4 Social Welfare
12 10
Provider Profit
8 6 4
Payoff3 Demand3 Demand2 Payoff2 Demand1 Payoff1
2
1 1
4
(c) Fix ω and vary α
ω=3; α1=0.6, α2=0.5, α3=0.4 Social Welfare
8
Provider Profit
Demand3
3
Social Welfare Demand/Payoff/Social Welfare
5
Provider Profit
5
(b) Fix ω and vary α 9
ω=2.5; α1=0.6, α2=0.5, α3=0.4
6
0
8
Social Welfare
7
1
1
6
Demand/Payoff/Social Welfare
Payoff3
Demand2 Demand1
α=0.6; ω1=2.5, ω2=3, ω3=3.5
8
8
(a) Fix ω and vary α
0
9
Social Welfare
Demand/Payoff/Social Welfare
0
α=0.5; ω1=2.5, ω2=3, ω3=3.5
9 Demand/Payoff/Social Welfare
Demand/Payoff/Social Welfare
α=0.4; ω1=2.5, ω2=3, ω3=3.5
Demand/Payoff/Social Welfare
12
1
2
3
4
5
6
7
(e) Fix α and vary ω
8
0
1
2
3
4
5
6
7
8
(f) Fix α and vary ω
Fig. 6. Performance comparison between solution 1 (no: green) and solution 2 (share: red). The simulation parameters for the power provider is a = 0.1, b = 0.5, and c = 0, i.e., the supply cost function is C(s) = 0.1s2 + 0.5s. For the parameters of the consumer gain functions, we first fix α and vary ω, then fix ω and vary α to show different cases. The iteration step size is fixed at γ = 0.5. All initial values of the price and demand are set to be random and it is verified through tests that the initial point does not affect the convergence point of the game. In Fig. 5, performance comparison details between the two scenarios are provided. One is without information sharing among consumers, i.e., each consumer responds to the power provider based on only the price information. The solution to this case is denoted by solution 1 (no). The other is with information sharing among consumers, i.e., each consumer responds to the power provider based on the price information and the aggregate demand infor-
667
mation of others. The solution to this case is denoted by solution 2 (share). Fig. 6 shows more simulation results when α varies from 0.4 to 0.6, and ω varies from 2.5 to 3.5. In the bar chart, solution 1 (no) is denoted by green bars, while red bars for solution 2 (share). The performance comparison metrics include each user’s energy demand, each user’s payoff, all users’ average payoff, power provider’s profit, and social welfare. Both figures illustrate that each consumer’s payoff is improved by information sharing, while the power provider’s profit drops and the social welfare slightly decreases. When comparing solution 2 (share) with solution 1 (no), we can see that the information at the supply side remains the same, while each consumer at the demand side gains more information by information sharing. From each consumer’s perspective, by contributing his own demand strategy, he can retrieve the aggregate demand information of others
IFAC DYCOPS-CAB, 2016 668 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
Algorithm 2 : Executed by each consumer j ∈ M. 1: Initialization. 2: repeat 3: Receive price pk from power provider. 4: Receive aggregate demand lk from information platform. 5: if k > 1 6: Record price change ∆p = pk − pk−1 . 7: Record aggregate demand change ∆l = lk − lk−1 . 8: Compute ∆p/∆l. 9: else 10: Set ∆p/∆l = 0. 11: end 12: Update demand dk+1 by (16). j 13: Communicate the updated demand dk+1 to power j provider. 14: Communicate the updated demand dk+1 to j information platform. 15: until demand does not change Algorithm 3 : Executed by the information platform. 1: Initialization. 2: repeat 3: Receive demand dkj from all consumers j ∈ M. 4: Compute aggregate demand lk by (6). 5: Broadcast aggregate demand lk to all consumers. 6: until aggregate demand does not change and increase his own payoff. It is actually a win-win situation for all consumers since every user’s payoff is improved. Therefore, we can conclude that information symmetry and sharing are crucial in game theory and are directly related to solution performance. 6. CONCLUSION AND FUTURE WORK In this paper, we have studied a bi-level game model for the demand response problem with one power provider and multiple consumers. The bi-level game consists of (1) a consumer-level noncooperative game and (2) a one-leaderone-follower Stackelberg game between the provider-level and consumer-level. An information sharing platform is designed for the scenario where all consumers share their demand information with each other. We have proved the existence of equilibrium for both games and proposed distributed algorithms for the supply side and demand side as well as the information platform. Numerical results are presented to illustrate the performance of the proposed algorithms and the effectiveness of information sharing for improving every user’s payoff. Serving as a starting point, the bi-level game model and information sharing mechanism analyzed in this paper can be further extended from several aspects. The scalability and convergence rate of the proposed approach will be studied. The equilibrium proof under more general cost and gain functions will also be interesting. In addition, we will explore the scenarios with multiple power providers and consumers, and compare different information sharing mechanisms such as partial population participation or cheating, both numerically and theoretically. 668
REFERENCES Altman, E. and Altman, Z. (2003). S-modular games and power control in wireless networks. IEEE Transactions on Automatic Control, 48(5), 839–842. Bressan, A. (2010). Noncooperative differential games. A tutorial. Technical report, Department of Mathematics, Penn State University. Chai, B., Chen, J., Yang, Z., and Zhang, Y. (2014). Demand response management with multiple utility companies: A two-level game approach. IEEE Transactions on Smart Grid, 5(2), 722–731. Chen, L., Li, N., Low, S.H., and Doyle, J.C. (2010). Two market models for demand response in power networks. Proc. IEEE SmartGridComm, 10, 397–402. Deng, R., Yang, Z., Chen, J., Asr, N.R., and Chow, M.Y. (2014). Residential energy consumption scheduling: A coupled-constraint game approach. IEEE Transactions on Smart Grid, 5(3), 1340–1350. Deng, R., Yang, Z., Chow, M.Y., and Chen, J. (2015). A survey on demand response in smart grids: Mathematical models and approaches. IEEE Transactions on Industrial Informatics, 11(3), 570–582. Fudenberg, D. and Tirole, J. (1991). Game theory. Cambridge, MA, USA: MIT Press. Ibars, C., Navarro, M., and Giupponi, L. (2010). Distributed demand management in smart grid with a congestion game. In Proc. IEEE SmartGridComm, 495– 500. Maharjan, S., Zhu, Q., Zhang, Y., Gjessing, S., and Basar, T. (2013). Dependable demand response management in the smart grid: A stackelberg game approach. IEEE Transactions on Smart Grid, 4(1), 120–132. Mohsenian-Rad, A.H., Wong, V.W., Jatskevich, J., Schober, R., and Leon-Garcia, A. (2010). Autonomous demand-side management based on game-theoretic energy consumption scheduling for the future smart grid. IEEE Transactions on Smart Grid, 1(3), 320–331. Neel, J.O., Reed, J.H., Gilles, R.P., et al. (2004). Convergence of cognitive radio networks. In Proc. IEEE WCNC, volume 4, 2250–2255. Nguyen, H.K., Song, J.B., and Han, Z. (2012). Demand side management to reduce peak-to-average ratio using game theory in smart grid. In Proc. IEEE INFOCOM Workshops, 91–96. Saad, W., Han, Z., Poor, H.V., and Ba¸sar, T. (2012). Game-theoretic methods for the smart grid: An overview of microgrid systems, demand-side management, and smart grid communications. IEEE Signal Processing Magazine, 29(5), 86–105. Samadi, P., Mohsenian-Rad, A.H., Schober, R., Wong, V.W., and Jatskevich, J. (2010). Optimal real-time pricing algorithm based on utility maximization for smart grid. In Proc. IEEE SmartGridComm, 415–420. Siano, P. (2014). Demand response and smart grids—A survey. Renewable and Sustainable Energy Reviews, 30, 461–478. Wang, C. and Xiu, N. (2000). Convergence of the gradient projection method for generalized convex minimization. Computational Optimization and Applications, 16(2), 111–120. Wu, C., Mohsenian-Rad, H., and Huang, J. (2012). Vehicle-to-aggregator interaction game. IEEE Transactions on Smart Grid, 3(1), 434–442.
ScienceDirect
IFAC-PapersOnLine 49-7 (2016) 663–668
Bi-level Demand Response Game with Bi-level Demand Response Game with Bi-level Demand Response Game with Bi-level Demand Response Game with Information Sharing among Consumers Information Sharing among Consumers Information Information Sharing Sharing among among Consumers Consumers
∗ ∗∗ ∗ ∗ Zhaohui Zhang ∗ Ruilong Deng ∗∗ Tao Yuan ∗ S. Joe Qin ∗ Zhaohui Zhang Deng ∗∗ ∗ ∗ Ruilong ∗∗ Tao Yuan ∗ ∗ S. Joe Qin ∗ ∗ Zhaohui Zhang ∗ Ruilong Deng ∗∗ Tao Yuan ∗ S. Joe Qin ∗ Zhaohui Zhang Ruilong Deng Tao Yuan S. Joe Qin ∗ ∗ Viterbi School of Engineering, University of Southern California, Los of Southern California, Los ∗ ∗ Viterbi School of Engineering, University University of ∗ Viterbi School of Angeles, CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) Viterbi School of Engineering, Engineering, University of Southern Southern California, California, Los Los CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) ∗∗Angeles, Angeles, CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) Department of Electrical and Computer Engineering, University of ∗∗Angeles, CA 90089 USA (e-mail: {zhaohuiz,tyuan,sqin}@usc.edu) Department of Electrical and Computer Engineering, University of ∗∗ ∗∗ ∗∗ Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada (email: [email protected]) Department of Electrical and Computer Engineering, University Alberta, Edmonton, AB, Canada (email: [email protected]) of Alberta, Edmonton, AB, Canada (email: [email protected]) Alberta, Edmonton, AB, Canada (email: [email protected]) Abstract: In we demand response problem grid Abstract: In In this this paper, paper, we we formulate formulate the the demand demand response response problem problem in in smart smart grid grid as as aa bi-level bi-level Abstract: game: a consumer-level game and aaresponse one-leader-one-follower game Abstract: In this this paper, paper,noncooperative we formulate formulate the the demand problem in in smart smartStackelberg grid as as aa bi-level bi-level game: a consumer-level noncooperative game and one-leader-one-follower Stackelberg game game: a consumer-level noncooperative game and a one-leader-one-follower Stackelberg game between the provider-level and the consumer-level. We prove the existence of a Nash Equilibrium game: a the consumer-level noncooperative game andWe a one-leader-one-follower Stackelberg game between provider-level and the consumer-level. prove the existence of a Nash Equilibrium between the and the consumer-level. We existence of Equilibrium for game Stackelberg for the game, focus between the provider-level provider-level andand thea We prove prove the the of aa Nash Nash Equilibrium for the the noncooperative noncooperative game and aconsumer-level. Stackelberg Equilibrium Equilibrium for existence the Stackelberg Stackelberg game, focus on on for the noncooperative game and a Stackelberg Equilibrium for the Stackelberg game, focus on the case with information sharing among all consumers, and design distributed algorithms for the noncooperative game and a Stackelberg Equilibrium for the Stackelberg game, focus on the case with information sharing among all consumers, and design distributed algorithms for for the the the case with information sharing among all consumers, and distributed algorithms for supply and demand side as well as the results are provided the caseside with information sharing among allinformation consumers, platform. and design designNumerical distributed algorithms for the the supply side and demand side side as well well as the the information platform. Numerical results are provided provided supply side and demand as as information platform. Numerical results are to illustrate the performance of the proposed algorithms and the effectiveness of information supply side and demand side asofwell asproposed the information platform. Numerical results are provided to illustrate the performance the algorithms and the effectiveness of information to illustrate the of algorithms and the effectiveness of information sharing for each payoff. to illustrate the performance performance of the the proposed proposed sharing for improving improving each consumer’s consumer’s payoff. algorithms and the effectiveness of information sharing for improving each consumer’s payoff. sharing for improving eachFederation consumer’s payoff. Control) Hosting by Elsevier Ltd. All rights reserved. © 2016, IFAC (International of Automatic Keywords: electric power systems, demand Keywords: electric power power systems, demand demand response, response, game game theory, theory, information information integration integration Keywords: Keywords: electric electric power systems, systems, demand response, response, game game theory, theory, information information integration integration
1. INTRODUCTION 1. INTRODUCTION INTRODUCTION 1. 1. INTRODUCTION Nowadays, taking advantages of widely deployed smart Nowadays, taking taking advantages advantages of of widely widely deployed deployed smart smart Nowadays, meters and two-way communication facilities of smart Nowadays, taking advantages of widely deployed smart meters and two-way communication facilities of meters and two-way communication facilities of smart grid, demand response, aiming to change the electricity meters and two-way communication facilities of smart grid, demand demand response, response, aiming aiming to to change change the the electricity electricity grid, usage patterns of end-users by the pricing strategies of grid, aiming electricity usage demand patternsresponse, of end-users end-users by to thechange pricingthe strategies of usage patterns of by the pricing strategies of the supply side, can lead to a significant improvement of usage patterns ofcan end-users bysignificant the pricing strategies of the supply side, lead to a improvement the supply side, can lead significant improvement of gird reliability (2014)]. Among all the side,and can efficiency lead to to a a [Siano significant improvement of gird supply reliability and efficiency [Siano (2014)]. Among all all gird reliability and efficiency [Siano (2014)]. Among the pricing programs, real-time pricing (RTP) has been gird reliability and efficiency [Siano (2014)]. Among all the pricing pricing programs, programs, real-time real-time pricing pricing (RTP) (RTP) has has been the widely considered to be the most efficient, the the pricing programs, real-time pricing (RTP)where has been been widely considered to be be the most most efficient, where the widely considered to the efficient, where the electricity price varies with every time interval (usually widely considered to be theevery mosttime efficient, where the electricity price varies with interval (usually electricity price varies with time interval (usually each hour or each 15 minutes) [Deng al. (2015)]. In electricity varies with every every time et (usually each hour hour price or each each 15 minutes) minutes) [Deng etinterval al. (2015)]. (2015)]. In each or 15 [Deng et al. In such a system, game theory can be leveraged to model the each hour or each 15 minutes) [Deng et al. (2015)]. In such a system, game theory can be leveraged to model the such aa system, game be to the interactive decision-making process of the power provider such system, game theory theory can can be leveraged leveraged to model model the interactive decision-making process of the the power power provider 1 interactive decision-making process of provider [Saad et al. (2012)]. and consumer 1 interactive decision-making process of the power provider [Saad et al. (2012)]. and consumer 1 1 and consumer 1 [Saad et al. (2012)]. and consumer [Saad et al.researches (2012)]. on various demand There have been extensive There have been extensive researches on various demand There have been extensive researches on various demand response game models that characterize behaviors of There have been extensive researches on the various demand response game models that characterize the behaviors of response game models that characterize the behaviors of the power providers and consumers. For example, Ibars response game models that characterize the behaviors of the power providers and consumers. For example, Ibars the power providers and consumers. For example, Ibars et al. (2010) and Nguyen et al. (2012) studied the demand the power providers and etconsumers. For example, Ibars et al. (2010) and Nguyen al. (2012) studied the demand et al. and Nguyen et al. studied the response game cost minimization and et al. (2010) (2010) andfocusing Nguyenon et energy al. (2012) (2012) studied the demand demand response game focusing on energy cost minimization and response game focusing on energy cost minimization peak-to-average ratio minimization. Mohsenian-Rad et al. response game focusing on energy cost minimizationetand and peak-to-average ratio minimization. Mohsenian-Rad al. peak-to-average ratio minimization. Mohsenian-Rad et al. (2010) and Deng et al. (2014) studied the noncooperative peak-to-average ratio minimization. Mohsenian-Rad et al. (2010) and Deng et al. (2014) studied the noncooperative (2010) and Deng et al. (2014) studied the noncooperative game among residential consumers. Chen et al. (2010) (2010) and Deng et al. (2014) studied Chen the noncooperative game among residential consumers. et al. (2010) game among residential consumers. Chen et al. studied the electricity market from the social game among residential consumers. Chen etperspective, al. (2010) (2010) studied the electricity market from the social perspective, studied the electricity market from the social perspective, aiming at optimization of the social welfare of studied the electricity market from the social perspective, aiming at optimization of the social welfare of the the entire entire aiming at optimization of the welfare of system. Wu et on and aiming optimization the social social welfare charging of the the entire entire system. at Wu et al. al. (2012) (2012)offocused focused on PHEV PHEV charging and system. Wu et focused on PHEV charging and discharging Maharjan (2013) and Chai system. Wu scenarios. et al. al. (2012) (2012) focused et on al. PHEV charging and discharging scenarios. Maharjan et al. (2013) and Chai discharging scenarios. Maharjan et al. (2013) and Chai et al. (2014) studied the scenario where multiple power discharging scenarios. Maharjan etwhere al. (2013) andpower Chai et al. (2014) studied the scenario multiple et al. studied the scenario where providers interact with multiple consumers. et al. (2014) (2014) studied scenario where multiple multiple power power providers interact withthe multiple consumers. providers interact with multiple consumers. providers interact with multiple consumers. In spite of the aforementioned works on different demand In spite of the aforementioned works on different demand In spite of the aforementioned works on different demand response game models under different scenarios, there is In spite of the aforementioned works on different demand response game models under different scenarios, there is response game models under different scenarios, there is lack of a unified hierarchical game structure that characresponse game models under different scenarios, there is lack of a unified hierarchical game structure that characlack of a unified hierarchical game structure that characlack of a unified hierarchical game structure that charac The work is supported in part by Alberta Innovates Technology
The work is supported in part by Alberta Innovates Technology The work is supported in part by Alberta Innovates Technology Futures (AITF) postdoctoral fellowship. The work is supported in part by Alberta Innovates Technology Futures (AITF) postdoctoral fellowship. 1 Futures (AITF)the postdoctoral fellowship. Throughout text, the terms “consumer” and “user” are inter1 Futures (AITF)the postdoctoral fellowship. Throughout text, the terms “consumer” and “user” are inter1 1 Throughout the text, the terms “consumer” and “user” are interchangeably used. 1 Throughout the text, the terms “consumer” and “user” are interchangeably used. changeably used. changeably used.
terizes the high-level leader-follower relationship between terizes the high-level leader-follower relationship between terizes the high-level leader-follower relationship between the supply side and demand side, and has flexibility of terizes the high-level leader-follower relationship between the supply side and demand side, and has flexibility of the supply side and demand side, and has flexibility of incorporating different game models within each side. Bethe supply side and demand side, and has flexibility of incorporating different game models within each side. Beincorporating different game models within each side. Besides, there is lack of study on comparing different inforincorporating different game models within each side. Besides, there is lack of study on comparing different inforsides, there is of on comparing different information sharing mechanisms game, while information sides, there is lack lack of study study in onthe comparing different information sharing mechanisms in the game, while information mation sharing mechanisms in the game, while information symmetry and sharing are crucial in game theory. mation sharing mechanisms in the game, while information symmetry and sharing are crucial in game theory. symmetry and sharing are crucial in theory. symmetry and sharing crucialpoint in game game theory. This paper serves as aaare starting for addressing the This paper serves as starting point for addressing the This paper serves as a starting point for addressing the above-mentioned problems, in which we focus on a microThis paper servesproblems, as a starting pointwefor addressing the above-mentioned in which focus on a microabove-mentioned problems, in which we focus on a microgrid one multiple conabove-mentioned which weand focus on a microgrid system system with with problems, one power powerin provider provider and multiple congrid system with power provider and multiple consumers, and the overall demand response game by grid system with one one multiple sumers, and model model the power overallprovider demand and response gameconby and model the overall demand response game asumers, bi-level model, comprising (1) a consumer-level noncoopsumers, and model the overall demand response noncoopgame by by aa bi-level model, comprising (1) aa consumer-level bi-level model, comprising (1) consumer-level noncooperative game and (2) aa one-leader-one-follower Stackelberg aerative bi-level model, comprising (1) a consumer-level noncoopgame and (2) one-leader-one-follower Stackelberg erative game (2) aa one-leader-one-follower Stackelberg game the provider-level and The erative game and and one-leader-one-follower Stackelberg game between between the(2) provider-level and consumer-level. consumer-level. The game between the provider-level and consumer-level. The structure has good extendability in terms of any game between the provider-level consumer-level. structure has good extendabilityand in terms of adding adding The any structure has good extendability in terms of adding any game on the supply side or changing any game on the structure has supply good extendability in terms ofgame adding any game on the side or changing any on the game on the supply side or changing any game on the demand side, as long as the game satisfies similar good game on side, the supply side or changing any game ongood the demand as long as the game satisfies similar demand side, as long as the game satisfies similar good properties. In addition, have designed an information demand side, long aswe game satisfies good properties. In as addition, wethe have designed ansimilar information properties. In we designed an information sharing for studied the case properties. In addition, addition, we have haveand designed sharing platform platform for consumers consumers and studiedan theinformation case when when sharing platform for consumers and studied the case all consumers share their demand information. Numerical sharing platform for their consumers andinformation. studied the Numerical case when when all consumers share demand all consumers share demand information. Numerical results are to win-win all consumers share their their demandthe information. Numerical results are provided provided to illustrate illustrate the win-win effectiveness effectiveness results are to illustrate the of sharing consumers. results are provided provided to among illustrate the win-win win-win effectiveness effectiveness of information information sharing among consumers. of information sharing among consumers. of information sharing among consumers. The rest of the paper is organized as follows. The system The rest of the paper is organized as follows. The system The rest of the paper is organized as follows. The system model is described in Section 2. In Section 3, we formulate The rest of the paper is organized as follows. The system model is described in Section 2. In Section 3, we formulate model is described in Section 2. In Section 3, we formulate the demand response problem as a bi-level game and model is described in Section 2. InasSection 3, wegame formulate the demand response problem a bi-level and the demand response problem as a bi-level game and prove the existence of equilibrium. In Section 4, we study the demand response problem as Ina Section bi-level 4,game and prove the existence of equilibrium. we study prove the existence of equilibrium. In Section 4, we study the information sharing design distributed prove thewith existence of equilibrium. In Section we study the case case with information sharing and and design 4, distributed the case information sharing and design distributed algorithms. Numerical results are provided Section the case with with information sharing designin algorithms. Numerical results are and provided indistributed Section 5 5 algorithms. Numerical results are provided in and conclusions are drawn in Section 6. algorithms. Numerical results are provided in Section Section 5 5 and conclusions are drawn in Section 6. and conclusions are drawn in Section 6. and conclusions are drawn in Section 6. 2. SYSTEM MODEL 2. SYSTEM MODEL 2. SYSTEM 2. SYSTEM MODEL MODEL 2.1 2.1 Two-way Two-way Communication Communication Infrastructure Infrastructure 2.1 2.1 Two-way Two-way Communication Communication Infrastructure Infrastructure Consider a microgrid system of Consider a microgrid system of one one power power provider provider and and Consider aa microgrid system of one power provider N consumers. As shown in Fig. 1, the infrastructure Consider microgrid system of one power provider and and N consumers. As shown in Fig. 1, the infrastructure N consumers. As in Fig. the infrastructure of the system contains two On the power layer, N consumers. As shown shown inlayers. Fig. 1, 1, the infrastructure of the system contains two layers. On the power layer, of of the the system system contains contains two two layers. layers. On On the the power power layer, layer,
Copyright © 2016, 2016 IFAC 663 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2016 IFAC 663 Copyright ©under 2016 responsibility IFAC 663Control. Peer review of International Federation of Automatic Copyright © 2016 IFAC 663 10.1016/j.ifacol.2016.07.252
IFAC DYCOPS-CAB, 2016 664 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
each user connects to the power provider via the power line. On the communication layer, users connect with each other and the power provider via the local area network (LAN). Through the LAN, the real-time twoway communication between the power provider and each user becomes feasible. In addition, users can share their information with each other via the LAN conveniently.
Air conditioner
User 1 Refrigerator …
Smart Meter
Washer
Local Area Network (LAN)
Power Provider
Power Line
Smart Meter User 2
Smart Meter
…
In this paper, we consider a quadratic gain function [Samadi et al. (2010)]: αj 2 ωj ωj dj − 2 dj , 0 ≤ dj < α j (4) Gj (dj ) = ωj ωj 2 , dj ≥ . 2αj αj where ωj > 0 and αj > 0 are pre-determined parameters, which vary among consumers, and also vary for the same consumer during different times of a day. We can see this gain function corresponds to a linear decreasing marginal benefit: ∂ 2 Gj (dj ) < 0, (5) ∂ 2 dj ωj when 0 ≤ dj < αj . 3. BI-LEVEL GAME FORMULATION
User N
3.1 Supply Side: Leader-level Fig. 1. Two-way communication infrastructure. In smart grid, the computation and optimization is implemented by programmed computers and smart meters. On the supply side, usually the pre-programmed computers are responsible for computing and implementing the realtime pricing strategies. On the demand side, each residential consumer is equipped with a smart meter, which can be pre-programmed to do computation in response to the real-time price, and to take automatic control of all household appliances based on the computation results. 2.2 Cost Function for Power Provider The cost function C(s) models the expense of supplying s unit of energy by the power provider. Demand response accommodates any form of cost functions as long as they satisfy the following two properties [Deng et al. (2015)]. Property 1. Increasing: the cost always increases when the supply amount increases. C(s1 ) < C(s2 ), ∀s1 < s2 . (1) Property 2. Strictly convex: the marginal cost always increases when the supply amount increases. ∂C(s2 ) ∂C(s1 ) < , ∀s1 < s2 . (2) s1 s2 The piece-wise linear function and quadratic function are two common choices. In this paper, we consider a quadratic cost function [Samadi et al. (2010)]: C(s) = as2 + bs + c, (3) where a > 0, b ≥ 0, and c ≥ 0 are three pre-determined parameters.
Let l denote the total energy demand from all consumers, i.e., dj , (6) l= j∈M
where M denote the set of all consumers. For the power provider, the profit from supplying s unit of electricity at the price of p is calculated as the revenue minus the cost function, i.e., Pp (s, p) = pl − C(s). (7) The local optimization problem at the supply side is: max Pp (s, p) s,p s≥l s.t. p ≥ 0.
(8)
It can be seen from (7) that once the total demand l is fixed, the increasing of the supply amount s will result in increasing of the cost and thus decreasing of the profit. Therefore, the provider tends to stay at the minimum supply amount that matches exactly with the demand s = l. (9) Substituting (9) into (7), we have (10) Pp (s, p) = ps − C(s). The price is chosen to maximize (10): ∂C(s) ∂C(s) ∂Pp (s, p) =p− =0⇒p= . (11) ∂s ∂s ∂s s=l Substituting (3) into (11), we have p(l) = 2al + b. (12) The price described in (12) is called the market-clearing price [Chen et al. (2010)], which is the local optimal choice of the power provider under the market-clearing condition.
2.3 Gain Function for Power Consumer 3.2 Demand Side: Follower-level The gain function Gj (dj ) models the power consumer j’s satisfaction degree obtained by consuming dj unit of energy. The gain function should be nondecreasing and concave, i.e., it is increasing before the energy consumption reaches a desired level, and gradually gets saturated when the desired level is satisfied [Deng et al. (2015)]. 664
Each consumer can respond to the power provider based on only the price information. As shown in Fig. 2, each user only communicates with the power provider to exchange the price and demand information. There is no communication or information sharing among consumers.
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
2 ∂ Pj (d) (14a) ∂d ∂d ≥ 0, ∀i = j ∈ M j i ∂ 2 Pj (d) ≤ 0, ∀i = j ∈ M. (14b) ∂dj ∂di Lemma 1. For the S-modular game, when NE exists and is unique, best response can be used to drive the solution converging to NE [Altman and Altman (2003)]. Theorem 1. NE of the noncooperative consumer game we formulated in this paper exists and is unique. Proof 1. Due to the space limitation, we omit the proof. Theorem 2. NE of the noncooperative consumer game we formulated in this paper can be achieved by best response. Proof 2. Due to the space limitation, we omit the proof.
Power Provider price
demand
demand
price
…
User 1
price
demand
…
User j
User N
Fig. 2. Without information sharing among consumers.
Power Provider price
demand
demand
price dj
User 1
N
price
demand
d1
d…
dN
User j
dj
…d
665
User N
Best Response Best response means that at each iteration, each consumer adapts his strategy to the strategies of others to maximize his own payoff. Mathematically, each consumer aims at solving (15) d∗j = arg max Pj (dj , d−j ),
1
Fig. 3. With information sharing among consumers. In reality, consumers can choose whether to share their demand information with each other or not. In this paper, we focus on the scenario where all consumers share their demand information and all shared information are authentic, as shown in Fig. 3. More complicated situations such as partial population participation or cheating will be considered as potential directions in our future work. Note that although consumers share demand information, the objective of each individual remains selfish. Each user still aims at maximizing his own gain while minimizing his payment. With the rational and selfish assumption, it is still a noncooperative game among consumers. We can utilize noncooperative game theory for building the model. Noncooperative Consumer Game The noncooperative game G = M, D, {Pj (·)}j∈M among consumers consists of three components: (1) Players: M = {1, 2, · · · , N } is a finite set of players. Each consumer in the system is a player. (2) Strategies: D = ×M j Dj is the strategy space of all players in the game. Each player j ∈ M chooses a demand strategy dj from his strategy set Dj . Let d−j = (d1 , · · · , dj−1 , dj+1 , · · · , dN ) denote the demand strategies of all consumers but j. We can write (dj , d−j ) for the overall demand profile d. (3) Payoff functions: the player j’s payoff is determined by the demand profile d. Each selfish and rational player j ∈ M chooses dj according to the other players’ strategies d−j to maximize Pj (dj , d−j ). For each consumer, the payoff from consuming dj unit of energy at the price of p is calculated as the gain function minus the payment, i.e., Pj (dj , d−j ) = Gj (dj ) − p(l)dj . (13) Definition 1. Nash Equilibrium (NE) and Ti [Fudenberg role (1991)]: A strategy profile d∗ = d∗j , d∗−j is called NE if and only if Pj (d∗ ) ≥ Pj dj , d∗−j , ∀j ∈ M, ∀dj ∈ Dj . Definition 2. An S-modular game restricts the payoff functions {Pj (·)} such that for ∀j ∈ M either of the following is satisfied [Neel et al. (2004)]: 665
dj ∈Dj
where Pj (·) is defined in (13). 3.3 Leader-Follower Interaction: Stackelberg Game In fact, the whole electricity market is a one-leader multifollower game, where the power provider leads the game by moving first, i.e., defining the electricity price, and then all consumers make their moves following the price afterwards, i.e., deciding the energy demand, and meanwhile consumers may share information with each other on the follower level. The main difference of a leader-follower game from a normal game lies in the situation of asymmetry of information. The leader chooses his strategy in advance, and then the follower makes the move accordingly. Therefore, many concepts and strategies in normal games are no longer suitable. For example, the equilibrium in leader-follower games may not satisfy NE conditions. In this paper, the overall game between the power provider and all consumers is now modeled by a bi-level game, comprising (1) a follower-level noncooperative game and (2) a one-leader-one-follower Stackelberg game. This model is due to the special structure of the game. On the leader side, the power provider does not care about the detail of how each consumer behaves. The provider’s supply and price are only affected by the aggregate behavior of all consumers, as shown in (9) and (12). Therefore, we can represent the response of the whole follower level by one single follower with strategy l = j∈M dj .
Another reason for this model formulation is due to the existing research on Stackelberg games. Stackelberg games are sourced from and have been extensively studied on the one-leader-one-follower case. For multi-leader-multifollower cases, there exists no special focus on the interactions among followers. Since as long as NE exists in the leader-level game, the game structures and equilibrium theories are well-formed. For more complicated cases where followers interact/cooperate/share information with each other, even for the single leader case, the equilibrium theories are not trivial and have not been well studied yet.
IFAC DYCOPS-CAB, 2016 666 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
One-Leader-One-Follower Stackelberg The Stack Game elberg demand response game Θ = Ω, S, ΦP (·), ΦL (·) is modeled as follows: (1) Players: Ω = {P, L}. The player set has two players: the player P is the leader, representing the power provider, while the player L is the follower, which is a virtual player representing all consumers. (2) Strategies: two sets of strategies P and L. The player P ’s strategy is the price p ∈ P, and the player L’s strategy is the aggregate demand l ∈ L. (3) Payoff functions: two sets of payoff functions ΦP (·): P × L → R and ΦL (·): P × L → RN , where ΦP (·) is described in (10), ΦL (·) = [P1 (·), · · · , PN (·)] is the payoff vector of all consumers. Definition 3. Best Response Set RL (p): The best response l∗ (p) ∈ RL (p) of the follower after observing the leader’s strategy p is l∗ (p) = j∈M d∗j (p).
Definition 4. Stackelberg Equilibrium (SE): A pair of strategies (pS , lS ) ∈ P × L is called SE if lS ∈ RL (pS ) and ΦP (p, l) ≤ ΦP (pS , lS ) for every pair (p, l) with l ∈ RL (p) [Bressan (2010)]. Lemma 2. If the sets P and L are compact metric spaces, and the payoff functions ΦP (·) and ΦL (·) are continuous, SE always exists [Bressan (2010)]. Theorem 3. SE of the one-leader-one-follower demand response Stackelberg game we formulated in this paper exists. Proof 3. Due to the space limitation, we omit the proof. 4. DISTRIBUTED ALGORITHMS WITH INFORMATION SHARING In this section, we focus on the scenario where all consumers share their demand information with each other. However, in practice, it is obvious that no one would like to share his information if he can obtain the information of others without sharing his own. Therefore the equilibrium is no one obtains any other information. This is the reason why information sharing is not the usual case. Power Provider price
demand
User 1 retrieve aggregate demand
…
User j
…
Specifically, we calculate the best response of the virtual follower in the Stackelberg game by achieving NE of the noncooperative game among consumers, and let the leader choose his optimal action accordingly. NE of the noncooperative consumer game is calculated by deriving the best response strategy of each consumer. The above calculation processes repeat until the overall demand response game converges. The convergence condition is that the power provider and all consumers do not revise their strategies any more. We begin with solving the best response problem for each consumer as formulated in (15), which could be directly solved by letting the first-order derivative equal zero, i.e., ∂Pj (dj , d−j ) = Gj (dj ) − 2a(l + dj ) − b = 0. ∂dj However, this method may not be applicable in practice since both a and b are the power provider’s cost function parameters. It is not realistic that the consumers know them. Due to the strict concavity of the payoff function, a gradient projection method [Wang and Xiu (2000)] is designed to solve (15), which does not require the functional parameters of the power provider: Dj ∂Pj dkj , d−j k+1 k dj = dj + γ ∂dkj Dj ∆p k dj . (16) = dkj + γ Gj (dkj ) − pk − ∆l On receiving each consumer’s demand strategy, the information sharing platform computes the aggregate demand by (6) and shares it with all consumers as well as the power provider. Based on the aggregate demand, the power provider updates his supply and price by (9) and (12) respectively, and broadcasts the updated price to all consumers. On receiving the updated price from the power provider and the aggregate demand from the information sharing platform, each consumer updates his demand strategy by (16). The iteration processes repeat until the game converges, i.e., no one revises his strategy any more. The algorithms for the supply side and demand side as well as the information platform are summarized as Algorithm 1, 2, and 3, respectively. Algorithm 1 : Executed by the power provider. 1: Initialization. 2: repeat 3: Receive demand dkj from all consumers j ∈ M. 4: Update supply sk by (9). 5: Update price pk by (12). 6: Broadcast updated price pk to all consumers. 7: until price does not change
User N
contribute demand
Demand Information Sharing Platform
Fig. 4. With demand information sharing platform. To avoid the above-mentioned situation, an information sharing platform can be set up, as shown in Fig. 4, which forces each user to contribute his own demand strategy in order to retrieve the aggregate demand information of others. We derive the game solution for the situation that (1) all consumers participate and (2) everyone contributes his real demand strategy, and design distributed algorithms for the supply side and demand side. 666
5. NUMERICAL RESULTS In this section, we provide numerical examples to evaluate the performance of the proposed bi-level demand response game with information sharing among consumers. For ease of illustration, we consider a simple case for a microgrid system with one power provider and three consumers. It can be extended to more users, with similar results.
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
3.5
Payoff1(share) Payoff2(share) Payoff3(share) Payoff1(no) Payoff2(no) Payoff3(no)
3 2.5 2 Payoff
3 Demand
3.5
Demand1(share) Demand2(share) Demand3(share) Demand1(no) Demand2(no) Demand3(no)
2.5
1.5 1
2
14
10 8 6 4
0.5
1.5 1
2
0
0
2
4
6
8
−0.5
10
ProviderProfit(share) AvUserPayoff(share) SociallWelfare(share) ProviderProfit(no) AvUserPayoff(no) SociallWelfare(no)
12
Payoff/Profit/Welfare
4
667
0
2
4
6
8
0
10
0
2
4
Iteration
Iteration
(a) Consumer demand
6
8
10
Iteration
(b) Consumer payoff
(c) Power provider profit
Fig. 5. Performance comparison between solution 1 (no) and solution 2 (share). 10 Social Welfare
10 8 Provider Profit
6 Demand3 Demand2
4
Payoff3
Demand1
Payoff2
2
Payoff1 1
2
3
4
7 6
Provider Profit
5 Demand3
4 3 2
Payoff2 Payoff1
5
6
7
0
8
1
2
3
4
5
6
7
4
Demand2 Payoff3 Payoff2 Payoff1
2 Demand1 1
2
3
4
5
6
7
(d) Fix α and vary ω
Demand3
3
7 6
Provider Profit
5 4 3
Demand3 Demand2
Payoff3
Payoff2 Payoff1
Demand1
2
8
0
Payoff3
Demand2
Payoff2
2 Demand1
Payoff1
1
2
3
4
5
6
7
8
14 ω=3.5; α1=0.6, α2=0.5, α3=0.4 Social Welfare
12 10
Provider Profit
8 6 4
Payoff3 Demand3 Demand2 Payoff2 Demand1 Payoff1
2
1 1
4
(c) Fix ω and vary α
ω=3; α1=0.6, α2=0.5, α3=0.4 Social Welfare
8
Provider Profit
Demand3
3
Social Welfare Demand/Payoff/Social Welfare
5
Provider Profit
5
(b) Fix ω and vary α 9
ω=2.5; α1=0.6, α2=0.5, α3=0.4
6
0
8
Social Welfare
7
1
1
6
Demand/Payoff/Social Welfare
Payoff3
Demand2 Demand1
α=0.6; ω1=2.5, ω2=3, ω3=3.5
8
8
(a) Fix ω and vary α
0
9
Social Welfare
Demand/Payoff/Social Welfare
0
α=0.5; ω1=2.5, ω2=3, ω3=3.5
9 Demand/Payoff/Social Welfare
Demand/Payoff/Social Welfare
α=0.4; ω1=2.5, ω2=3, ω3=3.5
Demand/Payoff/Social Welfare
12
1
2
3
4
5
6
7
(e) Fix α and vary ω
8
0
1
2
3
4
5
6
7
8
(f) Fix α and vary ω
Fig. 6. Performance comparison between solution 1 (no: green) and solution 2 (share: red). The simulation parameters for the power provider is a = 0.1, b = 0.5, and c = 0, i.e., the supply cost function is C(s) = 0.1s2 + 0.5s. For the parameters of the consumer gain functions, we first fix α and vary ω, then fix ω and vary α to show different cases. The iteration step size is fixed at γ = 0.5. All initial values of the price and demand are set to be random and it is verified through tests that the initial point does not affect the convergence point of the game. In Fig. 5, performance comparison details between the two scenarios are provided. One is without information sharing among consumers, i.e., each consumer responds to the power provider based on only the price information. The solution to this case is denoted by solution 1 (no). The other is with information sharing among consumers, i.e., each consumer responds to the power provider based on the price information and the aggregate demand infor-
667
mation of others. The solution to this case is denoted by solution 2 (share). Fig. 6 shows more simulation results when α varies from 0.4 to 0.6, and ω varies from 2.5 to 3.5. In the bar chart, solution 1 (no) is denoted by green bars, while red bars for solution 2 (share). The performance comparison metrics include each user’s energy demand, each user’s payoff, all users’ average payoff, power provider’s profit, and social welfare. Both figures illustrate that each consumer’s payoff is improved by information sharing, while the power provider’s profit drops and the social welfare slightly decreases. When comparing solution 2 (share) with solution 1 (no), we can see that the information at the supply side remains the same, while each consumer at the demand side gains more information by information sharing. From each consumer’s perspective, by contributing his own demand strategy, he can retrieve the aggregate demand information of others
IFAC DYCOPS-CAB, 2016 668 June 6-8, 2016. NTNU, Trondheim, NorwayZhaohui Zhang et al. / IFAC-PapersOnLine 49-7 (2016) 663–668
Algorithm 2 : Executed by each consumer j ∈ M. 1: Initialization. 2: repeat 3: Receive price pk from power provider. 4: Receive aggregate demand lk from information platform. 5: if k > 1 6: Record price change ∆p = pk − pk−1 . 7: Record aggregate demand change ∆l = lk − lk−1 . 8: Compute ∆p/∆l. 9: else 10: Set ∆p/∆l = 0. 11: end 12: Update demand dk+1 by (16). j 13: Communicate the updated demand dk+1 to power j provider. 14: Communicate the updated demand dk+1 to j information platform. 15: until demand does not change Algorithm 3 : Executed by the information platform. 1: Initialization. 2: repeat 3: Receive demand dkj from all consumers j ∈ M. 4: Compute aggregate demand lk by (6). 5: Broadcast aggregate demand lk to all consumers. 6: until aggregate demand does not change and increase his own payoff. It is actually a win-win situation for all consumers since every user’s payoff is improved. Therefore, we can conclude that information symmetry and sharing are crucial in game theory and are directly related to solution performance. 6. CONCLUSION AND FUTURE WORK In this paper, we have studied a bi-level game model for the demand response problem with one power provider and multiple consumers. The bi-level game consists of (1) a consumer-level noncooperative game and (2) a one-leaderone-follower Stackelberg game between the provider-level and consumer-level. An information sharing platform is designed for the scenario where all consumers share their demand information with each other. We have proved the existence of equilibrium for both games and proposed distributed algorithms for the supply side and demand side as well as the information platform. Numerical results are presented to illustrate the performance of the proposed algorithms and the effectiveness of information sharing for improving every user’s payoff. Serving as a starting point, the bi-level game model and information sharing mechanism analyzed in this paper can be further extended from several aspects. The scalability and convergence rate of the proposed approach will be studied. The equilibrium proof under more general cost and gain functions will also be interesting. In addition, we will explore the scenarios with multiple power providers and consumers, and compare different information sharing mechanisms such as partial population participation or cheating, both numerically and theoretically. 668
REFERENCES Altman, E. and Altman, Z. (2003). S-modular games and power control in wireless networks. IEEE Transactions on Automatic Control, 48(5), 839–842. Bressan, A. (2010). Noncooperative differential games. A tutorial. Technical report, Department of Mathematics, Penn State University. Chai, B., Chen, J., Yang, Z., and Zhang, Y. (2014). Demand response management with multiple utility companies: A two-level game approach. IEEE Transactions on Smart Grid, 5(2), 722–731. Chen, L., Li, N., Low, S.H., and Doyle, J.C. (2010). Two market models for demand response in power networks. Proc. IEEE SmartGridComm, 10, 397–402. Deng, R., Yang, Z., Chen, J., Asr, N.R., and Chow, M.Y. (2014). Residential energy consumption scheduling: A coupled-constraint game approach. IEEE Transactions on Smart Grid, 5(3), 1340–1350. Deng, R., Yang, Z., Chow, M.Y., and Chen, J. (2015). A survey on demand response in smart grids: Mathematical models and approaches. IEEE Transactions on Industrial Informatics, 11(3), 570–582. Fudenberg, D. and Tirole, J. (1991). Game theory. Cambridge, MA, USA: MIT Press. Ibars, C., Navarro, M., and Giupponi, L. (2010). Distributed demand management in smart grid with a congestion game. In Proc. IEEE SmartGridComm, 495– 500. Maharjan, S., Zhu, Q., Zhang, Y., Gjessing, S., and Basar, T. (2013). Dependable demand response management in the smart grid: A stackelberg game approach. IEEE Transactions on Smart Grid, 4(1), 120–132. Mohsenian-Rad, A.H., Wong, V.W., Jatskevich, J., Schober, R., and Leon-Garcia, A. (2010). Autonomous demand-side management based on game-theoretic energy consumption scheduling for the future smart grid. IEEE Transactions on Smart Grid, 1(3), 320–331. Neel, J.O., Reed, J.H., Gilles, R.P., et al. (2004). Convergence of cognitive radio networks. In Proc. IEEE WCNC, volume 4, 2250–2255. Nguyen, H.K., Song, J.B., and Han, Z. (2012). Demand side management to reduce peak-to-average ratio using game theory in smart grid. In Proc. IEEE INFOCOM Workshops, 91–96. Saad, W., Han, Z., Poor, H.V., and Ba¸sar, T. (2012). Game-theoretic methods for the smart grid: An overview of microgrid systems, demand-side management, and smart grid communications. IEEE Signal Processing Magazine, 29(5), 86–105. Samadi, P., Mohsenian-Rad, A.H., Schober, R., Wong, V.W., and Jatskevich, J. (2010). Optimal real-time pricing algorithm based on utility maximization for smart grid. In Proc. IEEE SmartGridComm, 415–420. Siano, P. (2014). Demand response and smart grids—A survey. Renewable and Sustainable Energy Reviews, 30, 461–478. Wang, C. and Xiu, N. (2000). Convergence of the gradient projection method for generalized convex minimization. Computational Optimization and Applications, 16(2), 111–120. Wu, C., Mohsenian-Rad, H., and Huang, J. (2012). Vehicle-to-aggregator interaction game. IEEE Transactions on Smart Grid, 3(1), 434–442.