In this note, we study the problem of distributed control over communication channels, where a number of distributed stations collaborate to stabilize a linear system. We quantify the rate requirements and obtain optimal signaling, coding and control schemes for decentralized stabilizability in such multicontroller systems. We show that in the absence of a centralized decoder at the plant, there is in general a rate loss in decentralized systems as compared to a centralized system. This result is in contrast with the absence of rate loss in the stabilization of multisensor systems. Furthermore, there is rate loss even if explicit channels are available between the stations. We obtain the minimum data rates needed in terms of the open-loop system matrix and the connectivity graph of the decentralized system, and obtain the optimal signaling policies. We also present constructions leading to stability. In addition, we show that if there are dedicated channels connecting the controllers, rate requirements become more lenient, and as a result strong connectivity is not required for decentralized stabilizability. We determine the minimum number of such external channels leading to a stable system, in case strong connectivity is absent.
This article explains the distinctions between robustness and resilience in control systems. Resilience confronts a distinct set of challenges, posing new ones for designing controllers for feedback systems, networks, and machines that prioritize resilience over robustness. The concept of resilience is explored through a three-stage model, emphasizing the need for a proactive preparation and automated response to elastic events. A toy model is first used to illustrate the tradeoffs between resilience and robustness. Then, it delves into contextual dualism and interactionism, and introduces game-theoretic paradigms as a unifying framework to consolidate resilience and robustness. The article concludes by discussing the interplay between robustness and resilience, suggesting that a comprehensive theory of resilience and quantification metrics, and formalization through game-theoretic frameworks are necessary. The exploration extends to system-of-systems resilience and various mechanisms, including the integration of AI techniques and non-technical solutions, like cyber insurance, to achieve comprehensive resilience in control systems. As we approach 2030, the systems and control community is at the opportune moment to lay scientific foundations of resilience by bridging feedback control theory, game theory, and learning theory. Resilient control systems will enhance overall quality of life, enable the development of a resilient society, and create a societal-scale impact amid global challenges such as climate change, conflicts, and cyber insecurity.
Strategic information disclosure, in its simplest form, considers a game between an information provider (sender) who has access to some private information that an information receiver is interested in. While the receiver takes an action that affects the utilities of both players, the sender can design information (or modify beliefs) of the receiver through signal commitment, hence posing a Stackelberg game. However, obtaining a Stackelberg equilibrium for this game traditionally requires the sender to have access to the receiver's objective. In this work, we consider an online version of information design where a sender interacts with a receiver of an unknown type who is adversarially chosen at each round. Restricting attention to Gaussian prior and quadratic costs for the sender and the receiver, we show that $\mathcal{O}(\sqrt{T})$ regret is achievable with full information feedback, where $T$ is the total number of interactions between the sender and the receiver. Further, we propose a novel parametrization that allows the sender to achieve $\mathcal{O}(\sqrt{T})$ regret for a general convex utility function. We then consider the Bayesian Persuasion problem with an additional cost term in the objective function, which penalizes signaling policies that are more informative and obtain $\mathcal{O}(\log(T))$ regret. Finally, we establish a sublinear regret bound for the partial information feedback setting and provide simulations to support our theoretical results.
We consider in this paper a class of stochastic team and nonzero-sum game problems with more than two agents who have access to decentralized information and may build their own subjective probability models to be used in the decision process. There is, in general, no compatibility between different models built by different agents, and this makes the available theory on teams and games inapplicable to our problem. We discuss different equilibrium solutions to the team and game problems in this multimodelling framework, and develop convergent algorithms which would lead to such an equilibrium under a number of conditions and for different probabilistic models. As a by-product of our analysis, we obtain a recursive algorithm which provides a solution to quadratic teams when the underlying distributions are not Gaussian.
We study power control in multicell CDMA wireless networks as a team optimization problem where each mobile attains its individual fixed target SIR level by transmitting with minimum possible power level. We derive conditions under which the power control problem admits a unique feasible solution. Using a Lagrangian relaxation approach similar to F. Kelly et al. (1998) we obtain two decentralized dynamic power control algorithms: primal and dual power update, and establish their global stability utilizing both classical Lyapunov theory and the passivity framework [J.T. Wen and M. Arcak, February 2004]. We show that the robustness results of passivity studies [(X. Fan et al., July 2004), (X. Fan et al., 2004)] as well as most of the stability and robustness analyses of F. Kelly et al. (1998) in the literature are applicable to the power control problem considered. In addition, some of the basic principles of call admission control are investigated from the perspective of the model adopted in this paper. We illustrate the proposed power control schemes through simulations.
This paper develops a solution method to obtain hierarchical noncooperative equilibria of a class of stochastic decision problems with more than two levels of hierarchy (in decision making), and wherein the decision makers (DMs) have access to nested dynamic information. In particular, we consider the three-person three-criteria quadratic decision problems wherein DM1 has access to the static observations and the controls of the other two DMs (in addition to his own static observation of the state of nature), and DM2 has access to the static observation and the control of DM3 (in addition to his own static observation). First DM1 announces his strategy and enforces it on both DM2 and DM3, and then DM2 announces his strategy and dictates it on DM3. Assuming that all three DMs are rational decision makers striving to minimize their expected costs, we first introduce the notion of hierarchical equilibrium for this decision problem, and then present a method for obtaining equilibrium strategies which exhibit the following feature: The equilibrium strategy of the decision maker (DM1) at the top of the hierarchy forces the other two decision makers to act in such a way so as to minimize jointly his expected cost function, while that of DM2 forces DM3 to minimize jointly the expected cost function of DM2 under the declared equilibrium strategy of DM1. An explicit derivation is given, in this framework, which leads to strategies that are linear in the dynamic part of the information available to DM1 and DM2. These strategies are, however, nonlinear functions of the static part of the information, even if the underlying statistics are Gaussian.
Recent years have witnessed significant advances in technologies and services in modern network applications, including smart grid management, wireless communication, cybersecurity as well as multi-agent autonomous systems. Considering the heterogeneous nature of networked entities, emerging network applications call for game-theoretic models and learning-based approaches in order to create distributed network intelligence that responds to uncertainties and disruptions in a dynamic or an adversarial environment. This paper articulates the confluence of networks, games and learning, which establishes a theoretical underpinning for understanding multi-agent decision-making over networks. We provide an selective overview of game-theoretic learning algorithms within the framework of stochastic approximation theory, and associated applications in some representative contexts of modern network systems, such as the next generation wireless communication networks, the smart grid and distributed machine learning. In addition to existing research works on game-theoretic learning over networks, we highlight several new angles and research endeavors on learning in games that are related to recent developments in artificial intelligence. Some of the new angles extrapolate from our own research interests. The overall objective of the paper is to provide the reader a clear picture of the strengths and challenges of adopting game-theoretic learning methods within the context of network systems, and further to identify fruitful future research directions on both theoretical and applied studies.
In this paper, we study a large population game with heterogeneous dynamics and cost functions solving a consensus problem. Moreover, the agents have communication constraints which appear as: (1) an Additive-White Gaussian Noise (AWGN) channel, and (2) asynchronous data transmission via a fixed scheduling policy. Since the complexity of solving the game increases with the number of agents, we use the Mean-Field Game paradigm to solve it. Under standard assumptions on the information structure of the agents, we prove that the control of the agent in the MFG setting is free of the dual effect. This allows us to obtain an equilibrium control policy for the generic agent, which is a function of only the local observation of the agent. Furthermore, the equilibrium mean-field trajectory is shown to follow linear dynamics, hence making it computable. We show that in the finite population game, the equilibrium control policy prescribed by the MFG analysis constitutes an −Nash equilibrium, where tends to zero as the number of agents goes to infinity. The paper is concluded with simulations demonstrating the performance of the equilibrium control policy.
We consider the Hegselmann-Krause model for opinion dynamics and study the evolution of the system under various settings. We first analyze the termination time of the synchronous Hegselmann-Krause dynamics in arbitrary finite dimensions and show that the termination time in general only depends on the number of agents involved in the dynamics. To the best of our knowledge, that is the sharpest bound for the termination time of such dynamics that removes dependency of the termination time from the dimension of the ambient space, and connects the convergence speed of the dynamics to the eigenvalues of the adjacency matrix of the connectivity graph in the Hegselmann-Krause dynamics. This answers an open question in the paper by Bhattacharyya et al. on how to obtain a tighter upper bound for the termination time. Furthermore, we study the asynchronous Hegselmann-Krause model from a novel game-theoretic approach and show that the evolution of an asynchronous Hegselmann-Krause model is equivalent to a sequence of best response updates in a well-designed potential game. We then provide a polynomial upper bound for the expected time and expected number of switching topologies until the dynamic reaches an arbitrarily small neighborhood of its equilibrium points, provided that the agents update uniformly at random. This is a step toward analysis of heterogeneous Hegselmann-Krause dynamics. Finally, we consider the heterogeneous Hegselmann-Krause dynamics and provide a necessary condition for the finite termination time of such dynamics. In particular, we sketch some future directions toward more detailed analysis of the heterogeneous Hegselmann-Krause model.
Many real-world applications of multi-agent reinforcement learning (RL), such as multi-robot navigation and decentralized control of cyber-physical systems, involve the cooperation of agents as a team with aligned objectives. We study multi-agent RL in the most basic cooperative setting -- Markov teams -- a class of Markov games where the cooperating agents share a common reward. We propose an algorithm in which each agent independently runs stage-based V-learning (a Q-learning style algorithm) to efficiently explore the unknown environment, while using a stochastic gradient descent (SGD) subroutine for policy updates. We show that the agents can learn an $\epsilon$-approximate Nash equilibrium policy in at most $\propto\widetilde{O}(1/\epsilon^4)$ episodes. Our results advocate the use of a novel \emph{stage-based} V-learning approach to create a stage-wise stationary environment. We also show that under certain smoothness assumptions of the team, our algorithm can achieve a nearly \emph{team-optimal} Nash equilibrium. Simulation results corroborate our theoretical findings. One key feature of our algorithm is being \emph{decentralized}, in the sense that each agent has access to only the state and its local actions, and is even \emph{oblivious} to the presence of the other agents. Neither communication among teammates nor coordination by a central controller is required during learning. Hence, our algorithm can readily generalize to an arbitrary number of agents, without suffering from the exponential dependence on the number of agents.
'/%3e%3cuse xlink:href='%23a' transform='rotate(23.036 .093 25.536)'/%3e%3cuse xlink:href='%23a' transform='rotate(45.87 1.273 16.18)'/%3e%3cuse xlink:href='%23a' transform='rotate(69.945 .996 12.078)'/%3e%3cuse xlink:href='%23a' transform='rotate(20.66 -19.689 31.932)'/%3e%3c/svg%3e)
