Recent Advances in Learning and Data-Driven Modeling of Complex Systems

Andrey Lokhov (Los Alamos National Lab)
Title: Learning of networked spreading models from noisy and incomplete data
Abstract Recent years have seen a lot of progress in algorithms for learning parameters of spreading dynamics from both full and partial data. Some of the remaining challenges include model selection under the scenarios of unknown network structure, noisy data, missing observations in time, as well a  s an efficient incorporation of prior information to minimize the number of samples required for an accurate learning. We introduce a universal learning method based on scalable dynamic message-passing technique that addresses these challenges often encountered in real data. The algorithm leverages available prior knowledge on the model and on the data, and reconstructs both network structure and parameters of a spreading model. We show that a linear computational complexity of the method with the key model parameters makes the algorithm scalable to large network instances.

Dante R. Chialvo (UNSAM)
Title: What is critical about the brain
Abstract Support for the role of critical phenomena as a fundamental property of brain function has evolved exponentially since its introduction by Bak and colleagues two and half decades ago.  In this lecture we review the most recent data-driven results at a wide range of scales, discuss its most relevant biological implications and identify newly open challenges.

Ferdinando Fioretto (Syracuse)
Title: Constrained-aware Machine Learning 
Abstract: In recent years, the integration of Machine Learning (ML) with challenging scientific and engineering problems has experienced remarkable growth. In particular, deep learning has proven to be an effective solution for unconstrained problem settings, but it has struggled to perform as well in domains where hard constraints, domain knowledge, or physical principles need to be taken into account. In areas such as power systems, materials science, and fluid dynamics, the data follows well-established physical laws, and ignoring these laws can result in unreliable and ineffective solutions. In this talk, we will delve into the need for constraint-aware ML. We will present how to integrate key constrained optimization principles within the training process of deep learning models, endowing them with the capability of handling hard constraints and physical principles. The resulting models will bring a new level of accuracy and efficiency to hard decision tasks, which will be showcased on energy and scheduling problems.

Francesco Caravelli (Los Alamos National Lab)
Title: Memristive networks and their applications
Abstract:There has been a lot of interest in nanoscale devices that seem to mimic some brain functionalities, such as plasticity. In this talk, we discuss exact and numerical techniques that can be used for nanoscale devices with memory, their connections to networks and Kirchhoff's laws, and their applications to study recent experiments on self-organizing nanowires. 
A large body of work has shown that PVP-coated self-organizing Ag nanowires have a dynamic response to an applied voltage that mimics the short-term plasticity of neuronal synapses. However, while every single experiment has a different connectivity pattern, a conductance response is observed that is not strongly dependent on the details of the nanowire network. In this talk, we use recent results on the study of memristive networks that apply to dynamical systems with constraints, such as Kirchhoff laws, to derive a mean field theory for networks of nanowires. We show how these mean field theories have potential applications that go beyond the specificity of the system, but that can be applied to a larger class of dynamical systems.

Guannan Qu (Carnegie-Mellon)
Title: Scalable reinforcement learning for multi-agent networked systems
Abstract: We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find policies such that the (discounted) global reward is maximized. A fundamental challenge in this setting is that the state-action space size scales exponentially in the number of agents, rendering the problem intractable for large networks. In this paper, we present our framework that exploits the network structure to conduct reinforcement learning in a scalable manner. The key feature in our framework is that we prove spatial decay properties for the Q function and the policy, meaning their dependence on faraway agents decays when the distance increases. Such spatial decay properties enable approximations by truncating the Q functions and policies to local neighborhoods, hence drastically reducing the dimension and avoiding the exponential blow-up in the number of agents. Lastly, we demonstrate the effectiveness of our approach in a microgrid inverter control example, showing our approach is significantly more scalable than benchmarks.
Bio: Guannan Qu has been an Assistant Professor at the Electrical and Computer Engineering Department of Carnegie Mellon University since September 2021. He received his B.S. degree in Electrical Engineering from Tsinghua University in Beijing, China in 2014, and his Ph.D. in Applied Mathematics from Harvard University in Cambridge, MA in 2019. He was a CMI and Resnick postdoctoral scholar in the Department of Computing and Mathematical Sciences at California Institute of Technology from 2019 to 2021. He is the recipient of Caltech Simoudis Discovery Award, PIMCO Fellowship, Amazon AI4Science Fellowship, and IEEE SmartGridComm Best Student Paper Reward. His research interest lies in control, optimization, and machine/reinforcement learning with applications to power systems, multi-agent systems, Internet of things, smart city, etc.

Hiroki Sayama (Binghamton University)
Title: Deriving Dynamical Model Equations from Temporal Network Data Using a Graph Rewriting Framework
Abstract:Many real-world networks dynamically change overtime. Such networks are collectively referred to as temporal networks. A wide variety of methods have already been developed and used for temporal network analysis, but they are predominantly descriptive so that they cannot provide dynamical, mechanistic explanation of observed network behaviors. To address this gap, here we propose a novel modeling method that derives dynamical model equations of temporal network behaviors directly from real-world temporal network data. We expect that this will help generate a more mechanistic “understanding” of what happened in the network under investigation, offering interpretable explanations of “how” and “why” the observed temporal network behaviors occurred. The proposed method is based on an unconventional approach by describing temporal network dynamics as repeated extraction and replacement of subgraphs across time. The dynamics of the subgraph densities and interactions are then formulated into dynamical model equations. In our presentation at CCS, we will showcase preliminary functionalities of the proposed method and its software implementation and will seek feedback from the audience. If successful, this project will produce a fundamental change in how temporal network data are analyzed and interpreted in scientific research and various applications.

Leonardo Rydin Gorjão (Norwegian University of Life Sciences)
Title: Complexity of the German weather-driven electricity spot prices - A data-driven analysis
Abstract: The integration of volatile renewable power sources can prove a central challenge in the transition to a sustainable energy system. Electricity markets are central in coordinating electric power generation across Europe. These markets rely evermore on short-term trading to facilitate the balancing of power generation and demand and to enable systems integration of small producers. Electricity prices are themselves afflicted by volatility induced endogenously from evolving market structures and schemes, as well as exogenously by varying power generation from different renewable and non-renewable generation. Electricity prices in these spot markets show pronounced fluctuations, featuring extreme peaks as well as occasional negative prices. In this presentation, we highlight a few distinct statistical properties of electricity prices from the European Power Exchange market, in particular the hourly day-ahead, hourly intraday, and 15-min intraday market prices. We utilise various statistical physics methods to quantify the fluctuations, correlations, and extreme events and reveal different time scales in the dynamics of the market. The short-term fluctuations show remarkably different characteristics for time scales below and above 12 hours. Fluctuations are strongly correlated and persistent below 12 hours, which contributes to extreme price events and strong multifractal behaviour. On longer time scales, they get anticorrelated and price time series revert to their mean, witnessed by a stark decrease of the Hurst coefficient after 12 hours. The long-term behaviour is strongly influenced by the evolution of a large-scale weather pattern with a typical time scale of four days. We elucidate this dependence in detail using a classification into circulation weather types. The separation in time scales enables a superstatistical treatment, which confirms the characteristic time scale of four days, and motivates the use of q-Gaussian distributions as the best fit for the empiric distribution of electricity prices.

Marc Vuffray (Los Alamos National Lab)
Title: Locating the source of forced oscillations in dynamical networks
Abstract: Forced oscillations in dynamical networks refers to a state where an external force with unknown frequency and location perturb the system. Forced oscillation events are a concern for power grids operators where malfunctioning equipment causes persisting periodic disturbances that can jeopardize the stability of the whole network. Localization of the source of such disturbances remains an outstanding challenge especially when the network parameters and connectivity are unknown to the observer. Here, we propose a new method for locating the source of forced oscillations which addresses this issue by performing a simultaneous dynamic model identification using a principled maximum likelihood approach. We illustrate the validity of the algorithm on a variety of examples where forcing leads to resonance conditions in the system dynamics. Our results establish that an accurate knowledge of system parameters is not required for a successful inference of the source and frequency of a forced oscillation.

Mehrnaz Anvari (Fraunhofer Institute)
Title: Data-driven load profiles and the dynamics of residential electricity consumption
Abstract: The dynamics of power consumption constitutes an essential building block for planning and operating sustainable energy systems. Whereas variations in the dynamics of renewable energy generation are reasonably well studied, a deeper understanding of the variations in consumption dynamics is still missing. Here, we analyse highly resolved residential electricity consumption data of Austrian, German and UK households and propose a generally applicable data-driven load model. Specifically, we disentangle the average demand profiles from the demand fluctuations based purely on time series data. We introduce a stochastic model to quantitatively capture the highly intermittent demand fluctuations. Thereby, we offer a better understanding of demand dynamics, in particular its fluctuations, and provide general tools for disentangling mean demand and fluctuations for any given system, going beyond the standard load profile (SLP). Our insights on the demand dynamics may support planning and operating future-compliant (micro) grids in maintaining supply-demand balance.

Misha Chertkov (U of A)
Title: U-Turn Diffusion
Abstract: We present a comprehensive examination of score-based diffusion models of AI for generating synthetic images. These models hinge upon a dynamic auxiliary time mechanism driven by stochastic differential equations, wherein the score function is acquired from input images. Our investigation unveils a criterion for evaluating efficiency of the score-based diffusion models: the power of the generative process depends on the ability to de-construct fast correlations during the reverse/de-noising phase. To improve the quality of the produced synthetic images, we introduce an approach coined "U-Turn Diffusion". The U-Turn Diffusion technique starts with the standard forward diffusion process, albeit with a condensed duration compared to conventional settings. Subsequently, we execute the standard reverse dynamics, initialized with the concluding configuration from the forward process. This U-Turn Diffusion procedure, combining forward, U-turn, and reverse processes, creates a synthetic image approximating an independent and identically distributed (i.i.d.) sample from the probability distribution implicitly described via input samples. To analyze relevant time scales we employ various analytical tools, including auto-correlation analysis, weighted norm of the score-function analysis, and Kolmogorov-Smirnov Gaussianity test. The tools guide us to establishing that the Kernel Intersection Distance, a metric comparing the quality of synthetic samples with real data samples, is minimized at the optimal U-turn time. The talk is based on a joint work with Hamidreza Behjoo  https://arxiv.org/abs/2308.07421 

Philippe Jacquod (HES-SO & UNIGE)
Title: Towards Real-Time Inference of Power Grid Characteristics with Physics-Informed Machine Learning
Abstract: As the energy transition unfolds, power grids are fed with more diverse and more fluctuating power generation. An important challenge is then to evaluate the amount of existing ancillary resources such as rotating inertia and primary control at any given time. As a matter of fact, these resources depend on the types of operating power plants, furthermore, their presence is crucial to guarantee the dynamic stability of the grid.
In this talk I will describe the first steps we took towards real-time inference of ancillary services in power grids from fluctuating operational data. The starting point is a reduced, continuous model of the power grid. The associated ODE -> PDE mapping introduces fields representing local impedances, droop controls and inertia. The impedance field is extracted from steady-state data, and we develop a physics-informed machine learning algorithm based on the swing equations to construct inertia and droop control fields. The approach is validated by direct comparison of frequency wave propagation in the PDE, continuous model vs. the ODE, discrete PanTaGruEl model of the synchronous grid of continental Europe. I will conclude with a discussion of the next steps to be undertaken to reach true, real-time inference of power grid characteristics.
*work in collaboration with J Fritzsch (U of Geneva), and M Chertkov and L Pagnier (U of Arizona)
L. Pagnier, J. Fritzsch, Ph. Jacquod, and M. Chertkov, IEEE Access 10, 65118 (2022)
M. Tyloo, L. Pagnier, and Ph. Jacquod, Science Advances 5, eaaw8359 (2019)
L. Pagnier, and Ph. Jacquod, PLoS ONE 14, e0213550 (2019)

Pietro De Lellis (Federico II)
Title: Inference of directed interactions in collective dynamics through information-theoretic metrics
Abstract: Pairwise interactions are critical to collective dynamics of natural and technological systems.Information theory is the gold standard to study these interactions, but recent work has identified  pitfalls in the way information flow is appraised through classical metrics—time-delayed mutual information and transfer entropy. These pitfalls have prompted the introduction of intrinsic mutual information to precisely measure information flow. In this talk, we discuss the potential use of intrinsic mutual information in the inference of directional influences to diagnose interactions from time-series of individual units. We explore this possibility within a minimalistic, mathematically tractable leader–follower model, for which we will show an excess of false inferences of intrinsic mutual information compared to transfer entropy. We will illustrate how this unexpected finding is linked to a fundamental limitation of intrinsic mutual information, which suffers from the same sins of time-delayed mutual information: a thin tail of the null distribution that favors the rejection of the null-hypothesis of independence.