stochastic

This paper studies gradient dynamics subject to additive stochastic noise, which may arise from sources such as stochastic gradient estimation, measurement noise, or stochastic sampling errors. To analyze the robustness of such stochastic gradient systems, the concept of small-covariance noise-to-state stability (NSS) is introduced, along with a Lyapunov-based characterization. Furthermore, the classical Polyak–Lojasiewicz (PL) condition on the objective function is generalized to the K-PL condition via comparison functions, thereby extending its applicability to a broader class of optimization problems. It is shown that the stochastic gradient dynamics exhibit small-covariance NSS if the objective function satisfies the K-PL condition and possesses a globally Lipschitz continuous gradient. This result implies that the trajectories of stochastic gradient dynamics converge to a neighborhood of the optimum with high probability, with the size of the neighborhood determined by the noise covariance. Moreover, if the K-PL condition is strengthened to a K_∞-PL condition, the dynamics are NSS; whereas if it is weakened to a general positive-definite-PL condition, the dynamics exhibit integral NSS. The results further extend to objectives without globally Lipschitz gradients through appropriate step-size tuning. The proposed framework is further applied to the robustness analysis of policy optimization for the linear quadratic regulator (LQR) and logistic regression.

Solutions of optimization problems, including policy optimization in reinforcement learning, typically rely upon some variant of gradient descent. There has been much recent work in the machine learning, control, and optimization communities applying the Polyak-Łojasiewicz Inequality (PLI) to such problems in order to establish an exponential rate of convergence (a.k.a. ``linear convergence'' in the local-iteration language of numerical analysis) of loss functions to their minima under the gradient flow. Often, as is the case of policy iteration for the continuous-time LQR problem, this rate vanishes for large initial conditions, resulting in a mixed globally linear / locally exponential behavior. This is in sharp contrast with the discrete-time LQR problem, where there is global exponential convergence. That gap between CT and DT behaviors motivates the search for various generalized PLI-like conditions, and this paper addresses that topic. Moreover, these generalizations are key to understanding the transient and asymptotic effects of errors in the estimation of the gradient, errors which might arise from adversarial attacks, wrong evaluation by an oracle, early stopping of a simulation, inaccurate and very approximate digital twins, stochastic computations (algorithm ``reproducibility''), or learning by sampling from limited data. We describe an ``input to state stability'' (ISS) analysis of this issue. We also discuss convergence and PLI-like properties of ``linear feedforward neural networks'' in feedback control. Much of the work described here was done in collaboration with Arthur Castello B. de Oliveira, Leilei Cui, Zhong-Ping Jiang, and Milad Siami. This is a short paper summarizing the slides presented at my keynote at the 2025 L4DC (Learning for Dynamics & Control Conference) in Ann Arbor, Michigan, 05 June 2025. A partial bibliography has been added.

When measuring importance of nodes in a network, the interconnections and dynamics are often supposed to be perfectly known. In this paper, we consider networks of agents with both uncertain couplings and dynamics. Network uncertainty is modeled by structured additive stochastic disturbances on each agent's update dynamics and coupling weights. We then study how these uncertainties change the network's centralities. Disturbances on the couplings between agents resul in bilinear dynamics, and classical centrality indices from linear network theory need to be redefined. To do that, we first show that, similarly to its linear counterpart, the squared H2 norm of bilinear systems measures the trace of the steady-state error covariance matrix subject to stochastic disturbances. This makes the H2 norm a natural candidate for a performance metric of the system. We propose a centrality index for the agents based on the H2 norm, and show how it depends on the network topology and the noise structure. Finally, we simulate a few graphs to illustrate how uncertainties on different couplings affect the agents' centrality rankings compared to a linearized model of the same system.

Long-term behaviors of biochemical reaction networks (BRNs) are described by steady states in deterministic models and stationary distributions in stochastic models. Unlike deterministic steady states, stationary distributions capturing inherent fluctuations of reactions are extremely difficult to derive analytically due to the curse of dimensionality. Here, we develop a method to derive analytic stationary distributions from deterministic steady states by transforming BRNs to have a special dynamic property, called complex balancing. Specifically, we merge nodes and edges of BRNs to match in- and out-flows of each node. This allows us to derive the stationary distributions of a large class of BRNs, including autophosphorylation networks of EGFR, PAK1, and Aurora B kinase and a genetic toggle switch. This reveals the unique properties of their stochastic dynamics such as robustness, sensitivity, and multimodality. Importantly, we provide a user-friendly computational package, CASTANET, that automatically derives symbolic expressions of the stationary distributions of BRNs to understand their long-term stochasticity.

Cells respond to biochemical and physical internal as well as external signals. These signals can be broadly classified into two categories: (a) ``actionable'' or ``reference'' inputs that should elicit appropriate biological or physical responses such as gene expression or motility, and (b) ``disturbances'' or ``perturbations'' that should be ignored or actively filtered-out. These disturbances might be exogenous, such as binding of nonspecific ligands, or endogenous, such as variations in enzyme concentrations or gene copy numbers. In this context, the term robustness describes the capability to produce appropriate responses to reference inputs while at the same time being insensitive to disturbances. These two objectives often conflict with each other and require delicate design trade-offs. Indeed, natural biological systems use complicated and still poorly understood control strategies in order to finely balance the goals of responsiveness and robustness. A better understanding of such natural strategies remains an important scientific goal in itself and will play a role in the construction of synthetic circuits for therapeutic and biosensing applications. A prototype problem in robustly responding to inputs is that of ``robust tracking'', defined by the requirement that some designated internal quantity (for example, the level of expression of a reporter protein) should faithfully follow an input signal while being insensitive to an appropriate class of perturbations. Control theory predicts that a certain type of motif, called integral feedback, will help achieve this goal, and this motif is, in fact, a necessary feature of any system that exhibits robust tracking. Indeed, integral feedback has always been a key component of electrical and mechanical control systems, at least since the 18th century when James Watt employed the centrifugal governor to regulate steam engines. Motivated by this knowledge, biological engineers have proposed various designs for biomolecular integral feedback control mechanisms. However, practical and quantitatively predictable implementations have proved challenging, in part due to the difficulty in obtaining accurate models of transcription, translation, and resource competition in living cells, and the stochasticity inherent in cellular reactions. These challenges prevent first-principles rational design and parameter optimization. In this work, we exploit the versatility of an Escherichia coli cell-free transcription-translation (TXTL) to accurately design, model and then build, a synthetic biomolecular integral controller that precisely controls the expression of a target gene. To our knowledge, this is the first design of a functioning gene network that achieves the goal of making gene expression track an externally imposed reference level, achieves this goal even in the presence of disturbances, and whose performance quantitatively agrees with mathematical predictions.

In biological processes such as embryonic development, hematopoietic cell differentiation, and the arising of tumor heterogeneity and consequent resistance to therapy, mechanisms of gene activation and deactivation may play a role in the emergence of phenotypically heterogeneous yet genetically identical (clonal) cellular populations. Mathematically, the variability in phenotypes in the absence of genetic variation can be modeled through the existence of multiple metastable attractors in nonlinear systems subject with stochastic switching, each one of them associated to an alternative epigenetic state. An important theoretical and practical question is that of estimating the number and location of these states, as well as their relative probabilities of occurrence. This paper focuses on a rigorous analytic characterization of multiple modes under slow promoter kinetics, which is a feature of epigenetic regulation. It characterizes the stationary distributions of Chemical Master Equations for gene regulatory networks as a mixture of Poisson distributions. As illustrations, the theory is used to tease out the role of cooperative binding in stochastic models in comparison to deterministic models, and applications are given to various model systems, such as toggle switches in isolation or in communicating populations and a trans-differentiation network.

Cellular reprogramming is traditionally accomplished through an open loop control approach, wherein key transcription factors are injected in cells to steer a gene regulatory network toward a pluripotent state. Recently, a closed loop feedback control strategy was proposed in order to achieve more accurate control. Previous analyses of the controller were based on deterministic models, ignoring the substantial stochasticity in these networks, Here we analyze the Chemical Master Equation for reaction models with and without the feedback controller. We computationally and analytically investigate the performance of the controller in biologically relevant parameter regimes where stochastic effects dictate system dynamics. Our results indicate that the feedback control approach still ensures reprogramming even when analyzed using a stochastic model.

In the mathematical modeling of cell differentiation, it is common to think of internal states of cells (quanfitied by activation levels of certain genes) as determining different cell types. We study here the "PU.1/GATA-1 circuit" that controls the development of mature blood cells from hematopoietic stem cells (HSCs). We introduce a rigorous chemical reaction network model of the PU.1/GATA-1 circuit, which incorporates current biological knowledge and find that the resulting ODE model of these biomolecular reactions is incapable of exhibiting multistability, contradicting the fact that differentiation networks have, by definition, alternative stable steady states. When considering instead the stochastic version of this chemical network, we analytically construct the stationary distribution, and are able to show that this distribution is indeed capable of admitting a multiplicity of modes. Finally, we study how a judicious choice of system parameters serves to bias the probabilities towards different stationary states. We remark that certain changes in system parameters can be physically implemented by a biological feedback mechanism; tuning this feedback gives extra degrees of freedom that allow one to assign higher likelihood to some cell types over others.

The study of stochastic biomolecular networks is a key part of systems biology, as such networks play a central role in engineered synthetic biology constructs as well as in naturally occurring cells. This expository paper reviews in a unified way a pair of recent approaches to the finite computation of statistics for chemical reaction networks.

Biochemical reaction networks in cells frequently consist of reactions with disparate timescales. Stochastic simulations of such multiscale BRNs are prohibitively slow due to the high computational cost incurred in the simulations of fast reactions. One way to resolve this problem is to replace fast species by their stationary conditional expectation values conditioned on slow species. While various approximations schemes for this quasi-steady state approximation have been developed, they often lead to considerable errors. This paper considers two classes of multiscale BRNs which can be reduced by through an exact QSS rather than approximations. Specifically, we assume that fast species constitute either a feedforward network or a complex balanced network. Exact reductions for various examples are derived, and the computational advantages of this approach are illustrated through simulations.

We review in a unified way results for two types of stochastic chemical reaction systems for which moments can be effectively computed: feedforward networks and complex-balanced networks.

A formalism for the study of random dynamical systems with inputs and outputs (RDSIO) is introduced. An axiomatic framework and basic properties of RDSIO are developed, and a theorem is shown that guarantees the stability of interconnected systems.

Reverse engineering of biological pathways involves an iterative process between experiments, data processing, and theoretical analysis. In this work, we engineer synthetic circuits, subject them to perturbations, and then infer network connections using a combination of nonparametric single-cell data resampling and modular response analysis. Intriguingly, we discover that recovered weights of specific network edges undergo divergent shifts under differential perturbations, and that the particular behavior is markedly different between different topologies. Investigating topological changes under differential perturbations may address the longstanding problem of discriminating direct and indirect connectivities in biological networks.

Chemical systems are inherently stochastic, as reactions depend on random (thermal) motion. This motivates the study of stochastic models, and specifically the Chemical Master Equation (CME), a discrete-space continuous-time Markov process that describes stochastic chemical kinetics. Exact studies using the CME are difficult, and several moment closure tools related to "mass fluctuation kinetics" and "fluctuation-dissipation" formulas can be used to obtain approximations of moments. This paper, in contrast, introduces a class of nonlinear chemical reaction networks for which exact computation is possible, by means of finite-dimensional linear differential equations. This class allows second and higher order reactions, but only under special assumptions on structure and/or conservation laws.

This conference paper (a) summarizes material from "A fundamental limitation to fold-change detection by biological systems with multiple time scales" (IET Systems Biology 2014) and presents additional remarks regarding (b) expansion techniques to compute FCD error and (c) stochastic adaptation and FCD

This paper deals with the stability of interconnections of nonlinear stochastic systems, using concepts of passivity and noise-to-state stability.

Conference version of ``Stability certification of large scale stochastic systems using dissipativity of subsystems''.

Multivariate Poisson random variables subject to linear integer constraints arise in several application areas, such as queuing and biomolecular networks. This note shows how to compute conditional statistics in this context, by employing WZ Theory and associated algorithms. A symbolic computation package has been developed and is made freely available. A discussion of motivating biomolecular problems is also provided.

Interactions between genes and gene products give rise to complex circuits that enable cells to process information and respond to external signals. Theoretical studies often describe these interactions using continuous, stochastic, or logical approaches. Here we propose a framework for gene regulatory networks that combines the intuitive appeal of a qualitative description of gene states with a high flexibility in incorporating stochasticity in the duration of cellular processes. We apply our methods to the regulatory network of the segment polarity genes, thus gaining novel insights into the development of gene expression patterns. For example, we show that very short synthesis and decay times can perturb the wild type pattern. On the other hand, separation of timescales between pre- and post-translational processes and a minimal prepattern ensure convergence to the wild type expression pattern regardless of fluctuations.

This paper deals with sparse approximations by means of convex combinations of elements from a predetermined "basis" subset S of a function space. Specifically, the focus is on the rate at which the lowest achievable error can be reduced as larger subsets of S are allowed when constructing an approximant. The new results extend those given for Hilbert spaces by Jones and Barron, including in particular a computationally attractive incremental approximation scheme. Bounds are derived for broad classes of Banach spaces. The techniques used borrow from results regarding moduli of smoothness in functional analysis as well as from the theory of stochastic processes on function spaces.