Accelerated primal-dual methods with enlarged step sizes and operator learning for nonsmooth optimal control problems - Song, Yongcun; Yuan, Xiaoming; Yue, Hangrui (2024)
We consider a general class of nonsmooth optimal control problems with partial differential equation (PDE) constraints, which are very challenging due to the nonsmooth objective functionals and the resulting high-dimensional and ill-conditioned systems after discretization. We focus on the application of a primal-dual method, with which different types of variables can be treated individually and thus its main computation at each iteration only requires solving two PDEs. Our target is to accelerate the primal-dual method with either larger step sizes or operator learning techniques. For the accelerated primal-dual method with larger step sizes, its convergence can be still proved rigorously while it numerically accelerates the original primal-dual method in a simple and universal way. For the operator learning acceleration, we construct deep neural network surrogate models for the involved PDEs. Once a neural operator is learned, solving a PDE requires only a forward pass of the neural network, and the computational cost is thus substantially reduced. The accelerated primal-dual method with operator learning is mesh-free, numerically efficient, and scalable to different types of PDEs. The acceleration effectiveness of these two techniques is promisingly validated by some preliminary numerical results. <br
https://cris.fau.de/converis/publicweb/Publication/307431606

An ADMM numerical approach to linear parabolic state constrained optimal control problems - Song, Yongcun; et al. (Numerische Mathematik, 2020)
Optimal control problems arising from systems modeled by linear parabolic equations may be difficult for both theoretical analysis and algorithmic design. For the case where there are additional constraints on the state variables, restrictive regularity assumptions are usually required to guarantee the existence of the associated Lagrange multiplier and thus some regularization type methods such as the Moreau-Yosida and Lavrentiev methods have been discussed in the literature. In this article, we study the application of the alternating direction method of multipliers (ADMM) to linear parabolic state constrained optimal control problems, and propose an ADMM numerical approach. We prove the convergence of the ADMM algorithm without any existence or regularity assumption on the Lagrange multiplier, and estimate its worst-case convergence rate in both the ergodic and nonergodic senses. An important feature of the ADMM approach is that it decouples the state constraints and the parabolic optimal control problems inside each iteration. We show the efficiency of the ADMM approach by testing some control problems in two space dimensions.
Keywords: 49J20; 90C25; 65K05

An alternating direction method of multipliers for elliptic equation constrained optimization problem - Song, Yongcun; et al. (Science China-Mathematics, 2017)
We propose an alternating direction method of multipliers (ADMM) for solving the state constrained optimization problems governed by elliptic equations. The unconstrained as well as box-constrained cases of the Dirichlet boundary control, Robin boundary control, and right-hand side control problems are considered here. These continuous optimization problems are transformed into discrete optimization problems by the finite element method discretization, then are solved by ADMM. The ADMM is an efficient first order algorithm with global convergence, which combines the decomposability of dual ascent with the superior convergence properties of the method of multipliers. We shall present exhaustive convergence analysis of ADMM for these different type optimization problems. The numerical experiments are performed to verify the efficiency of the method.
Keywords: ADMM; optimal control problems; elliptic equation constrained

An Augmented Lagrangian Method for State Constrained Linear Parabolic Optimal Control Problems - Wang, Hailing; Yu, Changjun; Song, Yongcun (Journal of Optimization Theory and Applications, 2024)
In this paper, we consider a class of state constrained linear parabolic optimal control problems. Instead of treating the inequality state constraints directly, we reformulate the problem as an equality-constrained optimization problem, and then apply the augmented Lagrangian method (ALM) to solve it. We prove the convergence of the ALM without any existence or regularity assumptions on the corresponding Lagrange multipliers, which is an essential complement to the classical theoretical results for the ALM because restrictive regularity assumptions are usually required to guarantee the existence of the Lagrange multipliers associated with the state constraints. In addition, under an appropriate choice of penalty parameter sequence, we can obtain a super-linear non-ergodic convergence rate for the ALM. Computationally, we apply a semi-smooth Newton (SSN) method to solve the ALM subproblems and design an efficient preconditioned conjugate gradient method for solving the Newton systems. Some numerical results are given to illustrate the effectiveness and efficiency of our algorithm.
CRIS-Team Scopus Importer:2024-07-26
Keywords: Augmented Lagrangian method; Convergence analysis; Convergence rate; Optimal control problem; State constraints

AN INEXACT UZAWA ALGORITHMIC FRAMEWORK FOR NONLINEAR SADDLE POINT PROBLEMS WITH APPLICATIONS TO ELLIPTIC OPTIMAL CONTROL PROBLEM - Song, Yongcun; et al. (SIAM Journal on Numerical Analysis, 2019)
We consider a class of nonlinear saddle point problems with various applications in PDEs and optimal control problems and propose an algorithmic framework based on some inexact Uzawa methods in the literature. Under mild conditions, the convergence of this algorithmic framework is uniformly proved and the linear convergence rate is estimated. We take an elliptic optimal control problem with control constraints as an example to illustrate how to choose application-tailored preconditioners to generate specific and efficient algorithms by the algorithmic framework. The resulting algorithm does not need to solve any optimization subproblems or systems of linear equations in its iteration; each of its iterations only requires the projection onto a simple admissible set, four algebraic multigrid V-cycles, and a few matrix-vector multiplications. Its numerical efficiency is then demonstrated by some preliminary numerical results.
Keywords: nonlinear saddle point problems; inexact Uzawa method; convergence analysis; subregularity; linear convergence rate; elliptic optimal control problem

A numerical approach to the optimal control of thermally convective flows - Song, Yongcun; Yuan, Xiaomig; Yue, Hangrui (Journal of Computational Physics, 2023)
The optimal control of thermally convective flows is usually modeled by an optimization problem with constraints of Boussinesq equations that consist of the Navier-Stokes equation and an advection-diffusion equation. This optimal control problem is challenging from both theoretical analysis and algorithmic design perspectives. For example, the nonlinearity and coupling of fluid flows and energy transports prevent direct applications of gradient type algorithms in practice. In this paper, we propose an efficient numerical method to solve this problem based on the operator splitting and optimization techniques. In particular, we employ the Marchuk-Yanenko method leveraged by the L^2-projection for the time discretization of the Boussinesq equations so that the Boussinesq equations are decomposed into some easier linear equations without any difficulty in deriving the corresponding adjoint system. Consequently, at each iteration, four easy linear advection-diffusion equations and two degenerated Stokes equations at each time step are needed to be solved for computing a gradient. Then, we apply the Bercovier-Pironneau finite element method for space discretization, and design a BFGS type algorithm for solving the fully discretized optimal control problem. We look into the structure of the problem, and design a meticulous strategy to seek step sizes for the BFGS efficiently. Efficiency of the numerical approach is promisingly validated by the results of some preliminary numerical experiment

Approximate and Weighted Data Reconstruction Attack in Federated Learning - Wang, Ziqi; Song, Yongcun; Zuazua Iriondo, Enrique (2024)
Federated Learning (FL) is a distributed learning paradigm that enables multiple clients to collaborate on building a machine learning model without sharing their private data. Although FL is considered privacy-preserved by design, recent data reconstruction attacks demonstrate that an attacker can recover clients' training data based on the parameters shared in FL. However, most existing methods fail to attack the most widely used horizontal Federated Averaging (FedAvg) scenario, where clients share model parameters after multiple local training steps. To tackle this issue, we propose an interpolation-based approximation method, which makes attacking FedAvg scenarios feasible by generating the intermediate model updates of the clients' local training processes. Then, we design a layer-wise weighted loss function to improve the data quality of reconstruction. We assign different weights to model updates in different layers concerning the neural network structure, with the weights tuned by Bayesian optimization. Finally, experimental results validate the superiority of our proposed approximate and weighted attack (AWA) method over the other state-of-the-art methods, as demonstrated by the substantial improvement in different evaluation metrics for image data reconstructions.<br
https://cris.fau.de/converis/publicweb/Publication/310887833

A Two-Stage Numerical Approach for the Sparse Initial Source Identification of a Diffusion-Advection Equation - Biccari, Umberto; Song, Yongcun; Yuan, Xiaomin; Zuazua Iriondo, Enrique (Inverse Problems, 2023)
We consider the problem of identifying a sparse initial source condition to achieve a given state distribution of a diffusion-advection partial differential equation after a given final time. The initial condition is assumed to be a finite combination of Dirac measures. The locations and intensities of this initial condition are required to be identified. This problem is known to be exponentially ill-posed because of the strong diffusive and smoothing effects. We propose a two-stage numerical approach to treat this problem. At the first stage, to obtain a sparse initial condition with the desire of achieving the given state subject to a certain tolerance, we propose an optimal control problem involving sparsity-promoting and ill-posedness-avoiding terms in the cost functional, and introduce a generalized primal-dual algorithm for this optimal control problem. At the second stage, the initial condition obtained from the optimal control problem is further enhanced by identifying its locations and intensities in its representation of the combination of Dirac measures. This two-stage numerical approach is shown to be easily implementable and its efficiency in short time horizons is promisingly validated by the results of numerical experiments. Some discussions on long time horizons are also included.<br
Keywords: initial source identification; inverse problem; optimal control; sparse control; diffusion-advection equations; non-smooth optimization; primal-dual algorithm

FedADMM-InSa: An Inexact and Self-Adaptive ADMM for Federated Learning - Song, Yongcun; Wang, Ziqi; Zuazua Iriondo, Enrique (Neural Networks, 2025)
Federated learning (FL) is a promising framework for learning from distributed data while maintaining privacy. The development of efficient FL algorithms encounters various challenges, including heterogeneous data and systems, limited communication capacities, and constrained local computational resources. Recently developed FedADMM methods show great resilience to both data and system heterogeneity. However, they still suffer from performance deterioration if the hyperparameters are not carefully tuned. To address this issue, we propose an inexact and self-adaptive FedADMM algorithm, termed FedADMM-InSa. First, we design an inexactness criterion for the clients' local updates to eliminate the need for empirically setting the local training accuracy. This inexactness criterion can be assessed by each client independently based on its unique condition, thereby reducing the local computational cost and mitigating the undesirable straggle effect. The convergence of the resulting inexact ADMM is proved under the assumption of strongly convex loss functions. Additionally, we present a self-adaptive scheme that dynamically adjusts each client's penalty parameter, enhancing algorithm robustness by mitigating the need for empirical penalty parameter choices for each client. Extensive numerical experiments on both synthetic and real-world datasets have been conducted. As validated by some tests, our FedADMM-InSa algorithm improves model accuracy by 7.8% while reducing clients' local workloads by 55.7% compared to benchmark algorithm
CRIS-Team Scopus Importer:2024-10-25
Keywords: Machine Learning; Federated learning; Distributed; Parallel; and Cluster Computing; Optimization and Control

Lattice structure design optimization under localized linear buckling constraints - Li, Ming; Song, Yongcun; Yang, Xingtong; Zhang, Kai (Computers & Structures, 2023)
An optimization method for the design of multi-lattice structures satisfying local buckling constraints is proposed in this study. In order to resolve the highly nonlinear large-scale optimization problem, a consecutive numerical approach is devised: macro-field optimization and microstructure embedding. The macro-field optimization finds an optimal elasticity tensor distribution among all feasible elastic continua based on a concept of free material optimization (FMO), which largely extends the design space compared with density-based topology optimization. After this, by an inverse design approach that approximates the elasticity tensor under local buckling constraint, an appropriate lattice structure is able to be embedded within each macro-element, where the local stress tensors are considered in particular to produce more reasonable microstructures. During the process, a machine learning approach is also devised to significantly reduce the number of material/lattice types and thus the overall computational costs. Ultimately, a lattice structure under requirements of overall stiffness and local buckling resistance is produced, as demonstrated via numerical examples.
CRIS-Team Scopus Importer:2024-12-16
Keywords: Free material optimization; Inverse homogenization; Lattice structure; Local buckling mode; Topology optimization

THE ADMM-PINNS ALGORITHMIC FRAMEWORK FOR NONSMOOTH PDE-CONSTRAINED OPTIMIZATION: A DEEP LEARNING APPROACH - Song, Yongcun; Yuan, Xiaoming; Yue, Hangrui (SIAM Journal on Scientific Computing, 2024)
We study the combination of the alternating direction method of multipliers (ADMM) with physics-informed neural networks (PINNs) for a general class of nonsmooth partial differential equation (PDE)-constrained optimization problems, where additional regularization can be employed for constraints on the control or design variables. The resulting ADMM-PINNs algorithmic framework substantially enlarges the applicable range of PINNs to nonsmooth cases of PDE-constrained optimization problems. The application of the ADMM makes it possible to separate the PDE constraints and the nonsmooth regularization terms for iterations. Accordingly, at each iteration, one of the resulting subproblems is a smooth PDE-constrained optimization which can be efficiently solved by PINNs, and another is a simple nonsmooth optimization problem, which usually has a closed-form solution or can be efficiently solved by various standard optimization algorithms or pretrained neural networks. The ADMM-PINNs algorithmic framework does not require one to solve PDEs repeatedly, and it is mesh-free, easy to implement, and scalable to different PDE settings. We validate the efficiency of the ADMM-PINNs algorithmic framework by different prototypical applications, including inverse potential problems, source identification in elliptic equations, control constrained optimal control of the Burgers equation, and sparse optimal control of parabolic equations.
CRIS-Team Scopus Importer:2025-02-04
Keywords: ADMM; deep learning; nonsmooth optimization; PDE-constrained optimization; physics-informed neural networks

The hard-constraint PINNs for interface optimal control problems - Lai, Ming-Chih; Song, Yongcun; Yuan, Xiaoming; Yue, Hangrui; Zeng, Tianyou (2024)
We show that the physics-informed neural networks (PINNs), in combination with some recently developed discontinuity capturing neural networks, can be applied to solve optimal control problems subject to partial differential equations (PDEs) with interfaces and some control constraints. The resulting algorithm is mesh-free and scalable to different PDEs, and it ensures the control constraints rigorously. Since the boundary and interface conditions, as well as the PDEs, are all treated as soft constraints by lumping them into a weighted loss function, it is necessary to learn them simultaneously and there is no guarantee that the boundary and interface conditions can be satisfied exactly. This immediately causes difficulties in tuning the weights in the corresponding loss function and training the neural networks. To tackle these difficulties and guarantee the numerical accuracy, we propose to impose the boundary and interface conditions as hard constraints in PINNs by developing a novel neural network architecture. The resulting hard-constraint PINNs approach guarantees that both the boundary and interface conditions can be satisfied exactly and they are decoupled from the learning of the PDEs. Its efficiency is promisingly validated by some elliptic and parabolic interface optimal control problem
https://cris.fau.de/converis/publicweb/Publication/316341855
Keywords: Optimal control; interface problems; physics-informed neural networks; discontinuity capturing neural networks; hard constraints