学习目标：

• 如何构建神经网络模型，来近似模型chaos系统
• 是否chaos系统可以通过AI模型被预测？比如一些critical pattern？
• 如何将AI模型与系统的稳定性相互关联？
• chaos理论的一些思考和机器学习的一些共性？

Challenges and Opportunities for Machine Learning in Fluid Dynamics5

Fluid dynamics presents challenges that differ from those tackled in many applications of machine learning, such as image recognition and advertising. In fluid flows it is often important to precisely quantify the underlying physical mechanisms in order to analyze them. Furthermore, fluids flows entail complex, multi-scale phenomena whose understanding and control remain to a large extent unresolved. Unsteady flow fields require algorithms capable of addressing nonlinearities and multiple spatiotemporal scales that may not be present in popular machine learning algorithms. In addition, many prominent applications of machine learning, such as playing video games, rely on inexpensive system evaluations and an exhaustive categorization of the process that must be learned. This is not the case in fluids, where experiments may be difficult to repeat or automate and where simulations may require large-scale supercomputers operating for extended periods of time.

Dimensionality reduction : POD, PCA and auto-encoders

Autoencoder: A neural network architecture used to compress and decompress high-dimensional data. They are powerful alternatives to the Proper Orthogonal Decomposition (POD).

PDE学习10

PDE可以统一写成：

$\xi$便是所要求的系数。

$\Theta$矩阵难免包含数值和测量误差，这里采用sparse regression来求解：

$l_0$ 问题为n-p hard，通过convex relaxation 为$l_1$, 或者 sequentially thresholded least squares(STLS) 为 $l_2$:

Numerical Differentition

Use a Gaussian smoothing kernel on noisy data prior to taking derivatives with finite differences. Convolve with a Gaussian.

Tikhonov differentiation finds a numerical derivative $\hat{f}’$ by balancing the closeness of the integral of $\hat{f}’$ to f with the smoothness of $\hat{f}’$.

In the discrete problem, A is a trapezoidal approximation to the integral and D is a finite approximation to the derivative. The problem has closed form solution given by,

PDE-FIND for Burger’s Equation

Samuel Rudy, 2016

This notebook demonstrates PDE-FIND on Burger’s equation with an added diffusive term.

The solution given is a single travelling wave, starting out as a Gaussian.

Populating the interactive namespace from numpy and matplotlib

(256, 101)
(256,)
(101,)

Text(0.5, 0, 't')


Construct $\Theta (U)$ and compute $U_t$

The function build_linear_system does this for us. We specify

D = highest derivative to appear in $\Theta$

P = highest degree polynomial of $u$ to appear in $\Theta$ (not including multiplication by a derivative.

time_diff and space_diff taken via finite differences

Printed out is a list of candidate functions for the PDE. Each is a column of $\Theta (U)$

['1',
'u',
'u^2',
'u^3',
'u_{x}',
'uu_{x}',
'u^2u_{x}',
'u^3u_{x}',
'u_{xx}',
'uu_{xx}',
'u^2u_{xx}',
'u^3u_{xx}',
'u_{xxx}',
'uu_{xxx}',
'u^2u_{xxx}',
'u^3u_{xxx}']


Solve for $\xi$

TrainSTRidge splits the data up into 80% for training and 20% for validation. It searches over various tolerances in the STRidge algorithm and finds the one with the best performance on the validation set, including an $\ell^0$ penalty for $\xi$ in the loss function.

PDE derived using STRidge
u_t = (-1.000987 +0.000000i)uu_{x}
+ (0.100220 +0.000000i)u_{xx}

Error using PDE-FIND to identify Burger's equation:

Mean parameter error: 0.15935000000000255 %
Standard deviation of parameter error: 0.06064999999999543 %


Now identify the same dynamics but with added noise.

The only difference from above is that finite differences work poorly for noisy data so here we use polynomial interpolation. With deg_x or deg_t and width_x or width_t we specify the degree number of points used to fit the polynomials used for differentiating x or t. Unfortunately, the result can be sensitive to these.

PDE derived using STRidge
u_t = (-1.009655 +0.000000i)uu_{x}
+ (0.102966 +0.000000i)u_{xx}

Error using PDE-FIND to identify Burger's equation with added noise:

Mean parameter error: 1.9657499999999966 %
Standard deviation of parameter error: 1.0002499999999996 %


ML-embedded Universal Differential Equations11

https://github.com/ChrisRackauckas/universal_differential_equations

Neural ODE:

The Universal Approximation Theorem (UAT) demonstrates that sufficiently large neural networks can approximate any nonlinear function with a finite set of parameters.

Universe ordinary differential equation (UODE):

a known mechanistic model form f with missing terms defined by some universal approximation(UA) $U_\theta$.

the cost function $C(\theta)$ by Euclidean distance:

Automated Identification of Nonlinear Interactions

Extend SInDy algorithm(类似time-delay DMD with sparse)， the idea is to find a sparse basis over a given candidate library minimizing the objective function using data.11

UDE approach to extend SinDy that embeds prior structural knowledge.

Take Lots-Volterra system as example:

a system of ordinary differential equations that incorporates the known structure but leaves room for learning unknown interactions between the the predator and prey populations.

0 1 0
1 -4 1
0 1 0

MIT-climate

Scaling the software to “real” problems

Neural ODE with batching on the GPU (without internal data transfers) with high order adaptive implicit ODE solvers for stiff equations using matrix-free Newton-Krylov via preconditioned GMRES and trained using checkpointed adjoint equations. See Yingbo Ma’s talk.

ROM machine learning-燃烧模拟-加速模拟

• 首先对数据的认识非常重要，deeply understand the problem you working with,
• 机器学习算法保证不变量
• 预测chaostis是悬而未决的

https://github.com/Willcox-Research-Group?language=python

Lift & Learn approach

• Full model

Example:

1. Incompressible N-S
2. Euler equations can be transformed to quadratic form by using pressure, velocity.
3. A nonlinear tubular reactor model
• ROM model：

Where, $c_r = V^Tc, A_r = V^TAV, H_r=V^TH(V\otimes V), B_r= V^TB$

• Learn model fram data (data driven regression approach)Solve least-squares problem:（最小二乘法）最终，将上式表达为：Thus to compute the ROM operators $\hat{c}, \hat{A}, \hat{H}, \hat{B}$ without explicit acess to the original high-dimensional opertor $c, A, H, B$.
• Tikhonov Regularization to alleviate overfitting the operators to the data 参考$\Gamma$ is a full-rank regularizer.

L2 regularizer $\Gamma = \lambda I$, deriving the ROM toward the globally stable system ${d\over dt}\hat{q}(t)=0$.

• Combustion dynamics$\vec{K}$ the inviscid flux terms, $\vec{K}_v$ the viscous flux terms, $\vec{S}$ is the source terms. $\rho$ density, $Y_l$ is the species mass fraction. CH4 + 2 O2 —> 2 H2O+ CO2.

FLOW MODELING WITH MACHINE LEARNING

• dimensionality reduction

Dimensionality reduction involves extracting key features and dominant patterns that may be used as reduced coordinates where the fluid is compactly and efficiently described (Taira et al. 2017). 参见5

• Detecting von Karman-type wakes with hydrodynamic sensors6

Machine Learning Based Detection of Flow Disturbances Using Surface Pressure Measurements7

the coordinate along the plate：

Kelvin’s 环量守恒：

LSEP和前缘分离涡初始点之间的联系

“This question may appear superficially simple to address since the LESP itself is due directly to pressure, and in fact—as we will discuss below—is proportional to the lowest Fourier mode on the plate surface. However, it is important to remember that in the high-amplitude cases of interest in this work, the flow response (and the surface pressure) is non-linearly dependent on the disturbed critical LESP, so that the surface pressures along the airfoil chord are not trivially related to the leading-edge suction.”

The CNN contains 16 separate filters,each utilizing a 5×5 tile of weights, so that the output of this layer is of size $R^{16 126 201}$

learned the functional relationship betweensurface pressures and the LESP and the angle of attack histories by machine-learned systemidentification(MLSID).

assume that the dynamics of the angle ofattack can be described by a linear system of first order differential equations with forcing:

Applying the EnKF to a vortex model9

The nonlinear state transition function $f_k$ propagates the state from step k-1 to step k by the following methods:

(1) computing the bound vortex sheet strength on the plate from the no-penetration condition, using the current vortex blob positions and strength;

(2) computing the volocities of the vortex blobs;

(3) advecting the plate and the vortex blobs;

(4) applying the vortex aggregation algorithm, reducing the strength of any aggregated blobs to zero instead of completely removing the blob;

(5) releasing a new vortex blob from each edge of the plate with strengths based on the Kutta condition at the trailing edge and the current estimate of the LESP_c at the leading edge.

Deep neural networks for waves assisted by the Wiener–Hopf method 学习笔记-黄迅

10000 items: $\psis$ of various (m,n) with random amplitudes $A{mn}$ ,

笔记2: 针对有物理限制的案例

$\mathcal{B}$ 是PDE的边界条件。

Darcy flow：

$\lambda$ is the weight (Lagrange multiplier) to softly enforce the boundary conditions.

参考

1. [2019 SIAM Conference on Applications of Dynamical Systems
2. https://github.com/pvlachas/RNN-RC-Chaos
4. Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics
5. Machine Learning for Fluid Mechanics
6. Detecting exotic wakes with hydrodynamic sensors Mengying Wang and Maziar S. Hemati
7. AIAA conference-2019,Machine Learning Based Detection of Flow Disturbances Using Surface Pressure Measurements Wei Hou∗1 , Darwin Darakananda†1 , and Jeff D. Eldredge‡1
8. Kiran Ramesh, Ashok Gopalarathnam, Kenneth Granlund, Michael V. Ol, and Jack R. Edwards. Discrete-vortex method with novel shedding criterion for unsteady aerofoil flows with intermittent leading-edge vortex shedding. Journal of Fluid Mechanics, 751:500–538, 2014
9. Data-assimilated low-order vortex modeling of separated flows,PHYSICAL REVIEW FLUIDS 3, 124701 (2018)
10. 2017- APPLIED MATHEMATICS Data-driven discovery of partial differential equations Samuel H. Rudy1,*, Steven L. Brunton2, Joshua L. Proctor3 and J. Nathan Kutz1
11. Universal Differential Equations for Scientific Machine Learning Christopher Rackauckasa - 2020-10-10