### 学习资源:

### Numpy

- Matrix arithmetic
- Tensor (aka. “arrays”)(note the name: Tensorflow)
- 1-D tensor(vector), 2-D tensor(matix)
- Arithmetic: +,-,*,/
- Matrix multiply == dot/inner product (np.dot)
- Element-wise multiply(*)

### Matplotlib

- Line charts
- Scatter plots

### Pandas

- Loading in tabular data

### Scipy(power version of numpy)

- Satistics, optimization, linear algebra, signal processing

### Scikit-Learn

- basic machine learning
- Classification and regression
- Machine learning as geometry rather than magic
- Shape of data(X and Y)

### The Steps of a Machine Learning Script

- Load in the data
- X and Y
- Typically use Pandas unless the data is to complex

- Split into train/test sets (sometimes)
- Sometimes “test” and “validation” are used interchangebly, and the “true test set” is something else- don’t get frazzled over this.

- Build a model
- OOP (object oriented programming)
- Tensorflow 2.0 standard is Keras API, very similar to Scikit-learn

- Fit the model(gradinet descent)
- model.fit(X,Y)

- Evaluate the model
- Make predictions
- model.predict(X)

## Outline

### What is machine learning

look for patterns for data and make better prediction in furture.

traning a nueral network is just to find a weight of minimum of cost function.

### Classfication and Regression

Classfication:

- predict a category/ lable
- yes or not / dog or cat
- SVM

Regression: (more features,nonlinear makes problem complicated)

- Fit a line or curve

*ŷ* = *mx* + *b*

The Decision Rule

- sigmoid functions

*ŷ* = *σ*(*a*), *a* = *w*1*x*1 + *w*2*x*2 + *b*

Probabilistic Interpretation

*p*(*y* = 1|*x*) = *σ*(*a*)

Logistic Regression with >2 Inputs

- What if we have > 2 input?
- Use matrices/ vectors

$p(y= 1| x)=\sigma(w^T x + b)= \sigma (\sum_{d=1}^D w_d x_d +b)$

**Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.**

Those say that $\triangle output$ is a linear function of the changes $\triangle w_j$ and $\triangle b$ in the weights and bias. make it much easier to figure out how changing the weights and biases will change the output.

sigmoid function is defined by:

ReLU may be even more biological plausible than sigmoid;

In Tnesorflow 2.0

- Is mostly an exercise in “API hunting”
- The actual math isn’t written in code (by you)
- Your job is to find the function that implements the math in question
- tf.keras.layers.Dense(Output_size)

1 | model = tf.keras.models.Sequential([ tf.keras.layers.Input(shape=(D,)), tf.keras.layers.Dense(1, activation='sigmoid')]) |

- cost function

1 | model.compile (optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy']) |

- Accuracy

1 | model.compile(optimizer='adam', loss = 'binary_crossentropy', metrics = ['accuracy']) |

- Training / Fitting

1 | r = model.fit (X_train, y_train, validation_data = (X_test, y_test), epochs=100)plt.plot(r.history['loss'],label='loss')plt.plot(r.history['val_loss'],label='val_loss') |

To be conclude,

- Prediction = Round(
*σ*(*Wx*+*b*) ) - Implemented with Keras
*Dense*layer *compile()*to specify optimizer (adam), loss (binary cross entropy), metrics(accuracy)*fit()*returns a history object, so we can plot loss per iteration

### Loss

1 | model.compile (optimizer = tf.keras.optimizers.SGD(0.001, 0.9), loss='mse') |

- MSE = mean squared error

$MSE={1\over N} \sum_{i=1}^N (y_i -\hat{y_i})^2$

### Minimizing Cost

The multidimensional equivalnet of *derivative* is the **gradient**

$\nabla_w J =({\partial J\over \partial w_1},{\partial J\over \partial w_2},…,{\partial J \over \partial w_D})$

$Set: \nabla_w J =0, solve \space for \space w$

TensorFlow done this for us by *automatic differentitation*

- Tensorflow automatically finds the gradient of loss wrt all tour model weights
- Tensorflow uses the gradients to train your model

### Gradient Descent

- In most cases, it is not possible to solve for $anbla J=0$, using training
*iterative*instead. (Linear Regression is the exception.) - gradient descent algorithm is inside the Keras
**fit**function:

1 | w,b = randomly initialized |

**The aim of our traning algorithm will be to minimize the cost $C(w,b)$ as a fucntion of the weights and biases.** In other words, we want to find a set of weights and biases which make the cost as small as possible.

Gradient descent can be viewed as a way of taking small steps in the direction which does the most to immediately decrease C.

**Stochastic gradient descent(SGD)** used to speed uop learning.

The idea is to estimate the gradient $\nabla C$ by computing $\nabla C_x$ for a small sample of randomly chosen training inputs. By averaging over this small sample it turns out that we can quickly get a good estimate of the true gradient $\nabla C$/

The Cauchy-Schwarz inequality:

for all vectors u and v of an inner product space, it is true that:

**构建一个network(2,3,1)**

1 | class Network(object): |

接下来，就要计算每一层的输出：

1 | def sigmoid(z): |

1 | def feedforward(self, a): |

1 | def SGD(self, training_data, epochs, mini_batch_size, eta, |

1 | def update_mini_batch(self, mini_batch, eta): |

具体代码可以在github上找到：地址 python3.5版本

1 | Finished training network. |

**如何计算***backprogation*?

*backprogation*?

该算法的核心是快速计算相对于每一个weight w 和每一个bias b的cost function 的偏导数：

先给weights一个标记，如上图所示。

同时，还有basis的标签。

这样我们就有了输入输出之间的联系：

写成矩阵的形式，或者说爱因斯坦简化形式：

突然发现，这个$w_{jk}^l$其实就是所谓的三阶张量呀，但这个张量没层数量可能不相同！怪不得叫tensorflow。

**对于cost function的两个假设**

the cost function 为单个训练集的平均，即

the cost is a function of the outputs a:

例如，quadratic cost:

接着，引入中间变量，*error* $\delta_j^l$

我们将得到四个关于误差的表达式** (chain rule)**：

**An equation for the error in the output layer**, $\delat^L$:

简写为Hadmard product形式：

简写为矩阵形式：

$\sigma’(z_j^L)$无非就是sigma函数的的在$z_j^l$的导数，计算得到$z_j^l$后，在其附近添加一个差值，利用差分非常好求。C是给定的一个形式，例如上面的quadratic cost，那对a求导也非常容易，得:

y是标签的值，a是训练的得到的值。

**An equation for the error $\delta^l$ in terms of the error in the next layer**, $\delat^{l+1}$:

Differentiating, we obtain

Substituting back we obtain:

矩阵形式：

其中，$(w^{l+1})^T$ is the transpose of the weight matrix $w^{l+1}$ for the $(l+1)^{th}$ layer. 假定我们知道上一$l+1$层的误差$\delta^{l+1}$。When we apply the transpose weight matrix, $(w^{l+1})^T$, we can think intuitively of this as moving the error *backward* through the network, giving us some sort of meaure of the error at the output of the l layer.

如此，通过计算L层的误差，然后通过迭代就可以倒退回每一层的误差。

如果将其在RNN上应用，类似于dynamical system里也有类似累积的效果，每一层相当于每一个时间步。误差类比于流动稳定性。流动在某个临界条件会发生不稳定性，往往这个临界点是我们非常关心的。。。。后面如何和chaotic系统结合还有待思考🤔貌似一个是正问题，一个是逆问题。

- 后面MIT的课程上，通过forward propagation dual number 来求解。ODE关联，优化问题。都是梯度下架问题。在ODE问题中，用隐式算法，同样会涉及逆问题。最后，貌似都转化为同一个问题。

**An equation for the rate of change of the cost with respect to any bias in the network**:

通过这个公式，我们就获得了b的偏导数。

**An equation for the rate of change of the cost with respect to any weight in the net work**:

When $a_in$ is small, the gradient terms ${\partial C \over \partial w}$ will also tend to be small, it learns slowly. — **low-activation**

Besides, consider the term $\sigma’(z_j^L)$ become very flat when approach to 0 or 1, it also learns slowly. — **saturated**

**The backprogation algorithm**

Input x: Set the corresponding activation $a^1$ for the input layer.

Feedforward: For each l=2,3,…,L compute $z^l = w^l a^{l-1}+b^l$ and $a^l=\sigma(z^l)$.

Output error $\delta^L$: Compute the vector $\delta^L = \nabla_a C \odot \sigma’(z^L).$

Backpropagate the error: For each l = L-1,L-2,…,2 compute

Output: The gradient of the cost function is given by

**The backprogation algorithm with a set of training examples**

计算过程不变，只不过最终梯度下架为计算的平均值。

1 | class Network(object): |

1 | class Network(object): |

learning rate is low, means that the partial derivatives of the cost function, $\partial C/\partial w$ and $\partial C / \partial b$ is small.

**Better choice of cost function: the cross-entropy**

对单个输入的神经网络来说，

则带来的好处是：

训练误差越大，则学习速度越快！

**“Regularization” methods (L1 and L2 regularization, dropout, and artificial expansion of the traning data)**

过拟合问题。

**regularized cross-entropy**

The effect of regularization is to make it so the network prefers to learn small weights, all other things being equal. **Regularization can be viewed as a way of compromising between finding small weights and minimizing the original cost function.**

**L1 regularization**

相比L2 regularization,

In both expressions, the effect of regularization is to shrink the weights. This accords with our intuition that both kinds of regularization penalize large weights(罚函数)。In L1 regularization, the weights shrink by a constant amount toward 0. In L2 regularization, the weights shrink by an amount which is proportion to w. And so when a particular weight has a large magnitude, |w|, L1 regularization shrinks the weight much less than L2 regularization does. By contrast, when |w| is small, L1 regularization shrinks the weight much more than L2 regularization. z最终

在这两个表达式中，正则化的效果都是缩小权重。这符合我们的直觉，即两种正则化都会惩罚较大的权重。但是权重缩减的方式是不同的。在L1正则化中，权重保持不变。在L2正则化中，权重与w成比例。因此，当|w|较大时，L1正则化比L2正则化缩小的慢。相比之下，当|w|较小，L1正则化比L2正则化压缩权重更多。最终结果是，L1正则化倾向于将网络的权重集中在相对少量的高重要性连接中，而其他权重则趋向于零。

**Dropout**

modify the network itself.

**改进代码执行：**

1 | """network2.py |

**Tensorflow 执行代码：**参考

Classfication:

https://colab.research.google.com/drive/1M_otG1koa2qwix4-ScwZUzF_NB9rg4vD?usp=sharing

Regression:

https://colab.research.google.com/drive/1k-sj3NP0YNoaIXVNdaXF9WtyY9sMqosR#scrollTo=x5BY2x58MBrt

### Aritificial Neural Networks(ANNS)

**Feedforward neural networks**

the signal just goes from left to right, we call it a “feedforward” neural network

Different neural will find different features with inputs

- a line : ax+b
- a neuron: $\sigma(w^T x+b)$
- Multiple Neurons per Layer:

$z_j = \sigma(w_j^T x+ b_j),for \space j=1…M$

Vectorize the neuron:

$Z=\sigma(W^T x +b)$

这一个单元：将M个特征的信息通过DxM的矩阵W转化到D个特征，然后非常类似于着色器里的sigma函数操作。整个过程一气呵成，GPU的作用可见一斑。

**Each neural network layer is a “feature transform”**

这种转化在neural network一层一层的迭代：

$z^{(1)} = \sigma(W^{(1)T}x + b^{(1)})\\$$z^{(2)}=\sigma(W^{2}T)z^{(1)}+b^{(2)})$

$z^{(3)}=\sigma(W^{(3)T}z^{(2)}+b^{3})$

…

$p(y=1|x)=\sigma(W^{(L)T}z^{(L-1)}+b^{(L)})$

**The Geometric Picture**

- The neuron is nice and interpretable
- Large weights = important feature
- Small weights = not import feature

$\hat{y} = ax^2 + bx + c, \\ Let: x_1 =x, x_2= x^2\\ Then: \hat{y} = w_1 x_1 + w_2 x_2 +b$

It’s nonlinear because of the **sigmoid**

$W^{(2)T} \sigma(W^{(1)T}x+b^{(1)})+b^{(2)}$

**Convolutional Neural Networks (CNNs)**

更适合图像。

**Recurrent Neural Networks(RNNs)**- for sequence

a later neuron may connect back to an earlier neuron to created a recurrent connection

A recurrent neural network is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior. Derived from feedforward neural networks, RNNs can use their internal state to process variable length sequences of inputs. This makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

The main use of RNNs are when using google or Facebook these interfaces are able to predict the next word that you are about to type. RNNs have loops to allow information to persist. This reduces the complexity of parameters, unlike other neural networks. These neural nets are considered to be fairly good for modeling sequence data. RNNs keep track of a hidden state, that encodes information about the history of the system dynamics.

RNNs have the potential to overcome these scalability problems and be applied to high-dimensional spatio-temporal dynamics.

Recurrent neural networks are a linear architectural variant of recursive networks. They have a “memory” thus it differs from other neural networks. This memory remembers all the information about, what has been calculated in the previous state. It uses the same parameters for each input as it performs the same task on all the inputs or hidden layers to produce the output.

One of the biggest uniqueness RNNs have is “UAP- Universal Approximation Property” thus they can approximate virtually any dynamical system. This unique property forces us to say recurrent neural networks have something magical about them.

RNNs model has been proven to perform extremely well on temporal data. It has several variants including **LSTMs** (long short-term memory), **GRUs** (gated recurrent unit) and **Bidirectional RNNs.** Building models for natural language, audio files, and other sequence data got a lot easier with sequence algorithms.

**Vanishing and Exploding Gradient Problem**-limited

This problem sent the major set back for RNNs to get popularity. Values in RNNs can explode or vanish due to a simple reason for remembering previous values. Previous values are good enough to confuse them and cause current values to keep increasing or decreasing & take-over algorithm. Indefinite loops get formed that brings whole network halt.

For example, neurons might get stuck in the loop where it keeps multiplying the previous number to new number which can go to infinity if all numbers are more than one or get stuck at zero if any number is zero.

**If the weight is small, multiply it all the time, it will become small. Otherwise it will become very large.**这样深度神经网络的训练也会失效。从深度网络的角度来讲，不同的层学习的速度差异很大，表现为网络中靠近输出的层学习的情况很好，靠近输入的层学习的很慢，有时甚至训练了很久，前几层的权值和刚开始随机初始化的值差不多。因此，梯度消失、爆炸，其根本原因在于反向传播训练法则backpropagation (BPTT)**，属于先天不足。

### Reasons for the disappearance of the gradient:

- Neural network
- The initialization weight is too small
- Activation function sigmoid (Relu)

### Reasons for gradient explosion:

- Neural network
- The initialization weight is too large

### How to solve it?

- Pre-training + fine-tuning

其基本思想是每次训练一层隐节点，训练时将上一层隐节点的输出作为输入，而本层隐节点的输出作为下一层隐节点的 输入，此过程就是逐层“预训练”；在训练完成后，再对整个网络进行“微调”

- Gradient shear, regular

梯度剪切这种方案主要是针对梯度爆炸问题，其思想是设置一个梯度阀值。

**Long Short-Term Memory**(LSTM)

LSTMs employ gates that effectively **remember** and **forget** information thus alleviating the problem of vanishing gradients，可以缓解Vanishing and Exploding Gradient Problem。复杂的门可以接下来更新的时候“记住”前几次训练的“残留记忆”。

中间的具体对应关系参考下面的建模方法的图c，有点复杂： $$

g*t^f=\delta_f (W_f[h*{t-1},o*t]+b_f) , \space\space g_t^i =\delta_i (W_i[h*{t-1},o*t]+b_i)\\\tilde{c}_t = tanh(W_c[h*{t-1},o

*t]+b_t), \space c_t = g_t^f \odot c*{t-1} +g

*t^i \odot \tilde{c}_t\\*

g_t^o = \delta_h(W_h[h{t-1},o_t]+b_h), \space\space h_t =g_t^o \odot tanh(c_t)

g_t^o = \delta_h(W_h[h

$$ 有点像电路板，*gtf*, *gti*, *gto*分别代表forget, input, output gates. *ot* is the observable input at time t, *ht* is the hidden state, *ct* is the cell state, W is weight matrices, b is biases. The symbol ⊙ denotes the element-wise product. *δ* is Sigmoid function. The dimension of the hidden state *dh* (number of hidden units) controls the capacity of the cell to encode history information. The hidden-to-output functional form *fho* is given by a linear layer: *ôt* + 1 = *Woht*, *Wo* ∈ *Rd*0 × *dh*

**Reservior Computing**

**Reservoir computing** is a framework for computation derived from recurrent neural network theory that maps input signals into higher dimensional computational spaces through the dynamics of a fixed, non-linear system called a reservoir.[1] After the input signal is fed into the reservoir, which is treated as a “black box,” a simple readout mechanism is trained to read the state of the reservoir and map it to the desired output.[1] The first key benefit of this framework is that training is performed only at the readout stage, as the reservoir dynamics are fixed.[1] The second is that the computational power of naturally available systems, both classical and quantum mechanical, can be utilized to reduce the effective computational cost.[2]

**建模方法**

机器学习预测时间序列。给定等间隔1/*trianglet*采样，可观测序列*o* ∈ *Rdo* ,{*o*1, …, *oT*}。输入前一个时间点的状态*ht* ∈ *Rdh*. *ht* = *fhh*(*ot*, *ht* − 1), *ôt* + 1 = *fho*(*ht*) *fhh* is hidden-to-hidden mapping, *fho* is the hidden-to-output mapping. 下面这四种模型都是这种结构，只是对f的实现和参数有所差异。参见：^{4}

示意图