### Numpy

• Matrix arithmetic
• Tensor (aka. “arrays”)(note the name: Tensorflow)
• 1-D tensor(vector), 2-D tensor(matix)
• Arithmetic: +,-,*,/
• Matrix multiply == dot/inner product (np.dot)
• Element-wise multiply(*)

### Matplotlib

• Line charts
• Scatter plots

### Scipy(power version of numpy)

• Satistics, optimization, linear algebra, signal processing

### Scikit-Learn

• basic machine learning
• Classification and regression
• Machine learning as geometry rather than magic
• Shape of data(X and Y)

### The Steps of a Machine Learning Script

• Load in the data
1. X and Y
2. Typically use Pandas unless the data is to complex
• Split into train/test sets (sometimes)
1. Sometimes “test” and “validation” are used interchangebly, and the “true test set” is something else- don’t get frazzled over this.
• Build a model
1. OOP (object oriented programming)
2. Tensorflow 2.0 standard is Keras API, very similar to Scikit-learn
• Fit the model(gradinet descent)
1. model.fit(X,Y)
• Evaluate the model
• Make predictions
1. model.predict(X)

## Outline

### What is machine learning

look for patterns for data and make better prediction in furture.

traning a nueral network is just to find a weight of minimum of cost function.

### Classfication and Regression

Classfication:

• predict a category/ lable
• yes or not / dog or cat
• SVM

Regression: (more features,nonlinear makes problem complicated)

• Fit a line or curve

= mx + b

The Decision Rule

• sigmoid functions

= σ(a), a = w1x1 + w2x2 + b

Probabilistic Interpretation

p(y = 1|x) = σ(a)

Logistic Regression with >2 Inputs

• What if we have > 2 input?
• Use matrices/ vectors

$p(y= 1| x)=\sigma(w^T x + b)= \sigma (\sum_{d=1}^D w_d x_d +b)$

Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.

Those say that $\triangle output$ is a linear function of the changes $\triangle w_j$ and $\triangle b$ in the weights and bias. make it much easier to figure out how changing the weights and biases will change the output.

sigmoid function is defined by:

ReLU may be even more biological plausible than sigmoid;

In Tnesorflow 2.0

• Is mostly an exercise in “API hunting”
• The actual math isn’t written in code (by you)
• Your job is to find the function that implements the math in question
• tf.keras.layers.Dense(Output_size)
• cost function

• Accuracy
• Training / Fitting

To be conclude,

• Prediction = Round( σ(Wx + b) )
• Implemented with Keras Dense layer
• compile() to specify optimizer (adam), loss (binary cross entropy), metrics(accuracy)
• fit() returns a history object, so we can plot loss per iteration

### Loss

• MSE = mean squared error

$MSE={1\over N} \sum_{i=1}^N (y_i -\hat{y_i})^2$

### Minimizing Cost

The multidimensional equivalnet of derivative is the gradient

$\nabla_w J =({\partial J\over \partial w_1},{\partial J\over \partial w_2},…,{\partial J \over \partial w_D})$

$Set: \nabla_w J =0, solve \space for \space w$

TensorFlow done this for us by automatic differentitation

• Tensorflow automatically finds the gradient of loss wrt all tour model weights
• Tensorflow uses the gradients to train your model

• In most cases, it is not possible to solve for $anbla J=0$, using training iterative instead. (Linear Regression is the exception.)
• gradient descent algorithm is inside the Keras fit function:

The aim of our traning algorithm will be to minimize the cost $C(w,b)$ as a fucntion of the weights and biases. In other words, we want to find a set of weights and biases which make the cost as small as possible.

Gradient descent can be viewed as a way of taking small steps in the direction which does the most to immediately decrease C.

Stochastic gradient descent(SGD) used to speed uop learning.

The idea is to estimate the gradient $\nabla C$ by computing $\nabla C_x$ for a small sample of randomly chosen training inputs. By averaging over this small sample it turns out that we can quickly get a good estimate of the true gradient $\nabla C$/

The Cauchy-Schwarz inequality:

for all vectors u and v of an inner product space, it is true that:

### 如何计算backprogation?

​ 该算法的核心是快速计算相对于每一个weight w 和每一个bias b的cost function 的偏导数：

### 对于cost function的两个假设

• the cost function 为单个训练集的平均，即

• the cost is a function of the outputs a:

An equation for the error in the output layer, $\delat^L$:

$\sigma’(z_j^L)$无非就是sigma函数的的在$z_j^l$的导数，计算得到$z_j^l$后，在其附近添加一个差值，利用差分非常好求。C是给定的一个形式，例如上面的quadratic cost，那对a求导也非常容易，得:

​ y是标签的值，a是训练的得到的值。

An equation for the error $\delta^l$ in terms of the error in the next layer, $\delat^{l+1}$:

Differentiating, we obtain

Substituting back we obtain:

• 后面MIT的课程上，通过forward propagation dual number 来求解。ODE关联，优化问题。都是梯度下架问题。在ODE问题中，用隐式算法，同样会涉及逆问题。最后，貌似都转化为同一个问题。

An equation for the rate of change of the cost with respect to any bias in the network:

An equation for the rate of change of the cost with respect to any weight in the net work:

When $a_in$ is small, the gradient terms ${\partial C \over \partial w}$ will also tend to be small, it learns slowly. — low-activation

Besides, consider the term $\sigma’(z_j^L)$ become very flat when approach to 0 or 1, it also learns slowly. — saturated

### The backprogation algorithm

1. Input x: Set the corresponding activation $a^1$ for the input layer.

2. Feedforward: For each l=2,3,…,L compute $z^l = w^l a^{l-1}+b^l$ and $a^l=\sigma(z^l)$.

3. Output error $\delta^L$: Compute the vector $\delta^L = \nabla_a C \odot \sigma’(z^L).$

4. Backpropagate the error: For each l = L-1,L-2,…,2 compute

5. Output: The gradient of the cost function is given by

### The backprogation algorithm with a set of training examples

​ 计算过程不变，只不过最终梯度下架为计算的平均值。

learning rate is low, means that the partial derivatives of the cost function, $\partial C/\partial w$ and $\partial C / \partial b$ is small.

### “Regularization” methods (L1 and L2 regularization, dropout, and artificial expansion of the traning data)

regularized cross-entropy

The effect of regularization is to make it so the network prefers to learn small weights, all other things being equal. Regularization can be viewed as a way of compromising between finding small weights and minimizing the original cost function.

L1 regularization

In both expressions, the effect of regularization is to shrink the weights. This accords with our intuition that both kinds of regularization penalize large weights(罚函数)。In L1 regularization, the weights shrink by a constant amount toward 0. In L2 regularization, the weights shrink by an amount which is proportion to w. And so when a particular weight has a large magnitude, |w|, L1 regularization shrinks the weight much less than L2 regularization does. By contrast, when |w| is small, L1 regularization shrinks the weight much more than L2 regularization. z最终

Dropout

modify the network itself.

Tensorflow 执行代码：参考

Classfication:

Regression:

### Aritificial Neural Networks(ANNS)

Feedforward neural networks

the signal just goes from left to right, we call it a “feedforward” neural network

Different neural will find different features with inputs

• a line : ax+b
• a neuron: $\sigma(w^T x+b)$
• Multiple Neurons per Layer:

$z_j = \sigma(w_j^T x+ b_j),for \space j=1…M$

Vectorize the neuron:

$Z=\sigma(W^T x +b)$

Each neural network layer is a “feature transform”

$z^{(1)} = \sigma(W^{(1)T}x + b^{(1)})\\$$z^{(2)}=\sigma(W^{2}T)z^{(1)}+b^{(2)}) z^{(3)}=\sigma(W^{(3)T}z^{(2)}+b^{3}) p(y=1|x)=\sigma(W^{(L)T}z^{(L-1)}+b^{(L)}) The Geometric Picture • The neuron is nice and interpretable • Large weights = important feature • Small weights = not import feature \hat{y} = ax^2 + bx + c, \\ Let: x_1 =x, x_2= x^2\\ Then: \hat{y} = w_1 x_1 + w_2 x_2 +b It’s nonlinear because of the sigmoid W^{(2)T} \sigma(W^{(1)T}x+b^{(1)})+b^{(2)} Convolutional Neural Networks (CNNs) 更适合图像。 Recurrent Neural Networks(RNNs)- for sequence a later neuron may connect back to an earlier neuron to created a recurrent connection Recurrent Neural Networks： A recurrent neural network is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior. Derived from feedforward neural networks, RNNs can use their internal state to process variable length sequences of inputs. This makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition. The main use of RNNs are when using google or Facebook these interfaces are able to predict the next word that you are about to type. RNNs have loops to allow information to persist. This reduces the complexity of parameters, unlike other neural networks. These neural nets are considered to be fairly good for modeling sequence data. RNNs keep track of a hidden state, that encodes information about the history of the system dynamics. RNNs have the potential to overcome these scalability problems and be applied to high-dimensional spatio-temporal dynamics. Recurrent neural networks are a linear architectural variant of recursive networks. They have a “memory” thus it differs from other neural networks. This memory remembers all the information about, what has been calculated in the previous state. It uses the same parameters for each input as it performs the same task on all the inputs or hidden layers to produce the output. One of the biggest uniqueness RNNs have is “UAP- Universal Approximation Property” thus they can approximate virtually any dynamical system. This unique property forces us to say recurrent neural networks have something magical about them. RNNs model has been proven to perform extremely well on temporal data. It has several variants including LSTMs (long short-term memory), GRUs (gated recurrent unit) and Bidirectional RNNs. Building models for natural language, audio files, and other sequence data got a lot easier with sequence algorithms. Vanishing and Exploding Gradient Problem-limited This problem sent the major set back for RNNs to get popularity. Values in RNNs can explode or vanish due to a simple reason for remembering previous values. Previous values are good enough to confuse them and cause current values to keep increasing or decreasing & take-over algorithm. Indefinite loops get formed that brings whole network halt. For example, neurons might get stuck in the loop where it keeps multiplying the previous number to new number which can go to infinity if all numbers are more than one or get stuck at zero if any number is zero. If the weight is small, multiply it all the time, it will become small. Otherwise it will become very large.这样深度神经网络的训练也会失效。从深度网络的角度来讲，不同的层学习的速度差异很大，表现为网络中靠近输出的层学习的情况很好，靠近输入的层学习的很慢，有时甚至训练了很久，前几层的权值和刚开始随机初始化的值差不多。因此，梯度消失、爆炸，其根本原因在于反向传播训练法则backpropagation (BPTT)**，属于先天不足。 ### Reasons for the disappearance of the gradient: 1. Neural network 2. The initialization weight is too small 3. Activation function sigmoid (Relu) ### Reasons for gradient explosion: 1. Neural network 2. The initialization weight is too large ### How to solve it? 1. Pre-training + fine-tuning 其基本思想是每次训练一层隐节点，训练时将上一层隐节点的输出作为输入，而本层隐节点的输出作为下一层隐节点的 输入，此过程就是逐层“预训练”；在训练完成后，再对整个网络进行“微调” 1. Gradient shear, regular 梯度剪切这种方案主要是针对梯度爆炸问题，其思想是设置一个梯度阀值。 1. Long Short-Term Memory (LSTM) LSTMs employ gates that effectively remember and forget information thus alleviating the problem of vanishing gradients，可以缓解Vanishing and Exploding Gradient Problem。复杂的门可以接下来更新的时候“记住”前几次训练的“残留记忆”。 中间的具体对应关系参考下面的建模方法的图c，有点复杂：$$ gt^f=\delta_f (W_f[h{t-1},ot]+b_f) , \space\space g_t^i =\delta_i (W_i[h{t-1},ot]+b_i)\\ \tilde{c}_t = tanh(W_c[h {t-1},ot]+b_t), \space c_t = g_t^f \odot c{t-1} +gt^i \odot \tilde{c}_t\\ g_t^o = \delta_h(W_h[h {t-1},o_t]+b_h), \space\space h_t =g_t^o \odot tanh(c_t)$\$ 有点像电路板，gtf, gti, gto分别代表forget, input, output gates. ot is the observable input at time t, ht is the hidden state, ct is the cell state, W is weight matrices, b is biases. The symbol ⊙ denotes the element-wise product. δ is Sigmoid function. The dimension of the hidden state dh (number of hidden units) controls the capacity of the cell to encode history information. The hidden-to-output functional form fho is given by a linear layer: ôt + 1 = Woht, Wo ∈ Rd0 × dh

Reservior Computing

Reservoir computing is a framework for computation derived from recurrent neural network theory that maps input signals into higher dimensional computational spaces through the dynamics of a fixed, non-linear system called a reservoir.[1] After the input signal is fed into the reservoir, which is treated as a “black box,” a simple readout mechanism is trained to read the state of the reservoir and map it to the desired output.[1] The first key benefit of this framework is that training is performed only at the readout stage, as the reservoir dynamics are fixed.[1] The second is that the computational power of naturally available systems, both classical and quantum mechanical, can be utilized to reduce the effective computational cost.[2]