LDA

# LDA

## 算法输入与目的

$$x = [x_1,x_2,x_3…]^T$$

$$y=w^T x$$

## 模型建立

$$Y_i = {y_1,y_2,y_3…}$$

$$\tilde{m_i}=\frac{1}{n_i}\sum_{y\in Y_i}y$$

$$\tilde{S_i}^2 = \sum_{y \in Y_i}(y-\tilde{m_i})^2$$

$$\tilde{m}=\frac{1}{n}\sum n_i*\tilde{m_i} \\max\quad J(w)=\frac{ \sum n_i(\tilde{m_i}-\tilde{m})^2}{\sum{\tilde{s_i}^2}}$$

## 变形

$$D_i = {x_1,x_2,x_3…}$$

$$m_i=\frac{1}{n_i}\sum_{x\in D_i}x$$

$$\tilde{m_i}=\frac{1}{n_i}\sum_{y\in Y_i}y=\frac{1}{n_i}\sum_{x\in D_i}w^Tx=w^T\frac{1}{n_i}\sum_{x\in D_i}x=w^Tm_i$$

$$S_i = \sum_{x \in D_i}(x-m_i)(x-m_i)^T$$

\begin{aligned} \tilde{S_i}^2&= \sum_{y \in Y_i}(y-\tilde{m_i})^2 \&= \sum_{x \in D_i}(w^Tx-w^Tm_i)^2 \&=\sum_{x \in D_i}(w^T(x-m_i))(w^T(x-m_i)) \&=\sum_{x \in D_i}(w^T(x-m_i)) ((x-m_i)^Tw) \&=w^TS_iw \end{aligned}

$$\tilde{m}=\frac{1}{n}\sum n_i\tilde{m_i}=w^T\frac{1}{n}\sum n_im_i=w^Tm$$

\begin{aligned} \&\sum n_i(\tilde{m_i}-\tilde{m})^2 \=&\sum n_i(w^Tm_i-w^Tm)^2 \=&w^T(\sum n_i(m_i-m)(m_i-m)^T) w \end{aligned}

## 模型总结

\begin{aligned} &S_B=\sum n_i(m_i-m)(m_i-m)^T \&S_W=\sum S_i \&\max\quad J(w)=\frac{ w^TS_Bw}{w^TS_ww} \end{aligned}

## 拉格朗日乘子法求极值

J(w)的值与w的长度无关，只和w的方向有关，我们不妨固定分子为1，则变为了

$$\max\quad J_2(w)=w^TS_Bw\quad,\quad w^TS_ww=1$$

$$L(w,\lambda) = w^TS_Bw+\lambda(1-w^TS_ww)$$

\begin{aligned} &\frac{\partial x^TAx}{\partial x} = (A^T+A)x \end{aligned}

\begin{aligned} &\frac{\partial L}{\partial w}=(S_B+S_B^T)w-\lambda(S_w+S_w^T)w=0 \end{aligned}

\begin{aligned} &2S_Bw-2\lambda S_ww=0 \\to&S_Bw=\lambda S_ww \\to&(S_w^{-1}S_B)w=\lambda w \end{aligned}