# LDA

## 算法输入与目的

$x = [x_1,x_2,x_3...]^T$

$y=w^T x$

## 模型建立

$Y_i = \{y_1,y_2,y_3...\}$

$\tilde{m_i}=\frac{1}{n_i}\sum_{y\in Y_i}y$

$\tilde{S_i}^2 = \sum_{y \in Y_i}(y-\tilde{m_i})^2$

$\tilde{m}=\frac{1}{n}\sum n_i*\tilde{m_i} \\\max\quad J(w)=\frac{ \sum n_i(\tilde{m_i}-\tilde{m})^2}{\sum{\tilde{s_i}^2}}$

## 变形

$D_i = \{x_1,x_2,x_3...\}$

$m_i=\frac{1}{n_i}\sum_{x\in D_i}x$

$\tilde{m_i}=\frac{1}{n_i}\sum_{y\in Y_i}y=\frac{1}{n_i}\sum_{x\in D_i}w^Tx=w^T\frac{1}{n_i}\sum_{x\in D_i}x=w^Tm_i$

$S_i = \sum_{x \in D_i}(x-m_i)(x-m_i)^T$

\begin{aligned} \tilde{S_i}^2&= \sum_{y \in Y_i}(y-\tilde{m_i})^2 \\&= \sum_{x \in D_i}(w^Tx-w^Tm_i)^2 \\&=\sum_{x \in D_i}(w^T(x-m_i))*(w^T(x-m_i)) \\&=\sum_{x \in D_i}(w^T(x-m_i))*((x-m_i)^Tw) \\&=w^TS_iw \end{aligned}

$\tilde{m}=\frac{1}{n}\sum n_i*\tilde{m_i}=w^T\frac{1}{n}\sum n_i*m_i=w^Tm$

\begin{aligned} \\&\sum n_i(\tilde{m_i}-\tilde{m})^2 \\=&\sum n_i(w^Tm_i-w^Tm)^2 \\=&w^T(\sum n_i(m_i-m)(m_i-m)^T) w \end{aligned}

## 模型总结

\begin{aligned} &S_B=\sum n_i(m_i-m)(m_i-m)^T \\&S_W=\sum S_i \\&\max\quad J(w)=\frac{ w^TS_Bw}{w^TS_ww} \end{aligned}

## 拉格朗日乘子法求极值

J(w)的值与w的长度无关，只和w的方向有关，我们不妨固定分子为1，则变为了

$\max\quad J_2(w)=w^TS_Bw\quad,\quad w^TS_ww=1$

$L(w,\lambda) = w^TS_Bw+\lambda(1-w^TS_ww)$

\begin{aligned} &\frac{\partial x^TAx}{\partial x} = (A^T+A)x \end{aligned}

\begin{aligned} &\frac{\partial L}{\partial w}=(S_B+S_B^T)w-\lambda(S_w+S_w^T)w=0 \end{aligned}

\begin{aligned} &2S_Bw-2\lambda S_ww=0 \\\to&S_Bw=\lambda S_ww \\\to&(S_w^{-1}S_B)w=\lambda w \end{aligned}

-----------------------本文结束感谢您的阅读-----------------------