Skip to main content
1 of 4
Raz
  • 135
  • 11

why in Gaussain on linear regression instead of $x$ they put $y_i-\theta^Tx_i$?

I am new to the machine learning area. Then I am studying a paper that considers the problem of logistic regression in Bayesian. Then I do not know in the $\pi$ formula in the following which one is prior?

Define the vector $Y=(y_1,\cdots,y_n)\in \{0,1\}^n$ and let $X$ be the $n\times d$ matrix with $x_i$ as $i^{th}$-row. We choose the prior $\pi_0$ to be a Gaussian distribution with zero mean and covariance matrix proportional to the inverse of the sample covariance matrix $\sum_X=\frac{1}{n} X^TX$. Plugging in the formula for the prior and likelihood, we find that the posterior density is given by $\pi(\theta)=\pi(\theta|X,Y)\propto \mathrm{exp}\left\{ Y^TX\theta - \sum_{i=1}^n \log(1+e^{\theta^Tx_i})-\alpha||\sum_X^{1/2}\theta||_2^2\right\}$.

And in general, in regression or logistic regression when they assume the prior or likelihood is Guassian why in the formula of Guassian $p(x)= \frac{1}{\sqrt(2\pi \sigma)}\mathrm{exp}(-\frac{(x-\mu)^2}{2\sigma^2})$ in stead of $x$ they put $y_i-\theta^tx_i$?

Raz
  • 135
  • 11