Probabilistic Derivation of Linear Regression Estimating Equations So this has been something I've been studying for some time.  The first time I studied it I glossed over the details and parts that I really didn't understand.  Now I'd like to tackle these points.
My sources of study are Andrew Ngs CSC229 lecture notes and Bishops Pattern Recognition and Machine Learning Textbook.
So we have $y(i) = dot(θ, x_i) + ε_i$, where θ is the parameter vector, and x_i the ith feature vector.  We assume ε_i be the noise of the model, and let it be gaussian i.e. $ε(i) ∼ N (0, σ^2)$.  Now I'm confused how we can infer the following:
$y_i | x_i; θ ∼ N (dot(θ, x_i), σ^2).$
Thats my first uncertainty.   
Second, the way the likelihood function is defined confuses me a bit.  Its defined as $L(θ) = p(y|X; θ)$.  Now, is the likelihood function always defined like this.  Like from Bayes theorem, we have $p(w|D) ~ p(D|w)p(w)$ where D is the observed data $D = {t1, . . . , tN }$ and w is the parameters.  The Bishop textbook defines p(D|w) as the likelihood function.  So would D in our context be y and w be θ.  I guess I'm just confused about all the variables etc.
 A: Question 1
Because we condition on the values of $x_i$, we can treat it as fixed and non-random.  Then, we can observe that $Y_i$, by its very definition, can be decomposed into a constant term $x_i'\theta$, often called the signal, and a random term $\varepsilon_i,$ often called the noise.
Now for any real $c$ and any real-valued random variable $Z$, we have $E(Z + c) = E(Z) + c$ and $\text{Var}(Z) = \text{Var}(Z+c)$.  When $Z$ happens to be normally distributed, so is $Z+c$.  The case of $Y_i=x_i'\theta + \epsilon_i$ is no different because the signal $x_i'\theta$ is a constant (even if $\theta$ is unknown), and the noise term $\varepsilon_i\sim N(0,\sigma^2)$.  Therefore, $Y_i$ is also normally distributed with mean $0+x_i'\theta=x_i'\theta$ and variance $\sigma^2$.
Question 2
Now consider the entire length-$n$ vector $y$ of observed responses and the matrix $X$ of covariates.
You can think of the likelihood function as telling you how probable or improbable your observed data would be if the a particular $\theta$ were the true model.
More precisely, under the assumptions of the model, the response $y$ (a vector of length $n$) that we actually observe is a realization of a random variable $Y\sim N_n(X\theta, \sigma^2I_n)$. The likelihood function is $L(\theta)=f(y|X;\theta)$, where $f(\cdot|X;\theta)$ the multivariate normal pdf with mean $X\theta$ and variance $\sigma^2I_n$.  Larger values of $L(\theta)$ indicate that, if $\theta$ were the true model parameter, the observed $y$ comes from a region of relatively more probability mass.  Smaller values of $L(\theta)$ tell us the opposite.
The idea is that the smaller the likelihood for a given $\theta$, less likely (note: the pdf is not a probability for continuous random variables, such as the normal distribution) it is that we could observe the actual data $y$ under the assumptions of our model.  Conversely, the higher the likelihood, the more likely it is that we could observe the actual data $y$.  This is why a very common objective of estimating model parameters is to maximize the likelihood. We want to find the parameter $\theta$ that makes the observed data as likely as possible! 
The notation from Bishop $p(D|w)$--$D$ for data, so it corresponds to $y$, and $w$ for weights, so it corresponds to $\theta$--has the same definition and interpretation but it is more explicit about being a pdf (not a probability, however).
A: $Y \mid X$ is normal since $Y = f(X) + \epsilon$ and the $f(X)$ is constant and $\epsilon \sim \mathcal{N}(0, \sigma^2)$. Therefore, $Y \mid X \sim \mathcal{N}(f(X), \sigma^2)$. You are right that there is an interesting relationship between a posterior density and a likelihood that differs only proportionally to the prior density. For this reason, noninformative priors often lead to posterior mode estimates that are very similar to MLEs in regular distributions.
It is useful to think of the $X$ in such analyses as part of the study design, fixed as in an experimental study. It is the $Y$ which may vary in ideal replications of the study design. What, then, is the probability of the sample $Y$ that we obtained under this experimental design? Well, by assuming a range of values for $\theta$, we can calculate this value over a range of possible probability models for $Y$. 
A: In the first one, you have error normally distributed with mean zero and variance sigma^2. Now let's drive the distribution of y (here y is a random variable since error is a random variable). You can go back to the regression equation (y=Xbeta + e) and take the expectation and variance to get the mean and variance of y. E(y)=Xbeta (mean is zero for error and expectation of Xbeta is equal to itself since in regression, you assume Xbeta to be constant (actual beta is a number)) and Var(y)=sigma^2 (variance of Xbeta is zero since you assume it to be constant similarly).
Regarding your point on likelihood, It is probably confusing due to differences in notations. You always consider likelihood as the p(data|parameters) (here data is y, since it is what you assume to be random) and you estimate the parameters that maximize this probability (i.e. that explains your data best).
I hope these help..
