# Likelihood when points aren't i.i.d

If we assume that we have a set of N data points given as $$\textbf{X}$$ and corresponding targets vectors $$\textbf{T}$$, where both represents matrices in this case. For an i.i.d we could write the likelihood (by using vectorization), through:

$$p(\textbf{T}\vert f,\textbf{X}) = \prod_{i=1}^{N}p(\textbf{t}_i\vert f,\textbf{X})$$ Where I've denoted f as an linear estimator of $$\textbf{t}_i$$. Now I want to write the likelihood for a not i.i.d, I thought it might look like:

$$p(\textbf{T}\vert f,\textbf{X})= \prod_{i=1}^{N}p(\textbf{t}_i\vert \textbf{t}_{i+1},...,\textbf{t}_N,f,\textbf{X})$$

But this doesn't seem right, it seems like I've forgotten something, and if that's not the case, it feels like I'm calculating the same points over and over. How can I fix this ?

Personally I think it should look like:

$$p(\textbf{T}\vert f,\textbf{X}=p(\textbf{t}_1\vert f,\textbf{X})*p(\textbf{t}_1\vert \textbf{t}_2,...,\textbf{t}_N,f,\textbf{X}) * p(\textbf{t}_2\vert f, \textbf{X})*p(\textbf{t}_2\vert \textbf{t_1},...,),....p(\textbf{t}_N$$

$$p(t_1, t_2, t_3, ..., t_N) = p(t_1)p(t_2|t_1)p(t_3|t_1, t_2) ... p(t_N| t_1,..., t_{N-1})$$
Why? For just two variables, it follows from the definition of conditional probability: $$\Pr(A,B) = \Pr(B) \cdot \Pr(A|B)$$
• Perhaps you can clarify something that I might have misunderstood, should I interpret it as $\textbf{T}$ is given in my case, therefore assuming that I have knowledge of these, or should I base my calculations around the fact that I dont have $\textbf{T}$, rather I calculate the likelihood for $\textbf{t}_1$ in the first place, and then extend this argument when adding more points ? – A.Maine Nov 15 '18 at 10:35
• perhaps I should reformulate, is this a correct interpretation? $$p(t_1, t_2,...,t_N) = p(t_1 \vert f,\bm{X})*p(t_2\vert t_1,f, X)*...*p(t_n \vert t_1,....,t_{N-1} \vert f, X)$$ – A.Maine Nov 15 '18 at 10:50
• If you add $f$ and $X$ to the conditioning set on the left hand side, then yes. The factorisation formula is fully general. By the way, I'm not sure what's what in your notation. Liklihoods usually refer to the joint density or mass of the data ($x$) conditional on (if Bayesian) or written as a function of (if frequentist) the parameter vector ($\theta$) one is trying to estimate, something like $p(x_i, .x_n| \theta)$ – CloseToC Nov 15 '18 at 10:55