0
$\begingroup$

I'm new to ML/stats so got confused with what I supposed was simple notation.

For simplicity, say I have a data set with just one column:

enter image description here

From probabilistic perspective, I had understood that X represents a random variable, and each row value is an observation (i.e. realization) of this random variable. X then has a distribution, which let's assume is Gaussian.

I got confused however when I studied the Maximum Likelihood Estimate (MLE) method. In MLE literature, it's assumed that the individual row values are iid, which then permits us to multiply their distributions:

enter image description here

Isn't iid however a property of random variables? In other words seems MLE is assuming that each row value by itself is a random variable with its own distribution. Doesn't this contradict the previous interpretation that X is a random variable and the row values are just specific realizations?

$\endgroup$
0
$\begingroup$

Yes, $X$ is a RV, and the listed values can be interpreted as its realizations. In MLE, we don't multiply their distributions. We again use $X$'s distribution but substitute our data points in it. $p(x_i|\theta)$ actually means $p_X(x_i|\theta)$, i.e. the value of PDF of $X$ at the specific point $x_i$ (something that represents some sort of probability measure). MLE just tries to maximize your probability of having those realizations.

And right, being iid is a property belonging to RVs although, not related to this discussion directly, technically constants are also RVs (with probability of equaling their value is $1$), and constants are also iid. But, here we're not talking about the iid-ness of constants/realizations. Each realization, $x_i$, is assumed to be drawn from the distribution of $X_i$ that has the same distribution with $X$. Consider the rows as grades of students $i=1,2,3,4$, i.e. $X_i$. Each student's grade can be thought of as a random variable. Having realizations of them doesn't change this.

This confusion arises in many other places, not just MLE; for instance, sample mean. Your sample mean is $(100+200+300+400)/4=250$. But, sample mean has also a distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.