The motivation for this question comes from the following: When I was studying statistics for the first time long ago, no one presented the mathematical concepts behind linear regression, like the one in the figure below, in which "Searching for θ that minimizes Error is the same as LOCATING the point Xθ as close as possible in the column space than any other point in the C(X).". This geometric approach, that came from Linear Algebra, helped me a lot when I started to learn with a more rigorous approach. Therefore, I'm looking for the same approach, but for the concept of Identifiability. Thank you.
Hi: Note that there's another definition of identifiability (one that's totally different from the one I'm referring to here ), but, for your picture, the definition would be that there does not exist a second projection onto the column space of $X$ that also minimizes error term. Note though that this is not the meaning of the term "identifiability" that is used in econometrics. In econometrics, "identifiability" has a different meaning that I'm not referring to here.