Can least-squares linear regression ever produce no solution at all?

Is it ever possible for least-squares linear regression (linear in both features and weights) NOT to produce a solution? That is, after we set each partial derivative to zero, can the resulting system of n linear equations in n weights be "inconsistent"? An example of "inconsistent system of linear equations" is two parallel lines (for 2 weights) or two parallel planes (for 3 weights). If the answer is yes, can you please give a simple numerical example with 2 or 3 data points?

• Differentiability is not necessary--it's just a distraction. A sum of squares always has a global minimum, because it is a continuous function, bounded below by zero, with an upper bound on any compact set. Thus, if you want there to be exceptions you will have to modify the problem by introducing constraints on the solution.
– whuber
Oct 20 '19 at 17:44

A linear system $$Ax=b$$ can have no-solution case due to the inconsistencies introduced in its equations. But, linear regression does not aim to satisfy $$Ax=b$$, it merely tries to minimize $$||Ax-b||_2$$. This is a simple optimization problem, and you'll have your solution, either unique or infinitely many. Intuitively, one can always draw a line as best fit to a given set of points.