Is weighted 2d regression the same as a projection of 3d regression?

Question

I have a set of data with two independent variables (x, y) and one dependent variable (z). I want to find a regression line that minimizes the perpendicular distances to the regression line, rather than the vertical distances as is customary for simple linear regression. From what I have read, this is a special case of Deming regression called orthogonal regression, which is a special case of total least squares.

I really only need the slope of the regression line for the x and y components. So, I was thinking I might be able to transform the problem into one with one independent variable (x) and one dependent variable (y), weighting each data point by the old z component. From what I have read, orthogonal regression of "2-dimensional" data such as this, has a convenient analytic formula in the complex plane. It also seems to be more efficient, computationally, than performing the regression with all three components. Admittedly, I don't know how to derive the regression formula for all three components.

Would the transformed problem yield the same results as the original? Since orthogonal regression computes errors using both dependent and independent variables, my intuition says they'd be the same. However, I lack the mathematical skills to confirm this.

Edit:
I realize now that my usage of "projection" is ambiguous. If you did principal components analysis on the data, you'd get three eigenvectors. I want to project the eigenvector with the smallest biggest z component, onto the xy plane, then rotate it 90 degrees.

Answer 1

In 2D, to find the best-fit line by orthogonal least squares, you can keep only the eigenvector with largest eigenvalue. Equivalently, you can remove the eigenvector with smallest eigenvalue.

In N-D however, to find the best-fit hyperplane by orthogonal least squares, only the second approach works: you remove the eigenvector with smallest eigenvalue.

That is, orthogonal regression is usually understood to mean finding a hyperplane that minimizes the sum of distances between the data points and the plane. In general a (hyper-)plane can be specified by a point $\mathbb{p}_0$ on the plane and a vector $\mathbb{n}$ normal to the plane, so that points $\mathbb{p}$ on the plane satisfy $$(\mathbb{p}-\mathbb{p}_0)\cdot\mathbb{n}=0$$ For a general point $\mathbb{x}$, its (signed) distance from the plane is then $$d=(\mathbb{x}-\mathbb{p}_0)\cdot\mathbb{n}$$ So orthogonal regression amounts to choosing the normal vector $\mathbb{n}$ to be the (covariance-)eigenvector with smallest eigenvalue. For example, if the $n$-D data fall on an $(n-1)$-D hyperplane, then this smallest eigenvalue will be $0$.

So "3D orthogonal regression" would usually mean fitting a plane, rather than a line. For a non-vertical plane, we can interpret it as a 2D function $Z(x,y)$. If the gradient of this function is $$\mathbb{g}\equiv\boldsymbol{\nabla}Z(x,y)=\begin{bmatrix}\tfrac{\partial Z}{\partial x}\\\tfrac{\partial Z}{\partial y}\end{bmatrix}$$ then the 3D vector $$\mathbb{n}=\begin{bmatrix}-\mathbb{g}\\\phantom{-}1\end{bmatrix}$$ will be normal to the plane.

The unit vector will be in the same direction, so the $(x,y)$ components of the orthogonal eigenvector will be aligned with the gradient of the plane.

I hope this helps!