2
$\begingroup$

In a linear regression of multiple variables, $$ y = X\beta^T + \epsilon $$, where $X$ and $y$ are the independent and dependent variables, we can estimate OLS $\beta$ as follows: $$ \hat \beta = (X^T X)^{-1} X^T y $$

I wonder if it is possible to design a neural network such that given $X$ and $y$ (with a variable number of rows) as input, it will output $\hat \beta$.

$\endgroup$
16
  • 1
    $\begingroup$ I have had trouble getting neural nets to work with variable length input vectors as inputs (of course there's much work on variable sequences) to work; I hope someone else here has interesting advice to share with us on this matter. $\endgroup$ Commented May 6, 2023 at 18:03
  • 2
    $\begingroup$ So you want the neural network to find the $\beta$ parameters? Why? It's like building an artificially intelligent robot to build you a hammer. You can simply use OLS for that for a much simpler and more robust solution. $\endgroup$
    – Tim
    Commented May 6, 2023 at 19:02
  • 2
    $\begingroup$ Then could you give us more details? Please edit to describe what exactly is the problem. $\endgroup$
    – Tim
    Commented May 6, 2023 at 19:08
  • 1
    $\begingroup$ Why do you need gradient updates at all? The "neural network" is simply directly computing $\hat \beta$ using matrix arithmetic. One example is to compute the SVD $X = USV^T$ and then $\hat \beta = V S^{-1} U^Ty$. This question reads like an XY Problem -- what problem are you trying to solve, and why is a neural network an essential part of computing the regression coefficients? $\endgroup$
    – Sycorax
    Commented May 7, 2023 at 1:19
  • 1
    $\begingroup$ It sounds like X-Y problem. Tell us what is the underlying problem you are trying to solve. Why do you think you need it? Please edit your question for more details, otherwise, it's not possible to answer it because we would have to make many guesses on what you mean. $\endgroup$
    – Tim
    Commented May 7, 2023 at 6:43

0

Browse other questions tagged or ask your own question.