Solving simple linear equation using Matrices [closed]

Question

I need to find a matrix A whose dimensions will be 1 x n and I have input matrix X whose dimensions are n x m and another matrix Y whose dimensions are 1 x m.

I have the following equation:

$A * X = Y$

To find A we can write the above equation as

$A = Inverse(X) * Y$

right?

But, as X is not a square matrix we cannot have its inverse. So, what will be the best possible way to solve A?

After a fair bit of research I have found that $X = L * B$ where L is the left inverse of A. Is this the right approach? If so how to find the left inverse in R Programming? I have tried the below code, but A and the PredA should have same values which they are not.

m = 5
n = 10

A = t(matrix(runif(n)))
X = matrix(round(runif(m * n)), nrow = n)

# Y = t(matrix(runif(n)))

# # Usually A * X = Y
Y = A %*% X

library(MASS)
# # Now to find A (Predicted A)
PredA = Y %*% ginv(X)

# # PredA should be equal to A right?
PredA == A # Returns False

What could be the basic thing that I'm missing?

I have tried two approaches:

$PredA1 = (X'*Y) * Inv(X*X')$

$PredA2 = Y * Inv(X)$

both resulting in same values (PredA1 = PredA2) but none of them are equal to the actual value (PredA1 not = A) and (PredA2 not = A)

NOTE: I am new to Matrix Algebra. So, I've surely missed something very basic

Answer 1

General Question

In general*, if $n<m$ your linear system is over-determined and will not have an exact solution, while if $m<n$ your system is under-determined and will have infinitely many solutions. (*assuming $X$ is full rank, for simplicity.)

In the over-determined case, the typical approach is to find an approximate solution $A$ that minimizes some norm of the residual $||AX-Y||$. For the standard $L_2$-norm, this will be the linear least squares solution.

In the under-determined case, there are various approaches, but a simple one is to choose the particular solution $A$ that itself has minimum norm $||A||$. For the $L_2$ case, this corresponds to a ridge regression solution.

In either $L_2$ case, the solution can be obtained using $X^+$, the pseudo-inverse of $X$.

I do not know how to get this in R, but in Matlab the command is pinv(). (Actually, in Matlab the \ command will solve the system correctly for whatever $m$ and $n$.) Note that numerically, you should really (almost) never explicitly compute an inverse to solve a system of equations. The details of the techniques used in practice (QR,SVD,...) are beyond the scope of this answer, however.

Specific Code

Note that in your example, if I understand it correctly the system is under-determined, but you use a "method of manufactured solutions" to get a particular $Y\equiv AX$ based on your pre-chosen $A$. I cannot say if your R syntax is correct.

However $A_{\mathrm{calc}}\neq A$ would not be surprising in general, due to roundoff error. If this seems strange to you, please read that linked answer!

EDIT: I answered this pretty late, so was sloppy due to tiredness. (The question may be closed/migrated, but I do not want to put out misinformation, so ...)

Given that your system is under-determined, then the "roundoff" comment really would apply more to $||A_{\mathrm{calc}}X-Y||\neq 0$. Given that you appear to generate a random $A$, in the case that your R solution is doing the pseudo-inverse method, then the most likely cause of your disagreement is that $A$ includes a component in the null-space of $X$, which has been changed in $A_{\mathrm{calc}}$ to lower its norm (i.e. in this case you would see $||A_{\mathrm{calc}}||<||A||$).

So you could very well see a "finite" difference in this case.