Regression to solve system of a system of linear equations? The question I am going to ask probably has a straight-forward answer, I just don't know how to frame my question in a way that would be 'normal.'
I have a set of a set of linear equations that I would like to do a regression on. The main equation will always look like the standard matrix linear equation system:
$Ax = b$
where A is a 3x3 matrix, x is 3x1 and b is 3x1. However, I can gather data to make 6 equations of this form and A should be the same for each one.
$Ax_1 = b_1 \\ Ax_2 = b_2 \\ Ax_3 = b_3 \\ Ax_4 = b_4 \\ Ax_5 = b_5 \\ Ax_6 = b_6$
Please note that $x_1$, $x_2$, $b_1$, $b_2$, etc. are not single values, but are 3x1 vectors. So for a given matrix A, I can produce 6 sets of x's ad b's. Because I am gathering data from sensors that will have noise, I don't want to just solve $Ax_1 = b_1$ and throw away the data I can get from the other 5 systems. Is there a way to do a linear regression not on a system of linear equations, but on a system of a system of linear equations?
 A: if you take the first row of A, i.e. $A_{1} = [a11,a12,a13]$ and multiply it by design matrix X that consists of x's stacked next to each other $X=[x1,x2,...,x6] \in M_{3,6}$ you should be getting the 1st coordinate of the b's $B_1=[b11,b21,b31,...,b61] \in M_{1,6}$
$B_1 = A_{1} X + \epsilon$
so essentially you run linear regression of X onto B, the first coordinate of b's, to get the 1st row of A, second coordinate to get the 2nd row and third coordinate for the 3rd row
i hope this is making sense.
some warning: think hard about the error. is it additive and independent of the x's? if it is true for one of the coordinates how about the combination of the three? it seems non-trivial question but i believe my suggestion will give you a reasonable approximation
A: Easy and fast approach. Run each of the six regressions separately, repeat experiment if needed to get more observations.
Per least-squares theory, there is an associated variance on each of your estimated parameters.
So weight all the results for each parameter by the inverse of its variance to the sum of 1/variance for each separate estimate obtained. In essence for correlated sensors, we do want to combine data, but without collinearity associated issues.
To the extent that the expected precision produced by another sensor is very poor, either gather more observations or remove it from your list. 
EDIT: This approach has a few advantages, in addition to fast and easy, over what has been proposed. Examination of single regression suggests if more data sampling is required and for what sensor. Also, one can delete an input from a poor sensor and not blindly incorporate it. In addition, there is a theoretical foundation for combining estimates based on their variances, while not clear if such a clear standing exists for the previously proposed method. Lastly, one can compute correlations between the sensors parameter estimates to gain insight.
