3
$\begingroup$

I am wondering how I can map this problem to something known. Let us start with a standard linear regression framework, and suppose we want to reconstruct an observed signal $y$ from single known components $\mathbf {A}$ that are mixed following a single unknown vector of weights $x$. We know $\mathbf {A}$ from reliable data and use it as a reference. Assuming those weights are positive, this is the solution to the NNLS problem:

$$ {\displaystyle \operatorname {arg\,min} \limits _{\mathbf {x} }\|\mathbf {Ax} -\mathbf {y} \|_{2}^{2}} \quad \textrm{subject to } x ≥ 0. $$

Now I am wondering, what if instead of the matrix $\mathbf{A}$, we know a parametric description of the single components in the form of, say, parameters of their distribution for each entry? My problem is that this $\mathbf{A}$ has now only "pointwise" information of the reference value when in reality, one might be more confident of certain entries than others (having different variances in the reference). It reminds me of some Bayesian models, but I wonder whether there is a simpler route to get there.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ In this comment, I'll outline the option of using the distribution on $\mathbf{A}$ as a prior for Bayesian inference. If you want the simplest version of this, do Maximum a Posteriori "Bayes", which is simply $\underset{\mathbf{x},\mathbf{A}}{\textrm{argmin}}\Vert\mathbf{A}\mathbf{x}-\mathbf{y}\Vert_2^2 - \log p_A(\mathbf{A})$ subject to $\mathbf{x}\geq\mathbf{0}$, where $p(\mathbf{A})$ is your distributional assumption for $\mathbf{A}$. If it is defined entrywise, it will be $\log p_A(\mathbf{A}) =\sum_{i,j} \log p_{i,j} (a_{i,j})$. $\endgroup$
    – John Madden
    Commented Nov 20, 2022 at 15:07
  • $\begingroup$ PS you have the tag non-negative matrix factorization, which refers to something different. $\endgroup$
    – John Madden
    Commented Nov 20, 2022 at 15:58
  • $\begingroup$ Thank you, @JohnMadden. I edited the tags, and I will also explore a MAP approach for this problem $\endgroup$
    – Gian
    Commented Nov 20, 2022 at 16:35

1 Answer 1

Reset to default
4
$\begingroup$

One type of model you describe is Total Least Squares (TLS, https://en.wikipedia.org/wiki/Total_least_squares), but it does not have the non-negative constraint you place on elements of the coefficient vector $\bf{x}$. In typical usages of this nomenclature, the errors in the elements of the matrix $\bf{A}$ are independent identically distributed (IIID), so have the same standard-deviation of errors, but in general, it does not need to be IID. See that page for more info (note that it has $\bf{X}$ instead of $\bf{A}$).

A slightly more general model is called Errors In Variables (EIV, https://en.wikipedia.org/wiki/Errors-in-variables_models)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.