3
$\begingroup$

I have a set of data (underway observation along cruise track) which includes one dependent variable A and three independent variables B, C, and D. It's known that A is related with each of the three factors, so I want to find a numerical relationship that allows me to express A by means of B, C, and D, and would like to be able to evaluate the relative contribution of each independent variable to A.

I'm aware PCA is the tool to use in this situation, and I've tried it in Excel, only to get a biplot where all four variables are treated equally as the same type of dependent variable, and variable A is used to explain those principal components. What I really want is to see how A behaves under the joint influence of B, C, and D, and I don't care how B, C, and D themselves relate to each other.

(I'm working in MATLAB.)

$\endgroup$
3
  • 2
    $\begingroup$ It seems that you don't need PCA at all, you need to regress A on B, C, and D. $\endgroup$
    – amoeba
    Commented Mar 30, 2015 at 16:56
  • $\begingroup$ My final goal is to be able to predict A with BCD based on what the current data give me. So I think I'll need the multivariate analysis to derive a relationship between A and BCD, rather than looking at A vs B, A vs C, A vs D respectively. $\endgroup$
    – Lin
    Commented Mar 30, 2015 at 20:25
  • 1
    $\begingroup$ Yes, @Lin, this is still called regression -- more precisely, "multiple regression" with A as dependent variable and B,C,D as independent variables. My point was that your question has nothing at all to do with PCA. $\endgroup$
    – amoeba
    Commented Mar 30, 2015 at 22:17

1 Answer 1

4
$\begingroup$

As amoeba wrote in the comments, this is a multiple regression problem and has nothing to do with PCA (see Linear regression on Wikipedia). I here use the standard assumptions of a linear model with additive noise:

$$ A = b ~ B + c ~ C + d ~ D + k + E. $$

Here $b$, $c$, and $d$ are the scalar regression coefficients for your three explanatory variables, $k$ is a constant offset, and $E$ captures everything that can't be fit into the other terms (the error).

In Matlab, to compute $A$ if given all the values on the left-hand side, one would write in matrix notation:

A = [B C D ones(n, 1)] * [b c d k]' + E

assuming that all variables are represented as column vectors, and that $n$ is the number of data points.

To solve this equation for the coefficients $b$, $c$, $d$, and $k$ for given $A$, you can use the backslash operator:

coeff = [B C D ones(n, 1)] \ A

This operator gives a least-squares solution, meaning $A$ is determined such that the sum of squared errors ($\sum_{i = 1}^n E_i^2$ or sum(E .^ 2) in Matlab) is as small as possible. You can then extract the single coefficients:

b = coeff(1)
c = coeff(2)
d = coeff(3)

These coefficients are estimates of how strongly the variables $B$, $C$, and $D$ contribute to $A$.


Your title says that you are interested in a "percentage of contribution" of the independent variables, and I assume that you mean percentages of explained variance. The problem is that if the variables $B$, $C$, and $D$ are correlated (share variance among themselves), there is no simple answer to that question, because the variance of their contributions to $A$, which are $b ~ B$, $c ~ C$, and $d ~ D$, do not combine additively. Assuming that there are no correlations, you can compute those contributions like this:

var([B C D] * diag([b c d])) / var(A) * 100

which gives you a three-element vector with estimates of the relative contribution in percent of the three independent variables.

$\endgroup$
12
  • 1
    $\begingroup$ This is a nice explanation and good Matlab examples. +1 in particular for the backslash operator. $\endgroup$
    – amoeba
    Commented Mar 31, 2015 at 20:47
  • $\begingroup$ This is an excellent answer to my question! Sorry I wasn't able to get back to you sooner. I see the multiple regression gives me a linear equation in which A is expressed by means of B C D. This should work on my data set because A is supposed to correlate with those 3 variables linearly. Just curious, if A is non-linearly associated with BCD, will this method still work? Another question is that you mentioned the regression works well only if BCD are independent to each other. Does it still hold true if they are somehow dependent, even slightly related to each other? Thanks for the help! $\endgroup$
    – Lin
    Commented Apr 1, 2015 at 18:18
  • $\begingroup$ @Lin, if the relation is only weakly nonlinear, you can still catch the linear part using regression. For strongly nonlinear relationships you need other methods, see e.g. support vector regression. $\endgroup$
    – A. Donda
    Commented Apr 1, 2015 at 18:23
  • $\begingroup$ The regression itself does not have problems with independent variables that are correlated among themselves (as long as they are not linearly dependent, i.e. [B C D] is not singular). The problem exists only for quantifying their relative contribution in the form of percent variance explained. $\endgroup$
    – A. Donda
    Commented Apr 1, 2015 at 18:25
  • 1
    $\begingroup$ @A.Donda, I will make sure to do that and we may chat further afterwards. Thank you for you time! $\endgroup$
    – Lin
    Commented Apr 2, 2015 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.