19
$\begingroup$

This may be a simple explanation (I'm hoping anyway).

I have done some regression analysis in Matlab using the regression toolbox. However, I have come across a study that states this:

"Using regression analysis, it was possible to set up a predictive model using only four sonic features that explain 60% of the variance"

The link to the article is here if needed: Article

I'm not 100% sure what this means, but I'm hoping its something simple. Also is 60% a good thing? I have tried to search for this but because there is always a percentage before the word 'variance', its hard to find answer.

$\endgroup$
0

2 Answers 2

17
$\begingroup$

I will try to explain this in simple terms.

The regression model focuses on the relationship between a dependent variable and a set of independent variables. The dependent variable is the outcome, which you’re trying to predict, using one or more independent variables.

Assume you have a model like this:

Weight_i = 3.0 + 35 * Height_i + ε

Now one of the obvious questions is: how well does this model work? In other words, how well the height of a person accurately predicts – or explains – the weight of that person?

Before we answer this question, we first need to understand how much fluctuation we observe in people’s weights. This is important, because what we are trying to do here is to explain the fluctuation (variation) in weights across different people, by using their heights. If people’s height is able to explain this variation in weight, then we have a good model.

The variance is a good metric to be used for this purpose, as it measures how far a set of numbers are spread out (from their mean value).

This helps us rephrase our original question: How much variance in a person’s weight can be explained by his/her height?

This is where the “% variance explained” comes from. By the way, for regression analysis, it equals the correlation coefficient R-squared.

For the model above, we might be able to make a statement like: Using regression analysis, it was possible to set up a predictive model using the height of a person that explain 60% of the variance in weight”.

Now, how good is 60%? It’s hard to make an objective judgement about this. But if you have other competing models – say, another regression model that uses the age of a person to predict his/her weight – you can compare different models based on how much variance is explained by them and decide which model is better. (There are some caveats to this, see ‘Interpreting and Using Regression’ -- Christopher H. Achen http://www.sagepub.in/books/Book450/authors)

$\endgroup$
2
  • 1
    $\begingroup$ That has certainly answered a large proportion of my question. In terms of why the authors are stating this like its of huge significance, I don't know. So, if this is the R-sqaured value and we go back to your example: say we did use a model for 'age' that had a variance of 80%, and then and model for 'height' that had a variance of 85% to predict a person's weight, I take it that the latter model would be more significant? Thanks for the book link, I purchased it last night as I'll be using regression quite a lot in the forthcoming months. $\endgroup$ Commented Mar 26, 2016 at 8:30
  • 1
    $\begingroup$ Yes, you can conclude that the latter model is better in its ability to predict (or, explain) weight of a person, ceteris paribus. BTW, you stated this as "model had a variance of 80%", but it should be "model explains 80% of the variance". $\endgroup$
    – Vishal
    Commented Mar 28, 2016 at 20:50
12
$\begingroup$

The authors are referring to the $R^2$ value for the model which is given by the formula

$$ \frac{\sum_{i=1}^{n} (\hat{y}_i - \bar{y})^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2} $$

where $y_i$ is the observed value, $\hat{y}_i$ the least squares fitted value for the $i^\text{th}$ data point and $\bar{y}$ is the overall mean. We sometimes think of $R^2$ as a proportion of variation explained by the model because of the total sum of squares decomposition

$$ \sum_{i=1}^{n} (y_i - \bar{y})^2 = \sum_{i=1}^{n} (\hat{y}_i - \bar{y})^2 + \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 , $$

the latter term being residual error that is not accounted for by the model. The $R^2$ basically tells us how much of the overall variation has been "absorbed into" the fitted values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.