Questions tagged [sums-of-squares]
sum of squares plays an important role in statistical models based on the normal distribution, like ANOVA.
177
questions
0
votes
0answers
13 views
Simple Sum of Squares Problem [closed]
I need your help: I want to solve a problem and I am stuck. This is known: X'X, (X'X)^-1, X'Y, SSM, sigma^2, n and p. Now I just want to calculate SSE, R^2 and the estimated covariance matrix, but I ...
1
vote
0answers
17 views
Distribution of the squared length of a multivariate normal vector [duplicate]
Let $\mathbf{X} \sim \mathcal{N}(\mathbf{\mu}, \mathbf{\Sigma})$ be a multivariate Normal variable with $d$ dimensions. I'm interested in the marginal distribution of $\|\mathbf{X}\|^2 = \sum_{i=1}^d ...
0
votes
0answers
15 views
What exactly is explained sum of squares, and why do we focus on minimizing error rather than maximizing explained variability?
In the third of my questions about decomposing the total sum of squares, I want to focus on the sum of squares of the regression.
I can make sense of what the sum of squares of the residuals means: ...
1
vote
1answer
39 views
In a PCA setting, is there any relationship with the sum of squares of the scores (t1) with the eigenvalue of that principal component?
When one computes the vector of scores (t1) using the principal component (p1) the data is being projected over the direction of biggest variation. One could measure the distance between the point ...
3
votes
1answer
47 views
What is the lowest sum of squared errors (sse) of a dataset?
Given $X \subset R^d$ a set of data points, $z \in R^d$ a vector, $SSE(X)=\sum_{i=1}^{|X|}||X_i-z||^2$, I wonder what is the best value of $z$ so that $SSE$ is the lowest?
I suspect that $z$ is the ...
1
vote
0answers
15 views
How are terms in SS (corrected sum of squares) independent?
What does the highlighted test mean? How are the terms not independent? Can someone elaborate in detail?
0
votes
0answers
6 views
Do LS-means correspond only to type 3 (marginal) ANOVA or do they match the type 2 too?
I understand, that sequential type 1 ANOVA corresponds to unadjusted, raw (data-based) arithmetic means, and marginal type 3 ANOVA corresponds to LS-means (model-based predictions). In case of no ...
0
votes
0answers
9 views
ANCOVA $\eta^2$ definition
I am struggling to find references to understand what is the exact definition of $\eta^{2}$ is two way anova.
Let's say I am estimating the following ANCOVA model:
$$
y_{ijk} = \mu + \alpha_i + \...
0
votes
1answer
34 views
Source for BIC with residual sum of squares?
Wikipedia states that under certain circumstances, $\mathrm{BIC} $ can be calculated as:
$$\mathrm{BIC} = n \ln(\mathrm{RSS}/n) + k \ln(n),$$
where $\mathrm{RSS}$ is the residual sum of squares. ...
1
vote
1answer
64 views
Kruskal–Wallis one-way analysis of variance is related to what kind of regression?
One-way anova is similar to regular linear regression because both use the F-test which involves sums of squares among other reasons.
Is Kruskal–Wallis one-way analysis of variance similar to some ...
3
votes
3answers
528 views
Residual sum of squares in a regression
I understand that in a linear regression model, the residual sum of squares will either remain same or fall with the addition of a new variable.
What if the two models were
$$
I \colon y_i=\...
2
votes
2answers
95 views
Simplifying the Matrix Form of the Solution to Ridge Regression
I'm trying to understand how to obtain the solution to an objective function by solving for the parameter vector $\theta$ in ridge regression. I found an example here from Naomi which takes an example ...
1
vote
0answers
295 views
What is the correct implementation of BIC with residual sum of squares?
BIC is most often calculated by maximizing the log likelihood function. However, it is also possible to calculate BIC with residual sums of squares. This is pretty easy to find online and not an issue ...
2
votes
0answers
105 views
Formula for type III sum of squares of the intercept term in linear multiple regression
assume we have the regression model:
$$Y = b_0 + b_1 x_1 + \dots + b_k x_k + \varepsilon $$
I know the formulas for all type III sum of squares for the regression terms except the formula for SS of ...
4
votes
2answers
115 views
Coefficient of determination relationship?
$R^2 = \frac{SSREG}{SSTOT}$ or $R^2 = 1-\frac{SSRES}{SSTOT}$
If $X$ is the predictor random variable for science SAT
and $Y$ is the predictor random variable science GPA
given by equation $$\hat Y =...
2
votes
1answer
403 views
Sum of Squared Error Chi-Square distribution degree of freedom in Multilinear Regression
In this link it says that $Y$ variables has zero covariance (because covariance matrix has only diagonal terms) which implies they are independent.
Actually in linear regression $Y$ takes its ...
0
votes
1answer
378 views
What is the difference between the residual, lack of fit and pure error In F test for Regression Analysis?
What is the difference between the residual, lack of fit and pure error and how to calculate each of them in ANOVA F test for nonlinear regression?
2
votes
1answer
67 views
Two way ANOVA, no difference between tests based on type I vs type II sums of squares
I'm learning about two way ANOVA possibly with interaction.
I'm following this tutorial http://www.sthda.com/english/wiki/two-way-anova-test-in-r
This is their code.
...
0
votes
1answer
12k views
Calculating SSE in R
I'm following the "Intro to Statistics" course in Data Camp and I'm having some trouble as it seems that the course is contradicting itself: https://s3.amazonaws.com/assets.datacamp.com/production/...
0
votes
0answers
46 views
Why is regression line represented as $y = b0 + b1 * x$?
I am new to Data Science and ran into Regression Line formula which is
$ y = b0 + b1 *x $ (where x is dependent variable, y is predicted variable)
I understood the meaning of this formula as a ...
0
votes
1answer
303 views
Better to Minimize Absolute Error or Sum of Squared Error?
I have an Excel model which predicts the number of customers for a given month. The prediction depends on a churn rate. I have the absolute error (actual vs predicted), along with squared error and ...
2
votes
1answer
1k views
Why error sum of squares has n-2 df (possibly not duplicate, please read on)? (Regression Question Series - Part 4)
In simple linear regression, the error sum of squares is given by
$$
\text{SSE} = \sum_{i=1}^n(y_i - \hat{y_i})^2 \\
\hat{\sigma}^2 = s^2 = \dfrac{\text{SSE}}{n-2}
$$
where $n-2$ is the degrees of ...
0
votes
1answer
97 views
Minimize Logged Sum of Squares?
When numerically maximizing the likelihood function it is standard practice to do this indirectly by minimizing the negative log-likelihood. When numerically minimizing the residual sum of squares (...
2
votes
0answers
88 views
R: anova(lm): What is the Sum Sq when we have two inputs
What is the formula to calculate the Sum sq column for the inputs?
Answers to some other questions state, that it should be
$RSS = \sum (\hat Y_i -\bar Y)^2$,
yet this is false, it only gives the ...
2
votes
2answers
589 views
Showing that $\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$ for the generalized linear model [closed]
Exercise :
Prove that for the generalized linear model, it is :
$$\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$$
Question : How would one proceed with proving that for the generalized ...
1
vote
1answer
52 views
Squaring floats between -1 and 1 reduces sum of squares, so why do it? [duplicate]
I have been learning basic statistical testing as it relates to agriculture and have become familiar with the common practice of summing squared raw deviation values, whether in something simple like ...
2
votes
2answers
156 views
Calculating F-statistic, why is SS used instead of just $r^2$?
The calculations and question are for a simple regression (one independent and one dependent variable).
SSres = SSy * (1 - $r^2$)
SSreg = SSy * $r^2$
F = (SSreg / DFreg)/(SSres / DFres)
So, I've ...
1
vote
1answer
247 views
Understanding ANOVA as regression / type III SS in R
I'm confused about how type III SS are calculated for a "main effect". According to what I have read, Type III SS is calculated by evaluating the change in the SSE by removing only the variable in ...
0
votes
1answer
874 views
2-way ANOVA in Python statsmodels yields different Sum of Squares than SPSS
I am learning to use Python for my statistical analyses, and while figuring out how to perform a 2-way ANOVA with statsmodels I found that my Python code yielded slightly aberrant values. Comparing ...
1
vote
0answers
97 views
In regression, when partitioning SS among predictors, what determines which predictors get the SS that can be attributed to more than one predictor?
In regression analysis, predictors sometimes correlate (and in my field, psychology, they always do; often because they are simply measurements of the same aspects of human psychology).
If predictors ...
4
votes
2answers
901 views
How should Type II SS be calculated in a mixed model?
I have a data set (and corresponding mixed model) which gets very different p-values for one of the two-way interactions when tested using Type I (sequential, taking care that it's last), and Type II (...
0
votes
0answers
31 views
Deriving the F-test from ${{SSE_R-SSE_F}\over{(n-q)-(n-p)}}/{{SSE_F}\over{n-p}}$
Given a Full and Reduced model, the F-test to see if the reduced model is significant is given by
$$
{{SSE_R-SSE_F}\over{(n-q)-(n-p)}}/{{SSE_F}\over{n-p}}
$$
I'm trying to understand how this is ...
2
votes
0answers
89 views
Find the Sum of Squares and using them complete an F-test for goodness of fit
I have this question ( would be grateful if someone could put it on for me)
https://gyazo.com/5afe5d7c9d12acdff8f9c55db74f97c9
And I am concerned with part b).
I know what $RSS = SSE$ is due to the ...
2
votes
1answer
2k views
Which one to choose Type-I, Type-II, or Type-III ANOVA? [duplicate]
I don't understand what the difference is between TypeI and TypeIII?
Since my background is not very mathematics, it's very difficult for me to understand this mathematical notation;
Type I SS: SS(A) ...
0
votes
1answer
174 views
Does forward model selection by $R^2$ or SSR differ?
I'm trying to implement forward selection and need to add a feature only if it will make the sum of squared residuals (SSR) lower.
I am using Python's statsmodels ...
0
votes
0answers
56 views
The Sums of squares in 3 way Anova in R output has changed as the order of the variables in the modes changes [duplicate]
I am trying to fit a 3 way ANOVA using R. Then i realized the sums of squares of the output has changed when the order of the variables in the model are changed.Can anyone figure out the reason for ...
5
votes
2answers
258 views
Mixed-effect model single term deletion — should I change my random effects?
In short
I recently had a little conversation on the lme4 project's GitHub on how to properly test the significance of effects in a mixed-effect model, which made ...
2
votes
0answers
198 views
Test significance using Type II ANOVA following Type III ANOVA?
I hope you can help me with a theoretical question about how to proceed in my analysis. I have come across many posts discussing differences between Type II and III SS Anovas, however, because I don't ...
1
vote
0answers
111 views
Trying to understand hierarchical testing of nested regression models
I've learned multiple regression, but never did I learn hierarchical regression (i.e., the hierarchical testing of nested regression models) before.
Based on the image attached, I assume because the ...
3
votes
1answer
816 views
In regression analysis, How can R^2 represent the total explained variance, if it can be computed from an equation with only unique contributions?
Background
In regression analysis, $R^2$, the squared multiple correlation, represents the proportion of explained variance by the regression model. Most software's default setting uses Type-III sums ...
5
votes
1answer
2k views
Is Square Root of the Variance of a Regression Coefficient the Standard Error?
Quick question, in the textbook "Introductory Econometrics", the variance of a Regression Coefficient is given as:
$var(\hat\beta_j) = \frac{\sigma^2}{SST_j(1-R_j^2)}$
where,
$SST_j$= $\sum_{i=1}^n (...
3
votes
2answers
133 views
Why do type III sums of squares require orthogonal contrasts?
I have read many times that one has to set orthogonal contrast to get correct type III sums of square. E.g. John Fox says
To compute Type-III tests using incremental F-tests, one needs
contrasts ...
0
votes
1answer
95 views
Obtaining Information criterion or log likelihood from a model's squared error
I am concerned with the formulas I've seen for the AIC or BIC when using a squared-errors instead of likelihood.
On the AIC wikipedia page, there is the cryptic formula for LL:
$ -{\frac {n}{2}}\ln(...
0
votes
1answer
467 views
Is equation for total sum of squares always valid?
I wanted to do simple calculations for sum of squares for linear regression using a very simple example.
1) Let's assume that we have 3 obervations [x,y]: [[1,1], [2,2], [3,3]].
2) I created simple ...
1
vote
0answers
34 views
How to calculate the mean and standard deviation of weighed Noncentral chi distribution?
I would like to calculate the mean and standard deviation of :
Dist=(X^2+Y^2+Z^2)^0.5
Knowing that X, Y and Z are normal noncentral and nonstandard variables. they have a mean different from 0 and ...
4
votes
2answers
458 views
Motivation for Ward's definition of error sum of squares (ESS)
Ward (1963) provides a commonly used criterion for hierarchical clustering. It's based on the following definition (p. 237):
Given a set of ratings for 10 individuals, $\{2, 6, 5, 6, 2, 2, 2, 0, 0, ...
4
votes
0answers
72 views
Basics relative to chi square, likelihood, fits,
I'm confused to separate all the different meanings and connections.
The background of my question: On the one hand related to lmer models and on the other hand to the goodness of a fit. And their ...
0
votes
1answer
196 views
How is SSerror calculated in a factorial design?
I have found that
SSerror = SStotal - SStreatment
which makes sense to me alongside an example of a one-way ANOVA, however, how does this apply to a factorial ANOVA?
1
vote
0answers
637 views
Sum of Squares in Randomized Complete Block design
I'm interested in the Randomized Complete Block design (RCB) with
three treatments, which I call $A$, $B$, and $C$,
and four blocks ($b=4$):
\begin{align}
\textrm{block 1: } (y_{A,1}; y_{C,1}; y_{B,...
4
votes
1answer
127 views
What is the physical intuition behind the equality $\sum_i (x_i - \bar x)^2 = \sum_i (x_i - \bar x) x_i$?
Suppose that $x_1,\ldots,x_n$ are real numbers, and let $\bar x$ denote the average $\frac{\sum_i x_i}n.$ I know how to prove on paper that the equality
$$\sum_i (x_i - \bar x)^2 = \sum_i (x_i - \bar ...
