Questions tagged [sums-of-squares]
sum of squares plays an important role in statistical models based on the normal distribution, like ANOVA.
199
questions
1
vote
0
answers
23
views
cannot reproduce the example for nested anova
I'm working on the example for nested ANOVA from the Handbook of Biological Statistics. Copy-pasteable data can be found at https://www.biostathandbook.com/nestedanova.html and below shows my ...
- 121
3
votes
0
answers
31
views
What exactly do Anova sums of squares represent in lmer models?
In a lme4::lmer model, the anova function returns an sequential analysis of variance table.
As explained on page 34 of http://...
- 251
2
votes
0
answers
27
views
Interpretation of default Type III Sums in Squares in R
Many posts (i.e. here and here) discuss how the Type III sums of squares produced by car::Anova() in R
are incorrect or nonsensical under R's default model parameterization, "contr.treatment"...
- 1,131
2
votes
1
answer
33
views
How is the matrix of corrected sum of squares and products defined?
In the article Johns, S. "On identifying the population of origin of each observation in a mixture of observations from two normal populations." Technometrics 12.3 (1970): 553-563, the ...
- 31
1
vote
1
answer
32
views
Summation of terms in survival function giving negative probability
The following expression represents a survival function for a group size (n), where survival is a percentage on a scale of 0 to 100.
$$
p(x) = 100 - \beta(x \cdot x) = 100 - \beta\sum_{i=1}^{n}x_i^2
$$...
- 410
0
votes
1
answer
27
views
Does Residual Sum of Squares always decreases with an additional independent variable? [duplicate]
^ What the question says. Trying to determine whether this statement is true or false.
user358380
0
votes
0
answers
8
views
Is there any formal proof that in R type-1 anova() run twice on 2 models with switched terms equals type-2 (looking at the last entry)?
I observed something interesting to me.
Let's assume that I have an unbalanced, additive model (no interactions) with 2 terms.
When I swap the order of the two terms and run type-1 ANOVA over them, ...
- 51
0
votes
1
answer
39
views
Synonyms of Residual Sum of Squares
As a novice, I'm finding it difficult to learn statistics, partly because there are often many different words for the same thing. When I'm reading about stats, I don't realize that the thing I'm ...
- 37
-3
votes
1
answer
1k
views
"Normalized mean squared error" says WHAT?
I know that mean squared error is a public and popular metric to evaluate the efficiency of the model and architecture. Also, it is the tool to evaluate the result in such if, the ...
- 105
0
votes
0
answers
56
views
Why type III sum of squares is not recommended to use when there is significant interaction?
I did notice that in the following example, I obtained a insignificant main effect for treatment when including interaction, although the treatment is clearly effective.
Any intuitive explanation?
...
- 525
-1
votes
2
answers
147
views
Why calculate Mean Squared Error for Regression?
Once we fit the model to the points why is it necessary to find the mean squared error? What happens if I don't calculate Mean Squared Error?
- 1
1
vote
0
answers
29
views
Which type of sum of squares corresponds to the partition of variance formula for two-way ANOVA?
Does anyone have a good reference for the formula for type I and type III sum of squares in two-way ANOVA?
I learned the following formula for partition of total sum of squares in two-way ANOVA. Is it ...
- 525
1
vote
1
answer
2k
views
The number of clusters in the K-means and the within-cluster SS
Given a collection of observatons $\{X_i\}_1^N$ and prespecify the number of clusters K. The K-means solves
$$
\underset{\{C_k\}_1^K}{\arg\min} \sum_{k=1}^K \sum_{i \in C_k}|| X_i - \mu_k||^2
$$
where ...
- 122
4
votes
1
answer
199
views
Conceptual/Math/brainteaser Question: Multiple Linear Regression
This is kind of a brainteaser and I'm struggling to solve it, any ideas on approaching this would be valued:
I thought about using substitution of the different x and z into the third regression ...
- 63
6
votes
1
answer
161
views
Confused: Why is lme4 changing techniques from Wald F tests to Wald Chisquare?
I've constructed some LMEMs that use dichotomous variables and their interactions as regressors, and I've become confused by the output. When I only assess a single interaction, using the code below (...
1
vote
0
answers
40
views
Fitting data to sums of squares of sinusoids
I am given some time series data $(t, y(t))$ sampled at regular intervals $t=0,s,2s,3s,\ldots,1$ for some step size $s$, obtained from a function
$$
y(t) = y(t, \vec A, \vec \delta) = \sum_{i=1}^N ...
- 11
0
votes
1
answer
70
views
Which type of sums of squares does lm-function in R use?
I ran a two-way ANCOVA in R:
ancova = lm(DV ~ IV1*IV2 + CV1 + CV2 + CV3, data = Data)
summary.aov(ancova)
Anybody know if this uses type III sums of squares?
I ...
- 1
2
votes
1
answer
35
views
How unbalanced is unbalanced for factorial ANOVA?
Colleagues and I conducted an experiment with participants randomly allocated to each condition using survey software. Unfortunately, the cell sizes ended up being more unbalanced than we anticipated: ...
- 723
10
votes
1
answer
1k
views
Is least squares means (lsmeans) statistical nonsense?
I recently came accross this quote from Brian Ripley, who seems to be well-regarded as a statistician.
"Some of us feel that type III sum of squares and so-called ls-means are statistical ...
- 3,164
3
votes
1
answer
547
views
Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables
Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly ...
- 3,490
1
vote
1
answer
37
views
Shortcut for computing RSS at different split point when building regression trees?
I'm coding regression trees from scratch in R. For a given ordered predictor variable, $X$, obviously I have to compute the RSS at each unique ordered value of $X_i$. When moving over to the next ...
- 11
0
votes
1
answer
321
views
Why contrast in ANOVA has one degree of freedom?
I thought I understood the degree of freedom after reading Wikipedia explanation, but came across the sum of squares for contrasts $\{c_i\}$ $$SS_C = \frac{(\sum_i {c_i \bar{y_i}})^2}{\sum_i c_i^2 /...
- 11
1
vote
0
answers
281
views
Contrasts in ANOVA - need intuitions to understand its sum of squares and its degree of freedom
I have two questions about contrasts in ANOVA for testing the hypothesis that $\sum_i c_i y_i = 0$.
How is the sum of squares for contrasts $\{c_i\}$, $SS_C = \frac{(\sum_i {c_i \bar{y_i}})^2}{\sum_i ...
- 11
1
vote
0
answers
22
views
Distribution of the squared length of a multivariate normal vector [duplicate]
Let $\mathbf{X} \sim \mathcal{N}(\mathbf{\mu}, \mathbf{\Sigma})$ be a multivariate Normal variable with $d$ dimensions. I'm interested in the marginal distribution of $\|\mathbf{X}\|^2 = \sum_{i=1}^d ...
- 111
1
vote
1
answer
313
views
In a PCA setting, is there any relationship with the sum of squares of the scores (t1) with the eigenvalue of that principal component?
When one computes the vector of scores (t1) using the principal component (p1) the data is being projected over the direction of biggest variation. One could measure the distance between the point ...
- 65
3
votes
1
answer
212
views
What is the lowest sum of squared errors (sse) of a dataset?
Given $X \subset R^d$ a set of data points, $z \in R^d$ a vector, $SSE(X)=\sum_{i=1}^{|X|}||X_i-z||^2$, I wonder what is the best value of $z$ so that $SSE$ is the lowest?
I suspect that $z$ is the ...
- 153
1
vote
0
answers
41
views
How are terms in SS (corrected sum of squares) independent?
What does the highlighted test mean? How are the terms not independent? Can someone elaborate in detail?
- 21
0
votes
1
answer
239
views
Source for BIC with residual sum of squares?
Wikipedia states that under certain circumstances, $\mathrm{BIC} $ can be calculated as:
$$\mathrm{BIC} = n \ln(\mathrm{RSS}/n) + k \ln(n),$$
where $\mathrm{RSS}$ is the residual sum of squares. ...
- 253
1
vote
1
answer
213
views
Kruskal–Wallis one-way analysis of variance is related to what kind of regression?
One-way anova is similar to regular linear regression because both use the F-test which involves sums of squares among other reasons.
Is Kruskal–Wallis one-way analysis of variance similar to some ...
user271077
0
votes
0
answers
73
views
Difficulty with averaging corrected sample variances of different degrees of freedom:
I have a number of measurement samples of which some have 2 measurements and some have 3. I wish to make the most accurate estimation of population variance I can, and understand that ignoring data ...
- 67
3
votes
3
answers
1k
views
Residual sum of squares in a regression
I understand that in a linear regression model, the residual sum of squares will either remain same or fall with the addition of a new variable.
What if the two models were
$$
I \colon y_i=\...
- 1,167
2
votes
2
answers
999
views
Simplifying the Matrix Form of the Solution to Ridge Regression
I'm trying to understand how to obtain the solution to an objective function by solving for the parameter vector $\theta$ in ridge regression. I found an example here from Naomi which takes an example ...
- 123
1
vote
0
answers
1k
views
What is the correct implementation of BIC with residual sum of squares?
BIC is most often calculated by maximizing the log likelihood function. However, it is also possible to calculate BIC with residual sums of squares. This is pretty easy to find online and not an issue ...
- 381
2
votes
0
answers
204
views
Formula for type III sum of squares of the intercept term in linear multiple regression
assume we have the regression model:
$$Y = b_0 + b_1 x_1 + \dots + b_k x_k + \varepsilon $$
I know the formulas for all type III sum of squares for the regression terms except the formula for SS of ...
- 101
6
votes
2
answers
168
views
Coefficient of determination relationship?
$R^2 = \frac{SSREG}{SSTOT}$ or $R^2 = 1-\frac{SSRES}{SSTOT}$
If $X$ is the predictor random variable for science SAT
and $Y$ is the predictor random variable science GPA
given by equation $$\hat Y =...
- 183
2
votes
1
answer
1k
views
Sum of Squared Error Chi-Square distribution degree of freedom in Multilinear Regression
In this link it says that $Y$ variables has zero covariance (because covariance matrix has only diagonal terms) which implies they are independent.
Actually in linear regression $Y$ takes its ...
- 85
0
votes
1
answer
695
views
What is the difference between the residual, lack of fit and pure error In F test for Regression Analysis?
What is the difference between the residual, lack of fit and pure error and how to calculate each of them in ANOVA F test for nonlinear regression?
- 1
2
votes
1
answer
200
views
Two way ANOVA, no difference between tests based on type I vs type II sums of squares
I'm learning about two way ANOVA possibly with interaction.
I'm following this tutorial http://www.sthda.com/english/wiki/two-way-anova-test-in-r
This is their code.
...
- 23
0
votes
1
answer
19k
views
Calculating SSE in R
I'm following the "Intro to Statistics" course in Data Camp and I'm having some trouble as it seems that the course is contradicting itself: https://s3.amazonaws.com/assets.datacamp.com/production/...
0
votes
0
answers
63
views
Why is regression line represented as $y = b0 + b1 * x$?
I am new to Data Science and ran into Regression Line formula which is
$ y = b0 + b1 *x $ (where x is dependent variable, y is predicted variable)
I understood the meaning of this formula as a ...
- 109
1
vote
1
answer
652
views
Better to Minimize Absolute Error or Sum of Squared Error?
I have an Excel model which predicts the number of customers for a given month. The prediction depends on a churn rate. I have the absolute error (actual vs predicted), along with squared error and ...
- 161
4
votes
1
answer
4k
views
Why error sum of squares has n-2 df (possibly not duplicate, please read on)? (Regression Question Series - Part 4)
In simple linear regression, the error sum of squares is given by
$$
\text{SSE} = \sum_{i=1}^n(y_i - \hat{y_i})^2 \\
\hat{\sigma}^2 = s^2 = \dfrac{\text{SSE}}{n-2}
$$
where $n-2$ is the degrees of ...
1
vote
1
answer
358
views
Minimize Logged Sum of Squares?
When numerically maximizing the likelihood function it is standard practice to do this indirectly by minimizing the negative log-likelihood. When numerically minimizing the residual sum of squares (...
- 583
3
votes
0
answers
152
views
R: anova(lm): What is the Sum Sq when we have two inputs
What is the formula to calculate the Sum sq column for the inputs?
Answers to some other questions state, that it should be
$RSS = \sum (\hat Y_i -\bar Y)^2$,
yet this is false, it only gives the ...
- 713
2
votes
2
answers
2k
views
Showing that $\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$ for the generalized linear model [closed]
Exercise :
Prove that for the generalized linear model, it is :
$$\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$$
Question : How would one proceed with proving that for the generalized ...
- 173
1
vote
1
answer
56
views
Squaring floats between -1 and 1 reduces sum of squares, so why do it? [duplicate]
I have been learning basic statistical testing as it relates to agriculture and have become familiar with the common practice of summing squared raw deviation values, whether in something simple like ...
- 113
2
votes
2
answers
357
views
Calculating F-statistic, why is SS used instead of just $r^2$?
The calculations and question are for a simple regression (one independent and one dependent variable).
SSres = SSy * (1 - $r^2$)
SSreg = SSy * $r^2$
F = (SSreg / DFreg)/(SSres / DFres)
So, I've ...
- 23
1
vote
1
answer
711
views
Understanding ANOVA as regression / type III SS in R
I'm confused about how type III SS are calculated for a "main effect". According to what I have read, Type III SS is calculated by evaluating the change in the SSE by removing only the variable in ...
- 11
0
votes
1
answer
1k
views
2-way ANOVA in Python statsmodels yields different Sum of Squares than SPSS
I am learning to use Python for my statistical analyses, and while figuring out how to perform a 2-way ANOVA with statsmodels I found that my Python code yielded slightly aberrant values. Comparing ...
- 55
2
votes
0
answers
222
views
In regression, when partitioning SS among predictors, what determines which predictors get the SS that can be attributed to more than one predictor?
In regression analysis, predictors sometimes correlate (and in my field, psychology, they always do; often because they are simply measurements of the same aspects of human psychology).
If predictors ...
- 477