Questions tagged [sums-of-squares]

sum of squares plays an important role in statistical models based on the normal distribution, like ANOVA.

Filter by
Sorted by
Tagged with
1 vote
1 answer
38 views

How does type II vs type III ANOVA work when there are more than two categorical predictors?

For a model with two categorical factors in the form: X = A + B Type I (used if the factors are balanced) calculates SS for each term in the model sequentially as follows: SS(A) then SS(B|A) = SS(A ...
Insect_biologist's user avatar
1 vote
0 answers
14 views

How to derive the Sum of Square for Latin Square with repetition

I am studying the Latin Square Design with n repetitions from https://online.stat.psu.edu/stat503/lesson/4/4.4 Let say we talk about the repetitions case 3 i.e. we ...
Brian Smith's user avatar
1 vote
0 answers
98 views

Partial eta square values summing to greater than 1

I am trying to run a factorial ANOVA analysis with partial effects, but it seems that my partial eta square values are totalling above 1, particularly when I specify my contrast with Type II or Type ...
BigTimeDataTime's user avatar
2 votes
1 answer
36 views

How to resolve confusing ANOVA results using type II sums of squares?

Suppose we have the following data from the R package afex: ...
bluepole's user avatar
  • 2,699
1 vote
1 answer
276 views

Do I calculate the loss of a regression on the normalized or denormalized data?

I worked on implementing a simple MLP network, which should guess a numerical value based on different values. Basically it's a regression task. This was a few months ago. Everything worked fine, the ...
SickerDude43's user avatar
3 votes
1 answer
79 views

What is "explained" by the explained/regression sum of squares?

We are in a regression setting. Let's start by defining some notation and terminology. $y_i$ is observation $i$ of some (response) variable $Y$. $\hat{y}_i$ is the value of $y_i$ predicted by a ...
Dave's user avatar
  • 62.5k
1 vote
2 answers
61 views

How can you convert sum of squares deviation to an r-squared value?

I would like to calculate the r-squared value for some regressions. The model (its in a GUI) I am using gives me "goodness of fit" in terms of the sum of squares deviation. I am using ...
vermicellion's user avatar
1 vote
1 answer
42 views

Computing sum squared distances without computing center

Given an even number of sample points in a plane, I want to compute the sum of squared distances from the sample center as part of estimating the Rayleigh parameter. One way of doing it is to compute ...
feetwet's user avatar
  • 1,142
3 votes
1 answer
117 views

Cannot understand why total SS=explained SS+unexplained SS

I cannot understand why total SS=explained SS+unexplained SS because geometrically the sum of two small squares is not equal to a big square. I wish someone could explain that to me. Thank you.
J.Liu's user avatar
  • 31
2 votes
1 answer
104 views

Different results for type I and type II ANOVA

I am performing an ANOVA for a research question. I want to to test whether "Grade", "Nim" and "Tim" has a significant influence on "value". My question is ...
FlKbr's user avatar
  • 23
1 vote
0 answers
72 views

Is there an interpretation for this identity from sum of squares?

So yesterday my professor wrote the following on the board: $$\frac{1}{n}\sum_{i = 1}^{n}\left( x_i - \bar{X}\right)^2 = \frac{1}{n}\sum_{i = 1}^{n} x_i^2 - \bar{X}^2$$ where $\bar{X} = \frac{1}{n}\...
FafaDog's user avatar
  • 140
1 vote
1 answer
211 views

Expected Value for Sum of Squares in Reduced Model

$\newcommand{\ol}[1]{\overline{#1}}\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$I am studying Design and Analysis of Experiments, 2nd Ed., by Dean, Voss, and ...
Adrian Keister's user avatar
1 vote
0 answers
36 views

cannot reproduce the example for nested anova

I'm working on the example for nested ANOVA from the Handbook of Biological Statistics. Copy-pasteable data can be found at https://www.biostathandbook.com/nestedanova.html and below shows my ...
foehn's user avatar
  • 131
6 votes
1 answer
775 views

What exactly do Anova sums of squares represent in lmer models?

In a lme4::lmer model, the anova function returns an sequential analysis of variance table. As explained on page 34 of http://...
mjandrews's user avatar
  • 281
3 votes
0 answers
246 views

Interpretation of default Type III Sums in Squares in R

Many posts (i.e. here and here) discuss how the Type III sums of squares produced by car::Anova() in R are incorrect or nonsensical under R's default model parameterization, "contr.treatment"...
Darren James's user avatar
  • 1,231
2 votes
1 answer
340 views

How is the matrix of corrected sum of squares and products defined?

In the article Johns, S. "On identifying the population of origin of each observation in a mixture of observations from two normal populations." Technometrics 12.3 (1970): 553-563, the ...
qwerty's user avatar
  • 31
1 vote
1 answer
84 views

Summation of terms in survival function giving negative probability

The following expression represents a survival function for a group size (n), where survival is a percentage on a scale of 0 to 100. $$ p(x) = 100 - \beta(x \cdot x) = 100 - \beta\sum_{i=1}^{n}x_i^2 $$...
Cybernetic's user avatar
0 votes
1 answer
282 views

Does Residual Sum of Squares always decreases with an additional independent variable? [duplicate]

^ What the question says. Trying to determine whether this statement is true or false.
user avatar
0 votes
1 answer
99 views

Synonyms of Residual Sum of Squares

As a novice, I'm finding it difficult to learn statistics, partly because there are often many different words for the same thing. When I'm reading about stats, I don't realize that the thing I'm ...
User1974's user avatar
-3 votes
1 answer
4k views

"Normalized mean squared error" says WHAT?

I know that mean squared error is a public and popular metric to evaluate the efficiency of the model and architecture. Also, it is the tool to evaluate the result in such if, the ...
john22's user avatar
  • 127
0 votes
0 answers
190 views

Why type III sum of squares is not recommended to use when there is significant interaction?

I did notice that in the following example, I obtained a insignificant main effect for treatment when including interaction, although the treatment is clearly effective. Any intuitive explanation? ...
hehe's user avatar
  • 733
-1 votes
2 answers
275 views

Why calculate Mean Squared Error for Regression?

Once we fit the model to the points why is it necessary to find the mean squared error? What happens if I don't calculate Mean Squared Error?
Mr.D's user avatar
  • 1
1 vote
0 answers
39 views

Which type of sum of squares corresponds to the partition of variance formula for two-way ANOVA?

Does anyone have a good reference for the formula for type I and type III sum of squares in two-way ANOVA? I learned the following formula for partition of total sum of squares in two-way ANOVA. Is it ...
hehe's user avatar
  • 733
1 vote
1 answer
5k views

The number of clusters in the K-means and the within-cluster SS

Given a collection of observatons $\{X_i\}_1^N$ and prespecify the number of clusters K. The K-means solves $$ \underset{\{C_k\}_1^K}{\arg\min} \sum_{k=1}^K \sum_{i \in C_k}|| X_i - \mu_k||^2 $$ where ...
HJ Liang's user avatar
  • 132
4 votes
1 answer
502 views

Conceptual/Math/brainteaser Question: Multiple Linear Regression

This is kind of a brainteaser and I'm struggling to solve it, any ideas on approaching this would be valued: I thought about using substitution of the different x and z into the third regression ...
delucaenzo's user avatar
9 votes
1 answer
586 views

Confused: Why is lme4 changing techniques from Wald F tests to Wald Chisquare?

I've constructed some LMEMs that use dichotomous variables and their interactions as regressors, and I've become confused by the output. When I only assess a single interaction, using the code below (...
DegreesOfWhat's user avatar
1 vote
0 answers
41 views

Fitting data to sums of squares of sinusoids

I am given some time series data $(t, y(t))$ sampled at regular intervals $t=0,s,2s,3s,\ldots,1$ for some step size $s$, obtained from a function $$ y(t) = y(t, \vec A, \vec \delta) = \sum_{i=1}^N ...
J Bausch's user avatar
1 vote
1 answer
360 views

Which type of sums of squares does lm-function in R use?

I ran a two-way ANCOVA in R: ancova = lm(DV ~ IV1*IV2 + CV1 + CV2 + CV3, data = Data) summary.aov(ancova) Anybody know if this uses type III sums of squares? I ...
ge36fiy's user avatar
  • 11
2 votes
1 answer
91 views

How unbalanced is unbalanced for factorial ANOVA?

Colleagues and I conducted an experiment with participants randomly allocated to each condition using survey software. Unfortunately, the cell sizes ended up being more unbalanced than we anticipated: ...
Gabriel's user avatar
  • 743
12 votes
1 answer
5k views

Is least squares means (lsmeans) statistical nonsense?

I recently came accross this quote from Brian Ripley, who seems to be well-regarded as a statistician. "Some of us feel that type III sum of squares and so-called ls-means are statistical ...
Joe King's user avatar
  • 3,815
5 votes
1 answer
1k views

Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables

Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly ...
ColorStatistics's user avatar
1 vote
1 answer
93 views

Shortcut for computing RSS at different split point when building regression trees?

I'm coding regression trees from scratch in R. For a given ordered predictor variable, $X$, obviously I have to compute the RSS at each unique ordered value of $X_i$. When moving over to the next ...
Matt B's user avatar
  • 11
3 votes
1 answer
1k views

Why contrast in ANOVA has one degree of freedom?

I thought I understood the degree of freedom after reading Wikipedia explanation, but came across the sum of squares for contrasts $\{c_i\}$ $$SS_C = \frac{(\sum_i {c_i \bar{y_i}})^2}{\sum_i c_i^2 /...
Tom's user avatar
  • 41
1 vote
0 answers
714 views

Contrasts in ANOVA - need intuitions to understand its sum of squares and its degree of freedom

I have two questions about contrasts in ANOVA for testing the hypothesis that $\sum_i c_i y_i = 0$. How is the sum of squares for contrasts $\{c_i\}$, $SS_C = \frac{(\sum_i {c_i \bar{y_i}})^2}{\sum_i ...
Tom's user avatar
  • 41
1 vote
0 answers
27 views

Distribution of the squared length of a multivariate normal vector [duplicate]

Let $\mathbf{X} \sim \mathcal{N}(\mathbf{\mu}, \mathbf{\Sigma})$ be a multivariate Normal variable with $d$ dimensions. I'm interested in the marginal distribution of $\|\mathbf{X}\|^2 = \sum_{i=1}^d ...
macjan's user avatar
  • 111
1 vote
1 answer
543 views

In a PCA setting, is there any relationship with the sum of squares of the scores (t1) with the eigenvalue of that principal component?

When one computes the vector of scores (t1) using the principal component (p1) the data is being projected over the direction of biggest variation. One could measure the distance between the point ...
Ralphns's user avatar
  • 65
4 votes
1 answer
569 views

What is the lowest sum of squared errors (sse) of a dataset?

Given $X \subset R^d$ a set of data points, $z \in R^d$ a vector, $SSE(X)=\sum_{i=1}^{|X|}||X_i-z||^2$, I wonder what is the best value of $z$ so that $SSE$ is the lowest? I suspect that $z$ is the ...
curiosus's user avatar
  • 303
1 vote
0 answers
86 views

How are terms in SS (corrected sum of squares) independent?

What does the highlighted test mean? How are the terms not independent? Can someone elaborate in detail?
stockulum's user avatar
0 votes
1 answer
519 views

Source for BIC with residual sum of squares?

Wikipedia states that under certain circumstances, $\mathrm{BIC} $ can be calculated as: $$\mathrm{BIC} = n \ln(\mathrm{RSS}/n) + k \ln(n),$$ where $\mathrm{RSS}$ is the residual sum of squares. ...
Nos's user avatar
  • 325
1 vote
1 answer
553 views

Kruskal–Wallis one-way analysis of variance is related to what kind of regression?

One-way anova is similar to regular linear regression because both use the F-test which involves sums of squares among other reasons. Is Kruskal–Wallis one-way analysis of variance similar to some ...
user avatar
0 votes
0 answers
79 views

Difficulty with averaging corrected sample variances of different degrees of freedom:

I have a number of measurement samples of which some have 2 measurements and some have 3. I wish to make the most accurate estimation of population variance I can, and understand that ignoring data ...
MCC's user avatar
  • 67
3 votes
3 answers
2k views

Residual sum of squares in a regression

I understand that in a linear regression model, the residual sum of squares will either remain same or fall with the addition of a new variable. What if the two models were $$ I \colon y_i=\...
Harry's user avatar
  • 1,387
3 votes
2 answers
3k views

Simplifying the Matrix Form of the Solution to Ridge Regression

I'm trying to understand how to obtain the solution to an objective function by solving for the parameter vector $\theta$ in ridge regression. I found an example here from Naomi which takes an example ...
theupandup's user avatar
1 vote
0 answers
2k views

What is the correct implementation of BIC with residual sum of squares?

BIC is most often calculated by maximizing the log likelihood function. However, it is also possible to calculate BIC with residual sums of squares. This is pretty easy to find online and not an issue ...
JElder's user avatar
  • 919
2 votes
0 answers
259 views

Formula for type III sum of squares of the intercept term in linear multiple regression

assume we have the regression model: $$Y = b_0 + b_1 x_1 + \dots + b_k x_k + \varepsilon $$ I know the formulas for all type III sum of squares for the regression terms except the formula for SS of ...
Kourabi's user avatar
  • 111
6 votes
2 answers
204 views

Coefficient of determination relationship?

$R^2 = \frac{SSREG}{SSTOT}$ or $R^2 = 1-\frac{SSRES}{SSTOT}$ If $X$ is the predictor random variable for science SAT and $Y$ is the predictor random variable science GPA given by equation $$\hat Y =...
Tree Garen's user avatar
2 votes
1 answer
2k views

Sum of Squared Error Chi-Square distribution degree of freedom in Multilinear Regression

In this link it says that $Y$ variables has zero covariance (because covariance matrix has only diagonal terms) which implies they are independent. Actually in linear regression $Y$ takes its ...
mertcan's user avatar
  • 105
0 votes
0 answers
953 views

What is the difference between the residual, lack of fit and pure error In F test for Regression Analysis?

What is the difference between the residual, lack of fit and pure error and how to calculate each of them in ANOVA F test for nonlinear regression?
MoBo's user avatar
  • 1
2 votes
1 answer
304 views

Two way ANOVA, no difference between tests based on type I vs type II sums of squares

I'm learning about two way ANOVA possibly with interaction. I'm following this tutorial http://www.sthda.com/english/wiki/two-way-anova-test-in-r This is their code. ...
user231906's user avatar
0 votes
1 answer
22k views

Calculating SSE in R

I'm following the "Intro to Statistics" course in Data Camp and I'm having some trouble as it seems that the course is contradicting itself: https://s3.amazonaws.com/assets.datacamp.com/production/...
René Martínez's user avatar

1
2 3 4 5