Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [sums-of-squares]

sum of squares plays an important role in statistical models based on the normal distribution, like ANOVA.

2
votes
1answer
33 views

Sum of Squared Error Chi-Square distribution degree of freedom in Multilinear Regression

In this link it says that $Y$ variables has zero covariance (because covariance matrix has only diagonal terms) which implies they are independent. Actually in linear regression $Y$ takes its ...
0
votes
0answers
7 views

Least square estimate for post-stratification sampling

I figured out via the normal linear regression method that Beta0 hat = ybar - Beta1 hat xbar. But I am not sure how to find out the least square estimate for Chat. Is anyone able to help me? Thanks!! ...
0
votes
0answers
12 views

Scheffe's S Method - help with finding the absolute value of l

I am trying to understand Scheffe's S Method and how the absolute value of l is found. In the example I am reading, the absolute value of l is |-.836| but I have absolutely no idea how this value was ...
0
votes
1answer
40 views

What is the difference between the residual, lack of fit and pure error In F test for Regression Analysis?

What is the difference between the residual, lack of fit and pure error and how to calculate each of them in ANOVA F test for nonlinear regression?
0
votes
0answers
11 views

High error value when estimating model parameters

I have a non linear system of ODEs and to estimate 4 of the model parameters I am using Matlab fmincon by minimising the sum of squared errors (SSE). I have only 5 ...
2
votes
1answer
30 views

Two way ANOVA, no difference between tests based on type I vs type II sums of squares

I'm learning about two way ANOVA possibly with interaction. I'm following this tutorial http://www.sthda.com/english/wiki/two-way-anova-test-in-r This is their code. ...
0
votes
1answer
2k views

Calculating SSE in R

I'm following the "Intro to Statistics" course in Data Camp and I'm having some trouble as it seems that the course is contradicting itself: https://s3.amazonaws.com/assets.datacamp.com/production/...
0
votes
0answers
43 views

Why is regression line represented as $y = b0 + b1 * x$?

I am new to Data Science and ran into Regression Line formula which is $ y = b0 + b1 *x $ (where x is dependent variable, y is predicted variable) I understood the meaning of this formula as a ...
0
votes
1answer
47 views

Better to Minimize Absolute Error or Sum of Squared Error?

I have an Excel model which predicts the number of customers for a given month. The prediction depends on a churn rate. I have the absolute error (actual vs predicted), along with squared error and ...
0
votes
0answers
201 views

Why error sum of squares has n-2 df (possibly not duplicate, please read on)? (Regression Question Series - Part 4)

In simple linear regression, the error sum of squares is given by $$ \text{SSE} = \sum_{i=1}^n(y_i - \hat{y_i})^2 \\ \hat{\sigma}^2 = s^2 = \dfrac{\text{SSE}}{n-2} $$ where $n-2$ is the degrees of ...
0
votes
1answer
40 views

Minimize Logged Sum of Squares?

When numerically maximizing the likelihood function it is standard practice to do this indirectly by minimizing the negative log-likelihood. When numerically minimizing the residual sum of squares (...
1
vote
0answers
30 views

R: anova(lm): What is the Sum Sq when we have two inputs

What is the formula to calculate the Sum sq column for the inputs? Answers to some other questions state, that it should be $RSS = \sum (\hat Y_i -\bar Y)^2$, yet this is false, it only gives the ...
2
votes
2answers
94 views

Showing that $\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$ for the generalized linear model [closed]

Exercise : Prove that for the generalized linear model, it is : $$\sum_{i=1}^n (y_i-\hat{y_i})(\hat{y_i} - \bar{y}) = 0$$ Question : How would one proceed with proving that for the generalized ...
1
vote
1answer
45 views

Squaring floats between -1 and 1 reduces sum of squares, so why do it? [duplicate]

I have been learning basic statistical testing as it relates to agriculture and have become familiar with the common practice of summing squared raw deviation values, whether in something simple like ...
2
votes
2answers
93 views

Calculating F-statistic, why is SS used instead of just $r^2$?

The calculations and question are for a simple regression (one independent and one dependent variable). SSres = SSy * (1 - $r^2$) SSreg = SSy * $r^2$ F = (SSreg / DFreg)/(SSres / DFres) So, I've ...
0
votes
1answer
78 views

Understanding ANOVA as regression / type III SS in R

I'm confused about how type III SS are calculated for a "main effect". According to what I have read, Type III SS is calculated by evaluating the change in the SSE by removing only the variable in ...
0
votes
0answers
63 views

Calculate variance lost after clustering

I'm studying clustering for system performance analysis and I'm interested in calculate the variance lost after clustering. Problem is this suppose I've 23 variables and 3000 observation. First I use ...
0
votes
1answer
351 views

2-way ANOVA in Python statsmodels yields different Sum of Squares than SPSS

I am learning to use Python for my statistical analyses, and while figuring out how to perform a 2-way ANOVA with statsmodels I found that my Python code yielded slightly aberrant values. Comparing ...
0
votes
0answers
14 views

Measure of error for non-linear systems?

I am trying to model a system that is ideally explained by the formula Q=Qi*(1+Di*b*t)^(-1/b), where: ...
0
votes
0answers
612 views

What's the difference between the SS in the variance and the TSS?

I'm trying to understand how these two statistics differ. My understanding is the variance is the sum of squares of the predictor divided by the degrees of freedom. On the other hand, the sum of ...
1
vote
0answers
58 views

In regression, when partitioning SS among predictors, what determines which predictors get the SS that can be attributed to more than one predictor?

In regression analysis, predictors sometimes correlate (and in my field, psychology, they always do; often because they are simply measurements of the same aspects of human psychology). If predictors ...
0
votes
0answers
23 views

Show that these two sums involving sample means are equivalent

Let $C_{ij}$ be observations with indices where $i=1, \cdots, M_j$, $j=1, \cdots, J$. That is, there are $J$ groups and each group $j$ has $M_j$ observations so that in total there are $N=\sum_{j=1}^{...
5
votes
2answers
368 views

How should Type II SS be calculated in a mixed model?

I have a data set (and corresponding mixed model) which gets very different p-values for one of the two-way interactions when tested using Type I (sequential, taking care that it's last), and Type II (...
0
votes
0answers
25 views

Deriving the F-test from ${{SSE_R-SSE_F}\over{(n-q)-(n-p)}}/{{SSE_F}\over{n-p}}$

Given a Full and Reduced model, the F-test to see if the reduced model is significant is given by $$ {{SSE_R-SSE_F}\over{(n-q)-(n-p)}}/{{SSE_F}\over{n-p}} $$ I'm trying to understand how this is ...
2
votes
0answers
52 views

Find the Sum of Squares and using them complete an F-test for goodness of fit

I have this question ( would be grateful if someone could put it on for me) https://gyazo.com/5afe5d7c9d12acdff8f9c55db74f97c9 And I am concerned with part b). I know what $RSS = SSE$ is due to the ...
2
votes
1answer
561 views

Which one to choose Type-I, Type-II, or Type-III ANOVA? [duplicate]

I don't understand what the difference is between TypeI and TypeIII? Since my background is not very mathematics, it's very difficult for me to understand this mathematical notation; Type I SS: SS(A) ...
0
votes
1answer
99 views

Does forward model selection by $R^2$ or SSR differ?

I'm trying to implement forward selection and need to add a feature only if it will make the sum of squared residuals (SSR) lower. I am using Python's statsmodels ...
0
votes
0answers
30 views

The Sums of squares in 3 way Anova in R output has changed as the order of the variables in the modes changes [duplicate]

I am trying to fit a 3 way ANOVA using R. Then i realized the sums of squares of the output has changed when the order of the variables in the model are changed.Can anyone figure out the reason for ...
5
votes
2answers
143 views

Mixed-effect model single term deletion — should I change my random effects?

In short I recently had a little conversation on the lme4 project's GitHub on how to properly test the significance of effects in a mixed-effect model, which made ...
2
votes
0answers
128 views

Test significance using Type II ANOVA following Type III ANOVA?

I hope you can help me with a theoretical question about how to proceed in my analysis. I have come across many posts discussing differences between Type II and III SS Anovas, however, because I don't ...
1
vote
0answers
47 views

Trying to understand hierarchical testing of nested regression models

I've learned multiple regression, but never did I learn hierarchical regression (i.e., the hierarchical testing of nested regression models) before. Based on the image attached, I assume because the ...
3
votes
1answer
241 views

In regression analysis, How can R^2 represent the total explained variance, if it can be computed from an equation with only unique contributions?

Background In regression analysis, $R^2$, the squared multiple correlation, represents the proportion of explained variance by the regression model. Most software's default setting uses Type-III sums ...
3
votes
1answer
1k views

Is Square Root of the Variance of a Regression Coefficient the Standard Error?

Quick question, in the textbook "Introductory Econometrics", the variance of a Regression Coefficient is given as: $var(\hat\beta_j) = \frac{\sigma^2}{SST_j(1-R_j^2)}$ where, $SST_j$= $\sum_{i=1}^n (...
2
votes
1answer
52 views

Why do type III sums of squares require orthogonal contrasts?

I have read many times that one has to set orthogonal contrast to get correct type III sums of square. E.g. John Fox says To compute Type-III tests using incremental F-tests, one needs contrasts ...
0
votes
1answer
82 views

Obtaining Information criterion or log likelihood from a model's squared error

I am concerned with the formulas I've seen for the AIC or BIC when using a squared-errors instead of likelihood. On the AIC wikipedia page, there is the cryptic formula for LL: $ -{\frac {n}{2}}\ln(...
0
votes
1answer
281 views

Is equation for total sum of squares always valid?

I wanted to do simple calculations for sum of squares for linear regression using a very simple example. 1) Let's assume that we have 3 obervations [x,y]: [[1,1], [2,2], [3,3]]. 2) I created simple ...
1
vote
0answers
28 views

How to calculate the mean and standard deviation of weighed Noncentral chi distribution?

I would like to calculate the mean and standard deviation of : Dist=(X^2+Y^2+Z^2)^0.5 Knowing that X, Y and Z are normal noncentral and nonstandard variables. they have a mean different from 0 and ...
4
votes
2answers
255 views

Motivation for Ward's definition of error sum of squares (ESS)

Ward (1963) provides a commonly used criterion for hierarchical clustering. It's based on the following definition (p. 237): Given a set of ratings for 10 individuals, $\{2, 6, 5, 6, 2, 2, 2, 0, 0, ...
4
votes
0answers
55 views

Basics relative to chi square, likelihood, fits,

I'm confused to separate all the different meanings and connections. The background of my question: On the one hand related to lmer models and on the other hand to the goodness of a fit. And their ...
0
votes
1answer
86 views

How is SSerror calculated in a factorial design?

I have found that SSerror = SStotal - SStreatment which makes sense to me alongside an example of a one-way ANOVA, however, how does this apply to a factorial ANOVA?
1
vote
0answers
306 views

Sum of Squares in Randomized Complete Block design

I'm interested in the Randomized Complete Block design (RCB) with three treatments, which I call $A$, $B$, and $C$, and four blocks ($b=4$): \begin{align} \textrm{block 1: } (y_{A,1}; y_{C,1}; y_{B,...
4
votes
1answer
119 views

What is the physical intuition behind the equality $\sum_i (x_i - \bar x)^2 = \sum_i (x_i - \bar x) x_i$?

Suppose that $x_1,\ldots,x_n$ are real numbers, and let $\bar x$ denote the average $\frac{\sum_i x_i}n.$ I know how to prove on paper that the equality $$\sum_i (x_i - \bar x)^2 = \sum_i (x_i - \bar ...
1
vote
0answers
91 views

ANOVA Type II vs Type II SSq for multiple factors

I have pondered type II vs type III sums of squares for some time and looked at posted examples of calculations for 2 main factors (e.g. this very nice page). That is, in the case of a model: X = A + ...
0
votes
1answer
36 views

What is the total sum of squares for these points? [closed]

y = (-2,2,3,4) and x = (3,5,8,12). Lin. Model is y = -2.5 + 0.6x Please explain how you got the answer. Thank you! I am confused on whether I should be calculating based on the x or y values, or ...
3
votes
0answers
2k views

Generalized eta squared

Partial eta-squared are very often used in psychological litterature. As underlined by some authors (e.g., Baguley, 2009; Bakeman, 2005; Olejnik & Algina, 2003), this standardized measure of ...
1
vote
0answers
246 views

Distribution of $(n-2)MSres/\sigma^2$ in simple linear regression

In simple linear regression, it was proved that $(n-2)MS_{RES}/\sigma^2$ follows a $\chi^2_{n-2}$ distribution.In order to prove this,the fact that $e_i$ or $y_i - \hat{y_i}$ follows a Normal ...
1
vote
0answers
20 views

Feature scaling getting high Error Value (sum of squared)

I got a test data set to work on in order to implement feature scaling and get better results with gradient descent. It looks like this: ...
0
votes
0answers
114 views

Derivation of SSE(reduced)-SSE(full) = $(C\hat \beta - h)'(C(X'X)^{-1}C)^{-1}(C\hat \beta - h) $

When stating hypothesis in matrix formulation, as H0: $C\hat\beta=h$, $SSE(reduced)-SSE(full)$ can be expressed as: $$(C\hat \beta - h)'(C(X'X)^{-1}C)^{-1}(C\hat \beta - h) $$ How is this result ...
3
votes
3answers
103 views

Residual sum of squares, two different equations, why are they equal?

One definition of the Residual Sum of Squares is: $$ S_r = (y-X\hat{\beta})^T(y-X\hat{\beta}) $$ And I think I understand it. Now I have seen a different definition: $$ S_r = y^Ty- \hat{\beta}^TX^...
2
votes
1answer
648 views

Splitting of treatment sum of squares in ANOVA

An apparently classic mantra appearing in about a dozen of ANOVA textbooxs i consulted reads roughly as: given any choice of $T-1$ orthogonal contrasts $C_1, \cdots, C_{T-1}$ in a (not necessarily ...