40

Degrees of freedom are non-integer in a number of contexts. Indeed in a few circumstances you can establish that the degrees of freedom to fit the data for some particular models must be between some value $k$ and $k+1$. We usually think of degrees of freedom as the number of free parameters, but there are situations where the parameters are not completely ...


17

Let's start by considering ordinary cubic splines. They're cubic between every pair of knots and cubic outside the boundary knots. We start with 4df for the first cubic (left of the first boundary knot), and each knot adds one new parameter (because the continuity of cubic splines and derivatives and second derivatives adds three constraints, leaving one ...


16

F tables: The easiest way of all -- if you can -- is to use a statistics package or other program to give you the critical value. So for example, in R, we can do this: qf(.95,5,6744) [1] 2.215425 (but you can as easily calculate an exact p-value for your F). Usually F tables come with an "infinity" degrees of freedom at the end of the table, but a few ...


15

Your model is saturated. Any model will use at least 1 degree of freedom. You have 2 factors with 4 levels each. They both require 3 additional degrees of freedom. The interaction consumes another 9 degrees of freedom. Summing those 1 + 3 + 3 + 9 = 16, but you have only 16 data. Thus, there are no degrees of freedom left with which to determine the ...


13

We can prove this for more general case of $p$ variables by using the "hat matrix" and some of its useful properties. These results are usually much harder to state in non matrix terms because of the use of the spectral decomposition. Now in matrix version of least squares, the hat matrix is $H=X(X^TX)^{-1}X^T$ where $X$ has $n$ rows and $p+1$ columns (...


13

I suspect this is what Bishop means: If you think of a neural net as a function that maps inputs to an output, then when you first initialize a neural net with small random weights, the neural net looks a lot like a linear function. The sigmoid activation function is close to linear around zero (just do a Taylor expansion), and small incoming weights will ...


13

I think that lmerTest is getting it right and ezanova is getting it wrong in this case. the results from lmerTest agree with my intuition/understanding two different computations in lmerTest (Satterthwaite and Kenward-Roger) agree they also agree with nlme::lme when I run it, ezanova gives a warning, which I don't entirely understand, but which should not ...


12

The main problem on model selection in mixed models is to define the degrees of freedom (df) of a model, truly. To compute df of a mixed model, one has to define the number of estimated parameters including fixed and random effects. And this is not straightforward. This paper by Jiming Jiang and others (2008) entitled "Fence methods for mixed model selection"...


12

It's the number of predictor (x) variables; the additional -1 in the formula is for the intercept - it's an additional predictor. The Y doesn't count. So in your example $k=2$ and the error df is $N-3$


11

Here is a less technical answer, perhaps more accessible to people with modest mathematical preparation. The term degrees of freedom (df) is used in connection with various test statistics but its meaning varies from one statistical test to the next. Some tests do not have degrees of freedom associated with the test statistic (e.g., Fisher's Exact Test ...


11

For the first question, the default method in SAS to find the df is not very smart; it looks for terms in the random effect that syntactically include the fixed effect, and uses that. In this case, since trt is not found in ind, it's not doing the right thing. I've never tried BETWITHIN and don't know the details, but either the Satterthwaite option (...


11

The Welch-Satterthwaite d.f. can be shown to be a scaled weighted harmonic mean of the two degrees of freedom, with weights in proportion to the corresponding standard deviations. The original expression reads: $$\nu_{_W} = \frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{s_1^4}{n_1^2\nu_1}+\frac{s_2^4}{n_2^2\nu_2}}$$ Note that $r_i=s_i^2/...


10

Wikipedia asserts that degrees of freedom of a random vector can be interpreted as the dimensions of the vector subspace. I want to go step-by-step, very basically through this as a partial answer and elaboration on the Wikipedia entry. The example proposed is that of a random vector corresponding to the measurements of a continuous variable for different ...


10

I generally agree with Ben's analysis but let me add a couple of remarks and a little intuition. First, the overall results: lmerTest results using the Satterthwaite method are correct The Kenward-Roger method is also correct and agrees with Satterthwaite Ben outlines the design in which subnum is nested in group while direction and group:direction are ...


10

What you are referring to is the Welch-Satterthwaite correction to the degrees of freedom. The $t$-test when the WS correction is applied is often called Welch's $t$-test. (Incidentally, this has nothing to do with SPSS, all statistical software will be able to conduct Welch's $t$-test, they just don't usually report both side by side by default, so you ...


10

There is a sentence prior to the passage quoted by the OP that I believe helps to interpret this: In statistics, the number of degrees of freedom (d.o.f.) is the number of independent pieces of data being used to make a calculation. (...). The number of degrees of freedom is a measure of how certain we are that our sample population is ...


10

I have not studied actual practice, so this reply cannot address that aspect of the question. As a general principle I would expect the treatment of significant digits in reporting the degrees of freedom (df) to be based on judgment related to significant figures. The principle is to be consistent: use the precision in one quantity that is appropriate for ...


9

In my classes, I use one "simple" situation that might help you wonder and perhaps develop a gut feeling for what a degree of freedom may mean. It is kind of a "Forrest Gump" approach to the subject, but it is worth the try. Consider you have 10 independent observations $X_1, X_2, \ldots, X_{10}\sim N(\mu,\sigma^2)$ that came right from a normal population ...


9

The correct terminology for the degrees of freedom that you need to compute is model degrees of freedom. You could also compute residual degrees of freedom. The model degrees of freedom are indeed calculated by adding up the degrees of freedom used by the parametric and non-parametric (or smooth) terms in your model. Here is an example of gam model ...


8

In regression there are two kinds of degrees of freedom. As described well below, the denominator or error or residual d.f. is relevant to regression models that have residuals and residual variance. As shown above, this is strongly a function of the sample size, and the more the merrier. On the other hand, numerator d.f. (terminology comes from what's in ...


8

Assume we are given a set of $n$ $p$-dimensional observations, $x_i \in \mathbb{R}^p$, $i = 1, \dotsc, n$. Assume a model of the form: \begin{align} Y_i = \langle \beta, x_i\rangle + \epsilon \end{align} where $\epsilon \sim N(0, \sigma^2)$, $\beta \in \mathbb{R}^p$, and $\langle \cdot, \cdot \rangle$ denoting the inner product. Let $\hat{\beta} = \delta(\{...


8

The case $n = k + 1$ corresponds to a saturated model, $$ \# \textrm{parameters} = \# \textrm{observations} $$ which is why you are seeing effectively an "infinite" penalisation. One of the contexts in which Akaike's Information Criterion along with a host of others were developed, and is used frequently today, is linear regression. It's not always clear ...


8

This is by design, you can refer to ?chisq.test: the degrees of freedom of the approximate chi-squared distribution of the test statistic, NA if the p-value is computed by Monte Carlo simulation. If you ask why is is to, then the answer is pretty simple. If you use standard $\chi^2$ test, then you compare your test statistic to theoretical test ...


7

You can get the population standard deviation by computing the variance via integration and then taking the square root. (It's not the only possible way to compute a variance but it's fairly routine integration for this problem.) Assuming you have a t$_{(\mu,\sigma^2,\nu)}$ distribution, you can replace $\mu$ by 0 without changing the variance. You can ...


7

It is conventional to round down to the nearest integer before consulting standard t tables The reason that was a convention is because tables don't have noninteger df. There's no reason to do it otherwise. which makes sense as this adjustment is conservative. Well, the statistic doesn't actually have a t-distribution, because he squared denominator ...


7

There are several approaches to determining the number of factors to extract for exploratory factor analysis (EFA). However, practically all of them boil down to be either visual, or analytical. Visual approaches are mostly based on visual representation of factors' eigenvalues (so called scree plot - see this page and this page), depending on extracted ...


7

You get the next 7 days off work, but you use the first day planning the rest of the days, so you have 6 days free. "the number of observations minus the number of necessary relations among these observations." -Walker Degrees of freedom can be very complex though and contrary to popular belief, see this answer for more.


7

Yes! Important technical note: For the rules stated below, define the "number of levels" for each factor to be the number of levels of that factor per each level of any/all factors that one is nested in. For example, if factor A is nested in factor B, then the "number of levels" for B is just the total number of levels for B, but the "number of levels" for ...


7

The most common convention in statistics is to use Greek letters for parameters ($\mu, \sigma$ for normal distributions, $\lambda$ for Poisson, $\beta$ when parameterizing the mean in regression and GLMS, etc). I'll assert this without any attempt to offer evidence. You can define your notation is almost any convenient way as long as it's clear, but $\nu$, ...


7

So far I always thought that degrees of freedom (dof) equal the number of observations minus the parameters to be estimated and are thus well defined. You're confusing two very different things. a) Let's start with the family of (univariate) t-distributions. There's a degrees of freedom parameter, $\nu$, that is just that - a parameter - freely definable ...


Only top voted, non community-wiki answers of a minimum length are eligible