New answers tagged

0

Adding more variables will increase R Squared whether or not the added variables have any statistically significant effect on the dependent variable. On the other hand, adjusted R Squared can increase or decrease.


0

A very important effect of adding more variables is that you can better adjust for confounding, which means better causal interpretation. Which sometimes might be more important than looking at the $R^2$. For example, imagine we want a regression of the form $Y = X\beta + \epsilon$ where $X$ and $Y$ is thought of as a random variable. Suppose now that ...


1

If you introduce more variables, the $R^2$ will always increase, it can never decrease. This follows mathematically from the observation that $$ (y-\beta_0-\beta_1 x_1-...-\beta_p x_p-\beta_{p+1} x_{p+1})^2 \leq(y-\beta_0-\beta_1 x_1-...-\beta_p x_p)^2$$ On the other hand, the adjusted $R^2$ makes an adjustement for the number of variables. It will ...


2

Let us first recall the test statistics: Wald test in the linear regression model For $\mathcal{W}$ we need an estimator of the southeast block of the variance-covariance matrix of the coefficients, \begin{eqnarray} \widehat{V}_{\mathcal{W}}&=&\left[\mathcal{I}_{22}(\widehat{\theta})-\mathcal{I}_{21}(\widehat{\theta})\mathcal{I}_{11}(\widehat{\...


0

For a linear regression you assume that the data come from a normal distribution, whose mean is a function of the predictor: $$y \sim N(\mu=\beta_0 + \beta_1 x, \sigma)$$ After fitting (using maximum likelihood), you get estimates for the $\beta$s (typically indicated by adding the hat ^: $\hat{\beta}_1$) and for $\hat{\sigma}$. The figure plots the normal ...


1

You seem to be confusing the sample with the population. There's a population of possible response-values at any given $x$ (hence the conditional distribution [$Y|x$]), but in your sample you might only observe a single $y$-value at that specific $x$. Or you might observe two values, or 200 values (in which case the ECDF of the sample y's at that x-value ...


1

$R^2=0.15$ might not sound so great, but it might be quite acceptable for what you're doing. If that value is too low to be interesting, then you have a practically insignificant result resulting in a statistically significant p-value due to a large sample size that detects subtle differences. This means that there probably is a difference (statistical ...


0

A short answer: Yes, the linearity between $X$ and $Y$ is an assumption. A somewhat longer answer: Statistical terminology might be confusing. "Linear" in "linear regression" means being linear in the coefficients, but "logistic" in "logistic regression" does not mean being logistic in the coefficients! Logistic regression is just a name, a convention, ...


0

You are right. In statistics linear models are such models which are linear with respect to coefficients $\beta_i$. However there is also "common meaning" of linear, denoting the linear relationship (with random error $\epsilon$) between the variables, for example $y=a\cdot x+b$. However many non linear relationships (in terms of $x$ and $y$) can be ...


0

Consider a simple linear regression with just one predictor but where the error gets bigger as $x$ increases. We could still have an expectation of zero, but the error and the predictor certainly aren’t independent. For example, $\epsilon_i\sim N(0,x_i^2)$ has an expectation of $0$ for every $i$ but does not have the error independent of the predictor.


2

This is documented. I suggest reading the vignette on "basics", where EMMs are described: EMMs defined The reference grid consists of combinations of predictors. The predictions for the reference grid are each linear combinations of the regression coefficients. You can find out what these are by doing something like this: rg <- ref_grid(model) rg@linfct ...


1

Using basis expansion one can easily extend simple linear regression into non-linear models. Here is an example of how basis expansion works (with Fourier and polynomial basis). Depending on the data, we can chose the right model to fit. In the link, we are trying to fit a periodic data, so it is better to use Fourier basis. Note that it is one ...


4

Well, depending on the nature of the response variable $Y$ there are many possible regression models using only one predictor $x$ (and an intercept, mostly.) Logistic regression when $Y$ is binary Other glm's (generalized linear models) If $Y$ is ordinal, ordinal regression, How to handle ordinal categorical variable as independent variable If $Y$ is ...


0

It doesn't really make sense to compare two models using the anova() function when one model is not a sub-model of the other. So the p-value there is not interpretable. I suppose you could use the AIC to compare them but it's a little funny to have a quadratic term in the model without the main effect. The main reason is because a location shift in the ...


5

A couple of points: As @Roland said, it is not a good idea to fit a model with a quadratic term of a variable without also including the linear term in the model. You cannot compare the quadratic and linear models using a $\chi^2$ test because they are not nested. Namely, this test requires that one model is a special case of the other. For example, the ...


Top 50 recent answers are included