New answers tagged

0 votes

Why does the normality assumption not affect Linear Regression in large samples?

And is that for both prediction and inference (i.e getting BLUE estimators)? No, not for both The estimator for the coefficients will approach a normal distribution, independent of the distribution ...
Sextus Empiricus's user avatar
9 votes

Why does the normality assumption not affect Linear Regression in large samples?

The assumption of the Normality of the error term in a regression that applies Least-Squares estimation methods, is used to make statistical inferences about the coefficients after estimation, it is ...
Alecos Papadopoulos's user avatar
0 votes

Deviance as a measure of fit

What you are missing here is the hypothesis testing. We have: $$H_0:\beta_{p+1} = \beta_{p+2} = ... = \beta_{q} = 0$$ $$\text{vs}$$ $$H_1:\beta_i \neq 0 \text{ for some } i \in \{p+1,...,q\}$$ where $...
sunnydk's user avatar
  • 57
2 votes

Ordinal regression, multinomial regression or linear regression with dummy variables?

This is really two questions. One about the choice of regression method, the other about treatment of categorical independent variables. The choice among ordinal, multinomial, or linear (OLS) ...
Peter Flom's user avatar
  • 117k
0 votes
Accepted

Binary logistic regression: p-value of predictor containing all cases of response=1

A couple notes first regarding your questions: Is logistic regression inappropriate for such data? In theory, yes. Models with this predictor explain the most variance, but reporting results with ...
Shawn Hemelstrand's user avatar
-1 votes

How do I choose a spatial unit for fixed effects model

You are not able at present to detect a parish effect, but you detect a country effect. This might be because the parish effect is not independent form the country. If you are not directly interested ...
CaroZ's user avatar
  • 594
3 votes
Accepted

Geometric understanding of linear regression

Rather than $Y=\beta X+\varepsilon,$ you need $Y= X\beta+\varepsilon.$ The matrix $X$ has $n$ rows and $p$ columns and $\beta$ has $p$ rows and just one column, so $X$ needs to be on the left and $\...
Michael Hardy's user avatar
1 vote

Linear Regression with Only Categorical Features: Evaluating the Model

Since the response is the Net Operating Income (NOI), if a strictly positive continuous variable, we should consider gamma regression that allows a specific pattern of variance increasing with the ...
DrJerryTAO's user avatar
  • 1,325
3 votes

Linear Regression with Only Categorical Features: Evaluating the Model

I'm not 100% certain what you're asking here. It seems like you're lamenting that the $R^2$ is not impressive even though the predictions sound reasonable. I'll just give my two cents and can edit ...
Demetri Pananos's user avatar
4 votes

Remove non-significant independent variabels and re-run multiple regression

No. You should report the results you've got. Three of your variables are important for your theory. The other two have been shown to be large in other studies. So, you should keep them all. Don't ...
Peter Flom's user avatar
  • 117k
2 votes

Multiple regression model and correlation between predictors

Adding to Shawn's excellent answer (+1): First: Models don't "know" things. Second: The question in the book is clearly about real data and Shawn did a great job analyzing that data. However,...
Peter Flom's user avatar
  • 117k
3 votes
Accepted

Multiple regression model and correlation between predictors

Setting their causal assumptions to the side (which aren't well explained in this part of the book), the correlation between radio spending and sales is more than double that of newspaper spending ...
Shawn Hemelstrand's user avatar
2 votes

ANOVA comparison different subsets of same data frame

It's because they are two different subsets. When you use anova for model comparison, the models have to be on the same data with different models, usually nested models. If you want to see if there ...
Peter Flom's user avatar
  • 117k
1 vote

About regression analysis with categorical variables

This really depends on what your research question is. If you're simply interested in the effect of the continuous variable, you can just run a regression and look at the Wald test for the coefficient....
Demetri Pananos's user avatar
2 votes

About regression analysis with categorical variables

You can attempt to build a multiple regression model. A standard approach to perform regression with categorical variables is called one hot encoding. You encode each categorical variable with $k$ ...
Marko Lalovic's user avatar
2 votes

About regression analysis with categorical variables

Multiple linear regression analysis could be an option. For polytomous nominal predictor variables, you would have to use binary code variables in the regression model (e.g., using dummy coding [0, 1] ...
Christian Geiser's user avatar
7 votes

Missing Coefficients in Linear Regression with Multiple Categorical Variables in R

The intercept is the predicted level of the dependent variable when all the independent variables are 0 (however that is coded in your data). There are a number of ways to parametrize categorical ...
Peter Flom's user avatar
  • 117k
9 votes
Accepted

Missing Coefficients in Linear Regression with Multiple Categorical Variables in R

You need to define a reference level for each separate categorical variable, which will be absorbed into the intercept. (Specifically, R does this automatically, by using the alphabetically first ...
Stephan Kolassa's user avatar
1 vote
Accepted

How to prove the square of the t-statistic is the F-statistic in a linear regression without Lagrange multipliers?

More appealing (than Lagrangian multipliers), intuitive from a geometrical point of view is to resort to perpendicular projection operators. It is apt to work in a more general setting: let us ...
User1865345's user avatar
  • 7,922
2 votes

multiple regression using rank?

Ordinal regression is based on the rank-ordering of the outcome observations, without requiring any assumptions about things like error distributions. The idea is that the linear predictor is related ...
EdM's user avatar
  • 90.1k
0 votes

Analyzing compositional data (sum of proportions = 1) using mixed models with explanatory variables for each proportion

See my answer on compositional response to https://stats.stackexchange.com/a/638757/284766 that includes a discussion on the book "Analyzing Compositional Data with R." Section 5.3 "...
DrJerryTAO's user avatar
  • 1,325
1 vote
Accepted

Predictive capacities of Generalized Linear Models and significance

Statistical significance and predictive power are not totally related. Take for example this statistically significant but nonetheless extremely poor fit. It yields pretty terrible predictions given ...
Shawn Hemelstrand's user avatar
1 vote

Mixed effect models where one fixed effect leads to very different outcomes

Is there a way to reveal this finding from one overall model, To subset the data, you have probably created a binary indicator for weaker and stronger performers. You could use this variable as a ...
Marjolein Fokkema's user avatar
8 votes
Accepted

Multicollinearity and control variables dilemma

There is so much wrong with what you quoted! First, collinearity is not the same as correlation and can involve more than two variables. Second, there is nothing wrong with any of the equations, and ...
Peter Flom's user avatar
  • 117k
2 votes

Analyzing Intervention Effects in a Natural Experiment with Uneven Measurement Points

Multilevel models do not require either that everyone get the same number of measurements or that they are equally spaced. Which MLM you should choose depends on other things such as the nature of the ...
Peter Flom's user avatar
  • 117k
4 votes

Handling non-significant beta coefficients

This often happens, moreso with observational data. If variables were included to test specific hypotheses, then you have your answer. If variables were included because past research showed they were ...
David Smith's user avatar
  • 1,178
8 votes

Handling non-significant beta coefficients

Any tips on what we can do with the data to change this? Why change this? A null result is still a result, and if you purposefully seek to find data to support your hypothesis, that isn't really how ...
Demetri Pananos's user avatar
0 votes
Accepted

What is the difference between linear regression on two time periods with a time dummy variable and two separate regressions for each time?

The usual OLS standard error for a coefficient depends on the estimate of the standard deviation of the error, $\hat\sigma$, which will vary between the two formulations. That is, the estimate of $\...
Noah's user avatar
  • 32.4k
0 votes

When should one use multiple regression with dummy coding vs. ANCOVA?

As long as you can write the null hypothesis clearly, the problem will be solved head-on... For a regression analysis with a dummy variable (categorical variable consisting of several groups), your ...
ANuo's user avatar
  • 1
8 votes

Regression model not significant, but the predictor significant

Your question is very similar to this question, except that you had a prior hypothesis about the association between $x$ and $y$. As this answer to that question points out, having too many predictors ...
EdM's user avatar
  • 90.1k
2 votes

Calculate SE of regression coefficients using p-value (for meta-analysis)

It is possible with that information. You need to backcalculate the value of $t$ corresponding to that $p$ with the relevant degrees of freedom. If you use R then qt(p/2, df) should do it. Then since ...
mdewey's user avatar
  • 17.7k
2 votes
Accepted

How to interpret the output of 'Linear mixed model fit by REML' in R?

It's not clear what vnr is here, but I will answer in a general sense what your output is saying. First I start with some less important bits. Formula: This is ...
Shawn Hemelstrand's user avatar
3 votes

How to proceed with my multiple logistic regression?

Regarding your first point: However, the problem is that when I add more than one variable (with any combination), the results become non-significant. I performed backward regression, and the two ...
Shawn Hemelstrand's user avatar
2 votes

How to proceed with my multiple logistic regression?

With a total of 30 cases (19 events and 11 non-events), your study is quite small. This can lead to a lack of statistical power, making it difficult to detect significant effects even if they exist. ...
Linus's user avatar
  • 53

Top 50 recent answers are included