New answers tagged

1 vote

Hows does coefficient $b_1$ change when estimating $b_1 x_1+b_2 x_2+b_3 x_3$ instead of $b_1 x_1+b_2 x_2$

By the Frisch–Waugh(–Lovell) theorem and the well know formula $\left(X^{\top}X \right)^{-1}X^{\top}y$ for the OLS estimator we have $$ \begin{align} \hat{b}_{1; \text{model 1}}&=\left( \left(M_{2}...
user avatar
  • 1,395
0 votes

Hows does coefficient $b_1$ change when estimating $b_1 x_1+b_2 x_2+b_3 x_3$ instead of $b_1 x_1+b_2 x_2$

We can find both $\hat{b}_{1, \text{model } 1}$ and $\hat{b}_{1, \text{model } 2}$ are closed form (take the appropriate element of $(X^T X)^{-1}X y$) thus the difference between them is also ...
user avatar
  • 2,395
1 vote

Is it possible to use cross-validation to estimate the reliability of a specific predictor?

Generally, cross-validation is used to assess predictive value. From that perspective, you could fit an otherwise identical model with and without a variable and assess the change in the root mean ...
user avatar
0 votes

How can we visualize multiple regression with 3 or more continuous variables or with categorical variables

I have used matrix plots before, but I also used another way. I made a regression plot between the measured variable and the main input variable. Then I took the residuals, and plotted those as a ...
user avatar
  • 11
0 votes

How can we visualize multiple regression with 3 or more continuous variables or with categorical variables

You usually use so-called matrix plots, a collection of 2-dimensional scatter plots. Categories are usually represented by different colors of the data points.
user avatar
  • 1
4 votes

How can we visualize multiple regression with 3 or more continuous variables or with categorical variables

3 continuous variables: make a row of such plots at 3-5 defined levels (or 'slices') of the third variable. 4 continuous variables: make a column of such plots at 3-5 defined levels of the fourth ...
user avatar
  • 13.2k
0 votes

Why are machine learning algorithms performing worse than standard multiple linear regression?

Assumptions give you power - when they are valid. When the assumptions of a linear regression (or any other simple model) are fulfilled, they will outperform more complex and flexible models. Let's ...
user avatar
  • 13.2k
1 vote
Accepted

Statistical test to a large dataset

A regression (a.k.a. an ANOVA in this case) would be a simple and appropriate way to analyse this data. You would need to reorganise this data frame a little to analyse it appropriately, by making the ...
user avatar
  • 13.2k
3 votes

Correct variable/data use for day of week predictors

Yes, days of the week can be modelled either as continuous 'seasonal' features in time series models (sometimes with an additional binary variable indicating holidays), or a categorical variable. ...
user avatar
  • 13.2k
0 votes

When independent variables covary (not correlate) in the regression

I appreciate the attempt to make a reproducible example, this helps to diagnose problems. However, it's important for you to set a seed using set.seed() so we can ...
user avatar
  • 13.2k
0 votes

What to do if the number of parameters and observations are almost equal?

There are general solutions to the problem of more predictors than observations, including regularisation (such as LASSO), using a Bayesian prior (see Modelling with more variables than data points ...
user avatar
  • 13.2k
0 votes

Interpret multiple logistic regression

The interpretation of fitted regression coefficients $\beta_i$ as causal effect requires some additional insight into your scenario. E.g., if you know for sure that your variable Intervention is not ...
user avatar
  • 6,248
1 vote

Hows does coefficient $b_1$ change when estimating $b_1 x_1+b_2 x_2+b_3 x_3$ instead of $b_1 x_1+b_2 x_2$

No, there is no closed form for the the change in the estimated coefficient $\hat{b}_1$ when including additional independent variables. Why? When adding other independent variables, say $x_4, x_5, ......
user avatar
  • 184
0 votes

How to determine standard errors for treatment and control effects from multiple regression output

Let's start by saving cat as a factor. Then we can play with this a bit to see how it might be done. ...
user avatar
1 vote
Accepted

Making my linear regression model meet assumptions causes a large increase in mean squared error

I decided to remove outliers in my data that are more than 2 standard deviations from the mean in any predictor column. I think this alone could very well cause the reduction in performance you see. ...
user avatar
  • 13.2k
4 votes

What are the consequences of not including random effects in a linear model when they should be added?

There are 2 questions here: ...the population level predictions (based on the fixed effects coefficients) are virtually identical between these two models (standard vs. mixed). Interestingly, however,...
user avatar
  • 13.2k
0 votes

Negative output from multiple linear regression for precipitation time series

Negative values are permissible in linear regression; the Gaussian distribution permits all possible values. You might consider log-transforming your data, although that will cause you problems with ...
user avatar
1 vote
Accepted

How to interpret MSE, RMSE and MAE

I think that you already understands what there is to understand. These statistics are used to measure the average precision of prediction models, and to compare the accuracy of different models. You ...
user avatar
  • 446
0 votes

Methods to predict multiple dependent variables

A reasonable possibility is to make a Principal Component Analysis (PCA) of the $q$ dependent variables $Y_i$ and construct other $q$ independent variables as linear combinations: $$\tilde{Y}_i = \...
user avatar
  • 101
0 votes

How to consider time from vaccination on final outbreak size

If Vax represents a fraction of individuals vaccinated in a facility The time since vaccination might be considered a moderator of the effect of vaccination, ...
user avatar
  • 66.3k
0 votes

Data points for some control variables missing in regression - still feasible?

As dipetkov says, multiple-imputation is the best solution here. And imputing binary and categorical variables is both reasonable and quite feasible. This is a vignette that shows how to do this in ...
user avatar
  • 13.2k
1 vote

Add a covariate as an additional independent variable, an interaction, or a random intercept?

To give an authoritative answer about the best way to model this would require some subject knowledge, because causal pathways can be complex. But based on the information you have presented, the ...
user avatar
  • 13.2k
3 votes

Can I use both Spearman's rank correlations and multiple linear regression on the same data?

If you have ordinal outcome data, why not use ordinal regression? Frank Harrell has some useful links in this answer. The lrm() and ...
user avatar
  • 66.3k
1 vote

How to modify a coefficient in a linear regression

Processes that reduce the magnitude of coefficients are called "regularization". Normally, though, you decide what regularization to do before performing the regression, rather than looking ...
user avatar
11 votes
Accepted

How to modify a coefficient in a linear regression

It is possible to fix a coefficient at any value, though that would be dubious without a strong justification (such as a known physical constant). Omitting the variable would be the same as fixing its ...
user avatar
  • 13.2k
7 votes

Why is a "Correction" Required in Multiple Hypothesis Testing?

There is a lot of arbitrariness in certain statistical practices. As other responses and previous discussions in the literature point out, it's very hard to justify the tradition that you correct for ...
user avatar
  • 23.4k
4 votes

Why is a "Correction" Required in Multiple Hypothesis Testing?

I'm frequently asked when multiple comparison adjustment should be used. Then I start talking about false discovery rate, type-I errors and so on, to conclude that I was not fully understood. So I ...
user avatar
8 votes

Why is a "Correction" Required in Multiple Hypothesis Testing?

This is a tricky topic: when exactly do you correct for multiple testing? The two extremes are both problematic: never correcting for multiple testing will result in too many false positives, always ...
user avatar
  • 518
3 votes
Accepted

Inclusion of polynomial term in multiple linear regression

I suppose you use a ordinary least square (OLS) model to find the average effect of some $x$ on some outcome $y$? In order to see if some additional variable (or transformation) like $x^2$ benefits ...
user avatar
  • 216
4 votes

Is one-hot encoding required for a binary categorical variable?

Both models will yield exactly the same predictions -- but they might be harder to interpret than standard encodings. Let's look at them. Please note, though, that by "binary categorical ...
user avatar
  • 293k
1 vote

Is one-hot encoding required for a binary categorical variable?

@Tim, I think both would give same performance because: Rationale#1:Mathematically, there's enough room for parameter to adjust (to give same y value). Rationale#...
user avatar
  • 151
1 vote

Should I include IVs in my Regression model if my DV is based on them?

The percentile ranks for each variable mean approximately the same thing as the original variables; they carry almost the same information. If all of the 7 categories are used for each measure and you ...
user avatar
  • 23.3k

Top 50 recent answers are included