32 votes
Accepted

Why do we do matching for causal inference vs regressing on confounders?

As I see it, there are two related reasons to consider matching instead of regression. The first is assumptions about functional form, and the second is about proving to your audience that functional ...
user avatar
  • 22.4k
32 votes

Why doesn't mean square error work in case of angular data?

The problem with using MSE directly on angle values is that values can be very close together on the circle, but will have a large square error. For example, 359 degrees and 1 degrees are very close ...
user avatar
  • 79.4k
23 votes

Why is controlling for too many variables considered harmful?

There is no such thing as a "sweet spot" for the number of variables to control for in order to get an unbiased estimate of the causal effect. Since we are talking about confounding, we must ...
user avatar
15 votes

How large of a dataset should I use for building a statistical model?

You should usually fill in missing data when you can Whilst 40K+ rows is certainly a substantial dataset, the important issue here isn't so much the size of the dataset, but the question of whether or ...
user avatar
  • 96.7k
14 votes
Accepted

I've already used my entire dataset in a regression, should I not use that as a prediction model?

With so few cases, train/test splits aren't helpful. You then lose power in training the model and precision in testing it. What you've done so far is fine. You could go on to estimate how well the ...
user avatar
  • 63.9k
12 votes

Why is controlling for too many variables considered harmful?

I would point out three things: (1) Generally (related to the estimation of causal effects) Usually you want to explain phenomena out there in the world with parsimonious models including variables ...
user avatar
  • 327
12 votes
Accepted

Lasso coefficient for some features is higher than Linear Regression Coefficient

As German Demidov notes, this is perfectly fine. The Lasso will shrink some of your coefficients to zero, but it does not have the property of shrinking all coefficients compared to the OLS estimate. ...
user avatar
11 votes

What's WRONG with my multiple regression model

Your comment "Now, I am not good with cars, so I do not know exactly what parts of the data here can help achieve better output" is very insightful! You've correctly identified your problem, ...
user avatar
10 votes
Accepted

How to decide the best form of BMI used in cox regression, categorical or continuous?

BMI might be associated continuously with outcome but not necessarily linearly. The best way to test that is to fit BMI as a continuous predictor flexibly, for example with restricted cubic splines as ...
user avatar
  • 63.9k
10 votes
Accepted

Why doesn't Logistic Regression require heteroscedasticity and normality of the residuals, neither a linear relationship?

Isn't that something that would require linear relationships? The assumption is that the effect of covariates is linear on the log odds scale. You might see logistic regression written as $$ \...
user avatar
9 votes

Which weights are the weights of parameters of Linear Regression?

A Neural network with one layer using linear activation is a linear regression. If you have a multi-layer neural network that only uses linear activations, it will reduce to linear regression. In ...
user avatar
  • 115k
9 votes

Negative constant for income

The intercept of -40348,806 is the prediction if all predictors have a value of zero. Thus, a woman (sex = 0) of age 0 and height 0 would be predicted to have a negative income of -40348,806. This is ...
user avatar
8 votes
Accepted

Is it cheating to use least squares to obtain a prior for Bayesian regression?

Note that "a solution given by least squares" in itself is not a probability distribution and can't therefore be used as prior. I assume you mean to center the prior around the LS estimator (...
user avatar
8 votes
Accepted

How to assess normality under the OLS assumptions?

The first two are totally wrong but are common misconceptions about the normality assumption in OLS regression (when we choose to make such an assumption, which we don’t have to do). There is no ...
user avatar
  • 31.8k
7 votes

I've already used my entire dataset in a regression, should I not use that as a prediction model?

In my opinion the best course of action, if possible, is to collect more data and then use that data to check your current model as well as maybe top 5 of the previous models you tried. Continuing ...
user avatar
7 votes

When do I need something "fancier" than multiple regression?

For my money, if your goal is to understand the relationship between your predictors and the outcome, multiple regression is absolutely fine here, BUT you need to worry a bit about multiple ...
user avatar
  • 5,466
7 votes
Accepted

How can we assess whether the statistical relationship between measurements in one year is significantly different from that in other years?

You're on the right track, and you can answer your question using some old-fashioned tools. In your current model, the intercept reflects the predicted average productivity when ...
user avatar
  • 5,466
7 votes
Accepted

Logistic regression model for hospital readmissions: accounting for multiple admissions

If you have actual discharge and readmission dates for each patient, then this might best be handled with repeated-event survival analysis. That's presented in the main R ...
user avatar
  • 63.9k
7 votes

Lasso coefficient for some features is higher than Linear Regression Coefficient

Lasso coefficients can shrink again while you get closer to the OLS solution. See for instance: Why under joint least squares direction is it possible for some coefficients to decrease in LARS ...
user avatar
7 votes

Which weights are the weights of parameters of Linear Regression?

This neural network does regression, but what it does is not equivalent to linear regression with the original features. So, you cannot just get the weights of either layer and build a linear ...
user avatar
  • 52.1k
6 votes
Accepted

Residual Plot vs Fitted, linearity and Heteroskedasticity

Since your reponse variable is a count, then you need to fit a model for count data, such as a poisson or negative binomial, for example: ...
user avatar
  • 49.6k
6 votes

If X^2 is not significant but X is significant, do I have to remove X^2 and run again the regression analysis?

Quite generally, insignificant variables do not have to be removed from regression models (this also holds for parameters belonging to terms such as $X^2$). Also, generally, standard regression tests ...
user avatar
6 votes

Linearity assumption violated - can I still draw conclusions from my model?

One troubling situation is when the regressor variables are not independent from each other. This can make it look like there is seemingly (causal) relationship that is not truly present. Example: Say ...
user avatar
6 votes

Linearity assumption violated - can I still draw conclusions from my model?

A few things to start with. I agree that nature is rarely linear, but I disagree that the approach you've taken is a reasonable attempt to account for this. Polynomial terms do add non-linearity, ...
user avatar
6 votes
Accepted

Can I interpret coefficients for "Year" as differences between years that are not explained by my predictors?

Well done. I think your interpretation is OK. Running anova() will give you a more general test for a time trend than how you described it---a test of whether the ...
user avatar
6 votes
Accepted

is it possible to estimate least squares model coefficients by linear programming?

No, least squares regression can't be written as a linear program because the squared error is nonlinear w.r.t. the parameters. But, some least squares regression problems can be written as quadratic ...
user avatar
  • 29.8k
6 votes
Accepted

Logistic Regression on multiple classes (Shouldn't it be only on binary?)

Extending my comment into an answer: There are several natural extensions to handle multiple classes, and two are built into scikit-learn. Multinomial logistic ...
user avatar
5 votes

How to write a piecewise regression model as a linear model?

The overall model has four parameters: $\alpha_0,$ $\alpha_1,$ $\beta_0,$ and $\beta_1.$ Therefore, if a solution is at all possible, we must be able to construct four corresponding variables $z_1, ...
user avatar
  • 290k
5 votes

How can I deal with a covariate being defined only for a subset of my sample?

By (arbitrary) convention, people say you have 'a problem' with collinearity when the variance inflation factor ($VIF$) is greater than or equal to $10$. When considering a pair of variables, that ...
user avatar
5 votes
Accepted

Prove: Least Squares Prediction Equation Contains Mean Point

One place to start would be to replace your $a^T$ vector with $\frac1n$ times a row vector of $n$ 1's times $x$. This is just the matrix version of finding the means ($\bar{x}$) of the columns. When ...
user avatar
  • 47k

Only top scored, non community-wiki answers of a minimum length are eligible