32
votes
Accepted
Why do we do matching for causal inference vs regressing on confounders?
As I see it, there are two related reasons to consider matching instead of regression. The first is assumptions about functional form, and the second is about proving to your audience that functional ...
- 22.4k
32
votes
Why doesn't mean square error work in case of angular data?
The problem with using MSE directly on angle values is that values can be very close together on the circle, but will have a large square error. For example, 359 degrees and 1 degrees are very close ...
- 79.4k
23
votes
Why is controlling for too many variables considered harmful?
There is no such thing as a "sweet spot" for the number of variables to control for in order to get an unbiased estimate of the causal effect. Since we are talking about confounding, we must ...
- 3,338
15
votes
How large of a dataset should I use for building a statistical model?
You should usually fill in missing data when you can
Whilst 40K+ rows is certainly a substantial dataset, the important issue here isn't so much the size of the dataset, but the question of whether or ...
- 96.7k
14
votes
Accepted
I've already used my entire dataset in a regression, should I not use that as a prediction model?
With so few cases, train/test splits aren't helpful. You then lose power in training the model and precision in testing it.
What you've done so far is fine. You could go on to estimate how well the ...
- 63.9k
12
votes
Why is controlling for too many variables considered harmful?
I would point out three things:
(1) Generally (related to the estimation of causal effects)
Usually you want to explain phenomena out there in the world with parsimonious models including variables ...
- 327
12
votes
Accepted
Lasso coefficient for some features is higher than Linear Regression Coefficient
As German Demidov notes, this is perfectly fine. The Lasso will shrink some of your coefficients to zero, but it does not have the property of shrinking all coefficients compared to the OLS estimate. ...
- 96.8k
11
votes
What's WRONG with my multiple regression model
Your comment "Now, I am not good with cars, so I do not know exactly what parts of the data here can help achieve better output" is very insightful! You've correctly identified your problem, ...
- 111
10
votes
Accepted
How to decide the best form of BMI used in cox regression, categorical or continuous?
BMI might be associated continuously with outcome but not necessarily linearly. The best way to test that is to fit BMI as a continuous predictor flexibly, for example with restricted cubic splines as ...
- 63.9k
10
votes
Accepted
Why doesn't Logistic Regression require heteroscedasticity and normality of the residuals, neither a linear relationship?
Isn't that something that would require linear relationships?
The assumption is that the effect of covariates is linear on the log odds scale. You might see logistic regression written as
$$ \...
- 26.4k
9
votes
Which weights are the weights of parameters of Linear Regression?
A Neural network with one layer using linear activation is a linear regression. If you have a multi-layer neural network that only uses linear activations, it will reduce to linear regression. In ...
Tim♦
- 115k
9
votes
Negative constant for income
The intercept of -40348,806 is the prediction if all predictors have a value of zero.
Thus, a woman (sex = 0) of age 0 and height 0 would be predicted to have a negative income of -40348,806.
This is ...
- 96.8k
8
votes
Accepted
Is it cheating to use least squares to obtain a prior for Bayesian regression?
Note that "a solution given by least squares" in itself is not a probability distribution and can't therefore be used as prior. I assume you mean to center the prior around the LS estimator (...
- 12.3k
8
votes
Accepted
How to assess normality under the OLS assumptions?
The first two are totally wrong but are common misconceptions about the normality assumption in OLS regression (when we choose to make such an assumption, which we don’t have to do).
There is no ...
- 31.8k
7
votes
I've already used my entire dataset in a regression, should I not use that as a prediction model?
In my opinion the best course of action, if possible, is to collect more data and then use that data to check your current model as well as maybe top 5 of the previous models you tried.
Continuing ...
- 4,387
7
votes
When do I need something "fancier" than multiple regression?
For my money, if your goal is to understand the relationship between your predictors and the outcome, multiple regression is absolutely fine here, BUT you need to worry a bit about multiple ...
- 5,466
7
votes
Accepted
How can we assess whether the statistical relationship between measurements in one year is significantly different from that in other years?
You're on the right track, and you can answer your question using some old-fashioned tools.
In your current model, the intercept reflects the predicted average productivity when ...
- 5,466
7
votes
Accepted
Logistic regression model for hospital readmissions: accounting for multiple admissions
If you have actual discharge and readmission dates for each patient, then this might best be handled with repeated-event survival analysis. That's presented in the main R ...
- 63.9k
7
votes
Lasso coefficient for some features is higher than Linear Regression Coefficient
Lasso coefficients can shrink again while you get closer to the OLS solution.
See for instance: Why under joint least squares direction is it possible for some coefficients to decrease in LARS ...
- 48.2k
7
votes
Which weights are the weights of parameters of Linear Regression?
This neural network does regression, but what it does is not equivalent to linear regression with the original features. So, you cannot just get the weights of either layer and build a linear ...
- 52.1k
6
votes
Accepted
Residual Plot vs Fitted, linearity and Heteroskedasticity
Since your reponse variable is a count, then you need to fit a model for count data, such as a poisson or negative binomial, for example:
...
- 49.6k
6
votes
If X^2 is not significant but X is significant, do I have to remove X^2 and run again the regression analysis?
Quite generally, insignificant variables do not have to be removed from regression models (this also holds for parameters belonging to terms such as $X^2$). Also, generally, standard regression tests ...
- 12.3k
6
votes
Linearity assumption violated - can I still draw conclusions from my model?
One troubling situation is when the regressor variables are not independent from each other. This can make it look like there is seemingly (causal) relationship that is not truly present.
Example:
Say ...
- 48.2k
6
votes
Linearity assumption violated - can I still draw conclusions from my model?
A few things to start with. I agree that nature is rarely linear, but I disagree that the approach you've taken is a reasonable attempt to account for this. Polynomial terms do add non-linearity, ...
- 26.4k
6
votes
Accepted
Can I interpret coefficients for "Year" as differences between years that are not explained by my predictors?
Well done. I think your interpretation is OK. Running anova() will give you a more general test for a time trend than how you described it---a test of whether the ...
- 75.8k
6
votes
Accepted
is it possible to estimate least squares model coefficients by linear programming?
No, least squares regression can't be written as a linear program because the squared error is nonlinear w.r.t. the parameters. But, some least squares regression problems can be written as quadratic ...
- 29.8k
6
votes
Accepted
Logistic Regression on multiple classes (Shouldn't it be only on binary?)
Extending my comment into an answer:
There are several natural extensions to handle multiple classes, and two are built into scikit-learn. Multinomial logistic ...
- 6,927
5
votes
How to write a piecewise regression model as a linear model?
The overall model has four parameters: $\alpha_0,$ $\alpha_1,$ $\beta_0,$ and $\beta_1.$ Therefore, if a solution is at all possible, we must be able to construct four corresponding variables $z_1, ...
- 290k
5
votes
How can I deal with a covariate being defined only for a subset of my sample?
By (arbitrary) convention, people say you have 'a problem' with collinearity when the variance inflation factor ($VIF$) is greater than or equal to $10$. When considering a pair of variables, that ...
5
votes
Accepted
Prove: Least Squares Prediction Equation Contains Mean Point
One place to start would be to replace your $a^T$ vector with $\frac1n$ times a row vector of $n$ 1's times $x$. This is just the matrix version of finding the means ($\bar{x}$) of the columns. When ...
- 47k
Only top scored, non community-wiki answers of a minimum length are eligible