11
votes
Accepted
What is the 'right' slope formula of a regression? deltas or Pearson?
For only two points they are the same.
The slope of simple linear regression is
$$
\hat \beta = \frac{\sum_i (x_i - \bar x) (y_i - \bar y)}{\sum_i (x_i - \bar x)^2}
$$
that is the same form you ...
Tim♦
- 120k
8
votes
Accepted
Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed
One of the issues when you introduce nonlinearities and interactions is that the change resulting in a change in a variable of interest depends on the starting value of that variable of interest and ...
- 35.4k
7
votes
What is the 'right' slope formula of a regression? deltas or Pearson?
These are fairly unrelated concepts.
In the first equation, that is the slope of the line connecting two points.
In the second, you are finding a line that fits multiple points best (according to a ...
- 35.4k
7
votes
Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed
This is a nice question, that touches on some interesting points in the history of neural networks (which I can only briefly mention here).
First, what you say is absolutely right if and only if the ...
- 6,161
5
votes
Dependent Variable takes on the values 0, 1, 2, 3 - What is the right (logistic) regression model to use?
If you stick really close to the data generating process, these are repeated binary decisions. I.e. each participants makes three decisions of choosing the local product or not (each time a 1/0 ...
- 24k
4
votes
Linear probability model with crossentropy (log) loss
You can't use cross-entropy loss with a linear model. Notice that it calculates $\log(\hat y)$ and $\log(1 - \hat y)$, while $\hat y$ can be negative or bigger than one, so logs would be undefined (<...
Tim♦
- 120k
4
votes
Linear probability model with crossentropy (log) loss
I anticipate there is a real problem with this approach because there is no way to enforce that $L$ can be computed in the case where -- as you mention -- $\hat{y}$ is outside the unit interval. In ...
- 27.7k
4
votes
Would there be any alternatives to a Logistic Regression or way to modify the Regression for what I'm looking for?
The probability will not hit 0% or 100% exactly, it approaches them asymptotically.
The uncertainty increases towards the extremes, which you will see if you include confidence intervals in your plot.
...
- 14.3k
4
votes
Dependent Variable takes on the values 0, 1, 2, 3 - What is the right (logistic) regression model to use?
Although ordinal regression is a generally useful choice for ordered outcomes, in this particular case a binary regression could also be considered.
In each of the 3 trials per individual there is a ...
- 68.7k
3
votes
Accepted
Why does the logistic model for binary logistic regression return the probability that the outcome was in class/category 1?
A logistic regression proposes a conditional binomial distribution and then estimates the parameters of those conditional distributions.
Binomial distributions are defined to give the probability of ...
- 35.4k
3
votes
Dependent Variable takes on the values 0, 1, 2, 3 - What is the right (logistic) regression model to use?
Yes, ordinal logistic regression (also referred to as the proportional odds model) is your best choice. If your outcome were to have 7+ ordered categories, linear regression may also be used, though ...
- 542
2
votes
Accepted
Same variable on both sides of a regression model
Your model reduces tor typical linear regression
$$\begin{align}
B_i - \hat{B_i} &= \beta_0 + \beta_1(A_i) + \beta_2(\hat{B}_i) +\ ... = \\
B_i &= \beta_0 + \beta_1(A_i) + (1+\beta_2)(\hat{B}...
Tim♦
- 120k
2
votes
How to motivate the definition of $R^2$ in `sklearn.metrics.r2_score`?
Squared correlation between the feature and the outcome
That would be the case if you have a single feature and the model is linear regression.
Squared correlation between the outcome and the ...
Tim♦
- 120k
1
vote
Accepted
What is the hat matrix and why is it inappropriate for GLMM standardized residuals?
Thanks to whuber for pointing out some of the entries for hat martices on Cross Validated. I did find this particular discussion of what a hat matrix useful to a degree. However, there really wasn't a ...
2
votes
Coefficient from Regressing the OLS Residual on X
The residual vector from OLS estimation is uncorrelated with the explanatory vectors in the regression model, so the estimated coefficient vector from the regression you are proposing would be the ...
- 102k
1
vote
Should I use standardised or unstandardised beta coefficient in this scenario?
You need to scale the data only for certain kinds of models, for numerical reasons. For example, when using regularization. If you are using a regression model without regularization, or random ...
Tim♦
- 120k
1
vote
Quantile regression necessarily has a solution with $r$ residuals equal to 0: why/how
I don't have time for a full answer now, but this is too much for a comment.
Quantile regression is solved via linear programming, so you will find some background at
Formulating quantile regression ...
- 67.4k
1
vote
GridSearchCV returns unrealistic AUC score with Logistic Regression
the initial dataset is widely unbalanced (92% / 8%) so I performed ADASYN oversampling to bring back our ratio to 50/50. So in that case X_train has been oversampled while (obviously) the validation ...
1
vote
Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed
You can interpret NN like that but it is pointless generally. The seeming utility in doing so for OLS is in decoupling of the partial derivative from other variables, I.e. $\partial/\partial x_1 f(x_1,...
- 57.1k
1
vote
GLM: how to treat multiple variables that all measure a confounding aspect in a slightly different way?
If you want to estimate the total causal effect of $x_0$ on the outcome $y$, you have to control for all the confounders. So you are on the right track.
If you are presuming a linear model, you need ...
- 7,566
1
vote
How do Measure "Robustness" in Statistics?
One method to examine "robustness" is to do a simulation analysis
Sometimes we have a statistical model and its estimators for some type of data, and we want to see if the estimators are &...
- 102k
1
vote
Linear probability model with crossentropy (log) loss
In glm function if the family=binomial is chosen, then link function must be logit like <...
1
vote
Probit/Logit or Linear regression Model?
I would go for a logit model, since modeling the probability of being a victim of a data break seems natural.
Alternatively, with your alternative summed response (which gives you count data), I would ...
- 67.4k
1
vote
Why in the Ridge regression, the coefficients cannot be 0?
what is the difference between "shrinking to exact 0" and "parameters can be 0"
If the OLS solution is non-zero (which means that $y$ is non-zero) then the ridge regression ...
- 51.4k
1
vote
Accepted
Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)
Note that $\hat{u}_t = \phi \hat{u}_{t-1} + v_t,\, v_t \sim \mathcal N\left(0, \sigma^2_{v}\right)$ implies $\hat{u}_t | \hat{u}_{t-1}\sim \mathcal N\left(\phi \hat{u}_{t-1}, \sigma^2_{v}\right)$ for $...
- 2,350
1
vote
Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)
It would appear that Feasible Generalized Least Square (FGLS) should be used when the covariance matrix of the errors $cov(u) = \sigma^2V$ has a completely unknown form. However, in this problem, you ...
- 361
1
vote
Would there be any alternatives to a Logistic Regression or way to modify the Regression for what I'm looking for?
My data is widely dispersed across the X axis for both my 1s and 0s
however there is slightly more 1s the higher the X and slightly more
0s the lower the X. So the probabilities listed in a standard ...
- 4,444
Only top scored, non community-wiki answers of a minimum length are eligible