11 votes
Accepted

What is the 'right' slope formula of a regression? deltas or Pearson?

For only two points they are the same. The slope of simple linear regression is $$ \hat \beta = \frac{\sum_i (x_i - \bar x) (y_i - \bar y)}{\sum_i (x_i - \bar x)^2} $$ that is the same form you ...
  • 120k
8 votes
Accepted

Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed

One of the issues when you introduce nonlinearities and interactions is that the change resulting in a change in a variable of interest depends on the starting value of that variable of interest and ...
  • 35.4k
7 votes

What is the 'right' slope formula of a regression? deltas or Pearson?

These are fairly unrelated concepts. In the first equation, that is the slope of the line connecting two points. In the second, you are finding a line that fits multiple points best (according to a ...
  • 35.4k
7 votes

Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed

This is a nice question, that touches on some interesting points in the history of neural networks (which I can only briefly mention here). First, what you say is absolutely right if and only if the ...
  • 6,161
5 votes

Dependent Variable takes on the values 0, 1, 2, 3 - What is the right (logistic) regression model to use?

If you stick really close to the data generating process, these are repeated binary decisions. I.e. each participants makes three decisions of choosing the local product or not (each time a 1/0 ...
  • 24k
4 votes

Linear probability model with crossentropy (log) loss

You can't use cross-entropy loss with a linear model. Notice that it calculates $\log(\hat y)$ and $\log(1 - \hat y)$, while $\hat y$ can be negative or bigger than one, so logs would be undefined (<...
  • 120k
4 votes

Linear probability model with crossentropy (log) loss

I anticipate there is a real problem with this approach because there is no way to enforce that $L$ can be computed in the case where -- as you mention -- $\hat{y}$ is outside the unit interval. In ...
4 votes

Would there be any alternatives to a Logistic Regression or way to modify the Regression for what I'm looking for?

The probability will not hit 0% or 100% exactly, it approaches them asymptotically. The uncertainty increases towards the extremes, which you will see if you include confidence intervals in your plot. ...
  • 14.3k
4 votes

Dependent Variable takes on the values 0, 1, 2, 3 - What is the right (logistic) regression model to use?

Although ordinal regression is a generally useful choice for ordered outcomes, in this particular case a binary regression could also be considered. In each of the 3 trials per individual there is a ...
  • 68.7k
3 votes
Accepted

Why does the logistic model for binary logistic regression return the probability that the outcome was in class/category 1?

A logistic regression proposes a conditional binomial distribution and then estimates the parameters of those conditional distributions. Binomial distributions are defined to give the probability of ...
  • 35.4k
3 votes

Dependent Variable takes on the values 0, 1, 2, 3 - What is the right (logistic) regression model to use?

Yes, ordinal logistic regression (also referred to as the proportional odds model) is your best choice. If your outcome were to have 7+ ordered categories, linear regression may also be used, though ...
2 votes
Accepted

Same variable on both sides of a regression model

Your model reduces tor typical linear regression $$\begin{align} B_i - \hat{B_i} &= \beta_0 + \beta_1(A_i) + \beta_2(\hat{B}_i) +\ ... = \\ B_i &= \beta_0 + \beta_1(A_i) + (1+\beta_2)(\hat{B}...
  • 120k
2 votes

How to motivate the definition of $R^2$ in `sklearn.metrics.r2_score`?

Squared correlation between the feature and the outcome That would be the case if you have a single feature and the model is linear regression. Squared correlation between the outcome and the ...
  • 120k
1 vote
Accepted

What is the hat matrix and why is it inappropriate for GLMM standardized residuals?

Thanks to whuber for pointing out some of the entries for hat martices on Cross Validated. I did find this particular discussion of what a hat matrix useful to a degree. However, there really wasn't a ...
2 votes

Coefficient from Regressing the OLS Residual on X

The residual vector from OLS estimation is uncorrelated with the explanatory vectors in the regression model, so the estimated coefficient vector from the regression you are proposing would be the ...
  • 102k
1 vote

Should I use standardised or unstandardised beta coefficient in this scenario?

You need to scale the data only for certain kinds of models, for numerical reasons. For example, when using regularization. If you are using a regression model without regularization, or random ...
  • 120k
1 vote

Quantile regression necessarily has a solution with $r$ residuals equal to 0: why/how

I don't have time for a full answer now, but this is too much for a comment. Quantile regression is solved via linear programming, so you will find some background at Formulating quantile regression ...
1 vote

GridSearchCV returns unrealistic AUC score with Logistic Regression

the initial dataset is widely unbalanced (92% / 8%) so I performed ADASYN oversampling to bring back our ratio to 50/50. So in that case X_train has been oversampled while (obviously) the validation ...
1 vote

Interpret neural network like the linear regression equation such as how much will Y change if we change X1 and keep the other variables fixed

You can interpret NN like that but it is pointless generally. The seeming utility in doing so for OLS is in decoupling of the partial derivative from other variables, I.e. $\partial/\partial x_1 f(x_1,...
  • 57.1k
1 vote

GLM: how to treat multiple variables that all measure a confounding aspect in a slightly different way?

If you want to estimate the total causal effect of $x_0$ on the outcome $y$, you have to control for all the confounders. So you are on the right track. If you are presuming a linear model, you need ...
  • 7,566
1 vote

How do Measure "Robustness" in Statistics?

One method to examine "robustness" is to do a simulation analysis Sometimes we have a statistical model and its estimators for some type of data, and we want to see if the estimators are &...
  • 102k
1 vote

Linear probability model with crossentropy (log) loss

In glm function if the family=binomial is chosen, then link function must be logit like <...
1 vote

Probit/Logit or Linear regression Model?

I would go for a logit model, since modeling the probability of being a victim of a data break seems natural. Alternatively, with your alternative summed response (which gives you count data), I would ...
1 vote

Why in the Ridge regression, the coefficients cannot be 0?

what is the difference between "shrinking to exact 0" and "parameters can be 0" If the OLS solution is non-zero (which means that $y$ is non-zero) then the ridge regression ...
1 vote
Accepted

Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

Note that $\hat{u}_t = \phi \hat{u}_{t-1} + v_t,\, v_t \sim \mathcal N\left(0, \sigma^2_{v}\right)$ implies $\hat{u}_t | \hat{u}_{t-1}\sim \mathcal N\left(\phi \hat{u}_{t-1}, \sigma^2_{v}\right)$ for $...
  • 2,350
1 vote

Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

It would appear that Feasible Generalized Least Square (FGLS) should be used when the covariance matrix of the errors $cov(u) = \sigma^2V$ has a completely unknown form. However, in this problem, you ...
  • 361
1 vote

Would there be any alternatives to a Logistic Regression or way to modify the Regression for what I'm looking for?

My data is widely dispersed across the X axis for both my 1s and 0s however there is slightly more 1s the higher the X and slightly more 0s the lower the X. So the probabilities listed in a standard ...
  • 4,444

Only top scored, non community-wiki answers of a minimum length are eligible