Search Results

However, what if instead of OLS I run a regularized least squares (elastic net, lasso, ridge, etc.) regression. … Would that give me any information I didn't already have when I originally ran the regularized regression? …

Suppose you add a feature $x$ to an OLS model. The AIC goes down by 100 but the new feature is not statistically significant. The expanded model should be preferred overall but is the estimated coeffi …

Suppose I have a model like this: $$ y = \beta_1x_1 + \beta_2x_2 + \beta_3z_1 +\beta_4z_2 + \epsilon $$ where $\epsilon$ is noise. It so happens that $$x_1 + x_2 = z_1 + z_2$$ but there is no othe …

Suppose I have this model: $$y = \beta_1x + \beta_2x^2 + \epsilon$$ I would like to fit it using OLS. In my data the correlation between $x$ and $x^2$ is $0.91$. After I rescale $x$ to zero mean and u …

We are minimizing either the $1$ norm of the residuals, least absolute value, or the $2$ norm, least squares. Least absolute value: $$\min_\beta||y - x \beta||_1$$ Least squares: $$\min_\beta||y …

Suppose I have a model like $$y =\alpha + x_1\beta $$ and that there exists another variable, $x_2$, that is correlated with both $y$ and $x_1$. However, changing $x_1$ will cause changes in $x_2$ …

I need to calculate the DFBETA statistics for a logistic regression. The number of columns in the $X$ matrix is $3000$ and the number of records is in the millions. …

How do I set up the dummies to fit a linear regression model? …

The Gauss-Markov theorem states that for a linear model $$y = X \beta + \epsilon $$ if both of the conditions are true $$\operatorname E[\epsilon \mid X] = 0$$ $$\operatorname{Var}(\epsilon) = \sig …

Suppose I have an OLS model like this: $$y = \beta_1x_1 + \beta_2x_2 + \epsilon$$ If you sum the variables first, $x_3 = x_1 + x_2$ and fit $$y = \beta_3x_3 +\epsilon$$ I expect that $\beta_3$ is the …

Suppose I am modeling some quantity $y$ as a function of a covariate $x$, using a model from the linear family. $$ y = \beta x + \epsilon $$ For simplicity let's assume $x$ is categorical with $2$ val …

I have a data set of $200$ observations of $10,000$ features. I want to use this data to make numeric predictions of a target variable $y$. Will trees, in particular XGBoost, be useful here? I feel li …

Suppose you have data on $X$ and $Y$. You're interested in predicting $Y$ but you're only interested in $Y$ when $X > n$. Should you use all values of $X, Y$ pairs or just the ones where $X > n$?

Suppose all the data is positive. Squaring it means that the bigger values in the hump will get multiplied by a bigger number than the smaller values in the tail. Doesn't that just exacerbate the prob …

Suppose I run a linear model with shrinkage, say elastic net, and only 2 coefficients remain. The model then looks like $$y = \beta_1 x_1 + \beta_2 x_2 + \epsilon$$ Can I say that $\beta_1$ is the i …