67 views

### Find correlations based on large multivariable time-based data with one output per dataset

I am not well versed in anything beyond basic statistics but have been tasked with coming up with a "grading" scale for wear on a part based on data we have collected. I am in need of help figuring ...
33 views

### Is it useful to use sparse regression (e.g. Lasso) when the number of observations is significantly larger than the number of covariates?

I'm learning about penalized/sparse regression and I noticed that the examples used for penalized/sparse regression, e.g. Lasso, are usually cases where the number of observations is significantly ...
13 views

### Strategy to analyze large ( 20 mill rows and 200 columns) to predict a single variable

I am curious to understand how data scientists attack exceedingly large datasets in order to build a regression model for y? How does one decide where to start from? Reduce a large number of columns ...
1k views

### L1 and L2 penalty vs L1 and L2 norms

I understand the usages of L1 and L2 norms however I am unsure of usage of L1 and L2 penalty when building models. From what I understand: L1: Laplace Prior L2: Gaussian Prior are two of the ...
5k views

### Why is the James-Stein estimator called a “shrinkage” estimator?

I have been reading about the James-Stein estimator. It is defined, in this notes, as $$\hat{\theta}=\left(1 - \frac{p-2}{\|X\|^2}\right)X$$ I have read the proof but I don't understand the ...
1k views

### Why we use Ridge regression instead of Least squares in Multicollinearity?

Why do we use Ridge regression instead of Least squares in Multicollinearity? Which one is correct: a. lower bias and higher variance b. lower bias with the same variance c. higher bias with a ...
19k views

### Is regression with L1 regularization the same as Lasso, and with L2 regularization the same as ridge regression? And how to write “Lasso”?

I'm a software engineer learning machine learning, particularly through Andrew Ng's machine learning courses. While studying linear regression with regularization, I've found terms that are confusing: ...
99 views

### Choosing between feature selection and regularization to overcome over-fitting in categorical regression

In order to overcome over-fitting during a regression process over categorical features, one can either 1) Apply L1/L2/Elastic regularization during the regression, for example as answered here ...
213 views

### How to identify important independent variables for a dependent variable?

I have a dependent variable (DV) and about 200 independent variables (IVs). I want to understand which of the 10-20 variables are important for this DV. I could do: PCA - However it'll only tell me ...
135 views

### Logistic sample and case numbers

I have some questions about binary logistic regression. For my research, I am planning to use 12 predictors, and my sample consists of 129 cases. However, I know of a 1 to 10 rule. Additionally, my ...
464 views

### Cross validation for variable selection and coefficient shrinkage?

Is cross validation an appropriate technique for variable selection and regression coefficient shrinkage? A former colleague of mine used 10-fold CV to compare the regression coefficients from the ...