835 views

### Why does machine learning work for high-dimensional data($n \ll p$)?

Consider the high dimensional data with which the number of features $p$ is much larger than the number of observations $n$. Machine learning algorithm is trained with the data. My first thought is ...
177 views

### In a neural network, why can't there be more weights than the number of observations?

After having this exact same issue with caret, I arrived at this thread. However, I do not intuitively understand why this answer is correct. Why can't there be ...
1k views

### Bias Variance tradeoff from a Bayesian perspective

I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
896 views

### What is parameter instability? How can I measure it? [closed]

What is parameter instability and how can I measure it? If my model is having a hard time to forecast out-of-time samples, could parameter instability or populational instability be the cause of it?
160 views

### Bias-Variance decomposition: Expectations over what?

The Bias Variance decomposition is a decomposition of an expectation, but I fail to follow what's actually assumed random specifically in this decomposition. Take the specific regression example ...
91 views

### Variable selection without strong theory: Can we do better than LASSO for prediction?

When a variable of interest has many plausible explanatory variables, and one lakes strong theoretical or subject-matter grounds for selecting among them, it is tempting to build a “kitchen sink” ...
129 views

### Analogy for the process of neural networks

Consider a basic neural network like what you would expect to see in any beginner tutorial or course, and attempts to classify images as either 'cat' or 'no cat'. I have a few questions that I've ...
88 views

### Total Cost Shrinkage

I have a question regarding Shrinkage Methods. I am currently writing a term paper about ridge regression and lasso and before explaining the two methods, I want to give some theory on why shrinking ...
90 views

### Training of a deep Artificial Neural Network

I have few doubts related to training a neural network with more parameters (weights and biases) than number of data points. I know there exists discussion (on this platform) related to training such ...
77 views

### Improving a boosted regression model or change?

I am looking at a data set that contains multiple predictors and a continuous response. Using dismo along with gbm I built (a terrible one?) model. Using the package sROC, I got an AUC or 0.48 - so my ...