Linked Questions

0
votes
3answers
835 views

Why does machine learning work for high-dimensional data($n \ll p$)?

Consider the high dimensional data with which the number of features $p$ is much larger than the number of observations $n$. Machine learning algorithm is trained with the data. My first thought is ...
0
votes
2answers
177 views

In a neural network, why can't there be more weights than the number of observations?

After having this exact same issue with caret, I arrived at this thread. However, I do not intuitively understand why this answer is correct. Why can't there be ...
6
votes
0answers
1k views

Bias Variance tradeoff from a Bayesian perspective

I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
3
votes
1answer
896 views

What is parameter instability? How can I measure it? [closed]

What is parameter instability and how can I measure it? If my model is having a hard time to forecast out-of-time samples, could parameter instability or populational instability be the cause of it?
5
votes
1answer
160 views

Bias-Variance decomposition: Expectations over what?

The Bias Variance decomposition is a decomposition of an expectation, but I fail to follow what's actually assumed random specifically in this decomposition. Take the specific regression example ...
1
vote
1answer
91 views

Variable selection without strong theory: Can we do better than LASSO for prediction?

When a variable of interest has many plausible explanatory variables, and one lakes strong theoretical or subject-matter grounds for selecting among them, it is tempting to build a “kitchen sink” ...
1
vote
3answers
129 views

Analogy for the process of neural networks

Consider a basic neural network like what you would expect to see in any beginner tutorial or course, and attempts to classify images as either 'cat' or 'no cat'. I have a few questions that I've ...
2
votes
2answers
88 views

Total Cost Shrinkage

I have a question regarding Shrinkage Methods. I am currently writing a term paper about ridge regression and lasso and before explaining the two methods, I want to give some theory on why shrinking ...
0
votes
1answer
90 views

Training of a deep Artificial Neural Network

I have few doubts related to training a neural network with more parameters (weights and biases) than number of data points. I know there exists discussion (on this platform) related to training such ...
1
vote
1answer
77 views

Improving a boosted regression model or change?

I am looking at a data set that contains multiple predictors and a continuous response. Using dismo along with gbm I built (a terrible one?) model. Using the package sROC, I got an AUC or 0.48 - so my ...
3
votes
1answer
46 views

Why do we need Forward Selection, Forward Stagewise and LARS for solve linear regression

Forward Selection, Forward Stagewise and LARS are all used to solve the regression problem: $...
0
votes
0answers
20 views

How matematics deal with the number of parameters chosen for regressions

When approximating a set of points, with a model of a function and error, there is a general tool/criteria to decide how many parameters are optimal? For example, a set of points in 2 dimension can ...

15 30 50 per page
1
2