17
votes
Accepted
Chance that bootstrap sample is exactly the same as the original sample
Note that at each observation position ($i=1, 2, ..., n$) we can choose any of the $n$ observations, so there are $n^n$ possible resamples (keeping the order in which they are drawn) of which $n!$ are ...
- 283k
6
votes
Accepted
Intuition behind m-out-of-n bootstrap
I would argue that it's not so much that the $m$ of $n$ bootstrap does smoothing as that it makes smoothing unnecessary.
There are two components to the $m$ of $n$ bootstrap. The first is sampling ...
- 38.7k
5
votes
Accepted
Can I take a random sample of my very large data set to overcome non-independence?
If you downsample time-series data it would not remove the dependence, it would just dilute it. Say that your data follows the relationship
$$
y_{t+1} = f(y_{t}) + \varepsilon_{t+1}
$$
so the current ...
Tim♦
- 138k
4
votes
Accepted
How often to subsample for classification?
I feel that your understanding of subsampling is not quite correct or your problem is too complex. For two classes subsampling is basically throwing away majority of samples from the larger class, so ...
- 331
4
votes
Accepted
How to: Normal sub-sampling out of a uniformly distributed data samples
Subsample your original data without replacement. The crucial point is to weight each of the original points with the density you are aiming for (parameterizing it, e.g., by the mean and the variance ...
- 124k
3
votes
Does sampling from a large dataset lead to correct inferences?
If you have the whole population, you are not really doing any inference of a variable, that only happens when you are taking a sample. Let's say you are using a model that predicts weight based on ...
- 3,644
3
votes
Accepted
What is a good introductory text on resampling methods?
You can look at:
Philip Good's book on resampling methods titled "Permutation Tests";
my texts on bootstrap methods published by Wiley, e.g. "Bootstrap Methods: A Guide for ...
2
votes
What is the effect of using survey sample weights for a sub-sample?
most nationally-representative survey weights are generated with those certain demographic characteristics (e.g. age, race, gender) as a part of their fundamental construction. unless you have a ...
- 202
2
votes
Accepted
Is it good practice to perform model parameter tuning on a random subsampling of a large dataset?
This question is really broad. Depends on the data and model, it can be a good practice and can be bad.
The overall idea is to think about the "complexity of data and model". We may need to review ...
- 36.9k
2
votes
What is the typical size of feature matrix for xgboost
The typical number of features is the number of features that you think are relevant. No more, no less.
It might be possible to tabulate the number of features used in every XGBoost model, and ...
- 91k
2
votes
Accepted
Variance of subsample from a distribution
A random sample of size $n$ from a sample of size $N$ from a $N(0, \sigma^2)$ should also be a sample of size $n$ from $N(0, \sigma^2)$. So I don't think anything changes and
$$ \bar{X}_n \sim N(0, \...
- 15.6k
2
votes
How to compute ESS (Effective Sample Size)?
It is not clear how you are subsampling and how you are estimating the correlation in your results, but your function systematically under-estimates the effective sample size by a factor of about 2.1 (...
- 181
2
votes
Accepted
Estimate linear regression coefficients and standard errors using sub-samples of dataset
The variance of an ordinary least squares estimate (that is, the square of its standard error) is inversely proportional to the sample size.
In practice, there is an additional form of uncertainty in ...
- 323k
2
votes
Sub-sampling a dataset to a different target distribution without replacement - bias correction?
For arbitrary distributions, there isn't any way to ensure that a subsample of $X$ has a similar distribution to $Y$, whether you use replacement or not. Consider the distributions where $X \sim U(1,2)...
- 9,327
2
votes
Accepted
Sampling 1 item from groups of correlated values and combining the statistics
I fail to grasp the objective and need clarification of the question. Are you trying to demonstrate a point that the sample shows a special pattern of uniformly distributed values unlike both narrow ...
- 1,544
1
vote
Accepted
Sub-sampling a dataset to a different target distribution without replacement - bias correction?
It looks like you are trying to use importance sampling, which is the technique that I would have suggested for this problem.
First, a note: your implementation is different to the usual way of ...
- 26
1
vote
Linear regression in very unbalanced data
Why not use a mixed model? Stratified sampling will obfuscate the natural frequencies of the classes in the population (assuming the sizes of your sample reflect the sizes in the population ...
- 36.5k
1
vote
Finding the variance of subsample-based estimation
This subsample is identical to taking an iid sample of $n$ observations from a Bernoulli$(P)$ distribution, whence $k^\prime$ has a Binomial$(n,P)$ distribution.
Let's prove this rigorously. To do so,...
- 323k
1
vote
Do both Bootstrap with and without replacement create a distribution?
Bootstrap methods require resampling using the same sample size as that of the original sample. If you use fewer (or more!) samples, this is called "upstrap"
The Upstrap.
This, however, has ...
- 11
1
vote
Can I take a random sample of my very large data set to overcome non-independence?
I will add a couple of points to Tim's answer, focussing on the original question, which was "My question is - is this valid? Am I missing anything here?". I think the approach can be valid, ...
- 331
1
vote
Does sampling from a large dataset lead to correct inferences?
Yes, this works. All data is a sample population. If you have enough to achieve some level of performance on some metric, than you have achieved your goal. There will generally be a point of ...
- 219
1
vote
Accepted
Repeated k-fold CV of sub sample - repeat the k-fold CV or repeat the sub sampling?
This depends what you want to use the validation result for:
if you want to validate (measure performance) the SVM trained on the first 1k cases (and then want to use that model for prediction), do ...
1
vote
Accepted
Variance of subsample
Let me just introduce some notation first to make this easier. Let's denote the total number of data points in the full sample as $N$. The variance of the full sample is
$$\sigma_N^2 = \frac{1}{N} \...
- 126
1
vote
Accepted
Subsampling to determine a standard error, how does it work?
There are a few problems in your code.
First of all, why use var(res) if you are interested in the standard deviation of the mean? Instead just use ...
- 1,457
1
vote
Accepted
What does it mean that coefficient is significant for full sample but not significant when split into two subsamples?
Your analysis results essentially answer your question. Much is from the smaller number of cases, but the lack of normally distributed residuals means that the p-values are not reliable, and there is ...
- 92.4k
1
vote
How to undersample with algorithms in R to solve class imbalance?
First off, 95%-5% is NOT an example of imbalanced dataset, I would consider downsampling if there was something of the order of 99.9%-0.1%. Most approaches should work just fine.
Edit: Seems like ...
- 392
Only top scored, non community-wiki answers of a minimum length are eligible