Questions tagged [resampling]
Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.
391
questions
1
vote
1
answer
33
views
Resampling a timeseries
I have a list of stock returns (say computed from the historical data) and would like to resample the historical return distribution. Naively doing bootstrapping means the samples are iid. I'm ...
- 594
1
vote
1
answer
18
views
How to generate data or sample data from a summary tables?
I have a dataset like this:
I only have summary tables of the "loneliness score" for overall, for each sex, each age group, and sex * age interactions. Besides this, I have 23 items that ...
- 11
2
votes
0
answers
13
views
How to calculate 2-D confidence intervals with the boot package in R? [closed]
I'm making a scatter plot of two statistics X(e) and Y(e) for various values of scalar parameter e. The sampling distribution of both X and Y is not normally distributed.
Now I want to calculate a 2-D ...
- 121
1
vote
0
answers
59
views
Bootstrap/resampling and bayesian regression, solving dependency problems
Let's say we have two experiment replications R1 and R2, (with some modifications in R2), deployed for intersecting (but not fully the same, let's say >50% but <80%) populations of participants ...
- 81
1
vote
1
answer
49
views
What are the minimal sample size requirements for cross-validation or bootstrapping?
I hope it makes sense to even ask these questions, but I'm wondering how can I evaluate the "validation" procedures that my data allow me to perform (i.e. cross-validation or bootstrap: ...
- 507
0
votes
1
answer
28
views
How many independet test-train splits (with independent training) should I perform?
I recently read some literature by Bagnall et al https://arxiv.org/pdf/1602.01711.pdf "The Great Time Series Classification Bake Off". If I understand them correctly, they advise to perform ...
- 197
0
votes
0
answers
43
views
Confidence interval and/or sampling distribution in double / two-phase sampling for stratification
Question: How to calculate a 95% confidence interval for estimated population mean in double / two-phase sampling for stratification where a simple random sample is taken at each phase?
Consider the ...
- 1
0
votes
1
answer
36
views
Resources to understand the bootstrap
I am looking for any recommendations on resources to learn more about bootstrapping from a theoretical and rigorous perspective in terms of bounds/guarantees/etc. Any books or papers would be very ...
0
votes
0
answers
36
views
Can balancing of the majority and minority classes (RUS/SMOTE/...) improve AUC of a binary classifier?
Given that AUC is a threshold independent measure, can undersampling or oversampling of the majority/minority class during training improve the performance of a binary classifier?
In my experience ...
- 45
1
vote
1
answer
33
views
Re-sampling at defined percentage of the population
Background: I have data that contains the severity scores of the pathology findings of tissues from different subjects with 100% sampling of the organ (let's call this dataset population, which ...
- 11
1
vote
1
answer
16
views
Sample size determination for finding the "top 100" largest X among multiple datasets
My data sources
Source A gives me a ranking of the top 1.000 largest X (say, companies).
Source B provides a similar ranking, comprising 7.000 X, with some differences to Source A (different values, ...
- 111
2
votes
0
answers
61
views
Train-Test Split with nested groups and multiple balancing factors
I have a large (~15,000) sample of data from individuals nested within families (with about half the data points sharing a family). I want to split the sample in to a training and test set so I can ...
- 41
0
votes
1
answer
33
views
In which specific situations is minority class oversampling useful? [duplicate]
I understand that, in the context of a binary classification problem, downsampling the majority class is a useful strategy to come up with a smaller, computationally friendly dataset. Using this ...
1
vote
2
answers
51
views
Sub-sampling a dataset to a different target distribution without replacement - bias correction?
Suppose i have dataset $X$ and $Y$, and i want to sub-sample from $X$ so that the resulting (sample) distribution is as close to that of $Y$ as possible.
One thing i can do is subsample with ...
- 1,232
0
votes
0
answers
14
views
Approximately Unbiased P-value vs Bootstrap Probability: which one should i choose?
Some references first:
How is approximately unbiased bootstrap better than a regular bootstrap with regards to hierarchical clustering?
Suzuki et al. 2004 https://www.researchgate.net/publication/...
- 1
1
vote
0
answers
108
views
For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?
This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size?
In the referenced post, we see that the bootstrap approach does not control type-1 ...
- 11
1
vote
0
answers
10
views
How to analyse serial brain sections probed for different proteins (ttests and potenial issue of independance)
Generic scenario: Brains have been collected from two different populations. They then have been cut into thin sections (serially) for the purpose of looking at the expression of proteins in specific ...
- 11
1
vote
0
answers
31
views
Distribution-free prediction intervals in linear regression
I've found some literature on the subject, but it is rather difficult to read. I am wondering if the following simplified method makes sense. My question is what part is correct in this methodology, ...
0
votes
1
answer
34
views
"SMOTE makes the assumption that the instance between a positive class instance and its nearest neighbors is also positive"
I am trying to get my head around this assertion by Liu, Y. et al (2011 pp. 7) about SMOTE oversampling technique that:
because SMOTE makes the assumption that the instance between a positive class ...
- 231
1
vote
1
answer
58
views
How to resample members from the population who didn't respond after the survey?
I have used stratified random sampling on population to generate the sample.
Now the issue is if after the survey is conducted some of the members in the sample ...
- 63
0
votes
0
answers
35
views
Bootstrapping with dependent data
I am trying to construct an example of the problems that arise when the Efron bootstrap is applied to dependent data. I have the following hypothetical time series data set: $\{100, 101, 102, 103, 104,...
- 141
0
votes
0
answers
66
views
Bootstrap residual resampling in R
Since I am quite new to this topic I have a question regarding bootstrap techniques in R.
I should generate a 10,000 bootstrapped time series by drawing with replacement from the residuals.
This ...
- 1
0
votes
0
answers
46
views
wild bootstrap of regression models
I am trying to show the differences between the residual, wild and pairs bootstrap on a regression model in R. I understand the differences in how they are calculated but I would like to know the main ...
0
votes
0
answers
32
views
Sampling Puzzle
There is a bag with N = 50 balls. Among which M = 10 balls are red, and N-M = 40 balls are blue. Further, say the red balls are numbered among themselves from 1 to 10, and the blue balls are numbered ...
0
votes
0
answers
20
views
Probability of failure of Uniform Sampling [duplicate]
Say I have a bag with 10 numbered balls, and I pick one ball at each time step and then put it back in the bag. Since each ball is equally likely, therefore the current situation represents a uniform ...
0
votes
0
answers
62
views
Bootstrap to Statistically Compare Accuracy of Different Approaches
I am currently dealing with a multi-class classification problem. I have two different approaches (in terms of feature engineering) to this problem. Intuitively, the result is obvious. However, I want ...
- 63
20
votes
3
answers
1k
views
Algorithm for sampling fixed number of samples from a finite population
I'm looking for an algorithm that would do the following:
Imagine that you need to sample uniformly at random and without replacement $k$ elements from a pool of $n$ elements. The catch is that $n$ is ...
Tim♦
- 121k
0
votes
0
answers
24
views
Comparing two uneven datasets, both of which were sampled in different ways
Dataset X is small (n=100). It was sampled based on a specific variable. Say, it took data based on 3 countries. Something like this:
...
- 111
1
vote
1
answer
51
views
Are you always supposed to evaluate the performance of regression models?
I'm a bit confused. I am doing an analysis where there are about 70 observations of my dependent variable. I'm planning to do a multiple linear regression or multivariate logistic regression to see ...
- 11
1
vote
0
answers
45
views
Different ways of resampling in OLS bootstrap
I have a constrained OLS model of form:
$$ y = X\beta + \epsilon
$$
After fitting, it turns out that there is heteroskedasticity present in residuals. Let's say I want to estimate the parameter ...
- 53
0
votes
0
answers
7
views
Sample Selection within motion planning data
The target of this study is to attempt to learn behavior of an unknown algorithm from raw data.
The environment in use is a 2D motion planning environment. We assume the algorithm behaves similarly to ...
- 101
0
votes
0
answers
8
views
Constructing a population random sample from subpopulations random samples of equal size
I have a very basic question about random sampling.
Consider the following:
Population
Population size
Sample size
Male
1200
200
Female
800
200
Where each sample for the population partitions have ...
0
votes
1
answer
33
views
glm result difference between multiple number of bootstrap simulations vs. one simulation based on mulstiple boostrap measurements
I am performing 1000 bootstrap iterations on my data. For each iteration, there are 18 measurements included in the sample, and I would perform GLM after each iteration. So in total, I get 1000 GLM ...
0
votes
0
answers
52
views
Resample multi-variate data to uniform density
Problem statement
I have two multivariate data sets A and B. For each item in the A, I want choose one from the B to create a pair $(a,b)$. I find candidates from the second data set by the criterion $...
- 203
2
votes
1
answer
403
views
Is a "trimmed" simple random sample still a simple random sample?
Suppose I have a population of size N (N is large, say a million), and I take a simple random sample of size 100. Denote the units in the sample as $u_1$, ... $u_{...
- 63
5
votes
3
answers
352
views
Resampling small datasets - Issue of overcounting?
Hypothetical situation here, as techniques like bootstrapping often fail for very small datasets.
Taking bootstrapping as an example nonetheless. we can easily compute the number of possible bootstrap ...
- 1,179
1
vote
0
answers
31
views
Bootstrapping allows retraining across the different bootstrapped datasets? [closed]
I am training a model and I would use bootstrapping since my dataset is really really small. Hence, I bootstrap a dataset, I train on it and then I get some validation error and metrics on the unseen ...
0
votes
0
answers
10
views
Significance testing the difference in dimensionality between conditions
I would like to test the hypothesis that under two experimental conditions there is a difference in data dimensionality.
Measures of dimensionality are complicated statistics of the data; for example, ...
- 427
0
votes
0
answers
55
views
Bootstrapping example ISL - pages 194-195
I'm currently learning about bootstrapping using the book Introduction to Statistical Learning, and am struggling to understand what the point of using the boot ...
- 123
1
vote
2
answers
98
views
How does pseudoreplication differ from resampling
I wonder how pseudoreplication (https://en.wikipedia.org/wiki/Pseudoreplication) differs from resampling (which is simply resampling with replacement from a given sample).
On the one hand, resampling ...
- 197
1
vote
2
answers
51
views
What is the theoretical justification of testing for equal mean/distribution via resampling?
If we have two data sets $X_1,\ldots,X_m$ and $Y_1,\ldots,Y_n$, each i.i.d., and wanted to determine whether $\mathbb{E}[X_1] = \mathbb{E}[Y_1]$ or not using $\bar{X}_m - \bar{Y}_n = \hat{\Delta}_{m,n}...
- 1,173
0
votes
0
answers
10
views
Highest amount of variation with resampling
I am looking for the right direction to find methods to solve the following case:
Let's say that I have a sample of 1000 people which represents a real-world population. I am creating clusters/groups ...
- 1
1
vote
1
answer
158
views
How does Michaud Resampling improve Mean-Variance Optimization?
Michaud Resampling claims to reduce estimation error through the following process:
Step 1. Sample a mean vector and covariance matrix of returns from distribution of both
centered at the original (...
- 13
6
votes
1
answer
461
views
How to sample $n$ observations from a multinomial distribution using binomial (or poisson) sampling?
Context
I have $n$ observations which I'd like to sample with replacement for the purpose of bootstrap. A way to think about it is that we have a multinomial distribution with $n$ classes and that we'...
- 20.4k
2
votes
1
answer
52
views
Can I use a permutation test to test the null hypothesis ''The difference between two groups is X''?
From what I read on permutation test, the null hypothesis is usually that there is no difference between the two groups.
I want to test if the difference between the mean of the two groups is $\theta$ ...
- 21
3
votes
0
answers
374
views
Resampling classes across weighted source distributions
I am sure this is a common problem, but googling only yielded false positives. I probably did not know what terms to search for. So here we go:
I have $n$ classes from $m$ different sources. Each ...
- 199
0
votes
1
answer
86
views
What statistical tests use resampling?
I have heard, that resampling techniques like bootstrapping are part of how many non-parametric statistical tests were developed/derived, but after going through the maths behind a few I have seen ...
- 113
1
vote
1
answer
243
views
Why does the best model in the training set have the worst test result?
I have trained eight models using 10-fold cross-validation, and evaluated the models by using resampling technique as described here. The result shows that SVM with sigmoid kernel (SVM-s) and random ...
- 13
0
votes
0
answers
82
views
Bootstrapped mean always almost identical to sample mean?
I'm running a simple bootstrapping experiment with the following code
...
1
vote
1
answer
232
views
Is my understanding of Random Forest algorithm with bootstrapping correct?
I want to know whether my understanding of RF algorithm is correct when using bootstrapping.
So, let's say I have a dataset of 100 observations. That dataset is then split into a 75/25 split of train ...
- 47