Questions tagged [resampling]

Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.

Filter by
Sorted by
Tagged with
1 vote
1 answer
33 views

Resampling a timeseries

I have a list of stock returns (say computed from the historical data) and would like to resample the historical return distribution. Naively doing bootstrapping means the samples are iid. I'm ...
  • 594
1 vote
1 answer
18 views

How to generate data or sample data from a summary tables?

I have a dataset like this: I only have summary tables of the "loneliness score" for overall, for each sex, each age group, and sex * age interactions. Besides this, I have 23 items that ...
  • 11
2 votes
0 answers
13 views

How to calculate 2-D confidence intervals with the boot package in R? [closed]

I'm making a scatter plot of two statistics X(e) and Y(e) for various values of scalar parameter e. The sampling distribution of both X and Y is not normally distributed. Now I want to calculate a 2-D ...
  • 121
1 vote
0 answers
59 views

Bootstrap/resampling and bayesian regression, solving dependency problems

Let's say we have two experiment replications R1 and R2, (with some modifications in R2), deployed for intersecting (but not fully the same, let's say >50% but <80%) populations of participants ...
1 vote
1 answer
49 views

What are the minimal sample size requirements for cross-validation or bootstrapping?

I hope it makes sense to even ask these questions, but I'm wondering how can I evaluate the "validation" procedures that my data allow me to perform (i.e. cross-validation or bootstrap: ...
  • 507
0 votes
1 answer
28 views

How many independet test-train splits (with independent training) should I perform?

I recently read some literature by Bagnall et al https://arxiv.org/pdf/1602.01711.pdf "The Great Time Series Classification Bake Off". If I understand them correctly, they advise to perform ...
  • 197
0 votes
0 answers
43 views

Confidence interval and/or sampling distribution in double / two-phase sampling for stratification

Question: How to calculate a 95% confidence interval for estimated population mean in double / two-phase sampling for stratification where a simple random sample is taken at each phase? Consider the ...
  • 1
0 votes
1 answer
36 views

Resources to understand the bootstrap

I am looking for any recommendations on resources to learn more about bootstrapping from a theoretical and rigorous perspective in terms of bounds/guarantees/etc. Any books or papers would be very ...
0 votes
0 answers
36 views

Can balancing of the majority and minority classes (RUS/SMOTE/...) improve AUC of a binary classifier?

Given that AUC is a threshold independent measure, can undersampling or oversampling of the majority/minority class during training improve the performance of a binary classifier? In my experience ...
1 vote
1 answer
33 views

Re-sampling at defined percentage of the population

Background: I have data that contains the severity scores of the pathology findings of tissues from different subjects with 100% sampling of the organ (let's call this dataset population, which ...
1 vote
1 answer
16 views

Sample size determination for finding the "top 100" largest X among multiple datasets

My data sources Source A gives me a ranking of the top 1.000 largest X (say, companies). Source B provides a similar ranking, comprising 7.000 X, with some differences to Source A (different values, ...
  • 111
2 votes
0 answers
61 views

Train-Test Split with nested groups and multiple balancing factors

I have a large (~15,000) sample of data from individuals nested within families (with about half the data points sharing a family). I want to split the sample in to a training and test set so I can ...
0 votes
1 answer
33 views

In which specific situations is minority class oversampling useful? [duplicate]

I understand that, in the context of a binary classification problem, downsampling the majority class is a useful strategy to come up with a smaller, computationally friendly dataset. Using this ...
1 vote
2 answers
51 views

Sub-sampling a dataset to a different target distribution without replacement - bias correction?

Suppose i have dataset $X$ and $Y$, and i want to sub-sample from $X$ so that the resulting (sample) distribution is as close to that of $Y$ as possible. One thing i can do is subsample with ...
  • 1,232
0 votes
0 answers
14 views

Approximately Unbiased P-value vs Bootstrap Probability: which one should i choose?

Some references first: How is approximately unbiased bootstrap better than a regular bootstrap with regards to hierarchical clustering? Suzuki et al. 2004 https://www.researchgate.net/publication/...
  • 1
1 vote
0 answers
108 views

For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?

This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size? In the referenced post, we see that the bootstrap approach does not control type-1 ...
  • 11
1 vote
0 answers
10 views

How to analyse serial brain sections probed for different proteins (ttests and potenial issue of independance)

Generic scenario: Brains have been collected from two different populations. They then have been cut into thin sections (serially) for the purpose of looking at the expression of proteins in specific ...
1 vote
0 answers
31 views

Distribution-free prediction intervals in linear regression

I've found some literature on the subject, but it is rather difficult to read. I am wondering if the following simplified method makes sense. My question is what part is correct in this methodology, ...
0 votes
1 answer
34 views

"SMOTE makes the assumption that the instance between a positive class instance and its nearest neighbors is also positive"

I am trying to get my head around this assertion by Liu, Y. et al (2011 pp. 7) about SMOTE oversampling technique that: because SMOTE makes the assumption that the instance between a positive class ...
  • 231
1 vote
1 answer
58 views

How to resample members from the population who didn't respond after the survey?

I have used stratified random sampling on population to generate the sample. Now the issue is if after the survey is conducted some of the members in the sample ...
  • 63
0 votes
0 answers
35 views

Bootstrapping with dependent data

I am trying to construct an example of the problems that arise when the Efron bootstrap is applied to dependent data. I have the following hypothetical time series data set: $\{100, 101, 102, 103, 104,...
  • 141
0 votes
0 answers
66 views

Bootstrap residual resampling in R

Since I am quite new to this topic I have a question regarding bootstrap techniques in R. I should generate a 10,000 bootstrapped time series by drawing with replacement from the residuals. This ...
  • 1
0 votes
0 answers
46 views

wild bootstrap of regression models

I am trying to show the differences between the residual, wild and pairs bootstrap on a regression model in R. I understand the differences in how they are calculated but I would like to know the main ...
0 votes
0 answers
32 views

Sampling Puzzle

There is a bag with N = 50 balls. Among which M = 10 balls are red, and N-M = 40 balls are blue. Further, say the red balls are numbered among themselves from 1 to 10, and the blue balls are numbered ...
0 votes
0 answers
20 views

Probability of failure of Uniform Sampling [duplicate]

Say I have a bag with 10 numbered balls, and I pick one ball at each time step and then put it back in the bag. Since each ball is equally likely, therefore the current situation represents a uniform ...
0 votes
0 answers
62 views

Bootstrap to Statistically Compare Accuracy of Different Approaches

I am currently dealing with a multi-class classification problem. I have two different approaches (in terms of feature engineering) to this problem. Intuitively, the result is obvious. However, I want ...
  • 63
20 votes
3 answers
1k views

Algorithm for sampling fixed number of samples from a finite population

I'm looking for an algorithm that would do the following: Imagine that you need to sample uniformly at random and without replacement $k$ elements from a pool of $n$ elements. The catch is that $n$ is ...
  • 121k
0 votes
0 answers
24 views

Comparing two uneven datasets, both of which were sampled in different ways

Dataset X is small (n=100). It was sampled based on a specific variable. Say, it took data based on 3 countries. Something like this: ...
  • 111
1 vote
1 answer
51 views

Are you always supposed to evaluate the performance of regression models?

I'm a bit confused. I am doing an analysis where there are about 70 observations of my dependent variable. I'm planning to do a multiple linear regression or multivariate logistic regression to see ...
  • 11
1 vote
0 answers
45 views

Different ways of resampling in OLS bootstrap

I have a constrained OLS model of form: $$ y = X\beta + \epsilon $$ After fitting, it turns out that there is heteroskedasticity present in residuals. Let's say I want to estimate the parameter ...
  • 53
0 votes
0 answers
7 views

Sample Selection within motion planning data

The target of this study is to attempt to learn behavior of an unknown algorithm from raw data. The environment in use is a 2D motion planning environment. We assume the algorithm behaves similarly to ...
  • 101
0 votes
0 answers
8 views

Constructing a population random sample from subpopulations random samples of equal size

I have a very basic question about random sampling. Consider the following: Population Population size Sample size Male 1200 200 Female 800 200 Where each sample for the population partitions have ...
0 votes
1 answer
33 views

glm result difference between multiple number of bootstrap simulations vs. one simulation based on mulstiple boostrap measurements

I am performing 1000 bootstrap iterations on my data. For each iteration, there are 18 measurements included in the sample, and I would perform GLM after each iteration. So in total, I get 1000 GLM ...
0 votes
0 answers
52 views

Resample multi-variate data to uniform density

Problem statement I have two multivariate data sets A and B. For each item in the A, I want choose one from the B to create a pair $(a,b)$. I find candidates from the second data set by the criterion $...
  • 203
2 votes
1 answer
403 views

Is a "trimmed" simple random sample still a simple random sample?

Suppose I have a population of size N (N is large, say a million), and I take a simple random sample of size 100. Denote the units in the sample as $u_1$, ... $u_{...
5 votes
3 answers
352 views

Resampling small datasets - Issue of overcounting?

Hypothetical situation here, as techniques like bootstrapping often fail for very small datasets. Taking bootstrapping as an example nonetheless. we can easily compute the number of possible bootstrap ...
  • 1,179
1 vote
0 answers
31 views

Bootstrapping allows retraining across the different bootstrapped datasets? [closed]

I am training a model and I would use bootstrapping since my dataset is really really small. Hence, I bootstrap a dataset, I train on it and then I get some validation error and metrics on the unseen ...
0 votes
0 answers
10 views

Significance testing the difference in dimensionality between conditions

I would like to test the hypothesis that under two experimental conditions there is a difference in data dimensionality. Measures of dimensionality are complicated statistics of the data; for example, ...
  • 427
0 votes
0 answers
55 views

Bootstrapping example ISL - pages 194-195

I'm currently learning about bootstrapping using the book Introduction to Statistical Learning, and am struggling to understand what the point of using the boot ...
  • 123
1 vote
2 answers
98 views

How does pseudoreplication differ from resampling

I wonder how pseudoreplication (https://en.wikipedia.org/wiki/Pseudoreplication) differs from resampling (which is simply resampling with replacement from a given sample). On the one hand, resampling ...
  • 197
1 vote
2 answers
51 views

What is the theoretical justification of testing for equal mean/distribution via resampling?

If we have two data sets $X_1,\ldots,X_m$ and $Y_1,\ldots,Y_n$, each i.i.d., and wanted to determine whether $\mathbb{E}[X_1] = \mathbb{E}[Y_1]$ or not using $\bar{X}_m - \bar{Y}_n = \hat{\Delta}_{m,n}...
  • 1,173
0 votes
0 answers
10 views

Highest amount of variation with resampling

I am looking for the right direction to find methods to solve the following case: Let's say that I have a sample of 1000 people which represents a real-world population. I am creating clusters/groups ...
  • 1
1 vote
1 answer
158 views

How does Michaud Resampling improve Mean-Variance Optimization?

Michaud Resampling claims to reduce estimation error through the following process: Step 1. Sample a mean vector and covariance matrix of returns from distribution of both centered at the original (...
  • 13
6 votes
1 answer
461 views

How to sample $n$ observations from a multinomial distribution using binomial (or poisson) sampling?

Context I have $n$ observations which I'd like to sample with replacement for the purpose of bootstrap. A way to think about it is that we have a multinomial distribution with $n$ classes and that we'...
  • 20.4k
2 votes
1 answer
52 views

Can I use a permutation test to test the null hypothesis ''The difference between two groups is X''?

From what I read on permutation test, the null hypothesis is usually that there is no difference between the two groups. I want to test if the difference between the mean of the two groups is $\theta$ ...
3 votes
0 answers
374 views

Resampling classes across weighted source distributions

I am sure this is a common problem, but googling only yielded false positives. I probably did not know what terms to search for. So here we go: I have $n$ classes from $m$ different sources. Each ...
0 votes
1 answer
86 views

What statistical tests use resampling?

I have heard, that resampling techniques like bootstrapping are part of how many non-parametric statistical tests were developed/derived, but after going through the maths behind a few I have seen ...
  • 113
1 vote
1 answer
243 views

Why does the best model in the training set have the worst test result?

I have trained eight models using 10-fold cross-validation, and evaluated the models by using resampling technique as described here. The result shows that SVM with sigmoid kernel (SVM-s) and random ...
  • 13
0 votes
0 answers
82 views

Bootstrapped mean always almost identical to sample mean?

I'm running a simple bootstrapping experiment with the following code ...
1 vote
1 answer
232 views

Is my understanding of Random Forest algorithm with bootstrapping correct?

I want to know whether my understanding of RF algorithm is correct when using bootstrapping. So, let's say I have a dataset of 100 observations. That dataset is then split into a 75/25 split of train ...

1
2 3 4 5
8