Questions tagged [resampling]

Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.

161 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
0 answers
50 views

Implications of Different Bootstrap Procedures for Estimating Difference in Means

I'm having trouble understanding the difference between two bootstrap procedures to evaluate the difference in means between two samples. As an example, consider the following scenario: My goal is to ...
Tomas Bencomo's user avatar
5 votes
0 answers
658 views

Block bootstrap for dependent data with unequal sampling intervals?

I have data from a natural archive (lake sediment). For various reasons it is usually impossible to sample the archive equally in time, and we end up with a time series where essentially we have ...
Gavin Simpson's user avatar
4 votes
0 answers
94 views

Accounting for repeated measures by resampling data and averaging parameter estimates misses the mark, but why?

Let's say I am looking at how unicorn herd size changes with food quantity. Unicorn herd sizes were surveyed at seven localities over the course of twelve months. Food quantity was assessed monthly, ...
Marco Plebani's user avatar
4 votes
1 answer
297 views

Ratio of two weighted sum of Bernoulli random variables

Given a vector $X=(x_1,\dots x_N) \in R_+^N$ and $N$ i.i.d Bernoulli random variable $A_i$ with $$P(A_i= 1)=p = 1 -P(A_i = 0)$$ First we define the random variable $T$ as $$ T = \frac{\sum_{i}x_i * ...
Vassily's user avatar
  • 141
4 votes
0 answers
249 views

When to use ordinary, balanced, antithetic, or permutation resampling for bootstrap?

I am using boot and would appreciate any explanation as to when using each of these resampling methods would be recommended in practice. I have come across Do who says that: Simulation results ...
Krantz's user avatar
  • 585
4 votes
0 answers
162 views

Optimal block size for spatial bootstrapping

Take a regression model: $$ y_s = X_s\beta + \epsilon $$ Where $E[\epsilon|X] = 0$, but $cor(\epsilon_s, \epsilon_{near}) > cor(\epsilon_s, \epsilon_{far})> 0$. In other words, $X$ is ...
generic_user's user avatar
  • 13.3k
4 votes
0 answers
1k views

How to calculate a p-value under the null from a bootstrap?

In another thread, I asked how to obtain a confidence interval of the difference in probability of success between two groups from a logit model and @Weihuang Wong showed me how to do this with a ...
bee guy's user avatar
  • 1,019
4 votes
0 answers
822 views

Bootstrapped prediction intervals: Quantile, median, SE...?

I am trying to construct prediction intervals for a non linear model via Boostrap. What I do is to apply the usual bootstrap procedure, here you have pseudo-code for 1000 iterations: ...
adrian1121's user avatar
  • 1,116
4 votes
0 answers
248 views

Creating a bootstrap null (H0) distribution. Investigating alternatives

I'm investigating the possibility of creating a non parametric bootstrap null distribution for hypothesis testing in multivariable regression analysis. The null hypothesis is based on the absence of ...
Bakaburg's user avatar
  • 2,907
4 votes
0 answers
4k views

Bootstrapping time series data: Circular block bootstrap

I have some very basic questions on circular block bootstrap applied to time series (dependent data). Let's suppose, I have a time series data like the one below. I know it's non stationary, but for ...
forecaster's user avatar
  • 8,133
4 votes
0 answers
2k views

Sampling and resampling data in R

My problem this time concerns sampling size-related errors, resample-based confidence intervals and a possible way to control for this error. My dataset consists of 50 measurements of certain cranial ...
Fedja Blagojevic's user avatar
4 votes
0 answers
873 views

Why might the mean of a bootstrapped distribution not equal the original summary statistic?

Background: I have n samples and their average. The mean of this empirical bootstrapped distribution seems quite different form the average of my original sample. My original average for the n samples ...
Alex Charl's user avatar
3 votes
0 answers
51 views

Formal testing for differences in kurtosis between two samples when bootstrapping suggests a difference

My question is similar to Testing difference in kurtosis between two samples where a comment suggested Unless you are looking for an enormous difference in kurtosis, it's unlikely any physically ...
StrongBad's user avatar
  • 280
3 votes
0 answers
42 views

Do autoregressive coefficients obtained with the Durbin-Levinson algorithm, the Yule-Walker system of equations, and OLS coincide?

I am applying the sieve bootstrap for time series introduced by Kress (1988), which requires the estimation of autoregressive models with the Durbin-Levinson algorithm (generalized to the multivariate ...
Mr Frog's user avatar
  • 339
3 votes
0 answers
245 views

Why do Posterior Sampling when we can Bootstrap?

I have just taken a stats course where half of the content was to do with sampling from posterior distributions. However, it was not clear why we were doing this when we could simply perform bootstrap ...
Cameron Chandler's user avatar
3 votes
0 answers
152 views

Bootstrap Resampling vs Repeated K-Fold Cross-Validation for Confidence Intervals

Why is bootstrap resampling with replacement used to construct confidence intervals over repeated K-fold cross-validation? Isn't it valid to use 10-fold CV repeated 10 times, where we garner 100 data ...
retrograde's user avatar
3 votes
0 answers
133 views

Are statistics calculated from Bootstrapped samples independent?

I was wondering if someone could point me to some reference stating the (in)dependence properties of statistics calculated from bootstrapped samples. In other words, are bootstrapped statistics (mean,...
xxxmsi's user avatar
  • 81
3 votes
0 answers
46 views

Hypothesis testing with composite populations of unequal size and variance

I am trying to test whether two populations have different means. Let's call the populations "Glaciated" and "Unglaciated." Each population comprises data collected at a number of rivers (9 for ...
Duncan Ocel's user avatar
3 votes
0 answers
78 views

Can we sample from a set of biased samples to get unbiased samples?

This is a follow-up question to How to uniformly sample vertices from a large graph with given distance from a fixed vertex?. Suppose I took a set $B$ of $n$ samples from a large but finite universe $...
fuz's user avatar
  • 221
3 votes
0 answers
557 views

Bootstrap Resampling Vs KDE Resampling

Let $\xi\in\mathbb{R}^{m}$ be a random vector with joint desity function $f$, and let $\widehat{\xi}_{1},\ldots,\widehat{\xi}_{N}$ be a sample of $\xi$. We have that the kernel density estimator (...
Diego Fonseca's user avatar
3 votes
2 answers
812 views

How to choose resample size when drawing without replacement?

Say I have some second-order statistic $m(x)$ where $x$ is a data vector of length $n$. Let's also assume that the limiting distribution of $x$ is gaussian-ish, but generally unknown, so that the ...
pretzlstyle's user avatar
3 votes
0 answers
344 views

want to calculate significance of pairwise sharing between lists. Standard hypergeometric test seems inappropriate

I want to calculate if the percentage of sharing of strings (genes in this case) between lists is significantly more than expected by chance for multiple pairwise comparisons between lists of strings. ...
user964689's user avatar
3 votes
0 answers
140 views

Estimation on evolving distribution with small updates

I have a set $X$ of $10^6$ elements and a time series of probability distributions $\mu_1,\mu_2,\ldots$ on $X$. I want to estimate the expected value of a function $f$ over each $\mu_t$. It is easy to ...
Chris Culter's user avatar
3 votes
0 answers
131 views

Is there a resampling method that blends subsampling with the bootstrap?

I apologize if this is an inappropriate question. I thought of it in class the other day, and I couldn't find a specific answer in my textbooks. I am familiar with the two basic techniques for ...
Cat C.'s user avatar
  • 31
3 votes
1 answer
298 views

Selecting uncorrelated samples from a set of bulk data that contains correlated and dependent samples

i have a set of data that is generated by expensive computational model evaluations, on a total data set of 10000 samples in 40 dimensions. This sample data set is composed of different data sets, ...
Sarmes's user avatar
  • 151
3 votes
0 answers
242 views

Bootstrapping fits to a small sample

I have a sample of experimentally measured survival times that are quite noisy and vary stochastically. The survival probability of these events (number of events with a survival time of t or more) is ...
avocado's user avatar
  • 31
3 votes
0 answers
402 views

Resampling or Basic Simulation and confidence intervals

I have a population of sales that might be won or lost. I know the rate that they are won from historical data. This case 30% of them historically win. To figure out how much money I will be making ...
DanTheMan's user avatar
  • 307
2 votes
2 answers
38 views

Should we (under- or over-) sample when training a ML model, if we care about edge cases?

I know this question has been somehow reiterated in multiple ways, but I have not yet found an answer that would explicitly apply to my case. I wish to train a classification model to predict who is ...
BloodthirstyPlatypus's user avatar
2 votes
0 answers
288 views

Train-Test Split with nested groups and multiple balancing factors

I have a large (~15,000) sample of data from individuals nested within families (with about half the data points sharing a family). I want to split the sample in to a training and test set so I can ...
mrpeverill's user avatar
2 votes
0 answers
571 views

For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?

This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size? In the referenced post, we see that the bootstrap approach does not control type-1 ...
3487564's user avatar
  • 21
2 votes
0 answers
24 views

Improving samples from a distribution

I have a long-standing problem regarding generating samples from a desired distribution, $p(x)$. I know the analytic form of $p(x)$. I have a mechanism that should draw samples from $p(x)$, and does ...
lux's user avatar
  • 153
2 votes
0 answers
125 views

How to determine sample weights, when the prevalence in the population is based on percentages

The central question is: How to do I determine the weights, when prevalence in the population is expressed as percentages? I have posted this related question on Stack Overflow: According to the ...
Tom's user avatar
  • 448
2 votes
0 answers
2k views

Interview question: effect of Gaussian resampling of the features over the predictors' confidence interval

In an interview question I was asked what would be the effect of adding a Gaussian noise to the features $\mathbf X\in\mathbb R^{n\times k}$ over the confidence interval of the parameters $\mathbf\...
marco's user avatar
  • 215
2 votes
1 answer
176 views

Statistical test for single sample vs empirical distribution?

I have a sort of strange problem, where my sample is difficult to obtain, but the population distribution is easy to obtain. Specifically, I have obtained a single observation. I would like to know ...
Him's user avatar
  • 2,247
2 votes
0 answers
106 views

Estimating Unique Population Sizes

I am trying to estimate the total number of unique visitors to my website. My website contains 500 different pages. I am able to: find the number of unique users to a given page find the number of ...
Jai Pancholi's user avatar
2 votes
0 answers
1k views

How to handle missing values in a bootstrap resampling distribution?

I am creating a function that calculates point and interval estimates for a statistic of interest. I get the interval estimate through bootstrapping. However, it occasionally occurs that the statistic ...
Jeffrey Girard's user avatar
2 votes
0 answers
30 views

Downsampling, AUROC and accuracy equal

I am using downsampling to create perfectly balanced classes in my target feature. I have found that accuracy is exactly equal to the AUROC score. I was thinking that this is because I've used ...
Floor's user avatar
  • 21
2 votes
0 answers
159 views

Correlating Two Time Series with Gaps in Data and of Different Lengths

I am attempting to correlate the time series from 4 separate tilt monitors that sample every 5 minutes. The time series have slightly different base times and end times, and the resulting arrays are ...
geojasm's user avatar
  • 21
2 votes
3 answers
1k views

How should I resample the training and testing set with imbalanced data whilst having meaningful performance metrics?

I have an imbalanced dataset of approx. 200 positive and 800 negative examples. I run nested cross-validation where i=5 and j=5; (i is inner and j is outer). The cross-validation procedure isn't the ...
Aalawlx's user avatar
  • 305
2 votes
0 answers
242 views

Practical questions about cluster bootstrap confidence intervals

I want to estimate the accuracy of a machine learning model. I have an independent test set with a vector of trusted labels and a corresponding vector of model-based predictions. If I assume the test ...
Jeffrey Girard's user avatar
2 votes
0 answers
391 views

Monte Carlo testing: number of required permutations

I want to perform a statistical hypothesis test, however I don't know the exact distribution of my test statistic under $H_0$. Therefore, I need to calculate a Monte Carlo estimate $\hat{p}$ of the ...
Andreas's user avatar
  • 55
2 votes
0 answers
167 views

"ANOVA on a non-random non-Normal sample from a Normal Population"

How can I run ANOVA or tests for statistical significance on a bi-modal sample that came from a normal population? Context: I was tasked with running an ANOVA to see if genotypes (treatments / ...
A Duv's user avatar
  • 121
2 votes
1 answer
1k views

Using SMOTE with grouped, paneled, or categorical data?

Let's say that I am building a classifier on imbalanced data. A sample of the data set looks like: ...
user1566200's user avatar
  • 1,047
2 votes
0 answers
3k views

logic of resamples function in caret R package

The caret R package includes functions to compare models via their resampling distributions. Specifically, it prescribes fitting multiple models using the same resampling profiles (i.e. same versions ...
user156902's user avatar
2 votes
0 answers
280 views

Computing Monte Carlo Error: Particle Filters

I want to ask a question about the Monte Carlo error of a particle filter. Assume we have information of our of the process of our true states, $x_t \forall t$ and hence, we generate our data $y_t$. (...
tintinthong's user avatar
2 votes
0 answers
141 views

Can Resampling be used for estimation and goodness of fit test?

I am trying to compare my data with empirical distributions. But I don't have enough data to cut them to estimation data and validation data. I am trying a resampling approach and would like to see if ...
Saad Touhbi's user avatar
2 votes
0 answers
653 views

Bootstrapping and resampling sizes

This bootstrap primer from Stanford says How many observations should we resample? A good suggestion is the original sample size. While I get that this advice might be referring specifically to ...
user1205901 - Слава Україні's user avatar
2 votes
0 answers
869 views

multiple test correction procedure for survey data

I clustered ~ 20,000 students into three groups. Now, I also have answers from some of those students to ~ 100 survey questions. I would like to compare the differences between the clusters regarding ...
Vitomir Kovanovic's user avatar
2 votes
0 answers
880 views

What does the class_weight function in keras do during training of Neural Networks?

I have a heavily imbalanced dataset with 170 columns and 2 million rows, there are also missing data in the set. As practiced, I drop all the null values, normalized the data using min-max method and ...
Gerard's user avatar
  • 181
2 votes
0 answers
90 views

What's the relation between the covariance estimation of the full dataset VERSUS the covariance estimations of the resampled/bootstrapped set

Consider the following Matlab code: ...
iwein's user avatar
  • 21