Questions tagged [resampling]
Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.
161
questions with no upvoted or accepted answers
6
votes
0
answers
50
views
Implications of Different Bootstrap Procedures for Estimating Difference in Means
I'm having trouble understanding the difference between two bootstrap procedures to evaluate the difference in means between two samples. As an example, consider the following scenario:
My goal is to ...
5
votes
0
answers
658
views
Block bootstrap for dependent data with unequal sampling intervals?
I have data from a natural archive (lake sediment). For various reasons it is usually impossible to sample the archive equally in time, and we end up with a time series where essentially we have ...
4
votes
0
answers
94
views
Accounting for repeated measures by resampling data and averaging parameter estimates misses the mark, but why?
Let's say I am looking at how unicorn herd size changes with food quantity. Unicorn herd sizes were surveyed at seven localities over the course of twelve months. Food quantity was assessed monthly, ...
4
votes
1
answer
297
views
Ratio of two weighted sum of Bernoulli random variables
Given a vector $X=(x_1,\dots x_N) \in R_+^N$ and $N$ i.i.d Bernoulli random variable $A_i$ with $$P(A_i= 1)=p = 1 -P(A_i = 0)$$
First we define the random variable $T$ as
$$
T = \frac{\sum_{i}x_i * ...
4
votes
0
answers
249
views
When to use ordinary, balanced, antithetic, or permutation resampling for bootstrap?
I am using boot and would appreciate any explanation as to when using each of these resampling methods would be recommended in practice.
I have come across Do who says that:
Simulation results ...
4
votes
0
answers
162
views
Optimal block size for spatial bootstrapping
Take a regression model:
$$
y_s = X_s\beta + \epsilon
$$
Where $E[\epsilon|X] = 0$, but $cor(\epsilon_s, \epsilon_{near}) > cor(\epsilon_s, \epsilon_{far})> 0$.
In other words, $X$ is ...
4
votes
0
answers
1k
views
How to calculate a p-value under the null from a bootstrap?
In another thread, I asked how to obtain a confidence interval of the difference in probability of success between two groups from a logit model and @Weihuang Wong showed me how to do this with a ...
4
votes
0
answers
822
views
Bootstrapped prediction intervals: Quantile, median, SE...?
I am trying to construct prediction intervals for a non linear model via Boostrap. What I do is to apply the usual bootstrap procedure, here you have pseudo-code for 1000 iterations:
...
4
votes
0
answers
248
views
Creating a bootstrap null (H0) distribution. Investigating alternatives
I'm investigating the possibility of creating a non parametric bootstrap null distribution for hypothesis testing in multivariable regression analysis.
The null hypothesis is based on the absence of ...
4
votes
0
answers
4k
views
Bootstrapping time series data: Circular block bootstrap
I have some very basic questions on circular block bootstrap applied to time series (dependent data).
Let's suppose, I have a time series data like the one below. I know it's non stationary, but for ...
4
votes
0
answers
2k
views
Sampling and resampling data in R
My problem this time concerns sampling size-related errors, resample-based confidence intervals and a possible way to control for this error. My dataset consists of 50 measurements of certain cranial ...
4
votes
0
answers
873
views
Why might the mean of a bootstrapped distribution not equal the original summary statistic?
Background: I have n samples and their average. The mean of this empirical bootstrapped distribution seems quite different form the average of my original sample. My original average for the n samples ...
3
votes
0
answers
51
views
Formal testing for differences in kurtosis between two samples when bootstrapping suggests a difference
My question is similar to Testing difference in kurtosis between two samples where a comment suggested
Unless you are looking for an enormous difference in kurtosis, it's
unlikely any physically ...
3
votes
0
answers
42
views
Do autoregressive coefficients obtained with the Durbin-Levinson algorithm, the Yule-Walker system of equations, and OLS coincide?
I am applying the sieve bootstrap for time series introduced by Kress (1988), which requires the estimation of autoregressive models with the Durbin-Levinson algorithm (generalized to the multivariate ...
3
votes
0
answers
245
views
Why do Posterior Sampling when we can Bootstrap?
I have just taken a stats course where half of the content was to do with sampling from posterior distributions. However, it was not clear why we were doing this when we could simply perform bootstrap ...
3
votes
0
answers
152
views
Bootstrap Resampling vs Repeated K-Fold Cross-Validation for Confidence Intervals
Why is bootstrap resampling with replacement used to construct confidence intervals over repeated K-fold cross-validation? Isn't it valid to use 10-fold CV repeated 10 times, where we garner 100 data ...
3
votes
0
answers
133
views
Are statistics calculated from Bootstrapped samples independent?
I was wondering if someone could point me to some reference stating the (in)dependence properties of statistics calculated from bootstrapped samples. In other words, are bootstrapped statistics (mean,...
3
votes
0
answers
46
views
Hypothesis testing with composite populations of unequal size and variance
I am trying to test whether two populations have different means. Let's call the populations "Glaciated" and "Unglaciated." Each population comprises data collected at a number of rivers (9 for ...
3
votes
0
answers
78
views
Can we sample from a set of biased samples to get unbiased samples?
This is a follow-up question to How to uniformly sample vertices from a large graph with given distance from a fixed vertex?.
Suppose I took a set $B$ of $n$ samples from a large but finite universe $...
3
votes
0
answers
557
views
Bootstrap Resampling Vs KDE Resampling
Let $\xi\in\mathbb{R}^{m}$ be a random vector with joint desity function $f$, and let $\widehat{\xi}_{1},\ldots,\widehat{\xi}_{N}$ be a sample of $\xi$. We have that the kernel density estimator (...
3
votes
2
answers
812
views
How to choose resample size when drawing without replacement?
Say I have some second-order statistic $m(x)$ where $x$ is a data vector of length $n$. Let's also assume that the limiting distribution of $x$ is gaussian-ish, but generally unknown, so that the ...
3
votes
0
answers
344
views
want to calculate significance of pairwise sharing between lists. Standard hypergeometric test seems inappropriate
I want to calculate if the percentage of sharing of strings (genes in this case) between lists is significantly more than expected by chance for multiple pairwise comparisons between lists of strings. ...
3
votes
0
answers
140
views
Estimation on evolving distribution with small updates
I have a set $X$ of $10^6$ elements and a time series of probability distributions $\mu_1,\mu_2,\ldots$ on $X$. I want to estimate the expected value of a function $f$ over each $\mu_t$. It is easy to ...
3
votes
0
answers
131
views
Is there a resampling method that blends subsampling with the bootstrap?
I apologize if this is an inappropriate question. I thought of it in class the other day, and I couldn't find a specific answer in my textbooks.
I am familiar with the two basic techniques for ...
3
votes
1
answer
298
views
Selecting uncorrelated samples from a set of bulk data that contains correlated and dependent samples
i have a set of data that is generated by expensive computational model evaluations, on a total data set of 10000 samples in 40 dimensions. This sample data set is composed of different data sets, ...
3
votes
0
answers
242
views
Bootstrapping fits to a small sample
I have a sample of experimentally measured survival times that are quite noisy and vary stochastically. The survival probability of these events (number of events with a survival time of t or more) is ...
3
votes
0
answers
402
views
Resampling or Basic Simulation and confidence intervals
I have a population of sales that might be won or lost. I know the rate that they are won from historical data. This case 30% of them historically win. To figure out how much money I will be making ...
2
votes
2
answers
38
views
Should we (under- or over-) sample when training a ML model, if we care about edge cases?
I know this question has been somehow reiterated in multiple ways, but I have not yet found an answer that would explicitly apply to my case. I wish to train a classification model to predict who is ...
2
votes
0
answers
288
views
Train-Test Split with nested groups and multiple balancing factors
I have a large (~15,000) sample of data from individuals nested within families (with about half the data points sharing a family). I want to split the sample in to a training and test set so I can ...
2
votes
0
answers
571
views
For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?
This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size?
In the referenced post, we see that the bootstrap approach does not control type-1 ...
2
votes
0
answers
24
views
Improving samples from a distribution
I have a long-standing problem regarding generating samples from a desired distribution, $p(x)$. I know the analytic form of $p(x)$.
I have a mechanism that should draw samples from $p(x)$, and does ...
2
votes
0
answers
125
views
How to determine sample weights, when the prevalence in the population is based on percentages
The central question is: How to do I determine the weights, when prevalence in the population is expressed as percentages?
I have posted this related question on Stack Overflow: According to the ...
2
votes
0
answers
2k
views
Interview question: effect of Gaussian resampling of the features over the predictors' confidence interval
In an interview question I was asked what would be the effect of adding a Gaussian noise to the features $\mathbf X\in\mathbb R^{n\times k}$ over the confidence interval of the parameters $\mathbf\...
2
votes
1
answer
176
views
Statistical test for single sample vs empirical distribution?
I have a sort of strange problem, where my sample is difficult to obtain, but the population distribution is easy to obtain. Specifically, I have obtained a single observation. I would like to know ...
2
votes
0
answers
106
views
Estimating Unique Population Sizes
I am trying to estimate the total number of unique visitors to my website.
My website contains 500 different pages.
I am able to:
find the number of unique users to a given page
find the number of ...
2
votes
0
answers
1k
views
How to handle missing values in a bootstrap resampling distribution?
I am creating a function that calculates point and interval estimates for a statistic of interest. I get the interval estimate through bootstrapping. However, it occasionally occurs that the statistic ...
2
votes
0
answers
30
views
Downsampling, AUROC and accuracy equal
I am using downsampling to create perfectly balanced classes in my target feature. I have found that accuracy is exactly equal to the AUROC score. I was thinking that this is because I've used ...
2
votes
0
answers
159
views
Correlating Two Time Series with Gaps in Data and of Different Lengths
I am attempting to correlate the time series from 4 separate tilt monitors that sample every 5 minutes. The time series have slightly different base times and end times, and the resulting arrays are ...
2
votes
3
answers
1k
views
How should I resample the training and testing set with imbalanced data whilst having meaningful performance metrics?
I have an imbalanced dataset of approx. 200 positive and 800 negative examples.
I run nested cross-validation where i=5 and j=5; (i is inner and j is outer). The cross-validation procedure isn't the ...
2
votes
0
answers
242
views
Practical questions about cluster bootstrap confidence intervals
I want to estimate the accuracy of a machine learning model. I have an independent test set with a vector of trusted labels and a corresponding vector of model-based predictions. If I assume the test ...
2
votes
0
answers
391
views
Monte Carlo testing: number of required permutations
I want to perform a statistical hypothesis test, however I don't know the exact distribution of my test statistic under $H_0$. Therefore, I need to calculate a Monte Carlo estimate $\hat{p}$ of the ...
2
votes
0
answers
167
views
"ANOVA on a non-random non-Normal sample from a Normal Population"
How can I run ANOVA or tests for statistical significance on a bi-modal sample that came from a normal population?
Context:
I was tasked with running an ANOVA to see if genotypes (treatments / ...
2
votes
1
answer
1k
views
Using SMOTE with grouped, paneled, or categorical data?
Let's say that I am building a classifier on imbalanced data. A sample of the data set looks like:
...
2
votes
0
answers
3k
views
logic of resamples function in caret R package
The caret R package includes functions to compare models via their resampling distributions. Specifically, it prescribes fitting multiple models using the same resampling profiles (i.e. same versions ...
2
votes
0
answers
280
views
Computing Monte Carlo Error: Particle Filters
I want to ask a question about the Monte Carlo error of a particle filter.
Assume we have information of our of the process of our true states, $x_t \forall t$ and hence, we generate our data $y_t$. (...
2
votes
0
answers
141
views
Can Resampling be used for estimation and goodness of fit test?
I am trying to compare my data with empirical distributions. But I don't have enough data to cut them to estimation data and validation data. I am trying a resampling approach and would like to see if ...
2
votes
0
answers
653
views
Bootstrapping and resampling sizes
This bootstrap primer from Stanford says
How many observations should we resample? A good suggestion is the
original sample size.
While I get that this advice might be referring specifically to ...
2
votes
0
answers
869
views
multiple test correction procedure for survey data
I clustered ~ 20,000 students into three groups. Now, I also have answers from some of those students to ~ 100 survey questions. I would like to compare the differences between the clusters regarding ...
2
votes
0
answers
880
views
What does the class_weight function in keras do during training of Neural Networks?
I have a heavily imbalanced dataset with 170 columns and 2 million rows, there are also missing data in the set. As practiced, I drop all the null values, normalized the data using min-max method and ...
2
votes
0
answers
90
views
What's the relation between the covariance estimation of the full dataset VERSUS the covariance estimations of the resampled/bootstrapped set
Consider the following Matlab code:
...