Questions tagged [resampling]

Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.

123 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
783 views

Comparison of the jacknife vs the bootstrap

I am interested in understanding the relative pros and cons of bootstrap versus jacknife resampling. Both are used in iterative algorithmic approaches to estimating the precision of a prediction or ...
5
votes
0answers
566 views

Block bootstrap for dependent data with unequal sampling intervals?

I have data from a natural archive (lake sediment). For various reasons it is usually impossible to sample the archive equally in time, and we end up with a time series where essentially we have ...
5
votes
1answer
832 views

Assessing the representativeness of population sampling

I am looking for some suggestions about assessing the representativeness of a particular dataset I am analyzing. In this dataset I am looking at the relationship between two variables (e.g., X and Y)...
4
votes
0answers
71 views

Optimal block size for spatial bootstrapping

Take a regression model: $$ y_s = X_s\beta + \epsilon $$ Where $E[\epsilon|X] = 0$, but $cor(\epsilon_s, \epsilon_{near}) > cor(\epsilon_s, \epsilon_{far})> 0$. In other words, $X$ is ...
4
votes
0answers
719 views

Why might the mean of a bootstrapped distribution not equal the original summary statistic?

Background: I have n samples and their average. The mean of this empirical bootstrapped distribution seems quite different form the average of my original sample. My original average for the n samples ...
3
votes
1answer
333 views

How to choose resample size when drawing without replacement?

Say I have some second-order statistic $m(x)$ where $x$ is a data vector of length $n$. Let's also assume that the limiting distribution of $x$ is gaussian-ish, but generally unknown, so that the ...
3
votes
0answers
110 views

Resampling points in R^n so that kernel density is roughly uniform

Let's say we have points $x_1,\ldots,x_n\in\mathbb{R}^N$ and let $X=\{x_1,\ldots,x_n\}$. I wish to produce a resampling $y_1,\ldots, y_m\in X$ (allowing repetitions) such that the new kernel density ...
3
votes
0answers
131 views

Estimation on evolving distribution with small updates

I have a set $X$ of $10^6$ elements and a time series of probability distributions $\mu_1,\mu_2,\ldots$ on $X$. I want to estimate the expected value of a function $f$ over each $\mu_t$. It is easy to ...
3
votes
0answers
116 views

Is there a resampling method that blends subsampling with the bootstrap?

I apologize if this is an inappropriate question. I thought of it in class the other day, and I couldn't find a specific answer in my textbooks. I am familiar with the two basic techniques for ...
3
votes
1answer
202 views

Selecting uncorrelated samples from a set of bulk data that contains correlated and dependent samples

i have a set of data that is generated by expensive computational model evaluations, on a total data set of 10000 samples in 40 dimensions. This sample data set is composed of different data sets, ...
3
votes
0answers
2k views

Sampling and resampling data in R

My problem this time concerns sampling size-related errors, resample-based confidence intervals and a possible way to control for this error. My dataset consists of 50 measurements of certain cranial ...
3
votes
0answers
223 views

Bootstrapping fits to a small sample

I have a sample of experimentally measured survival times that are quite noisy and vary stochastically. The survival probability of these events (number of events with a survival time of t or more) is ...
3
votes
0answers
373 views

Resampling or Basic Simulation and confidence intervals

I have a population of sales that might be won or lost. I know the rate that they are won from historical data. This case 30% of them historically win. To figure out how much money I will be making ...
2
votes
0answers
32 views

Central limit theorem for resampling

What is the analog of the central limit theorem or concentration theorem for resampling, say, an i.i.d. samples? Are there any references for this topic? As a simple example, suppose there are $n$ i....
2
votes
0answers
26 views

Correlating Two Time Series with Gaps in Data and of Different Lengths

I am attempting to correlate the time series from 4 separate tilt monitors that sample every 5 minutes. The time series have slightly different base times and end times, and the resulting arrays are ...
2
votes
0answers
49 views

Can we sample from a set of biased samples to get unbiased samples?

This is a follow-up question to How to uniformly sample vertices from a large graph with given distance from a fixed vertex?. Suppose I took a set $B$ of $n$ samples from a large but finite universe $...
2
votes
0answers
53 views

“ANOVA on a non-random non-Normal sample from a Normal Population”

How can I run ANOVA or tests for statistical significance on a bi-modal sample that came from a normal population? Context: I was tasked with running an ANOVA to see if genotypes (treatments / ...
2
votes
0answers
98 views

Bootstrap Resampling Vs KDE Resampling

Let $\xi\in\mathbb{R}^{m}$ be a random vector with joint desity function $f$, and let $\widehat{\xi}_{1},\ldots,\widehat{\xi}_{N}$ be a sample of $\xi$. We have that the kernel density estimator (...
2
votes
0answers
370 views

Replicate weights and the Current Population Survey

The Current Population Survey’s Annual Social and Economic Supplement (abbreviated, oddly, as ASEC) is the USA’s longest running and most detailed annual survey of income and employment. Along with ...
2
votes
0answers
1k views

logic of resamples function in caret R package

The caret R package includes functions to compare models via their resampling distributions. Specifically, it prescribes fitting multiple models using the same resampling profiles (i.e. same versions ...
2
votes
0answers
532 views

How to calculate a p-value under the null from a bootstrap?

In another thread, I asked how to obtain a confidence interval of the difference in probability of success between two groups from a logit model and @Weihuang Wong showed me how to do this with a ...
2
votes
0answers
165 views

Computing Monte Carlo Error: Particle Filters

I want to ask a question about the Monte Carlo error of a particle filter. Assume we have information of our of the process of our true states, $x_t \forall t$ and hence, we generate our data $y_t$. (...
2
votes
0answers
87 views

Can Resampling be used for estimation and goodness of fit test?

I am trying to compare my data with empirical distributions. But I don't have enough data to cut them to estimation data and validation data. I am trying a resampling approach and would like to see if ...
2
votes
0answers
457 views

Bootstrapping and resampling sizes

This bootstrap primer from Stanford says How many observations should we resample? A good suggestion is the original sample size. While I get that this advice might be referring specifically to ...
2
votes
0answers
819 views

What does the class_weight function in keras do during training of Neural Networks?

I have a heavily imbalanced dataset with 170 columns and 2 million rows, there are also missing data in the set. As practiced, I drop all the null values, normalized the data using min-max method and ...
2
votes
0answers
77 views

Which resampling methods are suitable in the presence of dummy variables?

I want to build a machine learning model using the caret package in R. Some of the features in my dataset are dummies taking the value 0 or 1. I would like to know which resampling methods can be used ...
2
votes
0answers
196 views

want to calculate significance of pairwise sharing between lists. Standard hypergeometric test seems inappropriate

I want to calculate if the percentage of sharing of strings (genes in this case) between lists is significantly more than expected by chance for multiple pairwise comparisons between lists of strings. ...
2
votes
0answers
44 views

Resampling in R to see if percentage of pairwise sharing between lists is significant

I have three lists of strings: ListA, ListB and ListC. First 3 lines of ...
2
votes
0answers
142 views

Creating a bootstrap null (H0) distribution. Investigating alternatives

I'm investigating the possibility of creating a non parametric bootstrap null distribution for hypothesis testing in multivariable regression analysis. The null hypothesis is based on the absence of ...
2
votes
1answer
56 views

Updating the head or tail of an ordered subset of rows using data.table in R

The concept is simpler than my title. I have a data.table that represents a sample taken from a population. My goal is to test the performance of several different prediction algorithms across sample ...
2
votes
0answers
59 views

Application of Permutation Tests

I don't know how to post the question more formally. Therefore, let me introduce an example. Suppose you want to estimate the following regression: $n_i = f(x_i)+\beta \cdot 1[x_i = j]$, $n_i$ is ...
2
votes
0answers
1k views

Bootstrapping time series data: Circular block bootstrap

I have some very basic questions on circular block bootstrap applied to time series (dependent data). Let's suppose, I have a time series data like the one below. I know it's non stationary, but for ...
2
votes
0answers
149 views

Estimating means of correlated distributions with long tails

Suppose I have a relatively large number of samples (~1k) drawn from a series (~40) of increasingly long-tailed distributions (going from approximately normal to approximately log-normal). I want to ...
2
votes
0answers
310 views

What is the correct way of generating a p-value for a correlation with resampling?

I have a vector of gene expression values, across 20 patients. Each patient also has a glucose measure (a continuous numeric vector). I want to find how significant the real correlation value is ...
2
votes
0answers
36 views

Using Resampling to understand a large table

I have a data set that is very large. The attributes (columns) are several thousand. Some are sparse others are not. Some are ordinal, others interval, nominal or ratio. The row size is 10s of ...
2
votes
0answers
72 views

Statistical thought experiment (possibly Bayesian) about survey sampling and propensity scores

For some practical application I recently came about the following thought experiment. Can anybody help? Suppose we administer a survey A to measure a variable $Y$. The response probabilities may ...
2
votes
0answers
265 views

How to assess stability of daily time series in sentiment analysis?

I developed a measure of "sentiment" and I have time based data and used the measure to derive a daily sentiment time series. I am looking for some way to establish reliability or maybe stability. For ...
1
vote
0answers
11 views

Downsampling, AUROC and accuracy equal

I am using downsampling to create perfectly balanced classes in my target feature. I have found that accuracy is exactly equal to the AUROC score. I was thinking that this is because I've used ...
1
vote
0answers
29 views

How does one estimate error of extrapolated value with only 2 data points?

I am trying to do a simple linear regression. I have only two data points with errors. How do I estimate the y-intercept with errors? Currently I have: ...
1
vote
0answers
40 views

When to use ordinary, balanced, antithetic, or permutation resampling for bootstrap?

I am using boot and would appreciate any explanation as to when using each of these resampling methods would be recommended in practice. I have come across Do who says that: Simulation results ...
1
vote
0answers
42 views

How to calculate p-value comparing bootstrap-based predicted probabilities and observed probabilities

I posted this question on stackoverflow first but I have got no answer so far, so I decided to post it here in the hope that here I might get an answer. I hope my procedure is acceptable. ...
1
vote
0answers
26 views

Bootstrap test of means using pooled groups - is there a problem?

In his book "Resampling: the New Statistics" (available for free online), Julian Simon presents several examples where he Bootstraps the differences in group means, assuming that the observations of ...
1
vote
2answers
154 views

How should I resample the training and testing set with imbalanced data whilst having meaningful performance metrics?

I have an imbalanced dataset of approx. 200 positive and 800 negative examples. I run nested cross-validation where i=5 and j=5; (i is inner and j is outer). The cross-validation procedure isn't the ...
1
vote
0answers
26 views

Practical questions about cluster bootstrap confidence intervals

I want to estimate the accuracy of a machine learning model. I have an independent test set with a vector of trusted labels and a corresponding vector of model-based predictions. If I assume the test ...
1
vote
0answers
186 views

Monte Carlo testing: number of required permutations

I want to perform a statistical hypothesis test, however I don't know the exact distribution of my test statistic under $H_0$. Therefore, I need to calculate a Monte Carlo estimate $\hat{p}$ of the ...
1
vote
0answers
45 views

How to bootstrap data with unknown correlation structure?

I am interested in how to validly bootstrap data with an unknown correlation structure. Let's say I am bootstrapping in order to obtain inference for some smooth function of the data similar to a ...
1
vote
0answers
547 views

SMOTE, Oversampling on text classification in Python

I am doing a text classification and I have very imbalanced data like Category | Total Records Cate1 | 950 Cate2 | 40 Cate3 | 10 Now I want to over ...
1
vote
0answers
131 views

resampling from a frequency distribution (histogram) to test for significance

Suppose I have count data on brands and the number of toys they make. ...
1
vote
0answers
160 views

Raking to converting a convenience sample to representative sample

I have a convenience sample of patients in a ZIP Code Tabulation Area (ZCTA) for whom I know their gender and age. I also have census data for the ZCTA (in particular I have 1) the percentage of ...