Questions tagged [resampling]
Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.
0
votes
0answers
5 views
How to interpolate/resample both dense and sparse points?
Suppose I have data like red points below
I would like to interpolate/resample these points at black ticks.
At right the points are sparse and it is obvious to interpolate them linearly or with ...
1
vote
0answers
28 views
Monte Carlo testing: number of required permutations
I want to perform a statistical hypothesis test, however I don't know the exact distribution of my test statistic under $H_0$. Therefore, I need to calculate a Monte Carlo estimate $\hat{p}$ of the ...
0
votes
1answer
18 views
Dealing with dataset imbalance: test if adjusting is necessary
I'm currently working on a project which uses a imbalanced dataset (two classes) for training, and I'm not sure if I should do a resampling procedure or not. Is there a way to actually test if it's ...
0
votes
0answers
12 views
How can we explain the fact that “Bagging reduces the variance while retaining the bias” mathematically?
I am able to understand the intution behind saying that "Bagging reduces the variance while retaining the bias".
What is the mathematically principle behind this intution? I checked with few experts ...
0
votes
0answers
14 views
How to bootstrap data with unknown correlation structure?
I am interested in how to validly bootstrap data with an unknown correlation structure. Let's say I am bootstrapping in order to obtain inference for some smooth function of the data similar to a ...
0
votes
0answers
8 views
How would you do a two-one-sided test with resampling (bootstrap)?
Say I have two samples and I want to test whether their mean difference is likely to be in the interval [-0.1, 0.1] (this is, I want to both reject the hypothesis that it is smaller that -0.1 or ...
0
votes
1answer
61 views
Hypothesis testing when you have the entire population?
I have an experiment that involves testing the route-finding ability of 3 different critters. They have to travel between 5 different points (essentially a travelling salesman problem) and for each ...
0
votes
1answer
34 views
Paired-t test for comparing two subgroups selected from the same underlying population?
Say I have a total population of N=100. Algorithm #1 is based on some predictors and it selects n1=10 subjects (i.e., subgroup #1). Algorithm #2 is based on another set of predictors and it selects n2=...
0
votes
1answer
14 views
Resampling to get equal predictive power per observation
Cross posted from data science due to lack of response
This is probably a thing I am just not searching for correctly, but essentially my idea is this: given some machine learning classification $C$ ...
0
votes
0answers
37 views
Handling imbalanced data for classification [duplicate]
What are the best ways to deal with imbalanced datasets for classifying whether or not individuals pay their tuition? The data is 75% positive class (paid) and 25% negative (unpaid). Some approaches I ...
0
votes
0answers
16 views
bootstrapping covariance matrices with different sampling procedures
The regression model has heteroskedasticity. The variance of error term depends on regressors. From boostrapping analysis, I got two different covariance matrices of $\beta$. The difference results ...
0
votes
0answers
21 views
When to adjust for oversampling
This may be a question that is too broad for a single answer, so I'll attempt a narrowing of sorts with the below restrictions:
Response is a binary outcome
Split full data set into a ...
0
votes
1answer
39 views
different p-values from resampling and traditional methods
I'm not sure if this is hard to answer without the original data, but let's give it a go.
I'm calculating the difference in proportions for two diagnostic methods:
...
0
votes
0answers
18 views
bootstrap correlated data intuition
To estimate the variance of a parameter, I know that a simple bootstrap will not work for correlated data. I also read about block bootstrap. I am trying to get an intuition about why it is necessary ...
0
votes
0answers
17 views
Resampling imbalanced data in a multi-view scenario
Assume a multi-view scenario, where multiple views of the same entity are available. If each data pair is assigned a label and the resulting scenario is highly imbalanced, what are proper ways of ...
0
votes
0answers
23 views
Enzyme kinetics: Bootstrapping parameter estimates
First things first - this is kind of a mixed problem from biological data analysis, I hope I am in the right place to ask my question(s).
Context
I have data from a fluorescence assay of binding of ...
1
vote
0answers
42 views
Can we sample from a set of biased samples to get unbiased samples?
This is a follow-up question to How to uniformly sample vertices from a large graph with given distance from a fixed vertex?.
Suppose I took a set $B$ of $n$ samples from a large but finite universe $...
1
vote
1answer
56 views
Process for oversampling data for imbalanced binary classe
I have about a 30% and 70% for class 0 (minority class) and class 1 (majority class). Since I do not have a lot of data, I am planning to oversample the minority class to balance out the classes to ...
0
votes
1answer
44 views
Precision and recall of imbalanced classes
I'm new and have searched many questions about this problem in this stack, but those answers aren't clear enough for me.
The point is the area under PR curve of my binary classes is the same as the ...
1
vote
0answers
285 views
SMOTE, Oversampling on text classification in Python
I am doing a text classification and I have very imbalanced data like
Category | Total Records
Cate1 | 950
Cate2 | 40
Cate3 | 10
Now I want to over ...
0
votes
1answer
13 views
Is there any point in splitting data before applying NHST?
If I have understood correctly, Null Hypothesis Significance Testing is a device, which eats data and a conservative guess, and outputs a 'probability' that the data was generated by the hypothesized ...
4
votes
1answer
234 views
Oversampling: whole set or training set
I have a rather small dataset of 4 000 points (140 features) to feed to a NN binary classifier. The problem is only ~700 of them represent the second class. Is it more common to resample the whole ...
2
votes
0answers
45 views
“ANOVA on a non-random non-Normal sample from a Normal Population”
How can I run ANOVA or tests for statistical significance on a bi-modal sample that came from a normal population?
Context:
I was tasked with running an ANOVA to see if genotypes (treatments / ...
0
votes
1answer
36 views
Hypothesis testing on subsamples and cross validation
When working on very large datasets, identifying effects rests more on quantification than significance and many questions/answers give insight on what to do regarding large samples like this (very ...
2
votes
1answer
95 views
Conceptual questions on ensemble learning and Boosting methods in Matlab
The documentation on ensemble methods in Matlab explains different ensemble algorithms for classification and regression tasks. I have normalized the raw feature set and using the normalized data for ...
1
vote
0answers
51 views
1
vote
1answer
48 views
Accounting for sensitivity of model to small changes in assumptions
I am running a Monte Carlo simulation of correlated asset returns, based on this Matlab function.
The model's inputs are a vector of expected means and standard deviations for 9 assets, and their ...
1
vote
0answers
54 views
Bootstrap Resampling Vs KDE Resampling
Let $\xi\in\mathbb{R}^{m}$ be a random vector with joint desity function $f$, and let $\widehat{\xi}_{1},\ldots,\widehat{\xi}_{N}$ be a sample of $\xi$. We have that the kernel density estimator (...
0
votes
0answers
20 views
Resampling methods in derivation of prediction error
I have made a model which predicts some value $Y_i$ given $X_i$ where $i \in 1,...,N$. My ultimate goal is to predict $Y=\sum_i^n Y_i$ and to be specific the interval in which $Y$ lies within some % ...
1
vote
0answers
91 views
resampling from a frequency distribution (histogram) to test for significance
Suppose I have count data on brands and the number of toys they make.
...
0
votes
0answers
11 views
Using mean estimates of subsets of data or entire data to estimate a parameter
"Suppose that we wish to invest a fixed sum of money in two financial assets that yield returns of X and Y , respectively, where X and Y are random quantities. We will invest a fraction α of our money ...
4
votes
0answers
37 views
Optimal block size for spatial bootstrapping
Take a regression model:
$$
y_s = X_s\beta + \epsilon
$$
Where $E[\epsilon|X] = 0$, but $cor(\epsilon_s, \epsilon_{near}) > cor(\epsilon_s, \epsilon_{far})> 0$.
In other words, $X$ is ...
0
votes
0answers
36 views
For statistical resampling, why do we use Mean Squared Error (MSE) and not Residual Standard Error (RSE) to measure the test set error rate?
Is this as trivial as 'we use RSE for training-set and MSE for test-set'?
0
votes
1answer
200 views
Is Tomek Link undersampling the same as Edited Nearest Neighbours with 1 neighbour?
From what I've read I've understood that undersampling the majority class with Tomek Links or Edited Nearest Neighbours with 1 neighbour should yield the same result. However, I've tried it on this ...
3
votes
1answer
73 views
Variance after resampling uniformly from a sample from a normal population
I have recently been looking into the Bootstrap, and I was wondering, if I were to have a sample $X=\{x_1,x_2,\dots,x_N\}$ that has $N$ samples, all i.i.d coming from a normal distribution, $N(\mu,\...
0
votes
1answer
17 views
Removing confounding variable when comparing distributions
I have the following situation and am hoping that someone can point me in the right direction. I am completing an experiment under 2 conditions (1 and 2) and measuring 2 variables (A and B). I am ...
13
votes
3answers
3k views
Why is bootstrapping useful?
If all you are doing is re-sampling from the empirical distribution, why not just study the empirical distribution? For example instead of studying the variability by repeated sampling, why not just ...
1
vote
1answer
273 views
What's the point of reporting bootstrap bias?
Suppose the data $X = (X_1,X_2,\cdots,X_n)$ is a vector of iid observations $X_i$ where each $X_i$ has marginal distribution $F(\theta)$. Suppose we observe $x = (x_1,\cdots,x_n)$ and $\hat \theta = \...
0
votes
0answers
71 views
K-fold cross validation: confusion over which model is actually chosen and validated
I am watching the online lectures of Machine Learning by Hastie and Tibshirani, which are quite good. But they discussed $k$-fold cross-validation and there is something confusing me. As they describe ...
1
vote
0answers
230 views
Replicate weights and the Current Population Survey
The Current Population Survey’s Annual Social and Economic Supplement (abbreviated, oddly, as ASEC) is the USA’s longest running and most detailed annual survey of income and employment. Along with ...
1
vote
2answers
169 views
Mean of the sampling distribution of the OLS estimator
Suppose b represents the OLS estimator, and B the true coefficient in the regression model without intercept
y = Bx + u.
Under certain assumptions b is unbiased ...
0
votes
0answers
18 views
ROSE resampling altering certain values in variable
When using the ROSE package's ROSE() function, I resample data to achieve balanced classes. One of the numeric variables in my dataset appears to get altered where negative values present. Now, when I ...
0
votes
0answers
76 views
Analyzing resampled predictions in caret package
Is there a way to compare model performance in the caret package by combining the resampled predictions from every fold/repeat? I am working with a small (<1000 rows) and severely imbalanced (4% ...
3
votes
2answers
212 views
Resample-move algorithm for Sequential Monte Carlo
I'm reading about the resample-move strategy, originally by Gilks and Berzuini, but my question will use the slightly more verbose description from the review of Doucet and Johansen, section 4.4, PDF ...
10
votes
4answers
342 views
Why do hypothesis tests on resampled datasets reject the null too often?
tl;dr: Starting with a dataset generated under the null, I resampled cases with replacement and conducted a hypothesis test on each resampled dataset. These hypothesis tests reject the null more than ...
2
votes
1answer
82 views
What is the difference between SIR and Rejection sampling in this case
Suppose we want a sample of size n from a truncated gaussian distribution with density
$f(x) = \dfrac{1}{\sqrt{2\pi}\sigma(1-\Phi((1-\mu )/\sigma ) }e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ , $x>1$
set
...
1
vote
2answers
624 views
Bootstrap p-value
Assume you created $B$ bootstrap replicates $T^{(i)}, i\in\{1,\dots,B\}$ for a test statistic $T$ since you don't know the distribution of this test statistic.
Why is the p-value approximated by $\...
0
votes
0answers
26 views
Does increasing sample size by reducing universe invalidate previous data?
My team and I are working on a project that aims to measure the impact of adding more public bicycle stations in Mexico City on private vehicle traffic. We selected 100 key locations, which are spread ...
1
vote
0answers
77 views
Raking to converting a convenience sample to representative sample
I have a convenience sample of patients in a ZIP Code Tabulation Area (ZCTA) for whom I know their gender and age.
I also have census data for the ZCTA (in particular I have 1) the percentage of ...
2
votes
1answer
225 views
How to choose resample size when drawing without replacement?
Say I have some second-order statistic $m(x)$ where $x$ is a data vector of length $n$. Let's also assume that the limiting distribution of $x$ is gaussian-ish, but generally unknown, so that the ...
