Questions tagged [resampling]
Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.
0
votes
0answers
18 views
Can one construct “original” data from a function of jackknifed data?
Say I have original, uncorrelated data, $x_i$, with $i = 1,2 \ldots N$. I can jackknife this data set (a simple delete-one)
$$
\bar{x}_{i} = \frac{1}{N-1}\sum_{j \neq i}x_{j} \quad\quad (1)
$$
to ...
0
votes
1answer
8 views
Resample dataframe by random subset of years
I have a dataset of 59 years worth of daily rainfall data.
I'd like to resample the data by 6 randomly selected years 1,000 times, using the entire year of data for each of the 6 years (so 2190 ...
1
vote
0answers
20 views
How does one estimate error of extrapolated value with only 2 data points?
I am trying to do a simple linear regression. I have only two data points with errors. How do I estimate the y-intercept with errors?
Currently I have:
...
1
vote
0answers
29 views
When to use ordinary, balanced, antithetic, or permutation resampling for bootstrap?
I am using boot and would appreciate any explanation as to when using each of these resampling methods would be recommended in practice.
I have come across Do who says that:
Simulation results ...
2
votes
0answers
15 views
Correlating Two Time Series with Gaps in Data and of Different Lengths
I am attempting to correlate the time series from 4 separate tilt monitors that sample every 5 minutes. The time series have slightly different base times and end times, and the resulting arrays are ...
1
vote
0answers
40 views
How to calculate p-value comparing bootstrap-based predicted probabilities and observed probabilities
I posted this question on stackoverflow first but I have got no answer so far, so I decided to post it here in the hope that here I might get an answer. I hope my procedure is acceptable.
...
2
votes
1answer
101 views
Bootstrap, Rubin's rules, and uncertainty of sub-estimates?
Can someone provide an intuition for why, when using bootstrap to calculate the variability of an estimate (say a regression coefficient $\beta$) we don't need to incorporate the uncertainty of each ...
0
votes
0answers
13 views
Appropriate visualization and statistical testing using resampling stats
I'm using a linear SVM to perform classification on a dataset.
It seems to me that there are many ways to visualize my results and report their statistical significance, and I'm unsure of best ...
0
votes
0answers
16 views
Adjusting predicted probabilities after resampling
Suppose I've got highly imbalanced data and I want to train a model, for binary classification. So I upsample the minority class or downsample the majority class or whatever. My question is whether ...
0
votes
0answers
24 views
Permutation tests for paired data with several variables
I am trying to answer my own previous question, with a permutation method.
I have a reference method $M_{ref}$ that I would like to compare with other methods $M_1$, $M_2$, $M_3$ when taking ...
1
vote
0answers
33 views
Calibration curve for mixed model (logistic) [closed]
I did some searching using the CV search function and was unable to locate any information on R packages or general approach to creating a calibration curve with a bootstrapped curve overlay (similar ...
1
vote
0answers
17 views
Bootstrap test of means using pooled groups - is there a problem?
In his book "Resampling: the New Statistics" (available for free online), Julian Simon presents several examples where he Bootstraps the differences in group means, assuming that the observations of ...
1
vote
2answers
50 views
How should I resample the training and testing set with imbalanced data whilst having meaningful performance metrics?
I have an imbalanced dataset of approx. 200 positive and 800 negative examples.
I run nested cross-validation where i=5 and j=5; (i is inner and j is outer). The cross-validation procedure isn't the ...
0
votes
0answers
12 views
Adding Bad Events from the past to the risk default model to avoid Down/Up sampling techniques
We have been trying to build a classification model for credit default prediction using two different models one being Random forest and another being the Logistic regression based scorecard model. ...
0
votes
0answers
34 views
how to use boot package to do stratified bootstrapping?
Here's a toy data set that replicates my problem. I am interested in knowing the confidence intervals of an empirical distribution that is composed of the scores of each school at the proportion that ...
0
votes
0answers
9 views
Hypothetical sample size
I have a data set of n=98 of values that is normal distributed. This data set has a certain spread and hence a certain standard error of the mean. I now want to test how many samples (ns) I would need ...
1
vote
0answers
23 views
Practical questions about cluster bootstrap confidence intervals
I want to estimate the accuracy of a machine learning model. I have an independent test set with a vector of trusted labels and a corresponding vector of model-based predictions. If I assume the test ...
0
votes
0answers
21 views
How to approximate continuous function from discrete uneven interval value?
To clarify, I have a discrete value of an unknown distribution. The value of this distribution domain is from 0 to 1. But the value which the discrete value is sampled has an unknown interval.
For ...
0
votes
0answers
23 views
General way to construct a confidence interval for a unknown constant to which a sample estimator converges
Assuming that a sample estimator converges to some unknown constant (a wild assumption to be sure) and without assuming the distribution of either the sample estimator or the variables from which it ...
0
votes
1answer
49 views
ROC and PR curves after over/under sampling in Unbalanced datasets
As I understood till now, ROC curves are not a good presentation of unbalanced datasets and PR curves are preferred because ROC curves are not sensitive to false positives. If we now use resample ...
0
votes
0answers
12 views
Why with probability close to 1 bootstrap and classical tests provide the same decision? Is it caused by loss of pivotality? How?
Here is an example on which my question is based can please someone tell me what is the relation between the probability close to one and the fact that we have same decisions as classical tests and ...
2
votes
1answer
47 views
Sample size calculation for Kendall's Tau in reproducibility study
Can anyone help with justifying (or rubbishing) a couple of aspects of a sample size calculation for a grant proposal.
A pilot study gave significant results for the correlation of a continuous (non-...
0
votes
0answers
17 views
How to interpolate/resample both dense and sparse points?
Suppose I have data like red points below
I would like to interpolate/resample these points at black ticks.
At right the points are sparse and it is obvious to interpolate them linearly or with ...
1
vote
0answers
105 views
Monte Carlo testing: number of required permutations
I want to perform a statistical hypothesis test, however I don't know the exact distribution of my test statistic under $H_0$. Therefore, I need to calculate a Monte Carlo estimate $\hat{p}$ of the ...
0
votes
1answer
31 views
Dealing with dataset imbalance: test if adjusting is necessary
I'm currently working on a project which uses a imbalanced dataset (two classes) for training, and I'm not sure if I should do a resampling procedure or not. Is there a way to actually test if it's ...
1
vote
1answer
130 views
How can we explain the fact that “Bagging reduces the variance while retaining the bias” mathematically?
I am able to understand the intution behind saying that "Bagging reduces the variance while retaining the bias".
What is the mathematically principle behind this intution? I checked with few experts ...
1
vote
0answers
34 views
How to bootstrap data with unknown correlation structure?
I am interested in how to validly bootstrap data with an unknown correlation structure. Let's say I am bootstrapping in order to obtain inference for some smooth function of the data similar to a ...
0
votes
0answers
15 views
How would you do a two-one-sided test with resampling (bootstrap)?
Say I have two samples and I want to test whether their mean difference is likely to be in the interval [-0.1, 0.1] (this is, I want to both reject the hypothesis that it is smaller that -0.1 or ...
0
votes
1answer
65 views
Hypothesis testing when you have the entire population?
I have an experiment that involves testing the route-finding ability of 3 different critters. They have to travel between 5 different points (essentially a travelling salesman problem) and for each ...
0
votes
1answer
43 views
Paired-t test for comparing two subgroups selected from the same underlying population?
Say I have a total population of N=100. Algorithm #1 is based on some predictors and it selects n1=10 subjects (i.e., subgroup #1). Algorithm #2 is based on another set of predictors and it selects n2=...
0
votes
1answer
16 views
Resampling to get equal predictive power per observation
Cross posted from data science due to lack of response
This is probably a thing I am just not searching for correctly, but essentially my idea is this: given some machine learning classification $C$ ...
0
votes
0answers
44 views
Handling imbalanced data for classification [duplicate]
What are the best ways to deal with imbalanced datasets for classifying whether or not individuals pay their tuition? The data is 75% positive class (paid) and 25% negative (unpaid). Some approaches I ...
0
votes
0answers
55 views
bootstrapping covariance matrices with different sampling procedures
The regression model has heteroskedasticity. The variance of error term depends on regressors. From boostrapping analysis, I got two different covariance matrices of $\beta$. The difference results ...
0
votes
1answer
42 views
different p-values from resampling and traditional methods
I'm not sure if this is hard to answer without the original data, but let's give it a go.
I'm calculating the difference in proportions for two diagnostic methods:
...
0
votes
0answers
43 views
bootstrap correlated data intuition
To estimate the variance of a parameter, I know that a simple bootstrap will not work for correlated data. I also read about block bootstrap. I am trying to get an intuition about why it is necessary ...
0
votes
0answers
22 views
Resampling imbalanced data in a multi-view scenario
Assume a multi-view scenario, where multiple views of the same entity are available. If each data pair is assigned a label and the resulting scenario is highly imbalanced, what are proper ways of ...
0
votes
0answers
26 views
Enzyme kinetics: Bootstrapping parameter estimates
First things first - this is kind of a mixed problem from biological data analysis, I hope I am in the right place to ask my question(s).
Context
I have data from a fluorescence assay of binding of ...
2
votes
0answers
47 views
Can we sample from a set of biased samples to get unbiased samples?
This is a follow-up question to How to uniformly sample vertices from a large graph with given distance from a fixed vertex?.
Suppose I took a set $B$ of $n$ samples from a large but finite universe $...
1
vote
1answer
135 views
Process for oversampling data for imbalanced binary classe
I have about a 30% and 70% for class 0 (minority class) and class 1 (majority class). Since I do not have a lot of data, I am planning to oversample the minority class to balance out the classes to ...
0
votes
1answer
51 views
Precision and recall of imbalanced classes
I'm new and have searched many questions about this problem in this stack, but those answers aren't clear enough for me.
The point is the area under PR curve of my binary classes is the same as the ...
1
vote
0answers
488 views
SMOTE, Oversampling on text classification in Python
I am doing a text classification and I have very imbalanced data like
Category | Total Records
Cate1 | 950
Cate2 | 40
Cate3 | 10
Now I want to over ...
0
votes
1answer
13 views
Is there any point in splitting data before applying NHST?
If I have understood correctly, Null Hypothesis Significance Testing is a device, which eats data and a conservative guess, and outputs a 'probability' that the data was generated by the hypothesized ...
4
votes
1answer
544 views
Oversampling: whole set or training set
I have a rather small dataset of 4 000 points (140 features) to feed to a NN binary classifier. The problem is only ~700 of them represent the second class. Is it more common to resample the whole ...
2
votes
0answers
51 views
“ANOVA on a non-random non-Normal sample from a Normal Population”
How can I run ANOVA or tests for statistical significance on a bi-modal sample that came from a normal population?
Context:
I was tasked with running an ANOVA to see if genotypes (treatments / ...
0
votes
1answer
40 views
Hypothesis testing on subsamples and cross validation
When working on very large datasets, identifying effects rests more on quantification than significance and many questions/answers give insight on what to do regarding large samples like this (very ...
2
votes
1answer
179 views
Conceptual questions on ensemble learning and Boosting methods in Matlab
The documentation on ensemble methods in Matlab explains different ensemble algorithms for classification and regression tasks. I have normalized the raw feature set and using the normalized data for ...
1
vote
0answers
53 views
1
vote
1answer
51 views
Accounting for sensitivity of model to small changes in assumptions
I am running a Monte Carlo simulation of correlated asset returns, based on this Matlab function.
The model's inputs are a vector of expected means and standard deviations for 9 assets, and their ...
1
vote
0answers
75 views
Bootstrap Resampling Vs KDE Resampling
Let $\xi\in\mathbb{R}^{m}$ be a random vector with joint desity function $f$, and let $\widehat{\xi}_{1},\ldots,\widehat{\xi}_{N}$ be a sample of $\xi$. We have that the kernel density estimator (...
0
votes
0answers
21 views
Resampling methods in derivation of prediction error
I have made a model which predicts some value $Y_i$ given $X_i$ where $i \in 1,...,N$. My ultimate goal is to predict $Y=\sum_i^n Y_i$ and to be specific the interval in which $Y$ lies within some % ...
