Questions tagged [bootstrap]
The bootstrap is a resampling method to estimate the sampling distribution of a statistic.
2
votes
0answers
12 views
Stepwise regression limitations avoided on bootstrap/independent datasets?
One of the prime objections to best-subset and stepwise regression techniques (forward selection and/or backward elimination) is that multiple hypothesis tests are conducted on the same dataset, ...
0
votes
0answers
13 views
Why we need to do bootstraping and extract confidence interval for classification problems?
My question is related to bootstraping and extracting confidence interval for classification problems. Let's say I have 25 number of data points and 2 features and use Gradient Boosting Machine (GBM) ...
2
votes
1answer
45 views
Appropriateness of nonparametric bootstrap methods to assess difference between two groups
This question is motivated by the discussion of this earlier question.
I have two samples $X$ and $Y$, where both samples have $n$ elements. Both samples represent optimal solutions returned from two ...
0
votes
0answers
11 views
Python: Compute Bootstrap Confidence Intervals for a Kernel Density Estimator [on hold]
I have plotted the probability density function of some data, using a Kernel Density Estimator.
Both the data, and the code I am using can be found here
As you can see, the txt-file contains a lot ...
0
votes
0answers
15 views
How to calculate bootstrap values for Fst?
I want to try to get the significance of my Nei Fst values through bootstrapping. I have a large data set of SNPs from RADseq data. They are in 4 populations: River, Lake, Outgroup1, Outgroup2. My ...
0
votes
0answers
6 views
How to do such Monte-Carlo operation in SPSS? [closed]
Say i have a set of 8 values (4.34, 4.32, 4.35, 4.45, 4.25, 4.09, 4.29, 4.46).
Step 1: I would like SPSS to find the CV of two randomly sampled values . And then repeat this process for 10000 times ...
0
votes
0answers
16 views
Can i get confidence intervals using bootMer in lme4? [closed]
I have a glmm model built using glmmTMB:
...
0
votes
1answer
8 views
Resample dataframe by random subset of years
I have a dataset of 59 years worth of daily rainfall data.
I'd like to resample the data by 6 randomly selected years 1,000 times, using the entire year of data for each of the 6 years (so 2190 ...
0
votes
0answers
10 views
Bootstrapping a statistics vs bootstrapping a model (e.g. random forest) [closed]
Let y_s be the sample statistics and y_b as bootstrap mean of the statistics.
When bootstrapping a statistics we use ...
3
votes
1answer
168 views
Using Random Forest variable importance for feature selection
I'm currently trying to convince my colleague that his method of doing feature selection is causing data leakage and I need help doing so.
The method they are using is as follows:
They first run a ...
0
votes
0answers
19 views
Calculating empirical p-value for Anderson-Darling test
I have the data $x_1,\ldots,x_n$. First, I fit the stable distribution to this data and obtain a vector of estimated parameters $\hat{\theta}$. Now, I want to use Anderson-Darling test for stable ...
1
vote
0answers
30 views
When to use ordinary, balanced, antithetic, or permutation resampling for bootstrap?
I am using boot and would appreciate any explanation as to when using each of these resampling methods would be recommended in practice.
I have come across Do who says that:
Simulation results ...
1
vote
0answers
18 views
Generate yearly time series
I've 5 years of historical hourly wind-speeds for a certain location available.
What I need to do is to estimate the distribution of the yearly average wind speed. What method do I need such that I ...
0
votes
0answers
16 views
overall variance of different time signals
I have two sensors mounted on two appliances and they measure a variable in the time domain every 1 second.
Now for the course of 1 hour, I want to make only one error bar that contains both signals....
1
vote
0answers
40 views
How to calculate p-value comparing bootstrap-based predicted probabilities and observed probabilities
I posted this question on stackoverflow first but I have got no answer so far, so I decided to post it here in the hope that here I might get an answer. I hope my procedure is acceptable.
...
1
vote
1answer
120 views
Does bootstrapped regression allow for inference?
Assuming a linear model of
$$ y = \beta_0+\beta_1x+\epsilon $$
I can construct a bootstrapped confidence interval for the estimate of $\beta_1$ by sampling with replacement from all of the $(x_i, y_i)...
0
votes
0answers
34 views
Model Validation Through Bootstrapping Optimism; p >>> n
On this forum, I've read quite a few posts on the use of bootstrapping a statistic known as optimism when evaluating various models for their out of sample, predictive performance. I personally have ...
0
votes
0answers
25 views
Bootstrapping vs. K-Fold: Is every data point in atleast one of the test set/out of bag - atleast once?
It's easy to see that in K-Fold cross-validation, that split training examples into K parts, in such a way that 1 of the K parts is considered to be the test set, and eventually as you shift which ...
3
votes
1answer
39 views
Why is one of the two approximations in the bootstrap worse than the other?
My statistics text has the following diagram:
$$\mathbb{V}_F(T_n) \overbrace{\approx}^{ \text{not so small} } \mathbb{V}_{\hat{F}_n}(T_n) \overbrace{\approx}^{ \text{small} } v_{\text{boot}}$$
...
0
votes
0answers
19 views
How to bootstrap single equation autoregressive models?
I have been reading many papers where confidence intervals for the impulse responses of autoregressive process have been boot-strapped. The question is, is the usual way sufficient? Take as an example ...
0
votes
0answers
17 views
Is it fair to consider rolling regression a form of bootstrapping?
The context is time series analysis. A few similarities between rolling regression and boostrapping jump out at me, in that both re-use observations to form new subsamples for estimation. However, ...
0
votes
0answers
22 views
Generate data with a given covariance matrix and given non-normal distribution
Question
I have a dataset of numbers, which I know to be correlated with a covariance matrix that I can reasonably estimate. This correlation has no (known) structure, such as being time, space, or ...
0
votes
1answer
26 views
Double Bootstrap Example
I am working on the double (iterated) Bootstrap. The idea is to take resamples from a given sample, compute the statistic of interest for this sample, and then standardize the result. When there is no ...
0
votes
0answers
18 views
Estimation-based bootstrap using GARCH(1,1) and Rugarch
I try to replicate the methodology proposed by Freedman and Peters (1984a, 1984b) which was applied in the famous paper by Brock, Lakonishok and LeBaron (1992) to generate many artificial log return ...
1
vote
1answer
31 views
Whether there is significant difference between two gender groups
A data set contains 5 columns:
There are 16 people in the dataset(using subjectID to identify them). Each year, when these people were ages 11, 12, 13, 14, and 15, they provided us a score $y$. We ...
2
votes
1answer
101 views
Bootstrap, Rubin's rules, and uncertainty of sub-estimates?
Can someone provide an intuition for why, when using bootstrap to calculate the variability of an estimate (say a regression coefficient $\beta$) we don't need to incorporate the uncertainty of each ...
2
votes
1answer
60 views
Residual Bootstrapp based on GARCH models with student-t distributed innovation
I want to generate 500 simulations of my original return time series.
My original return series (n = 4000) exhibits significant serial autocorrelation at lag 1 & 2, is non-normally distributed (...
1
vote
2answers
44 views
Diagnostics for Bootstrap
I would like to know if there are any agreed on diagnostics one should run after any simple bootstrapping procedure. For example, I bootstrap a quite simple parameter like a median or a correlation to ...
0
votes
0answers
20 views
Confidence Intervals on Indirect Effect: Moderated Mediation/Conditional Path Analysis in R
I'm trying to bootstrap confidence intervals for my estimates of the indirect effects of a mediation model. Because it is actually (first-stage, a-path) moderated mediation, I need to actually do this ...
2
votes
2answers
92 views
Can bootstrapping improve the test power, when the sample size is small? [closed]
When the sample is small and study is under-powered, can bootstrapping be used to generate data and increase the sample size (and improve the test power)?
Is there any way that bootstrapping can be ...
2
votes
1answer
91 views
Bootstrap for goodness of fit
Suppose I want to test whether a die is fair. I toss 60 times with outcome $X = (40,20,0,0,0,0)$ where $i$th coordinate denotes times of $i$th face occurs.Consider Hypothest test($\alpha = 0.05$):
$$...
0
votes
0answers
21 views
Bootstrapping non-random samples
I perform some analysis (with the goal to select some relevant variables that are related between two experiments) on a group of samples (n = 158) but I would like to know how robust are, before going ...
0
votes
0answers
17 views
Error/Confidence Interval estimation on very specific problem
I'm developing a method which estimates a value $t$ for a very specific problem.
Assume that we have an individual $s$ with its associated data but where $t$ is not precisely known and a reference ...
0
votes
0answers
13 views
Appropriate visualization and statistical testing using resampling stats
I'm using a linear SVM to perform classification on a dataset.
It seems to me that there are many ways to visualize my results and report their statistical significance, and I'm unsure of best ...
0
votes
0answers
12 views
What is a reliable way to obtain an optimism-correct AUC with confidence limits?
I have seen that Frank Harrell's rms package does not offer a CI for Somers Dxy (and subsequently the c-statistic/AUROC).
I am trying to look at a method with good performance and a plan to ...
7
votes
1answer
159 views
Multiple comparisons correction for dependent comparisons
In this blog post the authors discuss simultaneously estimating quantiles, and constructing a simultaneous confidence envelope for the estimation which covers the whole quantile function. They do this ...
1
vote
0answers
12 views
Why does the computational time of maximum likelihood estimators depend only on the number of UNIQUE observations?
The following paper says that the computational time of least squares, maximum likelihood, and M-estimators in general depend only on the number of UNIQUE observations.
Intuitively it makes sense to ...
1
vote
0answers
9 views
Best way to test the difference in means for Tweedie distributions?
What is the best way to test the difference in means for two samples of data that follow a Tweedie distribution?
(zero inflated, tweedie power parameter ~1.7)
The data represents the number of ...
0
votes
0answers
12 views
Theoretically is it better to sample with or without replacement to approximate a rank test?
Let's say we want to do a rank test since we do not know the exact distribution.
Say we have 3 different sample sizes. Which of the following tests (exact permutation, sample with replacement, sample ...
3
votes
1answer
68 views
Proportion Test: Z-test vs bootstrap / simulation - different results
I'm learning hypothesis testing, and going through the following example:
The CEO of a large electric utility claims that 80 percent of his 1,000,000 customers are very satisfied with the service ...
1
vote
0answers
34 views
Calibration curve for mixed model (logistic) [closed]
I did some searching using the CV search function and was unable to locate any information on R packages or general approach to creating a calibration curve with a bootstrapped curve overlay (similar ...
0
votes
0answers
13 views
Should log-log relationship of relative std.err. vs. sample size always be linear?
I am seeing a strong linear relationship between log-log plot of number of samples vs std.error of the mean estimate. As far as I can tell, this is expected (e.g. link). Since std. error is calculated ...
0
votes
1answer
30 views
A simple concept question regarding Bootstrapping
I am having a difficulty understanding whether I can use the bootstrapping method for prediction.
First off, my data is as follows.
where personal Income is the dependent variable (y), and GPA, Age, ...
1
vote
0answers
60 views
determine quantile for prediction with quantile regression (forests)
Having a fitted quantile regression (forest) model is great. However, how does one choose the best quantile to perform the actual prediction?
One idea would be to use bootstrapping. In other words, ...
0
votes
0answers
18 views
Bootstrapping ANCOVA to deal with violations of assumptions
I have a smallish data set with three randomised intervention groups ($n$s between 13 and 22), with pre-post variables. I want to run an ANCOVA with the pre-data as a covariate and post as an outcome ...
1
vote
0answers
17 views
Bootstrap test of means using pooled groups - is there a problem?
In his book "Resampling: the New Statistics" (available for free online), Julian Simon presents several examples where he Bootstraps the differences in group means, assuming that the observations of ...
0
votes
0answers
92 views
In a bootstrap sample, what is the probability that the 5th observation is the same as the 1st observation?
I know that the probability of jth observation being in a sample is:
p(jth obs in bootstrap sample) = (1 - 1/n) ^ n
p(5) = (1 - 1/n)^n
p(3) = (1 - 1/n)^n
Can you give me a hint on how to go about ...
1
vote
1answer
64 views
Bootstrap based bias correction
Assume we have a probablistic model $f_{\theta}(x)$ and try to estimate the parameter $\theta$ based on data $x$ with some procedure that yields a biased estimator $$E[\hat{\theta}]=\theta + \eta,$$ ...
0
votes
1answer
18 views
2-sample bootstrap hypothesis test - comparing locations but different estimators in two samples
I have two independent samples X and Y where $x_i \sim F$ and $y_i \sim G$. Two different estimators A and B map X to $x_0$ and Y to $y_0$ respectively. I'd like to compare $x_0$ and $y_0$.
The ...
0
votes
0answers
20 views
Multinomial Random Variable - significance test & bootstrapping
If I have a random variable $X$ that follows a multinomial distribution $(p_1, p_2, \dotsc, p_n)$, is there anyway to test if two of the levels have equal probability, for example, if $p_1 = p_2$?
...
