Tag Info

17

Note that at each observation position ($i=1, 2, ..., n$) we can choose any of the $n$ observations, so there are $n^n$ possible resamples (keeping the order in which they are drawn) of which $n!$ are the "same sample" (i.e. contain all $n$ original observations with no repeats; this accounts for all the ways of ordering the sample we started with). For ...

12

One way to understand this choice is to think of the sample at hand as being the best representation you have of the underlying population. You may not have the whole population to sample from any more, but you do have this particular representation of the population. A truly random re-sample from this representation of the population means that you must ...

4

I feel that your understanding of subsampling is not quite correct or your problem is too complex. For two classes subsampling is basically throwing away majority of samples from the larger class, so the remaining amount will be approximately the same to the amount of samples from the smaller class. If, as you write, subsampling lead you to unstable results, ...

4

Subsample your original data without replacement. The crucial point is to weight each of the original points with the density you are aiming for (parameterizing it, e.g., by the mean and the variance you want). In R: original <- runif(1e6) hist(original) n.sample <- 1e4 mm <- 0.3 sdev <- 0.2 weights <- dnorm(original,mean=mm,sd=sdev) ...

3

I would argue that it's not so much that the $m$ of $n$ bootstrap does smoothing as that it makes smoothing unnecessary. There are two components to the $m$ of $n$ bootstrap. The first is sampling just $m$ observations; the second is knowing the convergence rate. A big part of the advantage of the subsampling is being able to handle the correct rate. If a ...

3

You can look at Philip Good's book on resampling methods or my texts on bootstrap methods as well as the classic book by Efron and Tibshirani.

2

The variance of an ordinary least squares estimate (that is, the square of its standard error) is inversely proportional to the sample size. In practice, there is an additional form of uncertainty in this general rule: the standard error itself has to be estimated because the variance of the error terms in the model is unknown. However, provided you obtain ...

2

A random sample of size $n$ from a sample of size $N$ from a $N(0, \sigma^2)$ should also be a sample of size $n$ from $N(0, \sigma^2)$. So I don't think anything changes and $$\bar{X}_n \sim N(0, \sigma^2/n)$$

2

It will not be benign. It will have the effect of biasing your estimates to think that the frequency of $X$ is more balanced than it really is. This effectively changes the distribution of your data. For instance, suppose that you have only one instance where $X=1$ and $X=0$ everywhere else. Then, applying the procedure you described will cause you to ...

2

most nationally-representative survey weights are generated with those certain demographic characteristics (e.g. age, race, gender) as a part of their fundamental construction. unless you have a strong justification to stray from the weights provided to users of the microdata, you should err on the side of sticking with the survey weights. in r, this would ...

2

This question is really broad. Depends on the data and model, it can be a good practice and can be bad. The overall idea is to think about the "complexity of data and model". We may need to review Bias and Variance trade-off, i.e., when under-fitting and over-fitting will happen and how to detect it. How to know if a learning curve from SVM model suffers ...

2

If you have the whole population, you are not really doing any inference of a variable, that only happens when you are taking a sample. Let's say you are using a model that predicts weight based on height, so it's $$w = a\cdot h + \epsilon$$ Where $\epsilon$ is some error. Somehow you have collected data on the whole population of the planet. Or even ...

1

Okay, so here is what I recommend: What not do to: Collapsing by ignoring certain level (for example month) could either be fine but more probably not, as it will have some effect and confounds your inference since you did not control for it. Assuming things and not really testing that assumptions hold. I'm not saying you should actually perform ...

1

Bootstrap methods require resampling using the same sample size as that of the original sample. If you use fewer (or more!) samples, this is called "upstrap" The Upstrap. This, however, has different properties and you should be careful as otherwise the precision of your estimates would be misleading.

1

Yes, this works. All data is a sample population. If you have enough to achieve some level of performance on some metric, than you have achieved your goal. There will generally be a point of diminishing returns on the size of the data. Thus, more data will make little difference. As long as you have enough to make an appropriate generalization on test data, ...

1

The typical number of features is the number of features that you think are relevant. No more, no less. It might be possible to tabulate the number of features used in every XGBoost model, and compute descriptive statistics of that data, but this is folly -- there's also an average telephone number, but no one's suggesting that you should call it. People ...

1

This depends what you want to use the validation result for: if you want to validate (measure performance) the SVM trained on the first 1k cases (and then want to use that model for prediction), do not subsample. However, as you then have 29k cases left out of training, use them for testing instead of doing k-fold cross validation. if you want to tune ...

1

There are a few problems in your code. First of all, why use var(res) if you are interested in the standard deviation of the mean? Instead just use sd(res) (or equivalently sqrt(var(res))). Also your Professor of course meant that the errors scales according to sample and subsample size, not the number of your Monte Carlo replicates !! So in your case you ...

1

Your analysis results essentially answer your question. Much is from the smaller number of cases, but the lack of normally distributed residuals means that the p-values are not reliable, and there is evidence that your model is not yet specified well enough. The point estimate of your coefficient of interest is the same to two significant figures in both ...

1

First off, 95%-5% is NOT an example of imbalanced dataset, I would consider downsampling if there was something of the order of 99.9%-0.1%. Most approaches should work just fine. Edit: Seems like this answer has this covered already, have a look here: https://stats.stackexchange.com/a/133385/41367; Now, if you have to rebalance, here are some options: ...

1

Is something like this what you were looking for? # create sample data data <- replicate(1000, c(cell = sample(1:3,1), year = sample(1:3,1), runif(1))) # create an empty list obs.list <- list() # for each unique cell-year pair create a list of observations matching the cell/year for(i in unique(data[1,])) { for(j in unique(data[2,])) { obs.list[[...

1

Ok, let's focus on some issues I can find here. As I already stated on the comments, the 'tracks' present in this arrangement characterize a split-block design with three replicates in each combination A x B. Pay attention to the term replicate, I will go back to it later. So, if you want to analyze your experiment in R, assuming you have a balanced data set ...

Only top voted, non community-wiki answers of a minimum length are eligible