0
$\begingroup$

I have a set of $M$ variables $\{ \lambda_N^{(i)} \}_{i=0}^{M(N)}$. The distribution of $\lambda$ depends on the $N$ parameter; for $N \rightarrow \infty$ the distribution shrinks around the mean value, but for finite $N$ the distribution is characterized by the presence of outliers which determine a fat tails decays. The presence of this outliers gradually (and slowly) disappear increasing $N$. I want to verify if the shrinking of my distribution follow the theoretical prediction. So, starting from a set $\{ \lambda_N^{(i)} \}_{i=0}^{M(N)}$ I look for an estimate of the variance, which is the variable I want to study, and a confident interval for it.
So I have to repeat it for different values of the $N$ parameter (say $n$ times) and perform a fit with the $n$ couples $(N, Var_{\lambda_N})$.
My doubt is how to associate the uncertainties with my variables $Var_{\lambda_N}$
My first attempt was this:
starting from the set $\{ \lambda_N^{(i)} \}_{i=0}^{M(N)}$ I perform the 2 following transformation:

  • $\lambda_N \rightarrow w_N \equiv \lambda_N- \overline{\lambda_N}$, with $\overline{\lambda_N} = \frac{1}{M(N)} \sum_{i=0}^{M(N)} \lambda_N^{(i)} $
  • $w_N \rightarrow z_N \equiv w_N^2$
    With the distribution of $z_N$ I estimated $Var_{\lambda_N} = \overline{z_N}$ and the standard deviation of $Var_{\lambda_N}$ with the standard deviation of $z_N$.
    The problem with this approach is that, due to the outlier's presence the standard deviation's estimate is too large.
    My second attempt was to use bootstrap method:
    Starting from the set of $M(N)$ values I perform $n_r$ resampling (with repetition), each one with $n_s$ element.
    The problem with this approach is that the estimation of the confident interval (always for the presence of outliers) is strongly dependent on the choice of $n_s$
    In fact, using a big $n_s$'s value, I increase the probability to find at least one outlier in a generic resampled set, which deeply increase the variance estimation of that set.
    So, with big $n_s$ I have a small variability between different resampled set (small confident interval), while decreasing $n_s$ I reduce the probability to extract an outlier, increasing the variability between different resampled set (bigger confident interval).
    I don't want that my confidence interval's estimate depend on the choice of $n_s$.
    Is there any reasonable reasonable basis to determine the right choice of $n_s$?
    Should I use a different method in situations like that (jack knife or others)?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.