meta user
network profile
| bio | website | |
|---|---|---|
| location | ||
| age | ||
| visits | member for | 1 year, 10 months |
| seen | 1 hour ago | |
| stats | profile views | 98 |
|
Apr 18 |
comment |
What's the distribution of $\bar{X}^{-1}$? @DilipSarwate $\bar X^{-1}$ is not converging to a point on the boundary of its support. $-2$ is on the interior of the support of $\bar X^{-1}$. $-2$ is at the boundary of the support of $\bar X$, not $\bar X^{-1}$. |
|
Apr 16 |
comment |
$N(\theta,\theta)$: MLE for a Normal where mean=variance It looks to me like there might be an error in your calculation of $\log f(x)$. I think it should be $\mbox{const} -\frac{n}{2} \log(\theta) - \frac{s}{2\theta} + t - \frac{n \theta}{2}$. As is, there is a positive probability chance that $1 - 4 \frac s n < 0$ which is a problem. |
|
Apr 16 |
comment |
What's the distribution of $\bar{X}^{-1}$? @DilipSarwate $(\bar X)^{-1} \to -2$ a.s. trivially by the SLLN, no ($\bar X \to -\frac 1 2$ on the same set as $\bar X^{-1} \to -2$)? I'm not sure what you are getting at, is there a problem? |
|
Apr 15 |
comment |
What's the distribution of $\bar{X}^{-1}$? @cardinal with regards to the edit, I take (say) "$\bar X$ has an asymptotic $N\left(\mu, \frac{\sigma^2} n \right)$ distribution" to mean $\sqrt n(\bar X - \mu) / \sigma \to N(0, 1)$ in distribution. I don't mean $\bar X$ converges to something depending on $n$ since obviously $n$ went to infinity. It's probably better to avoid confusion since I guess this language isn't standard so your edit is good. |
|
Apr 15 |
comment |
What's the distribution of $\bar{X}^{-1}$? @Glen_b Well, I specified that $\mu \ne 0$ and since $X$ is continuous $P(X = 0) = 0$ so I think that is enough. The delta method applies even when the mean of $X^{-1}$ does not exist. I was already pretty sure of this, but I simulated it anyways just to make sure and, indeed, the delta method works when we take $X \sim \mathcal U(-2, 1)$. |
|
Apr 14 |
comment |
Given that one can sample $X \sim f(x)$, is there an easy way to sample $Y \sim k \cdot f(g(y))$ (such as $k \cdot f(e^y)$)? I don't think there should be a reasonable way to do this in general with a single draw from $f$. You either need some special conditions on $g(x)$ and $f(x)$ or be willing to settle for an approximation, considering that if you could get exact samples like this in general then you could devise samplers for getting a sample from any distribution by choosing $g$ and $f$ appropriately. I'm also not hopeful for your related problem since it would imply that we could simulate from $f(\theta | x)$ if $\theta$ is real provided $f(\theta)$ is easy to sample from and $f(x|\theta)$ is symmetric. |
|
Mar 21 |
comment |
Asymptotic distribution of MLE (log-normal) Have you tried the delta method? |
|
Mar 4 |
comment |
Book for non parametric statistics I disagree, it should be fine for learning from. If I'm remembering correctly, he wrote it for people who hadn't seen nonparametric methods before, such computer science students he teaches. |
|
Mar 4 |
comment |
Book for non parametric statistics Nonparametric statistics is a large field, but I'd guess All of Nonparametric Statistics by Larry Wassermann should be a reasonable starting point. I don't know if I would call it "advanced level" but he sketches the proofs of many of the results in the book. Topics include the bootstrap, smoothing techniques, density estimation, regression, and lots of other things. No nonparametric Bayes, however. |
|
Mar 2 |
comment |
How should I mentally deal with Borel's paradox? @cardinal We did not use a textbook, we used the instructors notes. The instructor spent his entire research career proving laws of large numbers for Banach space valued random elements, and apparently had no need for such things. As a result, he didn't teach them. We learned the topics that he found important for his work. The other professor who taught probability used Billingsley and wasn't as short sighted. I picked up what I know by reading Billingsley in my own time. |
|
Feb 16 |
comment |
Are penalized regression techniques greedy algorithms? @RobertF everyone and their grandmother has come up with a different way of conducting the optimization, and they are all iterative in nature (like Newton-Raphson, although that involves getting a Hessian and the $L_1$ penalty is not differentiable). For the LASSO, for example, the LARS algorithm can be used to get the trajectory of the parameter values as the penalization term is varied. These methods all go to a local minimum, but if everything is convex then there is only one local optimum, and it is the global optimum. |
|
Feb 9 |
comment |
Unbiased estimator for variance or Maximum Likelihood Estimator? @Corone look up a formal definition of maximum likelihood if you don't believe me. Or, you could try to write down the likelihood as a function of $\eta$ in a sensible way if (say) $\theta$ is the mean of the normal distribution and $\eta = g(\theta)$ for your function $g(\cdot)$ and realize that the likelihood you wrote down wouldn't be able to distinguish between normal distributions having different means. The likelihood isn't a density, so I don't know what you mean when you say "the value of $\eta$ that has the most probability density." |
|
Feb 9 |
comment |
Unbiased estimator for variance or Maximum Likelihood Estimator? @Corone no, the "ML thing" (I assume you mean the invariance property) works for any function, not just bijections. If $\hat \theta_{ML} = 4$ then the ML estimator of $g(\theta)$ is 1. This is essentially a matter of definition; if $\eta = g(\theta)$ where $g$ is not an injection, we define $L(\eta)$ to be $\sup_{\theta: g(\theta) = \eta} L(\theta)$. With this definition, it is trivial that the invariance property holds for any $g$. |
|
Feb 8 |
comment |
Unbiased estimator for variance or Maximum Likelihood Estimator? Nowhere in the OP do I see that scaling factor, and as far as I can tell it hasn't been edited... |
|
Feb 8 |
comment |
Unbiased estimator for variance or Maximum Likelihood Estimator? (-1) nowhere in the question does it state that his data is normal. OP is asking, if knows the underlying family of distributions (Poisson, Normal, Gamma, or whatever) should he base his estimator on ML even if it is biased, or just use the generic (nonparametric) unbiased estimator anyways. For the Poisson distribution, there is a lot more than just a scaling factor separating the MLE of the variance and the unbiased estimator the OP gave. |
|
Feb 1 |
comment |
Are splines overfitting the data? I don't really see where your statistician friend is coming from, to be honest. You can overfit with polynomials and splines just the same. Overfitting comes from your class of models having excessive capacity; what distinguishes the performance of various models is how they restrict their capacity. For (natural) splines, it is knot placement and number, for polynomials it is the degree. |
|
Feb 1 |
comment |
Are splines overfitting the data? Polynomials are far more sensitive to anomalies within the data than splines are. An outlier anywhere in the data set has a massive global effect, while in splines the effect is local. |
|
Jan 30 |
comment |
kernel density estimation of a Dirichlet distribution @Pierre Yes, but you already presumably have a posterior sample of $\eta$'s as part of your sampling algorithm, and $f(\theta | \eta, y)$ should be available as a Dirichlet density. |
|
Jan 30 |
comment |
kernel density estimation of a Dirichlet distribution Obviously, this also sidesteps the issues that KDEs run into on the boundaries, as in Corone's answer. |
|
Jan 15 |
comment |
Fitting Lognormal Distribution in WinBugs @act00 the conjugate prior for the lognormal distribution is the normal distribution (or normal-inverse-gamma if you want to put a prior on $(\mu, \sigma^2)$). Why? Because you can transform the data so that it is normal, and then use the conjugate prior for the normal. However, if you want to use the lognormal directly it is given by dlnorm(mu, tau) where tau is $1 / \sigma^2$ in the usual parameterization; just stick a normal-gamma prior on those two. |
