All Questions
712 questions with no upvoted or accepted answers
8
votes
0
answers
1k
views
Applying a variance-stabilizing transform to a fitted function (rather than data)
Outline
I'm working with data corrupted by a mixed Poisson-Gaussian noise model (for example with images gathered in astronomy or electron microscopy), and have been using the generalized Anscombe ...
- 163
8
votes
0
answers
1k
views
Estimation of log-likelihood via importance sampling
I am looking at a model trained with stochastic gradient variational Bayes. In this paper an importance sampler is proposed to estimate the likelihood:
$$p(x) \approx {1 \over S} \sum_{s=1}^S {p(x|...
- 14k
7
votes
0
answers
663
views
Computation of log-likelihood in semi-supervised naive bayes
I have the following 2 questions about log-likelihood computation in semi-supervised Naive Bayes.
I have read on several documents online that, in every EM iteration of the semi-supervised Naive ...
- 123
7
votes
1
answer
125
views
Measurement error in maximum counts
I'm familiar with the concept of a mean value of data and the variation around the mean. Is it possible to quantify variation around maximum values?
For example, take the below data collected across ...
- 14.6k
6
votes
0
answers
681
views
Gradient-informed global optimization
I am looking for a review or comparison of global optimization techniques where the gradient of the function is available and utilized to speed up search, like the following:
A hybrid descent method ...
- 1,033
5
votes
0
answers
235
views
Running maximum of $\sum_{1\leq k\leq n} X_i$ for Cauchy random variables $X_i$
Suppose $X_i$ are $\mathrm{Cauchy}(0,~\gamma)$ IID RV's and let $S_n=X_1+\cdots+X_n$ be their sum. Does an expression exist for the CDF of the running maximum up to an index $1 \leq k \leq n$?
Edit:
...
- 61
5
votes
0
answers
145
views
How to prove oracle properties in Fan and Li (2001) paper
I am studying Fan and Li's 2001 paper "Variable selection via nonconcave penalized Likelihood andits oracle properties" but I am having troubles understanding Theorem 1 proof (page 1359). I ...
- 1,348
5
votes
0
answers
387
views
Is my explanation of profile likelihood plots correct?
Using the metafor package in R to conduct a mixed-effects meta-analysis and meta-regression, I checked the profile likelihood ...
- 491
5
votes
0
answers
244
views
How to test if coefficients of two linear regressions differ in practice?
The Chow test is often suggested to test if the coefficients of two linear regressions differ or if a single linear regressions is more appropriate.
However, Chow test assumes equality of the ...
- 3,112
5
votes
0
answers
943
views
Computing likelihood of a mixed-effect model manually
My question has to do with how to manually compute the likelihood of a mixed-effect model.
I understand how to determine the likelihood of a fixed-effect model manually.
For example, if I make up ...
- 535
5
votes
0
answers
548
views
Learning hidden Markov model where transition/emission/initial probabilities aren't independent
I'm working on a problem that I've cast as an HMM, except that unlike the "traditional" case where the transition probabilities $a(i,j) = p(s_i = j \,|\, s_{i-1}=i)$, emission probabilities $b(j,o) = ...
- 59
4
votes
0
answers
76
views
Why do Likelihood Functions sometimes have Integrals?
Suppose we have a Mixed Effects Regression Model:
$$Y = X \beta + z b + \epsilon$$
$$b \sim N(0, \Sigma_{b})$$
$$\epsilon \sim N(0, \sigma^{2}I)$$
$$\Sigma_{b} =
\begin{bmatrix}
\sigma^{2}_{b1} & ...
4
votes
0
answers
246
views
How can you evaluate the representativeness of a sample for a given distribution?
Problem:
I am looking for a metric to find the representativeness of a sample for a given distribution, being the representativeness of a random sample as the degree of capacity of the sample to ...
- 41
4
votes
0
answers
75
views
Estimation of the density at the bound of the support of a real random variable
Let $X$ be a random variable with real values and with density $f$.
Assume the support $f$ is bounded with supremum $m$ and has a positive value at that supremum:
$$\forall x > m, f(x) = 0 \text{ ...
- 2,629
4
votes
0
answers
827
views
Avoid numerical overflow problem in likelihood due to $\exp$
There is a trick called exp-normalization which is used for dealing with overflow for ratios of the type $$\frac{\exp(x_i)}{\sum_j \exp(x_j)} = \frac{\exp(x_i-b)}{\sum_j \exp(x_j-b)}$$ by using the ...
- 6,724
4
votes
0
answers
381
views
AIC Comparison for MLM with Different Distributions
Thank you in advance for your time and consideration! I am a non-mathematically-inclined graduate student in communication just learning multilevel modeling.
We are running different models - some ...
- 61
4
votes
0
answers
76
views
How Does Variance Propagate From Likelihood Function To MCMC Posterior?
Suppose we are trying to obtain the posterior distribution of three parameters that influence a discretely observed population. The likelihood function is unfortunately intractable, as it is a mix of ...
- 51
4
votes
0
answers
108
views
Expectation Maximisation (EM) Algorithm
Some of my parameters do not have a closed form solution. Thus, for these parameters the M-step is implemented via a one-step Newton-Raphson update, i.e.,
\begin{equation}
\theta^{t+1} = \theta^t - \...
- 843
4
votes
0
answers
223
views
How to prove that the prior for which Bayes rule is also the minimax rule, is the least favorable prior?
I have read in the book Mathematical Statistics: A Decision Theoretic Approach by Thomas Ferguson that The prior for which the Bayes rule is also minimax rule, then that prior is Least favorable prior....
- 51
4
votes
0
answers
579
views
How can I compare parametric and semiparametric survival models?
On a given dataset, I am running a semiprametric Cox proportional hazards model, together with a series of parametric models (Weibul, gamma, lognormal, exponential, etc.).
How can I know which is ...
- 241
4
votes
0
answers
276
views
Log-likelihood calculation on separate test set
I'm looking for a "hack" in R that would allow me to calculate the log-likelihood of a GLM fit on a separate test set easily regardless of the distribution. For instance for a Gamma GLM, this is how ...
- 191
4
votes
0
answers
66
views
Brownian bridge to unknown via extremum
Suppose, I know what's the minimum $\min$ of a random walk $w_t$ in period $[0,\Delta t]$. I also know $w_0$ and $\sigma$. How to construct the Brownian bridge for the latter period?
I guess it's not ...
- 62.3k
4
votes
1
answer
120
views
Is it possible to use an estimated Likelihood without summary statistics for MCMC sampling?
I am currently trying to perform MCMC sampling using a (stochastic) model, for which I cannot derive a likelihood function, but which allows me to draw samples $y_\theta \sim p_{y|\theta}$, where $p_{...
- 2,671
4
votes
0
answers
116
views
Basics relative to chi square, likelihood, fits,
I'm confused to separate all the different meanings and connections.
The background of my question: On the one hand related to lmer models and on the other hand to the goodness of a fit. And their ...
- 3,493
4
votes
0
answers
125
views
Understand the empirical likelihood
When I read the talk notes by Owen about empirical likelihood,
http://www.ms.uky.edu/~mai/sta709/Owen2005.pdf
I am confused about the solve of solution when I read the page 24.
Let's say we want to ...
- 560
4
votes
1
answer
524
views
MLE estimation of Autoregressive Conditional Poisson model
The density of an Autoregressive Conditional Poisson ACP(p,q) model is defined as
$$ f(x | \lambda_{t}) = \frac{\lambda_{t}^{x}\exp[-\lambda_{t}]}{x!},$$
where
$$\lambda_{t} = \omega + \sum_{j = 1}...
- 5,156
4
votes
0
answers
1k
views
Maximum Prediction in Gaussian Process
A Gaussian process (GP) is defined as a collection of random variables with a joint Gaussian distribution (Rasmussen 2006). It is well known that given observations $\left \{ \mathbf{x},\mathbf{y}\...
- 2,214
4
votes
0
answers
4k
views
What is the exact log-likelihood of an AR(2) model?
Let's say we have the following AR(2) model:
$y_t=\phi_0+\phi_1y_{t-1}+\phi_2y_{t-2}+e_t, \; e_t\sim N(0,\sigma^2_e)$
with T observations in total.
Working out the conditional log-likelihood is ...
- 141
4
votes
0
answers
1k
views
Comparing likelihoods from non-nested models
Short: I have a series of joint probabilities (likelihoods) for how likely sample $Q$ belongs to group $K$. I need to compute a p-value describing how "significant" the "top" group is compared to ...
- 4,323
4
votes
0
answers
773
views
Approximating the marginal likelihood in Bayesian Model Comparison
Given some data $y$, my interest centers around a collection of models $\{\mathcal{M}_1,\mathcal{M}_2,\cdots,\mathcal{M}_L\}$ representing competing hypotheses about $y$. Each model $\mathcal{M}_l$ ...
- 4,042
4
votes
0
answers
157
views
Cauchy MLE Reliability of Asymptotic Results
I have the following regression model
$$
p_i = x'_i\beta +s \varepsilon_i
$$
with sample size $n \approx 150$ and 4 independent variables.
I have reason to believe and $\varepsilon_i$ is distributed ...
- 4,042
4
votes
0
answers
151
views
Likelihood Function for Complicated Transformations
Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
- 41
4
votes
0
answers
94
views
Estimating the mean from knowing the first n largest values
There is a sample of n values that are the first n largest values of a population.
Is there a way of getting any statistic such as mean or dispersion from such piece of information provided that the ...
- 1,087
4
votes
0
answers
386
views
Expectation of density ratio of two iid variables
Let $X \sim N(0,1)$ and $Y \sim N(0,1)$ be independent RVs and let $f$ be their density function. I'd like to compute the expectation of the density ratio
\begin{align}
\mathbb{E}\left[\frac{f(X)}{f(Y)...
- 308
4
votes
0
answers
79
views
Non-Analytic extrapolation
I have some samples of a stable real-world process. Its is polymodal, and does not cleanly fit any of the "textbook" analytic distributions. I need to make very accurate estimates of the maximum ...
- 9,853
4
votes
0
answers
114
views
fitting the tail of a distribution in a regression tree
I have 3 integer valued time series $a_t$, $b_t$ and $y_t$ with $k$ observations. I want to fit $y_t$ with the 2 first, and for that purpose I use a regression tree like this:
test all combinations ...
- 141
4
votes
0
answers
936
views
Interpretation of a log likelihood function for PROC NLMIXED in SAS
I have a data set of skewed nutrient intake values, from around 7800 individuals, of whom around 3000 had two measures of daily nutrient intake (the others only had one measure), so this is a repeated ...
- 3,910
3
votes
0
answers
47
views
Is there an analytical solution to the distribution of a sum of observations drawn from a Frechet distribution?
Let $X_i$ be an iid draw from a Frechet distribution. Let $\alpha_i \in \mathbb{R}$.
Is there an analytical expression of the distribution of $\alpha_1X_1 + \alpha_2X_2 + \alpha_3X_3$? That is, can I ...
- 31
3
votes
0
answers
137
views
Function proportional to the log likelihood for the Gaussian distribution
The following question is crossposted from MathStackExchange upon recommendation from the MSE community and a lack of responses on my post over there.
Consider the following problem from a course on ...
- 243
3
votes
1
answer
486
views
Simulating likelihood ratio test (LRT) pvalue using Monte Carlo method
I'm trying to figure out my assignment to simulate lrt test p-value output using the Monte Carlo method. As far as I understand, the lrt test is supposed to test for "better", more accurate ...
- 143
3
votes
0
answers
113
views
What likelihood to use to model sample means from a Pareto-like distribution?
Suppose there is a random variable with Lomax (Pareto Type II) probability density
$$
P(x; c) = \frac{c}{(1 + x )^{c + 1}}, \quad x \ge 0, c > 0.
$$
Let's draw n_samples=30000 samples of length ...
- 31
3
votes
0
answers
646
views
What is the Cox-Reid adjusted likelihood used for?
In Love et al 2014 they use an adjusted likelihood function that I would like to understand better in terms of its use case (i.e. when might I wish to use it). They say:
We then maximize the Cox-Reid ...
- 9,680
3
votes
0
answers
75
views
What can a p-value (& sign) tell me about the marginal posterior distribution of a model parameter, and when?
EDIT: The tl;dr here would broadly be: given that both frequentist standard errors and a quadratic approximation of a Bayesian joint posterior can be obtained from the square root of the diagonal ...
3
votes
1
answer
299
views
Method of collecting and comparing outliers from sets of sets of populations
Background
I am a PhD student co-supervising a Master's student in our lab. I am mostly familiar with discrete mathematics, signal processing, and programming simulations. My statistics background ...
3
votes
0
answers
106
views
On a heuristic example to show that the likelihood function "does not contain all the information in the data."
As part of self-study, I am reviewing arguments I found tricky from Larry Wasserman's course notes "Intermediate Statistics Fall 2016, Lecture Notes 6: Likelihood". In particular, I have a ...
- 2,630
3
votes
0
answers
309
views
How to construct the likelihood function of compound Poisson process?
Since in the compound Poisson process (CPP), the jumps occur according to the Poisson process with intensity $\lambda(t)$. The jumps size is iid random variables and itself independent of the Poisson ...
- 51
3
votes
0
answers
309
views
Maximum Likelihood - Normal Errors - When is the Jacobian needed?
I am considering the following non-linear model
$$h(z) - \lambda_0 - \lambda_1 z - \lambda_2x = v$$
where $v \sim \mathcal N(0,\sigma^2)$ unobserved error and where $\lambda_j$ are unknown ...
- 5,565
3
votes
0
answers
64
views
Are they cheating?
I'm trying to determine the probability that three students cheated on a recent exam.
In a nutshell: What are the odds that three students sitting next to each other in a class of 22 could select (...
- 31
3
votes
0
answers
555
views
Convergence rate of the maximum of Weibull random variables to a Gumbel distribution
Given a sequence of iid samples $X_1, \dots, X_n,$ where each $X_i$ comes from a Weibull distribution with shape parameter $k$ and scale parameter $\lambda$. Then it is a well-known result that the ...
- 93
3
votes
1
answer
61
views
Problem with two correlated random normals
Imagine you have a two-dimensional multivariate normal random variable with $\mu = [0, 0]$ and $\Sigma\ = \begin{bmatrix}1 & r\\r & 1\end{bmatrix}$. (Conceptually, you have two random normal ...
- 131