All Questions
2,181 questions
0
votes
0
answers
13
views
Title: Handling Skewed Importance Sampling Weights for High-Dimensional Log-Likelihoods
Question:
I am performing importance sampling (IS) for a Bayesian inference problem with the following setup:
1. Data and Model
My data has ( D = 1300 ) dimensions.
The log-likelihood, $ \log p(x \...
0
votes
0
answers
13
views
Convolution with a pathological distribution part 2
This post is a follow-up to this previous one, based on what I learned from this second one.
Problem Definition
Consider a polygon with vertices $V_1,\dots,V_n \in \mathbb{R}^2$ and let
\begin{aligned}...
- 473
3
votes
0
answers
47
views
Is there an analytical solution to the distribution of a sum of observations drawn from a Frechet distribution?
Let $X_i$ be an iid draw from a Frechet distribution. Let $\alpha_i \in \mathbb{R}$.
Is there an analytical expression of the distribution of $\alpha_1X_1 + \alpha_2X_2 + \alpha_3X_3$? That is, can I ...
- 31
1
vote
0
answers
33
views
What makes a curve a good fit in the context of logistic regression
As I wanted to gain a better intuition between why separation is a problem in the context of logistic regression, I did create in R two models, one where y is perfectly separated at $x=5$, and one ...
- 81
2
votes
1
answer
101
views
Is there any question as to what the likelihood function for a geometric distribution is? [closed]
Because I've read it is either
$g(x) =
\prod_{i=1}^∞\ p(1-p)^{x_i}$
or
$g(x) =
\prod_{i=1}^∞\ p(1-p)^{x_i-1}$
So I'm really confused.
Reference: https://math.stackexchange.com/questions/4429910/...
- 23
0
votes
0
answers
38
views
Outer product approximation of derivatives of likelihood
Lately, I have been reading Muthén's paper, "Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika,...
- 1
1
vote
0
answers
22
views
Convert units, get different results when fitting extreme value distribution with extRemes
I am using the fevd() and lr.test() functions to examine precipitation using the extRemes R ...
- 11
0
votes
0
answers
17
views
Proof of the simplification of the likelihood function [duplicate]
There are many references that quote that under the assumption that $x_1,x_2,\ldots x_n$ are i.i.d., the likelihood function can be simplified as follows:
$$P(x_1,x_2,\ldots ,x_n|\theta)=P(x_1|\theta)...
- 1,150
0
votes
0
answers
20
views
Why is the logarithm of the likeilhood function significant? [duplicate]
In maximum likelihood we often use the logarithm of the likelihood function rather than the likelihood function directly. One reason for this is that it is convenient for numerical optimization ...
- 146
1
vote
0
answers
33
views
Can we use fisher scoring on a restricted likelihood function?
I have a question on how to optimize the RMLE in mixed effects regression models.
Starting with a mixed effects model:
$$y = X\beta + Zu + e$$
$$u \sim N(0, G), \quad e \sim N(0, R)$$
Where:
$y$ is ...
- 433
5
votes
1
answer
311
views
How to prevent negative variance estimates in likelihood optimization?
I have this regression model in which 100 people have 20 measurements taken:
$$ y_{ij} = \beta_0 + \beta_1 x_{ij} + \beta_2 t_{ij} + u_i + \epsilon_{ij} $$
Where:
$ i = 1, 2, ..., 100 \text{ (patient ...
- 433
0
votes
1
answer
43
views
Bayes Factor for two exact hypotheses on normally distributed data with unknown variance
Let's assume that the observations $x_i$ are normally distributed
$$
x_i \sim N(\mu, \sigma^2)
$$
and that the variance $\sigma^2$ is unknown. The Bayes Factor to compare two point hypotheses on the ...
- 63
0
votes
0
answers
36
views
Interpretation of $f(x;c\theta)$
I am trying to understand the meaning of the notation $f(x;c\theta)$, where $c$ is known constant. For example, assume the random variable $X$ is Poisson-distributed. When we write $f_X(x;2\theta)$, ...
- 1
1
vote
0
answers
48
views
Hessian for log likelihood of regression with respect to covariance matrix
I am interested in the residual covariance of a multivariate regression model. The regression is
$$
Y_t = X_t \beta + \varepsilon_t
$$
and I have a log likelihood as follows
$$
\mathcal{L}(\Sigma) = \...
- 11
1
vote
1
answer
78
views
Expectation of the minimum of random variables (Exponential + Erlang)
Consider the following random variable
$$
Z=\min_i\{X_i+Y_i\}
$$
for $-n\leq i\leq n$, where $X_i\overset{\mathrm{iid}}{\sim}\text{Exp}(\lambda)$, $Y_i\overset{\mathrm{iid}}{\sim}\text{Erlang}(|i|,\...
- 150
3
votes
1
answer
85
views
Maximum of two independent gamma variables
Let $X_1$, $X_2$ be two independent random variables with different gamma distributions, and $X = \max\{X_1, X_2\}$.
Are there known results for the distribution of $X$? Actually I only need to know $\...
- 1,191
1
vote
0
answers
12
views
Analysis of the relative likelihood with parametric bootstrapping
Using parametric bootstrapping, I find that the relative likelihood of model A is $l_A$ and of model B is $l_B$. Repeating the analysis several times yields a distribution of likelihood values for ...
0
votes
0
answers
25
views
Linearity of and pointwise equality in expectation of min() function
Consider the expressions $f = c + s*E[min(a/s, X)]$ and $g = E[min(c + a, c+sX)]$ where
c >= 0
0 < s <= 1
a >= 0
X ~ Poisson($\lambda$/s)
I'd like to think that $f = g$, reasoning as ...
3
votes
1
answer
42
views
Maximum Likelihood Estimation for Pairs of Observations
I have $n$ pairs of observations $(x_i,y_i)$, where each $y_i$ is distributed according to $\text{Pois}(\theta x_i)$, and I wish to do a maximum likelihood estimation for $\theta$ only based on this ...
4
votes
2
answers
75
views
Confidence regions of optimized parameters in maximum log likelihood fits
I am using a numerical optimization algorithm to maximize a log-likelihood function, $\mathcal{L}$. The log-likelihood function has a fixed number of parameters, $\{\theta_i\}$. These parameters are ...
- 333
0
votes
0
answers
34
views
Flattening a likelihood
Background
Let $y_1,y_2,\dots,y_K$ be a sequence of measurements.
I've derived a likelihood $\mathcal{L}(y|i)$ to solve a classification problem via the Bayesian classifier
\begin{equation}
p_k(i)=\...
- 473
0
votes
0
answers
16
views
Bishop gradient calculation
In section 3.1.1 of Pattern Recognition and Machine Learning by Christopher Bishop, it is written that
$$\ln p(\mathbf{t} | \mathbf{w}, \beta) = \frac{N}{2} \ln \beta - \frac{N}{2} \ln (2 \pi) - \beta ...
- 11
1
vote
0
answers
37
views
How can I measure Monte Carlo convergence in distribution with heavy tails?
I'm performing a Monte Carlo study on a simple agent based simulation, and I'm trying to formulate a heuristic for the number of MC samples to use. I'm able to measure convergence of statistics like ...
0
votes
0
answers
36
views
Fitting a regression line which passes through the anchor point
In our setting, we have data $X_1, \ldots, X_n$, which can be ordered as $X_{1,n}\leq X_{2,n}\leq \ldots \leq X_{n,n}$ and we have the points $(-\log (1-\frac{i}{n+1}), X_{i,n})$ for $i=1,\ldots,n$.
...
- 656
0
votes
0
answers
23
views
Estimated degrees of freedom of a GAM comparison
I want to evaluate the significance of including two smooth terms in a GAM by comparing that model to a baseline model that does not include those predictors. A simulation of this case can be ...
- 1
1
vote
0
answers
36
views
Comparing GLMM with LMM with -2*log-likelihood
Is it possible/recommended to compare the -2*Log-Likelihood (-2LL) value of a Generalized Linear Mixed Model (GLMM) against the -2LL value (and/or AIC/AICC/BIC) of a Linear Mixed Model (LMM) with the ...
- 135
0
votes
0
answers
42
views
Manually calculate loglikelihood of fitted model
My goal is to calculate the loglikelihood of a fitted model on some unseen data.
To this end I defined a function that calculates the loglikelihood by hand on some new data. However, as a sanity check ...
- 1
0
votes
0
answers
21
views
Mixture of factor analyzers: Correct expression for likelihood function?
I'm reading these notes about mixtures of factor analyzers, where the following generative model is described:
The conditional distribution of the data $x$ is stated (in Sec 3, eqn 9) as:
$$
P(x \mid ...
- 43
1
vote
1
answer
58
views
Identify maximum in quadratic regression
I am looking for a way to find the maximum in a quadratic regression.
Specifically, I have two variables X and Y. Y is a discrete and commonly used scale representing the severity of a disease, ...
- 433
3
votes
2
answers
253
views
Regression and independent random vectors
Lets consider that data samples are generated from random vectors $(X_1, Y_1)...(X_N, Y_N)$ of cross-sectional data. For regression one usually assumes that the error distribution is I.I.D. normally ...
- 321
0
votes
0
answers
39
views
Null distribution for 1 vs 2 source detection likelihood ratio test
I am wanting to construct a likelihood ratio test something like the following:
I have $N$ observations, let's say in 2D. Each observation comes with its own uncertainty estimate, i.e. from its own ...
- 279
1
vote
0
answers
37
views
Gaussian linear model marginal likelihood under g-prior
Consider a Gaussian linear model with an $ n \times 1 $ outcome vector $ y $ and an $ n \times p $ matrix of centered predictors $ X $:
$ y = \iota\alpha + X\beta + \varepsilon \quad \quad \varepsilon ...
- 730
0
votes
1
answer
48
views
Monte Carlo method for likelihoods ratio density estimation
I recently started reading Stephen Kay's Fundamentals of Statistical Signal Processing - Detection Theory (Volume II) and there is something I do not fully understand about likelihoods and hypothesis ...
- 103
2
votes
1
answer
91
views
Birnbaum's Theorem: Strong belief in a model $\implies$ the likelihood function must be used as a data reduction device?
Working through understanding section 6.3.2 (pg. 292-294) in Casella and Berger's Statistical Inference (2nd-ed).
The following definitions and principles are given:
Definition (Experiment): An ...
5
votes
2
answers
346
views
What is the median of the minimum or maximum of multiple samples?
Suppose I have a variable with a known distribution, and suppose I sample that variable k times and record the minimum. If I repeat this many times, will the median of the minimum converge to a ...
- 55
0
votes
1
answer
35
views
Why my Rho-square on Multinomial Logit Model (McFadden) so small?
When I'm using MNL, and try to find my rho square, it's found out to be so small. It is $0.0139$. For a good fit model, the rho square has to be between $0.2$-$0.4$. Is there any reason why it's so ...
- 1
2
votes
1
answer
39
views
analytical asymptotic approximation of the expected maximum, mean, and minimum distance of nearest neighbours in unit ball
Say I uniformly at random distribute $x = n^3$ (independent identically distributed) points in a ball of radius $r=1$ in $\mathbb{R}^3$.
What can be said about the expected maximum, minimum, and mean ...
- 277
3
votes
1
answer
230
views
Basic question about deriving MAP estimator
Say we have a random process $X(t, u)$ parametrized by $t$ and $u$ that generates data $x$. We also have a prior on $u$, $p(u)$.
Am I correct in stating that the expression to find the maximum a ...
- 43
5
votes
1
answer
216
views
In a sum of high-variance lognormals, what fraction comes from the first term?
If $X_i \overset{\textrm{iid}}{\sim} \text{Lognormal}(0, \sigma^2)$ for $i=1,\ldots,n$ and $Y_1 = X_1 / \sum_{j=1}^n X_j$, then I would expect that a particular* limiting distribution of $Y_1$, ...
- 195
4
votes
2
answers
111
views
Confusion over Fisher-scoring algorithm
Given a probability model $f(X;\theta)$ and a set of i.i.d. observations $x_1,\ldots,x_n$ which we assume to be drawn from some true parameter $f(X; \theta_0)$, we can perform maximum-likelihood ...
- 316
0
votes
0
answers
17
views
Declustering impact, stationarity and discretization
I have a seasonal time series, and I am considering declustering (before any other preprocessing steps) it using runs declustering. If I observe an extremal index of 1, can I claim that my data is i.i....
- 1
0
votes
0
answers
21
views
Correlation Coefficent is higher when likelihood of an event is lower, how does this occur?
I have different variables that I am interested in if they influence pass/fail rates.
To see what variables I might use as a leading indicator, I've pulled different variables such as "tutoring&...
0
votes
2
answers
47
views
Negative log-likelihood, high BIC, high R-squared, low error, using a difference-in-differences (DiD) methodology [closed]
I am trying to see the impact of Brexit on UK imports. My dependent variable are EU exports to the rest of world. I have monthly data from 2013 to 2023, also data is in billions of GBP.
When I do ...
- 1
0
votes
0
answers
30
views
How to obtain likelihood ($P(B/R)$ given the prior $P(R)$ and the posterior $P(R/B)$
I am working on a topic related to multiple-choice response. I would like to measure the efficiency of the information source (or a student’s information search) and I believe Bayesian statistics is ...
- 21
1
vote
0
answers
34
views
Closed-Form Lambda for Yeo-Johnson-Transformed Normal-Inverse-Gaussian-Distributed Random Variables
I would like to know whether there exists a closed-form solution for the $\lambda$-parameter that maximizes the log-likelihood function of Yeo-Johnson transformed random variables that (before the ...
- 435
2
votes
0
answers
41
views
Likelihood from posterior [closed]
This question is strange and perhaps silly but it would be very useful for my research. Is there any method to find the likelihood given a prior distribution and its corresponding posterior ...
- 21
0
votes
0
answers
16
views
Convergence of a Bayesian classifier
Background
Let $y_k$ be a noisy measurement at time $k$ and let $\{p_{k-1}(i)\}_{i=1}^n$ be (a discrete) prior probability distribution. Using Bayes rule, one can update the prior in function of $y_k$ ...
- 473
0
votes
0
answers
55
views
Does the mean of the maxima of a set of distributions converge?
This question is related to a recent one I posted. In that question I ask what statistic might best represent the central tendency of the true discrete distribution of a property for a sample for ...
- 165
3
votes
3
answers
125
views
What statistic best estimates the sample mean in case of missing data in a distribution?
I have samples of particles and am interested in the particle lengths. The problem is that the samples are assessed using image analysis. As the particles overlap, the measurements are incomplete and ...
- 165
0
votes
0
answers
19
views
Comparing GLMs with different fitted distributions
I have a scenario where I need to compare some generalized liner models (with same link function, target variable, but not necessarily nested) with k fold cross validation, using a cost function to ...
- 25