Questions tagged [profile-likelihood]

The profile likelihood function is an inference function constructed from the likelihood function. If the likelihood function depends on many parameters and only some are of interest, then the other parameters are removed "by concentration", which means that they are "maxed out".

Filter by
Sorted by
Tagged with
0 votes
0 answers
28 views

Profile (quasi-)likelihood score tests

Suppose I have a log-likelihood or quasi-log-likelihood for my data in terms of the parameter vectors $\theta$ and $\psi$: $$L(\theta;\psi)=\frac{1}{T}\sum_{t=1}^T{\log{f(y_t|\theta;\psi)}}.$$ (I am ...
cfp's user avatar
  • 535
0 votes
0 answers
37 views

Marginal likelihood and profile likelihood: Gaussian case

Consider a model such that the log-likelihood function of a $n$-dimensional parameter $\theta$ is given by (can be approximated by) $$\tag{1} L(\theta)=L(\widehat \theta) -\frac12 (\theta - \widehat \...
G. Gare's user avatar
  • 73
1 vote
1 answer
72 views

Profile likelihood vs quadratic log-likelihood approximation

I want to compare two alternative approaches for evaluating the uncertainty of the multi-dimensional MLE $\widehat \theta$ based on a log-likelihood function $l$: Compute a Fisher-information-based ...
G. Gare's user avatar
  • 73
0 votes
0 answers
60 views

Approximating profile likelihood confidence intervals when I only have a score function and not a likelihood

I'm working on a modeling problem where I can define a score function that looks a lot like a binomial likelihood, but the model isn't really binomial. I'd like to use profile likelihood to estimate ...
famulare's user avatar
  • 362
1 vote
0 answers
32 views

Maximum Likelihood with a sign restriction

Suppose that we have a log-likelihood function of five parameters and an observed data sample $y=[y_1,\ldots,y_N]$: $$\mathcal{l}(\beta_1, \beta_2, \beta_3, \beta_4, \beta_5;\;y) =\log f_Y(y;\;\beta_1,...
MinChul Park's user avatar
7 votes
1 answer
1k views

Tweedie Dispersion Parameter Estimation Methods

In the book Generalized Linear Models with Examples in R - Dunn and Smyth, in Chapter 6.8, it is recommended to use the Pearson estimator of the dispersion - "This makes the Pearson estimator the ...
Kozolovska's user avatar
  • 1,333
1 vote
0 answers
75 views

Confidence Interval for factor loadings of 2PL model

I estimated a 2PL model obtained for some dichotomous data presenting answers of 12 "yes/no" questions from about 200 individuals. Now I want to calculate the confidence intervals for the ...
mugdi's user avatar
  • 90
6 votes
1 answer
179 views

justification for 'population prediction intervals'?

Suppose we are living in a frequentist world and want to compute confidence intervals on some quantity that is a complicated function of the parameters $q_1 = f(\Theta)$ (i.e., there's no closed-form ...
Ben Bolker's user avatar
  • 43.1k
2 votes
2 answers
265 views

Relationship Between "Profile Likelihood" and "EM Algorithm"?

I was reading Rao (2017) (Ch3) on profile likelihood. An example is provided which shows how the parameters of a Weibull Distribution can be estimated using the "profile likelihood approach"...
stats_noob's user avatar
  • 8,502
1 vote
3 answers
470 views

How to correct Chi-square's p-value when working with very unbalanced contingency tables?

I'm studying the association between a rare disease and smoking. Because the disease is rare, my contingency table is highly unbalanced with way more Non-Diseased than Diseased individuals, ...
je2018's user avatar
  • 51
-1 votes
1 answer
490 views

How to get the $95\%$ Wald test confidence interval for $\theta$? [closed]

Suppose that iid random samples $X_i$ from a discrete CDF $F(x)$ on $\{x_1,\dots, x_n\}$ with mean $EX=\theta$. We want to estimate $F(x)$. We consider empirical likelihood for $F(x)=\sum_{i=1}^n p_i ...
Hermi's user avatar
  • 687
1 vote
0 answers
45 views

Pooling Profile Penalised LTRs in multiple imputation

I am analysizing data from a clinical trial. I used multiple imputation to impute the (binary) outcome variable, which is the only variable with missing data. All of the covariates are categorical and ...
Simone's user avatar
  • 11
3 votes
1 answer
508 views

Calculation of p-values in logistf package

I'm trying to understand profile likelihood used in the logistf package. In the code, it seems the p-values are calculated by: ...
jon's user avatar
  • 31
5 votes
3 answers
2k views

How to easily obtain the profile likelihood 95% confidence interval for a predicted value in a logistic regression model in R?

I'm a fish biologist and we often use a logistic regression to estimate what we refer to as the L50, i.e. the length at which you expect one fish out of two (50%) to have developed gonads. How to ...
julienbio99's user avatar
5 votes
1 answer
459 views

Meaning of Invariance of Maximum Likelihood Estimator

In Casella-Berger, the invariance of MLE is defined as: Assuming that $\hat{\theta}$ is MLE of $\theta$, then for any function $\tau$, $\tau(\hat{\theta})$ is MLE of $\tau(\theta)$. In the case of a ...
Artem Moskalev's user avatar
4 votes
2 answers
183 views

Reference for profile likelihood estimation

I wonder if there is a book talking about profile likelihood in detail, about the parameter computation procedure (grid search, newton-raphson method, EM algorithm); also, about estimation matters: ...
wut's user avatar
  • 107
1 vote
1 answer
568 views

How can concentrated (profile) log marginal likelihood be used to optimize the mean and scale(outputscale) parameters in Gaussian Process Regression?

The log marginal likelihood which is used in Gaussian Process Regression comes from a Multivariate Normal pdf Gaussian Processes for Machine Learning, p.19, eqn. 2.30, Surrogates, Chapter 5, eqn. 5.4 \...
m-julian's user avatar
0 votes
1 answer
168 views

Finding a confidence interval for difference of proportions

Let two independent random variables, $Y_1$ and $Y_2$ that have binomial distribution have parameters $n_1 = n_2 = 100$, $p_1$ and $p_2$, respectively, be observed to be equal to $y_1 = 50$ and $y_2 = ...
user avatar
6 votes
2 answers
691 views

Plot profile likelihood

So, I have a generated data from a nonlinear model $$y_i = \eta_1 - 2\theta \eta_2 x_i + \eta_2 x_i^2 + e_i,$$ where $e_i \sim N(0,\sigma^2)$. What I want is to find the profile log-likelihood for $\...
Jeronimo B's user avatar
0 votes
0 answers
148 views

R: Getting Wrong Profile Likelihood Confidence Interval Estimates

I am trying to estimate the profile likelihood confidence interval (CI) of the parameters ($\xi$, $\sigma$) of the Generalized Pareto Distribution (GPD). However, the lower estimate (left CI) of $\xi$ ...
Blg Khalil's user avatar
4 votes
1 answer
256 views

How to interpret my profile likelihood plots (three-level meta-analysis)?

I conducted a three-level meta-analysis to test effects of non-driving related tasks on take-over quality in highly automated driving. When it now comes to interpretation of the results, I struggle to ...
timc's user avatar
  • 51
1 vote
0 answers
123 views

Can't reproduce confint profile likelihood results

I'm trying to follow this answer here and the linked paper. But I can't seem to get the same results as the confint gives. Sometimes I get something very simmilar, ...
Maverick Meerkat's user avatar
1 vote
0 answers
138 views

Standard errors of parameters in hyperparameter tuning

I have a model with the parameters $\textbf{θ} = (\textbf{φ}, ψ)$, which I trying to estimate. There is an out-of-the-box solution for ML estimating $\textbf{φ}$ if $ψ$ is fixed. If I treat $ψ$ as ...
Prsed's user avatar
  • 11
7 votes
4 answers
1k views

How to create an interval estimate based on being within some percentage of the maximum value of the likelihood function?

When we do maximum likelihood estimation of a (let's say scalar) parameter $\theta$, we get a point estimate wherever the likelihood function is maximized. However, if we want an interval estimate, ...
Dave's user avatar
  • 61k
2 votes
3 answers
353 views

Compare which of two groups is MORE similar to a third group?

I know I can use an ANOVA (+ post-hoc tests) to determine the equality (or inequality) of three groups: (i.e., H0: each of 3 means are equal; H A: at least 1 of 3 means is not equal). However, how ...
theforestecologist's user avatar
1 vote
0 answers
62 views

error code in finding confidence interval? [closed]

I optimized parameters from a function using mle2 from bbmle package. After I got the values, I tried to find profile likelihood ...
Asunray's user avatar
  • 11
9 votes
4 answers
2k views

Confidence interval for difference between two predicted probabilities in R

In R, I have estimated a logistic regression and calculated two predicted probabilities (with 95% confidence intervals) using the code shown: ...
Rushmore's user avatar
  • 111
3 votes
2 answers
1k views

Profile likelihood confidence interval proof

I have read that for null hypothesis $H_0 : \beta = \beta_0$, the likelihood ratio statistic from profile-likelihood is $$LR = 2 (\log L_p(\hat{\beta_p}) - \log L_p(\beta_0))$$ where $\hat{\beta_p}$ ...
JDoe2's user avatar
  • 670
0 votes
0 answers
30 views

How to practically calculate confidence interval from the following confidence equation?

How to use the following equation to calculate the confidence interval: $$CI_{j,\alpha}(y) = \left\{p | -2PL_j(p) \leq \min_\theta - 2LL(y|\theta) + \Delta(\alpha) \right\}.$$ Here, CI is confidence ...
Paichu's user avatar
  • 101
2 votes
1 answer
113 views

How is to maximize a function $f(x,y)$ for values of $y$?

Maximum Likelihood Method: Likelihood function asks what value of parameter, $\theta$, makes the data set most probable. Let the distribution is $$f(x;p)=\binom{3}{x}p^x(1-p)^{n-x},\quad x=0,1,2,3.$$ ...
time's user avatar
  • 1,587
6 votes
1 answer
330 views

Profile Likelihood: why optimize all other parameters while tracing a profile for a partitcular one?

Profile likelihood is sometimes used to get estimates for the confidence limits of parameters from an n-dimension parameter fit to a model. It can be used for example instead of Monte Carlo estimation....
rhody's user avatar
  • 215
7 votes
2 answers
2k views

MLE: Marginal vs Full Likelihood

Suppose I have a statistical model with parameters $\boldsymbol{\theta}=\{\theta_1,\theta_2,\dots,\theta_n\}$ of which only a single parameter, say $\theta_1$, is of interest to me. Suppose also that ...
Estacionario's user avatar
2 votes
0 answers
134 views

Profile log-likelihood function empirical similarity

I´m trying to deduce the profile log-likelihood function in "Rule-Based and Case-Based Reasoning in Housing Prices" (page 14) by Gayer et al. (2007). Given is: $$\mathbf{y}=\alpha \mathbf{1} + \...
To Mate's user avatar
  • 87
4 votes
1 answer
2k views

Confidence intervals for GLMM: bootstrap vs likelihood profile

I've built a relatively large negative binomial GLMM (~50000 observations, 10 covariates in conditional model, 1 covariate as zero-inflation model, 3 random intercepts (650, 26, 26 levels in each ...
sacuL's user avatar
  • 380
2 votes
2 answers
782 views

In a GLM, are the Maximum likelihood estimators for the regression coefficients always normally distributed?

I'm doing a Poisson regression and using the confint function in R to generate confidence intervals for my regression coefficient. These result in different ...
WBM's user avatar
  • 423
2 votes
1 answer
162 views

Is the profile likelihood (dependent on one parameter) always a concave function?

Let $\theta = (\theta_1, \theta_2, \dots)$ be a vector of parameters, and let $L(\theta) = L(\theta|y)$ denote the likelihood function with respect to observed data $y$. Following the notation from ...
Alex's user avatar
  • 4,340
5 votes
2 answers
317 views

Confidence interval on the percentage difference of two binomial distributions

I have survey data for women and men following a binomial distribution. Their means are $p_1$ and $p_2$ respectively. I have calculated $(p_1-p_2)/p_2$ and would like to attach a confidence interval ...
Big AL's user avatar
  • 151
5 votes
0 answers
370 views

Is my explanation of profile likelihood plots correct?

Using the metafor package in R to conduct a mixed-effects meta-analysis and meta-regression, I checked the profile likelihood ...
Johanna's user avatar
  • 481
5 votes
1 answer
461 views

Confidence intervals with penalized likelihood

I am trying to perform parameter estimation using something like a maximum likelihood ratio method, however I need to add a penalty term to constrain nuisance parameters which describe certain ...
Ben Farmer's user avatar
2 votes
0 answers
1k views

Firth's method for logistic regression - interpretation of R output

I have a multivariate, multinomial logistic regression model with exclusively continuous covariates. After some examination, I found that I had a problem of quasi-complete separation. The textbook ...
StatsSorceress's user avatar
2 votes
1 answer
2k views

Existence and uniqueness of MLE

I am aiming to show that the MLE of $(\alpha, \lambda)$ for $X_1, \dots, X_n \sim \Gamma(\alpha,\lambda)$ exists and is unique, under the assumption that the $X_i$ are all positive and unequal. I am ...
Orest Xherija's user avatar
3 votes
1 answer
455 views

Can we calculate the MLE of $\mu$ and $\sigma^2$ of normally distributed data using the profile likelihood approach?

My definition of profile likelihood is that given a vector of parameters $(\theta_1, \theta_2)$, with $\theta_1$ the parameter of interest, and $\theta_2$ a nuisance parameter -- If $L(\theta_1, \...
user1398057's user avatar
  • 2,345
4 votes
1 answer
215 views

How do I compute the estimated values of x for a beta-binomial distribution?

I understand how to set up a binomial probability distribution. I'm trying to extend my understanding to the beta-binomial. On Wikipedia, there is a beta-binomial example given at https://en....
Jim Lewis's user avatar
  • 536
1 vote
0 answers
466 views

Calculating the amount of underestimation

Coverage rate for a parameter is 91.2%, and the nominal coverage rate is 95%. If the confidence interval is based on asymptotic standard normal, then the amount of coverage 91.2% implies that the ...
ABC's user avatar
  • 1,705
1 vote
0 answers
246 views

Restricted Maximum Likelihood Estimation for Linear Mixed Model

The maximum likelihood estimation procedure for linear mixed model is described in this link. It seems to me that something is wrong there. In their Restricted Maximum Likelihood section the first ...
user81411's user avatar
  • 771
5 votes
1 answer
393 views

Confidence interval for GLM or the maximum of a function?

Imagine I have a set of (xi,yi) measures. I can show it on a scatter plot I want to choose the value of x that maximizes y, or I could fit a function and find the values of the parameters that ...
skan's user avatar
  • 1,064
2 votes
1 answer
769 views

Profile likelihood

I am considering a normal distribution with mean $\beta_1 + \beta_2\exp(-\phi x)$ and variance $\sigma^2$, i.e. $y \sim N(\beta_1 + \beta_2\exp(-\phi x), \sigma^2) $. My aim is to calculate the ...
denby47's user avatar
  • 87
1 vote
0 answers
700 views

Discordant significance of OR’s confidence interval and p-value in glm(quasibinomial) model [R]

I’m currently trying to test if differences in proportions of people infected by malaria (RDT positive) between clusters with high or low coverage of control intervention are significant. Therefore I’...
ThomasK's user avatar
  • 11
5 votes
1 answer
334 views

Determining if two profile likelihood curves are significantly different

I want to compare two profile likelihood curves and determine if they are significantly different from one another. For example are the following curves significantly different from one another: I ...
Ben Haley's user avatar
  • 311
2 votes
0 answers
218 views

Parameter distribution from profile likelihood

This question is regarding profile likelihood to obtain CI of parameters. There is a published parameter values that best fits (using minimum chi square)the data for some mathematical model say y=f(...
Sheeja Krishnan's user avatar