3
$\begingroup$

Let's consider the family of transformations given by $$g_a(Y)=\begin{cases} \frac{e^{aY}-1}{a} & \text{ for } a\neq 0 \\ Y & \text{ for } a=0 \end{cases}$$ for $Y\in\mathbb{R}$. Analogous to the estimation of the Box-Cox parameter $\lambda$, the parameter $a\in\mathbb{R}$ can be estimated using a profile likelihood approach. Let $Y_1,...,Y_n\in\mathbb{R}$ be independent responses together with corresponding predictors $\mathbf{x}_1,...,\mathbf{x}_n\in\mathbb{R}^n$. We assume for $a$ that there exist $\textbf{b}\in\mathbb{R}^p$ and $\sigma^2>0$ such that $g_a(Y_i)\sim N(\textbf{x}_i^T \textbf{b},\sigma^2)$ for $i=1,...,n$.

Here is my questions:

  1. Can we derive such a density function $f_Y$ of the untransformed observations $Y_i$. If so, would it be $$f_{Y_i}(\textbf{x})=\frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{1}{2}\cdot \frac{(\textbf{x}-\textbf{x}_i^T\textbf{b})^2}{\sigma^2}}$$

  2. How would we find the log-likelihood function $\ell(a,b,\sigma^2;y_1,...,y_n)$? I know that $\sum_{i=1}^n \log(f_{Y_i}(y_i))$.

Thanks in advance.

$\endgroup$
1
  • 3
    $\begingroup$ Your (1) states that $Y_i$ and $g_a(Y_i)$ are identically distributed. That's clearly the case for $a=0,$ but do you believe that continues to be true for nonzero $a$? Maybe there are some typographical errors in your formulas? $\endgroup$
    – whuber
    Dec 13, 2022 at 15:54

1 Answer 1

4
$\begingroup$

Only one calculation is needed: we have to differentiate $g_a.$ Everything else is just substitutions and taking logarithms.

For convenience, let $Z=g_a(Y)$ and write $\mu = \mathbf x_i^\prime \beta.$ You say $Z$ has a Normal distribution, which means it has a density function given by

$$f(z) = \frac{1}{\sigma\sqrt{2\pi}}\, \exp\left(-\frac{1}{2\sigma^2}(z-\mu)^2\right)$$

The transformation $y\to g_a(y) = z$ is everywhere increasing, differentiable, and one-to-one. For $a\ne 0$ its differential is

$$\mathrm d z = \mathrm d\left(\frac{e^ay-1}{a}\right) = e^{ay}\mathrm dy.$$

Consequently, $Z$ has a continuous distribution supported on the image of $g_a$ with probability element

$$h(y)\mathrm d y = f(z(y))\mathrm dz = \frac{1}{\sigma\sqrt{2\pi}}\, \exp\left(-\frac{1}{2\sigma^2}((e^{ay}-1)/a-\mu)^2\right)\, e^{ay}\mathrm dy .$$

The logarithm of the density $h$ is

$$\log h(y) = -\log(\sigma) - \frac{1}{2}\log (2\pi) - \frac{1}{2\sigma^2}\left(\frac{e^{ay}-1}{a}-\mu\right)^2 + ay.$$

Consequently, the log likelihood of $n$ independent observations according to your model is the sum of the logs of their individual likelihoods,

$$\Lambda(a,\beta,\sigma; \mathbf x, \mathbf y)) = - n\log\sigma - \frac{n}{2}\log(2\pi) - \frac{1}{2\sigma^2}\sum_{i=1}^n \left(\frac{e^{ay_i}-1}{a}-\mathbf x_i^\prime \beta \right)^2 + a\sum_{i=1}^n y_i.$$

When $a=0$, taking the limit of the forgoing as $a\to 0$ reduces each $(e^{ay_i}-1)/a$ to $y_i$ and the $+ay_i$ terms vanish, reproducing the usual likelihood for this Normal regression model.

$\endgroup$
3
  • $\begingroup$ Nice one! Is it possibel to find ML-estimastes for $\hat{b}$ and $\hat{\sigma}^2$ for a fixed value of $a$? I tried to read about ML-estimates but didn't get so much from this. $\endgroup$
    – Joey Adams
    Dec 13, 2022 at 23:54
  • $\begingroup$ And notice. Should you not include a negative sign in the exponent in both $f(z)$ and $h(y)dy$? $\endgroup$
    – Joey Adams
    Dec 14, 2022 at 0:12
  • 1
    $\begingroup$ Good catch! I left that out of the first two equations, but the subsequent formulas involving logarithms are correct. For any fixed value of $a,$ this is an Ordinary Least Squares (OLS) problem. The ML estimate of $b$ is the OLS estimate and the ML estimate of $\sigma^2$ is $1-2/n$ times the OLS estimate of $\sigma^2.$ $\endgroup$
    – whuber
    Dec 14, 2022 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.