1
$\begingroup$

A week ago, I asked a question concerning the Taylor expansion of an arbitrary distribution function. As noted by a member of the forum, the question was vague and perhaps incorrect. I had asked this question in my quest to understand a particular derivation in a book titled "Sign-based methods in linear statistical models" by Boldin et al (1997), as to me it appeared to be a Taylor series. This is the excerpt from the book (with some preliminary assumptions):

Consider a one parameter model

\begin{equation} x_i=c_i\theta+\varepsilon_i,\quad i=1,\cdots,n, \end{equation}

with parameter $\theta\in \mathbb{R}$. Further assume the random errors are independent and identically distributed. Their common distribution function is \begin{equation} F(u)=P\{\varepsilon_i<u\} \end{equation} with the following conditions:

  1. $F(0)=1/2$
  2. $F(x)$ has a continuous density $f(x)$ in a neighbourhood of zero, and $f(0)>0$.
  3. $f(x)$ is absolutely continuous in a neighbourhood of zero and $f'(0)=0$
  4. $F(x)$ satisfies the Lipschitz condition at zero.

Consider the vector of signs \begin{equation} S(X)=(\text{sign }x_1,\cdots,\text{sign }x_n)' \end{equation} The possible values of random vector $S(X)$ are vectors consisting of $+1$ and $-1$. Let \begin{equation} S=(s_1,\cdots,s_n)' \end{equation} be an arbitrary vector of this form. Then the likelihood $P\{S(X)=s\mid \theta \}$ is given by the formula

\begin{eqnarray} P\{S(X)=s\mid \theta \}&=&\prod\limits_{t=1}^{n}(P\{x_i>0\})^{(1+s_i)/2}(P\{x_i<0\})^{(1-s_i)/2}\\ &=&2^{-n}\prod\limits_{t=1}^n[1+2f(0)c_is_i\theta+o(\theta)]\\ &=&2^{-n}\left[1+2f(0)\left(\sum\limits_{i=1}^{n}c_is_i\right)\theta+o(\theta)\right] \end{eqnarray}

My question is as follows: I do not exactly understand how line 2 of the likelihood function is derived. I initially thought it is a Taylor expansion, but as it was noted it could be that I am completely wrong about this. Thanks in advance!

$\endgroup$
2
$\begingroup$

You need to relate the probabilities to $F,$ so begin with

$$\Pr(x_i \gt 0) = \Pr(c_i\theta + \varepsilon\gt 0) = \Pr(\varepsilon\gt -c_i\theta) = F(-c_i\theta),$$

immediately giving

$$\Pr(x_i \lt 0) = 1 - \Pr(x_i \gt 0) - \Pr(x_i=0) = 1 - F(-c_i\theta)$$

because $F$ is continuous.

The key is the "$o(\theta)$" term, indicating the evaluation is to be made for arbitrarily small $|\theta|.$ This implies all the $-c_i\theta$ are small, too, which means $F$ will be evaluated in a neighborhood of $0.$ The assumptions are made so that you can approximate $F$ by its derivative at $0.$ That is, for sufficiently small $|x|,$ use

$$F(x) = F(0) + F^\prime(0)x + o(x) = \frac{1}{2} + f(0)x + o(x).$$

Using this approximation with $x=-c_i\theta$ lets us approximate each term in the product as

$$\begin{aligned} &\Pr(x_i\gt 0)^{(1+s_i)/2}\, \Pr(x_i\lt 0)^{(1-s_i)/2} \\ &= \left(\frac{1}{2} - f(0)c_i\theta + o(c_i\theta)\right)^{(1+s_i)/2}\, \left(\frac{1}{2} + f(0)c_i\theta + o(c_i\theta)\right)^{(1-s_i)/2} . \end{aligned}$$

We may treat each of these factors in the same way by applying the Binomial Theorem (equivalently, the MacLaurin series for a binomial power), which implies

$$(1 + x)^p = 1 + px + o(x).$$

Whence, plugging in appropriate values for $x$ and $p$ and factoring out $1/2,$ the factors can be expressed

$$\left(\frac{1}{2} \mp f(0)c_i\theta + o(c_i\theta)\right)^{(1\pm s_i)/2} = 2^{-(1\pm s_i)/2}\left(1 \mp \frac{1\pm s_i}{2}\,2f(0)(c_i\theta) + o(\theta)\right).$$

Finally, the product of these terms can be taken using the standard rules of algebra, noting that $o(c_i\theta) = o(\theta),$ $\theta o(\theta) = o(\theta),$ and $o(\theta)^2 = o(\theta)$ (all of which are easy implications of the definition of $o$). After that simplification, the terms will have been reduced to

$$2^{-1} \left(1 + 2 f(0) s_i c_i \theta + o(\theta)\right).$$

Factoring out the $n$ copies of $2^{-1}$ from the product of all these terms produces the second line of the derivation.

$\endgroup$
3
  • $\begingroup$ Thank you so much! You are a saint! However, my confusion still remains regarding a point you had made last week! If you recall, I made a remark about Taylor series for an arbitrary distribution function, with which you found an issue. However, as I had suspected and as you have pointed out yourself, it is indeed a MacLaurin series. Could you please kindly also elaborate on my mistake in the technical phrasing of that question, which led to confusion. I am mainly asking this for the sake of clarity and a deeper understanding. Thank you again! $\endgroup$ – Carl Nov 12 '20 at 17:53
  • 1
    $\begingroup$ If you're referring to stats.stackexchange.com/questions/494800/…, it's too vague to determine what you are asking about. In light of the present question one can guess what you're getting at but it still needs more details. $\endgroup$ – whuber Nov 12 '20 at 17:56
  • 1
    $\begingroup$ Upon second inspection, you are correct. I seem to struggle to convey certain ideas on the forum. Hopefully this will improve in the future. Once again, I am grateful for your response and making this clear for me. $\endgroup$ – Carl Nov 12 '20 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.