Help me understand how the following likelihood function is derived

A week ago, I asked a question concerning the Taylor expansion of an arbitrary distribution function. As noted by a member of the forum, the question was vague and perhaps incorrect. I had asked this question in my quest to understand a particular derivation in a book titled "Sign-based methods in linear statistical models" by Boldin et al (1997), as to me it appeared to be a Taylor series. This is the excerpt from the book (with some preliminary assumptions):

Consider a one parameter model

$$$$x_i=c_i\theta+\varepsilon_i,\quad i=1,\cdots,n,$$$$

with parameter $$\theta\in \mathbb{R}$$. Further assume the random errors are independent and identically distributed. Their common distribution function is $$$$F(u)=P\{\varepsilon_i with the following conditions:

1. $$F(0)=1/2$$
2. $$F(x)$$ has a continuous density $$f(x)$$ in a neighbourhood of zero, and $$f(0)>0$$.
3. $$f(x)$$ is absolutely continuous in a neighbourhood of zero and $$f'(0)=0$$
4. $$F(x)$$ satisfies the Lipschitz condition at zero.

Consider the vector of signs $$$$S(X)=(\text{sign }x_1,\cdots,\text{sign }x_n)'$$$$ The possible values of random vector $$S(X)$$ are vectors consisting of $$+1$$ and $$-1$$. Let $$$$S=(s_1,\cdots,s_n)'$$$$ be an arbitrary vector of this form. Then the likelihood $$P\{S(X)=s\mid \theta \}$$ is given by the formula

$$\begin{eqnarray} P\{S(X)=s\mid \theta \}&=&\prod\limits_{t=1}^{n}(P\{x_i>0\})^{(1+s_i)/2}(P\{x_i<0\})^{(1-s_i)/2}\\ &=&2^{-n}\prod\limits_{t=1}^n[1+2f(0)c_is_i\theta+o(\theta)]\\ &=&2^{-n}\left[1+2f(0)\left(\sum\limits_{i=1}^{n}c_is_i\right)\theta+o(\theta)\right] \end{eqnarray}$$

My question is as follows: I do not exactly understand how line 2 of the likelihood function is derived. I initially thought it is a Taylor expansion, but as it was noted it could be that I am completely wrong about this. Thanks in advance!

1 Answer

You need to relate the probabilities to $$F,$$ so begin with

$$\Pr(x_i \gt 0) = \Pr(c_i\theta + \varepsilon\gt 0) = \Pr(\varepsilon\gt -c_i\theta) = F(-c_i\theta),$$

immediately giving

$$\Pr(x_i \lt 0) = 1 - \Pr(x_i \gt 0) - \Pr(x_i=0) = 1 - F(-c_i\theta)$$

because $$F$$ is continuous.

The key is the "$$o(\theta)$$" term, indicating the evaluation is to be made for arbitrarily small $$|\theta|.$$ This implies all the $$-c_i\theta$$ are small, too, which means $$F$$ will be evaluated in a neighborhood of $$0.$$ The assumptions are made so that you can approximate $$F$$ by its derivative at $$0.$$ That is, for sufficiently small $$|x|,$$ use

$$F(x) = F(0) + F^\prime(0)x + o(x) = \frac{1}{2} + f(0)x + o(x).$$

Using this approximation with $$x=-c_i\theta$$ lets us approximate each term in the product as

\begin{aligned} &\Pr(x_i\gt 0)^{(1+s_i)/2}\, \Pr(x_i\lt 0)^{(1-s_i)/2} \\ &= \left(\frac{1}{2} - f(0)c_i\theta + o(c_i\theta)\right)^{(1+s_i)/2}\, \left(\frac{1}{2} + f(0)c_i\theta + o(c_i\theta)\right)^{(1-s_i)/2} . \end{aligned}

We may treat each of these factors in the same way by applying the Binomial Theorem (equivalently, the MacLaurin series for a binomial power), which implies

$$(1 + x)^p = 1 + px + o(x).$$

Whence, plugging in appropriate values for $$x$$ and $$p$$ and factoring out $$1/2,$$ the factors can be expressed

$$\left(\frac{1}{2} \mp f(0)c_i\theta + o(c_i\theta)\right)^{(1\pm s_i)/2} = 2^{-(1\pm s_i)/2}\left(1 \mp \frac{1\pm s_i}{2}\,2f(0)(c_i\theta) + o(\theta)\right).$$

Finally, the product of these terms can be taken using the standard rules of algebra, noting that $$o(c_i\theta) = o(\theta),$$ $$\theta o(\theta) = o(\theta),$$ and $$o(\theta)^2 = o(\theta)$$ (all of which are easy implications of the definition of $$o$$). After that simplification, the terms will have been reduced to

$$2^{-1} \left(1 + 2 f(0) s_i c_i \theta + o(\theta)\right).$$

Factoring out the $$n$$ copies of $$2^{-1}$$ from the product of all these terms produces the second line of the derivation.

• Thank you so much! You are a saint! However, my confusion still remains regarding a point you had made last week! If you recall, I made a remark about Taylor series for an arbitrary distribution function, with which you found an issue. However, as I had suspected and as you have pointed out yourself, it is indeed a MacLaurin series. Could you please kindly also elaborate on my mistake in the technical phrasing of that question, which led to confusion. I am mainly asking this for the sake of clarity and a deeper understanding. Thank you again! – Carl Nov 12 '20 at 17:53
• If you're referring to stats.stackexchange.com/questions/494800/…, it's too vague to determine what you are asking about. In light of the present question one can guess what you're getting at but it still needs more details. – whuber Nov 12 '20 at 17:56
• Upon second inspection, you are correct. I seem to struggle to convey certain ideas on the forum. Hopefully this will improve in the future. Once again, I am grateful for your response and making this clear for me. – Carl Nov 12 '20 at 17:58