I would like, after further considerations about this problem, to reformulate this question of mine again. I kept a record of the past words and remarks as the appendix below. I think the question that really bothers me can be stated as follows:

If $\theta \in \mathbb{R}^{k}$, if $W$ is a random vector in $\mathbb{R}^{r}$ from a distribution having $\theta$ as the value of its parameter vector, if $q \leq k$, if $f: \mathbb{R}^{r} \times \mathbb{R}^{q} \to \mathbb{R}$ is measurable (in the suitable sense), if the distributions $\{ \mathscr{P}_{f(W, b)} \}_{b \in \mathbb{R}^{q}}$ induced by the random variable $f(W, \cdot)$ is identifiable as $\mathbb{R}^{q}$, and if (at no cost of specialization as can be examined) $\beta$ is such that the first moment of $\mathscr{P}_{f(W, \beta)}$ is $= 0$, then is it necessary that $\beta$ is a subvector of $\theta$?

An example of the above more general setting is a orthogonal projection model (identifiable linear model with stochastic regressors).

It took a long while to raise the question in my mind in the current form. I could say it would be what is closest to what I would like to know at the current stage.


Words and remarks before the above revised version of this question are in the following paragraphs:

Let $p \in \mathbb{N}$; let $\theta \in \mathbb{R}^{p}$; let $(X,Y) \sim F^{\theta}_{X,Y}$ be a random vector in $\mathbb{R}^{2}$ such that $F^{\theta}_{X,Y}$ is the joint CDF of $X$ and $Y$. If there is exactly one $b \in \mathbb{R}$ such that $Y = Xb + U$ and $\mathbb{E}(U \mid X) = 0$, is it necessary that $b$ is a component of $\theta$? How to prove it no matter the answer is affirmative or not? Thanks (for sure at least to Whuber's questions below that pushed to form the current neat form of the question).

It seems that until now did I realize the crux of my question. Thanks again to the feedback providers below, directly or not. Let me use this simple example below to illustrate my confusion. I hope this example would also explain the first paragraph better. Suppose the expectation $\mathbb{E}(Y - Xb)^{2}$ is finite. If $F_{U}^{b}$ is the CDF of $U$, then the expectation can be obtained via two integrals, namely $$ \mathbb{E} (Y - Xb)^{2} = \int_{\mathbb{R}^{2}} (y - xb)^{2} dF_{X,Y}(x,y) = \int_{\mathbb{R}} u^{2} dF^{b}_{U}(u). $$ If the expectation is regarded as the last integral, then it depends on $b$ by assumption. I wonder if $F_{X,Y}$ also depends on $b$ when regarded as the first integral? The notation $\mathbb{E}(Y - Xb)^{2}$ itself provides no information about the distinction, correct? This example does not fit the question title very much; however, I would ask the reader to examine it with a more careless viewpoint unless this mismatch does make the reader completely clueless.

  • 2
    $\begingroup$ I think you might need to explain what you mean by a "parameter ... entering" a function. Since you posit just one function $f_{x,y},$ there are no "parameters" per se. A parameter makes sense only when that function is part of a family of functions. A parameter is used to determine which member of the family it is. $\endgroup$
    – whuber
    Oct 19, 2018 at 23:10
  • 2
    $\begingroup$ Thank you, but I still cannot make sense of your question. How can a unique object like $f_{x,y}$ possibly "depend" on anything at all? $\endgroup$
    – whuber
    Oct 20, 2018 at 16:04
  • 2
    $\begingroup$ That nicely illustrates the confusion: by "bivariate normal PDF" you are referring to an arbitrary member of a two-parameter family of distribution functions $f_{\mu,\sigma},$ but where you write "if this is true for every PDF or not," suddenly you are referring to just a single density function rather than a family. Because, no matter what you might be trying to ask, the answer is likely to depend (very much) on the family, it's crucial that you maintain this distinction and not confuse distributions with families of distributions. $\endgroup$
    – whuber
    Oct 20, 2018 at 16:58
  • 2
    $\begingroup$ I'm afraid I cannot understand it. You attempt to define "parameter enters" by pointing out that the expectation of a random variable depends on its distribution. That's inadequate as a definition or even as a hint about what you intend. $\endgroup$
    – whuber
    Oct 20, 2018 at 18:37
  • 1
    $\begingroup$ Having some trouble with the notation. It sure is compact. Reminds me of Perl language, easier to write than to read. Maybe try for the explanation for "dummies." $\endgroup$
    – Carl
    Oct 20, 2018 at 20:04

1 Answer 1


The question puts two functions into evidence, which I will call $\theta$ and $t.$ They are maps from a space $\mathcal F$ of distributions defined on $\mathbb{R}^2$ (or, more generally, any set on which distributions may be defined) into a "parameter space" $\Theta\subset\mathbb{R}^p$ or the real numbers $\mathbb R.$

The parameterization associates the parameter values with the distributions and so may be considered an invertible map

$$\theta:\mathcal{F} \to \Theta\subset \mathbb{R}^p.$$

The regression parameter $b$ of the question is an example of a property of a distribution. That there is exactly one $b$ for each $f\in\mathcal F$ means $b$ can be considered the value $t(f)$ of some function

$$t:\mathcal F \to \mathbb{R}.$$

Note that the components of $\theta$ are automatically properties according to this definition, because the $i^\text{th}$ component is the composition of $\theta$ and the projection $\pi_i:\mathbb{R}^p\to \mathbb{R},$

$$\theta_i:\mathcal F \to \mathbb{R}^p \to \mathbb R\\\theta_i(f)=\pi_i(\theta(f)),$$

and is therefore a real-valued function defined on $\mathcal F.$

The question asks whether, for any possible such $\theta$ and $t,$ it is necessarily the case that there exists a component $i,$ $1\le i \le p,$ such that

$$t(f) = \theta_i(f)$$

for all $f\in \mathcal F.$ That is clearly not true, because distributions have infinitely many properties but there are only a finite number $p$ of components of $\theta.$ For instance, for any number $x$ the map $t+x:\mathcal{F}\to\mathbb R$ given by

$$(t+x)(f) = t(f) + x$$

is not the same property as $t.$

We could generalize the question to ask whether $t$ must depend somehow on the parameter. This is readily shown to be the case because the invertibility of $\theta$ implies that for any parameter $p\in\Theta$ there is a unique $f = \theta^{-1}(p)\in\mathcal F$ associated with $p$ and

$$t(f) = t(\theta^{-1}(p)) = (t\circ \theta^{-1})(p)$$

defines a function

$$t\circ\theta^{-1}:\mathbb{R}^p \to \mathbb{R}.$$

To sum up these two conclusions in words we may say

The parameters are properties of a finitely parameterized family of distributions, but they are not the only properties. All properties are functions of the parameters, however.

Although I have been silent about technical issues of continuity (or measurability or differentiability, depending on the application), the same analysis holds quite generally assuming we apply the same criteria to "properties" as we do to "parameters."

  • $\begingroup$ (+1) Good job. I especially like that the notation did not put you off. I thought the answer was obvious, but was at a loss to express that in formal language due to my incomplete understanding of the notation. $\endgroup$
    – Carl
    Oct 21, 2018 at 1:01
  • $\begingroup$ Thanks. Is it true that, if there is no reparameterization, then $b$ has to be a component of $\theta$? $\endgroup$
    – Yes
    Oct 21, 2018 at 16:19
  • $\begingroup$ For example, if $(x,y)$ comes from a PDF $f_{x,y}$, if $b \in \mathbb{R}$, then the PDF of the random variable $y - xb$ has a parameter $b$. Then, if we have to, should we write something like $\mathbb{E}^{b} (y - xb)^{2}$ to emphasize the expectation is taken with respect to a certain parameter value? I think the answer is yes if we are integrating $(y-xb)^{2}$ with respect to the distribution of the difference random variable. But should we also write $\mathbb{E}^{b} (y - xb)^{2}$ if the integration is taken with respect to the joint distribution of $x,y$? $\endgroup$
    – Yes
    Oct 21, 2018 at 16:29
  • $\begingroup$ If still interested, please refer to the extended version of my question, which is inspired by the comments. $\endgroup$
    – Yes
    Oct 21, 2018 at 16:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.