-1
$\begingroup$

I think it is easiest if I describe my question using an example:

Let

$$ \mathbf{x} \triangleq [ x_1,x_2,x_3] $$ and for sake of explanation let's assume that $$ \{x_i \in [0,3] \}_{i=1}^{3} $$ which is to say that each element in the array $\mathbf{x}$ can take one of four values: 0,1,2 or 3. All combinations are allowed including repetitions e.g. $\mathbf{x}=[1,1,1]$.

Let us further say that $\mathbf{x}$ follows the distribution $P(\mathbf{X}\mid \boldsymbol \theta)$ and a sample is drawn thus: $\mathbf{x} \sim P(\mathbf{X}\mid \boldsymbol \theta)$.

The expected value of $\mathbf{X}$ is simply $$ \mathbb{E}[\mathbf{X}] = \sum_i \mathbf{x}_i \cdot P(\mathbf{x}_i) $$

But (here starts the real question) suppose that I now apply an operation to $\mathbf{X}$ e.g. concatenation of a reversed copy of itself i.e. $$ f(\mathbf{x}) = \mathbf{x} \cup \texttt{reverse}(\mathbf{x}) $$ Where if we have drawn a sample e.g. $\mathbf{x} = [1,2,3]$ then $f(\mathbf{x}) = [1,2,3,3,2,1]$.

What can we say about the expectation of $f(\mathbf{X})$? I believe I am onto the Law of the unconscious statistician (https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician) but I do not understand if the law holds for my above function (i.e. concatenation which is the operation of interest here).

I.e. what can we say about the expected value of $\mathbf{X}$ is simply $$ \mathbb{E}[f(\mathbf{X})] = \sum_i f(\mathbf{x}_i) \cdot P(f(\mathbf{x}_i)) $$ in terms of the original distribution over $\mathbf{x}$, if anything at all?

An explanation with an example would be most helpful (or even an explained proof) since I do not currently understand how this can possibly work (for my function $f(\cdot)$ since it is neither differentiable nor the inverse monotonic).

$\endgroup$
9
  • 1
    $\begingroup$ This notation is inconsistent (and somewhat nonsensical in places, although the general intent is evident), which makes it difficult to determine what you're really trying to ask. Your definition of $\mathbf x$ in particular is that of a vector valued random variable. Thus, $E[\mathbf x]$ also is a vector. Furthermore -- I hope this is easy to see -- your function $f:\mathbb R^3\to\mathbb R^6$ is linear. All you have to do is apply linearity of expectation to conclude $E[f(\mathbf x)] = f(E[\mathbf x]).$ If you differ with this interpretation, please clarify. $\endgroup$
    – whuber
    May 25 at 23:42
  • $\begingroup$ Please explain where it is inconsistent and I shall endeavour to fix it so that it makes more sense. Your definition of $f$ is further incorrect since $f$ is a discrete map and not continuous (but I may have misunderstood you?) I do not think we mean the same thing? But again, please tell me where I can improve on the notation. I struggle to see how the expectation of $\mathbf{x} \in \mathbb{Z}^3$ -- but I may misunderstand here, is a concatenation of a reverse copy a linear operation? $\endgroup$
    – Astrid
    May 25 at 23:49
  • $\begingroup$ If you insist on that nit, $f$ has a cpntinuous extension to all of $\mathbb R^3.$ It is represented by the matrix $$\pmatrix{1&0&0&0&0&1\\0&1&0&0&1&0\\0&0&1&1&0&0}.$$ As far as notation goes, the first formula is idiosyncratic and not likely to be understood by many; the second uses set brackets in an unusual way; and the text suggests you are thinking of an expectation as a number rather than a vector, indicating other potential problems with understanding the notation. $\endgroup$
    – whuber
    May 26 at 11:24
  • $\begingroup$ I’m sorry I don’t agree. The first formula is just an array definition. Standard fare. The second is domain assignment of each element. Also standard. And the standard definition of the expectation does give a number for one random variable, and an array for arrays. I’m not insisting on f being discrete, it is fundamental to my question. Imposing another form imposes another question not of interest. $\endgroup$
    – Astrid
    May 26 at 13:32
  • $\begingroup$ Re "standard fare:" the notation of computing does not have a well-understood mathematical or statistical meaning. By using it, you both obscure the issues and, apparently, confuse yourself. $\endgroup$
    – whuber
    May 26 at 13:48

1 Answer 1

1
$\begingroup$

I think this is a lot simpler than you think it is :)

We have a random vector $\mathbf X=(X_1,X_2, \cdots, X_n)$, with expectation $E[\mathbf X] = (E[X_1], E[X_2], \cdots, E[X_n])$.

The expectation of a vector is just a vector of each element's expectation.

Then, the reverse concatenation $(\mathbf X, \mathbf X_\text{rev})=(X_1,\cdots, X_n, X_n, \cdots, X_1)$ has expectation:

$$ \begin{aligned} E[(\mathbf X, \mathbf X_\text{rev})] &= (E[X_1], \cdots, E[X_n], E[X_n],\cdots, E[X_1]) \end{aligned} $$

$\endgroup$
6
  • $\begingroup$ Interesting and thank you! My confusion persists a bit. I want to treat the whole vector as the sample rather than the individual bits. Is there a world in which that is done? So we would have sufficient statistics over vectors and not elements as shown in your answer? $\endgroup$
    – Astrid
    May 26 at 2:44
  • $\begingroup$ Hi @Astrid. But the whole vector isn't the sample - only the first half (i.e. $\mathbf X$) is the sample (under how you've defined your problem). I'm not sure I understand what you're asking. $\endgroup$
    – Alex J
    May 26 at 3:03
  • $\begingroup$ No you’re right apologies; same question but regarding only the first half. So treating the first half as the sample and then providing statistics over that. $\endgroup$
    – Astrid
    May 26 at 4:34
  • $\begingroup$ Still not sure I understand - what about your question is not answered by my second sentence (starting from "We have a random vector [...]")? If I replaced the words "random vector" with "random sample", my argument still holds. $\endgroup$
    – Alex J
    May 26 at 4:39
  • $\begingroup$ You’ve explained it very well. Thank you again. I’ll have to collect my thoughts to see if I can pose a better question. $\endgroup$
    – Astrid
    May 26 at 4:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.