4
$\begingroup$

Suppose we have a multivariate data set, $s = (s_1, s_2, ... s_p)$ and each $s_i$ is distributed with a distribution that has finite support (we'll call each $s_i$ a "source"). Let us denote the support of $s_i$ to be $S_{s_i}$. Then, if we can say that $$S_s = S_{s_1} \times S_{s_2} \times \dots \times S_{s_p} \tag{1}$$ where $\times$ denotes the Cartesian product, what can we say about the distribution of $s$ (i.e. the joint distribution of $(s_1,s_2,...s_p)$? The paper I am reading says the following:

We should note that the domain separability assumption refers to the condition that the convex support of the joint density of sources can be written as Kronecker product of the convex supports of the individual source marginals. The domain separability assumption essentially implies that the boundaries of the values that a source can take is not dependent on the values of other sources, and is a necessary condition for the mutual independence of sources.

I need help breaking this down.

  1. Why did we switch from the Cartesian product to the Kronecker product? Does this mean that the convex support is different than just the support of the distribution? (I have no training in convex analysis, so if that is what needs to be learn't here please let me know).

  2. Equation 1 is noted as a necessary condition for independence, but not sufficient condition. At other places, the paper also uses language implying that the sources need not be independent, but can be independent (and even correlated). My questions to this are:

    a.) When Equation 1 is satisfied, what does it imply about the joint distribution? Why? Any examples to help solidify the understanding here? The paper says that Equation 1 relaxes the independence assumption between the random variables, but what does it definitely imply? And why?

    b.) Are there situations for a joint PDF where you can't write the support of the joint distribution function as a Cartesian product of the marginal's support? I suppose there must be based on the language used in the document, but example may help me understand this better?


Edit

After some more research I found this image which visualizes some of the concepts.

img

The difference between domain separability and independence is that independence forces your joint pdf to be a certain shape, while domain separability only forces your joint pdf to be in a certain region. My interpretation is that since $S_s$ is the support of the new region ($S_{S_1,S_2}$) in the figure, it is assumed that the joint pdf must be non-zero in this region?

$\endgroup$
1
$\begingroup$

The "convex support" is the convex hull of the support. For instance, for the PDF shown at the bottom right of the figure, the support (defined as the smallest close set containing all values with probability $1$) is the union of the disjoint intervals $[0,1/3]$ and $[2/3,1].$ Its convex hull (defined as the smallest convex set containing it) is the interval $[0,1]:$ we have to include the numbers intervening between $1/3$ and $2/3$ to assure convexity.

The quotation almost surely means "Cartesian product" (a topological operation) rather than "Kronecker product" (a linear algebraic operation).

The real interest in the question concerns relationships between (convex) supports of marginal distributions and the (convex) support of the joint distribution. I think a few simple pictures might help illuminate the subtleties.

Consider distributions defined over the unit square $[0,1]\times [0,1]\subset\mathbb{R}^2,$ shown in the upper left plot of the figure.

Figure

Let's work with the distributions that are uniform over the blue triangles shown. Specifically, for any $0 \le t \lt 1,$ let the density (PDF) be constant within the region $|x-y| \ge t$ (shown in blue) and zero elsewhere. The support of such a distribution is a union of two triangles.

The corresponding marginal densities (which are the same in each dimension) are plotted in the bottom row. The pattern should be obvious: when $t \le 1/2,$ the support of the marginal distribution is the full interval $[0,1].$ When $t \gt 1/2,$ the support is the union of the intervals $[0,t]$ and $[1-t,1].$ The convex support is always $[0,1].$

Notice that the convex support of the joint distribution is the unit rectangle only for $t=0.$ For larger $t,$ the smallest convex set containing the two blue triangles is a hexagon. These hexagons comprise the blue triangles and the gray intervening rectangles, all with dotted black outlines.

These provide clear examples where the Cartesian product of the marginal supports is strictly greater than either the joint support or the joint convex support. In particular, the joint density can be zero throughout a substantial portion of its convex support.

If you are concerned primarily about convex supports, you can construct a variant of these examples in which the joint distribution also includes triangles at the upper right and lower left corners, thereby making the convex support equal to the entire unit square.

(BTW, these are all examples with compact support, which is what I think the question must mean by "finite" support. Distributions with finite supports would necessarily be discrete and notions of convexity wouldn't be appropriate for them.)

$\endgroup$
0
$\begingroup$

If he really means Cartesian product, it is easy. The assumption that the joint support is the Cartesian product of the marginal supports sets, simply means there is no logically necessary relations between the variables.

They might be independent or dependent. However, if the joint support is not the Cartesian product of the marginal supports, then the variables are necessarily dependent and independence is ruled out.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.