Implications of bounded support on regression equation

I am reading an academic paper right now, and the following claim is made without much explanation:

Suppose that the joint distribution $$F(\textbf y)$$ of the $$n \times 1$$ random vector $$\textbf Y$$ has a density $$f(\textbf y )$$ with bounded support $$\mathscr Y$$. Without loss of generality assume that the true coefficient equals $$\beta_0=0$$ and that $$\sigma^2=1.$$

Because $$\textbf Y$$ has bounded support $$\mathscr Y$$ there is a set $$B \subset \mathbb R^m$$ such that $$|\textbf y' \Sigma^{-1}\textbf X \beta|<1$$ for all $$\beta \in B$$ and $$\textbf y \in \mathscr Y$$.

I don't understand how we are able to conclude that $$|\textbf y' \Sigma^{-1}\textbf X \beta|<1$$. Can anyone explain how this follows from the assumption of bounded support?

• I think you will need some bounds on the elements of $X$ also. Did they assume so? Commented Dec 12, 2021 at 17:14
• The proof treats X as fixed. I suppose you could regard $Y=X \beta + e$ as a constraint. Commented Dec 12, 2021 at 19:42
• It is page 9 here: ssc.wisc.edu/~bhansen/papers/gauss.pdf Commented Dec 12, 2021 at 19:44

Because $$Y$$ has bounded support $$\mathcal S,$$ the set $$\mathcal S \Sigma^{-1}X = \{y\Sigma^{-1}X\mid y\in\mathcal S\}$$ is bounded, too. Let $$c\gt 0$$ be any upper bound of the norms of its elements.

For $$\beta\in\mathbb{R}^m$$ and $$y\in\mathcal S,$$ the Cauchy-Schwarz Inequality implies

$$|y \Sigma^{-1}X\beta| \le ||y\Sigma^{-1}X||\,||\beta|| \le c||\beta||.$$

We may therefore choose for $$B$$ any nonempty subset of ball of radius $$1/c$$ in $$\mathbb{R}^m,$$ for which $$\beta\in B$$ implies

$$||y\Sigma^{-1}X||\,||\beta|| \le (c)(1/c) = 1,$$

QED.

This demonstration began with the claim that when $$\mathcal S \subset \mathbb{R}^n$$ is bounded and $$A = \Sigma^{-1}X = (a_{ij})$$ is any $$n\times m$$ matrix, then $$\mathcal S A \subset \mathbb{R}^n$$ is bounded, too. To see this, let $$s$$ be any upper bound for the norms of elements of $$\mathcal S$$ and let $$a$$ be an entry in $$A$$ with largest absolute value. Then for any $$y\in \mathcal S,$$ write $$y_k$$ for an entry of largest absolute value and--noting that $$|y_k|^2 \le \sum_i |y_i|^2 = ||y||^2$$--observe that

$$||yA||^2 = \sum_{i=1}^m \left(\sum_{j=1}^n y_j a_{ji}\right)^2 \le \sum_{i=1}^m\left(\sum_{j=1}^n |y_k| |a|\right)^2 = mn||y||^2|a|^2 \le mns|a|^2 ,$$

demonstrating $$mns|a|^2$$ is an upper bound for $$\mathcal S A.$$

This should make it clear that $$\mathbb{R}^m$$ and $$\mathbb{R}^n$$ can be replaced by any dual normed linear spaces and $$A$$ need only be a bounded linear operator for the result to be true. In other words, everything results from the fact that finite matrices represent bounded operators.

Finally, the converse is not always true. A characteristic counterexample has $$\mathcal S = \{(x,0)\} \subset \mathbb R^2$$ and $$B \subset \{(0,y)^\prime\}\subset \mathbb R^2$$ (with $$\Sigma^{-1}X$$ the identity transformation). By construction, $$|y\beta|=0$$ for all $$y=\mathcal S$$ and all $$\beta\in B,$$ even though $$\mathcal S$$ is unbounded.

This helps show that the point to the claim in the question is that a bounded set $$\mathcal S$$ is bounded in all possible directions (as represented by $$\Sigma^{-1}X\beta$$). In the counterexample, $$\mathcal S$$ is bounded in some special directions (namely, those spanned by the vector $$(0,1)^\prime$$) but not in all of them. When the image of the map $$\beta \to \Sigma^{-1}X \beta$$ does not span all possible directions (that is, $$\Sigma^{-1}X$$ is not of full rank), such counterexamples can always be constructed.

A try a proof using the notation of the linked paper A Modern Gauss-Markov Theorem. As this is a finite-sample result, $$n$$ is a fixed integer $$\Sigma>0$$ is a known, fixed, positive-definite (posdef) matrix, and the design matrix $$X$$ is assumed known, so a constant. The range of $$Y$$, $$\mathscr{Y}$$, is a bounded set, so contained in a compact set, so we can assume it is compact.

Interest is in the expression $$y^T \Sigma^{-1} X\beta$$ which we can write as the continuous function $$T(y,\beta)=y^T \Sigma^{-1} X\beta$$ it is not necessary to include the constants as arguments. For each $$\beta$$, the image set $$T(\mathscr{Y},\beta)$$ is a compact set, by continuity of $$T$$. It is clear that $$T(\mathscr{Y}, 0)=\{ 0 \}$$ so the condition $$| T | < 1$$ is fulfilled at least for one $$\beta$$, namely $$\beta=0$$.

Now write $$\sup_{\beta\in B,~ y \in \mathscr{Y}} = | T(\mathscr{Y}, B) |$$ Now fix one nonzero $$\beta$$ and observe that $$| T(\mathscr{Y}, c\beta)| = |c| |T(\mathscr{Y}, \beta) |$$ so is increasing linearly along $$\beta$$-rays from the origin. If this last value is $$| c | G >0$$, the condition $$|T | <1$$ will be fulfilled for $$|c| < G^{-1}$$.

We can conclude that the sought-for set $$B$$ is non-empty, larger than just $$\{ 0 \}$$, and it is in fact a cone.