Questions tagged [projection]
For on-topic questions involving the mathematical concept projection, a linear transformation $P$ such that $P=P^2$. Please include also a more statistical methods tag. For purely mathematical questions about projections it is better to ask on math SE https://math.stackexchange.com/
95 questions
0
votes
0
answers
49
views
Making forecasts based on cyclical data
I am trying to develop a more robust methodology for a forecast model. This attempts to project the final number of recruits for this cycle based on comparing the current recruitment cycle to previous ...
1
vote
1
answer
130
views
Wilks' theorem, confidence regions on a projected subspace?
Background
My context is I am thinking about maximum likelihood estimation of a parameter $\theta\in \mathbb{R}^r$ by drawing samples $\{x_i\}$ from a distribution which depends on $\theta$. I know ...
- 146
10
votes
2
answers
307
views
Does the conditional expectation operator have an interpretable decomposition like the projection matrix does in linear algebra?
I'm trying to draw a parallel between the concept of projections in a finite linear space to an infinite linear space.
Here is the set-up, first in the finite dimensional case, and then second in the ...
- 111
0
votes
0
answers
23
views
How to interpret Diffusion Maps for the iris dataset?
This might be a poor exercise but I'm trying to understand the methods of paper and if it makes sense to adapt my linear-based workflow with PCA to non-linear manifold methods; thought trying out ...
- 1,482
3
votes
1
answer
98
views
Geometric understanding of linear regression
I am reading up on linear regression from mit 16.850
Here is how the lecture goes:
Given: $Y_{n,1}$ (targets), $X_{n, p}$ (data), $t_{p, 1}$ (the parameters I'm optimizing over), True model: $Y = \...
- 2,683
3
votes
1
answer
94
views
Name of PDF? - projecting uniform probability distribution on the unit circle to the x-axis
Consider a uniform probability distribution on a circle of radius r, i.e. $\{(x,y) \in \mathbb{R}^2: x^2 + y^2 = r^2 \}$.If we wish to project onto the x-axis, we can consider each point on the circle ...
- 225
0
votes
0
answers
51
views
How do we prove that the rows or columns of a hat matrix sum up to 1, when mean function includes an intercept? [duplicate]
Every textbook I encounter tells me that this is simply a meaningful relationship of a hat matrix, without explaining why:
If H is the hat (projection) matrix, and our X matrix has full rank and a ...
- 1
0
votes
0
answers
47
views
Exogeneity of volatility shocks in Local projection model
I want to estimate the impact of volatility shocks on cross-assets spillovers. I have series of spillovers, and I want to use a Local Projection model, and the volatility of some financial assets ...
1
vote
2
answers
534
views
Proof of Frisch-Waugh-Lovell Theorem
In the book I am currently reading, they propose the following proof:
$\hat{Y} = X_1 \hat{\beta}_1 + X_2 \hat{\beta}_2$
Now because $M_1 = I - P_1$, we can always derive a formula for $X_2$:
$X_2 = ...
- 677
2
votes
0
answers
186
views
Relationship between conditional expectation and regression
I would be grateful if you could help me clear up some confusion regarding conditional expectation and regression.
I have seen two formulations of the linear regression framework:
$$Y=a+bX+\varepsilon\...
2
votes
1
answer
107
views
So how can I project the data with the eigenvectors from LDA?
I have data that look like this.
And my goal is to reduce this 3D dimension into 2D dimension so it might looks like this. Turning the angle so the distance between all classes becomes maximum.
So ...
- 425
0
votes
1
answer
717
views
Proof that leverage points are between 0 and 1 inclusive
I would really appreciate it if anyone can guide me through this.
I have a $n \times (p+1)$ matrix $X$. The projection matrix $P = X(X'X)^{-1}X'$. I want to prove that $P(i,i)$ is in $[0,1]$, where $P(...
2
votes
1
answer
73
views
OLS regression and dummy variables: fitted values of the subsample equal actual values?
Given $X=(d\; X_1)$, we want to prove that
$$(d'd)^{-1}d'P_{[d X_1]}y=(d'd)^{-1}d'y,$$ where $P_{[X]}y=\hat{y}.$ That is, restricting the regression to the subsample for which $d_i=1$, we have that ...
- 65
1
vote
0
answers
89
views
The principal submatrix of projection matrix with Gaussian design
I've come across a phenomenon from a simulation that I'm very curious about.
But I don't know how to start my analysis. So, I am asking for some guidance.
Thanks!
Denote by $\mathbf{H}$ the principal ...
- 115
1
vote
2
answers
99
views
Hayashi Econometrics Seemingly Unrelated Regressions (SUR) Eq 4.5.13'-15'
According to 4.5.13
$\hat{A}_{mh}=\left(\frac{1}{n}\sum_{i=1}^{n}z_{im}x_{i}^{\prime}\right)\left(\frac{1}{n}\sum_{i=1}^{n}x_{i}x_{i}^{\prime}\right)^{-1}\left(\frac{1}{n}\sum_{i=1}^{n}x_{i}z_{ih}^{\...
- 41
1
vote
0
answers
81
views
Pythagorean Theorem for inclusive Kullback-Leibler
Let $\mu$ be some probability measure and consider its information projection defined by $P_* = \arg\min_{P \in \mathcal{P}} KL(P || \mu)$ and $\mathcal{P}$ is some convex family of probability ...
- 141
0
votes
0
answers
29
views
Cross-covariance in context of Andrews plot
As shown in this Cross-Validated post
Close curves on an Andrews plot
I don't understand how, in the accepted answer, the cross-covariance can be defined as,
$$\int_{-\pi}^{\pi}f_xf_ydt$$
Considering ...
1
vote
0
answers
30
views
Range of error in linear regression [closed]
In linear regression whose matrix form is
$$Y = Xb + E$$
where $X=[X_1,X_2]$.
If the error of using only $X_1$ (i.e $ Y = X_1b+E$ ) is $x_1$, and the error of using $X_2$ (i.e $ Y=X_2b+E$) is $e_2$, ...
- 121
0
votes
0
answers
33
views
What function yields a hat matrix with the smallest Frobenius norm to a given hat matrix?
Phrasing Attempt 1
If I have one function $f_1: \mathcal{X} \rightarrow \mathbb{R}^{D_1}$ that yields a particular hat matrix $P_1 \in \mathbb{R}^{N \times N}$, how do I find the function $f_2: \...
- 1,057
0
votes
0
answers
8
views
Why the inverse for unknown coefficients vector? [duplicate]
From my understanding, this formula is used for least-squares when we're interested in minimizing the distance between a point and some space we are projecting on.
Somebody can correct me if this is ...
- 1
0
votes
0
answers
74
views
Why Expectation for estimates of projection in least squares
I understand that in the least square setting, the projection of $\mathbf{y}$ onto the column space of $\mathbf{X}$ is the vector $\mathbf{Xb}$, where the vector $$\mathbf{b=(X'X)^{-1}Xy}$$ Thus the ...
- 109
0
votes
1
answer
156
views
Projection operator: why squared norm of the sum of them is equal (or smaller) than the sum of the squared norms?
I am working through the proof of Lemma 2 in this paper (page 25, need it for my own research) and I am stuck at the very first step. Here, I will formulate a bit simplified version of this step.
...
- 733
3
votes
1
answer
67
views
Prediction error for ARMA process
Let $X(t)= \phi X_{t-12}+Z_t+\theta Z_{t-1}$ where $Z_t\sim WN(0,1)$. I need to find prediction error for projecting $X_t$ onto $H_{t-3}(X)$ (Hilbert space).
So, I know that $X_t \perp P_{H_{t-3}}X_t$ ...
- 380
1
vote
2
answers
229
views
How to show that smoothing spline fit preserves the local regression part of the fit
We need to show that a smoothing spline of $y_i$ to $x_i$ retains the local regression part of the fit.
For linear regression, this problem seems trivial because it is relatively easy to move from $...
- 27
1
vote
0
answers
34
views
Finding Covariance from linear algebra projection chapter
I am trying to solid background of linear regression by using linear algebra.
In linear algebra, there are some chapters that related to linear regression. (orthogonality, Projection)
I learned some ...
- 11
1
vote
0
answers
43
views
In PCA, why does maximising projections maximise variance?
My understanding could be wrong as the lecture series from my University isn't clear and there are no links to share.
But having rewatched a few times, I think the point being made is:
. We want the ...
- 109
0
votes
1
answer
248
views
Linear Models Hat matrix
For OLS in matrix form, we are taught that Hat matrix is $X(X^TX)^-X^T$, and is idempotent etc, i.e. when it multiplies with itself, it will self cancel and thus lead back to the same Hat matrix.
I ...
- 185
5
votes
1
answer
2k
views
In machine learning, especially deep learning, is it a principle to only do linear projections to a smaller dimension size?
Linear projection (or fully connected layer) is perhaps one of the most common operations in deep learning models. When doing linear projection, we can project a vector $x$ of dimension $n$ to a ...
- 163
4
votes
1
answer
832
views
Why are covariance matrices projected by both right and left multiply?
I've been doing a lot of Kalman filtering work recently. I've derived all the equations starting from a basic linear inverse problem, so strictly speaking I know where everything comes from. I also ...
- 1,367
1
vote
1
answer
59
views
Can I use a single letter or symbol to represent "actual" in a table?
I have a chart that displays numbers for 2017, 2018, 2019 and 2020, but the 2020 numbers are based on projections - I want to display the "actual" value on the data label next to the ...
- 113
1
vote
0
answers
133
views
Finding a Projection Plane in Dimensionality Reduction (e.g., Multidimensional Scaling)
I have a set of data points in high-dimensional space that I wish to map onto a lower dimension (3D or 2D).
Question :
How do I obtain the Projection (Hyper)Plane (e.g., its normal vector or its set ...
- 11
5
votes
1
answer
2k
views
Intuition behind projection matrix
I'm new to machine learning and came across projection matrix . In a random thread it was interpreted as
The matrix $X(X^\text{T} X)^{-1} X^\text{T}$ is a projection matrix, as it does precisely that:...
- 65
3
votes
1
answer
2k
views
How is conditional expectation related to projection? [duplicate]
I vaguely remember seeing somewhere that the conditional expectation $E(Y|X)$ can be interpreted as projection of random variable $Y$ onto random variable $X$. My question is:
Is the aforementioned ...
- 3,028
3
votes
1
answer
139
views
Projection vs fixed effects
Suppose I have $n$ observation indexed by $i$ and that each observation is part of a group $g$. I want to compare two regressions.
First regression:
$$
Y_i=\beta X_i + \alpha F_{g(i)}+\varepsilon_i
$$
...
- 257
1
vote
0
answers
140
views
Orthogonalization vs fixed effects in instrumental variable estimation
Suppose that I have a set of observations indexed by $i$. Each observation belongs to a group $g$. Let's define by $\hat{Z}_i=Z_i - E\left[Z_i\vert g(i)\right]$ is the residual after projecting $Z$ on ...
- 257
3
votes
1
answer
1k
views
Projecting a covariance matrix to a lower dimensional space
I have a point $\mathbf{x}$ in 3-dimensional space, which is measured with a degree of uncertainty. The point falls within a unit cube, and the uncertainty is assumed to follow a multivariate normal ...
- 31
0
votes
1
answer
56
views
Project time series from previous time series examples and characteristics
Say I want to open a shop but first I want to project the likely sales in the first 5 years to see if it is a viable option.
I have data pertaining to 100s of other start ups, including their success ...
- 133
1
vote
1
answer
481
views
Question about Regression error and the residual maker matrix
Starting with the 'residual maker' defined by M in:
$e= y-\hat{y} = Y-X(X'X)^{-1}X'Y = [I-X(X'X)^{-1}X']Y =MY$
where e is the regression residual.
one common equality i see relating the regression ...
- 711
3
votes
0
answers
126
views
Is $\hat \beta$ of general least squares an orthogonal projection?
$\hat \beta_{GLS} = (X'V^{-1}X)^{-1}X'V^{-1}Y \\
Y=X\hat \beta_{GLS} + \epsilon=X(X'V^{-1}X)^{-1}X'V^{-1}Y + \epsilon = \\X(X'V^{-1}X)^{-1}X'V^{-1}Y + (I-X(X'V^{-1}X)^{-1}X'V^{-1})Y$
It is clear that ...
- 3,670
1
vote
0
answers
82
views
Intercept of linear projection
I'm reading a paper and wondering whether I get things right: Say I have a variable Y and two further variables $X_1$ and $X_2$. I want to determine the part of $Y$ which is not linearly explained by $...
- 11
2
votes
1
answer
2k
views
Projecting new samples onto PCA space is failing
After performing PCA I would like to project any new samples to the principal component space (I would like to see how samples cluster together). I did the PCA analysis in R:
...
2
votes
1
answer
788
views
Simple Linear Regression: Hat-Value $h_i$
I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
- 199
2
votes
0
answers
91
views
Boundary point errors in PCA projection using sklearn
I am preparing a small example of a projection using python, numpy and sklearn to perform <...
- 473
1
vote
2
answers
274
views
Difficulty in understanding the proof of the Wold decomposition theorem
The proof of the Wold Decomposition [1] of $x_t$ involves the definition of the process
$$w_t = x_t - P_{\mathcal{M}_{t-1}^x} x_t,$$
where $x_t$ is a stationary zero-mean process,
$\mathcal{M}_n^x = ...
- 307
3
votes
0
answers
228
views
What is the geometric meaning of correlation matrix
I recently read this article explaining the geometric meaning of covariance matrix.
http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/
My question is : is there an ...
- 1,509
2
votes
1
answer
298
views
Best linear prediction as a projection in a Hilbert space $L^2$
Consider two random variables $Y$ and $X$. In the context of the best linear prediction, if we would like to predict $Y$ given $X$ known, we derive the solution solving the following minimize problem
...
- 1,007
2
votes
0
answers
253
views
The joint distribution of Y=AX and Z=BX given a projection matrix A and residual maker matrix B, and a random vector X with known pdf?
This question follows on from a previous question I asked which was answered. It turns out my question lacked some important details, which was revealed by the answer posted on that thread. This is ...
- 63
1
vote
0
answers
230
views
Least Squares Fit Covariance Matrix
How do you perform a least squares fit for $\Sigma$ in the equation $v = u^T\Sigma u$? The term $v$ is a vector of observed variances of a projected Gaussian distribution. The matrix $u$ is made of ...
4
votes
1
answer
774
views
Higher moments of linear regression residuals?
I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations.
Background
In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
- 641
4
votes
1
answer
1k
views
Expected value as an orthogonal projection
I'm reading a paper in which the expected value of a random variable, $\mathbb{E}[X]$, is characterized as an orthogonal projection. This is on page 10. I've seen the geometric interpretation of ...
- 439