Questions tagged [projection]
For on-topic questions involving the mathematical concept projection, a linear transformation $P$ such that $P=P^2$. Please include also a more statistical methods tag. For purely mathematical questions about projections it is better to ask on math SE https://math.stackexchange.com/
87
questions
2
votes
0
answers
17
views
Relationship between conditional expectation and regression
I would be grateful if you could help me clear up some confusion regarding conditional expectation and regression.
I have seen two formulations of the linear regression framework:
$$Y=a+bX+\varepsilon\...
0
votes
0
answers
11
views
So how can I project the data with the eigenvectors from LDA?
I have data that look like this.
And my goal is to reduce this 3D dimension into 2D dimension so it might looks like this. Turning the angle so the distance between all classes becomes maximum.
So ...
- 317
0
votes
1
answer
200
views
Proof that leverage points are between 0 and 1 inclusive
I would really appreciate it if anyone can guide me through this.
I have a $n \times (p+1)$ matrix $X$. The projection matrix $P = X(X'X)^{-1}X'$. I want to prove that $P(i,i)$ is in $[0,1]$, where $P(...
0
votes
0
answers
14
views
Is Cartesian coordinate system better than GCS for semi variogram analysis?
My research focuses on semi-variograms for interpolation and mapping using geographic CS projected data in the geoR package in R....
- 1
2
votes
1
answer
54
views
OLS regression and dummy variables: fitted values of the subsample equal actual values?
Given $X=(d\; X_1)$, we want to prove that
$$(d'd)^{-1}d'P_{[d X_1]}y=(d'd)^{-1}d'y,$$ where $P_{[X]}y=\hat{y}.$ That is, restricting the regression to the subsample for which $d_i=1$, we have that ...
- 65
1
vote
0
answers
49
views
The principal submatrix of projection matrix with Gaussian design
I've come across a phenomenon from a simulation that I'm very curious about.
But I don't know how to start my analysis. So, I am asking for some guidance.
Thanks!
Denote by $\mathbf{H}$ the principal ...
- 115
0
votes
2
answers
64
views
Hayashi Econometrics Seemingly Unrelated Regressions (SUR) Eq 4.5.13'-15'
According to 4.5.13
$\hat{A}_{mh}=\left(\frac{1}{n}\sum_{i=1}^{n}z_{im}x_{i}^{\prime}\right)\left(\frac{1}{n}\sum_{i=1}^{n}x_{i}x_{i}^{\prime}\right)^{-1}\left(\frac{1}{n}\sum_{i=1}^{n}x_{i}z_{ih}^{\...
- 11
1
vote
0
answers
58
views
Pythagorean Theorem for inclusive Kullback-Leibler
Let $\mu$ be some probability measure and consider its information projection defined by $P_* = \arg\min_{P \in \mathcal{P}} KL(P || \mu)$ and $\mathcal{P}$ is some convex family of probability ...
- 141
0
votes
0
answers
22
views
Cross-covariance in context of Andrews plot
As shown in this Cross-Validated post
Close curves on an Andrews plot
I don't understand how, in the accepted answer, the cross-covariance can be defined as,
$$\int_{-\pi}^{\pi}f_xf_ydt$$
Considering ...
1
vote
0
answers
25
views
Range of error in linear regression [closed]
In linear regression whose matrix form is
$$Y = Xb + E$$
where $X=[X_1,X_2]$.
If the error of using only $X_1$ (i.e $ Y = X_1b+E$ ) is $x_1$, and the error of using $X_2$ (i.e $ Y=X_2b+E$) is $e_2$, ...
- 121
0
votes
0
answers
28
views
What function yields a hat matrix with the smallest Frobenius norm to a given hat matrix?
Phrasing Attempt 1
If I have one function $f_1: \mathcal{X} \rightarrow \mathbb{R}^{D_1}$ that yields a particular hat matrix $P_1 \in \mathbb{R}^{N \times N}$, how do I find the function $f_2: \...
- 771
0
votes
0
answers
8
views
Why the inverse for unknown coefficients vector? [duplicate]
From my understanding, this formula is used for least-squares when we're interested in minimizing the distance between a point and some space we are projecting on.
Somebody can correct me if this is ...
- 1
0
votes
0
answers
69
views
Why Expectation for estimates of projection in least squares
I understand that in the least square setting, the projection of $\mathbf{y}$ onto the column space of $\mathbf{X}$ is the vector $\mathbf{Xb}$, where the vector $$\mathbf{b=(X'X)^{-1}Xy}$$ Thus the ...
- 99
0
votes
1
answer
95
views
Projection operator: why squared norm of the sum of them is equal (or smaller) than the sum of the squared norms?
I am working through the proof of Lemma 2 in this paper (page 25, need it for my own research) and I am stuck at the very first step. Here, I will formulate a bit simplified version of this step.
...
- 713
3
votes
1
answer
61
views
Prediction error for ARMA process
Let $X(t)= \phi X_{t-12}+Z_t+\theta Z_{t-1}$ where $Z_t\sim WN(0,1)$. I need to find prediction error for projecting $X_t$ onto $H_{t-3}(X)$ (Hilbert space).
So, I know that $X_t \perp P_{H_{t-3}}X_t$ ...
- 288
1
vote
2
answers
193
views
How to show that smoothing spline fit preserves the local regression part of the fit
We need to show that a smoothing spline of $y_i$ to $x_i$ retains the local regression part of the fit.
For linear regression, this problem seems trivial because it is relatively easy to move from $...
- 27
1
vote
0
answers
29
views
Finding Covariance from linear algebra projection chapter
I am trying to solid background of linear regression by using linear algebra.
In linear algebra, there are some chapters that related to linear regression. (orthogonality, Projection)
I learned some ...
- 11
1
vote
0
answers
24
views
In PCA, why does maximising projections maximise variance?
My understanding could be wrong as the lecture series from my University isn't clear and there are no links to share.
But having rewatched a few times, I think the point being made is:
. We want the ...
- 109
0
votes
1
answer
196
views
Linear Models Hat matrix
For OLS in matrix form, we are taught that Hat matrix is $X(X^TX)^-X^T$, and is idempotent etc, i.e. when it multiplies with itself, it will self cancel and thus lead back to the same Hat matrix.
I ...
- 143
4
votes
1
answer
856
views
In machine learning, especially deep learning, is it a principle to only do linear projections to a smaller dimension size?
Linear projection (or fully connected layer) is perhaps one of the most common operations in deep learning models. When doing linear projection, we can project a vector $x$ of dimension $n$ to a ...
- 153
2
votes
1
answer
446
views
Why are covariance matrices projected by both right and left multiply?
I've been doing a lot of Kalman filtering work recently. I've derived all the equations starting from a basic linear inverse problem, so strictly speaking I know where everything comes from. I also ...
- 1,197
1
vote
1
answer
48
views
Can I use a single letter or symbol to represent "actual" in a table?
I have a chart that displays numbers for 2017, 2018, 2019 and 2020, but the 2020 numbers are based on projections - I want to display the "actual" value on the data label next to the ...
- 113
1
vote
0
answers
86
views
Finding a Projection Plane in Dimensionality Reduction (e.g., Multidimensional Scaling)
I have a set of data points in high-dimensional space that I wish to map onto a lower dimension (3D or 2D).
Question :
How do I obtain the Projection (Hyper)Plane (e.g., its normal vector or its set ...
- 11
5
votes
1
answer
1k
views
Intuition behind projection matrix
I'm new to machine learning and came across projection matrix . In a random thread it was interpreted as
The matrix $X(X^\text{T} X)^{-1} X^\text{T}$ is a projection matrix, as it does precisely that:...
- 65
2
votes
0
answers
1k
views
How is conditional expectation related to projection?
I vaguely remember seeing somewhere that the conditional expectation $E(Y|X)$ can be interpreted as projection of random variable $Y$ onto random variable $X$. My question is:
Is the aforementioned ...
- 2,412
2
votes
1
answer
122
views
Projection vs fixed effects
Suppose I have $n$ observation indexed by $i$ and that each observation is part of a group $g$. I want to compare two regressions.
First regression:
$$
Y_i=\beta X_i + \alpha F_{g(i)}+\varepsilon_i
$$
...
- 227
1
vote
0
answers
105
views
Orthogonalization vs fixed effects in instrumental variable estimation
Suppose that I have a set of observations indexed by $i$. Each observation belongs to a group $g$. Let's define by $\hat{Z}_i=Z_i - E\left[Z_i\vert g(i)\right]$ is the residual after projecting $Z$ on ...
- 227
3
votes
1
answer
840
views
Projecting a covariance matrix to a lower dimensional space
I have a point $\mathbf{x}$ in 3-dimensional space, which is measured with a degree of uncertainty. The point falls within a unit cube, and the uncertainty is assumed to follow a multivariate normal ...
- 31
0
votes
1
answer
43
views
Project time series from previous time series examples and characteristics
Say I want to open a shop but first I want to project the likely sales in the first 5 years to see if it is a viable option.
I have data pertaining to 100s of other start ups, including their success ...
- 133
1
vote
1
answer
370
views
Question about Regression error and the residual maker matrix
Starting with the 'residual maker' defined by M in:
$e= y-\hat{y} = Y-X(X'X)^{-1}X'Y = [I-X(X'X)^{-1}X']Y =MY$
where e is the regression residual.
one common equality i see relating the regression ...
- 601
3
votes
0
answers
87
views
Is $\hat \beta$ of general least squares an orthogonal projection?
$\hat \beta_{GLS} = (X'V^{-1}X)^{-1}X'V^{-1}Y \\
Y=X\hat \beta_{GLS} + \epsilon=X(X'V^{-1}X)^{-1}X'V^{-1}Y + \epsilon = \\X(X'V^{-1}X)^{-1}X'V^{-1}Y + (I-X(X'V^{-1}X)^{-1}X'V^{-1})Y$
It is clear that ...
- 3,027
1
vote
0
answers
70
views
Intercept of linear projection
I'm reading a paper and wondering whether I get things right: Say I have a variable Y and two further variables $X_1$ and $X_2$. I want to determine the part of $Y$ which is not linearly explained by $...
- 11
2
votes
1
answer
1k
views
Projecting new samples onto PCA space is failing
After performing PCA I would like to project any new samples to the principal component space (I would like to see how samples cluster together). I did the PCA analysis in R:
...
2
votes
1
answer
513
views
Simple Linear Regression: Hat-Value $h_i$
I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
- 179
2
votes
0
answers
74
views
Boundary point errors in PCA projection using sklearn
I am preparing a small example of a projection using python, numpy and sklearn to perform <...
- 463
1
vote
2
answers
243
views
Difficulty in understanding the proof of the Wold decomposition theorem
The proof of the Wold Decomposition [1] of $x_t$ involves the definition of the process
$$w_t = x_t - P_{\mathcal{M}_{t-1}^x} x_t,$$
where $x_t$ is a stationary zero-mean process,
$\mathcal{M}_n^x = ...
- 297
3
votes
0
answers
176
views
What is the geometric meaning of correlation matrix
I recently read this article explaining the geometric meaning of covariance matrix.
http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/
My question is : is there an ...
- 1,489
2
votes
1
answer
189
views
Best linear prediction as a projection in a Hilbert space $L^2$
Consider two random variables $Y$ and $X$. In the context of the best linear prediction, if we would like to predict $Y$ given $X$ known, we derive the solution solving the following minimize problem
...
- 1,017
1
vote
0
answers
179
views
The joint distribution of Y=AX and Z=BX given a projection matrix A and residual maker matrix B, and a random vector X with known pdf?
This question follows on from a previous question I asked which was answered. It turns out my question lacked some important details, which was revealed by the answer posted on that thread. This is ...
- 53
1
vote
0
answers
218
views
Least Squares Fit Covariance Matrix
How do you perform a least squares fit for $\Sigma$ in the equation $v = u^T\Sigma u$? The term $v$ is a vector of observed variances of a projected Gaussian distribution. The matrix $u$ is made of ...
4
votes
1
answer
581
views
Higher moments of linear regression residuals?
I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations.
Background
In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
- 561
4
votes
1
answer
1k
views
Expected value as an orthogonal projection
I'm reading a paper in which the expected value of a random variable, $\mathbb{E}[X]$, is characterized as an orthogonal projection. This is on page 10. I've seen the geometric interpretation of ...
- 347
0
votes
0
answers
36
views
Linear project when X includes a constant
In Hamilton's text I ran across the following statement: if $X$ includes a constant then the linear projection of $aY+b$ is $aP(Y\mid X)+b$ where $P(Y\mid X)$ is the linear projection of $y$ on $x$. I’...
- 243
0
votes
0
answers
235
views
Linear regression with feature representation confusion - relationship of design matrix column space to the feature space?
I am trying to visualise the geometry of linear regression with feature representation. I have a regression problem with $n$ data pairs $\mathcal{D}:=\{(\mathbf{x},y)_{i}\}_{i=1}^{n}$, independent ...
- 223
3
votes
0
answers
345
views
What's wrong with my solution to canonical correlation analysis (CCA) using the SVD? [duplicate]
I am working through the derivations for solving CCA in A Tutorial on Canonical Correlation Methods. Right now, I am trying to solve CCA using SVD (bottom of page 95:7). For completeness, I include ...
- 1,594
2
votes
1
answer
2k
views
t-SNE, PCA and another technique - which one? [closed]
I am comparing dimension reduction techniques and I am utilizing them for data visualizations onto a plane – projections in 2D space.
The input into a projection/dimension reduction techniques is a ...
- 31
5
votes
1
answer
820
views
Conditional expectation and variable decomposition
Suppose that $X$ and $Y$ have an uknown joint distribution $f_{XY}$.
How can I formally demostrate that it always exists a unique decomposition of the form :
$$ Y = E[Y|X] +\epsilon $$
without ...
- 417
0
votes
1
answer
32
views
Model Selection - 6 month forecast given the past 25 months
As the title states, the problem at hand is asking me to predict the next 6 months values when given the past 25 months. In my opinion, the training data will be quite thin so traditional time series ...
- 121
0
votes
0
answers
38
views
Can anyone help me with manipulating the projection matrices?
I am trying to prove the below equality to prove that the squared correlation of coefficient is equal to r-square(i.e. $r^2 = R^2$): $$Y'M_\iota \hat{Y} = \hat{Y}'M_\iota \hat{Y}$$ where $\hat{Y} = X\...
- 65
0
votes
1
answer
1k
views
Projection of a vector on unit sphere
I am currently reading the paper G.Salton et al, A Vector Space Model for Automatic Indexing, November 1975, Volume 18
It is about indexing documents.
Representing them in vectors to find ...
- 229