Questions tagged [projection]

For on-topic questions involving the mathematical concept projection, a linear transformation $P$ such that $P=P^2$. Please include also a more statistical methods tag. For purely mathematical questions about projections it is better to ask on math SE https://math.stackexchange.com/

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

projection matrix and matrix position

I am currently making exercise regarding deriving the variance of an unbiased but inefficient estimator in Restricted Least Square(RLS) model that is shown below $\boldsymbol{b = \hat{\beta}_{ols}-X'...
1
vote
2answers
58 views

Difficulty in understanding the proof of the Wold decomposition theorem

The proof of the Wold Decomposition [1] of $x_t$ involves the definition of the process $$w_t = x_t - P_{\mathcal{M}_{t-1}^x} x_t,$$ where $x_t$ is a stationary zero-mean process, $\mathcal{M}_n^x = ...
3
votes
0answers
58 views

What is the geometric meaning of correlation matrix

I recently read this article explaining the geometric meaning of covariance matrix. http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/ My question is : is there an ...
0
votes
1answer
21 views

Best linear prediction as a projection in a Hilbert space $L^2$

Consider two random variables $Y$ and $X$. In the context of the best linear prediction, if we would like to predict $Y$ given $X$ known, we derive the solution solving the following minimize problem ...
1
vote
0answers
26 views

The joint distribution of Y=AX and Z=BX given a projection matrix A and residual maker matrix B, and a random vector X with known pdf?

This question follows on from a previous question I asked which was answered. It turns out my question lacked some important details, which was revealed by the answer posted on that thread. This is ...
0
votes
0answers
19 views

How can I recover full dimensional VAR model coefficients after fitting a VAR model to a dimensionality reduced (via PCA) dataset?

I am using PCA to reduce dimensionality prior to fitting a multivariate time-series dataset to a VAR (vector autoregressive) model. Is there any way to convert a PCA-derived VAR model to a full ...
1
vote
0answers
54 views

Least Squares Fit Covariance Matrix

How do you perform a least squares fit for $\Sigma$ in the equation $v = u^T\Sigma u$? The term $v$ is a vector of observed variances of a projected Gaussian distribution. The matrix $u$ is made of ...
1
vote
0answers
55 views

Higher moments of linear regression residuals?

I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations. Background In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
2
votes
1answer
123 views

Expected value as an orthogonal projection

I'm reading a paper in which the expected value of a random variable, $\mathbb{E}[X]$, is characterized as an orthogonal projection. This is on page 10. I've seen the geometric interpretation of ...
0
votes
0answers
33 views

Linear project when X includes a constant

In Hamilton's text I ran across the following statement: if $X$ includes a constant then the linear projection of $aY+b$ is $aP(Y\mid X)+b$ where $P(Y\mid X)$ is the linear projection of $y$ on $x$. I’...
0
votes
0answers
115 views

Linear regression with feature representation confusion - relationship of design matrix column space to the feature space?

I am trying to visualise the geometry of linear regression with feature representation. I have a regression problem with $n$ data pairs $\mathcal{D}:=\{(\mathbf{x},y)_{i}\}_{i=1}^{n}$, independent ...
2
votes
0answers
173 views

What's wrong with my solution to canonical correlation analysis (CCA) using the SVD

I am working through the derivations for solving CCA in A Tutorial on Canonical Correlation Methods. Right now, I am trying to solve CCA using SVD (bottom of page 95:7). For completeness, I include ...
2
votes
1answer
652 views

t-SNE, PCA and another technique - which one? [closed]

I am comparing dimension reduction techniques and I am utilizing them for data visualizations onto a plane – projections in 2D space. The input into a projection/dimension reduction techniques is a ...
2
votes
1answer
122 views

Conditional expectation and variable decomposition

Suppose that $X$ and $Y$ have an uknown joint distribution $f_{XY}$. How can I formally demostrate that it always exists a unique decomposition of the form : $$ Y = E[Y|X] +\epsilon $$ without ...
0
votes
0answers
13 views

Embedding Dimension finds same projection direction with more projection dimensions

I have the following data setup: $$X \in \mathbb{R}^{n \times d}$$ $$Y \in \{1, ..., K\}^n$$ And I want to find a low-dimensional representation (something like PCA): $$A^r = embedding(X, Y, r)$$ ...
0
votes
1answer
16 views

Model Selection - 6 month forecast given the past 25 months

As the title states, the problem at hand is asking me to predict the next 6 months values when given the past 25 months. In my opinion, the training data will be quite thin so traditional time series ...
1
vote
0answers
33 views

Can anyone help me with manipulating the projection matrices?

I am trying to prove the below equality to prove that the squared correlation of coefficient is equal to r-square(i.e. $r^2 = R^2$): $$Y'M_\iota \hat{Y} = \hat{Y}'M_\iota \hat{Y}$$ where $\hat{Y} = X\...
0
votes
1answer
492 views

Projection of a vector on unit sphere

I am currently reading the paper G.Salton et al, A Vector Space Model for Automatic Indexing, November 1975, Volume 18 It is about indexing documents. Representing them in vectors to find ...
0
votes
1answer
97 views

Vertically translated depreciation curve: Update the exponential regression coefficient

I have an exponential regression equation that I use to predict the condition of roads. The equation can be found on page 53 of the original master's thesis: Development of a Flexible Framework for ...
1
vote
1answer
87 views

Road condition: Project future condition by appending part of a deterioration curve?

I inspect a road network's condition every three years: ...
0
votes
1answer
125 views

What is wrong with my computation of projections on the first principal component?

Following the links in parenthesis (link1, link2) I wrote a bit of code in MATLAB to simulate a PCA. After running the code and plotting the results I obtain this ...
2
votes
0answers
201 views

Markov chain which is also a projection

Let $P$ be the transition matrix of a Markov chain, and assume that $P^2=P$. One immediate conclusion is that $P=P^\infty$. Furthermore, assume that there is a state $i$ such as each state $j$ (...
4
votes
1answer
448 views

How to show that demeaning the data in design matrix does not change the hat matrix

When I have a design matrix $$X = \begin{bmatrix} 1 & x_{11} & \ldots & x_{1k}\\ 1 & x_{21} & \ldots & x_{2k}\\ \vdots & \vdots & \ddots & \vdots\\ 1 & ...
3
votes
1answer
265 views

Method of alternating projections for linear fixed effects models [closed]

The standard fixed effects model (in econometrics, mostly) is $$ y = \mathbf{X\beta} + \mathbf{D\alpha} + \epsilon $$ where $\mathbf{D}$ is a set of factors, potentially with thousands of levels. ...
0
votes
1answer
241 views

Orthogonal and stationary basis of projection

Consider a set of $p$ stationary correlated time signals of identical duration $T$. I would like to de-correlate them by projection on an orthogonal basis but preserving stationarity. Without the ...
0
votes
1answer
345 views

Can I Use The Same Principal Components Of One Dataset To Graph Another Dataset In R?

I have two matrices of equal dimensions $(m \times n)$. If it helps, one matrix is the raw data, while the other is transformed. I've plotted PC1 vs PC2 for both datasets individually, but then I ...
5
votes
1answer
752 views

Intuition using linear algebra that the rank of the projection matrix equals the rank of the design matrix

Using linear algebra to explain, can someone show the intuition? I can show that the ranks are the same by using properties of rank but can't get my head around the whole projection thing more than ...
2
votes
1answer
415 views

Projected Gradient Descent

Consider the primal SVM problem: $$ \frac12||w||^2 +\frac Cm \sum_{i=1}^m \max(0,1-y_iw\cdot x_i) $$ We want to find a solution with a bounded norm, by using SGD with a projection onto the convex set:...
0
votes
2answers
2k views

Projection Matrix in linear regression(and difference between Projection Matrix in linear Algebra)

In linear algebra class I learned, that $$\begin{equation*} \hat{Y} = X \hat{\beta} = X\,\left(X^\prime X \right)^{-1} \, X^\prime Y = P\,Y \end{equation*}$$ , where \begin{equation*} P \equiv X\,\...
2
votes
0answers
44 views

What model should I use for retirement forecasting?

I am a HR professional looking to self learn statistical modeling for new responsibilities at work. I need to forecast no. of employees who may retire next 10 years. What would be simple way to ...
2
votes
1answer
202 views

Projection Matrix and linear model(Question about notation)

I just want to ask a question about notation in this exercise. In Equation $X^*_{.,i} =M_{X.,-i} * X_i$ ; $X^*_{.,i}$ means ith column of original matrix $M_{X.,-i}$ means orthogonal projection ...
1
vote
1answer
638 views

Sampling from marginal distribution using joint sample? [duplicate]

I have a sample of a multivariate distribution, and I am interested in obtaining a sample from the marginals. I know the right way to do so is by simply taking the corresponding entries. What is the ...
3
votes
0answers
205 views

Temporal Multi Dimensional Scaling

Let's say I apply a multidimensional scaling(MDS) to a dynamic dataset of $n$ points (eg, time series). At each step I will obtain a projection (in 2/3D) of the $n$ points. If nothing meaningful ...
4
votes
3answers
2k views

Since a projection matrix is idempotent, symmetric and square, why isn't it just the identity matrix?

I was working on a question on projection matrix. Since, projection matrix is idempotent, symmetric and square matrix, it must always be equal to $I$ (Identity matrix). This can be shown by ...
6
votes
3answers
694 views

Projecting data on a sphere

I am used to working with PCA, tSNE, LLEs... They all do a great job projecting the data on a plane (or on linear subspaces of $\mathbb{R}^n$). Is there any other embedding technique that projects the ...
2
votes
0answers
54 views

Changing the basis of random variables

Let $X_1$ and $X_2$ be two independent (1-dimensional) random variables and let $Y_1 = f^1(X_1, X_2)$ ($f^1$ is a deterministic function) be a (1-dimensional) random variable too. Question Does ...
2
votes
0answers
550 views

Find projection matrix using partitioned matrices

If $X$ is a ($n$, $p+1$) design matrix, partition $X$ to be $X$=[$J$ $X$*] where $J$ is a ($n$,$1$) vector of all $1$'s, and $X$* is a ($n$,$p$) matrix. Let $H_X$ be a projection matrix, where $H_X$...
1
vote
1answer
149 views

How to project data onto a model (specifically, GMM)?

I'm using data to train a Gaussian mixture model (GMM). I then take a sample and would like to see its projection on the GMM 'space'. I can think of an optimization problem such as this: consider $y$ ...
6
votes
1answer
956 views

Relationship between Linear Projection and OLS Regression

In Wooldridge's Econometric Analysis of Cross Section and Panel Data, he defines linear projection of $y$ on $1,\mathbf{x}$, in the following way: Let's assume that $Var(\mathbf{x})$ is positive-...
2
votes
1answer
126 views

Independence of random projection of Gaussian random vector

If $v$, $y$ and $h$ are independent Gaussian random vectors, are $|v^H y|^2$ and $|v^H h|^2$ independent?
0
votes
1answer
423 views

Transforming data on the Principal Components axis?

x y 2.5 2.4 0.5 0.7 2.2 2.9 1.9 2.2 3.1 3.0 2.3 2.7 2 1.6 1 1.1 1.5 1.6 1.1 0.9 Eigenvalues: ...
13
votes
3answers
8k views

Utility of the Frisch-Waugh theorem

I am supposed to teach the Frish Waugh theorem in econometrics, which I have not studied. I have understood the maths behind it and I hope the idea too "the coefficient you get for a particular ...
1
vote
0answers
56 views

Proof: Ratio of same normals is uniform?

I am currently working through a proof that uses (without reference) the following result. I cannot find a proof to this result, and would appreciate any help. Denote the stacked regression as $Y = X\...
6
votes
1answer
784 views

Do the principal components change if we apply PCA more than once (recursively) on data?

Consider a set $X=(X_1; \dots; X_n)$ of $n$ data points such that $X_i \in \mathbb{R}^d$ is a column vector. Let $Y = \text{pca_proj}(X)$ denote the projection of points in $X$ according to the PCA ...
4
votes
0answers
93 views

Projection of a spherical t-distribution

I'm working with an $n$-dimensional spherical t-distribution for which the density is defined as $f(x) = \frac{\Gamma(\frac{n+\nu}{2})}{(\pi \sigma^2\nu)^{\frac n2}\Gamma(\frac \nu2)} \left( 1 + \...
1
vote
1answer
719 views

The projection matrix and proof of an unbiased estimator for sigma-squared

Hi, given this information we are meant to prove that the above estimator is unbiased. I understand the proof for the most part (below). What I do not understand is the intuitive reason why the ...
0
votes
2answers
151 views

$E(u x)=0$ and endogeneity

I am confused as to how Endogeneity arises. I understand all the examples but something basic just doesn't fit: The usual assumption is that $E(xu) \ne 0$. But, we can always find $\beta$'s that ...
1
vote
0answers
245 views

Proving Orthogonal Properties of Projection Matrix

I am working through a multi-part proof of how orthogonal projection matrices give specific results from their properties. I read through the Gauss-Markov model theory to get a start. This is only a ...
1
vote
1answer
130 views

Projection Matrix Help

I'm having trouble understanding the following in a review textbook I'm using, particularly the long equality with inverses and transposes, and the subsequent conclusion regarding the rank and trace ...
2
votes
2answers
2k views

Linear Discriminant Analysis Newbie question

Does linear discriminant analysis always project the points to a line? Most of the graphical illustrations of LDA that I see online use an example of 2 dimensional points which are projected onto a ...