Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [projection]

For on-topic questions involving the mathematical concept projection, a linear transformation $P$ such that $P=P^2$. Please include also a more statistical methods tag. For purely mathematical questions about projections it is better to ask on math SE https://math.stackexchange.com/

0
votes
0answers
24 views

Expected value as an orthogonal projection

I'm reading a paper in which the expected value of a random variable, $\mathbb{E}[X]$, is characterized as an orthogonal projection. This is on page 10. I've seen the geometric interpretation of ...
0
votes
0answers
33 views

Linear project when X includes a constant

In Hamilton's text I ran across the following statement: if $X$ includes a constant then the linear projection of $aY+b$ is $aP(Y\mid X)+b$ where $P(Y\mid X)$ is the linear projection of $y$ on $x$. I’...
0
votes
0answers
80 views

Linear regression with feature representation confusion - relationship of design matrix column space to the feature space?

I am trying to visualise the geometry of linear regression with feature representation. I have a regression problem with $n$ data pairs $\mathcal{D}:=\{(\mathbf{x},y)_{i}\}_{i=1}^{n}$, independent ...
1
vote
0answers
59 views

What's wrong with my solution to canonical correlation analysis (CCA) using the SVD

I am working through the derivations for solving CCA in A Tutorial on Canonical Correlation Methods. Right now, I am trying to solve CCA using SVD (bottom of page 95:7). For completeness, I include ...
1
vote
1answer
330 views

t-SNE, PCA and another technique - which one? [closed]

I am comparing dimension reduction techniques and I am utilizing them for data visualizations onto a plane – projections in 2D space. The input into a projection/dimension reduction techniques is a ...
2
votes
1answer
40 views

Conditional expectation and variable decomposition

Suppose that $X$ and $Y$ have an uknown joint distribution $f_{XY}$. How can I formally demostrate that it always exists a unique decomposition of the form : $$ Y = E[Y|X] +\epsilon $$ without ...
0
votes
0answers
12 views

Embedding Dimension finds same projection direction with more projection dimensions

I have the following data setup: $$X \in \mathbb{R}^{n \times d}$$ $$Y \in \{1, ..., K\}^n$$ And I want to find a low-dimensional representation (something like PCA): $$A^r = embedding(X, Y, r)$$ ...
0
votes
1answer
15 views

Model Selection - 6 month forecast given the past 25 months

As the title states, the problem at hand is asking me to predict the next 6 months values when given the past 25 months. In my opinion, the training data will be quite thin so traditional time series ...
0
votes
0answers
70 views

Projection of data point on to principal components

So I would like to understand how much each principal component contributes to explain a given data point (not all data points). I've used the following code to extract 100 principal components from ...
1
vote
0answers
33 views

Can anyone help me with manipulating the projection matrices?

I am trying to prove the below equality to prove that the squared correlation of coefficient is equal to r-square(i.e. $r^2 = R^2$): $$Y'M_\iota \hat{Y} = \hat{Y}'M_\iota \hat{Y}$$ where $\hat{Y} = X\...
0
votes
1answer
298 views

Projection of a vector on unit sphere

I am currently reading this paper G.Salton et al, A Vector Space Model for Automatic Indexing November 1975, Volume 18 It is about indexing documents. Representing them in vectors to find ...
0
votes
1answer
84 views

Vertically translated depreciation curve: Update the exponential regression coefficient

I have an exponential regression equation that I use to predict the condition of roads. The equation can be found on page 53 of the original master's thesis: Development of a Flexible Framework for ...
1
vote
1answer
78 views

Road condition: Project future condition by appending part of a deterioration curve?

I inspect a road network's condition every three years: ...
0
votes
1answer
111 views

What is wrong with my computation of projections on the first principal component?

Following the links in parenthesis (link1, link2) I wrote a bit of code in MATLAB to simulate a PCA. After running the code and plotting the results I obtain this ...
2
votes
0answers
133 views

Markov chain which is also a projection

Let $P$ be the transition matrix of a Markov chain, and assume that $P^2=P$. One immediate conclusion is that $P=P^\infty$. Furthermore, assume that there is a state $i$ such as each state $j$ (...
4
votes
1answer
329 views

How to show that demeaning the data in design matrix does not change the hat matrix

When I have a design matrix $$X = \begin{bmatrix} 1 & x_{11} & \ldots & x_{1k}\\ 1 & x_{21} & \ldots & x_{2k}\\ \vdots & \vdots & \ddots & \vdots\\ 1 & ...
3
votes
1answer
172 views

Method of alternating projections for linear fixed effects models

The standard fixed effects model (in econometrics, mostly) is $$ y = \mathbf{X\beta} + \mathbf{D\alpha} + \epsilon $$ where $\mathbf{D}$ is a set of factors, potentially with thousands of levels. ...
0
votes
1answer
148 views

Orthogonal and stationary basis of projection

Consider a set of $p$ stationary correlated time signals of identical duration $T$. I would like to de-correlate them by projection on an orthogonal basis but preserving stationarity. Without the ...
0
votes
1answer
205 views

Can I Use The Same Principal Components Of One Dataset To Graph Another Dataset In R?

I have two matrices of equal dimensions (m x n). If it helps, one matrix is the raw data, while the other is transformed. I've plotted PC1 vs PC2 for both datasets individually, but then I realized I ...
5
votes
1answer
591 views

Intuition using linear algebra that the rank of the projection matrix equals the rank of the design matrix

Using linear algebra to explain, can someone show the intuition? I can show that the ranks are the same by using properties of rank but can't get my head around the whole projection thing more than ...
3
votes
1answer
328 views

Projected Gradient Descent

Consider the primal SVM problem: $$ \frac12||w||^2 +\frac Cm \sum_{i=1}^m \max(0,1-y_iw\cdot x_i) $$ We want to find a solution with a bounded norm, by using SGD with a projection onto the convex set:...
0
votes
2answers
1k views

Projection Matrix in linear regression(and difference between Projection Matrix in linear Algebra)

In linear algebra class I learned, that $$\begin{equation*} \hat{Y} = X \hat{\beta} = X\,\left(X^\prime X \right)^{-1} \, X^\prime Y = P\,Y \end{equation*}$$ , where \begin{equation*} P \equiv X\,\...
2
votes
0answers
43 views

What model should I use for retirement forecasting?

I am a HR professional looking to self learn statistical modeling for new responsibilities at work. I need to forecast no. of employees who may retire next 10 years. What would be simple way to ...
3
votes
1answer
158 views

Projection Matrix and linear model(Question about notation)

I just want to ask a question about notation in this exercise. In Equation $X^*_{.,i} =M_{X.,-i} * X_i$ ; $X^*_{.,i}$ means ith column of original matrix $M_{X.,-i}$ means orthogonal projection ...
1
vote
1answer
457 views

Sampling from marginal distribution using joint sample? [duplicate]

I have a sample of a multivariate distribution, and I am interested in obtaining a sample from the marginals. I know the right way to do so is by simply taking the corresponding entries. What is the ...
2
votes
0answers
129 views

Temporal Multi Dimensional Scaling

Let's say I apply a multidimensional scaling(MDS) to a dynamic dataset of $n$ points (eg, time series). At each step I will obtain a projection (in 2/3D) of the $n$ points. If nothing meaningful ...
4
votes
3answers
1k views

Since a projection matrix is idempotent, symmetric and square, why isn't it just the identity matrix?

I was working on a question on projection matrix. Since, projection matrix is idempotent, symmetric and square matrix, it must always be equal to $I$ (Identity matrix). This can be shown by ...
5
votes
3answers
511 views

Projecting data on a sphere

I am used to working with PCA, tSNE, LLEs... They all do a great job projecting the data on a plane (or on linear subspaces of $\mathbb{R}^n$). Is there any other embedding technique that projects the ...
3
votes
0answers
44 views

Changing the basis of random variables

Let $X_1$ and $X_2$ be two independent (1-dimensional) random variables and let $Y_1 = f^1(X_1, X_2)$ ($f^1$ is a deterministic function) be a (1-dimensional) random variable too. Question Does ...
2
votes
0answers
437 views

Find projection matrix using partitioned matrices

If $X$ is a ($n$, $p+1$) design matrix, partition $X$ to be $X$=[$J$ $X$*] where $J$ is a ($n$,$1$) vector of all $1$'s, and $X$* is a ($n$,$p$) matrix. Let $H_X$ be a projection matrix, where $H_X$...
1
vote
1answer
112 views

How to project data onto a model (specifically, GMM)?

I'm using data to train a Gaussian mixture model (GMM). I then take a sample and would like to see its projection on the GMM 'space'. I can think of an optimization problem such as this: consider $y$ ...
5
votes
1answer
730 views

Relationship between Linear Projection and OLS Regression

In Wooldridge's Econometric Analysis of Cross Section and Panel Data, he defines linear projection of $y$ on $1,\mathbf{x}$, in the following way: Let's assume that $Var(\mathbf{x})$ is positive-...
2
votes
1answer
107 views

Independence of random projection of Gaussian random vector

If $v$, $y$ and $h$ are independent Gaussian random vectors, are $|v^H y|^2$ and $|v^H h|^2$ independent?
0
votes
1answer
160 views

Transforming data on the Principal Components axis?

x y 2.5 2.4 0.5 0.7 2.2 2.9 1.9 2.2 3.1 3.0 2.3 2.7 2 1.6 1 1.1 1.5 1.6 1.1 0.9 Eigenvalues: ...
13
votes
3answers
6k views

Utility of the Frisch-Waugh theorem

I am supposed to teach the Frish Waugh theorem in econometrics, which I have not studied. I have understood the maths behind it and I hope the idea too "the coefficient you get for a particular ...
1
vote
0answers
51 views

Proof: Ratio of same normals is uniform?

I am currently working through a proof that uses (without reference) the following result. I cannot find a proof to this result, and would appreciate any help. Denote the stacked regression as $Y = X\...
5
votes
1answer
516 views

Do the principal components change if we apply PCA more than once (recursively) on data?

Consider a set $X=(X_1; \dots; X_n)$ of $n$ data points such that $X_i \in \mathbb{R}^d$ is a column vector. Let $Y = \text{pca_proj}(X)$ denote the projection of points in $X$ according to the PCA ...
4
votes
0answers
80 views

Projection of a spherical t-distribution

I'm working with an $n$-dimensional spherical t-distribution for which the density is defined as $f(x) = \frac{\Gamma(\frac{n+\nu}{2})}{(\pi \sigma^2\nu)^{\frac n2}\Gamma(\frac \nu2)} \left( 1 + \...
1
vote
1answer
532 views

The projection matrix and proof of an unbiased estimator for sigma-squared

Hi, given this information we are meant to prove that the above estimator is unbiased. I understand the proof for the most part (below). What I do not understand is the intuitive reason why the ...
0
votes
2answers
137 views

$E(u x)=0$ and endogeneity

I am confused as to how Endogeneity arises. I understand all the examples but something basic just doesn't fit: The usual assumption is that $E(xu) \ne 0$. But, we can always find $\beta$'s that ...
1
vote
0answers
162 views

Proving Orthogonal Properties of Projection Matrix

I am working through a multi-part proof of how orthogonal projection matrices give specific results from their properties. I read through the Gauss-Markov model theory to get a start. This is only a ...
1
vote
1answer
98 views

Projection Matrix Help

I'm having trouble understanding the following in a review textbook I'm using, particularly the long equality with inverses and transposes, and the subsequent conclusion regarding the rank and trace ...
1
vote
2answers
1k views

Linear Discriminant Analysis Newbie question

Does linear discriminant analysis always project the points to a line? Most of the graphical illustrations of LDA that I see online use an example of 2 dimensional points which are projected onto a ...
6
votes
2answers
10k views

difference between linear projection and linear regression (OLS)?

In http://www.wouterdenhaan.com/numerical/slidesbayesian.pdf (approximately from page 7 to 13), ordinary least squares and linear projection are said to be different. But from my linear algebra class, ...
8
votes
1answer
696 views

Regularization and projection onto the $l_*$ ball

I'm trying to understand how regularization works in term of projections onto a $l_*$ ball, and Euclidean projection onto the simplex. I'm not sure I understand what we mean when we project the ...
5
votes
1answer
178 views

Making two vectors uncorrelated in terms of Kendall Tau correlation

Assume that we have two normalized $n\times 1$ vectors $\bf x$ and $\bf y$. In terms of Pearson correlation, these two signals are uncorrelated if ${\bf x}^T {\bf y} = 0$. Now, assume that ${\bf x}^T {...
3
votes
2answers
442 views

2D projection to maximise separability

I have a set of 500 points in 5D. Each point belongs to one of five classes, and the class labels are known. I’d like to visualise the dataset in 2D such that the classes would be separated as much ...