Questions tagged [projection]

For on-topic questions involving the mathematical concept projection, a linear transformation $P$ such that $P=P^2$. Please include also a more statistical methods tag. For purely mathematical questions about projections it is better to ask on math SE https://math.stackexchange.com/

Filter by
Sorted by
Tagged with
1
vote
0answers
12 views

Can I decompose a signal in interval T between symmetric and anti-symmetric signals?

I have computed many samples of a signal $y_n[t]; t\in \{-T/2, T/2\}$ for $n=1,\ldots,N_{\text{samples}}$. I want to model this with an even function $f(t;\vec{\theta}) = f(-t; \vec{\theta})$ where $\...
1
vote
0answers
10 views

KPCA - projection of new data doesnt work for polynomial kernel [closed]

I have written my own KPCA code and I m finding a hard time finding the projection of new data for the polynomial and sigmoid kernel. The gaussian and laplacian kernel give good results. Below you can ...
0
votes
1answer
26 views

Linear Models Hat matrix

For OLS in matrix form, we are taught that Hat matrix is $X(X^TX)^-X^T$, and is idempotent etc, i.e. when it multiplies with itself, it will self cancel and thus lead back to the same Hat matrix. I ...
0
votes
0answers
20 views

Best linear unbiased estimator that relies on a linear combination of the projection matrix?

I was studying and found this exercise in Erich Lehmann book, Theory of Point Estimation, chapter 2 about Unbiasedness. My doubt is about exercise (b). I was calculating the variance of $\delta^*$ ...
3
votes
0answers
35 views

In machine learning (especially deep learning), is it a princlple to only do linear projections to a smaller dimension size?

Linear projection (or fully connected layer) is perhaps one of the most common operations in deep learning models. When doing linear projection, we can project a vector ...
0
votes
1answer
40 views

Why are covariance matrices projected by both right and left multiply?

I've been doing a lot of Kalman filtering work recently. I've derived all the equations starting from a basic linear inverse problem, so strictly speaking I know where everything comes from. I also ...
0
votes
0answers
26 views

What can I learn about the dimensions with highest variance of a matrix $M\approx L^TR$ from looking at $L$?

I have a high-dimensional, symmetric data matrix $M\in\mathbb{R}^{d\times d}$ , which is factorized by two matrices $L, R\in \mathbb{R}^{n\times d}$ : $L^TR\approx M$, where $n$ is much smaller than $...
0
votes
0answers
9 views

Matrix Derivation Related to Nested Models (Orthogonal Projection?)

The problem is: Full Model: $$ y=I\beta_0+X_1\beta_1+X_2\beta_2+\epsilon$$ Reduced Model: $$ y=I\beta_0+X_1\beta_1+\epsilon$$ Now this is where I'm confused: It can be shown that in this case $$(C\...
1
vote
1answer
25 views

Can I use a single letter or symbol to represent “actual” in a table?

I have a chart that displays numbers for 2017, 2018, 2019 and 2020, but the 2020 numbers are based on projections - I want to display the "actual" value on the data label next to the ...
1
vote
0answers
23 views

Finding a Projection Plane in Dimensionality Reduction (e.g., Multidimensional Scaling)

I have a set of data points in high-dimensional space that I wish to map onto a lower dimension (3D or 2D). Question : How do I obtain the Projection (Hyper)Plane (e.g., its normal vector or its set ...
0
votes
0answers
29 views

Projection shortcuts in Resnets implemented as 2D convolutions

I'm currently preparing for a presentation on the well-known Resnet ("Deep Residual Learning for Image Recognition") paper and couldn't find a satisfying answer to my question yet. My ...
3
votes
1answer
136 views

Intuition behind projection matrix

I'm new to machine learning and came across projection matrix . In a random thread it was interpreted as The matrix $X(X^\text{T} X)^{-1} X^\text{T}$ is a projection matrix, as it does precisely that:...
2
votes
0answers
44 views

How is conditional expectation related to projection?

I vaguely remember seeing somewhere that the conditional expectation $E(Y|X)$ can be interpreted as projection of random variable $Y$ onto random variable $X$. My question is: Is the aforementioned ...
2
votes
1answer
47 views

Projection vs fixed effects

Suppose I have $n$ observation indexed by $i$ and that each observation is part of a group $g$. I want to compare two regressions. First regression: $$ Y_i=\beta X_i + \alpha F_{g(i)}+\varepsilon_i $$ ...
1
vote
0answers
22 views

Orthogonalization vs fixed effects in instrumental variable estimation

Suppose that I have a set of observations indexed by $i$. Each observation belongs to a group $g$. Let's define by $\hat{Z}_i=Z_i - E\left[Z_i\vert g(i)\right]$ is the residual after projecting $Z$ on ...
2
votes
1answer
73 views

Projecting a covariance matrix to a lower dimensional space

I have a point $\mathbf{x}$ in 3-dimensional space, which is measured with a degree of uncertainty. The point falls within a unit cube, and the uncertainty is assumed to follow a multivariate normal ...
0
votes
0answers
19 views

Whitening on projection matrices

The projection matrix $P = I -xx^T\in \mathbf{R}^{d \times d}$ has a zero eigenvalue and eigenvalues equal to one with multiplicity $d-1$. Is it possible to apply whitening transform on $P$ taking ...
1
vote
1answer
32 views

Project time series from previous time series examples and characteristics

Say I want to open a shop but first I want to project the likely sales in the first 5 years to see if it is a viable option. I have data pertaining to 100s of other start ups, including their success ...
1
vote
1answer
49 views

Question about Regression error and the residual maker matrix

Starting with the 'residual maker' defined by M in: $e= y-\hat{y} = Y-X(X'X)^{-1}X'Y = [I-X(X'X)^{-1}X']Y =MY$ where e is the regression residual. one common equality i see relating the regression ...
3
votes
0answers
44 views

Is $\hat \beta$ of general least squares an orthogonal projection?

$\hat \beta_{GLS} = (X'V^{-1}X)^{-1}X'V^{-1}Y \\ Y=X\hat \beta_{GLS} + \epsilon=X(X'V^{-1}X)^{-1}X'V^{-1}Y + \epsilon = \\X(X'V^{-1}X)^{-1}X'V^{-1}Y + (I-X(X'V^{-1}X)^{-1}X'V^{-1})Y$ It is clear that ...
0
votes
0answers
32 views

Neural ODEs, augmentation and subspace “projection”

The answer to the neural ODE question, the Augmented neural ODEs paper is mentioned. There, the following process happens: 2D data is augmented by padding with 1 zero 3D data is augmented once again ...
1
vote
0answers
30 views

Intercept of linear projection

I'm reading a paper and wondering whether I get things right: Say I have a variable Y and two further variables $X_1$ and $X_2$. I want to determine the part of $Y$ which is not linearly explained by $...
2
votes
1answer
250 views

Projecting new samples onto PCA space is failing

After performing PCA I would like to project any new samples to the principal component space (I would like to see how samples cluster together). I did the PCA analysis in R: ...
1
vote
1answer
169 views

Simple Linear Regression: Hat-Value $h_i$

I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
2
votes
0answers
28 views

Boundary point errors in PCA projection using sklearn

I am preparing a small example of a projection using python, numpy and sklearn to perform <...
1
vote
2answers
103 views

Difficulty in understanding the proof of the Wold decomposition theorem

The proof of the Wold Decomposition [1] of $x_t$ involves the definition of the process $$w_t = x_t - P_{\mathcal{M}_{t-1}^x} x_t,$$ where $x_t$ is a stationary zero-mean process, $\mathcal{M}_n^x = ...
3
votes
0answers
82 views

What is the geometric meaning of correlation matrix

I recently read this article explaining the geometric meaning of covariance matrix. http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/ My question is : is there an ...
1
vote
1answer
63 views

Best linear prediction as a projection in a Hilbert space $L^2$

Consider two random variables $Y$ and $X$. In the context of the best linear prediction, if we would like to predict $Y$ given $X$ known, we derive the solution solving the following minimize problem ...
1
vote
0answers
41 views

The joint distribution of Y=AX and Z=BX given a projection matrix A and residual maker matrix B, and a random vector X with known pdf?

This question follows on from a previous question I asked which was answered. It turns out my question lacked some important details, which was revealed by the answer posted on that thread. This is ...
1
vote
0answers
109 views

Least Squares Fit Covariance Matrix

How do you perform a least squares fit for $\Sigma$ in the equation $v = u^T\Sigma u$? The term $v$ is a vector of observed variances of a projected Gaussian distribution. The matrix $u$ is made of ...
3
votes
1answer
204 views

Higher moments of linear regression residuals?

I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations. Background In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
3
votes
1answer
503 views

Expected value as an orthogonal projection

I'm reading a paper in which the expected value of a random variable, $\mathbb{E}[X]$, is characterized as an orthogonal projection. This is on page 10. I've seen the geometric interpretation of ...
0
votes
0answers
33 views

Linear project when X includes a constant

In Hamilton's text I ran across the following statement: if $X$ includes a constant then the linear projection of $aY+b$ is $aP(Y\mid X)+b$ where $P(Y\mid X)$ is the linear projection of $y$ on $x$. I’...
0
votes
0answers
159 views

Linear regression with feature representation confusion - relationship of design matrix column space to the feature space?

I am trying to visualise the geometry of linear regression with feature representation. I have a regression problem with $n$ data pairs $\mathcal{D}:=\{(\mathbf{x},y)_{i}\}_{i=1}^{n}$, independent ...
3
votes
0answers
340 views

What's wrong with my solution to canonical correlation analysis (CCA) using the SVD? [duplicate]

I am working through the derivations for solving CCA in A Tutorial on Canonical Correlation Methods. Right now, I am trying to solve CCA using SVD (bottom of page 95:7). For completeness, I include ...
2
votes
1answer
1k views

t-SNE, PCA and another technique - which one? [closed]

I am comparing dimension reduction techniques and I am utilizing them for data visualizations onto a plane – projections in 2D space. The input into a projection/dimension reduction techniques is a ...
3
votes
1answer
251 views

Conditional expectation and variable decomposition

Suppose that $X$ and $Y$ have an uknown joint distribution $f_{XY}$. How can I formally demostrate that it always exists a unique decomposition of the form : $$ Y = E[Y|X] +\epsilon $$ without ...
0
votes
1answer
17 views

Model Selection - 6 month forecast given the past 25 months

As the title states, the problem at hand is asking me to predict the next 6 months values when given the past 25 months. In my opinion, the training data will be quite thin so traditional time series ...
1
vote
0answers
34 views

Can anyone help me with manipulating the projection matrices?

I am trying to prove the below equality to prove that the squared correlation of coefficient is equal to r-square(i.e. $r^2 = R^2$): $$Y'M_\iota \hat{Y} = \hat{Y}'M_\iota \hat{Y}$$ where $\hat{Y} = X\...
0
votes
1answer
770 views

Projection of a vector on unit sphere

I am currently reading the paper G.Salton et al, A Vector Space Model for Automatic Indexing, November 1975, Volume 18 It is about indexing documents. Representing them in vectors to find ...
0
votes
1answer
118 views

Vertically translated depreciation curve: Update the exponential regression coefficient

I have an exponential regression equation that I use to predict the condition of roads. The equation can be found on page 53 of the original master's thesis: Development of a Flexible Framework for ...
1
vote
1answer
96 views

Road condition: Project future condition by appending part of a deterioration curve?

I inspect a road network's condition every three years: ...
0
votes
1answer
189 views

What is wrong with my computation of projections on the first principal component?

Following the links in parenthesis (link1, link2) I wrote a bit of code in MATLAB to simulate a PCA. After running the code and plotting the results I obtain this ...
2
votes
0answers
279 views

Markov chain which is also a projection

Let $P$ be the transition matrix of a Markov chain, and assume that $P^2=P$. One immediate conclusion is that $P=P^\infty$. Furthermore, assume that there is a state $i$ such as each state $j$ (...
5
votes
1answer
671 views

How to show that demeaning the data in design matrix does not change the hat matrix

When I have a design matrix $$X = \begin{bmatrix} 1 & x_{11} & \ldots & x_{1k}\\ 1 & x_{21} & \ldots & x_{2k}\\ \vdots & \vdots & \ddots & \vdots\\ 1 & ...
3
votes
1answer
378 views

Method of alternating projections for linear fixed effects models [closed]

The standard fixed effects model (in econometrics, mostly) is $$ y = \mathbf{X\beta} + \mathbf{D\alpha} + \epsilon $$ where $\mathbf{D}$ is a set of factors, potentially with thousands of levels. ...
0
votes
1answer
271 views

Orthogonal and stationary basis of projection

Consider a set of $p$ stationary correlated time signals of identical duration $T$. I would like to de-correlate them by projection on an orthogonal basis but preserving stationarity. Without the ...
0
votes
1answer
462 views

Can I Use The Same Principal Components Of One Dataset To Graph Another Dataset In R?

I have two matrices of equal dimensions $(m \times n)$. If it helps, one matrix is the raw data, while the other is transformed. I've plotted PC1 vs PC2 for both datasets individually, but then I ...
8
votes
1answer
2k views

Intuition using linear algebra that the rank of the projection matrix equals the rank of the design matrix

Using linear algebra to explain, can someone show the intuition? I can show that the ranks are the same by using properties of rank but can't get my head around the whole projection thing more than ...
3
votes
2answers
698 views

Projected Gradient Descent

Consider the primal SVM problem: $$ \frac12||w||^2 +\frac Cm \sum_{i=1}^m \max(0,1-y_iw\cdot x_i) $$ We want to find a solution with a bounded norm, by using SGD with a projection onto the convex set:...