Questions tagged [orthogonal]
THIS TAG HAS UNCLEAR/VAGUE SCOPE. Please use more specific tags instead.
49
questions
0
votes
0answers
14 views
Hopfield network: Less orthogonality -> Store more (why?)
I've been reading the paper on Hopfield networks and testing out my implementation. One of the test were between 32x32 bit images and 32x32 random images generated from a gaussian distribution.
With ...
2
votes
1answer
129 views
Orthogonality and uncorrelated
In linear regression suppose we parition the regressors X (with k variables and n observations) into two sets X1 (with k1 variables) and X2 (with k2 variables) where k1 and k2 sum to k.
I found some ...
0
votes
0answers
9 views
varimax rotatin in PCA with non-standardized scores
I am working with a PCA. I am thinking about varimax rotating the loadings but I have some reservations about it after reading about it in the past few days. First, can I get out the varimax rotated ...
1
vote
0answers
27 views
What is the difference in the “solutions” of FastICA and Infomax?
So, I feel like I can understand the basic difference between FastICA and Infomax:
Infomax tries to minimize the mutual information between variable.
FastICA tries to maximize the non-gaussianity of ...
0
votes
0answers
45 views
In Probabilistic PCA, Where does the arbitrary orthogonal matrix(rotation matrix) come from?
I'm working on studying Probabilistic PCA based on the paper (Tipping & Bishop, 1999), I can follow the idea that the maximum likelihood function would reach the stationary point when the the ...
1
vote
1answer
50 views
Alternatives to PCA with orthogonal datasets?
http://blog.audio-tk.com/2008/02/04/dimensionality-reduction-principal-components-analysis/
"It is obvious that PCA does not respect the manifold structure. One has to use 3 dimensions to describe ...
1
vote
1answer
36 views
Contrasts in a Completely Randomized Design (Unbalanced)
Four catalysts that may affect the concentration of one component in a three-component liquid mixture are being investigated. Consider a completely randomized experiment, where $n_1 = 5$, $n_2 = 4$, $...
0
votes
0answers
12 views
Genstat warning message an71
When running an anova analysis of an experiment using balanced incomplete blocks using Genstat, I get this warning message:
Standardized residuals approximate if not an orthogonal design
Please ...
1
vote
0answers
53 views
Independence of components in PCA
Let's have spatio-temporal dataset ($Y \in \mathbb{R}^{L \times T}$). Where $L$ stands for spatial grid points and $T$ for time. Now let's say that the noise of the system follows a multivatiate ...
0
votes
2answers
24 views
How to directly know the backward selection model when independent variables are orthogonal?
According to this output, the independent variables are orthogonal.
Please tell me,
when doing the backward selection, why it can be directly known that it should be reduced to 5th order model?
1
vote
0answers
46 views
A Primer on Orthogonal GARCH Model Covariance Matrix
I am trying to replicate Table 3a: Correlation Matrix from this paper (Page 11): http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.201.7226&rep=rep1&type=pdf. (I believe there is a ...
0
votes
1answer
51 views
Why the first principal component is mostly negative while the second component is mostly positive?
I am running PCA for a fleet management data frame $X$, where each column is a city, each row is a date, there are 50 cities and 500 dates.
I run PCA on $A=X^{T}X$.
Then the first component $v_{1}$ ...
0
votes
1answer
67 views
Scatter Plot - Basics [closed]
I am stuck in understanding a basic scatter plot. I am working in two dimensions i.e. there are two variables X & Y.
So, the question is that in the scatter plot, what do the two axes mean?
...
2
votes
0answers
21 views
Variance of Random Vector in the Circular Orthogonal Ensemble
Let $x$ be a (uniformly) randomly chosen column of a random orthogonal matrix (of size $K$ x $K$) distributed according to Haar measure. What is $\mathbb{E}[x]$, $\mathbb{E}[x x^T]$, $Cov(x, x)$, and $...
1
vote
1answer
30 views
Granger's representation theorem: Johansen's version
In his book 'Likelihood based inference in cointegrated Var', in order to get the expression for the Granger's representation theorem,, Johansen claims that:
(1)
$$\beta \bot(\alpha' \bot \beta \bot ...
0
votes
0answers
17 views
Find a vector that satisfies the following: i) it has a given correlation with a second vector and ii) it is orthogonal to a set of vectors
I would like to generate a vector $\vec{u}$ of dimension $n$, so that i) it has a given correlation $r$ with a second vector $\vec{v}$ and ii) it is orthogonal to a set of $m$ vectors $A = \{\vec{w}_1,...
1
vote
1answer
135 views
Orthogonal contrasts, ANOVA, why are there only as many contrasts there are degrees of freedom?
For example, if I have the data
$$
\begin{array}{l|l|l|l|l|l|l}
A & low & & medium & & high & \\ \hline
B & standard & new & ...
0
votes
0answers
119 views
Using varimax – rotated PCA for clustering via Gaussian Mixture Model?
After extracting the Principle Components of my data, I apply Gaussian Mixture Models for clustering. I used a subset of the orthogonal basis of the Principle Components and projected my data onto ...
1
vote
0answers
158 views
Orthogonal contrasts for coefficients of regression
Suppose that we want to test the following hypothesis
$H_{0}:b_{1}+b_{3}-2b_{2}=0$
where $b_{1},b_{2},b_{3}$ are coefficient derived from a linear regression.We can see that $H_{0}$ is similar to ...
5
votes
2answers
5k views
Orthogonality of residuals in linear regression
In multiple linear regression, I came across the statement that both $e$(residual) and predicted $y$ are projections of actual y and $e$ is orthogonal to predicted $y$.
I was trying to visualize the ...
0
votes
1answer
293 views
Which rotation type for principal component regression?
I would like to perform a principal component regression (PCR), but feel a little confused about the rotation type to be used in the principal component analysis (PCA) step.
First I perform a PCA to ...
1
vote
0answers
244 views
How is multivariate Gaussian distribution is determined by its second moments alone?
The following statement is given in Unsupervised Learning chapter of the book Elements of Statistical Learning.
Since the multivariate Gaussian distribution is determined by its
second moments ...
10
votes
2answers
3k views
The linear transformation of the normal gaussian vectors
I am facing difficulty in proving the following statement. It is given in a research paper found on Google. I need help in proving this statement!
Let $X= AS$, where $A$ is orthogonal matrix and $...
1
vote
0answers
80 views
Statistics: orthogonality vs uncorrelatedness vs independence [duplicate]
In this post I would like someone to summarize and relate these 3 concepts of statistics (in the context of stats).
1) I remember that uncorrelated does NOT imply independence (e.g. the case where ...
0
votes
1answer
447 views
Does orthogonal and zero mean of two RV X,Y imply that they are uncorrelated?
I understand that two uncorrelated RV X,Y are orthogonal if at least one of both is of zero mean. But can you reverse this statement if you expand the preconditions to both RV X,Y being of zero mean?
...
1
vote
1answer
66 views
which angle and axis to chose to get a 90 degrees angle between those 2 vectors
I am suddenly puzzled by ho to know (when in 3D) with respect to which axis is the vector being rotated when the dot product between then is =0. for example: if i rotate 90degrees (pi/2 radians) along ...
0
votes
2answers
207 views
orthogonality in $2D$ vs higher dim vectors
considering that $2$ vectors such as $x_2=\begin{bmatrix}1 & 1 \end{bmatrix}$ and $y_2=\begin{bmatrix} -1 & 1 \end{bmatrix}$ are orthogonal in $2D$ (i.e. their scalar product is $0$) however ...
10
votes
3answers
3k views
Why are PCA eigenvectors orthogonal but correlated?
I've seen some great posts explaining PCA and why under this approach the eigenvectors of a (symmetric) correlation matrix are orthogonal. I also understand the ways to show that such vectors are ...
1
vote
1answer
2k views
Orthogonal initialization of weight matrix
Searching for the way to initialize the matrix weights as orthogonal (i.e. W*W^T = I and all the eigenvalues are equal either 1 or -1),(I was wrong) I found this ...
2
votes
0answers
312 views
Calculating orthogonalized impulse response functions for vector error corrrection models
Background:
I am working on orthogonal impuls response functions (OIRFs) for vector error correction models (VECMs). Its an exercise to develop understanding. I am given a bivariate VECM:
$$ \Delta ...
1
vote
0answers
621 views
Advantage of orthogonal polynomials
What is the sense or background of orthogonal polynomials (regarding using mixed models)? I would like to know why they shall or should be orthogonal. Is it to build independent sample points?
On Is ...
0
votes
0answers
2k views
How to standardize the data matrix before applying SVD for PCA?
I am trying to enhance the contrast in the images I get after scanning a surface using Thermography (Principal Component Thermography ~Rajic, which is basically an application of Principal Component ...
2
votes
1answer
150 views
Balancing out in an orthogonal design
A definition of orthogonality in the context of statistics is
An experimental design is orthogonal if the effects of any factor
balance out (sum to zero) across the effects of the other factors.
...
1
vote
1answer
301 views
Statistically orthogonal - explanation?
I did see the related question here but my question is more related to the actual explanation of the orthogonality itself.
So the following design is orthogonal (this is a latin square to be precise):...
3
votes
1answer
923 views
What is the intercept term in a mixed effects model using orthogonal polynomials to model time?
I'm using a mixed effects model (lmer) in R to model eye-tracking data using orthogonal polynomials (poly(time,3)) for time. The response variable is log(looks to target/looks to competitor). The ...
5
votes
0answers
353 views
Orthogonality in ANOVA and Regression Analysis
I read the following (Wikipedia) regarding contrast coding of categorical variables:
Unlike when used in ANOVA, where it is at the researcher’s discretion whether they choose coefficient values ...
1
vote
1answer
339 views
Distribution involving orthogonal matrix
If $Y∼N(\mu,I\sigma^2)$ and $Q$ is any orthogonal matrix of appropriate dimension, how do I find the distribution of $QY$?
1
vote
0answers
964 views
Uses of the Helmert matrix
In chapter 8 of "Matrix Algebra from a Statistician's Perspective", the author describes the construction of an orthogonal matrix, the first row of which is proportional to some row vector of non-...
10
votes
2answers
2k views
Linear regression: *Why* can you partition sums of squares?
This post refers to a bivariate linear regression model, $Y_i = \beta_0 + \beta_1x_i$ . I have always taken the partitioning of total sum of squares (SSTO) into sum of squares for error (SSE) and sum ...
12
votes
1answer
2k views
What are multivariate orthogonal polynomials as computed in R?
Orthogonal polynomials in an univariate set of points are polynomials that produce values on that points in a way that its dot product and pairwise correlation are zero. R can produce orthogonal ...
10
votes
1answer
4k views
Optimization with orthogonal constraints
I am working on computer vision, and have to optimize an objective function involves matrix $X$ and matrix $X$ is an orthogonal matrix.
$$maximize \ \ f(X)$$
$$ s.t \ \ X^T X=I$$
Where $I$ is the ...
1
vote
1answer
238 views
How to force the slope=1 in orthogonal regression
I would like to model the functional relationship between two acoustic measurements of traffic noise exposure, e.g. noise during day (x) and noise during night (y), expressed in Decibel values, using ...
1
vote
0answers
131 views
Planned contrasts - pros & cons of reducing a 2x3 design into 6-level single factor?
I have a 2x3 factorial design, and wish to explore specific hypotheses with planned (orthogonal) contrasts.
Some people recommend that in order to specify the contrasts, this design can be replaced ...
4
votes
0answers
196 views
Dropping columns from an orthogonal design matrix?
Hello: I’m working with a three factor (ANOVA) design that I wish to use in an MCMC chain to estimate the parameters for the main effects and treatment interactions. I wish to run MCMC analyses ...
1
vote
1answer
753 views
Orthogonal polynomials for regression
Is it possible to define orthogonal polynomials on the interval $[0, +\infty[$ ? Maybe using the Gram-Schmidt process from the monomial basis $(1, x, x^2, ...)$?
My problem is that I have some data ...
2
votes
1answer
126 views
PCA: Cannot Understand one part of Derivation of Principal Comonents
I'm stuck on something in the derivation of the Principal Components.
We have random vectors $X$ of dimension $p$. We want to find linear combinations of $X$, $a'X,$ where $a \in \mathbb{R}^{p}$ that ...
0
votes
0answers
32 views
4
votes
0answers
53 views
Marginal distribution of distribution on the Unitary group [closed]
I am trying to understand the behavior of distributions over the Unitary group (i.e. the set of square matrices $P$ such that $P^tP = I_d$), or in general distribution over the Stiefel manifold (set ...
4
votes
4answers
2k views
Orthogonalized regression reference?
I'm not sure what this is called but I remember seeing a colleague of mine doing a multivariate regression much like
$$Y \sim X_1 + X_2 + X_3$$
and then he said he would "orthogonalize" so ...
