Questions tagged [orthogonal]

THIS TAG HAS UNCLEAR/VAGUE SCOPE. Please use more specific tags instead.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

What is the appropriate design for a discrete choice experiment?

I need to develop a discrete choice experiment in R, which is an unlabeled experiment. However, after delving into the theory behind discrete choice experiments, it remains unclear whether there is a ...
Amy95's user avatar
  • 1
0 votes
0 answers
38 views

Prove two orthogonal contrasts are uncorrelated

I am reading some ANOVA notes and have a problem with proving two orthogonal contrasts are uncorrelated. Here is what I tried: Assume that $\bar X_i, i=1,\dots,n$ are uncorrelated. Suppose we have two ...
Matata's user avatar
  • 641
1 vote
0 answers
23 views

Partially orthogonal error term in Regression

Suppose I have a model: $Y = X_1 \beta_{1} + X_2\beta_2 + C$, where $C$ is independent to $X_1$ but not $X_2$. If we naively perform linear regression, say by concatenating $X= (X_1 X_2)$ and ...
Tommy Tang's user avatar
3 votes
1 answer
98 views

Two-way ANOVA with interaction term / orthogonal design

The two-way ANOVA model with interaction for some continuous variable $y$ can be expressed as $$y = X\mu + \varepsilon,$$ where $X$ is the design matrix (the first column of $X$ contains the constant, ...
Syd Amerikaner's user avatar
0 votes
1 answer
100 views

interpreting polynomial regression output when the regressors are orthogonal (vs. raw)

I want to show an inverted U-shape relationship between two variables: "minutes spent in a room A" and "trustworthiness in others". The hypothesis is that those who have low and ...
nina_stats's user avatar
1 vote
0 answers
58 views

How can I add the constraint that regression parameters are orthogonal to a design matrix in an MCMC algorithm?

Suppose I have a linear regression like $$y_i=X\boldsymbol{\beta} + \alpha_i + \epsilon_i,$$ where $\epsilon_i\sim N(0,\sigma^2)$ are i.i.d. Further, suppose that I want to add the constraint that $\...
Ron Snow's user avatar
  • 2,069
0 votes
0 answers
27 views

In what sense are "all regression predictors in a balanced factorial ANOVA orthogonal"?

This question Why are all regression predictors in a balanced factorial ANOVA orthogonal? asks why all predictors in a balanced ANOVA setting are orthogonal. My question is in what sense are the ...
Syd Amerikaner's user avatar
2 votes
1 answer
128 views

Is the coefficient of an orthogonal independent variable affected by the near-multicollinearity of two other independent variables?

Suppose there is a regression with three variables $X_1, X_2, X_3$. Say $X_1$ and $X_2$ have near-multicollinearity, but $X_3$ is (nearly) orthogonal to both. Will $\beta_3$ experience the same ...
Rudolf Schuster's user avatar
3 votes
1 answer
188 views

What estimation methods other than ordinary least squares guarantee the orthogonality of predictions and residuals?

From my question here, it is evident that estimation approaches to linear regression other than ordinary least squares can result in the predictions and residuals lacking orthogonality, despite the ...
Dave's user avatar
  • 60.9k
1 vote
0 answers
66 views

Quadratic regression with orthogonal polynomials vs. raw polynomials with QR decomposition

I'm using rstanarm to estimate random slopes for second-order polynomial coefficients. My model has the basic form: ...
sjs_999's user avatar
  • 11
1 vote
1 answer
188 views

Prove OLS consistency

Consider the linear model $$ Y={\underbrace{X_i}_{K\times 1 }}^\top\beta+U_i $$ and assume (0) There is no intercept in the model (1) $E(X_i U_i)=0_K$ [orthogonality] (2) $E(X_i X_i^\top)$ has rank $K$...
Star's user avatar
  • 816
1 vote
0 answers
46 views

The p value of my OPLS-DA model was -nan(ind), I don't know, what's mean, and the residual MS was infinite, I need help here?

I obtained for the first time strange cross-validation results of my OPLS-DA model, the CV ANOVA was -nan(ind), SD residual was inf, MS residual was inf also, moreover, F test equal to zero. The ...
user357173's user avatar
0 votes
0 answers
25 views

Cross-covariance in context of Andrews plot

As shown in this Cross-Validated post Close curves on an Andrews plot I don't understand how, in the accepted answer, the cross-covariance can be defined as, $$\int_{-\pi}^{\pi}f_xf_ydt$$ Considering ...
WorseThanEinstein's user avatar
4 votes
1 answer
444 views

some thought about independence and orthogonal, please comment on this if it's wrong

It seems that linearly independent is totally different from independent of random variable concept. Non-zero vectors Orthogonality must imply linearly independence. In Statistics, the relation of ...
LJNG's user avatar
  • 301
1 vote
0 answers
344 views

decision boundaries of random forests

I was told that decision boundaries of RandomForests can be non-orthogonal. See Figure 7-5 in Geron's book Hands-On Machine Learning with Scikit-Learn & TensorFlow p.g. 187 edition 1. This is not ...
NNN's user avatar
  • 155
0 votes
0 answers
27 views

Proving non-correlation with very disperse distributions

I'm fairly new to statistics and came up with a problem. I have a sample with a variation coefficient CV = 0.517 for variable x, and I want to prove this variable is not correlated with a second ...
lafinur's user avatar
  • 235
0 votes
0 answers
46 views

One-dimensional subspace clustering

Consider the inner product space $(\mathbb{R}^n, \langle\cdot,\cdot\rangle)$ and suppose that there are one-dimensional orthogonal subspaces $\{V_i\}_{i=1}^n$ such that $\mathbb{R^n} = \oplus_{i=1}^n ...
Steve's user avatar
  • 35
0 votes
0 answers
77 views

Are eigenvectors of PCA guaranteed to be orthonormal?

Are eigenvectors (principal components) of PCA orthonormal or only orthogonal ? Or only some of them are orthonormal or they are orthonormal if data were normalized before doing PCA ?
Daniel Wiczew's user avatar
2 votes
1 answer
54 views

Check orthogonality of batched vectors, of non square matrix

I have a batch of vectors $X$ that have row vectors of size $n$, and batch size of $k$, so $$\begin{bmatrix} v_{11} & ... & v_{1n} \\ v_{21} & ... & v_{2n} \\ &\;\;\vdots \notag \\ ...
Daniel Wiczew's user avatar
2 votes
2 answers
375 views

Prove two orthogonal contrasts are statistically independent

Linear combination $C=\sum_{i=1}^{n} a_i \bar{X}_i$ is called a (estimated) contrast if $\sum_{i=1}^{n} a_i=0$. Two contrasts are called orthogonal if $\sum_{i=1}^{n} a_i b_i = 0$; simplest example ...
Esmailian's user avatar
  • 606
0 votes
1 answer
264 views

Using Orthogonal Main Effects Plan to select profiles for conjoint analysis

I am trying to create a code in Python to select orthogonal profiles given some attributes and levels. For eg: ...
DS_1's user avatar
  • 1
3 votes
0 answers
105 views

Estimating Moving Average Impact Matrix after running VECM

How to estimate moving average impact matrix after running VECM model in R? It is given as $ \hat{\beta_c}( \hat{\alpha_c}$ $\hat{\Gamma}$ $\hat{\beta_c}$)$^{-1}$ $\hat{\alpha_c}'$, where $\alpha$, $\...
Raghav Goyal's user avatar
2 votes
1 answer
968 views

Orthogonality and uncorrelated

In linear regression suppose we parition the regressors X (with k variables and n observations) into two sets X1 (with k1 variables) and X2 (with k2 variables) where k1 and k2 sum to k. I found some ...
user584534's user avatar
1 vote
1 answer
155 views

Alternatives to PCA with orthogonal datasets?

http://blog.audio-tk.com/2008/02/04/dimensionality-reduction-principal-components-analysis/ "It is obvious that PCA does not respect the manifold structure. One has to use 3 dimensions to describe ...
user avatar
2 votes
1 answer
408 views

Contrasts in a Completely Randomized Design (Unbalanced)

Four catalysts that may affect the concentration of one component in a three-component liquid mixture are being investigated. Consider a completely randomized experiment, where $n_1 = 5$, $n_2 = 4$, $...
Chesso's user avatar
  • 423
1 vote
0 answers
113 views

Independence of components in PCA

Let's have spatio-temporal dataset ($Y \in \mathbb{R}^{L \times T}$). Where $L$ stands for spatial grid points and $T$ for time. Now let's say that the noise of the system follows a multivatiate ...
Xbel's user avatar
  • 123
0 votes
2 answers
121 views

How to directly know the backward selection model when independent variables are orthogonal?

According to this output, the independent variables are orthogonal. Please tell me, when doing the backward selection, why it can be directly known that it should be reduced to 5th order model?
aab's user avatar
  • 1
1 vote
0 answers
180 views

A Primer on Orthogonal GARCH Model Covariance Matrix

I am trying to replicate Table 3a: Correlation Matrix from this paper (Page 11): http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.201.7226&rep=rep1&type=pdf. (I believe there is a ...
OGARCH's user avatar
  • 65
0 votes
1 answer
601 views

Why the first principal component is mostly negative while the second component is mostly positive?

I am running PCA for a fleet management data frame $X$, where each column is a city, each row is a date, there are 50 cities and 500 dates. I run PCA on $A=X^{T}X$. Then the first component $v_{1}$ ...
eight3's user avatar
  • 235
0 votes
1 answer
191 views

Scatter Plot - Basics [closed]

I am stuck in understanding a basic scatter plot. I am working in two dimensions i.e. there are two variables X & Y. So, the question is that in the scatter plot, what do the two axes mean? ...
Sameer Saurabh's user avatar
2 votes
0 answers
94 views

Variance of Random Vector in the Circular Orthogonal Ensemble

Let $x$ be a (uniformly) randomly chosen column of a random orthogonal matrix (of size $K$ x $K$) distributed according to Haar measure. What is $\mathbb{E}[x]$, $\mathbb{E}[x x^T]$, $Cov(x, x)$, and $...
winter_stats's user avatar
1 vote
1 answer
103 views

Granger's representation theorem: Johansen's version

In his book 'Likelihood based inference in cointegrated Var', in order to get the expression for the Granger's representation theorem,, Johansen claims that: (1) $$\beta \bot(\alpha' \bot \beta \bot ...
Alchemy's user avatar
  • 131
0 votes
0 answers
22 views

Find a vector that satisfies the following: i) it has a given correlation with a second vector and ii) it is orthogonal to a set of vectors

I would like to generate a vector $\vec{u}$ of dimension $n$, so that i) it has a given correlation $r$ with a second vector $\vec{v}$ and ii) it is orthogonal to a set of $m$ vectors $A = \{\vec{w}_1,...
user242367's user avatar
3 votes
1 answer
414 views

Orthogonal contrasts, ANOVA, why are there only as many contrasts there are degrees of freedom?

For example, if I have the data $$ \begin{array}{l|l|l|l|l|l|l} A & low & & medium & & high & \\ \hline B & standard & new & ...
baxx's user avatar
  • 938
0 votes
0 answers
246 views

Using varimax – rotated PCA for clustering via Gaussian Mixture Model?

After extracting the Principle Components of my data, I apply Gaussian Mixture Models for clustering. I used a subset of the orthogonal basis of the Principle Components and projected my data onto ...
Mofongo's user avatar
1 vote
0 answers
244 views

Orthogonal contrasts for coefficients of regression

Suppose that we want to test the following hypothesis $H_{0}:b_{1}+b_{3}-2b_{2}=0$ where $b_{1},b_{2},b_{3}$ are coefficient derived from a linear regression.We can see that $H_{0}$ is similar to ...
G1I2A's user avatar
  • 85
5 votes
2 answers
15k views

Orthogonality of residuals in linear regression

In multiple linear regression, I came across the statement that both $e$(residual) and predicted $y$ are projections of actual y and $e$ is orthogonal to predicted $y$. I was trying to visualize the ...
Spartan07's user avatar
0 votes
1 answer
588 views

Which rotation type for principal component regression?

I would like to perform a principal component regression (PCR), but feel a little confused about the rotation type to be used in the principal component analysis (PCA) step. First I perform a PCA to ...
Onur Tekel's user avatar
1 vote
0 answers
455 views

How is multivariate Gaussian distribution is determined by its second moments alone?

The following statement is given in Unsupervised Learning chapter of the book Elements of Statistical Learning. Since the multivariate Gaussian distribution is determined by its second moments ...
ironman's user avatar
  • 692
11 votes
2 answers
5k views

The linear transformation of the normal gaussian vectors

I am facing difficulty in proving the following statement. It is given in a research paper found on Google. I need help in proving this statement! Let $X= AS$, where $A$ is orthogonal matrix and $...
ironman's user avatar
  • 692
1 vote
0 answers
106 views

Statistics: orthogonality vs uncorrelatedness vs independence [duplicate]

In this post I would like someone to summarize and relate these 3 concepts of statistics (in the context of stats). 1) I remember that uncorrelated does NOT imply independence (e.g. the case where ...
SheppLogan's user avatar
0 votes
1 answer
866 views

Does orthogonal and zero mean of two RV X,Y imply that they are uncorrelated?

I understand that two uncorrelated RV X,Y are orthogonal if at least one of both is of zero mean. But can you reverse this statement if you expand the preconditions to both RV X,Y being of zero mean? ...
Alon's user avatar
  • 235
1 vote
1 answer
532 views

which angle and axis to chose to get a 90 degrees angle between those 2 vectors

I am suddenly puzzled by ho to know (when in 3D) with respect to which axis is the vector being rotated when the dot product between then is =0. for example: if i rotate 90degrees (pi/2 radians) along ...
SheppLogan's user avatar
0 votes
2 answers
258 views

orthogonality in $2D$ vs higher dim vectors

considering that $2$ vectors such as $x_2=\begin{bmatrix}1 & 1 \end{bmatrix}$ and $y_2=\begin{bmatrix} -1 & 1 \end{bmatrix}$ are orthogonal in $2D$ (i.e. their scalar product is $0$) however ...
SheppLogan's user avatar
11 votes
3 answers
6k views

Why are PCA eigenvectors orthogonal but correlated?

I've seen some great posts explaining PCA and why under this approach the eigenvectors of a (symmetric) correlation matrix are orthogonal. I also understand the ways to show that such vectors are ...
huckleberry's user avatar
1 vote
1 answer
3k views

Orthogonal initialization of weight matrix

Searching for the way to initialize the matrix weights as orthogonal (i.e. W*W^T = I and all the eigenvalues are equal either 1 or -1),(I was wrong) I found this ...
Slowpoke's user avatar
  • 163
2 votes
0 answers
364 views

Calculating orthogonalized impulse response functions for vector error corrrection models

Background: I am working on orthogonal impuls response functions (OIRFs) for vector error correction models (VECMs). Its an exercise to develop understanding. I am given a bivariate VECM: $$ \Delta ...
Nic Doe's user avatar
  • 21
2 votes
0 answers
970 views

Advantage of orthogonal polynomials

What is the sense or background of orthogonal polynomials (regarding using mixed models)? I would like to know why they shall or should be orthogonal. Is it to build independent sample points? On Is ...
Ben's user avatar
  • 3,353
0 votes
0 answers
3k views

How to standardize the data matrix before applying SVD for PCA?

I am trying to enhance the contrast in the images I get after scanning a surface using Thermography (Principal Component Thermography ~Rajic, which is basically an application of Principal Component ...
Rumi-Thermo's user avatar
2 votes
1 answer
244 views

Balancing out in an orthogonal design

A definition of orthogonality in the context of statistics is An experimental design is orthogonal if the effects of any factor balance out (sum to zero) across the effects of the other factors. ...
John V's user avatar
  • 95