Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

Filter by
Sorted by
Tagged with
1
vote
1answer
25 views

Vec operator and covariance matrix

You have a matrix containing $T$ observations of each of $K$ random variables \begin{align} U = \begin{bmatrix} u_{11} & \dots & u_{1T} \\ \vdots & \ddots & \vdots ...
1
vote
1answer
29 views

Scaling numbers with constraints

I have four numbers $X_1$, $X_2$, $X_3$ and $X_4$ that range from [-2,2]. I want to scale these numbers ($x_1=scaled(X_1)$) with the following constraints: $$x_1+x_2+x_3+x_4=1,$$ subject to $$x_1+x_2&...
2
votes
0answers
36 views

How to calculate the eigenvector?

I've been struggling to solve this math problem for two days... So I calculated the mean of all samples (0,0). Put it into the equation and got V as \begin{array} {rrr} 4 & 2 \\ 2 & 2 \end{...
0
votes
1answer
19 views

Variance of projected data

What's the expression for the variance of projection of $N$ data points onto a vector $\mathbf{v}$ which is not necessarily a unit vector? I could only find an expression when $\mathbf{v}$ is taken to ...
0
votes
0answers
28 views

First Principal Component Direction

I am trying to derive the first principal component direction from the definition and need help in finding which step is going wrong. Here's my attempt: $\mathbf{X} \in \mathbb{R}^{N \times p}$ is the ...
0
votes
0answers
7 views

How to define data within a 1D matrix as categorical

Is there any way to clearly delineate that the data contained within a matrix is categorical to the reader? Ie: is there a symbol that I can use to mark the data as qualitative and not quantitative.
0
votes
0answers
10 views

Is a vector, which is orthogonal to solution of undetermined system of linear equations, in a null space of such system? [migrated]

Let's have the following linear system $Ax = b$ and assume that $x_0$ is a solution this linear system, where $A$ is an $n \times m$ matrix, with $n < m$, $x$ is an $m$-dimensional column vector ...
0
votes
0answers
12 views

Understanding of undetermined system of linear equations [closed]

I have two questions regarding an undetermined system of linear equations. I believe they are somehow connected, thus just one question. Let's have the following linear system $Ax = b$ and assume that ...
0
votes
0answers
10 views

Signs of eigenvectors : Dual PCA

I'm trying to perform a DUAL PCA with numpy, this is are the steps I'm following: 0 - Standardize X, where X is for instance (m,n) 1 - Find eigenvalues, eigenvectors of X.T dot X Plot the projections (...
0
votes
0answers
7 views

Can you find the determinant of a matrix given its row echelon form? [migrated]

I believe I stumbled on a resource for quickly computing the determinant of a matrix by converting it to row echelon form then acting upon the diagonal elements, either they're sum or product. However,...
0
votes
0answers
13 views

Property of Covariance Matrices and Symmetric Matrices

I have a question about covariance matrices. I have read one interesting property that, all symmetric matrices are diagonalizable. Suppose we have a data matrix $X$ that has only $m$ independent ...
0
votes
0answers
99 views

@whuber’s solution to generating correlated vector to an existing one

Here https://stats.stackexchange.com/a/313138 @whuber describes a beautiful solution to generating a correlated vector to an existing one. The thing i cant figure out is $SD()$ in following expression:...
0
votes
1answer
26 views

Covariance and Correlation Matrices

I have a somewhat dumb question. When determining the correlation or covariance (doesn't matter I suppose) amongst random vectors, is the covariance computed among features or among observations? For ...
1
vote
0answers
14 views

Finding Covariance from linear algebra projection chapter

I am trying to solid background of linear regression by using linear algebra. In linear algebra, there are some chapters that related to linear regression. (orthogonality, Projection) I learned some ...
0
votes
1answer
9 views

Least squares for fitting a line with some thickness?

One can use least squares to fit a line to a set of points. However, these lines lack "thickness". (When I say "thickness", I refer to the orthogonal distance below and above the ...
1
vote
1answer
54 views

Why is $E(ee')$ a matrix?

One of the assumptions of linear regression is that $E(ee') = \sigma^2 * I $, where I is the identity matrix and sigma squared is the variance of residuals. Why is $E(ee')$ a matrix, though? $ee'$ is ...
0
votes
0answers
12 views

Term for the space describing the relationships between variables?

I'm looking for a term, which I've been referring to as the "data space", though I'm reasonably sure there's a proper term for it: Let's say I've got two variables, and when one doubles, the ...
0
votes
1answer
24 views

Can someone clearly explain the feature maps, representer theorem and kernels?

I know that we need feature maps for representing non linear function as a linear function. And linear function can be represented as a vector and vectors can be easily manipulated by computer like a ...
3
votes
0answers
69 views

Decorrelation, PCA and rotation

I am not a PCA expert, nor do I have a good knowledge of linear algebra, so bear with me and my ignorance. I am trying to understand how the authors of some papers I have been reading decorrelate two ...
0
votes
0answers
17 views

Reverse-Mode Automatic Differentiation with respect to a Matrix: How to “Matrix Multiply” 4D Tensors?

This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
0
votes
0answers
32 views

Using eigenvalues of the covariance matrix to reduce noise in my data

I have an idea to help reduce the noise in my signal but am stuck with a significant problem. I have a very noisy data set $y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$ I am fitting this ...
0
votes
0answers
35 views

Multi-class LDA (pairwise classification)

From this post: $ w=S_{W}^{-1}(μ1−μ2), $ is used to estimate $w_{0}=\frac{1}{2}(μ_{1}−μ_{2})^{T}S_{W}^{-1}(μ_{1}−μ_{2})−log(\frac{P1}{P2}),$ However, this is for a situation where there are only 2 ...
0
votes
0answers
24 views

Why PCA(Principal component analysis) can reduce the linear relationship between variables

Assuming we have centralized data,the covariance matrix of the sample is X'X. This is Because: $$ Cov(X)=\frac{1}{n-1} \begin{pmatrix} X_1'X_1 &...&X_1'X_n \\ ...&...&...\\ X_n'X_1&...
1
vote
1answer
170 views

Pseudo-inverse matrix for multivariate linear regression

In Andrew Ng's Machine Learning course lecture 4.6 on "Normal Equation", he says that in order to minimize $J(\theta) = \frac{1}{2m}\sum\limits_{i=1}^{m}({h_{\theta}}(x^{(i)}) - y^{(i)})^2$, ...
0
votes
0answers
24 views

How to find a non-linear manifold for an implicit linear function in the neighborhood of a seed point?

I am trying to understand functional analysis as an infinite-dimensional extension of linear-analysis. In this process, I came across the above query and willing to get a solution and validation of ...
0
votes
1answer
60 views

Solving for Z when Z^T Z = A using R

First time question asker here! Thanks in advance for any suggestions! So here's the issue. I start with a matrix Z which is samples by features. I create a covariance matrix $A = Z^T Z$. Then I ...
0
votes
1answer
19 views

Linear Regression Coefficients

In simple linear regression, we have that given some $(n \times 1)$ matrix of response observations $y$ and a $(n \times p)$ matrix of observations $x$, the least-squares solution for $\beta$ is $$\...
0
votes
1answer
40 views

Eigenvalues of idempotent matrix of rank $r$

In the proof for the following theorem in Linear Models in Statistics, Render & Schaalje $\textbf{Theorem 5.5}$ Let y be distributed as $N_p\left({\mathbf{\mu}, \mathbf{\Sigma}}\right)$, let $\...
2
votes
1answer
30 views

Resulting shapes when partitioning the constraint matrix $\boldsymbol{A}$ in linear programming

\begin{equation} \boldsymbol{A} = \begin{bmatrix} {1}_n^\top \otimes \mathbb{I}_m \\ \mathbb{I}_n \otimes {1}_m^\top \end{bmatrix} \in \mathbb{R}^{(m+n)\times mn} \end{equation} If the above matrix ...
1
vote
0answers
42 views

EFA Parallel Analysis

First time poster, I'm looking for some assistance with parallel analysis in R. I am doing exploratory factor analysis (EFA) on a 22 item questionnaire (n=6598) and looking for an effective way to ...
1
vote
0answers
20 views

detect incalculable variables

I have a bunch of equations in the form as follows. a+b+c+d=10 c+d+e+f=12 d+e+c=13 Where I am tying to calculate the values of each variable (many more equations ...
0
votes
0answers
44 views

Laplace transform of sum of $N$ IID random variables where $N$ is itself random

Let $\{Y_i\}$ be a sequence of IID random variables so that $Y_i \sim Exp(\lambda)$ or equivalently $Y_i \sim CPH_1(1, -\lambda)$ (continuous phase-type distribution). Let $N$ be a discrete random ...
0
votes
0answers
30 views

What are some beginner's references on algebraically structured models, and what's their connection with group actions and Fourier transform?

Since I didn't get an answer, I asked it on mathoverflow. I'm looking at a short-term position, whose project is on estimation in algebraically structured models. It also mentions in the required ...
0
votes
1answer
56 views

Determine what equation is being modelled with linear regression?

I am performing linear regression using the equation $x^{*}=(A^TA)^{-1}A^Tb$, from: $Ax=b$ → $A^TAx^{*}=A^Tb$ → $(A^TA)^{-1}A^TAx^{*}=(A^TA)^{-1}A^Tb$ → $x^{*}=(A^TA)^{-1}A^Tb$ I am trying to model an ...
1
vote
0answers
32 views

Calculating natural rate of unemployment

I have sample data on unemployment rate in a market and am looking to calculate the natural unemployment rate. The natural unemployment rate I obtained is constant over a time period, which is not a ...
1
vote
0answers
14 views

How to find upper and lower bound

Let $\Sigma \in S_{++}^n$ be a symmteric positive definte matrix with all diagonal entries one. Let $U \in R^{n \times k_1}$, $W \in R^{n \times k_2}$, $\Lambda \in R^{k_1 \times k_1}$ and $T \in R^{...
0
votes
0answers
42 views

Why does the smallest eigenvalue of the covariance matrix of a GMM equal the common variance?

I've been reading a paper "Introduction to Tensor Decompositions and Their Applications in Machine Learning". In it, the author describes an algorithm for estimating the means of the ...
0
votes
0answers
16 views

Show that ABA=0 if and only if B is of the form C-PCP for some symmetric matrix C

If B is a symmetric matrix such that ABA is well-defined, then show that ABA=0 if and only if B is of the form C-PACPA for some symmetric matrix C. PA here is the projection matrix of A.
4
votes
0answers
75 views

Physical interpretation of $U$ and $V$ matrices in SVD

I have a question about the physical interpretation of $U$ and $V$ matrices in SVD. I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
-1
votes
0answers
11 views

Can any distance be expressed by the product scalar or inner scalar? [duplicate]

I know that a distance could be expressed by inner scalars or scalar products. Is this true for all metrics (i.e., respecting the three axioms: the identity of indiscernible, symmetry, triangle ...
0
votes
0answers
84 views

Probabilities concerning n-spheres

Let $S_{n-1}(R)=\{ {\bf x}\in {\mathbb{R}}^n : ||{\bf x}||^2=R^2\}$ be the sphere in ${\mathbb{R}}^n$ with radious $R>0.$ Let the projection map $\tau_m({\bf x})=(x_1,...,x_m)$ with $m\leq n.$ Now, ...
0
votes
0answers
15 views

Visualizing data using vectors

Say there are 10 houses and we have three pieces of information for each of them, area, nbedrooms, price I can view this as 10 different vectors in space where there are 3 axes. Basically 10 arrows ...
0
votes
1answer
25 views

Correlation bounds in terms of spectral radius

My application requires that the following is true for random variables $X$, $Y$ in $\mathbb{R}^d$ with $E[X]=0, E[Y]=0$ and spectral radius $\rho$ $$\rho(E[XX]^{-1}E[XY]E[YY]^{-1}E[YX])<0.5$$ Can ...
0
votes
0answers
34 views

Using matrices to find estimates in GLMs

How do we use matrices to find estimates in Linear Models? A 4 × 400 relay race is run as follows. There are four runners, each of whom runs 400 meters. The first runner carries a baton, which she ...
1
vote
1answer
41 views

Convert the following expression w.r.t to the whole dataset instead of element of the dataset?

I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got $$w = \sum ^N _{i=1} \alpha_i x_i$$ Is it possible to convert ...
1
vote
1answer
39 views

Is the concept of Kernel in Linear Algebra and kernel for SVD the same?

Is the term kernel used in Sklearn to execute the SVD machine learning algorithm conceptually related to the notion of a kernel in linear algebra ( null space )? Or do they happen to use this same ...
4
votes
1answer
292 views

Prove that sample covariance matrix is positive definite [duplicate]

Consider the $p \times p$ sample covariance matrix: $$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c} \quad \quad \quad \mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
1
vote
0answers
12 views

how to adjust a result for a variable

Okay. I'm not sure if I'm phrasing it well but here is the problem. I have a set of data which comes out as a density measurement of proteins, these are expressed in a numerical values and correspond ...
2
votes
0answers
81 views

Derivation of skewness and kurtosis algebra of random variables

In algebra of random variables, the symbolic rule for computing variance of random variable $X\in\mathbb{R}^{n\times p}$ multiplied by a coefficent vector, $a\in\mathbb{R}^p$, is $$\text{Var}(X\cdot a)...
1
vote
0answers
160 views

What's the importance of parallel eigenvectors?

I'm studying eigenvectors. I read that if a matrix is symmetric and if the eigenvalues are real numbers, the eigenvectors will be perpendicular. However, I have no idea what it means (if anything) ...

1
2 3 4 5
11