Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

203 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
1answer
35 views

Rotation/Translation beween 2 Maps?

I have 2 maps, a location value is represented $(x,y)$ on Map1 and $(X,Y)$ on Map2. I want to know how much rotation and translation is needed to match Map1 and Map2. i.e. I need to calculate $\...
1
vote
0answers
97 views

Is there a solution for Canonical Correlation Analysis on large sparse matrices?

I'm trying to run CCA over two views which are sparse matrices. The two views are very high dimensional (e.g. 300k, 400k) with 1m samples. CCA needs the input views to be zero mean but I won't be ...
1
vote
0answers
148 views

Calculating Covariance Matrix from pooled sum of squares and products about the mean?

I am given the following (labeled as the "pooled sum of squares and products about the mean" $$ \begin{matrix} x_1 & x_1 & x_2 & x_3 & x_4\\ x_1 & 19.1434 ...
1
vote
0answers
58 views

Prove that $f^{-1}$SSE in SSE = SST - SSTR is the pooled unbiased estimate of the common covariance $\Sigma$ where f = n-J

At the onset I know I need to prove that $E(f^{-1}) = \Sigma$ I just don't know how to go about doing this. I know $\frac{SSE}{\sigma^2}$ follows a $\chi^2$ distribution with, I believe, n-J ...
1
vote
0answers
305 views

How does evenly spaced X simplify the linear regression?

What effect does it have on the H-matrix or the regression line?
1
vote
0answers
148 views

PRML: Why the elements of y(x) sums to 1? (Chapter 4.1.3)

I could not understand the reason why PRML (Pattern Recognition and Machine Learning by Christopher Michael Bishop) says: An interesting property of least-squares solutions with multiple target ...
1
vote
0answers
31 views

Random samples for particular cov(X,X), and cov(X,X.^2)

I have an empirical sample of multivariate data and I want to generate surrogate data X that matches certain characterizations of my data: cov(X,X) = A, and cov(X,X.*X) = B where X.*X is the Hadamard ...
1
vote
0answers
253 views

Are the Magnitudes of PCA Weights Reflective of their Importance?

I have a data matrix with N entries and n features. I wanted to find which features are the most important in explaining the data. So, I started with PCA, but PCA components are linear combination ...
1
vote
0answers
2k views

Proving that kernels are closed under addition and scalar multiplication

We have to show that given k_1 and k_2 are both kernels, the sum k_1 + 2*k_2 is a kernel as well. I am attempting to show this by using Mercer's theorem: $k(x,y) = <a(x),a(y)>$. Showing that $...
1
vote
0answers
41 views

What should be rank of matrix with compositional data variables?

I'm doing compositional data analysis on 'Baysite' dataset under 'compositions' package in r. Aim is to find nature of the relationship of its permeability to the mix of its four ingredients: A: ...
1
vote
1answer
52 views

Matrix operations in business

I have a chemical engineering background but I have decided to go into the corporate world instead of industry. I now work in payment streams and have been doing data science and optimizations. What I ...
1
vote
0answers
566 views

How to compare diagonal elements of precision matrix (the inverted correlation matrix)?

Let $$C=\begin{pmatrix}C_{11} & C_{12}\\ C_{21} & C_{22}\end{pmatrix}$$ be a $p\times p$ correlation matrix with positive entries, where $C_{11}$ is a $q\times q$ matrix. Define $D=C^{-1}=(d_{...
1
vote
0answers
77 views

Solve probability equation for one variable

Given: p = $a^l$ * ($\frac{c}{a^l + b^l}$ + $\frac{b}{a^l + c^l}$) Where a, b, c, p are known and are probabilities. Solve for l. (1 equation and 1 unknown) Does a closed form solution to this ...
1
vote
0answers
1k views

Trace of a Covariance Matrix (OLS vs. Ridge)

I am trying to assert that that the trace of the covariance of OLS is more than that of ridge regression to argue why the variance of OLS is more than the variance of ridge: $$ \frac{1}{n} (X\beta+\...
1
vote
0answers
91 views

is covariance matrix and eigen vector question

is every covariance matrix 's eigenvector is orthogonal? why symmetric matrix's eigenvector is orthogonal? can you show some example? and the reason? $$\Sigma {\bf U_i} = \lambda_i U_i$$ additional ...
1
vote
0answers
27 views

LITTERATURE SEARCH: Linear model on Vector space

I need a reference to an abstract treatment of the linear model on a space V where X is a normal variable (on V) with mean in a subspace of V, where all key results are given in this abstract ...
1
vote
0answers
117 views

Mean subtraction and addition purpose

I read Efficient and Accurate Approximations of Nonlinear Convolutional Networks paper. I have this point I don't understand in section 2.1. But may be, my question will be more general. it is known ...
1
vote
0answers
349 views

Correlated Normal Random Variables with Circulant Matrix

Let $\Sigma \in \mathbb{R}^{n \times n}$ be a circulant, symmetric, positiv definite matrix. To generate correlated random variables $Y$ with the covariance matrix $\Sigma$, one has: $$ Y = C X $$ ...
1
vote
0answers
595 views

Create a precision matrix and get desired covariance matrix

I am trying to build a Gaussian graphical model for a simulation. I want to achieve the following: Simulate an undirected graph structure (Markov network). Take nodes as variables and edges as ...
1
vote
1answer
50 views

Matrix constrains in optimization problem

I have a optimization problem with the conventional form: $\arg \min f(X)$ $s.t.:$ "some specific elements of X are zero" Ex: $X(1,2)=X(3,4)=...=0$ "some specific elements of X are one" Ex: $X(4,...
1
vote
0answers
33 views

How to optimize a generalized trace problem in dimensionality reduction

I know how to solve this problem in dimensionality reduction. $argmax_{X}$ $Trace[XLX^T]$ with $XX^T=I$ ,where $L$ is symmetric, $X$ is unitary, and $I$ is identity matrix. But I'd like to know how ...
1
vote
0answers
24 views

Distribution of the maximum vector (over i.i.d. set) and its dot product with the eigenvectors spanning the rest.

Let $z_1,\dots,z_n$ be i.i.d. draws from $N(0,\Sigma)$, where $\Sigma$ is a $p\times p$ matrix. Assume that $p>n$. Suppose (up to re-labeling) that $z_n=\max_i \|z_i\|_2$. Consider the eigenvectors ...
1
vote
0answers
504 views

Interpretation of (diagonalized) inverse covariance matrix

There are several threads here about covariance matrix and inverse covariance matrix interpretation (here, here or here). However, I was wondering how to interpret the inverse covariance matrix (or ...
1
vote
0answers
243 views

how to solve weighted low rank approximation problem with diagonal weight matrix

Without weights the low rank approximation problem can be solved in terms of svd of the original matrix.But is there any way to solve the problem following problem $$\min_C{\sum_{i=1}^n \sum_{j=1}^d (...
1
vote
0answers
64 views

Linear form arising in expected value of empirical variance of non-independent variables

Consider a normal vector $Y \sim \mathcal N(\mu, V)$ with $\mu \in\mathbb R^n$ and $V\in\mathbb R^{n\times n}$. I am interested in the expected value of $$ {1\over n-1} \left( Y'Y - {1\over n} (\...
1
vote
0answers
290 views

Recommendation system and baseline predictors

I'm participating in programming contest, where I have a data, and where the first number is a user, second number is a movie, and the third is a number in then-points rating. ...
1
vote
0answers
283 views

MECE statistics

Consider a sample of a real quantity $X$. Say that we divide this sample into two mutually exclusive and collectively exhaustive (MECE) groups. Say we know the mean and median of these two groups, $\...
1
vote
1answer
85 views

Alternative form for weighted least squares

Coefficients $\beta$ can be estimated from $y$ by weighted least squares with: $ \hat\beta = (X^T\Sigma^{-1}X)^{-1} X^T \Sigma^{-1} y $ where $\Sigma$ is the covariance matrix of the noise. Let $N$ ...
1
vote
0answers
962 views

Propogation of error in a matrix inversion

I'm trying to find the deterministic error bounds for some parameters calculated through distance geometry. The equation can be simplified to the following form: $ \left[\begin{matrix} x_1 \\ x_2 \\ ...
1
vote
0answers
109 views

Representing a multivariate normal with a scaled variance

I would like to model an observation to have a multivariate normal distribution but am having some trouble figuring out the linear algebra. So, let us start with a distribution that I know how to ...
1
vote
0answers
91 views

Laplacian Eigenmaps Derivation Question

I had read a few papers on Laplacian Eigenmaps and have been a bit confused on 1 step in the standard derivation. First I just want to deal with the 1-D case. We are given that we want to find the ...
1
vote
0answers
173 views

Fitting a linear model with few extreme values

I want to estimate a parameter (let's call it x) by some other paramaters via some linear model. Usually I take lm() in R for such purposes. However, in my situation the parameter x is mostly very ...
1
vote
0answers
112 views

Can a very bad Coefficient of determination ($R^{2}$) not be indicative of model performance?

Thanks in advance for the advice. I am trying to build a generalized linear model that has many predictors. The $R^{2}$ value of the model is quite low (.21), but when I use the model to predict ...
1
vote
0answers
81 views

Computing directly comparable wavelet features on variable-length training examples

Consider a classification problem in which the raw data are snippets of a larger 1-D time series signal. In my application, the signal is the response of a motion sensor as a function of time (the raw ...
1
vote
0answers
35 views

Regression factors and covariance matrix

I am trying to follow someone else's notes. They have two matrices. One is called comfact (company factors). This is a 580 x 5 matrix. The 580 rows represent 580 ...
1
vote
1answer
951 views

Fast Mahalanobis distance computation with singular covariance matrix

I'm trying to calculate the following Mahlanobis distance. $x^{T}$pinv($C$)$x$ Since covariance matrix, $C$ is singular, pinv($C$) means pseudo-inverse of C. However, my $C$ is very large, so it's ...
0
votes
0answers
6 views

How to compute a spatial covariance matrix within a cluster?

I'm trying to implement this paper, which is a method to obtain superpixels through a SLIC-based approach, and at some point I need to calculate a spatial covariance matrix for each cluster - or ...
0
votes
0answers
9 views

Word vectors projection and residual features

I am working on a project that uses word vectors from word2vec. I can come up with semantic feature vectors by subtracting pairs of word vectors, for example I can say a gender vector is formed by ...
0
votes
0answers
18 views

LDA (linear discriminant analysis) for images: strange eigenvalues

I have the following dataset: I represent each image as a $(67 \times 67, 1)$ vector and add it to the dataframe. df.head() My goal is to determine the ...
0
votes
0answers
45 views

Finding inverse matrix $(X'X)^{-1}$ with $X$ as design matrix

I'm relatively new to all this and I am trying to figure out how I can derive the matrix $(X'X)^{-1}$ when I have given $x_1, x_2, x_3$ and $y$. $X$ is the design matrix in that case but not sure how ...
0
votes
0answers
31 views

First Principal Component Direction

I am trying to derive the first principal component direction from the definition and need help in finding which step is going wrong. Here's my attempt: $\mathbf{X} \in \mathbb{R}^{N \times p}$ is the ...
0
votes
0answers
8 views

How to define data within a 1D matrix as categorical

Is there any way to clearly delineate that the data contained within a matrix is categorical to the reader? Ie: is there a symbol that I can use to mark the data as qualitative and not quantitative.
0
votes
0answers
13 views

Signs of eigenvectors : Dual PCA

I'm trying to perform a DUAL PCA with numpy, this is are the steps I'm following: 0 - Standardize X, where X is for instance (m,n) 1 - Find eigenvalues, eigenvectors of X.T dot X Plot the projections (...
0
votes
0answers
13 views

Property of Covariance Matrices and Symmetric Matrices

I have a question about covariance matrices. I have read one interesting property that, all symmetric matrices are diagonalizable. Suppose we have a data matrix $X$ that has only $m$ independent ...
0
votes
0answers
100 views

@whuber’s solution to generating correlated vector to an existing one

Here https://stats.stackexchange.com/a/313138 @whuber describes a beautiful solution to generating a correlated vector to an existing one. The thing i cant figure out is $SD()$ in following expression:...
0
votes
1answer
26 views

Covariance and Correlation Matrices

I have a somewhat dumb question. When determining the correlation or covariance (doesn't matter I suppose) amongst random vectors, is the covariance computed among features or among observations? For ...
0
votes
0answers
12 views

Term for the space describing the relationships between variables?

I'm looking for a term, which I've been referring to as the "data space", though I'm reasonably sure there's a proper term for it: Let's say I've got two variables, and when one doubles, the ...
0
votes
1answer
27 views

Can someone clearly explain the feature maps, representer theorem and kernels?

I know that we need feature maps for representing non linear function as a linear function. And linear function can be represented as a vector and vectors can be easily manipulated by computer like a ...
0
votes
0answers
27 views

Reverse-Mode Automatic Differentiation with respect to a Matrix: How to “Matrix Multiply” 4D Tensors?

This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
0
votes
0answers
49 views

Using eigenvalues of the covariance matrix to reduce noise in my data

I have an idea to help reduce the noise in my signal but am stuck with a significant problem. I have a very noisy data set $y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$ I am fitting this ...