Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [matrix]

A matrix (plural matrices) is a rectangular array of numbers, symbols, or expressions arranged in rows and columns. The individual items in a matrix are called its elements or entries.

0
votes
0answers
28 views

Derivative for row vector

Confused in the example 2.1 in the cs231n tutorial. Let y be a row vector with C components computed by taking the product of another row vector x with D components and a matrix W that is D rows by C ...
0
votes
0answers
25 views

Gaussian random fields: matrix and convolution sampling

I should be able to generate a stationary GRF from white noise in two different ways: multiplying the white noise vector by the square root of a covariance matrix with appropriate kernel; taking the ...
0
votes
0answers
9 views

Is there a relationship between the L2-norm and the Pearson chi-square test?

Suppose I have two datasets, $\mathbf{a}$ and $\mathbf{b}$, with some data weighting term (or error term), $\mathbf{e}$. I can compute the L2-norm for these two sets by the following: $$(\mathbf{a}-\...
1
vote
1answer
44 views

Why the representation in the form of $Z'X(X'X)^{-1}X'Z$ can not be simplified into $Z'Z$

Representation similar to $Z'X(X'X)^{-1}X'Z$ frequently appear to e.g. 2SLS. I think that $Z'X(X'X)^{-1}X'Z = Z'XX^{-1}X'^{-1}X'Z = Z'(XX^{-1})(X'^{-1}X')Z = Z'Z$. So why it seems that in the context ...
1
vote
0answers
11 views

How to modify the rows of a matrix, depending on the row (Making a Haar Matrix) [migrated]

I am writing a package that requires making very large Haar matrices ($2^{28} \approx 270$ million rows and columns). For example: $$ H_2 = \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} $$ $...
1
vote
0answers
30 views

VAR(p) Model Covariances and Moment Equation

I'm currently going through the book Analysis of Financial Time Series by Ruey S. Tsay and reached the following statement (The book can be found here, with VAR(1) included in the preview): Where: $...
0
votes
0answers
4 views

Number of features to initialise in Low Rank Matrix Factorisation

When aiming to randomly initialise the values in my feature vector, is there a way to determine how many features I should use?
0
votes
0answers
13 views

A question about the change in value of multiple correlation coefficient on multiplying each value with a variable quantity

Question: I know the formula of multiple correlation coefficient is; ( |R| is the correlation matrix and R11 is the cofactor of the (1,1)th element of R. But I really cannot figure out, how to deal ...
0
votes
1answer
26 views

Matrix Approach to Linear Regression Model

How do we interpret a data matrix, $X$ which does not have $1s$ as the first column? Does it refer to No Intercept Form? Could it also be interpreted as the mean deviated form? I understand that for ...
1
vote
0answers
22 views

How to simulate random correlation matrix containing off diagonal structures

I want to simulate a correlation matrix which has some off-diagonal structures and also should have some hierarchical structures. For simulating correlation matrices which contain hierarchical ...
1
vote
0answers
13 views

reorganise a data structure

I need to restructure my data, at the moment it looks like: Question number (39 total) in the header Participant ID | Response ( x 39 columns referring to Q numbers in the header) | Demographic x ...
2
votes
1answer
42 views

probability that matrix $2\times2$ of Random variables is Invertible

Let $X_1, X_2, X_3, X_4$ to be Variables, and let $A$ be the following matrix: $$ \left[\begin{matrix} X_1 & X_2\\ X_3 & X_4 \end{matrix}\right] $$ assume that $X_1, X_2, X_3, X_4$ are ...
0
votes
0answers
15 views

The miracle of the Lanczos/conjugate gradient algorithm

Lanczos/Arnoldi/Rietz/CG-like algorithm share the same core strategy... In each, a little miracle appears, most of the Gram-Schmidt inner products are zeroes ! In others words, new direction need only ...
1
vote
1answer
134 views

Measure of stability

I am working on a machine learning project when I realized I add a question. This is not programming, nor statistic, nor a probability question, but a real pure mathematical question. So I think my ...
0
votes
0answers
34 views

Decomposition of a diagonal matrix

I want to decompose a diagonal matrix $\Lambda \in R^{n \times n}$ such that $$ \Lambda \approx A\Sigma A^T $$ where $\Sigma \in R^{k \times k}$ is a diagonal matrix and $A \in R^{n \times k}$ is a ...
0
votes
0answers
14 views

How do the following 2 matrix equations equal each other

I am reading a book about machine learning at the moment. In it they make the following statement: $U^TAU^{-1} = UA^{-1}U^T $ But I can't understand why. Would someone be able to explain to me ...
0
votes
0answers
27 views

Notation for Matrix Concatenation

I have an input data matrix $X$. In fact, I have $N$ of these input matrices, so I identify each as $X_i$ using an index $i \in \{1,\ldots,N\}$. Thinking about something like cross validation, I want ...
0
votes
0answers
38 views

How to find basis functions for the given problem, by fitting cubic spline?

I am trying to understand splines and basis functions from the book, Elements of Statistical Learning. I'm trying to solve the following problem: Consider a problem with one predictor X and one ...
1
vote
0answers
31 views

CNN - Is this a Toeplitz Matrix?

I have been reading through Chapter 9 of www.deeplearningbbook.org, where convolutional networks are being described. The following image represents the output of a 2D convolution, without kernel ...
0
votes
1answer
46 views

How are biases updated when 'batch size' > 1?

This is my network represented in matrices: (a dot represents an arbitrary number) Feed-forwarding: (I omitted nesting it all in an activation function for the sake of brevity) Backpropagation The ...
0
votes
0answers
23 views

Self Attention in the Transformer learning algorithm

in this article can somebody tell me where the heck the Wq, Wk, and Wv matrices in the “Self-Attention in Detail” section come from a little more intuitively and specifically since the article doesn't ...
0
votes
0answers
11 views

Solve an equation of two unknowns to fit a distribution and mean

Apologize in advance for the drawn out question (found at the end). But I want to give a comprehensive picture of the problem. I have an equation of the following form: $T_{ij}^{sim}=A_i*O_i^{^{obs}...
2
votes
1answer
55 views

Matrices: system that is “computationally singular” versus “exactly singular” [closed]

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
1answer
82 views

Simulation of low rank and sparse matrix

I am having trouble simulating a matrix which is low rank and sparse (sparse along both rows and columns). One way to simulate a low-rank matrix is by generating a random matrix, then taking SVD and ...
0
votes
0answers
26 views

Computation of more variables than equations

Sorry if you find this question too basic. I need some inputs to solve the problem. I am decomposing a matrix $Y$ of size $N\times N$ as the summation of two matrices $Y_1$ and $Y_2$ such that the ...
1
vote
0answers
18 views

Recursively expressing matrix inverse

Let $X$ be an $D \times N$ matrix. Let $I$ be a $D \times D$ identity matrix. Also let $y$ be a $N \times 1$ column vector. Suppose we are trying to solve $(X X ^T + k I) w = Xy$ for a $D$ dimensional ...
0
votes
1answer
41 views

Calculate the implied correlation for missing cells in a correlation matrix in R

I have a correlation matrix in R. Many of the correlations are specified, but there are some that are "NA". eg, A __ B __ C A 100% NA 25% B NA 100% 50% C 25% 50% ...
0
votes
0answers
11 views

What kind of sparse matrix representation is this?

I am putting together a wrapper for a quadratic programming library. I am going through the C example here but I don't understand the indexing used for the matrices. The relevant excerpt is below, ...
0
votes
0answers
14 views

Rank Test for a Matrix

Suppose I have a matrix A corrupted with noise and I am looking for a test that tests the null hypothesis that the matrix A, rank(A)==1 v.s. rank(A)>1. I checked a little the literature and this paper ...
6
votes
1answer
693 views

Difference between Cholesky decomposition and log-cholesky Decomposition

Is there any difference between a Cholesky decomposition and a log-cholesky decomposition? If yes, what is the difference? In the paper "An R package for dynamic linear models" by Giovanni Petris ( ...
1
vote
0answers
25 views

Extracting latent vectors from autoencoder similar to SVD

I have read that there is an equivalency between a linear autoencoder and performing SVD. SVD can be used in collaborative filtering, for example, factorization of a user-movies matrix $\mathbf{M}$ ...
0
votes
0answers
10 views

Multivariate fixed point iteration: the wrt variable on both sides of update equation and not are different?

I try to understand more about the update in multivariate fixed point iteration. I saw the examples where the updates have the same variable (the wrt. variable of partial differentiation) on both ...
0
votes
0answers
35 views

Reliability Logistic Regression - train and evaluate the model [duplicate]

I have built an Logistic Regression model in R. The class that I want to predict, is very unbalanced (99 vs 1). My first finding is that this Logistic model does a better job if I train it on a ...
4
votes
2answers
108 views

Raising a variance-covariance matrix to a negative half power

I want to implement the following formula where $C$ is variance-covariance matrix of variables x, y, and z: $$C = \begin{bmatrix}cov(x,x)&cov(x,y)&cov(x,z)\\cov(y,x)&cov(y,y)&cov(y,z)\...
0
votes
1answer
127 views

PCA in psych package with more columns than rows

Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
0
votes
0answers
15 views

multivariate multiple regression, testing if a variable leads y's at the same time

I have a what I understand to be a multivariate multiple predictive regression. The y's are different variables and I am attempting to see if these are lead by w at the same time. I use the standard ...
1
vote
0answers
29 views

Obtaining hard, overlapping clusters using non-negative matrix factorization

From my understanding non-negative matrix factorization (NMF) provides a natural way to obtain soft clusters from a non-negative $n$x$m$ data matrix $X$. NMF decomposes $X$ into two non-negative ...
1
vote
0answers
49 views

Analyze similarity matrix using linear mixed model

Let's say I have a similarity matrix where every subject is compared to every other subject on some similarity measure (e.g., body movement synchrony). These subjects are divided into two groups, say ...
3
votes
1answer
65 views

Is there a (matrix) operation that can count elements in vector pairs?

For two vectors $x \in \{0, 1, 2\}^{n}$ and $y \in \{0, 1, 2\}^{n},$ I need to generate a matrix $C\in \mathcal{R}^{3\times3}$ where $C_{i,j}$ equals the number of indexes $t$ where $x[t]=i$ and $y[t]...
3
votes
0answers
93 views

Rayleigh quotient, traces and LDA optimization problem

I've been working about Linear Discriminant Analysis the last weeks, and after reading many articles, I see some aspects of this problem not very clear. The LDA optimization problem is formulated by ...
0
votes
0answers
14 views

Finding probability vectors from a matrix equation

I have $q$ $n$-dimensional vectors $\vec y_i$ and a matrix $\hat B$ of shape $n\times m$. I'm looking for $q$ $m$-dimensional vectors $\vec x_i$ such that: $\vec y_i=\hat B \vec x_i$ each vector $\...
1
vote
1answer
36 views

Explained Sums of Squares in matrix notation

I am currently reading Appendix C from Gujarati Basic Econometrics 5e. It deals with the Matrix Approach to Linear Regression Model. I am unable to decipher how the author went from equation 7.4.19 ...
1
vote
0answers
72 views

Is pairwise distance matrix useful to k-means?

The k-means implemented in scikit-learn precomputes distances but I don't how these distances are used. In its standard version, k-means is known to compute only the distances between the points and ...
0
votes
0answers
17 views

math notation for batch matrix multiply

I use batch matrix multiply (torch.bmm) very often in my models and I want to write them down in math notation for documentation purposes. Does bmm have a standard ...
2
votes
1answer
44 views

Positive definiteness of Grammian with respect to Gaussian process' covariance function

A Gaussian process indexed by $T \subseteq \mathbb{R}^d$ is a collection of random variables $\{ X_t : t \in T\}$, for which each finite subset is distributed as a multivariate Gaussian. Let $G$ be a ...
2
votes
0answers
133 views

Obtain within-group Gram matrix out of distance matrix

Gram matrix Let $\bf X$ be a n x p dataset with columns (variables) centered. Then p x p $\bf X'X$ is the total scatter matrix ...
0
votes
0answers
16 views

calculate probabilities with Markov / Matrix calculations

I am trying to get a little bit back into matrix calculations after I have sucessfully ignored it for about 25 years. Certainly you're laughing at me for this question. Here it comes: I thought I'd ...
1
vote
0answers
17 views

Predicting categories based on a similarity matrix

I am looking for some help in organising some analyses. I will describe what I am trying to do with a fictional example and then talk about some of the things I've thought about already. Example I ...
3
votes
0answers
48 views

Are null space of matrix and kernel function same?

I have recently started learning about machine learning and have come across kernels and null spaces. I understand that null space is the set of all vectors that satisfy the equation A.v = 0 (Where A ...
0
votes
1answer
34 views

Principle Component Analysis: Why Are Many Solutions Possible?

The following example is from chapter 2.12 of Deep Learning, by Goodfellow, Bengio, and Courville: I don't understand what is meant by the very last part: "Many solutions are possible, because we can ...