Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [matrix]

A matrix (plural matrices) is a rectangular array of numbers, symbols, or expressions arranged in rows and columns. The individual items in a matrix are called its elements or entries.

0
votes
0answers
14 views

Notation question for multiplying matrix rows by vector

I have a matrix A and row vector b such that; $A=\begin{bmatrix} a_{11} & a_{12} & a_{13} & \dots & a_{1n} \\ a_{21} & a_{22} & a_{23} & \dots & a_{2n} \\ ...
0
votes
0answers
8 views

How to draw a random matrix from Wishart distribution in Python? [migrated]

I am interested in drawing a random matrix from Wishart, and I wonder if there is anything in numpy/scipy for it. Google returns this for R, but haven't found anything for python.
1
vote
0answers
57 views

What is the most intuitive proof that Gaussian kernel is positive definite?

I have general form of Gaussian kernel $K(x,x')=\exp(-\|x-x'\|^{2})$ (just not considering $\sigma$). I tried to prove its positive definiteness via Gram matrix properties, but couldn't. Is there any ...
3
votes
2answers
171 views

adding a small constant to the diagonals of a matrix to stabilize

I have a large correlation matrix (110x110) with some small eigenvalues (about 20 < 0.1). It has been suggested that adding a constant (about 0.1) to the diagonals will help to stabilize the matrix....
0
votes
0answers
15 views

Build customer ratings from subscription datetime data for a recommendation system

I want to build a recommendation system with only some customer's subscription and unsubscription date. I have a database that looks like: ...
0
votes
0answers
9 views

Measure the change of feature set over time

I have two matrices mat1 and mat2, the same number of columns but the different number of rows. You can imagine that ...
1
vote
0answers
24 views

Simplifying a covariance expression

Let $I=\begin{pmatrix}I_1\\\vdots\\ I_n \end{pmatrix}$ be a random vector, and $\Omega$ and $\Omega_I$ two random variables. I am trying to simplify the following equation (which worth $\frac{\rho_{I\...
4
votes
2answers
73 views

Steps of Matrix Multiplication

It may seem kind of silly, but can anyone please show me the intermediate steps implied by the second equality in this derivation? $$e^\prime e = \left(y - Xb\right)^\prime\left(y - Xb\right) = y^\...
3
votes
1answer
62 views

The form of the Log-Likelihood Function in Mixed Linear Models

Let us assume the following mixed effects model: $y = X\beta+Zu+e$ where $y$ is a vector of n observable random variables, $\beta$ is a vector of $p$ fixed effects, $X$ and $Z$ are known matrices, ...
0
votes
0answers
13 views

How to apply Lagrange Multiplier to a matrix?

I am learning about the Lagrange Multiplier and I see how to apply it to a set of equations but I don't know how to apply it to matrices. Suppose I have: ...
1
vote
1answer
26 views

Given $m$ $n$-dimensional vectors, how to create a vector perpendicular to all of them?

Given $m$ vectors, $x_1$, $x_2$, ... $x_m$ with all $x_i \,\, \epsilon \,\, \mathcal{R}^n$, $i=1,2... m$ and $m < n$. How to sample a vector $x_{m+1}$ perpendicular to all the vectors $x_1$, $...
3
votes
2answers
64 views

What is the Joint Density Function of a Three-Level Mixed-Effects Model?

This is a follow-up question to a question I posted earlier. Obviously, maximum-likelihood estimation of mixed-effects models requires the joint density function. Let us assume a two-level mixed ...
4
votes
1answer
81 views

Mixed Models: How to derive the mixed-model equations?

In the context of best linear unbiased predictors (BLUP), Henderson specified the mixed-model equations (see Henderson (1950): Estimation of Genetic Parameters. Annals of Mathematical Statistics, 21, ...
2
votes
1answer
69 views

Interpreting matrix notation to run MLE in R

I am trying to re-create some indicators from the World Bank, using the methodology described in this paper, and I need to do maximum likelihood estimation, preferably using R. The aim is to get an ...
7
votes
2answers
713 views

What is the problem with $p > n$?

I know that this is the solving system of linear equation problem. But my question is why it is a problem the number of observation is lower than the number of predictors how can that thing happen? ...
1
vote
0answers
53 views

Multilevel Modeling: Minimization Problem when A = B + TCT'

I currently study multilevel model using Leeuw & Meijer (2008) Handbook of multilevel analysis. On page 65, they state the following theorem: If $A = B + TCT'$ with $A$ and $B$ positive definite, ...
0
votes
0answers
27 views

Is there a problem with the “low-rank matrix approximation”?

I know that "rank" is the number or independent rows in the matrix and I know that the resulting matrix of the "low-rank matrix approximation" algorithm has lower rank than the original matrix. 1- But ...
0
votes
0answers
25 views

Writing the matrix form of a linear regression model?

I don't know how to write a simple linear regression model in a matrix form.. in our book we are given a table having values of $ x,y,x2,y2,xy.$ . I created a very small example and I attached it as ...
1
vote
0answers
32 views

Gaussian random fields: matrix and convolution sampling

I should be able to generate a stationary GRF from white noise in two different ways: multiplying the white noise vector by the square root of a covariance matrix with appropriate kernel; taking the ...
0
votes
0answers
13 views

Is there a relationship between the L2-norm and the Pearson chi-square test?

Suppose I have two datasets, $\mathbf{a}$ and $\mathbf{b}$, with some data weighting term (or error term), $\mathbf{e}$. I can compute the L2-norm for these two sets by the following: $$(\mathbf{a}-\...
1
vote
1answer
46 views

Why the representation in the form of $Z'X(X'X)^{-1}X'Z$ can not be simplified into $Z'Z$

Representation similar to $Z'X(X'X)^{-1}X'Z$ frequently appear to e.g. 2SLS. I think that $Z'X(X'X)^{-1}X'Z = Z'XX^{-1}X'^{-1}X'Z = Z'(XX^{-1})(X'^{-1}X')Z = Z'Z$. So why it seems that in the context ...
1
vote
0answers
34 views

VAR(p) Model Covariances and Moment Equation

I'm currently going through the book Analysis of Financial Time Series by Ruey S. Tsay and reached the following statement (The book can be found here, with VAR(1) included in the preview): Where: $...
0
votes
0answers
13 views

A question about the change in value of multiple correlation coefficient on multiplying each value with a variable quantity

Question: I know the formula of multiple correlation coefficient is; ( |R| is the correlation matrix and R11 is the cofactor of the (1,1)th element of R. But I really cannot figure out, how to deal ...
0
votes
1answer
28 views

Matrix Approach to Linear Regression Model

How do we interpret a data matrix, $X$ which does not have $1s$ as the first column? Does it refer to No Intercept Form? Could it also be interpreted as the mean deviated form? I understand that for ...
1
vote
0answers
25 views

How to simulate random correlation matrix containing off diagonal structures

I want to simulate a correlation matrix which has some off-diagonal structures and also should have some hierarchical structures. For simulating correlation matrices which contain hierarchical ...
1
vote
0answers
13 views

reorganise a data structure

I need to restructure my data, at the moment it looks like: Question number (39 total) in the header Participant ID | Response ( x 39 columns referring to Q numbers in the header) | Demographic x ...
2
votes
1answer
44 views

probability that matrix $2\times2$ of Random variables is Invertible

Let $X_1, X_2, X_3, X_4$ to be Variables, and let $A$ be the following matrix: $$ \left[\begin{matrix} X_1 & X_2\\ X_3 & X_4 \end{matrix}\right] $$ assume that $X_1, X_2, X_3, X_4$ are ...
0
votes
0answers
17 views

The miracle of the Lanczos/conjugate gradient algorithm

Lanczos/Arnoldi/Rietz/CG-like algorithm share the same core strategy... In each, a little miracle appears, most of the Gram-Schmidt inner products are zeroes ! In others words, new direction need only ...
1
vote
1answer
134 views

Measure of stability

I am working on a machine learning project when I realized I add a question. This is not programming, nor statistic, nor a probability question, but a real pure mathematical question. So I think my ...
0
votes
0answers
38 views

Decomposition of a diagonal matrix

I want to decompose a diagonal matrix $\Lambda \in R^{n \times n}$ such that $$ \Lambda \approx A\Sigma A^T $$ where $\Sigma \in R^{k \times k}$ is a diagonal matrix and $A \in R^{n \times k}$ is a ...
0
votes
0answers
34 views

Notation for Matrix Concatenation

I have an input data matrix $X$. In fact, I have $N$ of these input matrices, so I identify each as $X_i$ using an index $i \in \{1,\ldots,N\}$. Thinking about something like cross validation, I want ...
1
vote
1answer
63 views

CNN - Is this a Toeplitz Matrix?

I have been reading through Chapter 9 of www.deeplearningbbook.org, where convolutional networks are being described. The following image represents the output of a 2D convolution, without kernel ...
0
votes
1answer
76 views

How are biases updated when 'batch size' > 1?

This is my network represented in matrices: (a dot represents an arbitrary number) Feed-forwarding: (I omitted nesting it all in an activation function for the sake of brevity) Backpropagation The ...
2
votes
1answer
128 views

Matrices: system that is “computationally singular” versus “exactly singular” [closed]

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
1answer
97 views

Simulation of low rank and sparse matrix

I am having trouble simulating a matrix which is low rank and sparse (sparse along both rows and columns). One way to simulate a low-rank matrix is by generating a random matrix, then taking SVD and ...
1
vote
0answers
20 views

Recursively expressing matrix inverse

Let $X$ be an $D \times N$ matrix. Let $I$ be a $D \times D$ identity matrix. Also let $y$ be a $N \times 1$ column vector. Suppose we are trying to solve $(X X ^T + k I) w = Xy$ for a $D$ dimensional ...
0
votes
1answer
44 views

Calculate the implied correlation for missing cells in a correlation matrix in R

I have a correlation matrix in R. Many of the correlations are specified, but there are some that are "NA". eg, A __ B __ C A 100% NA 25% B NA 100% 50% C 25% 50% ...
0
votes
0answers
11 views

What kind of sparse matrix representation is this?

I am putting together a wrapper for a quadratic programming library. I am going through the C example here but I don't understand the indexing used for the matrices. The relevant excerpt is below, ...
0
votes
0answers
14 views

Rank Test for a Matrix

Suppose I have a matrix A corrupted with noise and I am looking for a test that tests the null hypothesis that the matrix A, rank(A)==1 v.s. rank(A)>1. I checked a little the literature and this paper ...
6
votes
1answer
901 views

Difference between Cholesky decomposition and log-cholesky Decomposition

Is there any difference between a Cholesky decomposition and a log-cholesky decomposition? If yes, what is the difference? In the paper "An R package for dynamic linear models" by Giovanni Petris ( ...
1
vote
0answers
34 views

Extracting latent vectors from autoencoder similar to SVD

I have read that there is an equivalency between a linear autoencoder and performing SVD. SVD can be used in collaborative filtering, for example, factorization of a user-movies matrix $\mathbf{M}$ ...
0
votes
0answers
10 views

Multivariate fixed point iteration: the wrt variable on both sides of update equation and not are different?

I try to understand more about the update in multivariate fixed point iteration. I saw the examples where the updates have the same variable (the wrt. variable of partial differentiation) on both ...
0
votes
0answers
35 views

Reliability Logistic Regression - train and evaluate the model [duplicate]

I have built an Logistic Regression model in R. The class that I want to predict, is very unbalanced (99 vs 1). My first finding is that this Logistic model does a better job if I train it on a ...
4
votes
2answers
134 views

Raising a variance-covariance matrix to a negative half power

I want to implement the following formula where $C$ is variance-covariance matrix of variables x, y, and z: $$C = \begin{bmatrix}cov(x,x)&cov(x,y)&cov(x,z)\\cov(y,x)&cov(y,y)&cov(y,z)\...
0
votes
1answer
175 views

PCA in psych package with more columns than rows

Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
0
votes
0answers
18 views

multivariate multiple regression, testing if a variable leads y's at the same time

I have a what I understand to be a multivariate multiple predictive regression. The y's are different variables and I am attempting to see if these are lead by w at the same time. I use the standard ...
1
vote
0answers
32 views

Obtaining hard, overlapping clusters using non-negative matrix factorization

From my understanding non-negative matrix factorization (NMF) provides a natural way to obtain soft clusters from a non-negative $n$x$m$ data matrix $X$. NMF decomposes $X$ into two non-negative ...
1
vote
0answers
52 views

Analyze similarity matrix using linear mixed model

Let's say I have a similarity matrix where every subject is compared to every other subject on some similarity measure (e.g., body movement synchrony). These subjects are divided into two groups, say ...
3
votes
1answer
66 views

Is there a (matrix) operation that can count elements in vector pairs?

For two vectors $x \in \{0, 1, 2\}^{n}$ and $y \in \{0, 1, 2\}^{n},$ I need to generate a matrix $C\in \mathcal{R}^{3\times3}$ where $C_{i,j}$ equals the number of indexes $t$ where $x[t]=i$ and $y[t]...
3
votes
0answers
115 views

Rayleigh quotient, traces and LDA optimization problem

I've been working about Linear Discriminant Analysis the last weeks, and after reading many articles, I see some aspects of this problem not very clear. The LDA optimization problem is formulated by ...