Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

A matrix (plural matrices) is a rectangular array of numbers, symbols, or expressions arranged in rows and columns. The individual items in a matrix are called its elements or entries.

0
votes
0answers
13 views

Self Attention in the Transformer learning algorithm

in this article can somebody tell me where the heck the Wq, Wk, and Wv matrices in the “Self-Attention in Detail” section come from a little more intuitively and specifically since the article doesn't ...
0
votes
0answers
11 views

Solve an equation of two unknowns to fit a distribution and mean

Apologize in advance for the drawn out question (found at the end). But I want to give a comprehensive picture of the problem. I have an equation of the following form: $T_{ij}^{sim}=A_i*O_i^{^{obs}...
0
votes
0answers
9 views

(Why) Is the ouput's dimension in a LSTM bound to the number of recurrent units, and how are recurrent outputs passed?

Background: Looking for specific/interesting information on the equations within LSTM-Networks, I found the paper LSTM: A Search Space Odyssey. It is frequently mentioned in other articles. To gain a ...
0
votes
0answers
6 views

Help with sum specific elements of a matrix in mathematica [migrated]

i trying to sum over specific index in a matrix. For example. If i have the matrix a = {{1, 2, 3, 3}, {4, 5, 6, 6}, {7, 8, 9, 9}, {1, 5, 9, 7}} i need to sum the index {2, 1} and {4, 3} , i.e 4 + ...
0
votes
0answers
12 views

Why is matrix not invertible (row form) [migrated]

Why matrix is not invertible if A has row 1 + row 2 = row 3? I'm interested in intuition so please elaborate on the answer please.
0
votes
0answers
6 views

Proximity Matrix in Hiearchical Clustering

I have a proximity matrix from a hiearchical clustering calculation. I have used the Pearson Correlation as my distance metric. Is it valid to test the statistical significance of this value using a ...
1
vote
1answer
46 views

Matrices: system that is “computationally singular” versus “exactly singular”

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
1answer
63 views

Simulation of low rank and sparse matrix

I am having trouble simulating a matrix which is low rank and sparse (sparse along both rows and columns). One way to simulate a low-rank matrix is by generating a random matrix, then taking SVD and ...
1
vote
0answers
18 views

Recursively expressing matrix inverse

Let $X$ be an $D \times N$ matrix. Let $I$ be a $D \times D$ identity matrix. Also let $y$ be a $N \times 1$ column vector. Suppose we are trying to solve $(X X ^T + k I) w = Xy$ for a $D$ dimensional ...
0
votes
1answer
37 views

Calculate the implied correlation for missing cells in a correlation matrix in R

I have a correlation matrix in R. Many of the correlations are specified, but there are some that are "NA". eg, A __ B __ C A 100% NA 25% B NA 100% 50% C 25% 50% ...
0
votes
0answers
11 views

What kind of sparse matrix representation is this?

I am putting together a wrapper for a quadratic programming library. I am going through the C example here but I don't understand the indexing used for the matrices. The relevant excerpt is below, ...
0
votes
0answers
22 views

Conditions for the Fisher Information matrix to be invertible

The Fisher Information Matrix is positive semi definite. So, it is not necessarily invertible. By the Multivariate Central Limit Theorem we know that $\sqrt{n}(\hat{\theta}−\theta)=S_{n}⟹\mathcal{N}(...
0
votes
0answers
11 views

Rank Test for a Matrix

Suppose I have a matrix A corrupted with noise and I am looking for a test that tests the null hypothesis that the matrix A, rank(A)==1 v.s. rank(A)>1. I checked a little the literature and this paper ...
6
votes
1answer
391 views

Difference between Cholesky decomposition and log-cholesky Decomposition

Is there any difference between a Cholesky decomposition and a log-cholesky decomposition? If yes, what is the difference? In the paper "An R package for dynamic linear models" by Giovanni Petris ( ...
1
vote
0answers
14 views

Extracting latent vectors from autoencoder similar to SVD

I have read that there is an equivalency between a linear autoencoder and performing SVD. SVD can be used in collaborative filtering, for example, factorization of a user-movies matrix $\mathbf{M}$ ...
0
votes
0answers
6 views

Multivariate fixed point iteration: the wrt variable on both sides of update equation and not are different?

I try to understand more about the update in multivariate fixed point iteration. I saw the examples where the updates have the same variable (the wrt. variable of partial differentiation) on both ...
0
votes
0answers
29 views

Reliability Logistic Regression - train and evaluate the model [duplicate]

I have built an Logistic Regression model in R. The class that I want to predict, is very unbalanced (99 vs 1). My first finding is that this Logistic model does a better job if I train it on a ...
4
votes
2answers
100 views

Raising a variance-covariance matrix to a negative half power

I want to implement the following formula where $C$ is variance-covariance matrix of variables x, y, and z: $$C = \begin{bmatrix}cov(x,x)&cov(x,y)&cov(x,z)\\cov(y,x)&cov(y,y)&cov(y,z)\...
0
votes
1answer
70 views

PCA in psych package with more columns than rows

Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
0
votes
0answers
14 views

multivariate multiple regression, testing if a variable leads y's at the same time

I have a what I understand to be a multivariate multiple predictive regression. The y's are different variables and I am attempting to see if these are lead by w at the same time. I use the standard ...
1
vote
0answers
28 views

Obtaining hard, overlapping clusters using non-negative matrix factorization

From my understanding non-negative matrix factorization (NMF) provides a natural way to obtain soft clusters from a non-negative $n$x$m$ data matrix $X$. NMF decomposes $X$ into two non-negative ...
1
vote
0answers
45 views

Analyze similarity matrix using linear mixed model

Let's say I have a similarity matrix where every subject is compared to every other subject on some similarity measure (e.g., body movement synchrony). These subjects are divided into two groups, say ...
0
votes
0answers
21 views

Is there a (matrix) operation that can count the elements in the vector?

For two vectors $x \in \{0, 1, 2\}^{n}$ and $y \in \{0, 1, 2\}^{n}$ And I need to generate a matrix $C\in \mathcal{R}^{3\times3}$, where $C_{i,j}$ equals to the number of index $t$, where $x[t]=i$ ...
2
votes
0answers
71 views

Rayleigh quotient, traces and LDA optimization problem

I've been working about Linear Discriminant Analysis the last weeks, and after reading many articles, I see some aspects of this problem not very clear. The LDA optimization problem is formulated by ...
0
votes
0answers
14 views

Finding probability vectors from a matrix equation

I have $q$ $n$-dimensional vectors $\vec y_i$ and a matrix $\hat B$ of shape $n\times m$. I'm looking for $q$ $m$-dimensional vectors $\vec x_i$ such that: $\vec y_i=\hat B \vec x_i$ each vector $\...
1
vote
1answer
34 views

Explained Sums of Squares in matrix notation

I am currently reading Appendix C from Gujarati Basic Econometrics 5e. It deals with the Matrix Approach to Linear Regression Model. I am unable to decipher how the author went from equation 7.4.19 ...
1
vote
0answers
38 views

Is pairwise distance matrix useful to k-means?

The k-means implemented in scikit-learn precomputes distances but I don't how these distances are used. In its standard version, k-means is known to compute only the distances between the points and ...
0
votes
0answers
15 views

math notation for batch matrix multiply

I use batch matrix multiply (torch.bmm) very often in my models and I want to write them down in math notation for documentation purposes. Does bmm have a standard ...
2
votes
1answer
39 views

Positive definiteness of Grammian with respect to Gaussian process' covariance function

A Gaussian process indexed by $T \subseteq \mathbb{R}^d$ is a collection of random variables $\{ X_t : t \in T\}$, for which each finite subset is distributed as a multivariate Gaussian. Let $G$ be a ...
2
votes
0answers
95 views

Obtain within-group Gram matrix out of distance matrix

Gram matrix Let $\bf X$ be a n x p dataset with columns (variables) centered. Then p x p $\bf X'X$ is the total scatter matrix ...
0
votes
0answers
15 views

calculate probabilities with Markov / Matrix calculations

I am trying to get a little bit back into matrix calculations after I have sucessfully ignored it for about 25 years. Certainly you're laughing at me for this question. Here it comes: I thought I'd ...
1
vote
0answers
17 views

Predicting categories based on a similarity matrix

I am looking for some help in organising some analyses. I will describe what I am trying to do with a fictional example and then talk about some of the things I've thought about already. Example I ...
2
votes
0answers
35 views

Are null space of matrix and kernel function same?

I have recently started learning about machine learning and have come across kernels and null spaces. I understand that null space is the set of all vectors that satisfy the equation A.v = 0 (Where A ...
0
votes
1answer
32 views

Principle Component Analysis: Why Are Many Solutions Possible?

The following example is from chapter 2.12 of Deep Learning, by Goodfellow, Bengio, and Courville: I don't understand what is meant by the very last part: "Many solutions are possible, because we can ...
0
votes
0answers
19 views

How to obtain a single indicator from a correlation matrix of 6 or N variables?

The problem that I will like to ask is the following. In the problem each cross-correlation matrix represent a different state (M states), with N different variables (in my case 6). For each state ...
1
vote
0answers
15 views

eigenvalue perturbation theory for kernel function

Let $S=\{x_i\}_{i=1}^n$ be a set of training examples, and let $K\in \mathbb{S}^n_+$ be the kernel matrix induced by $S$ and some kernel function $k$ (i.e., $K_{ij}=k(x_i,x_j)$). I was wondering how ...
0
votes
0answers
46 views

Variance of a random variable by matrix

I'm trying to calculate $ \text{VAR}(\mathbf{XW}) $ where $ \mathbf{X} \sim \mathcal{N}(\mu, \sigma^2) $ and $\mathbf{X}$ is a matrix of size $ N\times D$ in which all entries are independant so both $...
0
votes
0answers
56 views

Difference between Dummy Variables, Complete Disjunctive Table, and One-Hot Encoding

These three transformations seem to perform similar (or the same) functions, and are used for similar purposes. What's the difference, if any, between these three techniques?
3
votes
0answers
28 views

what is the hessian matrix of $f(W) = \sqrt{Tr(W A_0 W A_1)}$? Here, $W$, $A_0$ and $A_1$ are positive semidefinite matrices and hermitian

what is the hessian matrix of $f(W) = \sqrt{Tr(W A_0 W A_1)}$? Here, $W$, $A_0$ and $A_1$ are positive semidefinite matrices and hermitian. For the time being, I obtain the derivative as $\frac{\...
1
vote
1answer
162 views

In restrict linear model, why Cov($A^T(X^TX)^-X^TY$) is positive definite?

In restricted linear model, we have $$Y = X\beta+\varepsilon$$ $$A^T\beta=b$$ Where $A^T\beta$ is estimable (i.e., there exists a matrix $D$, such that $D^TX=A^T$) and $A$ is a $p\times q$ matrix ...
7
votes
3answers
67 views

Converting the beta coefficient from matrix to scalar notation in OLS regression

I've found for my econometrics exams that if I forget the scalar notation, I can often save myself by remembering the matrix notation and working backwards. However, the following confused me. Given ...
0
votes
1answer
31 views

Help in understanding the ridge regression solution break down?

I tried to follow Jann Goschenhofer's answer here, but I don't understand How $x_i^T$ in $Criterion_{Ridge} = \sum_{i=1}^{n}(y_i-x_i^T\beta)^2 + \lambda \sum_{j=1}^p\beta_j^2$ became just $X$ without ...
1
vote
0answers
9 views

How many scale points can you have on a Van Westendorp survey?

I am conducting a Van Westendorp pricing survey. The market I am looking at has a wide range of prices (£3,000 to £33,000), how many scale points can I use for my Van Westendorp question to accurately ...
2
votes
1answer
30 views

How to compute adjustment weights to match marginal sums of a matrix?

I have a feeling this should be very simple, but I somehow got stuck thinking about it. I have $\it X$, which is a 15 x 18 matrix containing non-negative real values. I smoothed this matrix using a ...
0
votes
0answers
39 views

Scaling a data matrix to execute an algorithm

Suppose that I have to perform a certain algorithm (MCMC) on data that are stored in a matrix. In order to accelerate the convergence, I want to scale the centering them around 0. What type of ...
5
votes
1answer
146 views

Must a matrix of sample pairwise covariances be PSD?

Consider a random vector $\mathbf{X}=(X)_{i=1}^n$. Then the covariance matrix $$C=\mathbb{E}[(\mathbf{X}-\mu(\mathbf{X}))(\mathbf{X}-\mu(\mathbf{X}))^\top]$$ is by definition positive-semidefinite. (...
0
votes
0answers
19 views

Deriving the gradient vector of a Probit model

Consider the probit regression model where the pdf of $y_{i}$ is $$f(y_{i};\mathbf{\beta}) = \mu_{i}^{y_{i}}\left ( 1 - \mu_{i} \right )^{1 - y_{i}},$$ where $y_{i} = \left \{ 0,1 \right \}$ and $\...
0
votes
0answers
13 views

Quantitatively, how are weights computed via PCA for variables with 2 classes of observations?

I'm reading a paper where authors investigate multiple variables that may lead to batches of tomatoes in an industrial setting becoming one of two categories, "rotten" or "fresh" upon inspection. ...
0
votes
0answers
15 views

Fourier Transform based imputation

Fourier Transform based imputation Can any body please assist me in understanding the Fourier Transform based imputation algorithm shown in Figure. I am struggling to understand ts and te. Link of ...
0
votes
1answer
291 views

User based Collaborative Filtering (with Python)

I am trying to understand how can I calculate the similarity between userid and itemid. Here is the user-based table.The table ...