The matrix tag has no wiki summary.
4
votes
1answer
51 views
Visualising relationship data
I have data about a certain workforce, with different worker types W1, W2, W3, W4, etc. I need a method to visualise the relationship between each worker type. In particular, I'm interested in ...
1
vote
0answers
11 views
R Matrix process with conditional additions [migrated]
I have to pre-process a big matrix. To make my example easier to understand I will use the following matrix:
Raw data
$\begin{array}{ccc}
18 & 0 & 14\\
12 & 13 & 0\\
15 & 0 & ...
0
votes
1answer
60 views
Minimize a function with respect to a matrix
I have two sets of vectors, A and B. Vectors from set A live in an m-dimensional space, ...
0
votes
0answers
16 views
How do I label the rows and columns in a diagonal matrix? [migrated]
i'm creating a diagonal matrix of variances in R, thus:
D <- diag( data $ Variances, length(data $ Variances), length(data $ Variances))
Does anyone know how to add row and column labels? The ...
3
votes
1answer
46 views
Hold-one-out linear regression : a shortcut?
For a series of observations $(\vec{x}_i, y_i), i = 1 \cdots N$ from the linear model $Y = \beta^T X + \epsilon$, the least squares estimate of $\beta$ is: $\hat{\beta} = (\mathbf{X}^T ...
2
votes
0answers
33 views
How to compare two matrices?
I am working on Markov transition matrices. I would like to find a statistical test to compare them.
The first matrix is considered the population transition matrix and the second one is obtained by ...
1
vote
0answers
46 views
Floating point issues when transforming an arbitrary correlation matrix to positive semi-definite
I'm following Peter Jackel's book "Monte Carlo Methods in Finance", where an algorithm for transforming malformed correlation matrices into acceptable correlation matrices (positive semi-definite) to ...
0
votes
0answers
24 views
How to prepare variance matrix in Statistica (or different tool)?
I have data which describe measurments in four different point for few days.
Four water parameters have been measured.
In other words, data contains following columns:
point name - we have four ...
0
votes
0answers
84 views
Adding content information to matrix factorization-based recommender
I'm currently using a matrix factorization method to generate recommendations (for info on this, check: Matrix Factorization Techniques for Recommender Systems). At the moment, my rating estimate is ...
0
votes
1answer
51 views
What happens to the covariance matrix when the errors are independent?
I was wondering, what happens to the covariance matrix of the errors, when I assume that all the errors are stochastically independent?
Is the covariance matrix still:
$$\sigma^2I = ...
0
votes
0answers
35 views
R Quadratic programming and constrained optimization problem [closed]
I want find a vector p which lies in the positive null space of a matrix S and as well minimize the value of a least squares function with respect to p. Constraints are thus that p>0 and S*p=0
Here is ...
0
votes
1answer
20 views
Matrix image regression or regression of subscales
A person is given 10 sample images (based on the same image) with corresponding techniques on how the image was achieved. I want to regress the matrix bitmap of the image of the 10 images on the image ...
0
votes
0answers
44 views
Rank of within-class scatter matrix in LDA
Let $N$ be the number of total training examples from $C$ classes. Could anyone tell me why the rank of the within-class scatter matrix $S_w=\sum_{i=1}^C(N_i-1)S_i$ (where $S_i$ is the covariance ...
0
votes
2answers
74 views
Covariance matrix explanation
I try to understand and visualize myself covariance matrix. Supposing I have a matrix A = [ 2 3 4; 5 5 6 ], how do I calculate its covariance matrix, and what is ...
3
votes
2answers
142 views
Getting rid of a huge categorical factor in multiple regression
I have a large regression problem with a lot of cases, but relatively few independent variables. One of them is a categorical factor with thousands of levels. Robust regression runs forever. In some ...
1
vote
1answer
80 views
Proof for two-sample Hotelling $T^2$ statistic?
I've been reading "A primer of multivariate statistics" by Richard J. Harris, page 546, which shows how to derive the Hotelling $T^2$ statistic, after seeing this related but different question (I ...
0
votes
2answers
55 views
Non-distance metrics in hierarchical clustering? [closed]
What happens, intuitively, when one uses non-distance metrics to calculate the distance matrix that feeds into a standard hierarchical clustering algorithm?
What mistakes will the algorithm make and ...
4
votes
1answer
74 views
Confusion about scatter matrices
I am learning about evaluating clustering outcome and am confused about the scatter matrices. Hoping to get some help here.
The within-cluster scatter matrix $S_W$is defined as:
$$
S_W=\sum _{ k=1 ...
0
votes
0answers
16 views
Comparison of second-order probability matrices
I've searched over the site for matrix comparison questions, and it seems that the biggest response is 'what do you need it for'.
I have two second-order probability matrices, in which the rows and ...
0
votes
0answers
77 views
How to compute the difference between 2 similarity matrices?
I had asked this question on Mathoverflow.net and someone suggested that this might be a better forum. So here is the question again
Hello, I have two n*n correlation matrices with values ranging ...
0
votes
0answers
96 views
Help with correlation concepts
EDIT: I rewrote the whole question after following the suggestions given in the comments below, I hope everything's clear now.
I'm not very good at statistics, but I need a few tips to understand ...
-3
votes
1answer
46 views
0
votes
3answers
411 views
“matrix is not positive definite” - even when highly correlated variables are removed
I am running a factor analysis in SPSS and get a "matrix is not positive definite" error from my correlation matrix. I've tried removing correlated variables, but I have to remove all variables down ...
1
vote
0answers
76 views
Internal reliability for ordinal scale using Spearman correlation matrix
I have a question about estimating the reliability (coefficient alpha) of a pilot scale by assessing its internal consistency. The pilot questionnaire generates ordinal data, 9 items each with 9 ...
0
votes
1answer
86 views
Clustering of set of matrices
I have 25 matrices 19x19 containing coherence measure between EEG electrodes. I want to divided them into some groups by clustering or any other method. I know how to deal with vectors, but I can't ...
0
votes
0answers
218 views
Variance Covariance matrix of regression coefficients
In a procedure, I am resampling (bootstrapping) from a Y vector and X matrix.
For each resample, I standardize.
Then I run a linear model, and obtain the regression coefficients for each variable of ...
6
votes
1answer
237 views
Calculating Jaccard or other association coefficient for binary data using matrix multiplication
I want to know is their any possible way to calculate Jaccard coefficient using matrix multiplication.
I used this code
...
0
votes
0answers
26 views
Why are the eigenvalues of a covariance matrix corresponding to the data's variance? [duplicate]
For multi-dimensional data, we can compute its covariance matrix, and then the eigenvalues and eigenvectors. It turns out that the eigenvector with the largest eigenvalue corresonds to the direction ...
1
vote
0answers
32 views
Observed info matrix via Hessian
In some resources, I saw that the observed information matrix is the negative of the expected value of the hessian matrix. However, in some other resources I saw that it is just the negative of the ...
0
votes
0answers
43 views
Confidence intervals based on asymptotic normality
I wrote a report about confidence intervals based on asymptotic normality. However, my supervisor said that the definition in the second paragraph is wrong, but I ...
0
votes
0answers
29 views
Find and fill values for the missing points using close/similar vectors
I have a matrix which looks like the attached image below, and I would like to know what's the statistically best educated guess for the missing steps (filled in orange) for the vector $\mathbf N$.
...
2
votes
0answers
69 views
Matrix completion: How to assign names to the completed columns?
I am wondering if this the right place to ask this question. Normally it should be :).
I am recently reading some papers about matrix completion such as in here, and here.
I didn't go through some ...
0
votes
0answers
99 views
Multivariate distance function in Excel
I want to do a simple nearest neighbour calculation in Excel over a multivariate space to get an idea of how my data clusters. I have a set of data points $\{X_1, X_2, \ldots X_n\}$ and a set of ...
3
votes
1answer
132 views
Sparse representations for denoising problems
I have read in a huge number of papers that sparse models (sparse coding, dictionary learning, sparse matrix factorization, ...) are good solutions for image denoising problems.
I know that ...
1
vote
1answer
97 views
In non-negative matrix factorization, are the coefficients of features comparable?
I'm using Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient. The result (I use 2 as rank) is like this:
...
2
votes
1answer
54 views
Why are eigenvalues so significant in DR techniques?
I've read PCA and few more dimension reduction algorithms and all of them talk about using eigen values and matrix operations. How are they so significant in discovering geometrical significance of ...
1
vote
1answer
151 views
Hotelling T^2 test derivation question
I am reading about the Hotelling $T^2$ test (A primer of multivariate statistic s by Richard J. Harris). It says here that the test can be seen as creating a linear combination of your variables and ...
1
vote
1answer
239 views
How to calculate error matrices, K hat and var(K hat)?
I have a very simple data set consisting of three columns: Ground Truth Canopy Class, Method 1 Canopy Class and Method 2 Canopy Class. Each row in the columns represents the canopy class (i.e. 1 ...
0
votes
0answers
22 views
How to decide the dimension of non-negative matrix factorization?
Should I use the explained variance of the non-negative matrix factorization estimate of the target matrix as the measurement, as what we usually do in PCA?
3
votes
1answer
251 views
Mahalanobis distance on singular data
I have an issue which I could not solve, although I tried and I got some help on R help forums too.
I am trying to calculate Mahalanobis distances on a data frame, where I have several hundreds of ...
2
votes
0answers
124 views
Transformation of multivariate normal sum of chi-squared
If $A$ is symmetric and $Y\sim\mathcal N(0,V)$, how can I show that $Y'AY\sim\sum_{i=1}^{t}(c_i * \chi^2(1))$ with 1 degree of freedom), where $c_i$ can be any scalar?
I multiplied out the canonical ...
1
vote
1answer
299 views
1
vote
1answer
187 views
Topic modeling, LDA and NMF
My objective is to implement a topic model for a large number of documents (20M or 30M). Let us assume that the number of topics is fixed at 50.
I think implementing an LDA for the above problem ...
2
votes
0answers
181 views
Principal component analysis, bootstrap and probability of eigenvalue collision?
This is really a side project of mine ... while writing on a paper on something totally different! I read (part of ) the excellent paper
"FINITE SAMPLE APPROXIMATION RESULTS
FOR PRINCIPAL COMPONENT ...
0
votes
0answers
49 views
finding significant pairs in co-occurrence matrix
For a matrix $M$ in which entries $m_{a,b}$ denote the number of co-occurrences between elements $a,b$ from two distinct sets $A$ and $B$, how do I identify pairs with a significantly high ...
0
votes
0answers
84 views
Singular covariance matrix and Spatially-correlated random effects [closed]
I'm interested in incorporating spatially-correlated random effects into my model to explicitly account for between-observation spatial autocorrelation, such that spatial autocorrelation decreases ...
12
votes
3answers
259 views
Why bother with low rank approximations?
If you have a matrix with n rows and m columns, you can use SVD or other methods to calculate a low-rank approximation of the given matrix.
However, the low rank approximation will still have n rows ...
7
votes
3answers
176 views
Sparsity-inducing regularization for stochastic matrices
It is well-known (e.g. in the field of compressive sensing) that the $L_1$ norm is "sparsity-inducing," in the sense that if we minimize the functional (for fixed matrix $A$ and vector $\vec{b}$) ...
5
votes
1answer
109 views
MCMC for structured matrix
I'm working with a Gaussian random field, which can be described by joint pdf as follows:
$$X\mid \sim N\left ( \mu, \sigma^{2} \left ( I-C \right )^{-1}\right )$$
where $C$ is a structured matrix, ...
2
votes
0answers
187 views
Problem when creating matrix of values based on covariance matrix
I want to simulate a data set with similar covariance structure as my
observed data (which is a SNP by gene p-value matrix, dim ~600k*8368), and have calculated a covariance matrix (dimensions ...
