# Tag Info

### Relationship between SVD and PCA. How to use SVD to perform PCA?

I wrote a Python & Numpy snippet that accompanies @amoeba's answer and I leave it here in case it is useful for someone. The comments are mostly taken from @amoeba's answer. ...
Accepted

### If I generate a random symmetric matrix, what's the chance it is positive definite?

If your matrices are drawn from standard-normal iid entries, the probability of being positive-definite is approximately $p_N\approx 3^{-N^2/4}$, so for example if $N=5$, the chance is 1/1000, and ...
• 14k

### Relationship between SVD and PCA. How to use SVD to perform PCA?

Let me start with PCA. Suppose that you have $n$ data points comprised of $d$ numbers (or dimensions) each. If you center this data (subtract the mean data point $\mu$ from each data vector $x_i$) you ...
• 511

### What is an example of perfect multicollinearity?

Here are a couple of fairly common scenarios producing perfect multicollinearity, i.e. situations in which the columns of the design matrix are linearly dependent. Recall from linear algebra that this ...
• 23.7k
Accepted

### Is the average of positive-definite matrices also positive-definite?

Yes, it is. jth asnwer is correct (+1) but I think you can get a much simple explanation with just basic Linear Algebra. Assume $A$ and $B$ are positive definite matrices for size $n$. By definition ...
• 44.8k
Accepted

### Completing a $3 \times 3$ correlation matrix — $2$ coefficients of the $3$ given

We already know $\gamma$ is bounded between $[-1,1]$ The correlation matrix should be positive semidefinite and hence its principal minors should be nonnegative Thus, \begin{align*} 1(1-\gamma^2)-0.6(...
• 3,435

### What is the intuition behind SVD?

Let $A$ be a real $m \times n$ matrix. I'll assume that $m \geq n$ for simplicity. It's natural to ask in which direction $v$ does $A$ have the most impact (or the most explosiveness, or the most ...
• 610
Accepted

### What justifies this calculation of the derivative of a matrix function?

There is a subtle but heavy abuse of the notation that renders many of the steps confusing. Let's address this issue by going back to the definitions of matrix multiplication, transposition, traces, ...
• 327k

### Why the sudden fascination with tensors?

As someone who studies and builds neural networks and has repeatedly asked this question, I've come to the conclusion that we borrow useful aspects of tensor notation simply because they make ...
Accepted

### Are 1-dimensional numpy arrays equivalent to vectors?

A NumPy array is a N-dimensional container of items of the same type and size. As a computer programming data structure, it is limited by resources and dtype --- there are values which are not ...
• 579

### A term for "number of columns" of a matrix

There is a concept of wide and narrow data, so maybe you could use the term „width“ for the number of columns after you define it in order to avoid the ambiguity.
Accepted

### Why can't we cancel these two matrices in the OLS estimator?

Why can't we simply cancel out the $X^T$ I remember asking myself almost exactly the same question 300 years ago upon seeing the regression equation $y=X\beta$ for the first time in my life. The ...
• 61.7k

### In a convolutional neural network (CNN), when convolving the image, is the operation used the dot product or the sum of element-wise multiplication?

Any given layer in a CNN has typically 3 dimensions (we'll call them height, width, depth). The convolution will produce a new layer with a new (or same) height, width and depth. The operation however ...
• 6,023
Accepted

### Why the default matrix norm is spectral norm and not Frobenius norm?

In general, I am unsure that the spectral norm is the most widely used. For example the Frobenius norm is used for to approximate solution on non-negative matrix factorisation or correlation/...
• 44.8k