Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

How to have a constant error[b-a] for a range of values [1d array] in decending order that are not linear but are semi-linear?

I am trying to generate n numbers in decending order (for the purpose of decreasing order of weights) such that their sum should be 100 and more importantly the difference between anytwo numbers ...
user avatar
  • 1
1 vote
0 answers
20 views

Optimal truncation in SVD

I am working with SVD on a matrix $$Y_{m,n} = T_{m,m} \Sigma D^T_{n,n} $$ where $T$ and $D$ describe the row and the column entities of Y, respectively. The truncated SVD takes the first $r$ ...
user avatar
0 votes
0 answers
12 views

What is the meaning of the inner product between two regression variables?

I have been analyzing the effect of design matrix columns on the contour line of the least squares regression. These contours obviously are ellipses when only two columns $\phi_1$ and $\phi_2$ are ...
user avatar
  • 11
0 votes
0 answers
39 views

Matrix operator to stack the rows of a matrix on the "diagonal" [migrated]

I am looking for a matrix operator (or a mathematical expression) that does the following: I have a matrix A of dimesion $3 \times 5$ : $$A = \begin{bmatrix}a_{11}&a_{12}&a_{13}&a_{14}&...
user avatar
  • 137
0 votes
0 answers
24 views

Maximizing a unique trace quadratic form

I am dealing with an unsupervised problem where I have ended up with the following maximization problem: $\max_{C\in \mathbb{R}^{p\times n}}\sum_{i=0}^{m} tr(CA^ixx^\top A^{i^\top}C^\top) \\ \mathrm{...
user avatar
1 vote
1 answer
56 views

Matrix dimensions in Linear Algebra vs Time series Analysis

I am confused or may misunderstand the dimensions of a Matrix when I was reading about time series analysis. From what I understand in linear Algebra, if we have a Matrix $A \in \mathbf{R}^{m*n}$, ...
user avatar
  • 13
3 votes
1 answer
51 views

Re-using computations in several least squares problems

I have $K$ least squares problems of the form $Y_k = X_k\beta_k$ for $k = 1, \dots, n$. If the matrix $X_k$ is the same for each index $k$, we can rewrite the problem as $Y_k = X \beta_k$. How can I ...
user avatar
0 votes
1 answer
19 views

How to understand the relations in matrix multiplications in deep learning?

In the scaled dot product attention they multiply a "softmaxed" matrix (which has shape (sequence_length, sequence_length) I think?) to the V matrix as shown What does the second purple ...
user avatar
1 vote
0 answers
9 views

duel regression with orthogonal constraints on coefficients [closed]

I'm trying to solve a problem where I need to find two deming style regression models onto two different data sets of equal dimension who's coefficients(A,B) satisfy the following criteria with ...
user avatar
  • 11
1 vote
0 answers
30 views

Positive semi definite matrix with negative eigenvalues?

From what I know, for any square real matrix A, a matrix generated with the following should be a positive semidefinite (PSD) matrix: ...
user avatar
0 votes
0 answers
27 views

How to measure whether a set of vectors are more similar to each other than a set of randomly selected vectors would be?

I have ~15 000 word embedding vectors of length 256, that can be categorized into several groups (sizes 2 to 1500) via heuristic considerations. My aim is to measure whether the embedding vectors ...
user avatar
0 votes
1 answer
12 views

Product of elements of a vector which has very large values and very low values ordered in decreasing order (MATLAB)

I have to compute the product of the elements in a vector V. The elements of my vector are in decreasing order and go from very large numbers (eg 5e^5) to very small numbers (e.g 1.8e^-8). I am using ...
user avatar
  • 137
0 votes
0 answers
20 views

Dot product and linear combination of Basis

I am currently going through the SVD intuition provided here. In the section "From intuition to definition", It says that, First, note that any vector $\textbf{x}$ can be described using ...
user avatar
0 votes
0 answers
23 views

how do I eliminate redundant columns in matrix of nominal data?

I have data like this: c1 c2 c3 c4 c5 r1 a a b c d r2 NA a b c a r3 NA b b b c r4 a b c b c Here, each row is a data ...
user avatar
  • 101
0 votes
0 answers
40 views

Statistical significance and angular comparison (similarity test) between two vectors, MCMC and resampling approaches

I want to compare statistically if two vectors of PC1 loadings differ in terms of their direction which can be measured by the angle between them (e.g. this CV thread). I would like if someone can ...
user avatar
  • 377
0 votes
0 answers
40 views

what is the difference between factor analysis and SVD? factanal() vs svd()

I am doing factor analysis. Some sources tell me that I should use factanal() to do (exploratory) factor analysis; my goal is to find common sources of latent ...
user avatar
  • 93
0 votes
0 answers
25 views

Closed form solution for a linear problem

Consider the following problem: $W = \text{argmin} \, || SWE^T - P ||_2$ , where $S \in R^{m\times k}, W \in R^{k\times l}, E\in R^{n\times l}, P\in R^{m\times n}$. Matrices S, E, P are known and I'm ...
user avatar
5 votes
2 answers
585 views

Introduction to Statistical Learning Eq. 4.32

Can someone please explain how the third line becomes the fourth line?
user avatar
  • 93
0 votes
0 answers
18 views

how to find the best linear unbiased estimator for scalar product [duplicate]

if we have known the E[Y]=τ, How could we prove that the best linear unbiased estimator of the scalar x.τ is x.Y
user avatar
  • 1
3 votes
1 answer
47 views

How to express the sample variance as a homogeneous quadratic form when the population variance is known?

Given $\widetilde{S}^{2}=\frac{\sum_{i=1}^{n}{(X_{i}-\mu)^2}}{n}$ where $X=(X_{1},X_{2},...,X_{n})^T$. I was trying to find the $n\times n$ matrix $A$ for this quadratic form; that is, $$\widetilde{S}^...
user avatar
  • 31
0 votes
0 answers
23 views

Why can the hyperdimensional plane be discribed as $\textbf{w} \cdot \textbf{x} - b$ for support vector machines

So given the picture and the related definitions from this answer: How does the equation $\textbf{w} \cdot \textbf{x}^{(i)} - b = -1$ hold for several vectors $\textbf{x}^{(i)}$ when $\textbf{w} \...
user avatar
0 votes
0 answers
19 views

Lagrangian maximisation of a convex function

Having two real matrices $X_1$ and $X_2$ of size $(n,p_1)$ and $(n,p_2)$ respectively. I am trying to find the optimal vectors $u$ and $v$ such that $|u^TX_1^TX_2v|$ is maximum, with the constrains $||...
user avatar
1 vote
1 answer
22 views

How to compare row entries in a sparse table with lots of missing values?

I have a dataset with ~1000 laptops and performance results across ~100 different benchmarks. Using the benchmark results, I want to give each laptop a single composite performance score, and rank the ...
user avatar
1 vote
1 answer
74 views

Quantifying how underdetermined a system of equations or optimization problem is

In linear systems we have an exact solution when we have as many equations as unknowns and the equations are linearly independent, e.g., $$ x_0 + 2 x_1 = 5 \\ x_0 + 3 x_1 = 7 \\ $$ has the unique ...
user avatar
2 votes
1 answer
39 views

On distributions over orthonormal sets: existing families, construction, and simulation

Have families of distributions over orthonormal sets been defined and studied in the literature? What are a couple examples and/or references? Are there known methods for constructing distributions ...
user avatar
  • 2,049
0 votes
0 answers
25 views

minimum determinant covariance matrix and covariance

I am trying to understand minimum determinant covariance. I gather from this stack exchange post that it tries to select a subset of data that is tightly distributed to exclude anomalies, and it does ...
user avatar
  • 123
0 votes
0 answers
71 views

How to calculate the bias b in support vector machine when the dual coefficient alpha is obtained?

For my example, I have two data points x = {(54001.988, 19999), (30021.983, 15000} and their labels are y = {1, -1}. I calculated the dual coefficient(Lagrange multipliers) alpha = {10000, 10000}. The ...
user avatar
0 votes
1 answer
28 views

Do SVM's find the minimal number of support vectors in the case of redundancy?

Consider the case where more than the minimal number of vectors lie on the lines (or hyperplanes in higher dimensions) defined by the margin found by the SVM algorithm. For example, see the image ...
user avatar
  • 436
1 vote
2 answers
55 views

distance between regression models

Consider two multivariate linear regression models (vector inputs and outputs) with the same domain observations. Namely, let: $X \in \mathcal{R}^{a \times N}$ be a matrix of domain observations (...
user avatar
1 vote
0 answers
41 views

Finite non-zero determinant constraint?

Assume I want to train a multivariable normal distribution on given data set T. One definition of multivariable normal distribution is this: A random vector X has a (multivariate) normal distribution ...
user avatar
  • 11
0 votes
0 answers
32 views

kernel PCA similarity matrix analogy

The standard explanation to linear PCA begins with the covariance matrix. That is, for a dataset $D$ of dimension $N \times d$, the covariance matrix is given as $\sum = \frac{D^{T}D}{N}$ where the ...
user avatar
  • 207
0 votes
0 answers
47 views

Show inequality for $var(\tilde{\beta})$ for a linear mixed model

Consider the usual linear mixed model: $$Y=X \beta+ZB+\epsilon $$ where Y and $\epsilon$ are $n$-dimensional random variables and $B$ is a $q$-dimensional random variable independent of $\epsilon$ so ...
user avatar
  • 253
2 votes
1 answer
40 views

Expectation of the product of two independent random vectors and a positive-definite matrix

I am trying to compute the following: $\mathbb{E}[X^T\Omega^{-1}\epsilon]$, where $X$ is a random matrix, $\epsilon$ is a random vector, $\Omega$ is a real positive-definite matrix, and $\mathbb{E}[X^...
user avatar
  • 131
1 vote
1 answer
60 views

How do we know that we attain a minimum when $X^TX$ does not have full rank?

Assume we have a linear model $E[\textbf{y}]=\textbf{X} \beta$. When we use least squares we get the normal equations $\textbf{X}^Ty=\textbf{X}^T\textbf{X}\hat{\beta}$. Assume that $\textbf{X}$ does ...
user avatar
0 votes
1 answer
61 views

linear model estimator proof

For a linear model, the noncentered form is $Y_i=\beta_0+\beta_1x_{i1}+...+\beta_kx_{ik}+\epsilon_i$ ---(a) the centered form is $Y_i=\alpha+\beta_1(x_{i1}-\bar{x}_1)+...+\beta_k(x_{ik}-\bar{x}_k)+\...
user avatar
1 vote
1 answer
88 views

How to prove Mean Squarred Error (MSE)

I would like to prove this equation of Mean Squared Error (MSE): m is the number of training instances. X is a m × n matrix containing all the feature values (excluding labels) of all instances in ...
user avatar
0 votes
0 answers
17 views

Help with polynomial long division equations in "Three Model Representation for an ARMA Model" section by Tsay

I have 2 follow up questions to the same titled question posted here 7-years ago. Relating to Tsay's book section "2.6.5 Three Model Representations for an ARMA Model": Question 1 relates to ...
user avatar
1 vote
1 answer
48 views

Fitting of a sum of vectors $Y \alpha = X \beta + \epsilon$ [duplicate]

Related question / Motivation This question is related to the question here. But the difference in this question is that the weights can be different on both sides of the equation. This question This ...
user avatar
0 votes
1 answer
64 views

Question regarding matrix notation

I'm trying to get my head around a statistical topic where I look at dental measurements on eleven girls and sixteen boys at four different ages. One matrix that shows up is the following one: \begin{...
user avatar
1 vote
0 answers
96 views

Using multivariate normal likelihood when determinant of covariance matrix is zero

Estimating a multivariate normal (MVN) model requires minimising the negative log-likelihood of MVN (constant term dropped): $$ \ell(X|\mu,\Sigma)=\frac{n}{2}\log|\Sigma|+\frac{1}{2}(X-\mu)^T\Sigma^{-...
user avatar
  • 749
1 vote
0 answers
27 views

Variance and Covariance of Fixed Effects expressed in Quadratic Form

I know that for a simple vector $x$ of length $n$, the variance of this vector $\sigma_x^2= \frac{1}{n}\sum_{i=1}^{n} x_{i}^{2}-\left(\frac{1}{n}\sum_{i=1}^{n} x_{i}\right)^{2}$ can be written as $x^{\...
user avatar
1 vote
0 answers
42 views

Relation between PCA eigen values and data visualization

We have matrix data $X$ which is $n\times d$. We use the covariance matrix/ design matrix/ gram matrix $X^T X$ to perform least-squares/ PCA. I compute the eigen basis representation of said matrix $$...
user avatar
  • 181
0 votes
0 answers
25 views

Proof that design matrices with entries near zero lead to high variance of least squares estimator

I asked this question on Math Stackexchange but it might be better-suited to this site. I have a linear model $Y = X\beta + \epsilon$ where $\epsilon \sim (0_n, \sigma^2 I_n)$. The matrix $X$ is $n \...
user avatar
  • 531
0 votes
0 answers
7 views

Why the inverse for unknown coefficients vector? [duplicate]

From my understanding, this formula is used for least-squares when we're interested in minimizing the distance between a point and some space we are projecting on. Somebody can correct me if this is ...
user avatar
  • 1
0 votes
0 answers
44 views

Conditional Density Of Independent Bernoulli Random Variables Given Their Sum

Let Yi's be m independent Bernoulli random variables with corresponding success probabilities pi's, and let S = sum of Yi's. I am trying to figure out a way to find the given conditional probability, ...
user avatar
  • 1
1 vote
0 answers
80 views

Diagonal of the projection of a design matrix with fixed effects

I am looking to compute the diagonal entries of a projection matrix $$ P(X) = X (X' X)^{-1} X' $$ where $X$ is a design matrix that contains high dimensional fixed effects, that is, $X = [A ~~ D]$ ...
user avatar
  • 111
0 votes
1 answer
99 views

How to think about the discontinuity introduced in vector spaces by one-hot encoding?

Consider a case where you have two features: feature 1 (f1) is numerical and can take any real number, feature 2 (f2) is categorial with 3 unique values. Say we use one-hot encoding for feature 2 and ...
user avatar
  • 103
1 vote
2 answers
78 views

How can you constrain a neural network layer to simply be a n-dimensional rotation layer?

I'm looking to constrain one layer of my neural network to specifically find the best rotation of its input in order to satisfy an objective. (My end goal, where $R$ is the rotation layer, is of the ...
user avatar
  • 23
0 votes
0 answers
36 views

Does the null space of the design matrix X have an interpretation in statistics?

I am going through the material I learned in my linear algebra classes and I am trying to view the material from a statistical perspective. This typically boils down to imagining what the theorems say ...
user avatar
3 votes
1 answer
143 views

When to use dot-product as a similarity metric

I'm trying to understand which similarity measure should be used in which situations. Roughly speaking, when should someone use the dot product to assess similarity between vectors? Roughly speaking, ...
user avatar
  • 628

1
2 3 4 5
13