# Tag Info

Accepted

### What is "reduced-rank regression" all about?

1. What is reduced-rank regression (RRR)? Consider multivariate multiple linear regression, i.e. regression with $p$ independent variables and $q$ dependent variables. Let $\mathbf X$ and $\mathbf Y$ ...
• 104k
Accepted

### How to perform isometric log-ratio transformation

The ILR (Isometric Log-Ratio) transformation is used in the analysis of compositional data. Any given observation is a set of positive values summing to unity, such as the proportions of chemicals in ...
• 321k
Accepted

### What's in a name: Precision (inverse of variance)

Precision is often used in Bayesian software by convention. It gained popularity because gamma distribution can be used as a conjugate prior for precision. Some say that precision is more "...
• 138k
Accepted

### What test can I use to compare slopes from two or more regression models?

To answer these questions with R code, use the following: 1. How can I test the difference between slopes? Answer: Examine the ANOVA p-value from the interaction of Petal.Width by Species, then ...
• 821
Accepted

### What is the difference between univariate and multivariate time series?

Univariate time series: Only one variable is varying over time. For example, data collected from a sensor measuring the temperature of a room every second. Therefore, each second, you will only have a ...
Accepted

### Correlated Bernoulli trials, multivariate Bernoulli distribution?

No, this is impossible whenever you have three or more coins. The case of two coins Let us first see why it works for two coins as this provides some intuition about what breaks down in the case of ...
• 378
Accepted

### categorizing a variable turns it from insignificant to significant

One possible explanation would be nonlinearities in the relationship between your outcome and the predictor. Here is a little example. We use a predictor that is uniform on $[-1,1]$. The outcome, ...
• 122k

### Why is correlation only defined between two variables?

Pearson correlation is defined as a measure of the linear relationship between two variables. For other relationships, like multidimensional relationships, we use other names. For instance: one could ...
• 75.7k
Accepted

### Why does component-wise median not make sense in higher dimensions?

The underlying concept is that a median splits the data (or a distribution) into two halves with equal amounts in each half (by count or probability). Even in one dimension the median is problematic. ...
• 321k
Accepted

### Is there a multivariate two-sample Kolmogorov–Smirnov test?

A 2004 article On a new multivariate two-sample test by Baringhaus and Franz maybe helpful, they provided a brief literature review on the two-sample multivariate GoF tests and then a R package ...
• 3,150

### Why is correlation only defined between two variables?

Correlation is defined between more than two variables, through a correlation matrix. This is not a single number of course, but that is only natural given that it is describing correlation between ...
• 123k
Accepted

### Multivariate vs Multiple time series

Multiple time series is just that: Multiple series instead of a single series. Multivariate time series is usually contrasted with univariate time series, where each observation at a time $t$ is a ...
• 11.8k

### What's in a name: Precision (inverse of variance)

Precision is one of the two natural parameters of the normal distribution. That means that if you want to combine two independent predictive distributions (as in a Generalized Linear Model), you add ...
• 15.1k
Accepted

### Discriminant analysis vs logistic regression

When the classes are well-separated, the parameter estimates for logistic regression are surprisingly unstable. Coefficients may go to infinity. LDA doesn't suffer from this problem. If there are ...
• 20.2k
Accepted

### Multivariate log-normal probabiltiy density function (PDF)

Just for the sake of completeness, I'll provide an answer here. This is a simple application of the multivariate change of variables theorem: say $Y = \Phi(X)$ where $\Phi$ is a smooth bijective ...
• 492
Accepted

### Robust covariance and OGK outlier detection

To justify the OGK, you need to see the following chain: Outlier detection and robust estimation are essentially equivalent problems, At its core, outlier detection is related to the problem of (...
• 22.5k

### In cluster analysis should I scale (standardize) my data if variables are in the same units?

Some simple and obviuos, universal considerations for multivariate analysis, including clustering. Case 1. Incomparable units. Height vs weight. You cannot compare, so the default decision is to ...
• 57.2k

### What are variable importance rankings useful for?

This is completely anecdotal, but I've found variable importance useful in identifying mistakes or weaknesses in GBMs. Variable importance gives you a kind of huge cross-sectional overview of the ...
• 1,663

### Why is correlation only defined between two variables?

Such a statistic would be hard to define and interpret. Say you have the variables $A$, $B$, and $C$. The pairwise correlation between $A$ and $B$ is close to $+1$ and the pairwise correlation between ...
• 138k

### Why is correlation only defined between two variables?

why do we only evaluate the correlation between two variables and not more than two variables? It can be more than 2 variables. Three point correlation function (3PC) is used in cosmology, The Three-...
• 2,520

### Two-dimensional Kolmogorov–Smirnov

Python implementation I have written a python implementation using numpy. You can find the code here, you may find more infomation in the docstring in the code. And here's another one (not by me). ...
• 261
Accepted

### Restricted Maximum Likelihood (REML) Estimate of Variance Component

NB. I simplify notation somewhat and do not use bold typesetting. The following rules for matrix differentials are useful: \begin{align} d\log \vert A\vert &= \mathrm{tr}(A^{-1}dA) \\ dA^{-1} &...
• 4,521
Accepted

### Correction for multiple testing in Multiple regression analysis

(NB: I started to write this before the question was edited; I don't comment here on the "empirical aspect", i.e., what is done in the literature, but rather on what "should" be done.) This is a good ...
• 22.8k
Accepted

• 12.5k
Accepted

### Confidence regions on bivariate normal distributions using $\hat{\Sigma}_{MLE}$ or $\mathbf{S}$

Assume first the the parameters $\boldsymbol\mu$ and $\boldsymbol\Sigma$ are known. Just as $\frac{x-\mu}\sigma$ is standard normal and $\frac{(x-\mu)^2}{\sigma^2}$ chi-square with 1 degree of ...
• 10.7k
Accepted

### Name of classification algorithm based on gaussian distributions estimated from data?

Probably Quadratic Discriminant Analysis. There are also names for different constraints you could make: Covariance matrices of both classes are equal - Linear Discriminant Analysis. Only diagonal ...
• 4,861
The comment seems to be a way of saying that we like large sample sizes in machine learning. The numerator is the numerator, whether you divide by $N$ or $N-1$, so all that matters to our discussion ...