Skip to main content
33 votes
Accepted

What is the meaning of double bars and 2 at the bottom in ordinary least squares?

You're talking about the $\ell_2$-norm (Euclidean norm) of the vector ($Xw - y$). If this foreign to you, briefly, the $\ell_p$-norm of a vector $u \in \mathbb{R}^{n}$, is: $$\|u\|_p = \big(\sum_{i=1}...
ilanman's user avatar
  • 4,787
32 votes
Accepted

In statistics, should I assume $\log$ to mean $\log_{10}$ or the natural logarithm $\ln$?

It's safe to assume that without explicit base $\log=\ln$ in statistics, because base 10 log is not used very often in statistics. However, other posters bring up a point that $\log_{10}$ or other ...
Aksakal's user avatar
  • 61.6k
32 votes
Accepted

Dot product vs Element-wise multiplication

The difference operationally is the aggregation by summation. With the dot product, you multiply the corresponding components and add those products together. With the Hadamard product (element-wise ...
Galen's user avatar
  • 9,219
28 votes
Accepted

Why do people use $\mathcal{L}(\theta|x)$ for likelihood instead of $P(x|\theta)$?

Likelihood is a function of $\theta$, given $x$, while $P$ is a function of $x$, given $\theta$. Roughly like so (excuse the quick effort in MS paint): In this sketch we have a single $x$ as our ...
Glen_b's user avatar
  • 285k
26 votes
Accepted

How should variables in statistics be understood?

Units are often understood to be part of a random variable. If someone writes that "$h_i$ is the height of the $i$th patient ", you're meant to assume that every $i$ uses the same unit and ...
Matt Krause's user avatar
  • 21.3k
25 votes
Accepted

Why are density functions sometimes written with conditional notation?

In a Bayesian context, the parameters are random variables, so in that context the density is actually the conditional density of $X \mid (\mu, \sigma)$. In that setting, the notation is very natural. ...
angryavian's user avatar
  • 2,328
22 votes
Accepted

what is the semicolon notation in joint probability?

What this notation says is that $$ f(x; z, y, \theta) $$ is a function of $x$ with "parameters" $z, y, \theta$. It is just a way to visually show that they are of different kind (e.g. data vs ...
Tim's user avatar
  • 139k
20 votes
Accepted

Matrix notation for logistic regression

In linear regression the Maximize Likelihood Estimation (MLE) solution for estimating $x$ has the following closed form solution (assuming that A is a matrix with full column rank): $$\hat{x}_\text{...
joceratops's user avatar
20 votes
Accepted

Notation: What does the tilde below of the expectation mean?

$z\sim q$ means that RV $Z$ is distributed with respect to $q$ function, i.e. $q(z)$, where $q(z)$ is a valid PDF/PMF. So, the expectation can be unfold as (assuming $z$ being continuous) $$\mathbb{E}...
gunes's user avatar
  • 57.6k
19 votes
Accepted

How would you write mathematically that a random variable follows some unknown distribution?

The notation I tend to see is something like $X\sim F_X$ to denote that $X$ is a random variable with $F_X$ as its CDF. I have seen people try to be brief and just write $X\sim F$, but this could ...
Dave's user avatar
  • 64.2k
18 votes

Matrix notation for logistic regression

@joceratops answer focuses on the optimization problem of maximum likelihood for estimation. This is indeed a flexible approach that is amenable to many types of problems. For estimating most models, ...
AdamO's user avatar
  • 63.3k
18 votes

What does "def" above an equals sign mean?

The "def" can be read as "definition", "is defined" or "is defined to be". See for example Wikipedia, in the last sentence of this section (just before the subsection labelled "Science"): https://en....
Glen_b's user avatar
  • 285k
17 votes

In statistics, should I assume $\log$ to mean $\log_{10}$ or the natural logarithm $\ln$?

It depends. Base 10 logarithms are pretty rare in equations. However, log-scale plots are often in base-10, though this should be pretty easy to verify from the labels on the axes. In a mathematical ...
Matt Krause's user avatar
  • 21.3k
17 votes
Accepted

The origin of the Wilkinson-style notation such as (1|id) for random effects in mixed models formulae in R

The notation | has been around in nlme docs since version 3.1-1 and that is probably late 1999; we can easily check that on CRAN ...
usεr11852's user avatar
  • 44.7k
17 votes
Accepted

Meaning of vertical bar | in loss function?

It denotes that the function is parameterized by $\theta$ and the $x_i$ are the inputs to the function. For example $f(x|\theta)=x\cdot \theta$ is the dot product of the input, $x$, and the parameters ...
Sycorax's user avatar
  • 92.2k
17 votes
Accepted

How to interpret Pearl's do notation?

The proper interpretation of the do notation is that the expression $\operatorname{do}(X=1)$ means you are forcing $X$ to have the value $1.$ You are intervening to make that happen. In the Directed ...
Adrian Keister's user avatar
16 votes
Accepted

Reference: who introduced the tilde "~" notation to mean "has probability distribution..."?

Early uses There are some earlier uses since 1961 by Ingram Olkin with others. Olkin, Ingram, and Robert F. Tate. "Multivariate correlation models with mixed discrete and continuous variables.&...
Sextus Empiricus's user avatar
16 votes
Accepted

Conditional variance notation

It does condition $Y$ on $X$. But only after centering $Y$ around $E\left( Y \mid X \right)$. $$\begin{align*} \tilde Y &= \left( Y - E\left( Y \mid X = x \right) \right) ^2 \\ V(Y \mid X = x) &...
shadowtalker's user avatar
  • 12.7k
15 votes
Accepted

How to read scientific notation output (numbers that include "e")?

The "e" is a symbol for base-10 scientific notation. The "e" stands for $\times 10^{\rm exponent}$. So -1.861246e-04 means $-1.861246 \times 10^{-4}$. In fixed-point notation that would be -0....
Gordon Smyth's user avatar
  • 13.1k
15 votes

Why is the Beta Distribution Called the Beta Distribution?

Florian Cajori, in History of Mathematical Notations Vol. II (1928), wrote ... in the same paper of 1730 Euler gave what we now call the "beta function." ... About a century after Euler's first ...
whuber's user avatar
  • 327k
15 votes
Accepted

What does a subscript on a probability represent?

Maybe this helps: $P$ is the distribution of random variable $x$ given the value of random variable $y$. And this distribution has parameters $\theta$. By varying the parameters, you get different ...
Abhinav Gupta's user avatar
14 votes
Accepted

What does 1 with an inequality in the subscript mean?

$\mathbb{1}_{x\ge a}$ is an indicator function, that is equal to $1$ when $x\ge a$ and zero otherwise. Multiplying by it is a fancy, math way of saying that everything else is equal to zero. In this ...
Tim's user avatar
  • 139k
14 votes

What does a subscript on a probability represent?

Abhinav Gupta gave a nice example (+1). The general answer is that you can use the subscript to carry descriptive information about the distribution. For example, the definition of independence can be ...
Tim's user avatar
  • 139k
14 votes
Accepted

What does Φ mean?

The Phyrexian or Greek letter Φ is being used to describe the CDF (cumulative distribution function) of the standard normal. The lowercase Greek letter φ is being used to describe the PDF (probability ...
Sextus Empiricus's user avatar
14 votes
Accepted

Sample notation: When to use capital $N$ vs lowercase $n$?

I'd say the second notation is common, although evidently not universal, namely that $n$ is used for sample size and $N$ for population size. By accident, perhaps, rather than design, it matches a ...
Nick Cox's user avatar
  • 58k
13 votes
Accepted

Why $P(C=c)$ instead of simply $P(C)$

It really depends on the definition of $C$. If $C$ is defined as an event, then use $P(C)$. If $C$ is defined as a random variable that represents, say, the categories of diseases, then use $P(C = ...
Zhanxiong's user avatar
  • 20.3k
13 votes

Why are flat priors said to be proportional to a constant?

I beg to disagree with the answer given by pche8701: the main reason a flat prior is introduced (in an improper setting) as $f(\theta)\propto c$ or $f(\theta)\propto 1$ which is equivalent but more ...
Xi'an's user avatar
  • 106k
12 votes
Accepted

Why there are two different logistic loss formulation / notations?

The short version Yes Yes The long version The nice thing about mathematical modeling is that it's flexible. These are indeed equivalent loss functions, but they derive from very different ...
shadowtalker's user avatar
  • 12.7k
12 votes
Accepted

Notation for two expressions with the same distribution

Notation : $\cos(U) \stackrel d=\sin(U)$ If $X$ and $Y$ follow same distribution then mathematically you can write $X \stackrel d=Y$.
A.D's user avatar
  • 941
12 votes

Why is entropy sometimes written as a function with a random variable as its argument?

This is notation that bypasses the distribution (similar to moments) As can be seen from the formula, the entropy is fully determined by the probability distribution of the random variable, not the ...
Ben's user avatar
  • 127k

Only top scored, non community-wiki answers of a minimum length are eligible