# Questions tagged [definition]

This tag indicates questions about definitions of statistical terms. Use a more general tag [terminology] for questions on statistical parlance that are not specifically about definitions.

389 questions
Filter by
Sorted by
Tagged with
222k views

### Why square the difference instead of taking the absolute value in standard deviation?

In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute ...
568k views

### What is the difference between fixed effect, random effect and mixed effect models?

In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect and mixed effect models?
18k views

### What is a data scientist?

Having recently graduated from my PhD program in statistics, I had for the last couple of months began searching for work in the field of statistics. Almost every company I considered had a job ...
40k views

### When should linear regression be called “machine learning”?

In a recent colloquium, the speaker's abstract claimed they were using machine learning. During the talk, the only thing related to machine learning was that they perform linear regression on their ...
60k views

### Maximum Likelihood Estimation (MLE) in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
13k views

### What, precisely, is a confidence interval?

I know roughly and informally what a confidence interval is. However, I can't seem to wrap my head around one rather important detail: According to Wikipedia: A confidence interval does not ...
214k views

### What are principal component scores?

What are principal component scores (PC scores, PCA scores)?
12k views

### What is meant by a “random variable”?

What do they mean when they say "random variable"?
16k views

### What is a contrast matrix?

What exactly is contrast matrix (a term, pertaining to an analysis with categorical predictors) and how exactly is contrast matrix specified? I.e. what are columns, what are rows, what are the ...
10k views

### Rigorous definition of an outlier?

People often talk about dealing with outliers in statistics. The thing that bothers me about this is that, as far as I can tell, the definition of an outlier is completely subjective. For example, ...
24k views

### What is the difference between posterior and posterior predictive distribution?

I understand what a Posterior is, but I'm not sure what the latter means? How are the 2 different? Kevin P Murphy indicated in his textbook, Machine Learning: a Probabilistic Perspective, that it is ...
12k views

### Is a time series the same as a stochastic process?

A stochastic process is a process that evolves over time, so is it really a fancier way of saying "time series"?
34k views

### What is the difference between an estimator and a statistic?

I learned that a statistic is an attribute you can obtain from samples.Taking many samples of same size, calculating this attribute for all of them and plotting the pdf, we get the distribution of the ...
2k views

### Wikipedia entry on likelihood seems ambiguous

I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.) It starts from the Wikipedia page on likelihood. They say ...
6k views

### How to describe statistics in one sentence?

When I first started learning statistics, procedures like the t-test, ANOVA, chi-squared and linear regression each appeared to be very different creatures. But now I realise these procedures each do ...
51k views

### Is variation the same as variance?

This is my first question on Cross Validated here, so please help me out even if it seems trivial :-) First of all, the question might be an outcome of language differences or perhaps me having real ...
17k views

### What is a Highest Density Region (HDR)?

In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book. One similar term is the Highest Posterior ...
38k views

### What intuitively is “bias”?

I'm struggling to grasp the concept of bias in the context of linear regression analysis. What is the mathematical definition of bias? What exactly is biased and why/how? Illustrative example?
4k views

### What is the intuition behind defining completeness in a statistic as being impossible to form an unbiased estimator of $0$ from it?

In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased ...
5k views

### What does “likelihood is only defined up to a multiplicative constant of proportionality” mean in practice?

I'm reading a paper where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners. As a likelihood example, they ...
13k views

### What is the difference between $E(X|Y)$ and $E(X|Y=y)$?

Generally, What is difference between $E(X|Y)$ and $E(X|Y=y)$? Former is function of $y$ and latter is function of $x$? It's so confusing..
106k views

### What is the difference between “margin of error” and “standard error”?

Is "margin of error" the same as "standard error"? A (simple) example to illustrate the difference would be great!
5k views

### What exactly is called “principal component” in PCA?

Suppose $u$ is the vector that maximizes the variance of the projection of the data with design matrix $X$. Now, I have seen materials that refer $u$ as the (first) principal component of the data, ...
2k views

### What's in a name: hyperparameters

So in a normal distribution, we have two parameters: mean $\mu$ and variance $\sigma^2$. In the book Pattern Recognition and Machine Learning, there suddenly appears a hyperparameter $\lambda$ in the ...
20k views

### What is the definition of a symmetric distribution?

What's the definition of a symmetric distribution? Someone told me that a random variable $X$ came from a symmetric distribution if and only if $X$ and $-X$ has the same distribution. But I think this ...
6k views

### Is “test statistic” a value or a random variable?

I am a student taking my first Statistics course now. I am confused by the term "test statistic". In the following (I saw this in some textbooks), $t$ seems to be a specific value calculated from a ...
1k views

### Is supervised learning a subset of reinforcement learning?

It seems like the definition of supervised learning is a subset of reinforcement learning, with a particular type of reward function that is based on labelled data (as opposed to other information in ...
2k views

### Tensors in neural network literature: what's the simplest definition out there?

In the neural network literature, often we encounter the word "tensor". Is it different from a vector? And from a matrix? Have you got any specific example that clarifies its definition? I'm a bit ...
2k views

### Is there more than one “median” formula?

In my work, when individuals refer to the "mean" value of a data set, they're typically referring to the arithmetic mean (i.e. "average", or "expected value"). If I ...
2k views

### Confounder - definition

According to M. Katz in his book Multivariable analysis (Section 1.2, page 6), "A confounder is associated with the risk factor and causally related to the outcome." Why must the confounder be ...
714 views

### What exactly is a distribution?

I know very little of Probability and Statistics, and am wishing to learn. I see the word "distribution" used all over the place in different contexts. For example, a discrete random variable has a "...
9k views

### What does “permutation invariant” mean in the context of neural networks doing image recognition?

I have seen a term "permutation invariant" version of the MNIST digit recognition task. What does it mean?
6k views

### How does “Fundamental Theorem of Factor Analysis” apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell). In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation ...
9k views

### What is the precise definition of a “Heywood Case”?

I had been using the term "Heywood Case" somewhat informally to refer to situations where an online, 'finite response' iteratively updated estimate of the variance became negative due to numerical ...
4k views

### Difference between the terms 'joint distribution' and 'multivariate distribution'?

I am writing about using a 'joint probability distribution' for an audience that would be more likely to understand 'multivariate distribution' so I am considering using the later. However, I do not ...
2k views

### Examples of a statistic that is not independent of sample's distribution?

This is the definition for statistic on wikipedia More formally, statistical theory defines a statistic as a function of a sample where the function itself is independent of the sample's ...
874 views

### do(x) operator meaning?

I have seen the $do(x)$ operator everywhere in some literature review I am doing on Causality (see, for instance this wikipedia entry). However, I cannot find a formal and general definition of this ...
14k views

### Do you reject the null hypothesis when $p < \alpha$ or $p \leq \alpha$?

This is clearly just a matter of definition or convention, and of almost no practical importance. If $\alpha$ is set to its traditional value of 0.05, is a $p$ value of 0.0500000000000... considered ...
83k views

### What is the difference between effectiveness and efficacy in determining the benefit of therapy 'A' on condition 'B'?

The context of this question is within a health framework i.e. looking at one or more therapies in the treatment of a condition. It appears that even well respected researchers confuse the terms ...
1k views

### Definition and delimitation of regression model

An embarrassingly simple question -- but it seems it has not been asked on Cross Validated before: What is the definition of a regression model? Also a support question, What is not a regression ...
4k views

### What is Thompson Sampling in layman's terms?

I am unable to understand Thompson Sampling and how it works. I was reading about Multi Arm Bandit and after reading Upper Confidence Bound Algorithm, many text suggested that Thompson Sampling ...
12k views

### What is the difference between univariate and multivariate time series?

I have the following time series data: ...
1k views

### What are Regularities and Regularization?

I am hearing these words more and more as I study machine learning. In fact, some people have won Fields medal working on regularities of equations. So, I guess this is a term that carries itself from ...
15k views

### What does “curvilinear” mean?

As far as I can tell, curvilinear is defined vaguely but means the same as nonlinear. Is that correct? Or does curvilinear have a distinct definition?
3k views

### What do we mean by hyperparameters? [duplicate]

Can anyone give me full details about what we mean by hyperparameters, and what in the Dirichlet distribution are called hyperparameters? A practice example for the estimation of those parameters ...
10k views

### Can the likelihood take values outside of the range [0, 1]? [duplicate]

I got a log-likelihood value of -34.82, so I am not getting whether the answer which I have got is right or not. Can the likelihood take values outside of the range $[0, 1]$?
2k views

### Is the linearity assumption in linear regression merely a definition of $\epsilon$?

I am revising linear regression. The textbook by Greene states: Now, of course there will be other assumptions on the linear regression model, such as $E(\epsilon|X)=0$. This assumption ...
2k views

### Definition of “percentile”

I'm now reading a note on Biostatistics written by PMT Education, and notice the following sentences in Section 2.7: A baby born at the 50th percentile for mass is heavier than 50% of babies. A ...