Questions tagged [definition]

This tag indicates questions about definitions of statistical terms. Use a more general tag [terminology] for questions on statistical parlance that are not specifically about definitions.

Filter by
Sorted by
Tagged with
444
votes
22answers
222k views

Why square the difference instead of taking the absolute value in standard deviation?

In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute ...
303
votes
9answers
568k views

What is the difference between fixed effect, random effect and mixed effect models?

In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect and mixed effect models?
186
votes
15answers
18k views

What is a data scientist?

Having recently graduated from my PhD program in statistics, I had for the last couple of months began searching for work in the field of statistics. Almost every company I considered had a job ...
101
votes
11answers
40k views

When should linear regression be called “machine learning”?

In a recent colloquium, the speaker's abstract claimed they were using machine learning. During the talk, the only thing related to machine learning was that they perform linear regression on their ...
95
votes
12answers
60k views

Maximum Likelihood Estimation (MLE) in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
90
votes
10answers
13k views

What, precisely, is a confidence interval?

I know roughly and informally what a confidence interval is. However, I can't seem to wrap my head around one rather important detail: According to Wikipedia: A confidence interval does not ...
80
votes
7answers
214k views

What are principal component scores?

What are principal component scores (PC scores, PCA scores)?
79
votes
8answers
12k views

What is meant by a “random variable”?

What do they mean when they say "random variable"?
52
votes
4answers
16k views

What is a contrast matrix?

What exactly is contrast matrix (a term, pertaining to an analysis with categorical predictors) and how exactly is contrast matrix specified? I.e. what are columns, what are rows, what are the ...
44
votes
8answers
10k views

Rigorous definition of an outlier?

People often talk about dealing with outliers in statistics. The thing that bothers me about this is that, as far as I can tell, the definition of an outlier is completely subjective. For example, ...
38
votes
3answers
24k views

What is the difference between posterior and posterior predictive distribution?

I understand what a Posterior is, but I'm not sure what the latter means? How are the 2 different? Kevin P Murphy indicated in his textbook, Machine Learning: a Probabilistic Perspective, that it is ...
32
votes
5answers
12k views

Is a time series the same as a stochastic process?

A stochastic process is a process that evolves over time, so is it really a fancier way of saying "time series"?
31
votes
9answers
34k views

What is the difference between an estimator and a statistic?

I learned that a statistic is an attribute you can obtain from samples.Taking many samples of same size, calculating this attribute for all of them and plotting the pdf, we get the distribution of the ...
30
votes
5answers
2k views

Wikipedia entry on likelihood seems ambiguous

I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.) It starts from the Wikipedia page on likelihood. They say ...
28
votes
18answers
6k views

How to describe statistics in one sentence?

When I first started learning statistics, procedures like the t-test, ANOVA, chi-squared and linear regression each appeared to be very different creatures. But now I realise these procedures each do ...
28
votes
2answers
51k views

Is variation the same as variance?

This is my first question on Cross Validated here, so please help me out even if it seems trivial :-) First of all, the question might be an outcome of language differences or perhaps me having real ...
25
votes
3answers
17k views

What is a Highest Density Region (HDR)?

In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book. One similar term is the Highest Posterior ...
23
votes
3answers
38k views

What intuitively is “bias”?

I'm struggling to grasp the concept of bias in the context of linear regression analysis. What is the mathematical definition of bias? What exactly is biased and why/how? Illustrative example?
22
votes
2answers
4k views

What is the intuition behind defining completeness in a statistic as being impossible to form an unbiased estimator of $0$ from it?

In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased ...
20
votes
5answers
5k views

What does “likelihood is only defined up to a multiplicative constant of proportionality” mean in practice?

I'm reading a paper where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners. As a likelihood example, they ...
20
votes
4answers
13k views

What is the difference between $E(X|Y)$ and $E(X|Y=y)$?

Generally, What is difference between $E(X|Y)$ and $E(X|Y=y)$? Former is function of $y$ and latter is function of $x$? It's so confusing..
20
votes
5answers
106k views

What is the difference between “margin of error” and “standard error”?

Is "margin of error" the same as "standard error"? A (simple) example to illustrate the difference would be great!
20
votes
1answer
5k views

What exactly is called “principal component” in PCA?

Suppose $u$ is the vector that maximizes the variance of the projection of the data with design matrix $X$. Now, I have seen materials that refer $u$ as the (first) principal component of the data, ...
19
votes
5answers
2k views

What's in a name: hyperparameters

So in a normal distribution, we have two parameters: mean $\mu$ and variance $\sigma^2$. In the book Pattern Recognition and Machine Learning, there suddenly appears a hyperparameter $\lambda$ in the ...
19
votes
2answers
20k views

What is the definition of a symmetric distribution?

What's the definition of a symmetric distribution? Someone told me that a random variable $X$ came from a symmetric distribution if and only if $X$ and $-X$ has the same distribution. But I think this ...
18
votes
3answers
6k views

Is “test statistic” a value or a random variable?

I am a student taking my first Statistics course now. I am confused by the term "test statistic". In the following (I saw this in some textbooks), $t$ seems to be a specific value calculated from a ...
17
votes
1answer
1k views

Is supervised learning a subset of reinforcement learning?

It seems like the definition of supervised learning is a subset of reinforcement learning, with a particular type of reward function that is based on labelled data (as opposed to other information in ...
17
votes
1answer
2k views

Tensors in neural network literature: what's the simplest definition out there?

In the neural network literature, often we encounter the word "tensor". Is it different from a vector? And from a matrix? Have you got any specific example that clarifies its definition? I'm a bit ...
16
votes
3answers
2k views

Is there more than one “median” formula?

In my work, when individuals refer to the "mean" value of a data set, they're typically referring to the arithmetic mean (i.e. "average", or "expected value"). If I ...
16
votes
1answer
2k views

Confounder - definition

According to M. Katz in his book Multivariable analysis (Section 1.2, page 6), "A confounder is associated with the risk factor and causally related to the outcome." Why must the confounder be ...
16
votes
3answers
714 views

What exactly is a distribution?

I know very little of Probability and Statistics, and am wishing to learn. I see the word "distribution" used all over the place in different contexts. For example, a discrete random variable has a "...
15
votes
2answers
9k views

What does “permutation invariant” mean in the context of neural networks doing image recognition?

I have seen a term "permutation invariant" version of the MNIST digit recognition task. What does it mean?
15
votes
1answer
6k views

How does “Fundamental Theorem of Factor Analysis” apply to PCA, or how are PCA loadings defined?

I'm currently going through a slide set I have for "factor analysis" (PCA as far as I can tell). In it, the "fundamental theorem of factor analysis" is derived which claims that the correlation ...
15
votes
1answer
9k views

What is the precise definition of a “Heywood Case”?

I had been using the term "Heywood Case" somewhat informally to refer to situations where an online, 'finite response' iteratively updated estimate of the variance became negative due to numerical ...
15
votes
5answers
4k views

Difference between the terms 'joint distribution' and 'multivariate distribution'?

I am writing about using a 'joint probability distribution' for an audience that would be more likely to understand 'multivariate distribution' so I am considering using the later. However, I do not ...
14
votes
2answers
2k views

Examples of a statistic that is not independent of sample's distribution?

This is the definition for statistic on wikipedia More formally, statistical theory defines a statistic as a function of a sample where the function itself is independent of the sample's ...
14
votes
2answers
874 views

do(x) operator meaning?

I have seen the $do(x)$ operator everywhere in some literature review I am doing on Causality (see, for instance this wikipedia entry). However, I cannot find a formal and general definition of this ...
14
votes
2answers
14k views

Do you reject the null hypothesis when $p < \alpha$ or $p \leq \alpha$?

This is clearly just a matter of definition or convention, and of almost no practical importance. If $\alpha$ is set to its traditional value of 0.05, is a $p$ value of 0.0500000000000... considered ...
14
votes
6answers
83k views

What is the difference between effectiveness and efficacy in determining the benefit of therapy 'A' on condition 'B'?

The context of this question is within a health framework i.e. looking at one or more therapies in the treatment of a condition. It appears that even well respected researchers confuse the terms ...
14
votes
3answers
1k views

Definition and delimitation of regression model

An embarrassingly simple question -- but it seems it has not been asked on Cross Validated before: What is the definition of a regression model? Also a support question, What is not a regression ...
14
votes
2answers
4k views

What is Thompson Sampling in layman's terms?

I am unable to understand Thompson Sampling and how it works. I was reading about Multi Arm Bandit and after reading Upper Confidence Bound Algorithm, many text suggested that Thompson Sampling ...
13
votes
1answer
12k views

What is the difference between univariate and multivariate time series?

I have the following time series data: ...
13
votes
2answers
1k views

What are Regularities and Regularization?

I am hearing these words more and more as I study machine learning. In fact, some people have won Fields medal working on regularities of equations. So, I guess this is a term that carries itself from ...
12
votes
4answers
15k views

What does “curvilinear” mean?

As far as I can tell, curvilinear is defined vaguely but means the same as nonlinear. Is that correct? Or does curvilinear have a distinct definition?
12
votes
1answer
3k views

What do we mean by hyperparameters? [duplicate]

Can anyone give me full details about what we mean by hyperparameters, and what in the Dirichlet distribution are called hyperparameters? A practice example for the estimation of those parameters ...
12
votes
2answers
10k views

Can the likelihood take values outside of the range [0, 1]? [duplicate]

I got a log-likelihood value of -34.82, so I am not getting whether the answer which I have got is right or not. Can the likelihood take values outside of the range $[0, 1]$?
12
votes
3answers
2k views

Is the linearity assumption in linear regression merely a definition of $\epsilon$?

I am revising linear regression. The textbook by Greene states: Now, of course there will be other assumptions on the linear regression model, such as $E(\epsilon|X)=0$. This assumption ...
11
votes
2answers
2k views

Definition of “percentile”

I'm now reading a note on Biostatistics written by PMT Education, and notice the following sentences in Section 2.7: A baby born at the 50th percentile for mass is heavier than 50% of babies. A ...
11
votes
2answers
1k views

Is population size a parameter, or sample size a statistic?

The definitions of a parameter and statistic pretty much agree: parameters and statistics are numerical characteristics or numerical summaries of a population and sample, respectively, for a given ...
11
votes
2answers
1k views

Mathematical definition of Infill Asymptotics

I am writing a paper that uses infill asymptotics and one of my reviewers has asked me to please provide a rigorous mathematical definition of what infill asymptotics is (i.e., with math symbols and ...

1
2 3 4 5
8