16
$\begingroup$

I am a newbie to statistics and found this.

In statistics, θ, the lowercase Greek letter 'theta', is the usual name for a (vector of) parameter(s) of some general probability distribution. A common problem is to find the value(s) of theta. Notice that there isn't any meaning in naming a parameter this way. We might as well call it anything else. In fact, a lot of distributions have parameters which are usually given other names. For example, it is common use to name the mean and deviation of the normal distribution μ (read: 'mu') and deviation σ ('sigma'), respectively.

But I still don't know what that means in plain English?

$\endgroup$
  • 10
    $\begingroup$ $\theta$ is just a mathematical symbol and means different things in different contexts. Sometimes $\theta$ is used to refer to a parameter to be estimated but there is no real answer to the question "What is $\theta$?". That's like asking "What is the letter A?". Your link even hints at this when it says "Notice that there isn't any meaning in naming a parameter this way. We might as well call it anything else.". $\endgroup$ – Macro Aug 22 '12 at 22:39
  • $\begingroup$ Its just a way to name a statistical parameter(which defines the distribution of the quantity associated with this 'parameter') with a special letter (other than English letters). $\endgroup$ – Stat-R Aug 22 '12 at 23:01
  • 4
    $\begingroup$ Most of us would take this quotation to be extremely plain English, indeed, but to make any progress we have to accept that the question is not about how to read English. What, then, could it be about? I submit that it is asking us to explain the technical terms in the quotation: the ones with which we are so familiar that we no longer see how strange they might be to the statistically uninitiated. This calls for us to address the meanings of distribution and parameters (of a distribution that is; not of a fitted curve or other deterministic model). $\endgroup$ – whuber Aug 23 '12 at 13:01
31
$\begingroup$

It is not a convention, but quite often $\theta$ stands for the set of parameters of a distribution.

That was it for plain English, let's show examples instead.

Example 1. You want to study the throw of an old fashioned thumbtack (the ones with a big circular bottom). You assume that the probability that it falls point down is an unknown value that you call $\theta$. You could call a random variable $X$ and say that $X=1$ when the thumbtack falls point down and $X=0$ when it falls point up. You would write the model

$$P(X = 1) = \theta \\ P(X = 0) = 1-\theta,$$

and you would be interested in estimating $\theta$ (here, the proability that the thumbtack falls point down).

Example 2. You want to study the disintegration of a radioactive atom. Based on the literature, you know that the amount of radioactivity decreases exponentially, so you decide to model the time to disintegration with an exponential distribution. If $t$ is the time to disintegration, the model is

$$f(t) = \theta e^{-\theta t}.$$

Here $f(t)$ is a probability density, which means that the probability that the atom disintegrates in the time interval $(t, t+dt)$ is $f(t)dt$. Again, you will be interested in estimating $\theta$ (here, the disintegration rate).

Example 3. You want to study the precision of a weighing instrument. Based on the literature, you know that the measurement are Gaussian so you decide to model the weighing of a standard 1 kg object as

$$f(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp \left\{ -\left( \frac{x-\mu}{2\sigma} \right)^2\right\}.$$

Here $x$ is the measure given by the scale, $f(x)$ is the density of probability, and the parameters are $\mu$ and $\sigma$, so $\theta = (\mu, \sigma)$. The paramter $\mu$ is the target weight (the scale is biased if $\mu \neq 1$), and $\sigma$ is the standard deviation of the measure every time you weigh the object. Again, you will be interested in estimating $\theta$ (here, the bias and the imprecision of the scale).

$\endgroup$
  • 1
    $\begingroup$ +1 FWIW, I recently posted a worked example along the same lines at stats.stackexchange.com/a/34894. Although it would be misleading to construe it as "plain English"--it does not shy from using technical terms--I made an effort to explain as clearly and briefly as possible what is going on, what assumptions are made, and how one works with a parameterized family of distributions to produce an estimate based on data. For some, this might be an informative adjunct to your answer here. $\endgroup$ – whuber Aug 23 '12 at 22:20
  • 1
    $\begingroup$ Great answer! I am confused when you state the scale is biased if mu != 1, though. In fact, upon "normalizing", the standard normal distribution becomes x ~ N(0, 1). Or, in English, the mu = 0 and the variance = 1. See e.g., en.wikipedia.org/wiki/… $\endgroup$ – Mike Williamson Jan 7 '15 at 20:15
  • $\begingroup$ I just mean that the instrument has a bias if it indicates something else than 1 kg when it measures a 1 kg object. Perhaps the word "scale" is confusing. Here it just designates the instrument. $\endgroup$ – gui11aume Jan 8 '15 at 2:42
3
$\begingroup$

What $\theta$ refers to depends on what model you are working with. For example, in ordinary least squares regression, you model a dependent variable (usually called Y) as a linear combination of one or more independent variables (usually called X), getting something like

$Y_i = b_0 + b_1x_1 + b_2x_2 + ... + b_px_p$

where p is the number of independent variables. The parameters to be estimated here are the $\beta s$ and $\theta$ is a name for all the $\beta s$. But $\theta$ is more general can apply to any parameters we want to estimate.

$\endgroup$
  • 3
    $\begingroup$ Peter, although you didn't say this exactly, I'm afraid this answer may give a novice the incorrect impression that the symbol $\theta$ will always refer to a parameter vector and, conversely, that this is the only way to refer to a parameter value. As my comment above indicates, I think the answer is nothing more than "$\theta$ is a mathematical symbol", making it not really a statistical question. $\endgroup$ – Macro Aug 22 '12 at 22:42
  • 1
    $\begingroup$ @Macro I think, in this context, it's clear that this is the meaning of $\theta$ that Kamilski wanted. Sure, any symbol can refer to anything. But in this paragraph, Macro means you, and not a course in Economics or a part of SAS or whatnot. $\endgroup$ – Peter Flom Aug 22 '12 at 22:46
  • 1
    $\begingroup$ ok well I don't think that analogy is really apt but I will take it as an attempt at hyperbole. In any case, I'm really referring to something very basic which is that mathematical novices often mistake notation as something inherently meaningful and as something other than what it is - simply a label. My point was that this answer (I think unintentionally) does nothing to dispel that idea. As you know, $\theta$ can refer to other things a statistician may encounter. For example, angles are often denoted by $\theta$. $\endgroup$ – Macro Aug 22 '12 at 23:03
  • 4
    $\begingroup$ This explanation, although it is clear and technically correct, does not explicitly involve any distributions whatsoever, and thus appears not to be relevant to the quotation in the question. $\endgroup$ – whuber Aug 23 '12 at 13:03
1
$\begingroup$

In plain English:

Statistical distribution is a mathematical function $f$ that tells you what is the probability of different values of your random variable $X$ that has the distribution $f$, i.e. $f(x)$ outputs a probability of $x$. There are different such a functions, but for now let consider $f$ as some kind of "general" function.

However, for $f$ to be universal, that is, one that is possible to apply to different data (that share similar properties), it needs parameters that change its shape so that it fits different data. A simple example of such a parameter is $\mu$ in normal distribution that tells where is the center (mean) of this distribution and so it can describe random variables with different mean values. Normal distribution has another parameter $\sigma$ and other distributions also have at least one such a parameters. The parameters are often called $\theta$, where for normal distribution $\theta$ is a shorthand for both $\mu$ and $\sigma$ (i.e. is a vector of the two values).

Why is $\theta$ important? Statistical distributions are used to approximate the empirical distributions of data. Say you have dataset of ages of a group of people and on average they are 50 years old and you want to approximate the distribution of their ages using a normal distribution. If normal distribution didn't allow for different values of $\mu$ (e.g. had a fixed value of this parameter, say $\mu=0$), then it would be useless for this data. However, since $\mu$ is not fixed, normal distribution could use different values of $\mu$, with $\mu=50$ being one of them. This is a simple example, but there are more complicated cases where the values of $\theta$ parameters are not so clear and so you have to use statistical tools for estimating (finding the most appropriate) $\theta$ values.

So you could say that statistics is about finding the best $\theta$ values given the data (Bayesians would say: given the data and priors).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.