New answers tagged probability
0
Likelihood and entropy measure different things. The former tells us how likely it is to observe the dataset $X$ given the parameters $\theta$. The latter is supposed to tell us how much uncertainty is in the distribution. So, comparing these two metrics is like comparing weight and hair length of a student: can be done, but somewhat pointless.
Let us ...
4
You have two questions so I'll answer them in turn
(1) Why are the maxima Gumbel distributed?
Essentially, this comes from the extremal types theorem which is a limiting distribution for block maxima. The most common limits distribution is the CLT which states
$$ \frac{\bar{X} - \mu}{\sigma} \to N(0,1) \text{ as } n \to \infty$$.
Now suppose we have data ...
5
Let $X_1,...,X_m \sim \text{SRSWOR}(n)$ denote the sampled values. For simplicity, suppose we select an odd number of values $m=2k+1$ from the distribution, so the median is the order statistic $X_{(k+1)}$. To facilitate our analysis, let $R(x) = \sum_{i=1}^m \mathbb{I}(X_i \leqslant x)$ denote the number of sample values that are no greater than $x$. ...
0
Yes. There are few proposed measures of calibration, but the most widely circulated is the so called Hosmer-Lemeshow statistic. At a 10,000 view, this test works by binning a risk model into deciles, and then calculating the expected event frequency versus the observed event frequency within each decile. The "x"-axis here is the average predicted ...
2
For a more brute force approach, consider the following. You have three numbers, you know the last one is 6. There are $6 \cdot 6=36$ possible combinations for the first two numbers (assuming they can be any number from 1 to 6, which your problem seems to state that they cannot). Asking that the sum of these three numbers is $<10$ is equivalent to asking ...
4
It is a bit difficult to point out where the mistake is because your solution is "too informal". Is the random variable $sum$ the sum of the first three dice or the sum of all dices? Depending on the definition, your probabilities change. Moreover, if $X=1$, then $sum$ might not even be defined if you consider the first definition (the sum of the ...
1
When we look at the sum of dice thrown, there's no importance if we got $(1,2,3),(1,4,1),(3,1,2)$ or any other permutation that gives us 6. This is called unordered sampling with replacement. "Unordered" because the order of results doesn't matter; "with replacement" because a result can occur more than once.
The number of possible ...
0
When calculating Probability#1, you say:
we'll have 1/3 chance that our number is higher than the third as well
This is not true since you proceed to this step only when your number is higher than a number in other envelope selected at the begining. So in this step you compare maximum from envelopes selected at the begining with the last envelope. So you ...
1
If we assume that all bitstrings are equally likely then the problem is quite simple. Each matched pair of bits in the bitstrings will match with fixed probability $\tfrac{1}{2}$ and the number of matches has distribution:
$$M \sim \text{Bin}(n, \tfrac{1}{2}).$$
Consequently, the probability of at least $n/2$ matches (for even $n$) is:
$$\begin{align}
\...
2
Let's elucidate this issue using a Venn diagram for the three sets at issue. Let $A$ be the orange circle, let $B$ be the violet circle and let $C$ be the green circle, and denote the probabilities of the intersection-areas in the diagram by $P_1,...,P_7$ (ignoring the area outside the three sets).
From this diagram we can see that:
$$\begin{align}
\mathbb{...
9
Regression analysis accommodates situations where the explanatory variables/covariates can have any values, including zero. There is no particular model needed for this --- it can be implemented in almost any model. If there are lots of zeros for a particular covariate, the only issue this creates is that it affects the leverage of the data points and one ...
3
Let's think about this for a moment. Continuous state space means that there are infinitely many states linked via transition probabilities such that they are all accessible by traversing a single graph (let's suppose). The expected values in such a case are asymptotic, meaning that they either have a cleverly constructed closed form (such as mean of a ...
2
The difference is like between the value of a function $f(x)$ given its argument $x$ and the definition of a function $f()$.
You can think of a random value (rv) as a mapping from events to real numbers, quite similar to mapping that a typical function $f(x)$ represents from $x$ to some numbers. In case of a rv the argument is an event (or a set of events), ...
3
The realization of a random variable is the value that was observed (though, as noticed in the comments, you can have random variables for non-observable things). For example, you treat the result of throwing a fair dice as a random variable $X$. Say that the result is five dots, $x=5$ is the realization. The “five objects” that you call “realizations” are ...
0
By using law of total expectation:
$$\mathbb{E}[e^{tXY}] = \mathbb{E}[\mathbb{E}[e^{tXY}|Y] = \mathbb{E}[e^{tXY}|Y=0]P(Y=0) + \mathbb{E}[e^{tXY}|Y=1]P(Y=1) = \mathbb{E}[e^{0}]0.7 + \mathbb{E}[e^{tX}]0.3 = 0.7 + 0.3M_{X}(t)$$
3
There's a simpler way to tackle it, using the Law of Iterated Expectations. For notational clarity, I will define $y$ as "future $x$", so your first line would be $\mathbb{E}(y|x) = \dots$. Writing out the Law as applied to this case gives us:
$$\mathbb{E}[y|x] = \mathbb{E}_{\theta | x} \mathbb{E}[y|\theta]$$
We have $\mathbb{E}[y|\theta] = n\...
0
To follow up on what @Tim said, Let me give some specific examples.
Optimizing for marginal revenue of investment: cut at the first edge that accounts for X% of revenue increase. X is decided by the amount required to increase head count in our department in next quarter, cause HR has asked for support evidence. We will make a case to hire new people to ...
1
For a, your answer looks fine.
For b, you need to think about possible factorization from the primes of XY.
Let's say XY = 12, for all pairs of X and Y whose product is 12, namely, 1 and 12, 2 and 6, 3 and 4, the process is still memoryless. This is the result in part a, stating all possible pairing retains memorylessness.
3
If you need to bin the predictions for business reasons, you probably want to do something like treating the bins differently, e.g. assigning them to different marketing campaigns. In such a case, you have in mind optimizing some objective (money, clicks, subscriptions, churn, etc). What you need to do is find such bins that are optimal given the objective. ...
0
Tracking links to previous "similar" problems, one finds
several over the last four years--some on this site and
some on the physics site. Unfortunately, none of them
is clearly stated and several mention more than one
probability model. So this "Answer" of mine has to be speculative.
Suppose the proportion $p_i$ of particles in state $i$
...
0
Looking at the ELBO equation in Andrew Ng's course, we have a model p representing the data generation process from latent to observable, and an approximation of this model with Q. The ELBO aims to optimise a function such that Q is as close to p as possible, thus giving Q a good fit of p.
This is what the argmax in the M-step finds, the distance here is a ...
0
There is a difference. If we denote the probability vector $\pi$, then the first case of 10,000 repetitions of $X \sim \text{Mult}(1, \pi)$ retains information about the $X_i$ in each of the 10,000 repetitions.
The second case of one repetition of $Y \sim \text{Mult}(10000, \pi)$ does not retain information about the $X_i$ in each of the 10,000 trials.
Here ...
0
To estimate the total number of cases sold we used this following formula:
$\hat{t}_{unb}=\frac{N}{n} \sum_{i \in \mathcal{S}} \hat{t}_{i}$
$\hat{t}_{unb}=\frac{45}{6} 20396.3$
$\hat{t}_{unb}=152972.088$
For the Standard Error we have:
$$
\begin{array}{l}
\begin{aligned}
\\
S \cdot E\left(\hat{t}_{\text {unb }}\right) &=\sqrt{V\left(\hat{t}...
3
scipy.stats.mean will give you the mean $\bar X$. Depending on which definition of the geometric distribution you use, the parameter $p$ is estimated either by $1/\bar X$ or $1/(1+\bar X)$
1
Have a look at the class-conditional densities in the left plot. There is some information to be taken from it:
$p(x|C_1)$ is bimodal, with a peak at x=0.2 and one at x=0.5, separated by a "valley" at roughly x=0.3
$p(x|C_2)$ is unimodal and symmetric, with it's center at roughly x=0.7
$p(x|C_1)$ and $p(x|C_2)$ intersect near x=0.58
$p(x|C_1)$ is ...
0
One way you could tackle this problem is to assume the family of distributions of the transition probability and fit it to the data. For example, you can assume $y_{t-1} \sim \mathcal{N}(y_{t}, \sigma^{2})$ and learn the variance/standard deviation using Maximum Likelihood. The important thing is by the Markov condition (I assume of first-order), any data ...
1
I would say this is a matter of terminology. What people in causality mean when they talk about mechanisms (also known as Independent Causal Mechanisms, causal Markov kernels, causal conditionals, etc.) are the "true" causal conditional distributions. In your case, you have only one mechanism $P(B \mid A)$.
As you correctly pointed out, the ...
4
Disclosure: I wrote the samc R package used in this answer
This answer is more of a supplement to Stephan Kolassa's answer
In it, he showed how to construct a transition matrix representing the problem:
Image credit: Stephan Kolassa's answer
Now, as indicated in comments/another answer, this matrix can be simplified a variety of different ways, but I'm ...
1
Seems very close to Shannon - related theorems. If you posit "HTH" as your "end of message" string, you want to estimate the chance that "HTH" shows up in random data. I suspect a little digging into his work will provide the equations/ formulas of interest.
And because I can't resist, "HTH"
1
linear or any n-dimensional hyperplane cuts the data so it minimizes the MSE (or any other metric used to measure distance). However if you have a probability distribution (including multi-dimensional distributions), it provides the location of the cluster and the likelihood of finding a sample in a specific region.
If the data is clustered, then a ...
1
The main challenge in this kind of question is whether you can identify a hard upper limit. If so, you can use a rescaled beta distribution (assuming there is also a hard lower limit), or possibly a flipped-and-shifted distribution like the gamma or the negative binomial (assuming there is no lower limit), all possibly with some zero inflation (which would ...
2
If possible I would show a plot of the distribution, rather than just some summary statistics, since it is more informative. Alternatively, if some summary statistics are required, I would use a five-number summary or similar, rather than trying to find a distribution which fits the data. Fitting a distribution might look like a more elegant solution, since ...
1
Mixture distributions make the most sense if we have grounds to suspect that the data belong to multiple different subpopulations, as per the textbook you cite. Usually, we don't know which of the subpopulation a given instance belongs to - and we may not even know how many such subpopulations exist.
As such, we could even consider straightforward ANOVA as a ...
2
It's always hard to prove a negative. However, I have never seen such an interpretation. The naive idea that the weighted sum of quantiles would be the corresponding quantile of the mixture distribution is of course false.
I would be interested in being proven wrong and look forward to downvotes (and actual such interpretations).
2
The chance of A being $>2\sigma$ from the mean is $2.28\%$.
pnorm(12, 10, 1, lower.tail = FALSE)
Thus the chance of A & B (2 independent events both occurring) is equal to $A*B$ or $0.052\%$ of the time.
Therefore expected frequency is once every 1932 times (agrees with the simulation results reasonably well).
The medium and mode is a bit more ...
12
There is a fun way to answer this problem using martingales, and in particular using https://en.wikipedia.org/wiki/Optional_stopping_theorem. I first saw this trick in the book A First Look at Rigorous Probability Theory by Jeffrey S. Rosenthal, in the martingale chapter. (I don't have the book in front of me at the moment but I'll edit and add a page or ...
1
Final formula appears to be
sum(i = 1 to length(pattern) : if (first i flips of pattern match the last i flips of pattern) then P(series of i flips matches first i flips of pattern)^-1 else 0)
(For a fair coin, this reduces to Conway's algorithm; it can probably be proven by a similar method.)
31
At any given point in the game, you're $3$ or fewer "perfect flips" away from winning.
For example, suppose you've flipped the following sequence so far:
$$
HTTHHHTTTTTTH
$$
You haven't won yet, but you could win in two more flips if those two flips are $TH$. In other words, your last flip was $H$ so you have made "one flip" worth of ...
17
First, you can refactor your R code to be (IMHO) a little more legible, also using pbapply::pbreplicate() to get a nice progress bar:
n_sims <- 1e5
library(pbapply)
results <- pbreplicate(n_sims,{
flips <- NULL
while(length(flips)<3 || !identical(tail(flips,3),c("H","T","H"))){
flips <- c(flips,...
7
Here is a somewhat clumsy brute-force method to obtain the probabilities and order statistics. Getting the mean will take more work.
So first just generate the possible sequences and associated probabilities where "HTH" are the last 3 flips (with that sequence not occurring previously). Then look for patterns. For integer patterns the go to ...
1
I can propose something, the idea is to use an optimization method based on the KDE function. Here is a short example in R:
First I define a function to calculate a KDE (most of this code in taken from a personnal project on github (JeremyGelb/spNetwork)
quartic_kernel <- function(d, bw){
u <- d/bw
k <- (15/16)*(1-u**2)**2
k <- k / bw
k &...
0
I do not know how to use the factorization theorem
You have to write the likelihood as a product of two functions. One function is allowed to have parameters in it, and one is not. The one with the parameter in it will have the sufficient statistic in it.
Also, How can I express this density function as an exponential
family? I have problems with the ...
0
I think you substituted $\exp(-yf(x))$ as the logistic loss, not $\log(1 + \exp(-yf(x))$. I get the expected answer when I substitute the latter term.
Feel free to take a crack at that yourself substituting in that term instead -- otherwise, I've provided the derivation below.
Let $\eta \triangleq \mathbf{Pr}[Y = 1 \mid \mathbf{x}]$, and let $z \triangleq f(\...
1
$k$ refers to the possible values for Outlook, because you'll need the sum of all Outlook probabilities be $1$. It's up to you to apply this to all features or not. The smoothing is useful for remedying zero probability cases, but it's also for applying regularization. The degree/strength of regularization depends on the value of $\alpha$ and can be chosen ...
0
Let us have n men corresponding to which there exist n hats. A match is said to occur if a man wears his own hat.
A Match occurs if
$Man_{i}$ wears $Hat_{i}$ , $Man_{j}$ wears $Hat_{j}$. Let us denote this event as $E_{i}$ .
A NO Match occurs if
$Man_{i}$ wears $Hat_{j}$ , $Man_{j}$ wears $Hat_{i}$. Let us denote this event as $\overline{E_{i}}$ .
...
2
It is indeed the inverse of $1-p$. The column is mislabelled, since $p$ is already defined.
They are writing $1-p$ as a fraction $1/M$, and telling you $M$. It can get hard to compare very small fractions intuitively, but writing them as $1$ in $M$ can be more intuitive (e.g. "how many times would we need to repeat this sampling from a normal, to expect ...
3
There is no difference. That you get slightly different results, even with the same random seed, must be because somewhat different algorithms are used in the two cases.
As an illustration, think about throwing coins (well, three-sided coins ...). Throwing 10000 coins once, or throwing one coin 10000 times, you will expect the same results.
1
If $t_1 = t_2 = t$
Below you see an example with the distribution of
$$x \sim Poisson(17460) \qquad y \sim Poisson(18000)$$
You can make several approximations
The standard error of $y-x$ will be $\sigma_{y-x} = \sqrt{x+y}$ and you can see that the $\pm 5 \sigma$ boundary is roughly a straight line.
You could approximate the $y-x$ as normal distributed.
...
1
The correct statement is that, for any probability cumulative distribution $F$ (in dimension one), defining the generalised inverse as
$$F^-(u)=\inf\{x;~F(x)\ge u\}$$
the random variable$$X=F^-(U)\sim F\tag{1}$$ meaning it is distributed from the distribution associated with $F$.
When $F$ is continuous and strictly increasing, this implies that
$$F(X)\sim\...
2
What happened in the future variables clues what had happened in the past, so the assumption $p(x_1|x_2)=p(x_1)$ is not correct. You should use Bayes theorem to calculate it:
$$p(x_1|x_2)=\frac{p(x_2|x_1)p(x_1)}{p(x_2)}$$
You can calculate $p(x_i)$ from total probability theorem, or alternatively do the same calculations using some matrix multiplications as ...
Top 50 recent answers are included