microhaus
  • Member for 1 year, 5 months
  • Last seen this week
Posterior of factors in factor analysis
Accepted answer
1 votes

The derivation relies on a result known as the matrix inversion lemma, or Woodbury matrix identity. From wikipedia: $$(A + UCV)^{-1} = A^{-1} - A^{-1}U(C^{-1} + VA^{-1}U)^{-1}VA^{-1}$$ Identifying $...

View answer
How to adapt a linear time Newton-Raphson numerical method for an optimisation problem with positivity constraints?
1 votes

The class of methods devised to solve this generic problem are known in the optimisation literature as projected Newton methods. The following is extracted from Bertsekas and Gafni (1983), with some ...

View answer
Gradient of Log Normalizing Constant - Does it have a name and do we know of any properties?
Accepted answer
1 votes

This answer assumes that you are interested in computing $\nabla_{\theta} \log Z$ rather than $\nabla_{x} \log Z$. Without foreclosing the possibility that there may exist other results relevant to ...

View answer
Relationship between variance of gradient and SGD convergence
Accepted answer
1 votes

Only in response to: I'm looking for a reference that talks about the relationship between the variance of the gradient and convergence of SGD. Whilst the Robbins-Monro stochastic approximation ...

View answer
Latent Dirichlet Allocation and topic distributions
Accepted answer
1 votes

I think it would be helpful in this instance to draw a distinction between: 1. How the hyperparametrisation encodes prior beliefs about the topic vectors $\phi_k$ sampled from the Dirichlet prior $p(\...

View answer
Kelly Criterion for game with payoff equal to normal distribution
1 votes

Some references. To supplement the comments of @Dave Harris, here are a cluster of references you might consider using to start further formalising what you've done already. Depending on your ...

View answer
Is my calculation for the MLE correct? How do I check whether it's biased?
Accepted answer
1 votes

In response to: So how do I find $E(x^2_i)$? Assuming there are no issues with the steps in the derivation up to that point, you can use the following standard result relating expectation, variance ...

View answer
Meaning of a varaible for calculating the partial derviative of MSE cost function
Accepted answer
1 votes

From the context you have provided, my reading is that $x^{(i)}_j$ is the $j$-th element of the $i$-th input vector $\mathbf{x}^{(i)}$, where there are $i = 1,..., m$ training instances. Addressing ...

View answer
Show that the maximum of $x_1,...,x_n \sim \mathrm{Uniform}(0,\theta)$ is a sufficient statistic for $\theta$. (From definition)
1 votes

Extended comment. I am somewhat confused by what it is you are seeking an answer for, and also by your workings. On the workings: Assuming that by $f(x^n; \theta)$ you refer to the likelihood function ...

View answer
Bounding the uniform deviation of the empirical risk from the risk over a finite function class
Accepted answer
1 votes

The statement the authors intended is (A) for the following reasons. Addressing the confusion. Much of your reluctance to consider $(A)$ as the correct statement amounted to insufficient attentiveness ...

View answer
Does Adaboost ensemble use bootstrapping?
1 votes

You understanding is completely correct. Starting at step 1, you initialise uniform weights $w_i = 1 / N$. Beginning at step 2, in say iteration $m = 1$, you sample a bootstrap dataset $\mathcal{B}...

View answer
Learning resources for Bayesian Dynamic Networks?
Accepted answer
1 votes

The following lecture video, "L10 - Gaussian graphical models and Ising models", here of the course CMU Probabilistic Graphical Models Spring 2019 has a small section towards the end about ...

View answer
Explain the expected value of an indicator variable in section 9.3.1 of PRML
Accepted answer
1 votes

Typo. There is a typo in the 2006 version of Bishop's PRML, which has been corrected in later versions. Instead, equation (9.39) on p443 should read "the expectation of the indicator variable $z_{...

View answer
Latent Dirichlet Allocation - dimensionality of the Dirichlet prior parameter
Accepted answer
1 votes

The question you raised is down to confusion concerning whether or not the $\beta_{k, \space :}$ is drawn from an identically parametrised Dirichlet distribution, or whether each $\beta_{k, \space :}$ ...

View answer
Smoothed CDF to calculate asymptotic normality
0 votes

Fix $z \in \mathbb{R}$ to be some value. Now for illustrative purposes, define corresponding i.i.d. random variables $Y_i = g(Z_i) = 1\{Z_i \leq z\}$. The following prompts should hopefully assist in ...

View answer
Bishop equation 3.67
0 votes

\begin{align} p(t \vert \mathbf{x}, \mathcal{D}) &= \sum^L_{i=1}p(t, \mathcal{M}_i \vert \mathbf{x}, \mathcal{D}) \tag{marginalisation} \\ &= \sum^L_{i=1} p(t \vert \mathbf{x}, \mathcal{M_i}, \...

View answer
Computing the Bayes estimator under weighted squared error loss - interchanging derivatives and integrals
Accepted answer
0 votes

Solution. Using the results in here linked by StubbornAtom. The loss function you have specified can be interpreted as a generalisation of squared error loss, known as weighted squared error loss. ...

View answer
Showing that $f_\varphi(x)$ is a member of the one-parameter exponential family and $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$
0 votes

Here is my attempt. 1.Show that $f(x; \varphi)$ is in the one-parameter exponential family. Assume that $\varphi > 0$. Rewriting $f(x; \varphi)$, we have that \begin{align*} f(x; \varphi) &= \...

View answer
Likelihood $L(\theta; \mathbf{y})$: Is $\theta$ a vector of parameters or is it a single parameter?
Accepted answer
0 votes

It depends on the nature of the assumptions on your $n$ random variables $Y_1, ..., Y_n$, which in turn will influence the kind of parametric statistical model that is specified. Here is a coin ...

View answer
Relationship between Bayes Rule and Bayesian Networks
0 votes

In response to I did not fully understand why we are multiplying the intermediate conditionals i.e.: $...Pr(Burglar|Storm=T)xPr(Cat|Storm=T)...$ and Is there a logical explanation for the ...

View answer
Meaning of θj in equation for partial derivative of MSE
Accepted answer
0 votes

In response to question and also following comment: Sorry, but could I ask a small follow-up question? If θj = 5 then should the jth element in θ also be 5? E.g - If on the left hand side θj= θ3 = 5 ...

View answer
Question on loss function notation in Elements of Statistical Learning II
0 votes

The use of the index $l$ is just a 'dummy indexing variable' - it is used to avoid confusion with the fact that the indexing variable $k$ has already been used in the numerator. You generally tend to ...

View answer
Question about the expression for the generative model in latent Dirichlet allocation (LDA)
0 votes

All that is going on here is a notational trick - using Iverson brackets to index or "pick out" the appropriate element/row from a vector/matrix of parameters on Multinomial distributions. ...

View answer
Interpretation of multivariate conditional gaussian function form?
0 votes

I decided to do a little digging after posting the comment above (I preferred not to spend time re-crunching through the algebra again) and thought I'd post what I found here. So this is far from a ...

View answer
1
2