Taylor
  • Member for 10 years
  • Last seen this week
Testing significance of correlation between time series
Accepted answer
3 votes

Bartlett's theorem is useful for this. If $\{\mathbf{X}_{t}\}$ is a bivariate time series whose components are defined by $$ X_{t1} = \sum_{k=-\infty}^{\infty} \alpha_k Z_{t-k,1}, \hspace{10mm} \{Z_{...

View answer
Cramér–Rao bound to multiple parameters
Accepted answer
3 votes

$\displaystyle {\boldsymbol {T}}(X)$ is a vector of estimators and $\displaystyle {\boldsymbol{\psi}}({\boldsymbol {\theta}})$ is its expectation. That is $$ \mathbb{E}\left[\displaystyle {\boldsymbol ...

View answer
sample autocovariance function vs autocovariance function
Accepted answer
3 votes

Maybe one autocovariance function is theoretical, but we estimate it with the sample autocovariance function. Yes, that's correct. One is based on random data, and the other is theoretical and based ...

View answer
Difference in AIC between AR(p) and AR(p+1) models
3 votes

Can the log-likelihood actually decrease when adding a parameter, is there something special about how the TSA package calculates AIC, am I doing something wrong in R, or is R pulling my leg ...

View answer
Exponential Twisting
Accepted answer
3 votes

It looks like you're having trouble with the derivative of a moment generating function. The reason they have the name they have is because they generate (via differentiation) moments (as long as you ...

View answer
Setting up a MCMC scheme for Multivariate Stochastic Volatility
Accepted answer
3 votes

This book chapter is a pretty good explanation of MCMC techniques for general state space models; you might want to start with this. For your particular model here, instead of looking at a review ...

View answer
Loss functions for regression proof
3 votes

This proof is easier if you iterate expectations. You want to show that $\mathbb{E}(y|x)$ is in a sense the best predictor of $t$, so you want to show that $$ \mathbb{E}[(y(x) - t)^2] \ge \mathbb{E}[\{...

View answer
Confusion: Conditioning a Discrete rv on a Continuous rv, "Sampling Importance Resampling"
Accepted answer
3 votes

$X^*$ is not discrete; it's continuous. The $X_i$'s are continuous, too. It is $X^*|X_{1},\ldots,X_n$ that's discrete. I'll copy and paste your $\TeX$ and and write over the stuff that's incorrect. I ...

View answer
Distribution for the canonical statistic of exponential family
Accepted answer
3 votes

You're pretty much there. Here are all your steps, with the intended answer added at the end. \begin{align*} P(t(\boldsymbol{y}) = \boldsymbol{t}) &= \sum_{t(\boldsymbol{y}) =\boldsymbol{t}} P(t(\...

View answer
Trying to understand a simple discrete Expectation proof of $E(h(X,Y))$
Accepted answer
3 votes

Going from the first line to the beginning of the second line, you aren't pulling $z$ out of the right summation. You are simply substituting $$ P(h(X,Y) = z) = \sum_{\{(x,y): h(x,y) = z\}}P(X=x,Y=y)....

View answer
Kalman filter swapping algorithmic steps
3 votes

Is it correct to swap these two and put the measurement update first and then the prediction so at each time step have a prediction for the next time step? Depends what you mean by "swap." Let $...

View answer
Expectation of a square root of a sample mean
3 votes

Say the parameter to each distribution is $\lambda$. First, $$ E\left[\frac{\bar{X}}{6}\right] = [6\lambda]^{-1} $$ by linearity. Then, because $Y = \sum_i X_i \sim \text{Gamma}(n,1/\lambda)$ \begin{...

View answer
prove that X converges in probability to 1
3 votes

Hint: $$ P(|X_n - 1|> \epsilon) = P(nY_n > \epsilon) = P(Y_n > \epsilon/n) = 1-P(Y_n\le \epsilon/n). $$

View answer
Show convergence of the first order statistic of independent uniform$(0,n)$ distributed random variables
Accepted answer
3 votes

If you mean $X_1, \ldots, X_n \overset{iid}{\sim} \text{Uniform}(0,n)$, then $1-\prod_{k = 1}^n(1-F_k(y))$ turns into $$ 1-\prod_{k =1}^n(1-y/n) = 1 - (1-y/n)^n \to 1 - e^{-y}. $$ This is the CDF of ...

View answer
Gaussian assumption in Kalman filter
Accepted answer
3 votes

The Gaussian assumption is used in the predict and update steps of the Kalman Filter. They are the reason you only have to keep track of means and variances. First, $Z_t|X_t$ is Normal. Second, $X_t|...

View answer
Law of total variance as Pythagorean theorem
3 votes

Statement: The Pythagorean theorem says, for any elements $T_1$ and $T_2$ of an inner-product space with finite norms such that $\langle T_1,T_2\rangle = 0$, $$ ||T_1+T_2||^2 = ||T_1||^2 + ||T_2||^2 \...

View answer
Why we use Ridge regression instead of Least squares in Multicollinearity?
3 votes

Your OLS estimator is $$ \hat{\beta}_{ols} = (X'X)^{-1}X'y, $$ while your ridge regression estimator is $$ \hat{\beta}_{ridge} = (X'X + \lambda I)^{-1}X'y. $$ Take the expectation and variance of ...

View answer
How should you express a negative binomial distribution (\w gamma function) in an exponential family form?
3 votes

I'm not sure you can. Unless $\phi \in \mathbb{N}$ is known. You can see the problem in the last line. \begin{align*} &f(y_i, \mu, \phi) \\ &= \frac{\Gamma (y+ \phi)}{\Gamma(\phi) \Gamma(y+1)}...

View answer
Likelihood of a state space model with multiplicative noise $p(y_k|x_k) = ? $
Accepted answer
3 votes

It is still Gaussian, but the state is affecting the scale, not the location of the observaions. Here $x_k$ represents the standard deviation of $y_k$. So $$ y_k|x_k \sim \text{Gaussian}(y_k;0, x_k^2)...

View answer
Distribution of Max of 2 Uniforms with different support
Accepted answer
3 votes

Let $Z=\max(X,Y)$. It's always true by independence that $$ F_Z(z) = P(\max(X,Y) \le z) = P(X \le z)P(Y \le z). $$ If $0 \le z \le 1$, then $P(\max(X,Y) \le z) = z^2/2$. If $1 < z \le 2$ then $P(\...

View answer
Sampling from Empirical CDF for Forecasting
3 votes

Just make sure you resample from your data with replacement, and you give every data point the same chance of being chosen. Then you should be good. Here's a quick R example: num_samples <- 3 ...

View answer
Minimisation of Kullback-Leibler divergence on an arbitrary parameter
Accepted answer
2 votes

\begin{align*} \partial_x D[p|q] &= \int_{-\infty}^{\infty} \bigg (\partial_xp(x)\log\frac{p(x)}{q(x)}+ p(x)\partial_x \log\frac{p(x)}{q(x)}\bigg)dx \\ &= \int_{-\infty}^{\infty}\partial_xp(x)\...

View answer
Using eigendecomposition to transform state vector in linear Gaussian state space model
Accepted answer
2 votes

Must we make an assumption that $\mathbf{Q}$ is known in order to do what the authors are suggesting? No, this is a matter of identifiability, and things like this are commonly done to make the model ...

View answer
Need handy formula for $Var[\max(V, K)]$
Accepted answer
2 votes

Finding the first moment Whenever I look up the Black-Scholes formula, I always worry about changing parameterizations and notations, so let's start off by reproducing the result in Hull's text. All ...

View answer
Inaccurate parameter estimates for state-space models?
2 votes

"Is this normal?" I don't use this software, and I'm not sure what you mean by "normal," but nothing jumps out at me. A few thoughts: Asymptotic distributions of MLE estimates ...

View answer
Which notation and why: $\text{P}()$, $\Pr()$, $\text{Prob}()$, or $\mathbb{P}()$
2 votes

This makes me think of Meyn and Tweedie's book. They use $P$ to denote the transition kernel for a Markov chain, and $\mathsf{P}$ for the law of the entire chain on $\mathsf{X}^{\infty}$. This answer ...

View answer
Characteristic Function and Quantile Function?
Accepted answer
2 votes

The inverse of a cdf $F : \mathbf{R} \mapsto [0,1]$ is usually $$ F^{-1}(p) = \inf\{ x : F(x) \ge p \}. $$ The way you invert a c.f. $\phi$ to get measures of intervals is $$ \mu([a,b]) = \lim_{T\...

View answer
Parameters identifiability / estimation in Bayesian linear state-space models
Accepted answer
2 votes

Adding to @Jarle Tufto's comment, the likelihood can be written as $$ p(y_{1:t} \mid r, \sigma_x, \mu, p) = \int p(y_{1:t} \mid x_{1:t}, \mu, p) \overbrace{p(x_{1:t} \mid r, \sigma_x)}^{{\text{AR(1)}}}...

View answer
Trying to understand linear state-space notation
Accepted answer
2 votes

SSMs aren't always identifiable without restrictions on the parameter space. There is a common distinction between centered and uncentered parameterizations. What you have written would be called the ...

View answer
Approximating the mathematical expectation of the argmax of a Gaussian random vector
2 votes

You can use the law of large numbers to approximate your expectation pretty easily. Edit: Analyticaly you can multiply a bunch of normal cdf evaluations together. For $i > 0$ \begin{align*} \...

View answer
1
3 4
5
6 7
17