All Questions
69,070
questions with no upvoted or accepted answers
28
votes
0
answers
2k
views
How does a Relevance Vector Machine (RVM) work?
Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs.
In the light of a question like How does a Support Vector Machine (SVM) ...
26
votes
1
answer
1k
views
Bootstrapping Generalized Least Squares
Scenario:
Consider the use of bootstrapping to estimate the distribution of model parameters fitted per a linear or nonlinear generalized least squares model. In particular, assume there is a ...
24
votes
1
answer
582
views
Blind source separation of convex mixture?
Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures:
\begin{align}
Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\
...&\\
Y_m &= a_{m1}X_1 + ...
24
votes
0
answers
767
views
Distribution of inverse Wishart to a power?
In a related question, I had asked about the norm induced by an inverse Wishart matrix. I am interested in generalizing that result somewhat. Let $A\sim\mathcal{W}_p\left(I,n\right)$, a Wishart matrix ...
20
votes
0
answers
873
views
Is there a general expression for ancillary statistics in exponential families?
An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
20
votes
1
answer
872
views
Physical/pictoral interpretation of higher-order moments
I'm preparing a presentation about parallel statistics. I plan to illustrate the formulas for distributed computation of the mean and variance with examples involving center of gravity and moment of ...
18
votes
0
answers
368
views
Empirical Bayes (In)Admissibility
Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators.
Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
18
votes
0
answers
11k
views
When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?
Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?).
The ...
18
votes
0
answers
2k
views
Implementation of CoVaR (a systemic risk measure) in R
I'm trying to estimate CoVaR using bivariate DCC GARCH in R. The concept of CoVaR is the dependence adjusted of VaR, which was first introduced by Adrian and Brunnermeier (2011). However, this ...
18
votes
0
answers
2k
views
Picking block length in a block bootstrap
I am using the Mann-Kendall test to assess trends in a data time-series. I believe there is autocorrelation in my data and therefore need to use a block bootstrap to correct for it. I have plotted ...
17
votes
0
answers
1k
views
Estimation of ARMA: state space vs. alternatives
I am interested in estimation of ARMA models. I understand that a popular approach is to write the model down in the state space form and then maximize the likelihood of the model using some ...
16
votes
0
answers
3k
views
Gamma hurdle model for continuous response?
I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of ...
16
votes
1
answer
716
views
Clustering & Time Series
I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset.
Now I want to see ...
15
votes
0
answers
495
views
Asymptotic property of tuning parameter in penalized regression
I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around.
To keep things simple, I'...
15
votes
1
answer
474
views
Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?
I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
14
votes
0
answers
209
views
What is the "direct likelihood" point of view in statistics?
I am reading a Springer title from 1997 called Applied Generalized Linear Models by James K. Lindsey. In the preface, Lindsey writes
For this text, the reader is assumed to have knowledge of basic ...
14
votes
0
answers
2k
views
Rademacher complexity of logistic regression
Consider logistic regression. We have the logistic loss function,
$\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
14
votes
1
answer
743
views
Visualizing many left-skewed distributions
I have a series of left-skewed/heavy tailed distributions that I would like to show. There are 42 distributions across three factors (labeled as A, ...
14
votes
0
answers
1k
views
What is Shannon's source entropy?
Suppose that ${X_n; Y_n}$ is a random process with a discrete alphabet, that is, taking on values in a discrete set for $n$ data length. They correspond to the input and output of a communication ...
14
votes
0
answers
764
views
How can I measure model performance with weighted logistic regression?
I am working with some survey data that uses probability weights. A number of sources explain that likelihood-based tests and fit statistics like likelihood-ratio, AIC, and BIC are not valid in the ...
14
votes
0
answers
1k
views
Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?
I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
14
votes
1
answer
558
views
Are non-square latin hypercubes viable?
At https://github.com/OpenMDAO/OpenMDAO-Framework/issues/599 it is stated that non-square Latin Hypercube experimental design is not well defined (I assume that for higher dimensions that means ...
14
votes
0
answers
11k
views
Time series regression with overlapping data
I am seeing a regression model which is regressing Year-on-Year stock index returns on lagged (12 months) Year-on-Year returns of the same stock index, credit spread (difference between monthly mean ...
13
votes
0
answers
153
views
Why is Binning, Weight of Evidence and Information Value so ubiquitous in the Credit Risk/Finance industry?
In the credit risk industry (and finance industry as a whole, at least here in the UK), there is a very common and accepted 'proper' way to build scorecards.
The general framework seems to be:
...
13
votes
0
answers
575
views
Convolutional neural network for multi-variate time series?
I want to use CNN architectures for classification of multivariate time-series, where we apply one label to each sequence.
I searched the net for the available designs in the literature and i found ...
13
votes
3
answers
638
views
Adjustments to (Linear Regression) Forecast
Full disclosure: I am not a statistician, nor do I claim to be one. I am a lowly IT administrator. Please play gentle with me. :)
I am responsible for collecting and forecasting disk storage use ...
13
votes
0
answers
278
views
What is tantile regression?
My question follows on this discussion of medials and tantiles vs medians and quantiles from earlier this year:
When would we use tantiles and the medial, rather than quantiles and the median?
As ...
13
votes
1
answer
2k
views
How to compare forecasting methods?
I have several intermittent data. Based on those data, I would like to compare several forecasting methods (Exponential Smoothing, Moving Average, Croston, and Syntetos-Boylan), and decide whether ...
13
votes
0
answers
5k
views
Conditional expectation subscript notation
This should be a relatively simple question. I'm trying to confirm my understanding of the subscript notation on expectations when the subscript denotes a conditioning. In the example
$$E_{Y|X}[(Y-f(...
13
votes
0
answers
388
views
Penalized spline confidence intervals based on cluster-sandwich VCV
This is my first post here, but I've benefited a lot from this forum's results popping up in google search results.
I've been teaching myself semi-parametric regression using penalized splines. ...
12
votes
0
answers
597
views
Is the Wilcoxon two-sample test maximally powered to detect proportional odds alternatives?
We know from the literature that
The Wilcoxon-Mann-Whitney two-sample rank sum test is optimal for detecting simple location shifts when comparing two continuous random variables that each have a ...
12
votes
0
answers
738
views
what is the mistake of convergence proof in Adam
Sashank J. Reddi et. al in their paper "On the convergence of Adam and beyond" say that, Adam's proof of convergence as stated in original paper is wrong. More than that, they point out that the value ...
12
votes
0
answers
571
views
Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)
Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as
$$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$
Here the kernel $K$ is an assigned smooth ...
12
votes
0
answers
3k
views
What is the KL divergence of distribution from Dirac delta?
The Kullback–Leibler (KL) divergence of two continuous distributions $P(x)$ and $Q(x)$ is defined as
$$D_{KL}(P \mid\mid Q) = \int_{X} P(x) \log{\left[\frac{P(x)}{Q(x)}\right]} dx$$
How can one ...
12
votes
1
answer
704
views
Bound the difference between Spearman's Correlation and Kendall's Correlation
I am trying to prove or disprove that the difference between Spearman's Correlation and Kendall's Correlation is no more than 1 (or less, the tighter the merrier).
I am assuming there are no ties.
...
12
votes
0
answers
3k
views
Interpreting and reporting gamm4 result
I am new to gam, and most of my knowledge comes from this document http://www3.nd.edu/~mclark19/learn/GAMS.pdf. Now I am using generalized addictive model with random effects to model some data, where ...
12
votes
2
answers
2k
views
Scaling the backward variable in HMM Baum-Welch
I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
12
votes
0
answers
1k
views
Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio
I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows:
Two URLs are compared for their ability to ...
12
votes
0
answers
4k
views
How to compare two distance matrices?
Suppose that I have two distance matrices for the same set of items.
By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
12
votes
1
answer
7k
views
Confidence Intervals for AUC using cross-validation
I am analyzing the performance of a predictive model with the AUC, area under the ROC curve. I repeat several times cross-validation, and I have different estimations of the AUC in each folder. For ...
11
votes
0
answers
823
views
Instrumental variables with interactions between endogenous variables
I have two endogenous variables $x_1$ and $x_2$ and am trying to estimate the following model:
$$y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_{12} x_{12}$$
where $x_{12} = x_1\times x_2$. I'm ...
11
votes
0
answers
535
views
When can a Gaussian Process solve an SDE?
Considering an SDE of the form
$$dX_t = \mu(X_t, t)dt + \sigma(X_t, t)dW_t ,$$
where $W_t$ is a Wiener process, is there a set of necessary and sufficient conditions on the structure of the functions $...
11
votes
0
answers
10k
views
Deriving linear regression gradient with MSE
So I've been tinkering around with the backpropagation algorithm and to try to get a better understanding of how it works and my calculus is quite rusty. I've derived the gradient for linear ...
11
votes
0
answers
2k
views
How to generate 2 correlated Beta random variables
I was wondering if it might be possible to generate 2 correlated $Beta$ random variables?
In other words, I want to generate two Beta random variables which can be said to have come from two Beta ...
11
votes
1
answer
261
views
Correlation between two binary variables within one categorical variable
The Problem:
I have measured two binary variables within 1 categorical variable with 5 levels.
Initially, I thought I'd be able to use Fisher's Exact test or some $N \times M \times K$ version of it. ...
11
votes
0
answers
2k
views
Is sparsity of topics a necessary condition for latent Dirichlet allocation (LDA) to work
I have been playing with the hyper-parameters of the latent Dirichlet allocation (LDA) model and am wondering how sparsity of topic priors play a role in inference.
I have not performed these ...
11
votes
0
answers
1k
views
Negative deviance explained by GAM with betareg in R
I am fitting the following model in "mgcv" package in R using option family=betar to predict a percentage cover response variable (...
11
votes
0
answers
258
views
Is autocorrelation not worth addressing with small N?
Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
11
votes
0
answers
685
views
What approaches use multiple eigenvectors in graph spectral clustering?
Background: In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by ...
11
votes
0
answers
2k
views
Empirical Prediction interval for time series forecast based on quantile regression
As Gardner notes "almost all point forecasts are wrong", so prediction intervals (PI) are necessary to quantify uncertainty and help us make informed decisions. There exists theoretical PI, and in ...