All Questions
73,762
questions with no upvoted or accepted answers
26
votes
4
answers
887
views
Blind source separation of convex mixture?
Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures:
\begin{align}
Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\
...&\\
Y_m &= a_{m1}X_1 + ...
- 825
23
votes
0
answers
1k
views
Is there a general expression for ancillary statistics in exponential families?
An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
- 2,380
22
votes
0
answers
17k
views
When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?
Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?).
The ...
- 2,170
21
votes
1
answer
1k
views
Estimation of ARMA: state space vs. alternatives
I am interested in estimation of ARMA models. I understand that a popular approach is to write the model down in the state space form and then maximize the likelihood of the model using some ...
- 64.6k
20
votes
1
answer
969
views
Physical/pictoral interpretation of higher-order moments
I'm preparing a presentation about parallel statistics. I plan to illustrate the formulas for distributed computation of the mean and variance with examples involving center of gravity and moment of ...
- 241
19
votes
0
answers
477
views
Empirical Bayes (In)Admissibility
Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators.
Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
- 4,106
19
votes
0
answers
2k
views
Implementation of CoVaR (a systemic risk measure) in R
I'm trying to estimate CoVaR using bivariate DCC GARCH in R. The concept of CoVaR is the dependence adjusted of VaR, which was first introduced by Adrian and Brunnermeier (2011). However, this ...
- 265
19
votes
0
answers
2k
views
Picking block length in a block bootstrap
I am using the Mann-Kendall test to assess trends in a data time-series. I believe there is autocorrelation in my data and therefore need to use a block bootstrap to correct for it. I have plotted ...
- 191
18
votes
0
answers
701
views
Is the Wilcoxon two-sample test maximally powered to detect proportional odds alternatives?
We know from the literature that
The Wilcoxon-Mann-Whitney two-sample rank sum test is optimal for detecting simple location shifts when comparing two continuous random variables that each have a ...
- 86.3k
18
votes
2
answers
917
views
Clustering & Time Series
I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset.
Now I want to see ...
- 416
17
votes
1
answer
521
views
Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?
I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
- 3,585
17
votes
1
answer
946
views
How can I measure model performance with weighted logistic regression?
I am working with some survey data that uses probability weights. A number of sources explain that likelihood-based tests and fit statistics like likelihood-ratio, AIC, and BIC are not valid in the ...
- 341
16
votes
0
answers
1k
views
what is the mistake of convergence proof in Adam
Sashank J. Reddi et. al in their paper "On the convergence of Adam and beyond" say that, Adam's proof of convergence as stated in original paper is wrong. More than that, they point out that the value ...
- 303
16
votes
3
answers
786
views
Adjustments to (Linear Regression) Forecast
Full disclosure: I am not a statistician, nor do I claim to be one. I am a lowly IT administrator. Please play gentle with me. :)
I am responsible for collecting and forecasting disk storage use ...
- 261
16
votes
0
answers
383
views
What is tantile regression?
My question follows on this discussion of medials and tantiles vs medians and quantiles from earlier this year:
When would we use tantiles and the medial, rather than quantiles and the median?
As ...
- 10.2k
16
votes
0
answers
2k
views
Rademacher complexity of logistic regression
Consider logistic regression. We have the logistic loss function,
$\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
- 798
16
votes
2
answers
931
views
Visualizing many left-skewed distributions
I have a series of left-skewed/heavy tailed distributions that I would like to show. There are 42 distributions across three factors (labeled as A, ...
- 5,960
16
votes
0
answers
543
views
Asymptotic property of tuning parameter in penalized regression
I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around.
To keep things simple, I'...
- 12.5k
16
votes
0
answers
1k
views
What is Shannon's source entropy?
Suppose that ${X_n; Y_n}$ is a random process with a discrete alphabet, that is, taking on values in a discrete set for $n$ data length. They correspond to the input and output of a communication ...
- 419
16
votes
0
answers
5k
views
How to compare two distance matrices?
Suppose that I have two distance matrices for the same set of items.
By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
- 443
15
votes
1
answer
749
views
Are non-square latin hypercubes viable?
At https://github.com/OpenMDAO/OpenMDAO-Framework/issues/599 it is stated that non-square Latin Hypercube experimental design is not well defined (I assume that for higher dimensions that means ...
- 5,233
15
votes
0
answers
13k
views
Time series regression with overlapping data
I am seeing a regression model which is regressing Year-on-Year stock index returns on lagged (12 months) Year-on-Year returns of the same stock index, credit spread (difference between monthly mean ...
- 318
14
votes
0
answers
667
views
Convolutional neural network for multi-variate time series?
I want to use CNN architectures for classification of multivariate time-series, where we apply one label to each sequence.
I searched the net for the available designs in the literature and i found ...
- 439
14
votes
0
answers
4k
views
What is the KL divergence of distribution from Dirac delta?
The Kullback–Leibler (KL) divergence of two continuous distributions $P(x)$ and $Q(x)$ is defined as
$$D_{KL}(P \mid\mid Q) = \int_{X} P(x) \log{\left[\frac{P(x)}{Q(x)}\right]} dx$$
How can one ...
- 173
14
votes
0
answers
968
views
Generalization of Fisher information for a discrete parameter
This is mainly a reference request. There must be some generalizations of the concept of Fisher information for discrete (say, integer-valued) parameters, and of related results such as the Cramer-...
- 74.6k
14
votes
0
answers
3k
views
Interpreting and reporting gamm4 result
I am new to gam, and most of my knowledge comes from this document http://www3.nd.edu/~mclark19/learn/GAMS.pdf. Now I am using generalized addictive model with random effects to model some data, where ...
- 1,095
14
votes
0
answers
1k
views
Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio
I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows:
Two URLs are compared for their ability to ...
- 683
14
votes
0
answers
454
views
Penalized spline confidence intervals based on cluster-sandwich VCV
This is my first post here, but I've benefited a lot from this forum's results popping up in google search results.
I've been teaching myself semi-parametric regression using penalized splines. ...
- 13.2k
14
votes
1
answer
12k
views
Regress residuals in second regression
I am wondering if anyone can point me to a paper/lecture notes on the rationale behind first running an OLS on a set of variables, and then in a second regression using the residuals of that ...
- 223
13
votes
0
answers
223
views
Logistic regression for classification: are there any analytical solutions for the out-of-sample accuracy?
I run a binary logistic regression, with a binary dependent variable and a continuous independent one.
Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. ...
- 2,092
13
votes
0
answers
693
views
Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)
Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as
$$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$
Here the kernel $K$ is an assigned smooth ...
Tim♦
- 135k
13
votes
1
answer
3k
views
How to generate 2 correlated Beta random variables
I was wondering if it might be possible to generate 2 correlated $Beta$ random variables?
In other words, I want to generate two Beta random variables which can be said to have come from two Beta ...
- 3,826
13
votes
1
answer
839
views
Bound the difference between Spearman's Correlation and Kendall's Correlation
I am trying to prove or disprove that the difference between Spearman's Correlation and Kendall's Correlation is no more than 1 (or less, the tighter the merrier).
I am assuming there are no ties.
...
- 131
13
votes
0
answers
352
views
Is probability fundamentally about reference classes (real or imagined)?
Question: It seems that frequentism and Bayesianism may not really be different as far as the the ultimate basis for what a probability is (relative frequency within a reference class) - it's just ...
user75138
13
votes
0
answers
2k
views
Negative deviance explained by GAM with betareg in R
I am fitting the following model in "mgcv" package in R using option family=betar to predict a percentage cover response variable (...
- 473
12
votes
0
answers
379
views
Are there Imbalanced learning problems where re-balancing/re-weighting demonstrably improves *accuracy*?
I have been looking into the imbalanced learning problem, where a classifier is often expected to be unduly biased in favour of the majority class. However, I am having difficulties identifying ...
- 53.3k
12
votes
0
answers
1k
views
How to calculate percent partial deviance explained by each predictor variable in a GAM model?
I am trying to find a sensible way to calculate the deviance explained by each predictor variable in a GAM model and need some input on my calculations.
Following Simon Wood's example on the thread ...
- 19.7k
12
votes
0
answers
1k
views
Instrumental variables with interactions between endogenous variables
I have two endogenous variables $x_1$ and $x_2$ and am trying to estimate the following model:
$$y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_{12} x_{12}$$
where $x_{12} = x_1\times x_2$. I'm ...
- 171
12
votes
0
answers
671
views
Interpreting regression coefficients based on Andrew Gelman's re-scaling method
I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model.
I have come ...
- 403
12
votes
0
answers
17k
views
How to find p value using estimate and standard error?
I am trying to check statement on p. 23 of Data Analysis Using Regression and Multilevel/Hierarchical Models
For example, consider two independent studies with effect estimates and standard errors ...
- 121
12
votes
0
answers
2k
views
Is sparsity of topics a necessary condition for latent Dirichlet allocation (LDA) to work
I have been playing with the hyper-parameters of the latent Dirichlet allocation (LDA) model and am wondering how sparsity of topic priors play a role in inference.
I have not performed these ...
- 3,482
12
votes
0
answers
339
views
Is autocorrelation not worth addressing with small N?
Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
12
votes
0
answers
750
views
What approaches use multiple eigenvectors in graph spectral clustering?
Background: In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by ...
- 2,382
12
votes
0
answers
2k
views
Empirical Prediction interval for time series forecast based on quantile regression
As Gardner notes "almost all point forecasts are wrong", so prediction intervals (PI) are necessary to quantify uncertainty and help us make informed decisions. There exists theoretical PI, and in ...
- 8,043
12
votes
0
answers
5k
views
How to use formative indicators in covariance-based SEM with lavaan?
I'm trying to build a covariance-based structural equation model (SEM) using both reflective and formative specifications of latent variables. I use the sem ...
- 325
12
votes
0
answers
3k
views
glmer in R: Significance estimates are not robust to order of data frame
I'm using a mixed effects model with logistic link function (using lme4 version 1.1-7 in R). However, I noticed that the estimates of significance for fixed effects change depending on the order of ...
- 131
12
votes
2
answers
2k
views
Scaling the backward variable in HMM Baum-Welch
I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
- 121
12
votes
0
answers
13k
views
How to normalize data prior to computation of covariance matrix
In all my self-study, I have come across many different ways in which people seem to normalize their data, prior to the computation of the covariance matrix. I am confused as to what ways are 'correct'...
- 1,615
12
votes
1
answer
4k
views
How do I identify the "Long Tail" portion of my distribution?
I have a number of series that would typically be described as normal skewed or Gamma distributed. For example, in a group of customers I may have calculated their spend over a fixed length of time. ...
- 121
12
votes
1
answer
307
views
Unit testing sampling methods
I'm writing a bit of code that makes pretty heavy use of sampling (eg, MCMC, Particle Filters, etc), and I would really like to test it to make sure that it's doing what I think it is before claiming ...
- 545