All Questions
73,070
questions with no upvoted or accepted answers
23
votes
0
answers
18k
views
When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?
Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?).
The ...
- 2,190
23
votes
0
answers
1k
views
Is there a general expression for ancillary statistics in exponential families?
An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
- 2,470
21
votes
1
answer
1k
views
Physical/pictoral interpretation of higher-order moments
I'm preparing a presentation about parallel statistics. I plan to illustrate the formulas for distributed computation of the mean and variance with examples involving center of gravity and moment of ...
- 251
20
votes
0
answers
2k
views
Implementation of CoVaR (a systemic risk measure) in R
I'm trying to estimate CoVaR using bivariate DCC GARCH in R. The concept of CoVaR is the dependence adjusted of VaR, which was first introduced by Adrian and Brunnermeier (2011). However, this ...
- 275
19
votes
0
answers
724
views
Is the Wilcoxon two-sample test maximally powered to detect proportional odds alternatives?
We know from the literature that
The Wilcoxon-Mann-Whitney two-sample rank sum test is optimal for detecting simple location shifts when comparing two continuous random variables that each have a ...
- 92.2k
19
votes
0
answers
492
views
Empirical Bayes (In)Admissibility
Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators.
Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
- 4,486
18
votes
0
answers
1k
views
What is Shannon's source entropy?
Suppose that ${X_n; Y_n}$ is a random process with a discrete alphabet, that is, taking on values in a discrete set for $n$ data length. They correspond to the input and output of a communication ...
- 439
17
votes
0
answers
5k
views
How to compare two distance matrices?
Suppose that I have two distance matrices for the same set of items.
By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
- 453
17
votes
1
answer
972
views
How can I measure model performance with weighted logistic regression?
I am working with some survey data that uses probability weights. A number of sources explain that likelihood-based tests and fit statistics like likelihood-ratio, AIC, and BIC are not valid in the ...
- 341
16
votes
0
answers
405
views
What is tantile regression?
My question follows on this discussion of medials and tantiles vs medians and quantiles from earlier this year:
When would we use tantiles and the medial, rather than quantiles and the median?
As ...
- 10.6k
16
votes
0
answers
2k
views
Rademacher complexity of logistic regression
Consider logistic regression. We have the logistic loss function,
$\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
- 808
16
votes
0
answers
1k
views
Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio
I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows:
Two URLs are compared for their ability to ...
- 703
16
votes
0
answers
561
views
Asymptotic property of tuning parameter in penalized regression
I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around.
To keep things simple, I'...
- 12.8k
16
votes
0
answers
13k
views
Time series regression with overlapping data
I am seeing a regression model which is regressing Year-on-Year stock index returns on lagged (12 months) Year-on-Year returns of the same stock index, credit spread (difference between monthly mean ...
- 328
15
votes
0
answers
1k
views
Generalization of Fisher information for a discrete parameter
This is mainly a reference request. There must be some generalizations of the concept of Fisher information for discrete (say, integer-valued) parameters, and of related results such as the Cramer-...
- 78.1k
15
votes
0
answers
4k
views
Interpreting and reporting gamm4 result
I am new to generalized additive models (GAMs), and most of my knowledge comes from this document. Now I am using a generalized additive model with random effects to model some data, where I want to ...
- 1,125
15
votes
0
answers
463
views
Penalized spline confidence intervals based on cluster-sandwich VCV
This is my first post here, but I've benefited a lot from this forum's results popping up in google search results.
I've been teaching myself semi-parametric regression using penalized splines. ...
- 13.3k
15
votes
1
answer
780
views
Are non-square latin hypercubes viable?
At https://github.com/OpenMDAO/OpenMDAO-Framework/issues/599 it is stated that non-square Latin Hypercube experimental design is not well defined (I assume that for higher dimensions that means ...
- 5,453
14
votes
0
answers
676
views
Convolutional neural network for multi-variate time series?
I want to use CNN architectures for classification of multivariate time-series, where we apply one label to each sequence.
I searched the net for the available designs in the literature and i found ...
- 439
14
votes
0
answers
4k
views
What is the KL divergence of distribution from Dirac delta?
The Kullback–Leibler (KL) divergence of two continuous distributions $P(x)$ and $Q(x)$ is defined as
$$D_{KL}(P \mid\mid Q) = \int_{X} P(x) \log{\left[\frac{P(x)}{Q(x)}\right]} dx$$
How can one ...
- 173
14
votes
1
answer
3k
views
How to generate 2 correlated Beta random variables
I was wondering if it might be possible to generate 2 correlated $Beta$ random variables?
In other words, I want to generate two Beta random variables which can be said to have come from two Beta ...
- 3,996
14
votes
1
answer
864
views
Bound the difference between Spearman's Correlation and Kendall's Correlation
I am trying to prove or disprove that the difference between Spearman's Correlation and Kendall's Correlation is no more than 1 (or less, the tighter the merrier).
I am assuming there are no ties.
...
- 141
14
votes
0
answers
2k
views
Negative deviance explained by GAM with betareg in R
I am fitting the following model in "mgcv" package in R using option family=betar to predict a percentage cover response variable (...
- 483
13
votes
0
answers
241
views
Logistic regression for classification: are there any analytical solutions for the out-of-sample accuracy?
I run a binary logistic regression, with a binary dependent variable and a continuous independent one.
Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. ...
- 2,261
13
votes
0
answers
1k
views
How to calculate percent partial deviance explained by each predictor variable in a GAM model?
I am trying to find a sensible way to calculate the deviance explained by each predictor variable in a GAM model and need some input on my calculations.
Following Simon Wood's example on the thread ...
- 20.3k
13
votes
0
answers
2k
views
Are these equivalent (for p-values): super-uniform, stochastically larger than / dominating the uniform, conservative?
In the literature and online, I have found three different wordings that I think refer to the same concept: stochastically larger than uniform (which I take is ...
- 231
13
votes
0
answers
700
views
Interpreting regression coefficients based on Andrew Gelman's re-scaling method
I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model.
I have come ...
- 413
13
votes
0
answers
375
views
How to construct confidence limits based on small stratified samples of finite populations?
Imagine a business wishes to audit its transactions. It has a database summarizing the transactions, which constitute a sampling frame for the population. It would be time-consuming and expensive to ...
- 323k
13
votes
0
answers
18k
views
How to find p value using estimate and standard error?
I am trying to check statement on p. 23 of Data Analysis Using Regression and Multilevel/Hierarchical Models
For example, consider two independent studies with effect estimates and standard errors ...
- 131
13
votes
0
answers
703
views
Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)
Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as
$$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$
Here the kernel $K$ is an assigned smooth ...
Tim♦
- 138k
13
votes
0
answers
366
views
Is autocorrelation not worth addressing with small N?
Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
13
votes
2
answers
2k
views
Scaling the backward variable in HMM Baum-Welch
I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
- 131
12
votes
0
answers
421
views
Are there Imbalanced learning problems where re-balancing/re-weighting demonstrably improves *accuracy*?
I have been looking into the imbalanced learning problem, where a classifier is often expected to be unduly biased in favour of the majority class. However, I am having difficulties identifying ...
- 54.5k
12
votes
0
answers
2k
views
Instrumental variables with interactions between endogenous variables
I have two endogenous variables $x_1$ and $x_2$ and am trying to estimate the following model:
$$y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_{12} x_{12}$$
where $x_{12} = x_1\times x_2$. I'm ...
- 171
12
votes
1
answer
337
views
Correlation between two binary variables within one categorical variable
The Problem:
I have measured two binary variables within 1 categorical variable with 5 levels.
Initially, I thought I'd be able to use Fisher's Exact test or some $N \times M \times K$ version of it. ...
- 571
12
votes
0
answers
2k
views
Is sparsity of topics a necessary condition for latent Dirichlet allocation (LDA) to work
I have been playing with the hyper-parameters of the latent Dirichlet allocation (LDA) model and am wondering how sparsity of topic priors play a role in inference.
I have not performed these ...
- 3,552
12
votes
0
answers
764
views
What approaches use multiple eigenvectors in graph spectral clustering?
Background: In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by ...
- 2,402
12
votes
0
answers
2k
views
Empirical Prediction interval for time series forecast based on quantile regression
As Gardner notes "almost all point forecasts are wrong", so prediction intervals (PI) are necessary to quantify uncertainty and help us make informed decisions. There exists theoretical PI, and in ...
- 8,185
12
votes
0
answers
5k
views
How to use formative indicators in covariance-based SEM with lavaan?
I'm trying to build a covariance-based structural equation model (SEM) using both reflective and formative specifications of latent variables. I use the sem ...
- 325
12
votes
0
answers
109
views
Words that estimate numerical proportions of a group
There seem to be more than a few papers that surround the use of certain words to define the probability of an event without actually using the numerical probability itself.
For example, if you are ...
- 7,262
12
votes
0
answers
13k
views
How to normalize data prior to computation of covariance matrix
In all my self-study, I have come across many different ways in which people seem to normalize their data, prior to the computation of the covariance matrix. I am confused as to what ways are 'correct'...
- 1,665
12
votes
0
answers
1k
views
Computing a bootstrap confidence interval for the prediction error with the percentile and the BCa method
I have two related questions regarding the computation of a non-parametric bootstrap confidence interval for the prediction error.
Setting: I have a sample S from a data population P and a learner L, ...
- 825
11
votes
0
answers
3k
views
Varying dispersion parameter (=dispformula) in glmmTMB in R to account for heteroscedasticity that originates from one predictor
I struggle with understanding the dispersion model and dispersion parameter of glmmTMB , and could not find answers anywhere.
I constructed a GLMM using ...
- 111
11
votes
0
answers
395
views
Official name of a common type of Bayesian simulation study
There is a kind of simulation study that is commonly used to validate an implementation of a Bayesian model:
For independent replication $i = 1, ..., n$:
Draw a set of "true" parameters ...
- 257
11
votes
1
answer
3k
views
Feature selection using chi squared for continuous features
I'm looking at univariate feature selection. A method that is often described, is to look at the p-values for a $\chi^2$-test. However, I'm confused as to how this works for continuous variables.
1. ...
- 211
11
votes
1
answer
3k
views
Expected value of softmax transformation of Gaussian random vector
Let $\mathbf w_1,\mathbf w_2,\ldots,\mathbf w_n \in \mathbb R^p$ and $\mathbf v \in \mathbb R^n$ be fixed vectors, and $\mathbf x \sim \mathcal N_p(\boldsymbol{\mu}, \mathbf{\Sigma})$ be an $p$-...
- 548
11
votes
1
answer
3k
views
How to test for the difference in skewness of two samples?
I have two samples. From looking at their densities, one appears symmetrical and the other from some right-tailed distribution. I would like to test that the two do not have the same skewness (...
- 21.6k
11
votes
0
answers
600
views
Does using bootstrapped samples improve parameter estimates for a fitted distribution?
The R package retimes has a function for fitting an ex-Gaussian distribution to a set of observations. The method involves taking multiple bootstrapped samples of the observations, and fitting the ex-...
- 213
11
votes
0
answers
7k
views
Precision and recall of a random classifier
My understanding of precision and recall tells me that there is a tradeoff between these two measures: you can improve one at the cost of the other.
However, when I think of a random classifier (on a ...
- 211
11
votes
0
answers
4k
views
Contours containing a given fraction of $(x,y)$ points
I often need make $(x,y)$ scatter plots that have many ($>10^5$) points. I've experimented with different ways of representing this many points that capture the distribution while getting around ...
- 283