All Questions

72,782 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
23 votes
0 answers
18k views

When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?

Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?). The ...
MiniQuark's user avatar
  • 2,210
23 votes
0 answers
1k views

Is there a general expression for ancillary statistics in exponential families?

An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
Henry.L's user avatar
  • 2,480
21 votes
1 answer
1k views

Physical/pictoral interpretation of higher-order moments

I'm preparing a presentation about parallel statistics. I plan to illustrate the formulas for distributed computation of the mean and variance with examples involving center of gravity and moment of ...
James Koppel's user avatar
20 votes
0 answers
2k views

Implementation of CoVaR (a systemic risk measure) in R

I'm trying to estimate CoVaR using bivariate DCC GARCH in R. The concept of CoVaR is the dependence adjusted of VaR, which was first introduced by Adrian and Brunnermeier (2011). However, this ...
drawar's user avatar
  • 275
19 votes
0 answers
729 views

Is the Wilcoxon two-sample test maximally powered to detect proportional odds alternatives?

We know from the literature that The Wilcoxon-Mann-Whitney two-sample rank sum test is optimal for detecting simple location shifts when comparing two continuous random variables that each have a ...
Frank Harrell's user avatar
19 votes
0 answers
497 views

Empirical Bayes (In)Admissibility

Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators. Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
Cagdas Ozgenc's user avatar
18 votes
0 answers
1k views

What is Shannon's source entropy?

Suppose that ${X_n; Y_n}$ is a random process with a discrete alphabet, that is, taking on values in a discrete set for $n$ data length. They correspond to the input and output of a communication ...
chk's user avatar
  • 439
17 votes
0 answers
2k views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
axk's user avatar
  • 818
17 votes
0 answers
5k views

How to compare two distance matrices?

Suppose that I have two distance matrices for the same set of items. By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
Ahmet Yılmaz's user avatar
17 votes
0 answers
13k views

Time series regression with overlapping data

I am seeing a regression model which is regressing Year-on-Year stock index returns on lagged (12 months) Year-on-Year returns of the same stock index, credit spread (difference between monthly mean ...
Vishal Belsare's user avatar
17 votes
1 answer
989 views

How can I measure model performance with weighted logistic regression?

I am working with some survey data that uses probability weights. A number of sources explain that likelihood-based tests and fit statistics like likelihood-ratio, AIC, and BIC are not valid in the ...
biased_estimator's user avatar
16 votes
0 answers
413 views

What is tantile regression?

My question follows on this discussion of medials and tantiles vs medians and quantiles from earlier this year: When would we use tantiles and the medial, rather than quantiles and the median? As ...
user78229's user avatar
  • 10.8k
16 votes
0 answers
1k views

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
jkndrkn's user avatar
  • 703
16 votes
0 answers
565 views

Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...
Nick Sabbe's user avatar
  • 12.9k
15 votes
0 answers
1k views

Generalization of Fisher information for a discrete parameter

This is mainly a reference request. There must be some generalizations of the concept of Fisher information for discrete (say, integer-valued) parameters, and of related results such as the Cramer-...
kjetil b halvorsen's user avatar
15 votes
1 answer
4k views

Interpreting and reporting gamm4 result

I am new to generalized additive models (GAMs), and most of my knowledge comes from this document. Now I am using a generalized additive model with random effects to model some data, where I want to ...
nan's user avatar
  • 1,135
15 votes
0 answers
465 views

Penalized spline confidence intervals based on cluster-sandwich VCV

This is my first post here, but I've benefited a lot from this forum's results popping up in google search results. I've been teaching myself semi-parametric regression using penalized splines. ...
generic_user's user avatar
  • 13.5k
14 votes
0 answers
1k views

How to calculate percent partial deviance explained by each predictor variable in a GAM model?

I am trying to find a sensible way to calculate the deviance explained by each predictor variable in a GAM model and need some input on my calculations. Following Simon Wood's example on the thread ...
Isabella Ghement's user avatar
14 votes
0 answers
679 views

Convolutional neural network for multi-variate time series?

I want to use CNN architectures for classification of multivariate time-series, where we apply one label to each sequence. I searched the net for the available designs in the literature and i found ...
Bob's user avatar
  • 439
14 votes
0 answers
378 views

How to construct confidence limits based on small stratified samples of finite populations?

Imagine a business wishes to audit its transactions. It has a database summarizing the transactions, which constitute a sampling frame for the population. It would be time-consuming and expensive to ...
whuber's user avatar
  • 327k
14 votes
0 answers
4k views

What is the KL divergence of distribution from Dirac delta?

The Kullback–Leibler (KL) divergence of two continuous distributions $P(x)$ and $Q(x)$ is defined as $$D_{KL}(P \mid\mid Q) = \int_{X} P(x) \log{\left[\frac{P(x)}{Q(x)}\right]} dx$$ How can one ...
sethaxen's user avatar
  • 173
14 votes
1 answer
3k views

How to generate 2 correlated Beta random variables

I was wondering if it might be possible to generate 2 correlated $Beta$ random variables? In other words, I want to generate two Beta random variables which can be said to have come from two Beta ...
rnorouzian's user avatar
  • 4,036
14 votes
1 answer
876 views

Bound the difference between Spearman's Correlation and Kendall's Correlation

I am trying to prove or disprove that the difference between Spearman's Correlation and Kendall's Correlation is no more than 1 (or less, the tighter the merrier). I am assuming there are no ties. ...
Pqqwetiqe's user avatar
  • 241
14 votes
0 answers
2k views

Negative deviance explained by GAM with betareg in R

I am fitting the following model in "mgcv" package in R using option family=betar to predict a percentage cover response variable (...
jatalah's user avatar
  • 483
13 votes
0 answers
244 views

Logistic regression for classification: are there any analytical solutions for the out-of-sample accuracy?

I run a binary logistic regression, with a binary dependent variable and a continuous independent one. Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. ...
robertspierre's user avatar
13 votes
0 answers
2k views

Are these equivalent (for p-values): super-uniform, stochastically larger than / dominating the uniform, conservative?

In the literature and online, I have found three different wordings that I think refer to the same concept: stochastically larger than uniform (which I take is ...
dlaehnemann's user avatar
13 votes
0 answers
707 views

Interpreting regression coefficients based on Andrew Gelman's re-scaling method

I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model. I have come ...
ksroogl's user avatar
  • 413
13 votes
0 answers
18k views

How to find p value using estimate and standard error?

I am trying to check statement on p. 23 of Data Analysis Using Regression and Multilevel/Hierarchical Models For example, consider two independent studies with effect estimates and standard errors ...
wildfluss's user avatar
  • 131
13 votes
0 answers
704 views

Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$ Here the kernel $K$ is an assigned smooth ...
Tim's user avatar
  • 139k
13 votes
0 answers
368 views

Is autocorrelation not worth addressing with small N?

Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
gung - Reinstate Monica's user avatar
13 votes
2 answers
2k views

Scaling the backward variable in HMM Baum-Welch

I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
itzjustricky's user avatar
12 votes
0 answers
452 views

Are there Imbalanced learning problems where re-balancing/re-weighting demonstrably improves *accuracy*?

I have been looking into the imbalanced learning problem, where a classifier is often expected to be unduly biased in favour of the majority class. However, I am having difficulties identifying ...
Dikran Marsupial's user avatar
12 votes
0 answers
405 views

Official name of a common type of Bayesian simulation study

There is a kind of simulation study that is commonly used to validate an implementation of a Bayesian model: For independent replication $i = 1, ..., n$: Draw a set of "true" parameters ...
landau's user avatar
  • 267
12 votes
0 answers
2k views

Instrumental variables with interactions between endogenous variables

I have two endogenous variables $x_1$ and $x_2$ and am trying to estimate the following model: $$y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_{12} x_{12}$$ where $x_{12} = x_1\times x_2$. I'm ...
Biblot's user avatar
  • 171
12 votes
1 answer
338 views

Correlation between two binary variables within one categorical variable

The Problem: I have measured two binary variables within 1 categorical variable with 5 levels. Initially, I thought I'd be able to use Fisher's Exact test or some $N \times M \times K$ version of it. ...
baffled's user avatar
  • 571
12 votes
0 answers
2k views

Is sparsity of topics a necessary condition for latent Dirichlet allocation (LDA) to work

I have been playing with the hyper-parameters of the latent Dirichlet allocation (LDA) model and am wondering how sparsity of topic priors play a role in inference. I have not performed these ...
kedarps's user avatar
  • 3,572
12 votes
0 answers
767 views

What approaches use multiple eigenvectors in graph spectral clustering?

Background: In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by ...
highBandWidth's user avatar
12 votes
0 answers
2k views

Empirical Prediction interval for time series forecast based on quantile regression

As Gardner notes "almost all point forecasts are wrong", so prediction intervals (PI) are necessary to quantify uncertainty and help us make informed decisions. There exists theoretical PI, and in ...
forecaster's user avatar
  • 8,365
12 votes
0 answers
5k views

How to use formative indicators in covariance-based SEM with lavaan?

I'm trying to build a covariance-based structural equation model (SEM) using both reflective and formative specifications of latent variables. I use the sem ...
jhg's user avatar
  • 325
12 votes
0 answers
110 views

Words that estimate numerical proportions of a group

There seem to be more than a few papers that surround the use of certain words to define the probability of an event without actually using the numerical probability itself. For example, if you are ...
Brandon Bertelsen's user avatar
12 votes
0 answers
13k views

How to normalize data prior to computation of covariance matrix

In all my self-study, I have come across many different ways in which people seem to normalize their data, prior to the computation of the covariance matrix. I am confused as to what ways are 'correct'...
Creatron's user avatar
  • 1,665
12 votes
0 answers
1k views

Computing a bootstrap confidence interval for the prediction error with the percentile and the BCa method

I have two related questions regarding the computation of a non-parametric bootstrap confidence interval for the prediction error. Setting: I have a sample S from a data population P and a learner L, ...
Gitte's user avatar
  • 825
11 votes
0 answers
3k views

Varying dispersion parameter (=dispformula) in glmmTMB in R to account for heteroscedasticity that originates from one predictor

I struggle with understanding the dispersion model and dispersion parameter of glmmTMB , and could not find answers anywhere. I constructed a GLMM using ...
eab's user avatar
  • 111
11 votes
0 answers
1k views

Bootstrap Prediction Interval: which residuals to use and which method?

I ask this question referring to the post: Bootstrap prediction interval, where a step by step method for calculating the prediction interval for linear regression models is explained. In the ...
user2683832's user avatar
11 votes
1 answer
3k views

Feature selection using chi squared for continuous features

I'm looking at univariate feature selection. A method that is often described, is to look at the p-values for a $\chi^2$-test. However, I'm confused as to how this works for continuous variables. 1. ...
Jondiedoop's user avatar
11 votes
3 answers
2k views

Need advice on change point (step) detection

I have a time series with lots of steps/jumps (data file here). A plot is given below. I would like to subtract an appropriate value for each of these square wave features to bring them back down to ...
vibe's user avatar
  • 301
11 votes
1 answer
3k views

Expected value of softmax transformation of Gaussian random vector

Let $\mathbf w_1,\mathbf w_2,\ldots,\mathbf w_n \in \mathbb R^p$ and $\mathbf v \in \mathbb R^n$ be fixed vectors, and $\mathbf x \sim \mathcal N_p(\boldsymbol{\mu}, \mathbf{\Sigma})$ be an $p$-...
dohmatob's user avatar
  • 558
11 votes
1 answer
5k views

Generalized additive model: choosing between cubic and thin-plate splines

I am using the gam function (from the mgcv package) to model a continuous response (a soil nutrient) in relation to a continuous ...
michael's user avatar
  • 241
11 votes
1 answer
3k views

How to test for the difference in skewness of two samples?

I have two samples. From looking at their densities, one appears symmetrical and the other from some right-tailed distribution. I would like to test that the two do not have the same skewness (...
Tal Galili's user avatar
  • 21.7k
11 votes
0 answers
600 views

Does using bootstrapped samples improve parameter estimates for a fitted distribution?

The R package retimes has a function for fitting an ex-Gaussian distribution to a set of observations. The method involves taking multiple bootstrapped samples of the observations, and fitting the ex-...
Josh's user avatar
  • 213

15 30 50 per page
1
2 3 4 5
1456