All Questions

73,762 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
26 votes
4 answers
887 views

Blind source separation of convex mixture?

Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures: \begin{align} Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\ ...&\\ Y_m &= a_{m1}X_1 + ...
anonymous's user avatar
  • 825
23 votes
0 answers
1k views

Is there a general expression for ancillary statistics in exponential families?

An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
Henry.L's user avatar
  • 2,380
22 votes
0 answers
17k views

When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?

Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?). The ...
MiniQuark's user avatar
  • 2,170
21 votes
1 answer
1k views

Estimation of ARMA: state space vs. alternatives

I am interested in estimation of ARMA models. I understand that a popular approach is to write the model down in the state space form and then maximize the likelihood of the model using some ...
Richard Hardy's user avatar
20 votes
1 answer
969 views

Physical/pictoral interpretation of higher-order moments

I'm preparing a presentation about parallel statistics. I plan to illustrate the formulas for distributed computation of the mean and variance with examples involving center of gravity and moment of ...
James Koppel's user avatar
19 votes
0 answers
477 views

Empirical Bayes (In)Admissibility

Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators. Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
Cagdas Ozgenc's user avatar
19 votes
0 answers
2k views

Implementation of CoVaR (a systemic risk measure) in R

I'm trying to estimate CoVaR using bivariate DCC GARCH in R. The concept of CoVaR is the dependence adjusted of VaR, which was first introduced by Adrian and Brunnermeier (2011). However, this ...
drawar's user avatar
  • 265
19 votes
0 answers
2k views

Picking block length in a block bootstrap

I am using the Mann-Kendall test to assess trends in a data time-series. I believe there is autocorrelation in my data and therefore need to use a block bootstrap to correct for it. I have plotted ...
caitiekay's user avatar
  • 191
18 votes
0 answers
701 views

Is the Wilcoxon two-sample test maximally powered to detect proportional odds alternatives?

We know from the literature that The Wilcoxon-Mann-Whitney two-sample rank sum test is optimal for detecting simple location shifts when comparing two continuous random variables that each have a ...
Frank Harrell's user avatar
18 votes
2 answers
917 views

Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
slotishtype's user avatar
17 votes
1 answer
521 views

Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?

I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
Blain Waan's user avatar
  • 3,585
17 votes
1 answer
946 views

How can I measure model performance with weighted logistic regression?

I am working with some survey data that uses probability weights. A number of sources explain that likelihood-based tests and fit statistics like likelihood-ratio, AIC, and BIC are not valid in the ...
biased_estimator's user avatar
16 votes
0 answers
1k views

what is the mistake of convergence proof in Adam

Sashank J. Reddi et. al in their paper "On the convergence of Adam and beyond" say that, Adam's proof of convergence as stated in original paper is wrong. More than that, they point out that the value ...
Виталик Бушаев's user avatar
16 votes
3 answers
786 views

Adjustments to (Linear Regression) Forecast

Full disclosure: I am not a statistician, nor do I claim to be one. I am a lowly IT administrator. Please play gentle with me. :) I am responsible for collecting and forecasting disk storage use ...
sbrown's user avatar
  • 261
16 votes
0 answers
383 views

What is tantile regression?

My question follows on this discussion of medials and tantiles vs medians and quantiles from earlier this year: When would we use tantiles and the medial, rather than quantiles and the median? As ...
Mike Hunter's user avatar
  • 10.2k
16 votes
0 answers
2k views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
axk's user avatar
  • 798
16 votes
2 answers
931 views

Visualizing many left-skewed distributions

I have a series of left-skewed/heavy tailed distributions that I would like to show. There are 42 distributions across three factors (labeled as A, ...
topepo's user avatar
  • 5,960
16 votes
0 answers
543 views

Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...
Nick Sabbe's user avatar
  • 12.5k
16 votes
0 answers
1k views

What is Shannon's source entropy?

Suppose that ${X_n; Y_n}$ is a random process with a discrete alphabet, that is, taking on values in a discrete set for $n$ data length. They correspond to the input and output of a communication ...
chk's user avatar
  • 419
16 votes
0 answers
5k views

How to compare two distance matrices?

Suppose that I have two distance matrices for the same set of items. By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
Ahmet Yılmaz's user avatar
15 votes
1 answer
749 views

Are non-square latin hypercubes viable?

At https://github.com/OpenMDAO/OpenMDAO-Framework/issues/599 it is stated that non-square Latin Hypercube experimental design is not well defined (I assume that for higher dimensions that means ...
naught101's user avatar
  • 5,233
15 votes
0 answers
13k views

Time series regression with overlapping data

I am seeing a regression model which is regressing Year-on-Year stock index returns on lagged (12 months) Year-on-Year returns of the same stock index, credit spread (difference between monthly mean ...
Vishal Belsare's user avatar
14 votes
0 answers
667 views

Convolutional neural network for multi-variate time series?

I want to use CNN architectures for classification of multivariate time-series, where we apply one label to each sequence. I searched the net for the available designs in the literature and i found ...
Bob's user avatar
  • 439
14 votes
0 answers
4k views

What is the KL divergence of distribution from Dirac delta?

The Kullback–Leibler (KL) divergence of two continuous distributions $P(x)$ and $Q(x)$ is defined as $$D_{KL}(P \mid\mid Q) = \int_{X} P(x) \log{\left[\frac{P(x)}{Q(x)}\right]} dx$$ How can one ...
sethaxen's user avatar
  • 173
14 votes
0 answers
968 views

Generalization of Fisher information for a discrete parameter

This is mainly a reference request. There must be some generalizations of the concept of Fisher information for discrete (say, integer-valued) parameters, and of related results such as the Cramer-...
kjetil b halvorsen's user avatar
14 votes
0 answers
3k views

Interpreting and reporting gamm4 result

I am new to gam, and most of my knowledge comes from this document http://www3.nd.edu/~mclark19/learn/GAMS.pdf. Now I am using generalized addictive model with random effects to model some data, where ...
nan's user avatar
  • 1,095
14 votes
0 answers
1k views

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
jkndrkn's user avatar
  • 683
14 votes
0 answers
454 views

Penalized spline confidence intervals based on cluster-sandwich VCV

This is my first post here, but I've benefited a lot from this forum's results popping up in google search results. I've been teaching myself semi-parametric regression using penalized splines. ...
generic_user's user avatar
  • 13.2k
14 votes
1 answer
12k views

Regress residuals in second regression

I am wondering if anyone can point me to a paper/lecture notes on the rationale behind first running an OLS on a set of variables, and then in a second regression using the residuals of that ...
Z_D's user avatar
  • 223
13 votes
0 answers
223 views

Logistic regression for classification: are there any analytical solutions for the out-of-sample accuracy?

I run a binary logistic regression, with a binary dependent variable and a continuous independent one. Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. ...
robertspierre's user avatar
13 votes
0 answers
693 views

Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$ Here the kernel $K$ is an assigned smooth ...
Tim's user avatar
  • 135k
13 votes
1 answer
3k views

How to generate 2 correlated Beta random variables

I was wondering if it might be possible to generate 2 correlated $Beta$ random variables? In other words, I want to generate two Beta random variables which can be said to have come from two Beta ...
rnorouzian's user avatar
  • 3,826
13 votes
1 answer
839 views

Bound the difference between Spearman's Correlation and Kendall's Correlation

I am trying to prove or disprove that the difference between Spearman's Correlation and Kendall's Correlation is no more than 1 (or less, the tighter the merrier). I am assuming there are no ties. ...
Pqqwetiqe's user avatar
  • 131
13 votes
0 answers
352 views

Is probability fundamentally about reference classes (real or imagined)?

Question: It seems that frequentism and Bayesianism may not really be different as far as the the ultimate basis for what a probability is (relative frequency within a reference class) - it's just ...
user avatar
13 votes
0 answers
2k views

Negative deviance explained by GAM with betareg in R

I am fitting the following model in "mgcv" package in R using option family=betar to predict a percentage cover response variable (...
jatalah's user avatar
  • 473
12 votes
0 answers
379 views

Are there Imbalanced learning problems where re-balancing/re-weighting demonstrably improves *accuracy*?

I have been looking into the imbalanced learning problem, where a classifier is often expected to be unduly biased in favour of the majority class. However, I am having difficulties identifying ...
Dikran Marsupial's user avatar
12 votes
0 answers
1k views

How to calculate percent partial deviance explained by each predictor variable in a GAM model?

I am trying to find a sensible way to calculate the deviance explained by each predictor variable in a GAM model and need some input on my calculations. Following Simon Wood's example on the thread ...
Isabella Ghement's user avatar
12 votes
0 answers
1k views

Instrumental variables with interactions between endogenous variables

I have two endogenous variables $x_1$ and $x_2$ and am trying to estimate the following model: $$y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_{12} x_{12}$$ where $x_{12} = x_1\times x_2$. I'm ...
Biblot's user avatar
  • 171
12 votes
0 answers
671 views

Interpreting regression coefficients based on Andrew Gelman's re-scaling method

I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model. I have come ...
ksroogl's user avatar
  • 403
12 votes
0 answers
17k views

How to find p value using estimate and standard error?

I am trying to check statement on p. 23 of Data Analysis Using Regression and Multilevel/Hierarchical Models For example, consider two independent studies with effect estimates and standard errors ...
wildfluss's user avatar
  • 121
12 votes
0 answers
2k views

Is sparsity of topics a necessary condition for latent Dirichlet allocation (LDA) to work

I have been playing with the hyper-parameters of the latent Dirichlet allocation (LDA) model and am wondering how sparsity of topic priors play a role in inference. I have not performed these ...
kedarps's user avatar
  • 3,482
12 votes
0 answers
339 views

Is autocorrelation not worth addressing with small N?

Consider a simple regression context in which there is a small set of response values, $Y$, and corresponding dates, $X$. (For simplicity, we can assume the dates are equally spaced.) We would like ...
gung - Reinstate Monica's user avatar
12 votes
0 answers
750 views

What approaches use multiple eigenvectors in graph spectral clustering?

Background: In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by ...
highBandWidth's user avatar
12 votes
0 answers
2k views

Empirical Prediction interval for time series forecast based on quantile regression

As Gardner notes "almost all point forecasts are wrong", so prediction intervals (PI) are necessary to quantify uncertainty and help us make informed decisions. There exists theoretical PI, and in ...
forecaster's user avatar
  • 8,043
12 votes
0 answers
5k views

How to use formative indicators in covariance-based SEM with lavaan?

I'm trying to build a covariance-based structural equation model (SEM) using both reflective and formative specifications of latent variables. I use the sem ...
jhg's user avatar
  • 325
12 votes
0 answers
3k views

glmer in R: Significance estimates are not robust to order of data frame

I'm using a mixed effects model with logistic link function (using lme4 version 1.1-7 in R). However, I noticed that the estimates of significance for fixed effects change depending on the order of ...
Sean Roberts's user avatar
12 votes
2 answers
2k views

Scaling the backward variable in HMM Baum-Welch

I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
itzjustricky's user avatar
12 votes
0 answers
13k views

How to normalize data prior to computation of covariance matrix

In all my self-study, I have come across many different ways in which people seem to normalize their data, prior to the computation of the covariance matrix. I am confused as to what ways are 'correct'...
Creatron's user avatar
  • 1,615
12 votes
1 answer
4k views

How do I identify the "Long Tail" portion of my distribution?

I have a number of series that would typically be described as normal skewed or Gamma distributed. For example, in a group of customers I may have calculated their spend over a fixed length of time. ...
Adam L's user avatar
  • 121
12 votes
1 answer
307 views

Unit testing sampling methods

I'm writing a bit of code that makes pretty heavy use of sampling (eg, MCMC, Particle Filters, etc), and I would really like to test it to make sure that it's doing what I think it is before claiming ...
duckworthd's user avatar

15 30 50 per page
1
2 3 4 5
1476