All Questions

67,273 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
37
votes
0answers
953 views

Are there statistical lessons from the "Bible Code" episode

Although this question is somewhat subjective, I hope it qualifies as a good subjective question according to the faq guidelines. It is based on a question that Olle Häggström asked me a year ago and ...
31
votes
0answers
1k views

Bound for Arithmetic Harmonic mean inequality for matrices?

NOTE: This question has originally been posted in MSE, but it did not generate any interest. It was first posted there, because the question itself is a pure matrix-algebra question. Nevertheless, ...
27
votes
0answers
1k views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
24
votes
1answer
1k views

Bootstrapping Generalized Least Squares

Scenario: Consider the use of bootstrapping to estimate the distribution of model parameters fitted per a linear or nonlinear generalized least squares model. In particular, assume there is a ...
22
votes
0answers
1k views

Fitting custom distributions by MLE

My question relates to fitting custom distributions in R but I feel it has enough of a probability element to remain on CV. I have an interesting set of data which has the following characteristics: ...
22
votes
1answer
554 views

Blind source separation of convex mixture?

Suppose I have $n$ independent sources, $X_1, X_2, ..., X_n$ and I observe $m$ convex mixtures: \begin{align} Y_1 &= a_{11}X_1 + a_{12}X_2 + \cdots + a_{1n}X_n\\ ...&\\ Y_m &= a_{m1}X_1 + ...
21
votes
0answers
710 views

Distribution of inverse Wishart to a power?

In a related question, I had asked about the norm induced by an inverse Wishart matrix. I am interested in generalizing that result somewhat. Let $A\sim\mathcal{W}_p\left(I,n\right)$, a Wishart matrix ...
18
votes
1answer
831 views

Physical/pictoral interpretation of higher-order moments

I'm preparing a presentation about parallel statistics. I plan to illustrate the formulas for distributed computation of the mean and variance with examples involving center of gravity and moment of ...
17
votes
0answers
805 views

Is there a general expression for ancillary statistics in exponential families?

An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
16
votes
0answers
9k views

When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?

Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?). The ...
16
votes
0answers
2k views

Implementation of CoVaR (a systemic risk measure) in R

I'm trying to estimate CoVaR using bivariate DCC GARCH in R. The concept of CoVaR is the dependence adjusted of VaR, which was first introduced by Adrian and Brunnermeier (2011). However, this ...
15
votes
0answers
312 views

Empirical Bayes (In)Admissibility

Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators. Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
15
votes
0answers
948 views

Estimation of ARMA: state space vs. alternatives

I am interested in estimation of ARMA models. I understand that a popular approach is to write the model down in the state space form and then maximize the likelihood of the model using some ...
15
votes
0answers
3k views

Gamma hurdle model for continuous response?

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of ...
15
votes
0answers
1k views

Picking block length in a block bootstrap

I am using the Mann-Kendall test to assess trends in a data time-series. I believe there is autocorrelation in my data and therefore need to use a block bootstrap to correct for it. I have plotted ...
15
votes
1answer
660 views

Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
14
votes
0answers
210 views

Is bootstrap problematic in small samples?

In "3 Things That Bother Me" (1988), Ed Leamer writes: Bootstrap estimates of standard errors are based on the assumption that the observed sample is the same as the true distribution, ...
14
votes
1answer
20k views

Standardization vs. Normalization for Lasso/Ridge Regression

I am aware it is common practice to standardize the features for ridge and lasso regression, however, would it ever be more practical to normalize the features on a (0,1) scale as an alternative to z-...
14
votes
0answers
466 views

Asymptotic property of tuning parameter in penalized regression

I'm currently working on asymptotic properties of penalized regression. I've read a myriad of papers by now, but there is an essential issue that I cannot get my head around. To keep things simple, I'...
14
votes
0answers
2k views

The role of scale parameter in GEE

I am learning the generalized estimating equations (GEE) and the geepack R package. There are some questions that I am a little confused. In a GEE-constructed ...
14
votes
1answer
455 views

Can I use optimally scaled variables for a factor analysis to account for rotation? If I can then how?

I have discussed this issue several times in this site, but I am asking it again for a final justification from the experts of our community. I wanted to extract four factors (I should call dimensions ...
14
votes
1answer
5k views

How to compute confidence interval in ANOVA with repeated measures?

I made a model using repeated measures univariate ANOVA in R. ...
13
votes
1answer
1k views

How to measure the reliability of a consensus ranking (problem from Kemeny-Snell book)

Suppose that $k$ experts are each asked to rank a set of $n$ objects in order or preference. Let allow ties in the rankings. John Kemeny and Laurie Snell in their 1962 year book "Mathematical models ...
13
votes
1answer
1k views

Tail inequality on sum of product of normal variables

For independent random variables $ x_1,..,x_n$ and $y_1,...,y_n$ following normal distribution $N(0,1)$, I need a simple estimate formula for $P(|\sum_1^n x_iy_i|\ge nt) \leq e^{(?)}$ for $t>1$. ...
13
votes
0answers
1k views

What is Shannon's source entropy?

Suppose that ${X_n; Y_n}$ is a random process with a discrete alphabet, that is, taking on values in a discrete set for $n$ data length. They correspond to the input and output of a communication ...
13
votes
0answers
742 views

How can I measure model performance with weighted logistic regression?

I am working with some survey data that uses probability weights. A number of sources explain that likelihood-based tests and fit statistics like likelihood-ratio, AIC, and BIC are not valid in the ...
13
votes
0answers
1k views

Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
13
votes
1answer
518 views

Are non-square latin hypercubes viable?

At https://github.com/OpenMDAO/OpenMDAO-Framework/issues/599 it is stated that non-square Latin Hypercube experimental design is not well defined (I assume that for higher dimensions that means ...
12
votes
0answers
172 views

What is the "direct likelihood" point of view in statistics?

I am reading a Springer title from 1997 called Applied Generalized Linear Models by James K. Lindsey. In the preface, Lindsey writes For this text, the reader is assumed to have knowledge of basic ...
12
votes
0answers
337 views

Validity of confidence interval for $\rho$ when $X\sim N_3(0,\Sigma)$ with $\Sigma_{ij}=\rho^{|i-j|}$

Suppose $X\sim N_3(0,\Sigma)$, where $\Sigma=\begin{pmatrix}1&\rho&\rho^2\\\rho&1&\rho\\\rho^2&\rho&1\end{pmatrix}$. On the basis of one observation $x=(x_1,x_2,x_3)'$, I have ...
12
votes
3answers
596 views

Adjustments to (Linear Regression) Forecast

Full disclosure: I am not a statistician, nor do I claim to be one. I am a lowly IT administrator. Please play gentle with me. :) I am responsible for collecting and forecasting disk storage use ...
12
votes
0answers
257 views

What is tantile regression?

My question follows on this discussion of medials and tantiles vs medians and quantiles from earlier this year: When would we use tantiles and the medial, rather than quantiles and the median? As ...
12
votes
0answers
2k views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
12
votes
1answer
2k views

How to compare forecasting methods?

I have several intermittent data. Based on those data, I would like to compare several forecasting methods (Exponential Smoothing, Moving Average, Croston, and Syntetos-Boylan), and decide whether ...
12
votes
1answer
672 views

Visualizing many left-skewed distributions

I have a series of left-skewed/heavy tailed distributions that I would like to show. There are 42 distributions across three factors (labeled as A, ...
12
votes
0answers
369 views

Penalized spline confidence intervals based on cluster-sandwich VCV

This is my first post here, but I've benefited a lot from this forum's results popping up in google search results. I've been teaching myself semi-parametric regression using penalized splines. ...
11
votes
1answer
3k views

RNN vs Kalman filter : learning the underlying dynamics?

Being recently interested in Kalman filters and Recurrent neural networks, it appears to me that the two are closely related, yet I can't find relevant enough litterature : In a Kalman filter, the ...
11
votes
0answers
547 views

Convolutional neural network for multi-variate time series?

I want to use CNN architectures for classification of multivariate time-series, where we apply one label to each sequence. I searched the net for the available designs in the literature and i found ...
11
votes
0answers
528 views

Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$ Here the kernel $K$ is an assigned smooth ...
11
votes
1answer
609 views

Bound the difference between Spearman's Correlation and Kendall's Correlation

I am trying to prove or disprove that the difference between Spearman's Correlation and Kendall's Correlation is no more than 1 (or less, the tighter the merrier). I am assuming there are no ties. ...
11
votes
1answer
2k views

Eigenfunctions and eigenvalues of the exponential kernel

What are the eigenfunctions and the eigenvalues of the exponential kernel? The exponential kernel is defined as $$k(x,x')=\sigma^2\exp\left(\frac{||x-x'||}{l}\right)$$ where both $\sigma>0$ and $l&...
11
votes
0answers
3k views

Interpreting and reporting gamm4 result

I am new to gam, and most of my knowledge comes from this document http://www3.nd.edu/~mclark19/learn/GAMS.pdf. Now I am using generalized addictive model with random effects to model some data, where ...
11
votes
2answers
2k views

Scaling the backward variable in HMM Baum-Welch

I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
11
votes
0answers
4k views

How to compare two distance matrices?

Suppose that I have two distance matrices for the same set of items. By a distance matrix I mean a square matrix whose (i,j)th entry holds the distance (in terms of cosine similarity) between ith and ...
11
votes
1answer
7k views

Confidence Intervals for AUC using cross-validation

I am analyzing the performance of a predictive model with the AUC, area under the ROC curve. I repeat several times cross-validation, and I have different estimations of the AUC in each folder. For ...
10
votes
0answers
692 views

Instrumental variables with interactions between endogenous variables

I have two endogenous variables $x_1$ and $x_2$ and am trying to estimate the following model: $$y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_{12} x_{12}$$ where $x_{12} = x_1\times x_2$. I'm ...
10
votes
0answers
429 views

When can a Gaussian Process solve an SDE?

Considering an SDE of the form $$dX_t = \mu(X_t, t)dt + \sigma(X_t, t)dW_t ,$$ where $W_t$ is a Wiener process, is there a set of necessary and sufficient conditions on the structure of the functions $...
10
votes
3answers
214 views

Is there a way to recover a temporal dependence structure in a time series from a regression against time?

Consider a time series: $X_1,X_2,...X_{n-1},X_n$ This series can also be written as a function of time $X(t)$, so that: $X_1,X_2,...X_{n-1},X_n = X(t_1),X(t_2),...X(t_{n-1}),X(t_n)$ Most ...
10
votes
0answers
1k views

Are there unbiased, non-linear estimators with lower variance than the OLS estimator?

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$ The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...
10
votes
1answer
2k views

Nested cross validation vs repeated k-fold

I know there are many topics(1,2,3), papers(1,2,3) and websites(1) that discuss this topic at length. However for the past two days I am reading all I can find about the subject and seems I hit a ...

15 30 50 per page
1
2 3 4 5
1346