For questions about statistical theory. Always include a more specific tag as well.

learn more… | top users | synonyms

0
votes
0answers
15 views

Multinomial loss derivative in gradient boost

I'm struggling hard to understand the derivative Jerome Friedman uses to extend gradient boosting to the multiclass case using multinomial logistic loss in his paper on Gradient Boosting: Greedy ...
0
votes
0answers
7 views

Representer Theorem: Is the RKHS $H$ the feature space or the estimator space?

I don't know how theoretical this page is, but I don't know a better place to ask this. It is more an intuitive question than a formal one. In reproducing kernel hilbert space theory we normally have ...
1
vote
1answer
26 views

Survival in time (Kaplan Meier) when start time is unknown: is it possible and what methods exist?

Is it possible to do survival analysis if one does not know the time the studied subject has already been ‘at risk’? This is question is more theoretical than practical, as in practice you would ...
1
vote
0answers
24 views

Alternatives to Job Sequencing using Optimization

I have to N jobs to be assigned in a sequence to a Machine/User. I know using optimization technique we can find an optimal sequence. But in my case there are lot of parameters (some to be minimized ...
0
votes
0answers
14 views

Pattern-mixture models

I am currently looking at pattern-mixture models but I don't see to understand them and I wonder if someone could help. I can see the model comes from the factorisation $ f(y,r;\phi, ...
2
votes
1answer
32 views

Learning the joint probability distribution of 3 variables from partial observations

I have a dataset composed of 3 random variables X, Y and Z. However, at each sample one of the random variable remains hidden. As an arbitrary example, the observation 1 is $(x_1,y_1)$, the ...
0
votes
0answers
16 views

ANOVA multi-way classification: concise presentation

I have read ANOVA from chapter 11 of 'Statistical Inference' by Casella & Berger. Their presentation is concise. However they stop at one-way classification. Can you suggest some other (chapters ...
3
votes
1answer
49 views

Under what conditions are these mixed model formulations equivalent?

I see models for "mixed effects" (i.e., models with fixed as well as random factors) specified in the literature in two ways, and I'd like to understand the conditions under which they are ...
1
vote
0answers
16 views

MLE: Does the scale of predictor variables affect whether the hessian is positive definite?

I am trying to fit a regression via maximum likelihood estimation, one of the regression terms involves $\beta_0e^{(\beta t)}$ where $t$ is measured in hours and has a range of 0 to 90 days. The ...
1
vote
1answer
38 views

Uniquely determine the two parameters of a distribution given pre-determined probabilities for two disjoint subsets of its support

Let $F(x| \alpha, \beta)$ denote the cumulative density function of a probability distribution. Let $[a,b]$ and $[c,d]$ be two disjoint subsets of the support of $F$. Suppose that $F(b) - F(a) = p$ ...
3
votes
0answers
38 views

Cox's Theorem: the necessity of (un)countably additivity

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
0
votes
0answers
19 views

Cox's Theorem: controversy surrounding the proposition domain size

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
1
vote
0answers
16 views

Cox's Theorem: ignorance, objective priors, and the Mind Projection Fallacy

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
3
votes
1answer
23 views

Cox's Theorem: controversy surrounding universal comparability

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
3
votes
0answers
45 views

The variety of problems with Cox's Theorem [closed]

So I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
3
votes
2answers
64 views

The concept of efficiency

I have some problems in understanding the concept of efficiency as related to an estimator. My sources (Mukhopadhyay, 2000 and Casella, Berger, 2002) do not treat this argument as I expected since ...
3
votes
2answers
117 views

Sufficiency of the sample

Given a sample $X_1,.....,X_n$ with common pdf $f(x;\theta)$, it is clear that the sample $\boldsymbol X$ itself is a sufficient statistic for $\theta$. I am aware of the meaning of "containing all ...
0
votes
0answers
27 views

Relation about the vc dimension and required number of examples

I started to watch the video series of Yaser-Abu-Mostafa, from caltech. The video series can be found here. I have two questions. In lecture-7, the professor says that: There have to be at least ...
3
votes
1answer
59 views

Theoretical Justifications for Random Forest

Is there any theoretical justifications for Random Forests in high dimensions? I notice the work "Uniform Convergence of Random Forests via Adaptive Concentration" which shows generalization of RF ...
2
votes
0answers
37 views

Hoes does using Tukey's test correct for multiple comparison problem?

I am curious about the intuition behind the Tukey's HSD. I know that it is designed for post-hoc test(WHEN and HOW part), but I want to know underlying theory that justifies its usage(WHY part). To ...
2
votes
1answer
21 views

GLMs: What features are required for valid link functions and statistical distributions?

Theory question here: For Generalized Linear Models, what features do we require for a valid link function and a valid statistical distribution. I feel like the tendency of using exponential-family ...
7
votes
0answers
101 views

Bayesians' positions on inductive skepticism [closed]

Philosopher Marc Lange gives an overview (pdf) of the debate on Hume's Problem of induction. Chapter 9 (starting on p. 80) is called "Bayesian approaches". I understand it as: the justification for ...
0
votes
0answers
30 views

Inverse of Lag Operator

I'm new to the concept of a "lag operator" $L$ where $Ly_t=y_{t-1}$ for some sequence $\{y_t\}_t$. Question: How do you prove both equalities below: $$ (1-\lambda L)^{-1}=1+\lambda L+\lambda^2 ...
1
vote
0answers
42 views

Concrete example for using $o(n)$ [closed]

I understand $O$ and $o$ notation, but I'm having difficulty applying the theory to real data. I'm working with a theorem which holds if $p^2\log(p) = o(n)$. I'm trying to use this in a real data ...
1
vote
0answers
22 views

Confidence band for mean of means

This is sort of a stats theory question, and I can't really convince myself of the correct way to view things: In normal regression, the confidence band (or interval) gives the bounds on the mean, ...
1
vote
0answers
20 views

Role of Distribution , likelihood function, and MLE in regression ?

I'm currently a beginner in learning statistics. In regression part, we started to learn (software) using MLE method for estimating parameters in the models. In order to calculate MLE, likelihood ...
0
votes
2answers
65 views

Incorrectly rejecting a null hypothesis

What is the theoretical chance that we (incorrectly) reject the null hypothesis in the following case? ...
-1
votes
1answer
55 views

Expectation of a random variable and the indicator random variable proof

I need to show that $E[T 1_A] = E[T|A]P(A)$. What I've got so far is $E[T1_A] = \int_{\Omega}T.1_A dP = \int_AT dP$. Now I know that $\int_AdP = P(A)$ however I'm lost as to how do I get the ...
6
votes
4answers
2k views

How to Prove that an Event Occurs Infinitely Often (Almost Surely)?

Exercise: There is a fair 6-sided die and a biased coin that has probability p > 0 of coming up heads on each toss. The die gets rolled infinitely often, and whenever you roll a 6, you then ...
5
votes
1answer
230 views

Theoretical results behind Artificial Neural Networks

I have just covered Artificial Neural Networks on Coursera's Machine Learning course and I would like to know more theory behind them. I find the motivation that they mimic biology somewhat ...
13
votes
2answers
753 views

Non-normal distributions with zero skewness and zero excess kurtosis?

Mostly theoretical question. Are there any examples of non-normal distributions that has first four moment equal to those of normal? Could they exist in theory?
0
votes
0answers
17 views

What class of functions can be covered by beta functions in Bayesian statistics?

I was thinking about Bayesian statistics, and one thought bothered me: In Bayesian statistics, we assume that the pdf $p(x)$ can be described as: $p(x)=\int f(x|\theta)g(\theta)d\theta$ usually ...
2
votes
2answers
181 views

Performing k-means clustering on a set of lines

I have a set of lines (y = numbers between 1 and 100, x= discrete) that I am trying to cluster to group similarly-shaped profiles. I have found that the profiles seem to cluster the cleanest when ...
1
vote
0answers
88 views

Generalization error of PCA and kernel PCA

I've been recently reading Shawe-Taylor et al. 2005, On the Eigenspectrum of the Gram Matrix and the Generalization Error of Kernel PCA, where the authors analyze the squared residual of kernel ...
3
votes
1answer
143 views

What is the VC Dimension of a Naive Bayes Classifier?

How do you calculate the VC dimension of a Naive Bayes classifier with say K features?
2
votes
0answers
55 views

Sample lower bound for binary classification in Linear Discriminant Analysis?

Below is a description of this problem: Suppose the label $Y\in\{1,0\}$ in binary classification satisfies $\Pr[Y=1]=\Pr[Y=0]=\frac{1}{2}$, and $p(X|Y=1)=\mathcal{N}(\mu_1,\Sigma)$, ...
1
vote
0answers
96 views

MECE statistics

Consider a sample of a real quantity $X$. Say that we divide this sample into two mutually exclusive and collectively exhaustive (MECE) groups. Say we know the mean and median of these two groups, ...
2
votes
0answers
58 views

Are complete sufficient statistics unique?

I'm under the impression that up to a one-to-one function complete sufficient statistics are unique. How can I show this?
0
votes
1answer
263 views

Estimating $R^2$ when some coefficients are forced (i.e., restricted coefficients)

I am running a regression in R, and wanted to find the right way to calculate the $R^2$. I have an identity that I am empirically testing with data that is y = x1 - x2 + x3 (unfortunately dont have an ...
1
vote
0answers
915 views

What is the definition of generalization error and its justification?

I was trying to understand rigorously what the goal of machine learning is. One could frame that one of the central goals of machine learning is to obtain the best possible function ever. But what ...
3
votes
0answers
50 views

Diferrencing vs Moving Average

Moving Average and differencing a series can both be used to remove seasonality. Does the difference of these two lie in the model they are used? Moving Average used in classical decomposition and ...
0
votes
1answer
97 views

Forecast error for AR and MA process

AR(p) process is denoted by: $X_t=\mu+\alpha_1(X_{t-1}-\mu)+\alpha_2(X_{t-2}-\mu)+...\alpha_p(X_{t-p}-\mu)+Z_t$ I don't understand forecast error. Let $\epsilon_{t+l}$ be the forecast error at $l$ ...
2
votes
3answers
700 views

SARIMA model equation

Can someone please tell me in the book here how is this SARIMA equation obtained? I know that AR(1)=$Y_t=\alpha_1Y_{t-1}+e_t$ Non Seasonal AR(1)=> $Y_t(1-\alpha_1B)=e_t$. My question is what ...
5
votes
3answers
194 views

What makes constant function an estimator?

This is a theoretical one. This question is inspired by recent question and discussion on bootstrap, where a constant estimator, i.e. a constant function $$f(x) = \lambda$$ was used as an example of ...
1
vote
0answers
16 views

Recaculating control limits when an out of control situation is displayed

When plotting control charts if a point plots outside the control limits then the reason for that point to plot outside is checked and if an assignable cause is found to be the cause then after ...
1
vote
0answers
26 views

Changing a target value that minimize the product being outside the specifications in quality control

I was doing problems in the article here. In the question on page 16 I don't understand part b) how to find a target value that minimize the product being outside the specifications. Can someone ...
1
vote
1answer
75 views

Distribution of Minimum of RVs

I'm having trouble seeing why for RVs $X_{1}, \ldots, X_{n}$ it is true that: $$Pr(min(X_{1}, \ldots, X_{n}) > x ) = Pr(X_{1} > x, \ldots, X_{n} > x)$$ In other words: Why is the event that ...
8
votes
1answer
411 views

Justifying and choosing a proper scoring rule

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
1
vote
0answers
35 views

Calculating $\bar c $ in C chart

In studying C chart I came accross this problem in Statistical Process control by Douglas .C.Montgomary. In this exercise it says: A control chart is used to control the fraction of non ...
0
votes
1answer
187 views

Auto correlation function of AR(p) process

I am doing a time series course and in the theory part there are few things I don't understand.In obtaining auto correlation function for AR(p) process it is done as: AR(p)=$X_t = α_1X_{t−1} + ...