For questions about statistical theory. Always include a more specific tag as well.

learn more… | top users | synonyms

0
votes
2answers
47 views

Incorrectly rejecting a null hypothesis

What is the theoretical chance that we (incorrectly) reject the null hypothesis in the following case? ...
-1
votes
1answer
37 views

Expectation of a random variable and the indicator random variable proof

I need to show that $E[T 1_A] = E[T|A]P(A)$. What I've got so far is $E[T1_A] = \int_{\Omega}T.1_A dP = \int_AT dP$. Now I know that $\int_AdP = P(A)$ however I'm lost as to how do I get the ...
6
votes
4answers
810 views

How to Prove that an Event Occurs Infinitely Often (Almost Surely)?

Exercise: There is a fair 6-sided die and a biased coin that has probability p > 0 of coming up heads on each toss. The die gets rolled infinitely often, and whenever you roll a 6, you then ...
3
votes
1answer
135 views

Theoretical results behind Artificial Neural Networks

I have just covered Artificial Neural Networks on Coursera's Machine Learning course and I would like to know more theory behind them. I find the motivation that they mimic biology somewhat ...
2
votes
0answers
16 views

Minimum dimension of sufficient statistics

Suppose that we have a parameter of k-dimensions. Say, for example, for normal (u,theta) both unknown then the parameter is of two dimensions, and n i.i.d. observations. Is it possible to find a ...
11
votes
2answers
277 views

Non-normal distributions with zero skewness and zero excess kurtosis?

Mostly theoretical question. Are there any examples of non-normal distributions that has first four moment equal to those of normal? Could they exist in theory?
0
votes
0answers
15 views

What class of functions can be covered by beta functions in Bayesian statistics?

I was thinking about Bayesian statistics, and one thought bothered me: In Bayesian statistics, we assume that the pdf $p(x)$ can be described as: $p(x)=\int f(x|\theta)g(\theta)d\theta$ usually ...
2
votes
2answers
68 views

Performing k-means clustering on a set of lines

I have a set of lines (y = numbers between 1 and 100, x= discrete) that I am trying to cluster to group similarly-shaped profiles. I have found that the profiles seem to cluster the cleanest when ...
0
votes
0answers
25 views

Term importance from document with predefined weight

I am having a set of document with different value weight on them. I am trying to understand which term from the documents trigger the highest values. I have a theory on how to do it and I would like ...
1
vote
0answers
45 views

Generalization error of PCA and kernel PCA

I've been recently reading Shawe-Taylor et al. 2005, On the Eigenspectrum of the Gram Matrix and the Generalization Error of Kernel PCA, where the authors analyze the squared residual of kernel ...
0
votes
0answers
38 views

What is the VC Dimension of a Naive Bayes Classifier?

How do you calculate the VC dimension of a Naive Bayes classifier with say K features?
2
votes
0answers
38 views

Sample lower bound for binary classification in Linear Discriminant Analysis?

Below is a description of this problem: Suppose the label $Y\in\{1,0\}$ in binary classification satisfies $\Pr[Y=1]=\Pr[Y=0]=\frac{1}{2}$, and $p(X|Y=1)=\mathcal{N}(\mu_1,\Sigma)$, ...
0
votes
0answers
17 views

Information matrix at Wald statistic

AFAIK, the Wald statistic of ML estimator uses the limit normality of MLE, and it looks as: assume to test $H_0 : \theta = \theta_0$ $T = (\hat{\theta} - \theta_0)' (I(\theta_0))^{-1} (\hat{\theta} ...
1
vote
0answers
64 views

MECE statistics

Consider a sample of a real quantity $X$. Say that we divide this sample into two mutually exclusive and collectively exhaustive (MECE) groups. Say we know the mean and median of these two groups, ...
1
vote
0answers
32 views

Are complete sufficient statistics unique?

I'm under the impression that up to a one-to-one function complete sufficient statistics are unique. How can I show this?
0
votes
0answers
18 views

Estimate total number of errors from a sample of independent error probabilities

I have a dataset in which I have linked clients by their name and date of birth. I want to estimate the number of errors this will create, i.e. the number of times I will incorrectly assign as 1 ...
0
votes
0answers
23 views

To be significant or to be stabile, what's more scientfically important?

Recently I discovered the techniques related to cross validation. Basically you can split up your data in n groups and then run your model on one part of the data and assess prediction reliability on ...
0
votes
1answer
151 views

Estimating $R^2$ when some coefficients are forced (i.e., restricted coefficients)

I am running a regression in R, and wanted to find the right way to calculate the $R^2$. I have an identity that I am empirically testing with data that is y = x1 - x2 + x3 (unfortunately dont have an ...
0
votes
0answers
345 views

What is the definition of generalization error and its justification?

I was trying to understand rigorously what the goal of machine learning is. One could frame that one of the central goals of machine learning is to obtain the best possible function ever. But what ...
3
votes
0answers
42 views

Diferrencing vs Moving Average

Moving Average and differencing a series can both be used to remove seasonality. Does the difference of these two lie in the model they are used? Moving Average used in classical decomposition and ...
0
votes
1answer
69 views

Forecast error for AR and MA process

AR(p) process is denoted by: $X_t=\mu+\alpha_1(X_{t-1}-\mu)+\alpha_2(X_{t-2}-\mu)+...\alpha_p(X_{t-p}-\mu)+Z_t$ I don't understand forecast error. Let $\epsilon_{t+l}$ be the forecast error at $l$ ...
1
vote
2answers
288 views

SARIMA model equation

Can someone please tell me in the book here how is this SARIMA equation obtained? I know that AR(1)=$Y_t=\alpha_1Y_{t-1}+e_t$ Non Seasonal AR(1)=> $Y_t(1-\alpha_1B)=e_t$. My question is what ...
5
votes
3answers
143 views

What makes constant function an estimator?

This is a theoretical one. This question is inspired by recent question and discussion on bootstrap, where a constant estimator, i.e. a constant function $$f(x) = \lambda$$ was used as an example of ...
1
vote
0answers
14 views

Recaculating control limits when an out of control situation is displayed

When plotting control charts if a point plots outside the control limits then the reason for that point to plot outside is checked and if an assignable cause is found to be the cause then after ...
1
vote
0answers
24 views

Changing a target value that minimize the product being outside the specifications in quality control

I was doing problems in the article here. In the question on page 16 I don't understand part b) how to find a target value that minimize the product being outside the specifications. Can someone ...
1
vote
1answer
70 views

Distribution of Minimum of RVs

I'm having trouble seeing why for RVs $X_{1}, \ldots, X_{n}$ it is true that: $$Pr(min(X_{1}, \ldots, X_{n}) > x ) = Pr(X_{1} > x, \ldots, X_{n} > x)$$ In other words: Why is the event that ...
5
votes
1answer
215 views

Justifying and choosing a proper scoring rule

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
0
votes
0answers
38 views

How to determine if a U chart or a C chart to be used

This is regarding theory of attribute type control charts.In a question it is given data for the number of non confirmities per 1000 meters of wire. I have to calculate control limits for a ...
1
vote
0answers
23 views

Calculating $\bar c $ in C chart

In studying C chart I came accross this problem in Statistical Process control by Douglas .C.Montgomary. In this exercise it says: A control chart is used to control the fraction of non ...
0
votes
0answers
20 views

Area of Opportunity in C chart and U chart

In the article whether to use C chart of U chart is determined by area of opportunity.But I don't understand what is meant by area of opportunity and how it relates to C chart and U chart.Can ...
0
votes
1answer
101 views

Auto correlation function of AR(p) process

I am doing a time series course and in the theory part there are few things I don't understand.In obtaining auto correlation function for AR(p) process it is done as: AR(p)=$X_t = α_1X_{t−1} + ...
3
votes
0answers
51 views

Is there a way to visualize what “Minimal sufficiency” and “Completeness” of a statistic means?

As defined (for example, in Wikipedia): Completeness is a property of a statistic in relation to a model for a set of observed data. In essence, it is a condition which ensures that the parameters of ...
2
votes
1answer
56 views

Control limits in $\bar{x}$ and R chart

I am learning a Quality Control course and in theory part I don't understand calculating the control limits for $\bar{x}$ and R chart.I want to know, say when I calculate the R chart control limits no ...
1
vote
1answer
65 views

Definition of Scale median

Lehmann, in Theory of Point Estimation p.212 (and also on p.169), defines scale median as the solution to: $${E(X)I(X\le c)} = {E(X)I(X\ge c)}$$ given $X$ is a positive random variable, and ...
0
votes
0answers
28 views

Regression using a priori knowledge

I am sorry if the title (and probably the question) is not very clear but I have a regression problem which might be a bit over my head if I want to do it well. I am only interested in getting some ...
3
votes
2answers
78 views

Why are econometric analyses valid when the subject of study is inherently different?

I am reading numerous articles pertaining to unemployment as references for my own work. Yet I've encountered many where they use long time series in countries which have had some sort of pertinent ...
0
votes
1answer
86 views

Ensemble in stochastic process

I am learning a time series and forecasting course.In the book "The Analysis of Time Series by Chris Chatfield" it says that We only have single outcome of the process and a single observation on ...
0
votes
0answers
39 views

Statistical learning theory

I am looking for some good books for statistical learning theory. For an introduction I went through "An elementary introduction statistical learning theory Kulakrani" It was a good read with less ...
1
vote
1answer
61 views

Requirements for a valid neural network activation function?

What rules define a valid neural network activation function, excluding biological plausibility? What set of principles do softmax, rectified linear units, hyperbolic tangent, sigmoid, etc. follow? ...
4
votes
0answers
49 views

Polling vs averaging in Random Forest models

Why is it that for Random Forest we take the average vote from each classifier in the ensemble rather than the average probability from each classifier in the ensemble? Is there theory behind why ...
0
votes
0answers
29 views

Linear-time approximation to kernel SVM?

Scaling kernel support vector machines to large datasets is a very challenging problem. For linear SVMs, PEGASOS is able to learn efficiently online, so training time scales linearly with the size of ...
0
votes
0answers
13 views

Intuition for uniform integrability as asymptotic light tails

I am trying to get an intuition to the concept of uniform integrability: $$ \lim_m \sup\lim_n \mathbb{E}[|X_n| I_{\{X_n \geq m \}}] \to 0 $$ Can this be seen as a form of "asymptotic light tails"? ...
2
votes
1answer
132 views

Logistic regression & categorical explanatory variables

I'd like to perform logistic regression with some categorical explanatory variables with more categories than just binary 0/1. Is this possible and why? I am inclined to think that this would give ...
3
votes
0answers
51 views

Formalizing pdf using both discrete and continuous densities

I'm trying to formalize the probability density function for a rather simple process, but I'm having difficulty writing it precisely. Specifically, consider simulating a 1-D Gaussian random walk ...
0
votes
0answers
41 views

Most Important Stat Theory Concepts — Interview [duplicate]

I have an interview with a top company for a data scientist position. I was made aware that they will be testing probability/statistical theory concepts. So the question: If you had 1 hour tops ...
1
vote
2answers
61 views

Mixed variable, joint distribution, How do we know which one is continuous distribution, which one is discrete

If we have one continuous r.v. $x$ and a discrete r.v. $y$ which takes one of the two values $y_1$ and $y_2$. Let's say we know the prior probabilities $P(y_1)$ and $P(y_2)$. From Bayes theorem we ...
8
votes
1answer
126 views

Why is variability measured relative to a point?

Why are measures of dispersion calculated relative to some central point? Why wouldn't, for instance, all possible non-repeated, pairwise differences in the dataset be a valid measure of spread?
3
votes
2answers
100 views

Combining models for prediction based on residual performance

I have never read or seen someone do this before, so I wanted to pose the question here. Suppose I fit a basic linear model, $\text{price of house} = \beta_0 + \beta_1*\text{taxes} + ...
1
vote
2answers
98 views

Can a nuisance multi-class classifier do better than binary classifier?

This is rather a theoretical question in order to save the trouble in trying to do empirical testing and is part of a bet, so I hope I am right... Say there are M classes in the data BUT you want to ...
2
votes
1answer
96 views

How can you derive the sample size to obtain a particular type II error rate?

I often see that the sample size for a $z$-test to achieve a particular type II error rate $\beta$ (at a given significance level, $\alpha$) is: $$n = ...