Questions tagged [theory]

For questions about statistical theory. Always include a more specific tag as well.

Filter by
Sorted by
Tagged with
0
votes
2answers
167 views

Incorrectly rejecting a null hypothesis

What is the theoretical chance that we (incorrectly) reject the null hypothesis in the following case? ...
-1
votes
1answer
533 views

Expectation of a random variable and the indicator random variable proof

I need to show that $E[T 1_A] = E[T|A]P(A)$. What I've got so far is $E[T1_A] = \int_{\Omega}T.1_A dP = \int_AT dP$. Now I know that $\int_AdP = P(A)$ however I'm lost as to how do I get the ...
8
votes
3answers
5k views

How to Prove that an Event Occurs Infinitely Often (Almost Surely)?

Exercise: There is a fair 6-sided die and a biased coin that has probability p > 0 of coming up heads on each toss. The die gets rolled infinitely often, and whenever you roll a 6, you then toss ...
13
votes
3answers
684 views

Theoretical results behind Artificial Neural Networks

I have just covered Artificial Neural Networks on Coursera's Machine Learning course and I would like to know more theory behind them. I find the motivation that they mimic biology somewhat ...
20
votes
2answers
4k views

Non-normal distributions with zero skewness and zero excess kurtosis?

Mostly theoretical question. Are there any examples of non-normal distributions that has first four moment equal to those of normal? Could they exist in theory?
4
votes
2answers
4k views

Performing k-means clustering on a set of lines

I have a set of lines (y = numbers between 1 and 100, x= discrete) that I am trying to cluster to group similarly-shaped profiles. I have found that the profiles seem to cluster the cleanest when ...
3
votes
0answers
400 views

Generalization error of PCA and kernel PCA

I've been recently reading Shawe-Taylor et al. 2005, On the Eigenspectrum of the Gram Matrix and the Generalization Error of Kernel PCA, where the authors analyze the squared residual of kernel ...
4
votes
1answer
817 views

What is the VC Dimension of a Naive Bayes Classifier?

How do you calculate the VC dimension of a Naive Bayes classifier with say K features?
2
votes
0answers
111 views

Sample lower bound for binary classification in Linear Discriminant Analysis?

Below is a description of this problem: Suppose the label $Y\in\{1,0\}$ in binary classification satisfies $\Pr[Y=1]=\Pr[Y=0]=\frac{1}{2}$, and $p(X|Y=1)=\mathcal{N}(\mu_1,\Sigma)$, $p(X|Y=0)=\...
1
vote
0answers
269 views

MECE statistics

Consider a sample of a real quantity $X$. Say that we divide this sample into two mutually exclusive and collectively exhaustive (MECE) groups. Say we know the mean and median of these two groups, $\...
2
votes
0answers
501 views

Are complete sufficient statistics unique?

I'm under the impression that up to a one-to-one function complete sufficient statistics are unique. How can I show this?
1
vote
1answer
435 views

Estimating $R^2$ when some coefficients are forced (i.e., restricted coefficients)

I am running a regression in R, and wanted to find the right way to calculate the $R^2$. I have an identity that I am empirically testing with data that is y = x1 - x2 + x3 (unfortunately dont have an ...
1
vote
0answers
3k views

What is the definition of generalization error and its justification?

I was trying to understand rigorously what the goal of machine learning is. One could frame that one of the central goals of machine learning is to obtain the best possible function ever. But what ...
4
votes
0answers
142 views

Diferrencing vs Moving Average

Moving Average and differencing a series can both be used to remove seasonality. Does the difference of these two lie in the model they are used? Moving Average used in classical decomposition and ...
0
votes
1answer
731 views

Forecast error for AR and MA process

AR(p) process is denoted by: $X_t=\mu+\alpha_1(X_{t-1}-\mu)+\alpha_2(X_{t-2}-\mu)+...\alpha_p(X_{t-p}-\mu)+Z_t$ I don't understand forecast error. Let $\epsilon_{t+l}$ be the forecast error at $l$ ...
2
votes
3answers
8k views

SARIMA model equation

Can someone please tell me in the book here how is this SARIMA equation obtained? I know that AR(1)=$Y_t=\alpha_1Y_{t-1}+e_t$ Non Seasonal AR(1)=> $Y_t(1-\alpha_1B)=e_t$. My question is what ...
7
votes
3answers
664 views

What makes constant function an estimator?

This is a theoretical one. This question is inspired by recent question and discussion on bootstrap, where a constant estimator, i.e. a constant function $$f(x) = \lambda$$ was used as an example of ...
1
vote
0answers
25 views

Recaculating control limits when an out of control situation is displayed

When plotting control charts if a point plots outside the control limits then the reason for that point to plot outside is checked and if an assignable cause is found to be the cause then after ...
1
vote
0answers
28 views

Changing a target value that minimize the product being outside the specifications in quality control

I was doing problems in the article here. In the question on page 16 I don't understand part b) how to find a target value that minimize the product being outside the specifications. Can someone ...
2
votes
1answer
98 views

Distribution of Minimum of RVs

I'm having trouble seeing why for RVs $X_{1}, \ldots, X_{n}$ it is true that: $$Pr(min(X_{1}, \ldots, X_{n}) > x ) = Pr(X_{1} > x, \ldots, X_{n} > x)$$ In other words: Why is the event that ...
24
votes
1answer
3k views

Choosing among proper scoring rules

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
1
vote
1answer
89 views

Calculating $\bar c $ in C chart

In studying C chart I came accross this problem in Statistical Process control by Douglas .C.Montgomary. In this exercise it says: A control chart is used to control the fraction of non confirming....
1
vote
1answer
2k views

Auto correlation function of AR(p) process

I am doing a time series course and in the theory part there are few things I don't understand.In obtaining auto correlation function for AR(p) process it is done as: $$\newcommand{\Var}{{\rm Var}}\...
4
votes
0answers
113 views

Is there a way to visualize what “Minimal sufficiency” and “Completeness” of a statistic means? [duplicate]

As defined (for example, in Wikipedia): Completeness is a property of a statistic in relation to a model for a set of observed data. In essence, it is a condition which ensures that the parameters of ...
2
votes
1answer
205 views

Control limits in $\bar{x}$ and R chart

I am learning a Quality Control course and in theory part I don't understand calculating the control limits for $\bar{x}$ and R chart.I want to know, say when I calculate the R chart control limits no ...
2
votes
1answer
445 views

Definition of Scale median

Lehmann, in Theory of Point Estimation p.212 (and also on p.169), defines scale median as the solution to: $${E(X)I(X\le c)} = {E(X)I(X\ge c)}$$ given $X$ is a positive random variable, and ${E(X)}&...
3
votes
2answers
85 views

Why are econometric analyses valid when the subject of study is inherently different?

I am reading numerous articles pertaining to unemployment as references for my own work. Yet I've encountered many where they use long time series in countries which have had some sort of pertinent ...
0
votes
1answer
1k views

Ensemble in stochastic process

I am learning a time series and forecasting course.In the book "The Analysis of Time Series by Chris Chatfield" it says that We only have single outcome of the process and a single observation on ...
3
votes
2answers
737 views

Requirements for a valid neural network activation function?

What rules define a valid neural network activation function, excluding biological plausibility? What set of principles do softmax, rectified linear units, hyperbolic tangent, sigmoid, etc. follow? ...
8
votes
1answer
651 views

Polling vs averaging in Random Forest models

Why is it that for Random Forest we take the average vote from each classifier in the ensemble rather than the average probability from each classifier in the ensemble? Is there theory behind why ...
2
votes
1answer
165 views

Linear-time approximation to kernel SVM?

Scaling kernel support vector machines to large datasets is a very challenging problem. For linear SVMs, PEGASOS is able to learn efficiently online, so training time scales linearly with the size of ...
3
votes
0answers
79 views

Formalizing pdf using both discrete and continuous densities

I'm trying to formalize the probability density function for a rather simple process, but I'm having difficulty writing it precisely. Specifically, consider simulating a 1-D Gaussian random walk ...
0
votes
0answers
89 views

Most Important Stat Theory Concepts — Interview [duplicate]

I have an interview with a top company for a data scientist position. I was made aware that they will be testing probability/statistical theory concepts. So the question: If you had 1 hour tops ...
2
votes
2answers
735 views

Mixed variable, joint distribution, How do we know which one is continuous distribution, which one is discrete

If we have one continuous r.v. $x$ and a discrete r.v. $y$ which takes one of the two values $y_1$ and $y_2$. Let's say we know the prior probabilities $P(y_1)$ and $P(y_2)$. From Bayes theorem we ...
8
votes
1answer
146 views

Why is variability measured relative to a point?

Why are measures of dispersion calculated relative to some central point? Why wouldn't, for instance, all possible non-repeated, pairwise differences in the dataset be a valid measure of spread?
3
votes
2answers
806 views

Combining models for prediction based on residual performance

I have never read or seen someone do this before, so I wanted to pose the question here. Suppose I fit a basic linear model, $\text{price of house} = \beta_0 + \beta_1*\text{taxes} + \beta_2*\text{...
2
votes
3answers
588 views

Can a nuisance multi-class classifier do better than binary classifier?

This is rather a theoretical question in order to save the trouble in trying to do empirical testing and is part of a bet, so I hope I am right... Say there are M classes in the data BUT you want to ...
2
votes
1answer
481 views

How can you derive the sample size to obtain a particular type II error rate?

I often see that the sample size for a $z$-test to achieve a particular type II error rate $\beta$ (at a given significance level, $\alpha$) is: $$n = \frac{(Z_{1-\alpha/2}+Z_{1-\beta})^2\sigma^2}{\...
0
votes
0answers
51 views

Mixed model weight matrix

Consider Henderson’s classic mixed model equations: What is the nature of the weight matrix, S-1? I believe that this is composed of the covariance matrix, but at the same time it is diagonal. How ...
1
vote
1answer
115 views

Is it valid to apply large scale statistics to a single case?

I have a small black box that displays a 0 or a 1 every time I press its button. I can tell you, truthfully, that over the past 20,000,000,000 times it has been pressed, 19,999,999,990 times it has ...
0
votes
1answer
248 views

When Monte Carlo simulation can't be used to simulate a statistical system?

My question is simple. Which are the general conditions for which a Monte Carlo simulation can be used to represent a statistical system? Or conversely, which are the statistical system that cannot be ...
0
votes
0answers
277 views

Approximating a binomial distribution with a mixture normal

This is purely a theoretical question (I legitimately can't think of a real application), but if you wanted to approximate a binomial distributed variable with a two-component mixture normal, is there ...
0
votes
1answer
597 views

Non significant Pearson correlations included in hierarchical regression?

I would like to perform hierarchical regression in which all variables are based on previous research/theory. But when I perform Pearson correlations, I found that some variables did not correlate to ...
0
votes
8answers
784 views

Does the presence of an outlier increase the probability that another outlier will also be present on the same observation?

**Edit: (10/26/13) More clear (hopefully) mini-rewrites added at the bottom** I'm asking this from a theoretical/general standpoint - not one that applies to a specific use case. I was thinking about ...
20
votes
2answers
1k views

Is there a statistical application that requires strong consistency?

I was wondering if someone knows or if there exists an application in statistics in which strong consistency of an estimator is required instead of weak consistency. That is, strong consistency is ...
0
votes
2answers
2k views

Theoretical expected value and variance

Let $X$ be a random variable having expected value $\mu$ and variance $\sigma^2$. Find the Expected Value and Variance of $Y = \frac{X−\mu}{\sigma}$. I would like to show some progress I've made so ...
3
votes
2answers
260 views

How do we understand the relationship between independent probabilities and real-world independence?

From what I have come to understand, the events A and B are considered independent for purposes of probability theory when $$ p(A \cap B) = p(A) \cdot p(B) $$ Now, supposing I flip two coins. I ...
0
votes
0answers
107 views

Frequency of data: is quarterly data better than half-year data? Why?

Hi have a dataset which I constructed using quarterly observations (from bank accounts). I could have also used half-year or yearly data, but I chose quarterly because I thought that higher frequency ...
5
votes
1answer
61 views

What is the name/relevant details of this exponential-family related structure?

Suppose that $X$ comes from an exponential family $$ p_\theta(x) = h(x)\exp(\theta x - A(\theta)), $$ and that, conditional on $X$, $Y$ also comes from an exponential family of the form $$ p_\eta(y\...
4
votes
2answers
736 views

Similarity theory: Testing whether dimensions are separable or integral

Note: I'm not referring to linear separability. I've found the interesting comment in Edelman, Shahbazi: "Renewing the respect for similarity" that for integral dimensions, Euclidean distance is ...