Questions tagged [theory]

For questions about statistical theory. Always include a more specific tag as well.

76 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
203 views

Cox's Theorem: ignorance, objective priors, and the Mind Projection Fallacy

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
5
votes
0answers
141 views

Cox's Theorem: the necessity of (un)countably additivity

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
4
votes
0answers
66 views

How do we call a more extreme case of fat tails than a power law?

According to Wikipedia the most extreme case of a fat tail follows a power law: The most extreme case of a fat tail is given by a distribution whose tail decays like a power law. That is, if the ...
4
votes
0answers
142 views

Diferrencing vs Moving Average

Moving Average and differencing a series can both be used to remove seasonality. Does the difference of these two lie in the model they are used? Moving Average used in classical decomposition and ...
3
votes
0answers
26 views

Do variable-selection methods (e.g. Elastic Net; Lasso) invalidate theory-based models in fields where little is known?

I'm caught in a bind about the relationship between theoretical models about how the world works and statistical methods for accurately predicting an outcome in fields where little is known. I ...
3
votes
0answers
70 views

Statistical Theory of Power Sampling (within R code)

I am curious about the statistical theory behind power.prop.test in R. I have dived into the code behind it and detailed it in Latex format. My question here is: ...
3
votes
1answer
47 views

Statistical theory proof intuition (UMVU estimators)

I've been working through this problem in Theoretical Statistics by Keener, but could not solve it. I looked up the answer and I do understand why it's correct, but I don't understand what intuition ...
3
votes
1answer
390 views

Isn't the Solomonoff Universal Prior Biased Like Any Other?

The Solomonoff universal prior is fixed relative to a specific choice of universal Turing machine (UTM). Now, I understand that a UTM can simulate any other UTM, so that they assign a complexity to ...
3
votes
0answers
265 views

Pattern-mixture models

I am currently looking at pattern-mixture models but I don't seem to understand them and I wonder whether someone could help. I can see the model comes from the factorisation $ f(y,r;\phi, \theta)=...
3
votes
0answers
149 views

How does using Tukey's test correct for multiple comparison problem?

I am curious about the intuition behind the Tukey's HSD. I know that it is designed for post-hoc test(WHEN and HOW part), but I want to know underlying theory that justifies its usage(WHY part). To ...
3
votes
0answers
399 views

Generalization error of PCA and kernel PCA

I've been recently reading Shawe-Taylor et al. 2005, On the Eigenspectrum of the Gram Matrix and the Generalization Error of Kernel PCA, where the authors analyze the squared residual of kernel ...
3
votes
0answers
79 views

Formalizing pdf using both discrete and continuous densities

I'm trying to formalize the probability density function for a rather simple process, but I'm having difficulty writing it precisely. Specifically, consider simulating a 1-D Gaussian random walk ...
2
votes
0answers
29 views

Why we say that a probability measure $P$ is defined on $(\Omega, \mathcal{F})$?

I was wondering why we say that a probability measure $P$ is defined on $(\Omega, \mathcal{F})$, being $\Omega$ the sample space and $\mathcal{F}$ one sigma-algebra, when actually we have that $P$ is ...
2
votes
0answers
145 views

Does every loss function correspond to MLE/MAP

Many of the losses used in regression/classification tasks correspond to maximum likelihood estimation (MLE) or maximum aposteriori (MAP) under a specific data likelihood distribution $p(\mathbf{y}|X,\...
2
votes
0answers
41 views

Operator Theoretic Perspectives on Bayesian Inference

On the Wikipedia page for conjugate prior there is a section "Analogy with Eigenfunctions", which describes a metaphor where observation of data is seen as a "linear operator" on the space of ...
2
votes
0answers
25 views

Correcting for sample overlap (loss of statistical independence) in two z-tests

Short form: How can I correct two p-values computed from normal distributions to account for the fact that part of the samples overlap? Long form: I have a sequence of i.i.d RVs of length $n$. The ...
2
votes
2answers
75 views

High-level time-series question: How does one study a series' trend?

I want to understand a series' trend, not the deviations from the trend. I would like to do analysis on the trend, such as run a multivariate regression, but every time-series source I read online ...
2
votes
0answers
44 views

Does it ever make sense to talk about the confidence (faith) in a probability value?

Let us suppose that we want to know what weather will be tomorrow. We ask two meteorologists and both give us an identical probabilistic answer: It will rain with probability of 30% It will snow ...
2
votes
0answers
76 views

Maximum likelihood estimator with atoms

MLE (maximum likelihood estimation) can be defined mathematically for discrete or continuous variables. But there is a technical specificity about variables being neither discrete nor continuous. ...
2
votes
0answers
45 views

Name of most general operator on probabilities whose output are also probabilities?

Not sure if the title is correct phrasing, I'll try to explain. Consider finite domain discrete probabilities for simplicity and illustration. The n-probabilities are contained on a (n-1)-simplex ...
2
votes
0answers
47 views

Derivation for the second order expansion in the paper “Additive Logistic Regression: A Statistical View of Boosting”

I need help with a derivation. In the paper "Additive Logistic Regression: A Statistical View of Boosting". The authors make a second order approximation about $f(x)=0$ to the following function $J(...
2
votes
0answers
111 views

Sample lower bound for binary classification in Linear Discriminant Analysis?

Below is a description of this problem: Suppose the label $Y\in\{1,0\}$ in binary classification satisfies $\Pr[Y=1]=\Pr[Y=0]=\frac{1}{2}$, and $p(X|Y=1)=\mathcal{N}(\mu_1,\Sigma)$, $p(X|Y=0)=\...
2
votes
0answers
499 views

Are complete sufficient statistics unique?

I'm under the impression that up to a one-to-one function complete sufficient statistics are unique. How can I show this?
2
votes
0answers
2k views

The formula for covariance in terms of joint cdf

I want to show that $$\newcommand{\cov}{\operatorname{cov}}\newcommand{\d}{\mathrm{d}}\cov(x,y) = \iint (F_{X,Y}(x,y) - F_X(x)F_Y(y))\,\d x\,\d y$$ However, I have no idea how to start. I know that $...
1
vote
0answers
8 views

To what degree can distribution fitting of a response variable inform GLM family and link selection?

This is more of a theoretical question. I have a response variable that is best described by the Box-Cox Power Exponential distribution, but there is no way to really "run a GLM" with this information ...
1
vote
0answers
12 views

Large difference in contingency table level affecting Cramer's V?

The Wikipedia article for Cramer's V states the following without citation: Note that as chi-squared values tend to increase with the number of cells, the greater the difference between r (rows) ...
1
vote
0answers
17 views

Who first invented/proposed/formulated the IQM (Interquartile Mean) and the Truncated/Trimmed Mean?

I've looked this up on google for quite some time but couldn't find the answer! Does any of you know who or how can I find who first proposed these statistics? How can access their respective ...
1
vote
0answers
27 views

A good literature for statistical learning theory

any recommendations for good literature on statistical learning theory? I mean, something what goes into more details than Elements of Statistical Learning, in terms of losses, empirical error ...
1
vote
0answers
32 views

Is it possible to train an RNN to predict projectile motion?

Projectile motion is given by a function $y = -9.81 x^2 + ax + b$ for some parameters $a$ and $b$. I'll simply assume for $x$ values to be distanced by 1, so $x_t = t$. I can then easily generate ...
1
vote
0answers
15 views

Understanding how to weight variables

I am trying to model the "recruitment" of a deer population where recruitment is a factor of "females counted in fall" and "fawns counted in fall". e.g femaleFall:fawnsFall = Recruitment. In some ...
1
vote
0answers
31 views

What is an example of a sample space in machine learning?

Let $X$ denote a random variable. Then from a rigorous mathematical perspective (books such as Durrett, Feller, Kolmogorov, etc.), $X$ is a function. $X: \Omega \to \mathbb{R}^n$. Domain of the ...
1
vote
0answers
29 views

Is causal relationship between two variables a theory?

I was asked if Y causes Z a theory, and I noticed that I have a gap in my knowledge on that. I know that theory could be causal, descriptive or predictive in its explanation. However, a theory is an ...
1
vote
1answer
72 views

Parameter not estimating due to singular information matrix and mutually exclusive categories in R

I have some data that has two categorical variables that are somewhat correlated (there is a row and a column of zeros where the levels are mutually exclusive), similar to the tabulation below. ...
1
vote
0answers
764 views

How to manually calculate odds ratio for continuous variables?

In school, long before learning about logistic models, I've been taught how to calculate odds ratios by hand. Formula was based on a contingency table, just like this: This is very easy to ...
1
vote
0answers
24 views

Clarification on the concept of “General formulation of the problem of statistical inference” - Wald

I'm looking for clarification on one part of this definition, but also some feedback on my interpretation of the whole concept. The definition comes from the end of the first chapter of On the ...
1
vote
0answers
56 views

Combining subjoint distributions to create a larger joint distribution

I am trying to construct large joint distributions through smaller joint distributions and I'm not sure how to approach the literature. I am curious if there exists a function which can take n ...
1
vote
0answers
146 views

Is it true that normalizing the output of a ReLu feedforward Neural Network that its Rademacher Complexity becomes a constant?

I was trying to understand what happened with the Rademacher Complexity: $$ R_S(F) = \frac{1}{m} \mathbb E_{\sigma} [\sup_{f \in F}\sum^m_{i=1} \sigma_i f(z_i)] $$ or $$ R_{P,m} = \mathbb E_{s \...
1
vote
0answers
27 views

Adverserial Verification of an XGboost Classifier

This paper proposes an algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i.e., networks that provably satisfy some desired input-output properties. The ...
1
vote
1answer
24 views

Using largest gradient at every iteration: good strategy?

Assume you have access to an oracle, which, given a set of labeled data $D = \{(x_1, y_1), ..., (x_n, y_n))\}$ returns a single data point $d^j_{max} = (x_i, y_i)$ with the property that its gradient ...
1
vote
0answers
39 views

Process for finding UMVU estimator

I've been working on a problem from Theoretical Statistics: Topics for a Core Course by Keener. I spent a few hours on it making very little progress before caving and looking up the solution. I don't ...
1
vote
0answers
259 views

Optimal value for the intercept term in SVM

(note that this problem is different from this one, since the latter considers primal's Lagrangian) Hi, I am trying to figure out the SVM's dual problem. The primal problem is $${\displaystyle {\...
1
vote
0answers
61 views

Supervised learning when there is a true underlying model vs. supervised learning when there is no underlying model?

(I found a more concise way of expressing my question - I'm leaving the original wording below for reference purposes) Is there a difference in the way we approach supervised learning problems when ...
1
vote
0answers
35 views

Hypothesis Testing as a Markov Process

Consider the following multiple hypothesis testing problem. We have a finite family of distributions $\mathfrak{P}$ indexed by a set $\mathcal{V}$, that is $\mathfrak{P} = \{P_v, v \in \mathcal{V}\}$. ...
1
vote
0answers
85 views

Machine Learning Correlated Records

I am just getting started with machine learning and I have a question that I have had a hard time researching as I am not sure how to refer to it. If I have a dataset: ...
1
vote
0answers
100 views

How to simulate an ARMA process with a regressor

I'd like someone to explain me the model differences between this two type of simulation: Variables creation: ...
1
vote
0answers
9 views

Asymptotic Behavior of Probabilities mapped to Discrete Outcomes

I'm having trouble finding a solution to the following problem: Assume that there are $N$ observations and each observation is associated with $K$ probabilities. For each $i \in N$, the ...
1
vote
0answers
79 views

Alternatives to Job Sequencing using Optimization

I have to N jobs to be assigned in a sequence to a Machine/User. I know using optimization technique we can find an optimal sequence. But in my case there are lot of parameters (some to be minimized (...
1
vote
0answers
48 views

MLE: Does the scale of predictor variables affect whether the hessian is positive definite?

I am trying to fit a regression via maximum likelihood estimation, one of the regression terms involves $\beta_0e^{(\beta t)}$ where $t$ is measured in hours and has a range of 0 to 90 days. The ...
1
vote
0answers
48 views

Confidence band for mean of means

This is sort of a stats theory question, and I can't really convince myself of the correct way to view things: In normal regression, the confidence band (or interval) gives the bounds on the mean, ...
1
vote
0answers
269 views

MECE statistics

Consider a sample of a real quantity $X$. Say that we divide this sample into two mutually exclusive and collectively exhaustive (MECE) groups. Say we know the mean and median of these two groups, $\...