A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [theory]

For questions about statistical theory. Always include a more specific tag as well.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Theory: When to use one multi-factorial ANOVA vs. many one-factorial ANOVAs

I was recently peer reviewing a paper that used statistical tests in an odd way that didn't seem right to me. However, I'm not an expert so just wanted to talk theoretically about some of the things I ...
0
votes
0answers
8 views

Understanding Spectral Density of a time series in terms of Regression on Sinusoids

This may be a very trivial question but I would like to understand spectrum density of a time series in terms of regression. This is what I know from reading on the net: A time series can be ...
1
vote
0answers
26 views

A good literature for statistical learning theory

any recommendations for good literature on statistical learning theory? I mean, something what goes into more details than Elements of Statistical Learning, in terms of losses, empirical error ...
3
votes
1answer
20 views

Maximum sample size for one-way ANOVA?

Lists of requirements for one-way ANOVA include the following: Samples should be mutually independent Samples should be from a population with a normal distribution Samples should have the same ...
0
votes
2answers
38 views

Can the AUC (logistic regression) be used as a measure of quality of the model? [duplicate]

If I have a good measure of the AUC, what does it tell me in relation to the model and its variables?
0
votes
1answer
15 views

How do you prove that the following 2 hypothesis classes' VC-dimensions are equal?

Given a hypothesis class $H=\{h:X\to\{0,1\}\}$. Let $c\in H$ be the correct predictor. Denote $H^c = \{c\Delta h:h\in H\}$, where $c\Delta h=(h\backslash c)\cup (c\backslash h)$. Please prove that ...
3
votes
2answers
65 views

Learning from multiple very varied data sets?

Suppose we have a set of objects $X$ (e.g. individual humans). Suppose also that humans can be described by a set of (potentially very high-dimensional) variables $V_i$, (e.g. $V_1$ is a picture of ...
1
vote
1answer
60 views

Learning problem when we have data from distributions $(p_i)$ when we care about (known) distribution $p^*$?

Suppose we have a dataset $D$ or multiple datasets $(D_i)$, with distributions $p_i:X\to \mathbb R$. Suppose there is another distribution $p^*$. All distributions are known, including $p^*$, but the $...
0
votes
1answer
30 views

What are some books that explain the origins of or principles behind common statistical methods?

A friend of mine asked whether I knew of such a book that offers both intuitive and formal explanations of how a wide range of statistical methods were originally derived. She wasn't looking for a ...
0
votes
0answers
38 views

Checking linearity

Suppose I want to see if Gaussian linear regression is an appropriate model for my data. Then let X be the design matrix (with each column corresponding to an explanatory variable). and let e be the ...
1
vote
2answers
34 views

understanding bayesian optimization: what is meant by dimension

What is meant by the dimension in bayesian optimization? In many papers it is stated that the dimension should be lower than 20 (then there are papers which are solving high-dimensional problem). But, ...
2
votes
1answer
28 views

Fitting Poisson Distribution To Censored Data Repeat Experiments

I am working on analyzing some data that come from the following experimental setup: Imagine doing repeated trials of an experiment where a lever is pulled and a random number shows up drawn from a ...
1
vote
0answers
14 views

Is it possible to train an RNN to predict projectile motion?

Projectile motion is given by a function $y = -9.81 x^2 + ax + b$ for some parameters $a$ and $b$. I'll simply assume for $x$ values to be distanced by 1, so $x_t = t$. I can then easily generate ...
0
votes
0answers
7 views

Is there a theorem that guarantees an infinitely many number of neural networks that satisfy universal approximation theorem (UAC)?

Given a empricial dataset and its target, Universal Approximation Theorem (UAC) guarantees the existence of at least one neural network that can memorize the target of the dataset. Is there any ...
1
vote
0answers
14 views

Understanding how to weight variables

I am trying to model the "recruitment" of a deer population where recruitment is a factor of "females counted in fall" and "fawns counted in fall". e.g femaleFall:fawnsFall = Recruitment. In some ...
1
vote
0answers
28 views

What is an example of a sample space in machine learning?

Let $X$ denote a random variable. Then from a rigorous mathematical perspective (books such as Durrett, Feller, Kolmogorov, etc.), $X$ is a function. $X: \Omega \to \mathbb{R}^n$. Domain of the ...
1
vote
1answer
17 views

Help understanding CART trees notation

I was reading the Decision Trees user guide of sklearn to understand some of the underlying mathematics behind trees. Everything was fine until I stumbled upon some notation I'm not understanding. ...
3
votes
0answers
78 views

Constructive mathematics and statistics [closed]

Usual non-constructive mathematics leads to some paradoxes (e.g. the Banach-Tarski paradox), which are directly related to the axiom of choice. In non-constructive mathematics, the axiom of choice (as ...
3
votes
1answer
63 views

P-values in multiple comparison t-test

I'm a biologist trying to teach my college class about the issue of multiple comparisons with t-tests. I've explained that the alpha value must be "combined" so to speak from each of the related ...
0
votes
0answers
12 views

Converting extrapolation point to interpolation point

Just wondering if this is possible via mathematical conversion. I thought some Kernels might do this in the kernel space. I got curious whether it's possible to create a general method that ...
1
vote
0answers
29 views

Is causal relationship between two variables a theory?

I was asked if Y causes Z a theory, and I noticed that I have a gap in my knowledge on that. I know that theory could be causal, descriptive or predictive in its explanation. However, a theory is an ...
1
vote
1answer
37 views

Parameter not estimating due to singular information matrix and mutually exclusive categories in R

I have some data that has two categorical variables that are somewhat correlated (there is a row and a column of zeros where the levels are mutually exclusive), similar to the tabulation below. ...
0
votes
1answer
53 views

Why does the No Free Lunch theorem imply that we must design algorithms to perform well on specific tasks?

I'm currently studying deep learning using the textbook Deep Learning (Goodfellow, Bengio, & Courville - 2015) and had a question regarding a concept of machine learning provided in the book. ...
1
vote
1answer
132 views

Can someone provide a brief explanation as to why reproducing kernel Hilbert space is so popular in machine learning?

I thought functional analysis was long thought to be old fashioned and generally a dead research area. It seems that all of a sudden there is a huge fascination with so-called reproducing kernel ...
0
votes
0answers
12 views

Studying the relationship between non-inependent variables

What are some approaches to studying the relationship between two non-independent variables? When doing linear regression to study the relationship between two variables, it is assumed that your "x" ...
0
votes
0answers
16 views

References for generalization bounds?

I'm looking for references (books, papers, lecture notes etc) on generalization bounds and their proofs. Specifically, I'm looking to fully understand the technique of defining a hypothesis class (or ...
2
votes
0answers
29 views

Why we say that a probability measure $P$ is defined on $(\Omega, \mathcal{F})$?

I was wondering why we say that a probability measure $P$ is defined on $(\Omega, \mathcal{F})$, being $\Omega$ the sample space and $\mathcal{F}$ one sigma-algebra, when actually we have that $P$ is ...
1
vote
0answers
560 views

How to manually calculate odds ratio for continuous variables?

In school, long before learning about logistic models, I've been taught how to calculate odds ratios by hand. Formula was based on a contingency table, just like this: This is very easy to ...
1
vote
0answers
23 views

Clarification on the concept of “General formulation of the problem of statistical inference” - Wald

I'm looking for clarification on one part of this definition, but also some feedback on my interpretation of the whole concept. The definition comes from the end of the first chapter of On the ...
1
vote
0answers
53 views

Combining subjoint distributions to create a larger joint distribution

I am trying to construct large joint distributions through smaller joint distributions and I'm not sure how to approach the literature. I am curious if there exists a function which can take n ...
2
votes
1answer
123 views

What is the point of a permutation F-test when all you need is one F-test for one-way ANOVA?

Say you have three groups, and each group has 5 observations. You can figure out if there is a significant difference between means with a simple one-way ANOVA. I read in my nonparametric book, one ...
0
votes
0answers
18 views

Regarding a Proof in ''Robustness and Generalization'' by Xu. and Mannor

I am trying to understand the mathematics of the paper "Robustness and Generalization". The problem that I am facing is probably very simple but I am just not able to wrap my head around it. There ...
5
votes
1answer
92 views

Find the values of $ a$ so that A is positive definite (p.d)

Let $A=(1-a)I_n + a J_n$ Find the values of $~a~$ so the Matrix is p.d ? Note:$~I_n~$ is the identity matrix and $~J_n$ is the $1's$ matrix. I know that $~~A$ is p.d $~iff~λ_i >0$ so, I need to ...
1
vote
1answer
55 views

Based on the ideas of Parameter Estimation and Fitting Probability Distributions, what stops us from making any function be a PDF(PMF)?

Currently I am doing an introduction to parameter estimation and fitting probability distributions to sets of data. So in a small synopsis what I understand the whole process to be like is the ...
2
votes
0answers
95 views

Does every loss function correspond to MLE/MAP

Many of the losses used in regression/classification tasks correspond to maximum likelihood estimation (MLE) or maximum aposteriori (MAP) under a specific data likelihood distribution $p(\mathbf{y}|X,\...
0
votes
1answer
40 views

Why isn't the least squares predictor $\Phi(\Phi^\top\Phi)^{-1}\Phi^\top$ simply the identity matrix? [duplicate]

Given target vector $y$. Want to predict it using linear regression $h(w) = w^Tx$ Let $\Phi$ be the least squares matrix, i.e., $\Phi = \begin{bmatrix} x_1^\top \\ \vdots\\ x_n^\top \end{bmatrix}$ ...
2
votes
0answers
39 views

Operator Theoretic Perspectives on Bayesian Inference

On the Wikipedia page for conjugate prior there is a section "Analogy with Eigenfunctions", which describes a metaphor where observation of data is seen as a "linear operator" on the space of ...
0
votes
0answers
31 views

multivariate linear regression without b_0 [duplicate]

I created a multivariate regression following the scheme $$y = \beta_0 + \sum^n_{i=1}\beta_i*x_i$$ and got an average deviation ofaround 5%. When I tried the regression without the $\beta_0$ I got a ...
6
votes
2answers
363 views

ELI5: The Logic Behind Coefficient Estimation in OLS Regression

Like a lot of people, I understand how to run a linear regression, I understand how to interpret its output, and I understand its limitations. My understanding of the mathematical underpinnings of ...
0
votes
1answer
110 views

Type I and Type II errors in Hypothesis Testing

I am confused about the last highlighted sentence regarding finding a subset S, for which BOTH Type I and Type II error probabilities are 0. For $P_\theta S$, which is the probability with which we ...
0
votes
1answer
163 views

R: how is the Pr(>|z|) in the results of glm.fit calculated and why?

I've been searching but I can't find anywhere an explanation of how the Pr(>|z|) column is calculated in the results of R's glm.fit function. I would really appreciate: a) an explanation so I can ...
0
votes
1answer
38 views

How to prove the distribution of Generalised Instrumental Variables Estimator

$\hat { \beta } _ { GI V } = ( X ^ { \prime } Z ( Z ^ { \prime } Z ) ^ { - 1 } Z ^ { \prime } X ) ^ { - 1 } X ^ { \prime } Z ( Z ^ { \prime } Z ) ^ { - 1 } Z^ { \prime }y\hspace{35pt}(a)$ $(a)$ is ...
2
votes
0answers
21 views

Correcting for sample overlap (loss of statistical independence) in two z-tests

Short form: How can I correct two p-values computed from normal distributions to account for the fact that part of the samples overlap? Long form: I have a sequence of i.i.d RVs of length $n$. The ...
5
votes
1answer
190 views

Uniqueness of Reproducing Kernel Hilbert Spaces

Digging in the definition of Reproducing Kernel Hilbert Spaces (RKHS) I came across the following example taken from pages 49-51 of [1]: Given the kernel $k(x,y) = \langle x,y\rangle^2$, with $x,y\in ...
1
vote
0answers
127 views

Is it true that normalizing the output of a ReLu feedforward Neural Network that its Rademacher Complexity becomes a constant?

I was trying to understand what happened with the Rademacher Complexity: $$ R_S(F) = \frac{1}{m} \mathbb E_{\sigma} [\sup_{f \in F}\sum^m_{i=1} \sigma_i f(z_i)] $$ or $$ R_{P,m} = \mathbb E_{s \...
2
votes
0answers
53 views

Where can I read about the theory behind GLMs in their most general form? [duplicate]

I was following along with the MIT Opencourseware "Statistics for Applications" and their analysis of GLMs only covers discussion of GLMs whose dependent variable $y$ is distributed according to a ...
0
votes
1answer
93 views

How to handle 'unfairness' of missing data in machine learning?

Let me explain what I mean by unfairness. Let's say I have a multi-class classification problem where I am trying to predict the 'best drug' (among multiple candidate drugs) for each patient. So ...
2
votes
1answer
3k views

XGBoost implementation for unbalanced data using scale_pos_weight parameter

I have a confusion regarding how cost sensitive custom metric can be used for training of unbalanced dataset (two class 0 and 1) in XGBoost. Metric: Cost = 10*#of false positives + 500*# of false ...
1
vote
0answers
26 views

Adverserial Verification of an XGboost Classifier

This paper proposes an algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i.e., networks that provably satisfy some desired input-output properties. The ...
2
votes
1answer
119 views

Why fixed design analysis for observational data [duplicate]

Why do we use a fixed design analysis of regression coefficients, even for observational data, where the design is not fixed? For instance: $Var[\hat \beta]=(X'X)^{-1}\sigma^2$ is conditional on $X$. ...