Questions tagged [theory]

For questions about statistical theory. Always include a more specific tag as well.

Filter by
Sorted by
Tagged with
3
votes
0answers
60 views

How do we call a more extreme case of fat tails than a power law?

According to Wikipedia the most extreme case of a fat tail follows a power law: The most extreme case of a fat tail is given by a distribution whose tail decays like a power law. That is, if ...
0
votes
0answers
15 views

Conceptually, what happens when you take (A - B) and divide it by (1 - B)? What is it used for?

I've come across this transformation a lot in my advanced stats course and I'm curious as to what it accomplishes. The basic structure is: A - B --------- 1 - B Here's an example of its use when ...
3
votes
0answers
70 views

Statistical Theory of Power Sampling (within R code)

I am curious about the statistical theory behind power.prop.test in R. I have dived into the code behind it and detailed it in Latex format. My question here is: ...
0
votes
0answers
6 views

Bounding the distributional error introduced by bootstrapping

Suppose I have some data $x_N$ of `size' $N$ which were drawn according to some measure $P_N(x_N)$. I'm imagining $x_N$ as being any of: an exchangeable / iid sequence of length $N$ a stationary ...
0
votes
0answers
13 views

Sum of pure errors

We know that in a simple linear regression model that the sum of all the residuals is 0 but why is it that the sum of all the pure errors is also 0? Is there a relation between them?
0
votes
0answers
9 views

Does the quoted line mean that the 6 cars would have gained more miles? If not, what does it mean?

I am reading Jayant Deshpande's "Life Time Data: Statistical Models and Methods" book and while reading about Right Random Censoring, I read about this example. Example of Random (right) Censoring ...
1
vote
2answers
49 views

Compare predicted versus actual outcomes in a GLM

I read somewhere that you could compute a "residual value" for a GLM by taking the actual values of your response variable divided by the predicted value of that response variable. For example, ...
1
vote
3answers
30 views

What is the terminology for the length of time an individual data point represents?

I'm having trouble conceptualizing this or putting it into word(s). It has been bothering me off and on for months now. As an example (financial, as that's all I know), let's say that Sarah wants to ...
0
votes
0answers
9 views

Existence of an optimal learning algorithm for a given distribution

I am working on Exercise 3.8 from Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David. In it we consider the binary classification problem where $\mathcal{X}$ is a set of data ...
3
votes
1answer
41 views

The effect on expectation of biasing a distribution by a monotonic function

Let $$ x \sim g(x)$$ where $g(x)$ denotes the pdf of $x$. Let the pdf of another variable $x^*$ be denoted by $f(x^*)$ and let $$f(x^*) \propto g(x^*) z(x^*) $$ where $z(x^*)$ is a monotonic ...
1
vote
0answers
12 views

Large difference in contingency table level affecting Cramer's V?

The Wikipedia article for Cramer's V states the following without citation: Note that as chi-squared values tend to increase with the number of cells, the greater the difference between r (rows) ...
2
votes
2answers
35 views

Optimization equivalence

Can someone help me with the step by step demonstration of the following equivalence used in SVM: $$maximize: m = \frac{1} {\|w\|} \equiv minimize: m =\frac{1} {2}\|w\|^2 $$ I would be most grateful ...
0
votes
0answers
28 views

minimum replicate number for comparative metagenome

I am in the process of designing an experiment that will look for differences in the gut bacteria of sick and healthy animals. The question that has been troubling us is how many animals should we ...
1
vote
0answers
17 views

Who first invented/proposed/formulated the IQM (Interquartile Mean) and the Truncated/Trimmed Mean?

I've looked this up on google for quite some time but couldn't find the answer! Does any of you know who or how can I find who first proposed these statistics? How can access their respective ...
0
votes
0answers
13 views

Theory: When to use one multi-factorial ANOVA vs. many one-factorial ANOVAs

I was recently peer reviewing a paper that used statistical tests in an odd way that didn't seem right to me. However, I'm not an expert so just wanted to talk theoretically about some of the things I ...
0
votes
0answers
8 views

Understanding Spectral Density of a time series in terms of Regression on Sinusoids

This may be a very trivial question but I would like to understand spectrum density of a time series in terms of regression. This is what I know from reading on the net: A time series can be ...
1
vote
0answers
27 views

A good literature for statistical learning theory

any recommendations for good literature on statistical learning theory? I mean, something what goes into more details than Elements of Statistical Learning, in terms of losses, empirical error ...
3
votes
1answer
100 views

Maximum sample size for one-way ANOVA?

Lists of requirements for one-way ANOVA include the following: Samples should be mutually independent Samples should be from a population with a normal distribution Samples should have the same ...
0
votes
2answers
62 views

Can the AUC (logistic regression) be used as a measure of quality of the model? [duplicate]

If I have a good measure of the AUC, what does it tell me in relation to the model and its variables?
0
votes
1answer
22 views

How do you prove that the following 2 hypothesis classes' VC-dimensions are equal?

Given a hypothesis class $H=\{h:X\to\{0,1\}\}$. Let $c\in H$ be the correct predictor. Denote $H^c = \{c\Delta h:h\in H\}$, where $c\Delta h=(h\backslash c)\cup (c\backslash h)$. Please prove that ...
3
votes
2answers
69 views

Learning from multiple very varied data sets?

Suppose we have a set of objects $X$ (e.g. individual humans). Suppose also that humans can be described by a set of (potentially very high-dimensional) variables $V_i$, (e.g. $V_1$ is a picture of ...
1
vote
1answer
64 views

Learning problem when we have data from distributions $(p_i)$ when we care about (known) distribution $p^*$?

Suppose we have a dataset $D$ or multiple datasets $(D_i)$, with distributions $p_i:X\to \mathbb R$. Suppose there is another distribution $p^*$. All distributions are known, including $p^*$, but the $...
0
votes
1answer
39 views

What are some books that explain the origins of or principles behind common statistical methods?

A friend of mine asked whether I knew of such a book that offers both intuitive and formal explanations of how a wide range of statistical methods were originally derived. She wasn't looking for a ...
0
votes
0answers
42 views

Checking linearity

Suppose I want to see if Gaussian linear regression is an appropriate model for my data. Then let X be the design matrix (with each column corresponding to an explanatory variable). and let e be the ...
1
vote
2answers
41 views

understanding bayesian optimization: what is meant by dimension

What is meant by the dimension in bayesian optimization? In many papers it is stated that the dimension should be lower than 20 (then there are papers which are solving high-dimensional problem). But, ...
2
votes
1answer
36 views

Fitting Poisson Distribution To Censored Data Repeat Experiments

I am working on analyzing some data that come from the following experimental setup: Imagine doing repeated trials of an experiment where a lever is pulled and a random number shows up drawn from a ...
1
vote
0answers
28 views

Is it possible to train an RNN to predict projectile motion?

Projectile motion is given by a function $y = -9.81 x^2 + ax + b$ for some parameters $a$ and $b$. I'll simply assume for $x$ values to be distanced by 1, so $x_t = t$. I can then easily generate ...
0
votes
0answers
9 views

Is there a theorem that guarantees an infinitely many number of neural networks that satisfy universal approximation theorem (UAC)?

Given a empricial dataset and its target, Universal Approximation Theorem (UAC) guarantees the existence of at least one neural network that can memorize the target of the dataset. Is there any ...
1
vote
0answers
15 views

Understanding how to weight variables

I am trying to model the "recruitment" of a deer population where recruitment is a factor of "females counted in fall" and "fawns counted in fall". e.g femaleFall:fawnsFall = Recruitment. In some ...
1
vote
0answers
30 views

What is an example of a sample space in machine learning?

Let $X$ denote a random variable. Then from a rigorous mathematical perspective (books such as Durrett, Feller, Kolmogorov, etc.), $X$ is a function. $X: \Omega \to \mathbb{R}^n$. Domain of the ...
1
vote
1answer
19 views

Help understanding CART trees notation

I was reading the Decision Trees user guide of sklearn to understand some of the underlying mathematics behind trees. Everything was fine until I stumbled upon some notation I'm not understanding. ...
3
votes
0answers
85 views

Constructive mathematics and statistics [closed]

Usual non-constructive mathematics leads to some paradoxes (e.g. the Banach-Tarski paradox), which are directly related to the axiom of choice. In non-constructive mathematics, the axiom of choice (as ...
3
votes
1answer
86 views

P-values in multiple comparison t-test

I'm a biologist trying to teach my college class about the issue of multiple comparisons with t-tests. I've explained that the alpha value must be "combined" so to speak from each of the related ...
0
votes
0answers
14 views

Converting extrapolation point to interpolation point

Just wondering if this is possible via mathematical conversion. I thought some Kernels might do this in the kernel space. I got curious whether it's possible to create a general method that ...
1
vote
0answers
29 views

Is causal relationship between two variables a theory?

I was asked if Y causes Z a theory, and I noticed that I have a gap in my knowledge on that. I know that theory could be causal, descriptive or predictive in its explanation. However, a theory is an ...
1
vote
1answer
64 views

Parameter not estimating due to singular information matrix and mutually exclusive categories in R

I have some data that has two categorical variables that are somewhat correlated (there is a row and a column of zeros where the levels are mutually exclusive), similar to the tabulation below. ...
0
votes
1answer
60 views

Why does the No Free Lunch theorem imply that we must design algorithms to perform well on specific tasks?

I'm currently studying deep learning using the textbook Deep Learning (Goodfellow, Bengio, & Courville - 2015) and had a question regarding a concept of machine learning provided in the book. ...
1
vote
1answer
275 views

Can someone provide a brief explanation as to why reproducing kernel Hilbert space is so popular in machine learning?

I thought functional analysis was long thought to be old fashioned and generally a dead research area. It seems that all of a sudden there is a huge fascination with so-called reproducing kernel ...
0
votes
0answers
20 views

References for generalization bounds?

I'm looking for references (books, papers, lecture notes etc) on generalization bounds and their proofs. Specifically, I'm looking to fully understand the technique of defining a hypothesis class (or ...
2
votes
0answers
29 views

Why we say that a probability measure $P$ is defined on $(\Omega, \mathcal{F})$?

I was wondering why we say that a probability measure $P$ is defined on $(\Omega, \mathcal{F})$, being $\Omega$ the sample space and $\mathcal{F}$ one sigma-algebra, when actually we have that $P$ is ...
1
vote
0answers
722 views

How to manually calculate odds ratio for continuous variables?

In school, long before learning about logistic models, I've been taught how to calculate odds ratios by hand. Formula was based on a contingency table, just like this: This is very easy to ...
1
vote
0answers
24 views

Clarification on the concept of “General formulation of the problem of statistical inference” - Wald

I'm looking for clarification on one part of this definition, but also some feedback on my interpretation of the whole concept. The definition comes from the end of the first chapter of On the ...
1
vote
0answers
56 views

Combining subjoint distributions to create a larger joint distribution

I am trying to construct large joint distributions through smaller joint distributions and I'm not sure how to approach the literature. I am curious if there exists a function which can take n ...
2
votes
1answer
215 views

What is the point of a permutation F-test when all you need is one F-test for one-way ANOVA?

Say you have three groups, and each group has 5 observations. You can figure out if there is a significant difference between means with a simple one-way ANOVA. I read in my nonparametric book, one ...
5
votes
1answer
93 views

Find the values of $ a$ so that A is positive definite (p.d)

Let $A=(1-a)I_n + a J_n$ Find the values of $~a~$ so the Matrix is p.d ? Note:$~I_n~$ is the identity matrix and $~J_n$ is the $1's$ matrix. I know that $~~A$ is p.d $~iff~λ_i >0$ so, I need to ...
1
vote
1answer
57 views

Based on the ideas of Parameter Estimation and Fitting Probability Distributions, what stops us from making any function be a PDF(PMF)?

Currently I am doing an introduction to parameter estimation and fitting probability distributions to sets of data. So in a small synopsis what I understand the whole process to be like is the ...
2
votes
0answers
141 views

Does every loss function correspond to MLE/MAP

Many of the losses used in regression/classification tasks correspond to maximum likelihood estimation (MLE) or maximum aposteriori (MAP) under a specific data likelihood distribution $p(\mathbf{y}|X,\...
0
votes
1answer
43 views

Why isn't the least squares predictor $\Phi(\Phi^\top\Phi)^{-1}\Phi^\top$ simply the identity matrix? [duplicate]

Given target vector $y$. Want to predict it using linear regression $h(w) = w^Tx$ Let $\Phi$ be the least squares matrix, i.e., $\Phi = \begin{bmatrix} x_1^\top \\ \vdots\\ x_n^\top \end{bmatrix}$ ...
2
votes
0answers
40 views

Operator Theoretic Perspectives on Bayesian Inference

On the Wikipedia page for conjugate prior there is a section "Analogy with Eigenfunctions", which describes a metaphor where observation of data is seen as a "linear operator" on the space of ...
0
votes
0answers
31 views

multivariate linear regression without b_0 [duplicate]

I created a multivariate regression following the scheme $$y = \beta_0 + \sum^n_{i=1}\beta_i*x_i$$ and got an average deviation ofaround 5%. When I tried the regression without the $\beta_0$ I got a ...

1
2 3 4 5