Mathematical theory of statistics, concerned with formal definitions and general results.

learn more… | top users | synonyms (1)

0
votes
0answers
17 views

Z Score Problem. Help!

I am confused on finding the z-score for this question: Suppose researchers are interested in estimating the prevalence of HIV in a population of adolescent intravenous drug users determined to ...
0
votes
0answers
16 views

Matrix Calculations and Multivariate Bayesian Methods

Suppose I have the matrix given by: $$X = \begin{bmatrix}1 & 0 & 0\\ 1 & 1 & 0 \\ 1 & 1 & 1 \end{bmatrix}$$ This matrix actually represents whether a user interacted with a ...
1
vote
0answers
32 views

Implicit estimator, it's variance and expectaion

Denote with $V_j$ the firm's value of $j$-th firm. We will speak of default, if the firm's value falls below predefined barrier $c \in \mathbb{R}$. The random variable $L_j$ will indicate, if $j$-th ...
0
votes
0answers
7 views

Evaluation of a binary classifier in regards of rounding, taking average scores

I'm using scikit-learn to evaluate a binary classifier: There are 2 Methods to get the Precision, Recall and F1 scores: 1.classification_report: ...
1
vote
1answer
122 views

Is it always true that $P(X| Y \text{ OR } Z) \le P(X|Y)$?

Consider the following argument: If $(X| Y \ \text{OR} \ Z)$ is true, $(X| Y)$ must be true. For example, if $f(t)=10 $ when $ t=1 $ or $ \ t=0$ is true, then $f(t)=10 $ when $ t=1$ must be true ...
1
vote
0answers
13 views

Does a random sample size produce a larger or smaller variance than a fixed sample size?

If you run two identical samples with the exception of the sample size where in both cases E(n)=1000, but in one the variance is 0 (ie. fixed sample size of 1000) and the other has non-zero variance, ...
0
votes
0answers
15 views

compare the similarity of two cluster results using igraph

i am comparing the results of two clusters using igraph, such as What is the intuition behind the variation of information (VI) metric for cluster validation? I see there were consistency for all ...
3
votes
1answer
25 views

Geometry of Rejection regions for tests (NP lemma, Karl-Rubin, UMPU, LMP, etc)?

The Neyman-Pearson lemma, the Karlin-Rubin theorem, and the other for UMPU tests for the exponential family, etc. They all define a most powerful Rejection Region (RR), in a certain class of tests. I ...
3
votes
1answer
22 views

A uniformly optimal statistical procedure? an exercise from The Bayesian Choice

The previous exercise is from the book 'The Bayesian Choice', page 87. What does the author mean by uniformly optimal stat. procedure? This exercise refers to a Decision theory chapter, in a section ...
0
votes
0answers
7 views

Comparing relative error measures

So I have a list of actual values and two models to predict these values. Evaluating the predictions gives me a relative (absolute) error of 50% percent for the first and 25% for the seconds. I'm ...
0
votes
0answers
8 views

Whitening transform for the case of two stochastic processes

The classic whitening transform allows us to find a linear transformation for a given random process X, yielding a new process Xw with a unity Covariance matrix. Given an extended problem with two ...
0
votes
0answers
7 views

Objects and properties classification

I have a list of properties and objects: ...
0
votes
0answers
57 views

Prediction Algorithms in Statistics

Here it is what I'm looking for: Having a random variable, with an unknown distribution of its values, is there any smart algorithm that can predict the next value of the variable based on n samples ...
5
votes
1answer
75 views

Calculating probability of an event involving two normally distributed random variables

I have two normally distributed, independent random variables $S^{(1)}$ and $S^{(2)}$: $$S^{(1)} \sim \mathcal{N}(\mu_1, \sigma_1^2),\ S^{(2)} \sim \mathcal{N}(\mu_2, \sigma_2^2)$$ Given two positive ...
2
votes
0answers
13 views

What does “Borel space”, unqualified, refer to? [migrated]

For examples of use, Google "in Borel space", without the quotes. I'm thinking it means either ℝ equipped with its Borel σ-algebra, or to Borel spaces in general (that is, topological spaces with a σ-...
0
votes
0answers
14 views

Amplitude of Sine wave [migrated]

How can I alter amplitude of a sine wave, such that it is in the range of [-1,30]? $y = A\sin(\theta t + \phi)$. I want $A$ to be 30 and $-A$ to be -1.
5
votes
2answers
126 views
+100

By which ways can we, in principle, evaluate whether a model succeeded in generalizing?

A disclaimer: By using an informal term such as "generalize", I am aware I am getting close to philosophical territory, and that my question could be seen as unsuitable for CV. I will do my best ...
0
votes
1answer
7 views

A measure for sustained activity (dispersion)

I need a numeric measure that determines a level of sustained activity. Let me describe a situation in which this could be useful. Imagine that you have a Twitter feed in which you are tracking words ...
1
vote
1answer
25 views

Order of ARMA models

Why we usually do not exceed ARMA(5,5) models in practice? Is there any mathematical justification for this?
8
votes
2answers
183 views

How to define a Rejection Region when there's no UMP?

Consider the linear regression model $\mathbf{y}=\mathbf{X\beta}+\mathbf{u}$, $\mathbf{u}\sim N(\mathbf{0},\sigma^2\mathbf{I})$, $E(\mathbf{u}\mid\mathbf{X})=\mathbf{0}$. Let $H_0: \sigma_0^2=\...
4
votes
0answers
19 views

Data Transformation Question - Multiplying data proportional to demographics

I have a bunch of data that is tied to demographic variables (Age, Sex, Income, Education, etc.). However, the data is sent by one person in a household for the entire house. It's numerical data and I ...
0
votes
1answer
25 views

Conditional distribution (on N) of arrival times in a nonhomogenous poisson process

Conditional on $N(t)$, given some $\lambda(t)$ characterizing some Nonhomogenous poisson point process, the distribution of an arrival time $t_i$ is $\lambda(t_i)/\int_{A}\lambda\left(t\right)dt$ ...
0
votes
0answers
18 views

Applying delta method on Euclidean distance

In order to estimate confidence interval of a k-dimension Euclidean distance, I need to use delta method to estimate standard ...
7
votes
2answers
65 views

How is the $ \chi^2 $ distribution defined at 0 for various degrees of freedom?

I'm trying to understand the $\chi^2$-distribution. Wikipedia has the following graph for the probability density function: This graph shows that for $ k = 1$, the PDF will be... infinite? The mode ...
0
votes
0answers
29 views

How to build model given data only?

Given data only and without any (prior) knowledge or information about the data, how to construct a model for the data and predict new data? There are many statistical models but I have no idea which ...
0
votes
0answers
40 views

When I transform a distribution to apply a test that assumes normality, is the transformation “lossless”?

Many times we deal with data that do not conform to assumptions of normality and/or homoscedasticity/homogeneity of variance. Yet the reality is that almost all analyses benefit from improved the ...
2
votes
2answers
95 views

How to define a rejection region?

This has always troubled me a bit. When I choose my hypothesis, do I define in some way the rejection region [RR], or, do I do that by choosing the test statistic I want to use? By fixing the ...
3
votes
1answer
31 views

How to test for group differences in poisson distributed variables with unequal length?

I would like to test group differences in poisson distributed variables. However, I'm struggle to find appropriate approach(using R), which encounters unequal ...
0
votes
0answers
12 views

Quantifying comparison of two or more 1D datasets (pdfs or cdfs)

I have two 1D data sets which represent the same field quantity and are of same size. Say A1 and A2 both are arrays of size 1 million . Now I have to quantify the comparison of these two data sets ...
1
vote
1answer
33 views

Figuring out which delta to use when calculating standard deviation

So I'm working on something for work where I'm trying to refine how we do our financial projections and one of the key components of that is determining monthly unit sales, specifically, what we're ...
0
votes
1answer
39 views

How to improve a regression model without making new measurements? [closed]

Assuming I can't make more measurements, how can I improve a linear model regression? I have the following data: ...
1
vote
0answers
28 views

Why predicted values differ among methods?

I am using the dataset Orthodont from nlme to simplify my example. ...
1
vote
0answers
29 views

Is there difference between “spectral decomposition” and “singular value decomposition”? [duplicate]

Am I right that "spectral decomposition" for symmetric matrix and "singular value decomposition" for non square matrix? Any clarification would be appreciated.
3
votes
0answers
47 views

Intuition behind using tensors ?

I am trying to build a recommendation model by using a tensor. In order to recommend an article to a user, I have built a model to predict users' preferable articles. I am labeling articles based on ...
9
votes
2answers
163 views

A mathematical explanation of Simpson's paradox

Ok, I know that there's already a question with this name! But it doens't have an answer. There is a comment by @gung, but I'm afraid I don't get it, probably because I don't have a clear ...
0
votes
0answers
53 views

Model selection in LASSO Regression

I fitted lasso regression to "diabetes" data from Efron paper (2004). I had the plot of the coefficients path, but how to know which the best model from this fitting/plot?
0
votes
0answers
12 views

Does Huber regression account for autocorrelated errors?

It seems that regression using the Huber penalty accounts for deviations from the Gaussian distribution in the residuals - but what about serial correlations?
0
votes
2answers
33 views

Assigning a transfer value of a Football player given performance scores

I just recently landed my dream internship at a football statistics company and I am eager to impress. I have an excel spreadsheet of every player in the major leagues along with the minutes they ...
0
votes
0answers
6 views

The derivative of the absolute value |x| [migrated]

I read about the derivative of the absolute value |x|, but why the absolute value is not differentiable at point zero, and when it becomes 1 or -1 {geometrically}? Thanks
3
votes
2answers
31 views

Finding levels/subsets that will predict labels with high probabilty

Assume some 1000 data-points of the following pattern X1 X2 Y 2 B 1 1 C 0 6 A 0 3 B 1 3 C 0 where $X_1 \in \{1, 2, 3, 4, 5, 6\}$, $X_2 \in \{A, B, C\}$ ...
1
vote
2answers
77 views

Generate $r_i$ following condition $\sum_{i=1}^n \left|\frac{r_i}{\sigma_i}\right|^2\leq\chi^2_{n,\alpha}$

How can I generate $r_i$ for $1 \leq i \leq n$, such that $\sum_{i=1}^n \left|\frac{r_i}{\sigma_i}\right|^2\leq\chi^2_{n,\alpha}$, where $\sigma_i^2$ is the variance of $r_i$ and $\chi^2_{n,\alpha}$ ...
0
votes
0answers
25 views

Why is prewhitening important?

I am writing code,geophysical time series processing.First step is to prewhiten values in time domain.Why is this step important? For example,I have found this on sas.com If, as is usually the case, ...
0
votes
0answers
20 views

How does the symmetric matrix A= X' X is found to be QDQ'? [duplicate]

I am confused of how does the symmetric matrix A= X'X is found to be QDQ' ? where D is the diagonal matrix of A, and Q is orthogonal matrix i.e Q'Q=I identity matrix.
0
votes
0answers
16 views

Why does this inequality about the false discovery proportion hold?

I am reading an article from Statistics in Medicine ("Tutorial in biostatistics: multiple hypothesis testing in genomics"). For my question it is only relevant to know that $V$ denote the number of ...
0
votes
0answers
11 views

Jeffreys Interval Calculations and Sample Size

I have a rough understanding of bayesian statistics and of the Jeffrey's Interval/priors. I understand that the Jeffrey's interval takes the form: $$p\mid x \sim \mathrm{Beta}(x + 0.5, n - x + 0.5)$$ ...
0
votes
1answer
25 views

Is there a test suitable for the analysis of my data?

I am trying to compare the performance of pipelines. These pipelines have been sorted into two groups, Insulated and Non-Insulated. I have a total of roughly 60 pipes which are a combination of ...
0
votes
1answer
24 views

Applying bayesian inference to a time based problem

I have a N*M matrix with N customers and M products. Each row of this matrix is a M dimensional vector like [1 3 4 1 5 ....] where each value represents how many times that customer has chosen this ...
5
votes
1answer
73 views

Variance computed using Taylor series does not agree with numerical experiment

I would like to estimate an angle $\theta\in\left(-\frac{\pi}{2},\frac{\pi}{2}\right)$ given the noisy observations of its sine and cosine (this is related to my earlier question). My estimator is ...
1
vote
1answer
47 views

Is real analysis necessary for statisticians? [closed]

I'm almost finishing the undergraduate course in Statistics. Is it worth take a course in Real Analysis if I don't intend to do the master's degree? This will help me in something in the labor market ?...
0
votes
0answers
32 views

Equivalent of standard error for variance?

I would like to know if there is an equivalent formula of standard error for variance or standard deviation, that is, I would like to know how far the variance of my sample is from the real variance ...