Mathematical theory of statistics, concerned with formal definitions and general results.

learn more… | top users | synonyms (1)

0
votes
0answers
17 views

What is variance of an election poll?

I was working through Harvard data science course: I came across this question: Assume you have $M$ polls with sample sizes $n_1,\ldots,n_M$. If the polls are independent, what is the average of the ...
0
votes
0answers
12 views

Can I Use Distance Matrix (Weighted) For GLM Fit (Using R)

I am new to statistics but a business user liking to base my decisions on prediction models. And I am relatively new to regression. I have data in the below format. (I have used only imaginary ...
0
votes
0answers
17 views

Closed form Pocock Rejection Region solution for joint MVN correlated RVs?

This is related to my previous question (link), but I have a simpler way of expressing it. Suppose $ \left( \begin{array}{ccc} \ Z_1 \\ Z_2 \end{array} \right)$ follows a Bivariate standard normal ...
3
votes
1answer
26 views

Determining actual number of observations in a dataset

I have two datasets one is a dataset with doctors in which I have the procedures they have performed at a given hospital where the actual number of procedures is not captured by this data since it is ...
0
votes
0answers
16 views

Scoring model/formula to rank based on accuracy of data sources

Let us say i have 5 different data sources. My aim is arrive a score to arrive the accuracy of this data sources. It is like below 1. Refer the section Data confidence calculation in ...
3
votes
1answer
61 views
+100

Estimate mean & standard deviation of set S if I know stats of an inner set and outer set

Suppose I have sets $S_1\subset S_2\subset S_3$. I know exactly the size, mean, and standard deviation of both $S_1$ and $S_3$ and want to estimate the mean and standard deviation of $S_2$, where I ...
-1
votes
0answers
10 views

How to explain the correlation between the mean of each element and it's corresponding PC1 coefficient in a data.frame?

I am doing some PCA analysis by R now,I have a data,which is a data.frame:a,ncol(a) =1000,nrow(a)=100,each row represent a sample,each col represent a feature,so when I use R do PCA analysis like ...
0
votes
0answers
11 views

Equations/Theories to segment a large dataset into various subgroup

I am doing a small research project on how to segment a large dataset (basically addresses and coordinates on map) into a various subgroupa (like 3 or 4 for example) using some statistics/linear ...
3
votes
1answer
49 views

Testing correlation and the t-statistic used in Simple Linear Regression

Given $H_0$ : $\rho=0$ and $H_A$ : $\rho\neq0$, we use the test statistic $t_{n-2}$ , which is $\frac{r\sqrt{n-2}}{\sqrt{1-r^2}}$. I have to show that $\frac{r\sqrt{n-2}}{\sqrt{1-r^2}}$ equals ...
1
vote
1answer
28 views

Log Inverse Gamma Distribution

Can any one help me to know the formula for mean and variance of log inverse gamma distribution. Thanks
4
votes
0answers
11 views

Relation between asymptotic relative efficiency for tests and estimators

The asymptotic relative efficiency for unbiased estimators is the limit of the ratio of the variances as the $n\rightarrow \infty$. Is there a relation to asymptotic relative efficiency according to ...
0
votes
0answers
12 views

The speed of convergence to the true probability depending on the prior

In a Bayesian Updating world, suppose there is a true probability of a binary outcome happening, $P_{true}$. My questions is does the speed of convergence depend on the prior.For instance, if my prior ...
0
votes
1answer
22 views

Does martingale model work for betting football matches?

Imagine I have 1 million USD and will be betting 1.000 USD on the win of FC Barcelona each time they play a match in La Liga (Spanish Tier 1 football league). If FC Barcelona loses or ties their last ...
0
votes
0answers
22 views

What is first order asymptotic distribution?

I am stuck with this term in a research paper(Hansen, 1999). Its written (p 351) "Hansen (1996) shows that a bootstrap procedure attains the first-order asymptotic distribution, so p-values ...
1
vote
0answers
24 views

Mulitvariate normal truncated conditional expectation

There is three-variable multivariate normal distribution. Denote 3 variables with $X_1$, $X_2$, $X_3$. Let $\mu_i$ be means, and $\sigma_i^2$ variances of respective variables, and let $\Sigma$ be ...
0
votes
0answers
16 views

Joint distribution R [migrated]

I have a list of 10 stocks, with each having a time series of log returns (AIG, JPM,...). I have calculated the log returns for each of the stocks as follows: ...
1
vote
0answers
12 views

How to quantizate two sets of data, and see there relationships are strong or weak?

Let say I got three set of data, one is U.S. Dollar Index, another is Gold Price, and the last is ...
0
votes
0answers
4 views

simulating AR(1) process for N variables in MATLAB [migrated]

I am quite new in programming, can somebody tell whether the following approach is correct in matlab? clc; clear; N = 15; T = 221; alpha = randn(N,1); % normally distributed intercept as ...
3
votes
1answer
43 views

Does skewness predicts variance?

(with apologizes to this question). Consider two distributions $G$, $F$ both uni-modal and absolutely continuous, square integrable and satisfying: $$F<_c G$$ this means that the standardized ...
2
votes
1answer
37 views

What is “Targeted Maximum Likelihood Expectation”?

I'm trying to understand some papers by Mark van der Laan. He's a theoretical statistician at Berkeley working on problems overlap significantly with machine learning. One problem for me (besides ...
1
vote
0answers
30 views

Confidence intervals sum of dependent variables

How to construct confidence intervals for sum of dependent random variables. Specifically, I want a high probability claim regarding the difference between the empirical mean and the true mean of the ...
5
votes
1answer
37 views

characteristic functions and symmetry

If the characteristic function of a random variable is a real-valued function, does this imply that the random variable must be symmetric about zero?
0
votes
0answers
22 views

Question about notation of expectation operators over multiple random processes

My question is what are suitable or accepted notations for taking expectations over multiple random processes in the same equation. Let a model for variable $y$ be given by $$y_{i ...
1
vote
1answer
19 views

What are the appropriate statistical methods to assess this type of hypothesis?

I have a general question as to which methods are considered "standard" or "best practice" in asssessing the following type of hypothesis. I am running simulations where I generate random graphs ...
0
votes
0answers
20 views

Is there a way to check completeness of certain sufficient statistics?

In general, given a p.d.f. or a p.m.f., is there a method to check if a certain statistic is complete? For example, consider a population $N(\theta,1)$ where $\theta$ is unknown and the statistic ...
2
votes
0answers
29 views

parameter and prediction confidence intervals

We have a correctly specified linear model $z = x\beta + \varepsilon$ with 100 independent observations. Given: $y = e^z$, $\sigma^2 = 4$ (assumed to be known), sample means $\bar{x} = -3$ and ...
1
vote
0answers
33 views

Theorem A3 in Asymptotic properties of MLE for the i.n.i.d. case

Can someone explain to me why in the theorem below (case in $R^1$) $$\lim \inf I(A\cap B(u))|X_k(u)| \leq I(A\cap B)|X_k|$$ Full text of proof: ...
5
votes
1answer
65 views

How to obtain the quantile function when an analytical form of the distribution is not known

The problem comes from page 377-379 of this [0] paper. Given a continuous distribution $F$ and a fixed $z\in\mathbb{R}$, consider: $$L_z(t)=P_F(|z-Z|\leq t)$$ and ...
3
votes
1answer
65 views

If $X$ takes on a Gamma Distribution, how can I find $X^2$, $X^3$, etc?

I am trying to take consecutive powers of a Gamma Distribution. For example, if $X \sim \textrm{Gamma}(k, \theta)$, I would like to find $X^2$, $X^3$, and in general $X^m$ for $m>0$. The pdf ...
3
votes
0answers
61 views

Draw balls from two different binomial distributions. What $r^2$ do you expect to find?

We have $N$ buckets in which we will put some balls. Before that, the buckets are split into two groups, group $A$ and group $B$. The number of balls that we will put in each bucket is drawn from a ...
1
vote
2answers
28 views

Will the mean of a set of means always be the same as the mean obtained from the entire set of raw data?

If I have calculated the mean for 4 data sets (which do have different sample sizes), can I then obtain an "overall mean" by calculating the "mean of the means"? If yes, will this "mean of the means" ...
3
votes
1answer
26 views

Question regarding covariance

I'm trying to prove a theorem, where it is given that each $X_i$ is independent and identically distributed with mean $\mu$ and variance $\sigma^2$. Within this theorem, I have multiple sub-results to ...
2
votes
0answers
29 views

Finding for constant in a standard deviation problem

Let $X_1, \ldots, X_n$ be sample observations. Show that for any constant $c$, $$ (n-1)s^2\le\sum(X_i-c)^2 $$ where $s$ is the standard deviation of the observations. My professor gave this ...
5
votes
1answer
65 views

when can I say there is a relationship between events?

I am analysing a weather data for a computer science course using hadoop.I am taking away all the weather related issues out of the analysing and just looking strictly at it in the eyes of statistics. ...
4
votes
2answers
654 views

What is this equation (pictured) called?

Could someone provide some Google-able words for this equation? $$Q_{\chi^2,d} = \left[2^{d/2}\Gamma\left(\frac{d}{2}\right)\right]^{-1}\int_{\chi^2}^\infty (t)^{\frac{d}{2}-1}e^{-\frac{t}{2}}dt$$ ...
5
votes
2answers
134 views

How to find $\displaystyle \dfrac{d}{dt} \left [\int_t^\infty xf(x)~dx \right ] $ (when $f(x)$ is a probability density function)

How can I solve this? I need intermediate equations. Maybe the answer is $-tf(x)$. $\displaystyle \dfrac{d}{dt} \left [\int_t^\infty xf(x)~dx \right ] $ $f(x)$ is probability density function. That ...
1
vote
1answer
35 views

Simulating Negative Binomial Arrivals

I am building a simulation. I want to generate arrivals according to a negative binomial process. The data will show the minute of each "customer" arrival and look something like the following: ...
3
votes
1answer
37 views

How to use the bunching estimator to estimate the elasticity of taxable income

Today my professor told us about a recent estimator that non-parametrically estimates the elasticity of taxable income. I understood why doing this gets around problems of previous studies that use ...
1
vote
0answers
24 views

Tail bound on dependent sum

Let R be a random variable, and let X_i be a nonlinear, nonconvex function that takes R and outputs a value in [-1,1]. I'm trying to get a tail bound on the following dependent sum: ...
13
votes
2answers
259 views

For which distributions is there a closed-form unbiased estimator for the standard deviation?

For the normal distribution, there is an unbiased estimator of the standard deviation given by: $$\hat{\sigma}_\text{unbiased} = \frac{\Gamma(\frac{n-1}{2})}{\Gamma(\frac{n}{2})} ...
2
votes
0answers
37 views

Why does this multi-response Guassian LASSO not give a sparse solution?

I tried the glmnet package to learn multi-response Gaussian family. I have looked at the coefficients of the final model. The result is odd. All the features have ...
0
votes
0answers
32 views

posterior predictive of standard Normal

I want to derive the predictive posterior distribution of $y$ for the case where $$p(\tilde{y}|\theta) \sim N(\theta,1)$$ and $$p(\theta|y) \sim N(\bar{y},\frac{1}{n}) \sim N(6,\frac{1}{9})$$. By ...
0
votes
0answers
14 views

Measure the Ranking of Variables

What I want to do is compare the ordering of variables determined by the ranking of each variable. For example: Say, I have a rating system that is made up of 5 different ratings - Excellent Good ...
-1
votes
2answers
124 views

can a software engineer be good at data analytics using statistics and R [closed]

I am a software engineer and new to R. I started learning but at many steps I end up learning stats concepts. So I was wondering if it's a good idea for a software developer to move to data ...
0
votes
0answers
5 views

Calculate trending of content from multiple sources

I'm not fantastically mathematical so please excuse any mis steps. Let me start with what I'm trying to do: I have news sources, for example: A. Washington Post B. TechCrunch C. Krebs on Security ...
0
votes
0answers
16 views

How to measure the relative strength among available categories based on their odds raitos?

I have 7 categories (suppose a, b, c, d, e, f, g) and I applied Ordinal logistic regression keeping one as a reference category( suppose here the reference category is g). I want to know which ...
0
votes
1answer
32 views

How to prove absolute lack of correlation

I have a huge dataset of 17 variables. I intended to use 15 of those to predict the 17th, and I could not find any model (ANN) to do so. I know that one of those variables definitely predicts the ...
3
votes
2answers
50 views

Error terms vs Innovations

I noticed that we sometimes call the error terms "innovations". I do not understand if this is in special situations or if these terms can be used one for another. Then, another question is "why do we ...
2
votes
1answer
45 views

Sum of Square decomposition

Question about the Total, Explained, and Residual Sum of Squares. I am in the simple linear regression model. Could you help me clarify why the residual sum of squares (SSE where E stands for errors) ...
2
votes
0answers
13 views

Statistical tests to compare sigmoidal regression equations

Data from my experiments (sample shown below) can be fitted using a sigmoid function. The equation I used to fit the data is: y = A2 + (A1-A2)/(1+(x/x0)^p). Each experiment yields data (and a ...