Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

0
votes
0answers
28 views

How do we prove this identity related to expectation and variance?

Prove that if $\textbf{a}$ is a vector of constants with the same dimension as the random vector $\textbf{X}$, then \begin{align*} \textbf{E}[(\textbf{X} - \textbf{a})(\textbf{X} - \textbf{a})^{\prime}...
0
votes
1answer
28 views

What Statistical Method is Useful for Comparing Two Different Groups?

I'm working on a project and I want to know what statistical method would be useful to see if my hypothesis is true. The project I'm working on is Calculus Rentention. My hypothesis is: do students, ...
0
votes
0answers
21 views

Variance Calculation Issue

I want to calculate variance for the given problem , i am confused when to use n or n-1 , if nothing is mentioned. also, what should i use in this case. please explain
1
vote
1answer
87 views

Given two related ratios within a population, derive a third ratio (eg. redheads, non-redheads, and skin cancer)

If People X are N times more likely to have attribute A than non-X People X are P percent of the population then What percentage of A's are X? Example (the numbers are just for illustration). If ...
0
votes
2answers
43 views

Calculating the variance and expected value of a multiple Coin Toss with Reward

I was wondering if you flipped 4 coin tosses, and you get 0.25 dollars for each coin that lands on tails and 0 dollars if it lands on head. How, would you calculate the variance and expected value.
2
votes
1answer
41 views

Statistical test to check if sampling was not uniform

Given a set of real numbers $X$ and a subset $Y \subset X$, I would like to demonstrate that $Y$ is not a result of a uniform sampling from $X$. One solution that I thought recall this, that is: ...
1
vote
1answer
56 views

Survival analysis with biased coins

Suppose I have five biased coins $c1, c2, c3, c4$ and $c5$ each with different probabilities of getting heads. Pr. heads for coin $c1=0.5$ Pr. heads for coin $c2=0.8$ Pr. heads for coin $c3=0.6$ Pr....
0
votes
0answers
23 views

Statistical technique to assess change in drug events over time - Interrupted time series analysis?

I have a dataset of the number of drug related events (e.g. arrest, non-fatal overdoses) a population of individuals have had. My aim is to see if a certain policy intervention that occurred on 1 June ...
1
vote
1answer
48 views

How to prove $P(a \geq b +c) \leq P(a \geq b) + P(c \leq 0)?$

How to prove $$P(a\geq b+c)\leq P(a\geq b)+P(c\leq0)?$$ Thanks.
0
votes
0answers
16 views

Measuring the confidence of a delta

Hope this is the right place to ask this. I'm measuring a single quantity, A, multiple times and each time the value is expected to be different(inherent error in the system). Call them ...
0
votes
0answers
5 views

Comparison of discrimination tests using, Thurstonian model and its d -values

The task is to measure two discrimination test methods "triangle test and tetrad". In both methods the possibility to guess correct is (1/3). In my study I used Chi square test to analyse data. ...
1
vote
0answers
31 views

Multivariate Taylor series for moments of a random variable

In the expectation propagation for the generative aspect model, Minka uses Taylor series for the parameter estimation of the topics $p(w\mid a)$ eq 31. I am a little confused in the last equation. He ...
0
votes
0answers
29 views

How is the L2 regularization derived? [duplicate]

I just proved to myself why the regularization is added rather than multiplied to loss function. I did so by taking the MLE formula... $$\operatorname{argmax}\sum \log(P(x_i\mid\Theta ))$$ and ...
1
vote
0answers
8 views

Question about books answer to estimating variance of block

The question is, Suppose that a single factor experiment with four factors and six blocks is conducted and the error sum of squares is 500 and the sum of squares for the blocks is 250. The question ...
0
votes
1answer
28 views

Predicted distribution over the new data sample

I'm reading Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville. In chapter 5 about Bayesian Statistics, to find the distribution over the new data sample $x^{m+1}$ after observing $...
0
votes
0answers
18 views

The limit distribution of Wilcoxon signed rank statistic?

An alternative representation of the Wilcoxon signed rank statistic $V$ is $V=\sum_{i\le j}\mathbb{I}_{\{X_i+X_j>0\}}=\sum_i\mathbb{I}_{\{X_i>0\}}+\sum_{i<j}\mathbb{I}_{\{X_i+X_j>0\}}$ ...
17
votes
4answers
1k views

The magic money tree problem

I thought of this problem in the shower, it was inspired by investment strategies. Let's say there was a magic money tree. Every day, you can offer an amount of money to the money tree and it will ...
0
votes
0answers
66 views

Reinforcement Learning: Bellman Equations and relations for optimal (state)/value function?

This question is about (stationary) Markov Decision Processes in the case of discounted reward. My questions are: Why is $v^*(s) = \sup_{a \in A} Q^*(s,a)$? Why is $Q^*(s,a) = E[R_t|S_t=s, A_t=a] + ...
5
votes
2answers
63 views

Why is ANOVA in R showing a different result than ANOVA from another source?

I am new to R and statistics both. I am trying to perform 2x2 ANOVA on an excel dataset where the 2 factors are Gender and Athletes and the dependent variable is Anger Expression. When I am ...
0
votes
0answers
18 views

Is a rolling $z$-score a good proxy for a derivative?

Given a time series $\{V_t\}_{t=1}^{n}$, where $t \mapsto V_t \in \mathbb{R}$, I want to have some smoothed notion of the "derivative" of this time series. It was recommended that I look at $$\tilde{...
1
vote
1answer
46 views

Brier Score of a Prediction, Mathematical Notation

I am currently working on a logistic regression model that is fitted on the base of a training set ($D_0$) and is used to predict the outcome (0 or 1) of an independent test set ($D_1$). As an ...
1
vote
1answer
30 views

Justification for removing μ out of brackets in definitions of average of deviations and variance

Here are three equations for some basic parameters. Average of deviations: $$\lim_{N\to \infty} \bar{d} = \lim_{N\to \infty} \left[\frac{1}{N}\sum(x_i-\mu) \right]= \lim_{N\to \infty} \left(\frac{1}{...
0
votes
0answers
38 views

Hypothesis testing with 1 or 2 sample in ornithology

I am doing an experiment with birds. I have set up the experiment to measure how certain foods affect bird flying speeds. When I implement a control, for example food A, I want to claim that the ...
0
votes
1answer
23 views

Deriving standard deviation from p-value and t-statistics

I have the results of a t-test for two groups, where the mean of one group is 3.4 units and the mean of the second group is 3.8. From the paired t-test, I know that t$_{17}$ = 2.8 and p = 0.013. Is ...
0
votes
0answers
11 views

With few respondents on a 1 to 10 scale, is it better to use a weighted mean or the median?

I have survey data where ~20 respondents replied to some questions on a scale of 1 to 10. For the most part, none of the distributions are Gaussian. A set of questions look to find out a participants ...
0
votes
2answers
53 views

normalize/standardize data

I hope someone here can point me on the right direction. I have the following dataset. I have fish adult and juvenile data that I want to compare side by side, however, the juvenile fish numbers are ...
0
votes
0answers
17 views

Krippendorff's alpha getting low after restricting data

I am calculating inter-rater agreement for ~350 raters, each of who rated 100 items using Likert scale. The original scale has range [-3,3], using this python implementation, I got $\alpha=.35$, which ...
0
votes
0answers
8 views

classify according to one variable and no classes

I have a problem, I think that is common but I don´t have idea about how to proceed. I am looking an elegant solution for it. I have N groups and two variables for each group. After that I have ...
0
votes
0answers
15 views

How to know where to cut off a list of values?

Suppose I have a list of values like this one: ...
0
votes
0answers
13 views

median survival derived from a mixture of exponential distributions

I recently got a very nice question... Let's say that we have one cohort, and we know the median survival of it. Then we split the cohort into two groups, and somehow we also know the median survival ...
0
votes
0answers
21 views

Confusing problem statement of exercise 7.4.3 in Wasserman's All of statistics

I am reading chapter 7 of Larry Wasserman's All of statistics and working on the exercises in the end of this chapter. The exercise 7.4.3 (on p. 104) states: (Computer Experiment.) Generate 100 ...
2
votes
1answer
62 views

Find the cause of periodic behaviour present in Google-search “war”

If you use Google Trends (a tool to illustrate the frequency of search terms) and search for the word "war" with USA as geographic region, you will be presented with the graph below showing monthly ...
0
votes
1answer
25 views

Getting From Concentration Inequality to Interval Length

I've seen this used some times and I would like to ask what steps are taken on the way to getting there: E.g. assuming bounded variance, we can use Chebyshev concentration inequality: for any $t>0$...
0
votes
0answers
34 views

ARIMA Model Ljung-Box Test Derivation

Initially I am aware of the fact that in order to analyse autocorrelations of residuals of ARMA(p,q) we apply Ljung-Box test and involve chi-square distribution (because autocorrelations are ...
0
votes
2answers
31 views

Taking constant out of multivariate normal

For a univariate normal distribution $X \sim N(0, k\sigma^2)$ we can take out the $k$ to get $\sqrt{k}X \sim N(0, \sigma^2)$. In the multivariate normal case is there something similar? If $\textbf{Y} ...
2
votes
1answer
66 views

Effect of class imbalance on logistic regression (mathematical basis)

A number of posts, and papers, state that logistic regression (LR) is robust in the face of class imbalance. Unless the imbalance is extreme (e.g., events=0.01 or less), with adequate sample sizes ...
0
votes
0answers
18 views

Row aggregation of multiple records

We have a syslog dataset with different Timestamps and 3 other features pertaining to syslog information such as Process, Trace information, SeverityType etc. Below is the dataset format with ...
0
votes
1answer
23 views

Kolmogorov Distribution D statistics

As far as I have searched the cumulative distribution function of 𝐾, asymptotically (kolmogorov distribution) is given by Pr(𝐾≤𝑥)=1−2∑∞𝑘=1(−1)𝑘−1𝑒−2𝑘2𝑥2=2𝜋√𝑥∑∞𝑘=1𝑒−(2𝑘−1)2𝜋2/(8𝑥2). But ...
1
vote
1answer
101 views

FDR calculation for each p value

I am trying to understand how FDR is calculated and what to understand from it. I selected R as language so the example is in R Lets look at an example of 10 samples ...
4
votes
0answers
74 views

Why is this statistic F-distributed?

A book I'm reading claims that the statistic: $\frac{(RSS_0 - RSS_1) / (p_1 - p_0)}{RSS_1 / (N - p_1 - 1)}$ has an F distribution. Why is this? I know that an F distribution is something like $\frac{\...
1
vote
1answer
37 views

conditional probability involving mixed variable types

I'm trying to answer the following question A defective coin minting machine produces coins whose probability of heads is a random variable $T$ with PDF $f_{T}(p) = 1+\mathrm{sin}(2\pi p)$ if $p \in ...
1
vote
2answers
28 views

estimate mean and standar deviation from proportions

I have normal distributed data but i don´t know the mean neither the stdv. What i know is that 67.9% of my data points are below 2500g and 0.21% are over 3000g. How can I estimate the mean and stdv ...
0
votes
1answer
57 views

Should one remove highly correlated variables before doing EFA?

I am doing Exploratory Factor Analysis with 7 items. One of them (call it v1) have high correlations with 3 others, and other 3 are mutually moderately correlated. ...
5
votes
3answers
100 views

Why is sufficient statistics/data reduction normally taught in Statistics?

In most upper-level classes on statistical inference, data reduction and sufficient statistics are normally taught, but without too much motivation. I understand sufficient statistics are important ...
1
vote
0answers
14 views

Wald test for complex-valued data

Let $\mathbf{x} = [x_1,\ldots,x_N] \in \mathbb{C}^N$ be a set of complex-valued data and let $p(\mathbf{x};\alpha)$ their joint probability density function parametrized by the unknown complex scalar $...
0
votes
0answers
14 views

Ordered statistics: Does the survivor function applied to a fixed point of the Mills ratio increase with the sample size

Let $X_1, ..., X_n$ be $n$ i.i.d. Let $F$ be their CDF and let $f$ be their PDF. Assume that $f$ is log-concave. The ordered statistic $\max_{i=1...n} X_i$ has for CDF $F_n\left(x\right) = \mathbb{P}\...
7
votes
2answers
882 views

Calculate variance without calculating the mean

Can we calculate the variance without using the mean as the 'base' point?
0
votes
0answers
16 views

Zero-Inflated Rare Keyword Impact Prediction

Suppose we have a dataset $X$ of features and a target binary prediction $y\in\{0,1\}$ for each datapoint. Each row of $X$ consists of counts (a bag of features). We can normalize each row of $X$ to ...
0
votes
0answers
22 views

Confidence interval in Chi squared testing

How to calculate confidence interval when performing a Chi-Square significance testing? For reference where it's used - http://www.evanmiller.org/ab-testing/chi-squared.html Can we apply the same ...