Questions tagged [self-study]

A routine exercise from a textbook, course, or test used for a class or self-study. This community's policy is to "provide helpful hints" for such questions rather than complete answers.

Filter by
Sorted by
Tagged with
97
votes
14answers
67k views

Books for self-studying time series analysis?

I started by Time Series Analysis by Hamilton, but I am lost hopelessly. This book is really too theoretical for me to learn by myself. Does anybody have a recommendation for a textbook on time ...
76
votes
3answers
146k views

An example: LASSO regression using glmnet for binary outcome

I am starting to dabble with the use of glmnet with LASSO Regression where my outcome of interest is dichotomous. I have created a small mock data frame below: <...
48
votes
4answers
28k views

How are regression, the t-test, and the ANOVA all versions of the general linear model?

How are they all versions of the same basic statistical method?
46
votes
8answers
4k views

Pitfalls in time series analysis

I am just starting out self-learning in time series analysis. I have noticed that there are a number of potential pitfalls that are not applicable to general statistics. So, building on What are ...
42
votes
4answers
6k views

Taking the expectation of Taylor series (especially the remainder)

My question concerns trying to justify a widely-used method, namely taking the expected value of Taylor Series. Assume we have a random variable $X$ with positive mean $\mu$ and variance $\sigma^2$. ...
40
votes
3answers
21k views

A generalization of the Law of Iterated Expectations

I recently came across this identity: $$E \left[ E \left(Y|X,Z \right) |X \right] =E \left[Y | X \right]$$ I am of course familiar with the simpler version of that rule, namely that $E \left[ E \...
38
votes
9answers
4k views

What is the relationship between $Y$ and $X$ in this plot?

What is the relationship between $Y$ and $X$ in the following plot? In my view there is negative linear relationship, But because we have a lot of outliers, the relationship is very weak. Am I right? ...
37
votes
5answers
4k views

Will the fact that my Italian son is going to attend a primary school change the expected number of Italian children to be present in his class?

This is a question stemming from a real-life situation, for which I have been genuinely puzzled about its answer. My son is due to start primary school in London. As we are Italian, I was curious to ...
37
votes
5answers
15k views

LDA vs word2vec

I am trying to understand what is similarity between Latent Dirichlet Allocation and word2vec for calculating word similarity. As I understand, LDA maps words to a vector of probabilities of latent ...
35
votes
3answers
34k views

How to take derivative of multivariate normal density?

Say I have multivariate normal $N(\mu, \Sigma)$ density. I want to get the second (partial) derivative w.r.t. $\mu$. Not sure how to take derivative of a matrix. Wiki says take the derivative ...
34
votes
6answers
4k views

Why shouldn't the denominator of the covariance estimator be n-2 rather than n-1?

The denominator of the (unbiased) variance estimator is $n-1$ as there are $n$ observations and only one parameter is being estimated. $$ \mathbb{V}\left(X\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\...
34
votes
3answers
4k views

What does the Akaike Information Criterion (AIC) score of a model mean?

I have seen some questions here about what it means in layman terms, but these are too layman for for my purpose here. I am trying to mathematically understand what does the AIC score mean. But at ...
31
votes
4answers
83k views

Find expected value using cdf

I'm going to start out by saying this is a homework problem straight out of the book. I have spent a couple hours looking up how to find expected values and have determined I understand nothing. ...
31
votes
4answers
2k views

Is the result of an exam a binomial?

Here is a simple statistics question I was given. I'm not really sure I understand it. X = the number of aquired points in an exam (multiple choice and a right answer is one point). Is X binomial ...
30
votes
2answers
21k views

What is the difference between censoring and truncation?

In the book Statistical Models and Methods for Lifetime Data , it is written : Censoring: When an observation is incomplete due to some random cause. Truncation: When the incomplete nature of ...
28
votes
6answers
15k views

What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
28
votes
4answers
4k views

Self study vs a taught education?

There is a question with similar intent on programmers.SE. That question has some quite good answers, but the general theme seems to be that without self study, you get no-where. Obviously there are ...
27
votes
4answers
85k views

McFadden's Pseudo-R2 Interpretation

I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this ...
27
votes
3answers
4k views

Which hospital should be chosen? One has a higher success rate, but the other has a higher overall success rate

I have a question about something that my statistics teacher said about the following problem. My question isn't even about the occurrence of Simpson's paradox in this situation. My question is simply ...
26
votes
5answers
20k views

Why does the variance of the Random walk increase?

The random walk that is defined as $Y_{t} = Y_{t-1} + e_t$, where $e_t$ is white noise. Denotes that the current position is the sum of the previous position + an unpredicted term. You can prove that ...
26
votes
1answer
1k views

What are the classical notations in statistics, linear algebra and machine learning? And what are the connections between these notations?

When we read a book, understanding the notations plays a very important role of understanding the contents. Unfortunately, different communities have different notation conventions for the formulation ...
25
votes
7answers
24k views

Two dice rolls - same number in sequence

I am currently studying Statistical Inference class on Coursera. In one of the assignments, the following question comes up. ...
25
votes
2answers
50k views

How to know if a data follows a Poisson Distribution in R?

I am an undergrad student and I have a project for my probability class. Basically, I have a dataset about the hurricanes that impacted my country for a series of years. In my probability Book, (...
24
votes
8answers
52k views

Looking for a good and complete probability and statistics book

I never had the opportunity to visit a stats course from a math faculty. I am looking for a probability theory and statistics book that is complete and self-sufficient. By complete I mean that it ...
24
votes
3answers
6k views

Functions of Independent Random Variables

Is the claim that functions of independent random variables are themselves independent, true? I have seen that result often used implicitly in some proofs, for example in the proof of independence ...
24
votes
3answers
21k views

Interpreting plot of residuals vs. fitted values from Poisson regression

I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What ...
21
votes
3answers
6k views

Neyman-Pearson lemma

I have read the Neyman–Pearson lemma from the book Introduction to the Theory of Statistics by Mood, Graybill and Boes. But I have not understood the lemma. Can anyone please explain the lemma to ...
19
votes
3answers
31k views

Why is nls() giving me “singular gradient matrix at initial parameter estimates” errors?

I have some basic data on emission reductions and cost per car: ...
19
votes
4answers
25k views

How to test if my distribution is multimodal?

When I plot a histogram of my data, it has two peaks: Does that mean a potential multi-modal distribution? I ran the dip.test in R (...
18
votes
4answers
635 views

What is the intuition behind the independence of $X_2-X_1$ and $X_1+X_2$, $X_i \sim N(0,1)$?

I was hoping someone could propose an argument explaining why the random variables $Y_1=X_2-X_1$ and $Y_2=X_1+X_2$, $X_i$ having the standard normal distribution, are statistically independent. The ...
18
votes
9answers
2k views

Reference Request: Generalized Linear Models

I am looking for an introductory to intermediate level book on Generalized Linear Models. Ideally, in addition to the theory behind the models, I would want it to include applications and examples in ...
18
votes
2answers
1k views

Advantages of the Exponential Family: why should we study it and use it?

So here I am studying inference. I would like that someone could enumerate the advantages of the exponential family. By exponential family, I mean the distributions which are given as \begin{align*} f(...
18
votes
2answers
411 views

Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, 1)$

What is the easiest way to see that the following statement is true? Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, ...
18
votes
1answer
13k views

Difference between Hidden Markov models and Particle Filter (and Kalman Filter)

Here is my old question I would like to ask if someone knows the difference (if there is any difference) between Hidden Markov models (HMM) and Particle Filter (PF), and as a consequence Kalman ...
17
votes
4answers
3k views

Can anyone clarify the concept of a “sum of random variables”

In my probability class the terms "sums of random variables" is constantly used. However, I'm stuck on what exactly that means? Are we talking about the sum of a bunch of realizations from a random ...
17
votes
3answers
1k views

Why are “time series” called such?

Why are “time series” called such? Series means sum of a sequence. Why is it time Series, not time sequence? Is time the independent variable?
17
votes
1answer
13k views

A proof for the stationarity of an AR(2)

Consider a mean-centred AR(2) process $$X_t=\phi_1X_{t-1}+\phi_2X_{t-2}+\epsilon_t$$ where $\epsilon_t$ is the standard white noise process. Just for sake of simplicity let me call $\phi_1=b$ and $\...
17
votes
1answer
7k views

Proof of LOOCV formula

From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ ...
17
votes
1answer
1k views

Quiz: Tell the classifier by its decision boundary

Given are the 6 decision boundaries below. Decision boundaries is violett lines. Dots and crosses are two different data sets. We have to decide which one is a: Linear SVM Kernelized SVM (Polynomial ...
17
votes
2answers
6k views

Hidden Markov Model vs Markov Transition Model vs State-Space Model…?

For my master's thesis, I am working on developing a statistical model for the transitions between different states, defined by serological status. For now, I won't give too many details into this ...
17
votes
1answer
503 views

Second moment method, Brownian motion?

Let $B_t$ be a standard Brownian motion. Let $E_{j, n}$ denote the event$$\left\{B_t = 0 \text{ for some }{{j-1}\over{2^n}} \le t \le {j\over{2^n}}\right\},$$and let$$K_n = \sum_{j = 2^n + 1}^{2^{2n}} ...
16
votes
1answer
7k views

Question on how to normalize regression coefficient

Not sure if normalize is the correct word to use here, but I will try my best to illustrate what I am trying to ask. The estimator used here is least squares. Suppose you have $y=\beta_0+\beta_1x_1$, ...
16
votes
8answers
28k views

How is interpolation related to the concept of regression?

Explain briefly What is meant by interpolation.How is it related to the concept of regression? interpolation is art of reading between the lines of a table and in elementary mathematics the term ...
16
votes
1answer
4k views

Construction of Dirichlet distribution with Gamma distribution

Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$,...
16
votes
3answers
1k views

Why do we need Bootstrapping?

I'm currently reading Larry Wasserman's "All of Statistics" and puzzled by something he wrote in the chapter about estimating statistical functions of nonparametric models. He wrote "Sometimes we ...
16
votes
4answers
2k views

Expected value of sample median given the sample mean

Let $Y$ denote the median and let $\bar{X}$ denote the mean, of a random sample of size $n=2k+1$ from a distribution that is $N(\mu,\sigma^2)$. How can I compute $E(Y|\bar{X}=\bar{x})$? Intuitively, ...
16
votes
2answers
107k views

How do I calculate the variance of the OLS estimator $\beta_0$, conditional on $x_1, \ldots , x_n$?

I know that $$\hat{\beta_0}=\bar{y}-\hat{\beta_1}\bar{x}$$ and this is how far I got when I calculated the variance: \begin{align*} Var(\hat{\beta_0}) &= Var(\bar{y} - \hat{\beta_1}\bar{x}) \\ &...
16
votes
5answers
3k views

Probability theory books for self-study

Are there any good books that explain important concepts of probability theory like probability distribution functions and cumulative distribution functions? Please, avoid referring books like "...
15
votes
2answers
7k views

Simulating draws from a Uniform Distribution using draws from a Normal Distribution

I recently purchased a data science interview resource in which one of the probability questions was as follows: Given draws from a normal distribution with known parameters, how can you simulate ...
15
votes
4answers
2k views

Classic linear model - model selection

I have a classic linear model, with 5 possible regressors. They are uncorrelated with one another, and have quite low correlation with the response. I have arrived at a model where 3 of the regressors ...