Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [self-study]

A routine exercise from a textbook, course, or test used for a class or self-study. This community's policy is to "provide helpful hints" for such questions rather than complete answers.

74
votes
3answers
139k views

An example: LASSO regression using glmnet for binary outcome

I am starting to dabble with the use of glmnet with LASSO Regression where my outcome of interest is dichotomous. I have created a small mock data frame below: <...
46
votes
8answers
4k views

Pitfalls in time series analysis

I am just starting out self-learning in time series analysis. I have noticed that there are a number of potential pitfalls that are not applicable to general statistics. So, building on What are ...
45
votes
4answers
27k views

How are regression, the t-test, and the ANOVA all versions of the general linear model?

How are they all versions of the same basic statistical method?
41
votes
4answers
6k views

Taking the expectation of Taylor series (especially the remainder)

My question concerns trying to justify a widely-used method, namely taking the expected value of Taylor Series. Assume we have a random variable $X$ with positive mean $\mu$ and variance $\sigma^2$. ...
39
votes
3answers
19k views

A generalization of the Law of Iterated Expectations

I recently came across this identity: $$E \left[ E \left(Y|X,Z \right) |X \right] =E \left[Y | X \right]$$ I am of course familiar with the simpler version of that rule, namely that $E \left[ E \...
38
votes
9answers
4k views

What is the relationship between $Y$ and $X$ in this plot?

What is the relationship between $Y$ and $X$ in the following plot? In my view there is negative linear relationship, But because we have a lot of outliers, the relationship is very weak. Am I right? ...
37
votes
5answers
4k views

Will the fact that my Italian son is going to attend a primary school change the expected number of Italian children to be present in his class?

This is a question stemming from a real-life situation, for which I have been genuinely puzzled about its answer. My son is due to start primary school in London. As we are Italian, I was curious to ...
37
votes
5answers
14k views

LDA vs word2vec

I am trying to understand what is similarity between Latent Dirichlet Allocation and word2vec for calculating word similarity. As I understand, LDA maps words to a vector of probabilities of latent ...
35
votes
3answers
4k views

What does the Akaike Information Criterion (AIC) score of a model mean?

I have seen some questions here about what it means in layman terms, but these are too layman for for my purpose here. I am trying to mathematically understand what does the AIC score mean. But at ...
34
votes
6answers
4k views

Why shouldn't the denominator of the covariance estimator be n-2 rather than n-1?

The denominator of the (unbiased) variance estimator is $n-1$ as there are $n$ observations and only one parameter is being estimated. $$ \mathbb{V}\left(X\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\...
32
votes
3answers
32k views

How to take derivative of multivariate normal density?

Say I have multivariate normal $N(\mu, \Sigma)$ density. I want to get the second (partial) derivative w.r.t. $\mu$. Not sure how to take derivative of a matrix. Wiki says take the derivative ...
31
votes
4answers
77k views

Find expected value using cdf

I'm going to start out by saying this is a homework problem straight out of the book. I have spent a couple hours looking up how to find expected values and have determined I understand nothing. ...
31
votes
4answers
2k views

Is the result of an exam a binomial?

Here is a simple statistics question I was given. I'm not really sure I understand it. X = the number of aquired points in an exam (multiple choice and a right answer is one point). Is X binomial ...
28
votes
2answers
20k views

What is the difference between censoring and truncation?

In the book Statistical Models and Methods for Lifetime Data , it is written : Censoring: When an observation is incomplete due to some random cause. Truncation: When the incomplete nature of ...
27
votes
4answers
4k views

Self study vs a taught education?

There is a question with similar intent on programmers.SE. That question has some quite good answers, but the general theme seems to be that without self study, you get no-where. Obviously there are ...
27
votes
3answers
4k views

Which hospital should be chosen? One has a higher success rate, but the other has a higher overall success rate

I have a question about something that my statistics teacher said about the following problem. My question isn't even about the occurrence of Simpson's paradox in this situation. My question is simply ...
26
votes
1answer
1k views

What are the classical notations in statistics, linear algebra and machine learning? And what are the connections between these notations?

When we read a book, understanding the notations plays a very important role of understanding the contents. Unfortunately, different communities have different notation conventions for the formulation ...
25
votes
7answers
18k views

Two dice rolls - same number in sequence

I am currently studying Statistical Inference class on Coursera. In one of the assignments, the following question comes up. ...
25
votes
2answers
47k views

How to know if a data follows a Poisson Distribution in R?

I am an undergrad student and I have a project for my probability class. Basically, I have a dataset about the hurricanes that impacted my country for a series of years. In my probability Book, (...
25
votes
6answers
12k views

What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
24
votes
5answers
18k views

Why does the variance of the Random walk increase?

The random walk that is defined as $Y_{t} = Y_{t-1} + e_t$, where $e_t$ is white noise. Denotes that the current position is the sum of the previous position + an unpredicted term. You can prove that ...
24
votes
4answers
79k views

McFadden's Pseudo-R2 Interpretation

I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this ...
24
votes
3answers
20k views

Interpreting plot of residuals vs. fitted values from Poisson regression

I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What ...
23
votes
3answers
6k views

Functions of Independent Random Variables

Is the claim that functions of independent random variables are themselves independent, true? I have seen that result often used implicitly in some proofs, for example in the proof of independence ...
22
votes
8answers
46k views

Looking for a good and complete probability and statistics book

I never had the opportunity to visit a stats course from a math faculty. I am looking for a probability theory and statistics book that is complete and self-sufficient. By complete I mean that it ...
19
votes
3answers
6k views

Neyman-Pearson lemma

I have read the Neyman–Pearson lemma from the book Introduction to the Theory of Statistics by Mood, Graybill and Boes. But I have not understood the lemma. Can anyone please explain the lemma to ...
18
votes
4answers
626 views

What is the intuition behind the independence of $X_2-X_1$ and $X_1+X_2$, $X_i \sim N(0,1)$?

I was hoping someone could propose an argument explaining why the random variables $Y_1=X_2-X_1$ and $Y_2=X_1+X_2$, $X_i$ having the standard normal distribution, are statistically independent. The ...
18
votes
1answer
951 views

Quiz: Tell the classifier by its decision boundary

Given are the 6 decision boundaries below. Decision boundaries is violett lines. Dots and crosses are two different data sets. We have to decide which one is a: Linear SVM Kernelized SVM (Polynomial ...
17
votes
4answers
3k views

Can anyone clarify the concept of a “sum of random variables”

In my probability class the terms "sums of random variables" is constantly used. However, I'm stuck on what exactly that means? Are we talking about the sum of a bunch of realizations from a random ...
17
votes
3answers
28k views

Why is nls() giving me “singular gradient matrix at initial parameter estimates” errors?

I have some basic data on emission reductions and cost per car: ...
17
votes
9answers
2k views

Reference Request: Generalized Linear Models

I am looking for an introductory to intermediate level book on Generalized Linear Models. Ideally, in addition to the theory behind the models, I would want it to include applications and examples in ...
17
votes
4answers
23k views

How to test if my distribution is multimodal?

When I plot a histogram of my data, it has two peaks: Does that mean a potential multi-modal distribution? I ran the dip.test in R (...
17
votes
3answers
1k views

Why are “time series” called such?

Why are “time series” called such? Series means sum of a sequence. Why is it time Series, not time sequence? Is time the independent variable?
17
votes
2answers
403 views

Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, 1)$

What is the easiest way to see that the following statement is true? Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, ...
17
votes
1answer
494 views

Second moment method, Brownian motion?

Let $B_t$ be a standard Brownian motion. Let $E_{j, n}$ denote the event$$\left\{B_t = 0 \text{ for some }{{j-1}\over{2^n}} \le t \le {j\over{2^n}}\right\},$$and let$$K_n = \sum_{j = 2^n + 1}^{2^{2n}} ...
16
votes
1answer
7k views

Question on how to normalize regression coefficient

Not sure if normalize is the correct word to use here, but I will try my best to illustrate what I am trying to ask. The estimator used here is least squares. Suppose you have $y=\beta_0+\beta_1x_1$, ...
16
votes
4answers
2k views

Expected value of sample median given the sample mean

Let $Y$ denote the median and let $\bar{X}$ denote the mean, of a random sample of size $n=2k+1$ from a distribution that is $N(\mu,\sigma^2)$. How can I compute $E(Y|\bar{X}=\bar{x})$? Intuitively, ...
16
votes
1answer
11k views

A proof for the stationarity of an AR(2)

Consider a mean-centred AR(2) process $$X_t=\phi_1X_{t-1}+\phi_2X_{t-2}+\epsilon_t$$ where $\epsilon_t$ is the standard white noise process. Just for sake of simplicity let me call $\phi_1=b$ and $\...
16
votes
5answers
3k views

Probability theory books for self-study

Are there any good books that explain important concepts of probability theory like probability distribution functions and cumulative distribution functions? Please, avoid referring books like "...
15
votes
2answers
7k views

Simulating draws from a Uniform Distribution using draws from a Normal Distribution

I recently purchased a data science interview resource in which one of the probability questions was as follows: Given draws from a normal distribution with known parameters, how can you simulate ...
15
votes
4answers
2k views

Classic linear model - model selection

I have a classic linear model, with 5 possible regressors. They are uncorrelated with one another, and have quite low correlation with the response. I have arrived at a model where 3 of the regressors ...
15
votes
3answers
1k views

Why do we need Bootstrapping?

I'm currently reading Larry Wasserman's "All of Statistics" and puzzled by something he wrote in the chapter about estimating statistical functions of nonparametric models. He wrote "Sometimes we ...
15
votes
3answers
53k views

Expected number of tosses till first head comes up

Suppose that a fair coin is tossed repeatedly until a head is obtained for the first time. What is the expected number of tosses that will be required? What is the expected number of tails that will ...
15
votes
2answers
101k views

How do I calculate the variance of the OLS estimator $\beta_0$, conditional on $x_1, \ldots , x_n$?

I know that $$\hat{\beta_0}=\bar{y}-\hat{\beta_1}\bar{x}$$ and this is how far I got when I calculated the variance: \begin{align*} Var(\hat{\beta_0}) &= Var(\bar{y} - \hat{\beta_1}\bar{x}) \\ &...
15
votes
1answer
11k views

Difference between Hidden Markov models and Particle Filter (and Kalman Filter)

Here is my old question I would like to ask if someone knows the difference (if there is any difference) between Hidden Markov models (HMM) and Particle Filter (PF), and as a consequence Kalman ...
15
votes
1answer
4k views

Derivation of change of variables of a probability density function?

In the book pattern recognition and machine learning (formula 1.27), it gives $$p_y(y)=p_x(x) \left | \frac{d x}{d y} \right |=p_x(g(y)) | g'(y) |$$ where $x=g(y)$, $p_x(x)$ is the pdf that ...
15
votes
2answers
5k views

Hidden Markov Model vs Markov Transition Model vs State-Space Model…?

For my master's thesis, I am working on developing a statistical model for the transitions between different states, defined by serological status. For now, I won't give too many details into this ...
14
votes
8answers
4k views

How many 2-letter words can you get from aabcccddef

(aa would be one of many, bb would not) I thought it would be 10!/8! But apparently I'm doing something wrong. Can anyone help me out because I'm stumped.
14
votes
8answers
25k views

How is interpolation related to the concept of regression?

Explain briefly What is meant by interpolation.How is it related to the concept of regression? interpolation is art of reading between the lines of a table and in elementary mathematics the term ...
14
votes
1answer
4k views

Construction of Dirichlet distribution with Gamma distribution

Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$,...