All Questions

1
vote
0answers
540 views

Statistical test for multiple conditions and features with replicates [closed]

I have data from a biological experiment of the following form: ...
8
votes
1answer
789 views

Mixing and dividing point processes

At the following figure at left side two realizations of point processes with different density (intensity) $\lambda_1$ and $\lambda_2$ is being mixed matching the center of the belonging areas to ...
13
votes
3answers
1k views

How to justify the error term in factorial ANOVA?

A probably very basic question about multi-factorial ANOVA. Assume a two-way design where we test both main effects A, B, and the interaction A:B. When testing the main effect for A with type I SS, ...
2
votes
1answer
200 views

If F = F1+F2 is some measured random field, how to find out if F2 has been correctly estimated and subtracted?

I have a random field, F, defined on the surface of a sphere pixellized into about a million points. I know that F must consist of two components - F1 and F2, F1 being a true signal and F2 being some ...
1
vote
1answer
367 views

Simple exponential smoothing

I simulated a time series using expressions (3.10a), (3.10b) from (Hyndman et al., 2008). Next, I'd like to use a simple exponential smoothing method to forecast for the next period. For a given ...
10
votes
1answer
6k views

Calculation of canonical link function in GLM

I thought that the canonical link function $g(\cdot)$ comes from the natural parameter of exponential family. Say, consider the family $$ f(y,\theta,\psi)=\exp\left\{\frac{y\theta-b(\theta)}{a(\psi)}-...
6
votes
4answers
2k views

Chi square statistic and contingency coefficients

I need some advice on explaining $\chi^2$ as a contingency measure from a pedagogical perspective. I usually introduce the individual terms in the $\chi^2$ sum as scaled deviations from independence; ...
1
vote
1answer
2k views

Why proc mixed give different DF than proc glimmix?

I am trying to move data analysis from proc mixed to glimmix for the easy of the ilink transformation. However I am finding a big difference on the denominator DF when using either of them. As ...
3
votes
1answer
652 views

Estimating upper bound of uniform distribution from max of sample

This is actually part of a problem from All of Statistics: $X_1, X_2, \ldots, X_n \sim \text{Uniform}(0, \Theta)$. And $Y = \text{Max}\{X_1,\ldots, X_n\}$. If you're given that $Y > c$, can you ...
10
votes
2answers
5k views

Hypothesis testing and total variation distance vs. Kullback-Leibler divergence

In my research I have run into the following general problem: I have two distributions $P$ and $Q$ over the same domain, and a large (but finite) number of samples from those distributions. Samples ...
0
votes
1answer
4k views

How do you plot interaction effects in SAS? [closed]

I want to plot interaction between two variables in SAS. One is continuous and the other is binary.
7
votes
1answer
1k views

Is my data distribution normal? (Tried Shapiro and Kolmogornov-Smirnov tests)

I have a 1D data set with 83163 data points, and I would like to know whether the data follows a normal distribution. I tried using shapiro.test and ks.test in R: d is a vector containing the data ...
12
votes
2answers
793 views

Parametric modelling of variance of count data

I am looking to model some data, but I am not sure what type of model I can use. I have count data, and I want a model that will give parametric estimates of both the mean and the variance of the ...
2
votes
3answers
6k views

PCA to decorrelate variables

I have 2 variables that I want to decorrelate. I was told I can use PCA to do so. I did PCA on the data and got all the parameters. Now how do I get the new set of transformed data that no longer ...
1
vote
1answer
3k views

What does T statistics of information coefficient indicate?

Hi I am looking for a clear explanation of T statistics concept. Especially in quantitative equity portfolio management context, what does T statistics of information coefficient indicate? How can I ...
3
votes
1answer
145 views

Differences in coefficients

Suppose I want to see whether $z$ is a confounder for a model with $y$ the outcome variable and $x$ the predictor. If I adjust for $z$, and the adjusted coefficient of $x$ changes versus the ...
4
votes
1answer
838 views

Compare modeled (fitted) paired data to actual data in forecasting problem (Excel sheet included)

In my endeavor to learn forecasting and improve my statistical knowledge, I've decided to forecast the population of a certain area. I've attached the excel file that I used. I know I'm using excel ...
4
votes
2answers
761 views

Is sequential Bayesian updating an option when using MCMC?

I have an implementation of the Griddy Gibbs sampler, but my observations on which I'm conditioning model parameters are too many in number, thus the likelihood underflows quickly, even with a log ...
12
votes
4answers
2k views

Examples of costly consequences from improper use of statistical tools

I suspect that most users of statistical tools are ancillary users (folks who have had little to no formal training in statistics). It’s very tempting for researchers and other professionals to apply ...
1
vote
1answer
98 views

What is the probability of observing not more than N positive outcomes

I have the population which is split into three classes; A, B and C. We know that the number of observations in each class is Na, Nb and Nc respectively. Assuming that my observations are binary ...
2
votes
1answer
29k views

How to obtain detrended data and demeaned data in Stata?

I would like to run KSS test (KAPETANIOS, G. – SHIN, Y. – SNELL, A. 2003: Testing for a UnitRoot in the Nonlinear STAR Framework) , but I don't know how to obtain demeaned data and detrended data. ...
5
votes
2answers
38k views

Interpreting coefficients from a VECM (Vector Error Correction Model)

I would like to ask a question about error correction terms from VECM if I may. I am currently working on a lot of time-series data and one of the questions I would like to address is whether there is ...
4
votes
3answers
19k views

Odd problem with a histogram in R with a relative frequency axis

Using either truehist() from MASS or just the normal hist() function in R with the prob=TRUE option, I'm getting very strange values for the y-axis. I was under the impression that these values should ...
5
votes
2answers
12k views

How to calculate mean and standard deviation of a count variable when the raw data is based on frequency categories?

I asked 312 people how many times they visited their preferred local supermarket in a month The results are as follows: 5% did not visit at all 7% visited once a month 33% visited twice a month 22% ...
80
votes
3answers
77k views

What is the lasso in regression analysis?

I'm looking for a non-technical definition of the lasso and what it is used for.
1
vote
1answer
314 views

How to develop intuition about parameter estimation in AR-GARCH model?

consider a typical ar-garch model: y = sum( bi*xi ) + epsilon, epsilon~garch(p,q) In such case a typical textbook says that we should first estimate all bi's ...
4
votes
1answer
508 views

Looking for correct distribution for modelling foosball scoring

I'm trying to adapt something like this association football prediction approach to 2v2 foosball games at our office. For football, they basically have an offense score and a defense score for each ...
1
vote
1answer
233 views

Finding the distribution and/or fitting a model (to a biological problem)

I am a bioinformatician and work on RNA-Seq data. The data contains a lot of reads (of length 80 bp in my case). These reads are fragments of those genes that were expressed. I map them back to my ...
4
votes
2answers
2k views

Vertical line graphs in R

I want to draw vertical line graph (like the first graph on this page): The deal is I have about ~15k data points whose frequencies (along Y axis) I want to present. I also want to be able to to ...
2
votes
2answers
1k views

What test to use for overlapping categories

i am analyzing a surveillance tool where people could be collected by one or both methods. So it kinda looks like a Venn diagram. What test would I use to test if there's significant differences by ...
8
votes
2answers
1k views

How to explain linear mixed models to laypeople?

I need to explain the concept of linear mixed models in an article targeted at a mainstream audience. Is there a way of communicating the gist of the concept in a sentence or two?
2
votes
1answer
1k views

Second moment of draws from a multivariate normal covariance matrix

Suppose I have an $N\times N$ covariance matrix that describes a multivariate normal joint distribution. Now take 100,000 draws of the covariance matrix. I measure the variance of values for each ...
3
votes
1answer
458 views

Mean Survival Time: Comparison of Continuous and Discrete Time

First, the expected survival time when time is time to event is treated as a continuous r.v. is given as the following: $$\int_0^{\infty} S(t)\, dt$$ The formula for the mean residual survival time $...
14
votes
8answers
4k views

What are the “hot algorithms” for machine learning?

This is a naive question from someone starting to learn machine learning. I'm reading these days the book "Machine Learning: An algorithmic perspective" from Marsland. I find it useful as an ...
3
votes
1answer
3k views

How to calculate a confidence Interval for a mean of subjects from two sessions?

How to calculate confidence interval for this mix of independent(different subjects) and dependent (data of same subject from two sessions) observations? There are 12 subjects and the mean is ...
14
votes
1answer
5k views

When over/under-sampling unbalanced classes, does maximizing accuracy differ from minimizing misclassification costs?

First of all, I would like to describe some common layouts that Data Mining books use explaining how to deal with Unbalanced Datasets. Usually the main section is named Unbalanced Datasets and they ...
7
votes
1answer
3k views

L1 regression versus L2 regression

I am trying to develop an intuition for why L1 regression is more expensive than L2 regression. Can somebody point me to some material that explains why this is the case
1
vote
1answer
776 views

Restricted mean residual life: Survival analysis: Discrete

I am hoping someone can help with this - perhaps trivial question of interpreting the formula below. It is the mean residual lifetime (restricted to the window of data) for the case of discrete time. ...
6
votes
1answer
3k views

Estimating error from repeated measurements

Say we use a ruler to make a measurement of the width of a block of wood. We get some value like 4.2 +/- 0.1 cm, where the error is our estimated error of our ruler'...
3
votes
2answers
397 views

Coordinate descent for binomial decision in elastic net for logistic regression

I'm looking to implement a coordinate descent solution to the Elastic Net (Friedman et al, 'Regularization paths for generalized linear models via coordinate descent') for logistic regression. The ...
5
votes
1answer
973 views

Usage of the Friedman test

I have done an experiment in which I have measured the energy required to chew three different food samples on 10 subjects with 3 repetitions per sample. I am interested therefore in finding out ...
5
votes
2answers
2k views

General linear hypothesis test statistic: equivalence of two expressions

Assume a general linear model $y = X \beta + \epsilon$ with observations in an $n$-vector $y$, a $(n \times p)$-design matrix $X$ of rank $p$ for $p$ parameters in a $p$-vector $\beta$. A general ...
12
votes
3answers
327 views

Is dynamically increasing the sample size okay, if stated a priori?

I'm about to do a study about the merits of one stimulus compared to another with a within-subject design. I have a permutation scheme that is designed to reduce order effects of some parts of the ...
15
votes
4answers
37k views

Interpreting exp(B) in multinomial logistic regression

This is somewhat of a beginner's question, but how does one interpret an exp(B) result of 6.012 in a multinomial logistic regression model? 1) is it 6.012-1.0 = 5.012 = 5012% increase in risk? or 2)...
8
votes
2answers
2k views

Is there a reference that legitimises the use of the unpooled z-test to compare two proportions?

The z-test to compare two proportions is $\newcommand{\p}{\hat{p}}\newcommand{\v}{\mathrm{Var}} z=\frac{\p_1-\p_2}{\sqrt{\v(\p_1-\p_2)}}$. Usually it is defined that $$\v(\p_1-\p_2)=\p(1-\hat{p})(1/...
2
votes
1answer
139 views

Chi square based measure for fairness?

I have some project categories $(PC_1, ... ,PC_n)$ and some institutions $(I_1, ..., I_n)$ and would like to determine how fairly projects are assigned to institutions and their project categories by ...
10
votes
2answers
508 views

A good book with equal stress on theory and math

I have had enough courses on statistics during my school years and at the university. I have a fair understanding of the concepts, such as, CI, p-values, interpreting statistical significance, ...
6
votes
2answers
534 views

Is there a good reason for the name “Missing at Random”?

Non-statisticians can be easily misled by the phrase "missing at random". Their natural interpretation tends to be closer to MCAR than to MAR. I had to explain the concept to an ecologist today, and ...
5
votes
1answer
180 views

Estimating a p-value when you can't compute it for the whole set

I have a two large lists and I want to compute the p-value that they are similar. The similarity algorithm is a black-box, but for this application we will trust it gives accurate p-values. My ...
2
votes
0answers
161 views

Testing endogeneity against decision probability

I am working with a model where the literature suggests there is potential endogeneity between the dependent variable and the primary independent variable of interest (a binary treatment). However, ...

15 30 50 per page