All Questions

7
votes
1answer
8k views

How do you compute confidence intervals for positive predictive value?

How do you compute confidence intervals for positive predictive value? The standard error is: $$SE = \sqrt{ \frac{PPV(1-PPV)}{TP+FP}} $$ Is that right? (here my concern is the denominator) Does ...
6
votes
2answers
262 views

Estimating event probability from historical time series with clear seasonality

I would like to predict the average number of days in a year for which two conditions are true: daily average temperature is below zero celsius the day was preceded by at least four days with daily ...
3
votes
0answers
3k views

How to carry out a Likert scale analysis?

Context: My research aims to assess whether parents who have been involved in a relationship with a social worker are at higher risk of child abuse/neglect. I am trying to establish whether there ...
1
vote
1answer
213 views

Boosting for regression systems

I am quite a newbie in this area: What are the boosting methods for regression systems? I know about Gradient boosting; are there any other approaches? Are there textbooks or tutorials devoted to ...
4
votes
1answer
550 views

Displaying relationships between nodes [closed]

I'm trying to create something similar to this. So, 3 different Node classes, and a whole bunch of relationships between them. In my case, there should be roughly half of the number of nodes present ...
8
votes
2answers
1k views

How can I correct for measurement error in the dependent variable in a logit regression?

I'm running a binary logit regression where I know the dependent variable is miscoded in a small percentage of cases. So I'm trying to estimate $\beta$ in this model: $prob(y_i) = 1/(1 + e^{-z_i})$ ...
5
votes
2answers
3k views

Correct specification of longitudinal model in lme4

I am trying to fit a multilevel longitudinal model and i have a question regarding how to specify it. The data consist of about 8k observations collected from about 3k individuals at four time ...
3
votes
1answer
5k views

Interpreting Gaussian probabilities greater than 1 [duplicate]

Possible Duplicate: Probability distribution value exceeding 1 is OK? I'm a bit confused how I am getting probabilities greater than 1 when calculating p(x | mu, sigma) when x = mu. For example, ...
17
votes
1answer
76k views

Bonferroni or Tukey? When does the number of comparisons become large?

Reading Field's Discovering Statistics Using SPSS (3rd Edition) I was struck by a bit about post-hoc tests in ANOVA. For those wanting to control the Type I error rate he suggests Bonferroni or Tukey ...
6
votes
2answers
197 views

Resampling within a survey to account for missing data

Suppose I have survey responses that look like this: ...
2
votes
2answers
508 views

Notation of probability matrix corresponding to a contingency table

Contingency tables tables are typically formatted as tables similar to matrices in mathematics, see this example. Is the equation below an accepted notation of expressing the probabilities of the ...
1
vote
1answer
251 views

How to find the effective bandwidth correctly using statistics?

I have done an experiment to find the effective n/w bandwidth. The data I got in kbps is 223, 221, 510, 220, 471, 229, 222, 221, 220, 221 How can I find the effective bandwidth? Averaging gives ...
10
votes
3answers
4k views

Measuring non-linear dependence

Covariance between two random variables defines a measure of how closely are they linearly related to each other. But what if the joint distribution is circlular? Surely there is structure in the ...
4
votes
2answers
12k views

Statistical power and minimum sample size for ANOVA with likert scale as dependent variable

I'm trying to compute the minimum sample size for a psychometric test based on 7 point Likert scales. I'd like to run ANOVA on each scale to look for differences between groups. Most online survey ...
0
votes
2answers
2k views

Simulation of maximum likelihood ratio test to test two poisson random variables

I have two random poisson variables $x_1$ and $x_2$ with value 10 and 25 respectively. I am interested to use likelihood ratio test to test the null hypothesis: $\lambda_1=\lambda_2$, versus alernate ...
6
votes
2answers
370 views

How can I compute regression for several longitudinal data sets (thus, with auto-correlated error)?

My actual project is a bit complicated, but I'll explain by analogy (which I hope facilitates response): I have 3 substances, say water, motor oil, and ethanol. For each substance, I have 5 samples ...
13
votes
2answers
3k views

Supervised learning with “rare” events, when rarity is due to the large number of counter-factual events

Suppose you get to observe "matches" between buyers and sellers in a market. You also get to observe characteristics of both buyers and sellers which you would like to use to predict future matches &...
6
votes
1answer
2k views

Why are cumulative residuals from regression on stock and index returns mean reverting

In the paper M. Avellaneda and J. H. Lee, Statistical arbitrage in the U.S. equities market, July 2008, in the Appendix on page 44, I have some questions. First he runs the regression of stock-...
5
votes
2answers
267 views

How can I demonstrate non-linearity without categorising a predictor?

I don't know what is the appropriate term for my question. The scenario is described as following. In the analysis there one dependent variable Y and two independent variable X1 and X2. All three ...
0
votes
1answer
218 views

How to get scored combination of features

My data looks like this (F=Features) ...
9
votes
2answers
1k views

What's the probability that a bookmaker is mispricing odds on soccer games?

An English soccer team plays a series of matches against different opponents of varying ability. A bookmaker offers odds for each match as to whether it will be a home win, away win, or draw. Part-way ...
7
votes
3answers
10k views

Forecasting beyond one season using Holt-Winters' exponential smoothing

I am using the Holt-Winters' exponential smoothing technique to forecast expenditure data 2 years into the furture. The monthly data has an increasing trend and annual seasonality. I'm using MS Excel ...
1
vote
1answer
398 views

What are the correct edge case values of precision and recall and how to integrate them into cross validation?

I'm doing 10-fold cross validation on a dataset. But in some folds there are edge cases that the denominator in precision-recall calculation is zero (tp + fp =0). What are the correct values for ...
4
votes
0answers
140 views

Inference and noninference parts of statistical theory [closed]

I was wondering from the view of dividing the topics of statistical theory into inference part and non-inference part, what inference topics and non-inference topics statistical theory is covering? ...
4
votes
1answer
5k views

How to choose a data subset in RapidMiner?

I'm working with a CSV which contains approximately 220,000 entries. My aim is to predict one of the attributes (ATT1) using the other 3 (ATT2, ATT3, ATT4). I've been able to do this using NaiveBayes,...
7
votes
1answer
746 views

Origin of strange formula for equilibrium standard deviation

In the paper M. Avellaneda and J. H. Lee, Statistical arbitrage in the U.S. equities market, July 2008, in the Appendix on page 46, how does he get equilibrium standard deviation as following: $$...
2
votes
2answers
359 views

How do I determine confidence intervals around weighted, probilized events?

How would I go about finding out the confidence intervals around a set of distinct bianary occurrences where each occurence has a different associated probability and each occurence is weighted? To ...
28
votes
6answers
6k views

What are some interesting and well-written applied statistics papers?

What are some good papers describing applications of statistics that would be fun and informative to read? Just to be clear, I'm not really looking for papers describing new statistical methods (e.g., ...
21
votes
4answers
28k views

Creating an index of quality from multiple variables to enable rank ordering

I have four numeric variables. All of them are measures of soil quality. Higher the variable, higher the quality. The range for all of them is different: Var1 from 1 to 10 Var2 from 1000 to 2000 ...
58
votes
2answers
31k views

Why only three partitions? (training, validation, test)

When you are trying to fit models to a large dataset, the common advice is to partition the data into three parts: the training, validation, and test dataset. This is because the models usually have ...
9
votes
2answers
2k views

How does one appropriately apply cross-validation in the context of selecting learning parameters for support vector machines?

The wonderful libsvm package provides a python interface and a file "easy.py" that automatically searches for learning parameters (cost & gamma) that maximize the accuracy of the classifier. ...
6
votes
1answer
353 views

Can anyone explain why I have obtained an anti-predictive Support Vector Machine?

I'm playing with support vector machines (SVM) using the e1071::svm() function in R, and I encountered a scenario where I asked it for a leave-one-out cross-validated classification of a 2-category ...
1
vote
1answer
82 views

Why doesn't it seem to be standard to multiply prior scale by fraction of non-zero predictor observations?

When doing multivariate regression, it's often the case that some predictors often have many zero values - dichotomous inputs, dummy coding of polychotomous inputs, interval coding, etc. The fraction ...
8
votes
1answer
153 views

Is there an equivalent of ARMA for rank correlation?

I am looking at extremely non linear data for which the ARMA/ARIMA models do not work well. Though, I see some autocorrelation, and I suspect to have better results for non linear autocorrelation. 1/ ...
15
votes
3answers
5k views

Is it ok to use Manhattan distance with Ward's inter-cluster linkage in hierarchical clustering?

I am using hierarchical clustering to analyze time series data. My code is implemented using the Mathematica function DirectAgglomerate[...], which generates ...
5
votes
1answer
221 views

Regularized fit from summarized data

I have a multiple linear regression problem $y=X\beta+\epsilon$. The number of observations $m$ is large, so by the time the data gets to me it's been summarized into: $m$ $X^TX$ $X^Ty$ $y^Ty$ $\sum_{...
18
votes
4answers
10k views

Determining best fitting curve fitting function out of linear, exponential, and logarithmic functions

Context: From a question on Mathematics Stack Exchange (Can I build a program), someone has a set of $x-y$ points, and wants to fit a curve to it, linear, exponential or logarithmic. The usual ...
3
votes
1answer
81 views

Determining the relationship between salesperson and products sold?

Context: I have a series of figures for car sales that show me (a) the usual number of car sales for particular models and (b) the number of car sales by a particular car salesperson for each model. ...
5
votes
2answers
3k views

Calculation of natural cubic splines in R

I am new to the use of cubic splines for regression purposes and wanted to find out 1) What is a good source (besides ESL which I read but am still uncertain) to learn about splines for regression? ...
7
votes
3answers
735 views

Binomial data analysis with all 0 responses for some treatment groups

We have a series of experiments where we measure virus transmission to plants when exposed to virus-infected insects for different time periods, so all of the experiments have similar types of ...
11
votes
1answer
2k views

Is there a way to explain a prediction from a random forest model?

Say I've got a predictive classification model based on a random forest (using the randomForest package in R). I'd like to set it up so that end-users can specify an item to generate a prediction for, ...
4
votes
2answers
4k views

Count data with mixed effects

EDIT: Thanks for the help so far. I have updated the question based on further work. I am interested in finding any difference in response of counts of two competing species to dryness and elevation....
0
votes
2answers
435 views

Check of R command and output of unbalanced repeated measures ANOVA

I'd love a check if anyone is willing! I am trying to see if there is a statistical difference in female size between sites. Over the years females were repeatedly sampled within sites. I have ...
12
votes
1answer
6k views

Conditional homoskedasticity vs heteroskedasticity

From Econometrics, by Fumio Hayashi (Chpt 1): Unconditional Homoskedasticity: The second moment of the error terms E(εᵢ²) is constant across the observations The functional form E(εᵢ²|xi) is ...
5
votes
2answers
3k views

Multidimensional scaling pseudo-code

I am planning to write a program that performs MDS. Any pointers to where I can access the pseudo-code for MDS? Thanks!
5
votes
1answer
398 views

Statistical equivalence in the birthday paradox

I was reading a Wikipedia article on the birthday paradox and stumbled upon the following statement: ...the pairings in a group of 23 people are not statistically equivalent to 253 pairs chosen ...
15
votes
1answer
9k views

Topic prediction using latent Dirichlet allocation

I have used LDA on a corpus of documents and found some topics. The output of my code is two matrices containing probabilities; one doc-topic probabilities and the other word-topic probabilities. But ...
6
votes
2answers
514 views

Review of case control matching algorithms?

Can you suggest a good review of case control matching algorithms? Algorithms that can be used to set up the matched pairs of one case and one control, or matched blocks of a case and multiple ...
3
votes
2answers
29k views

How to compute the standard error of measurement (SEM) from a reliability estimate?

SPSS returns lower and upper bounds for Reliability. While calculating the Standard Error of Measurement, should we use the Lower and Upper bounds or continue using the Reliability estimate. I am ...
34
votes
2answers
7k views

Kullback–Leibler vs Kolmogorov-Smirnov distance

I can see that there are a lot of formal differences between Kullback–Leibler vs Kolmogorov-Smirnov distance measures. However, both are used to measure the distance between distributions. Is there a ...

15 30 50 per page