0
votes
0answers
171 views

Interpreting 95%CIs - why perform formal statistical tests or multiple comparisons?

95% (or other %) confidence intervals indicate the range in which we expect the true mean to lie with 95% probability (= 'confidence') according to the Central Limit Theorem, given the precision of ...
5
votes
2answers
3k views

How is the Weighted Gini Criterion defined?

I am interested in trying out and/or implementing the Weighted Random Forest (WRF) algorithm described in Chen, Liaw, Breiman. How is the Weighted Gini impurity actually defined? What implementations ...
2
votes
1answer
4k views

(Multiple) Correspondence Analysis for count data entered as binary variables

I have a data set, 1014 cases and 55 variables which are binary and is in the form of ...
0
votes
1answer
108 views

Expected value of a product of random variables

Knowing the distribution, \begin{array}{c c c c} & \quad &X=1\quad &X=-1\quad &X=2 \\ &Y=1\quad &0.25\quad &0.25\quad &0 \\ &Y=2\quad &0\quad ...
1
vote
1answer
160 views

Expected value calculation

I have great difficulty in calculating $\mathrm{E}\left(\sqrt{T_{1} T_{2}}\right)$ where $T_{i}$ is a random variable distributed according to the Birnbaum-Saunders distribution. Are there any ...
3
votes
2answers
930 views

Can someone give a simple guide of Dirichlet process clustering?

I am trying to understand the Dirichlet process and Dirichlet process clustering, but because of my lack of knowledge of stochastic process theory and measure theory, I cannot understand what the ...
0
votes
0answers
127 views

Significance of adjacency in correlation matrix with ordered variables

The following values are from the Discrete module in BayesTraits. Each value in the matrix is the difference in log-likelihood values between a 4-parameter model of independent evolution and an ...
1
vote
0answers
45 views

Girsanov Theorem

I am interested in understanding Kallenberg's Proof of Girsanov's theorem. The Theorem says: If $Q=Z_{t}P$ and $Q$,$P$ are probability measures and $Z$ is a.s. continuous, then for all local ...
1
vote
2answers
364 views

Fit VAR model with unknown order in Matlab

I have a multivariate observed time series $Y_t$ and I want to find the best fitting VAR process for it. I have the econometric toolbox in Matlab and can use 'vgxvarx' if I pre-specify an order for ...
3
votes
1answer
732 views

K-means clustering for usage profiling

I am trying to use k-means clustering to profile mobile device usage behaviour for IT users. My data consists of different system and user level variable/readings like number of calls/sms, cpu/memory ...
0
votes
0answers
50 views

Unbalanced repeated measures

I'm new to using R and therefore I have some doubts. I have a data table with measures gland secretion of chickens at two ages different, sometimes we just need a measure therefore is not balanced. ...
2
votes
1answer
638 views

Performing multiple comparisons on linear mixed model

I have collected data on gas fluxes from plots of soil subjected to 5 different treatments ("D2", "K2", "M", "N", and "O2"), which also possessed variable clay contents. The experiment was laid out in ...
0
votes
0answers
122 views

A concise way to describe the distribution of words in a text

I have a text, and I count, for each distinct word, the number of times that word appears in the text. In some texts, there are many rare words, so the distribution will have a long "tail" of words ...
3
votes
2answers
154 views

Which statistic to prove that one data point dominates the proportions in dataset?

I'm very new to statistics so apologies if this is a dumb question. I've got a grasp of the concepts but not real-life usage of them (but I am learning!). I have a skewed distribution where one ...
3
votes
1answer
224 views

Simple question about variance

I always thought I understood the concept of variance, but this one confuses me. $ \begin{eqnarray} X & := & (1, 2, 3)\\ E(X) & = & (1 + 2 + 3) / 3 = 2\\ Var(X) & = & Cov(X, ...
2
votes
1answer
639 views

1D weighted Isotonic Regression (PAV): a simple description of the algorithm

I have to use IsotonicRegression class from scikit-learn with non-uniform point weights: in method IsotonicRegression.fit parameter ...
11
votes
3answers
14k views

Area under curve of ROC vs. overall accuracy

I am a little bit confusing about the Area Under Curve (AUC) of ROC and the overall accuracy. Will the AUC be proportional to the overall accuracy? In other words, when we have a larger overall ...
0
votes
0answers
160 views

Floating point error while computing correlation for transformed data using proc corr

I get a floating point overflow error while using proc corr to compute correlations among transformations (exponential and log) ...
5
votes
2answers
844 views

Does it make sense to compute confidence intervals and to test hypotheses when data from whole population is available?

Does it make sense to compute confidence intervals and to test hypotheses when data from the whole population is available? In my opinion, the answer is no, since we can accurately compute the true ...
1
vote
0answers
444 views

Uniformly Most Powerful (UMP) test for Uniform(t,t+1)

Suppose $X_1,...,X_n$ follow a $U(t,t+1)$ distribution. Show that the UMP level alpha test for testing $H_0:t<=t^*$ vs $H_1:t>t^*$ has form: Reject $H_0$ if and only if ...
0
votes
1answer
94 views

Comparison of horizon forecasted values for two models (ARIMA and NAR)?

I fitted ARIMA and NAR models to 10 different univariate time series. These time series are water characteristics at a water treatment plant (CO$_2$, TDS, TH, pH, ...). After fitting suitable models ...
1
vote
1answer
409 views

Statistical notation in plain English

I have a very limited understanding of statistics, and am having trouble translating an answer to this question from quant.SE, "Separating the wheat from the chaff", the answer being: Larry Harris ...
0
votes
2answers
135 views

algorithm for finding similar items

I have a group of items S, say there are 100 items in S, and I know some features of these items, i.e., color, size, ...Now, I have another group of items P, say there are 10000 items in P. What can I ...
3
votes
0answers
60 views

Estimating the parameters of a normal distribution of probabilities

I know the question sounds weird or perhaps, silly. And my limited knowledge in statistics might have caused ambiguity in presenting the problem. But still, here's the case: Suppose there's a ...
1
vote
0answers
163 views

Confirmatory factor analysis for a newly constructed questionnaire

I am about to develop a new questionnaire with 142 questions I have performed exploratory factor analysis and 30 latent factors were extracted. I was wondering if it is necessary to perform ...
1
vote
1answer
554 views

Bootstrapping of RNA-Seq data: normal distribution?

I have an RNA_Seq dataset (FPKM values) obtained from the ENCODE Caltech datasets for K562 cells; I also have 6 different lists of genes and I want to show that the expression values of my panel of ...
4
votes
1answer
264 views

General theorems for consistency and asymptotic normality of maximum likelihood

I'm interested in a good reference for results concerning asymptotic properties of maximum likelihood estimators. Consider a model $\{f_n(\cdot \mid \theta): \theta \in \Theta, n \in \mathbb N\}$ ...
0
votes
2answers
2k views

Differences between planning, prediction and forecasting

What are the differences between planning, prediction and forecasting in relation to data mining and business intelligence? It would be great if you could provide some concrete examples of these ...
0
votes
0answers
213 views

Explaining the difference between baseline data and subsequent experiment data if 1.00 is baseline

I have what I guess is a simple question really and it's probably as much about the English language as about stats. I should explain that this isn't my strong point at all but I have to write up ...
1
vote
1answer
174 views

What are the most common subsets?

This has been originally asked at math.stackexchange where it was suggested that I try Cross Validated. My background is not mathematics. This is an IT problem that may have a mathematical solution: ...
1
vote
1answer
147 views

Poisson vs Negative Binomial

In the case of count data where my variance is 6.34 and mean is 5.44, which would be a better fit to the data - Poisson or negative binomial? And what will be the assumptions ?
0
votes
1answer
429 views

Entering dummy variables and other covariates into the regression model

I am controlling for education-level (two dummy variables) and another covariate in my regression model. I'm conducting a hierarchical regression analysis. I was confused about how I should be ...
1
vote
0answers
427 views

Help needed with interpreting mixed-model and factorial ANOVA

I really need help with an analysis I am doing. The study involves a comparison of two different treatments for bulimia (weight concern treatment and perfectionism treatment) and a combination of the ...
1
vote
1answer
145 views

Machine learning to catch fraud

I work for a company that ships material (a couple of thousand shipments per day) around the world. In order to ship anything a customer has to declare the weight of a shipment and declare the ...
0
votes
1answer
236 views

Linear regression: explanation of slope constraints

I am given regression equations, one showing linear regression of x on y, and the other y on x. Both refer to the same set of data-points. ...
5
votes
2answers
819 views

Variance of a smoothed AR(1) process

The query I have relates to calculating the variance of AR(1) processes that are smoothed with a simple moving average. So: In an AR(1) process of the form: $$ X_t=c+\varphi X_{t-1}+\varepsilon_t, ...
32
votes
5answers
7k views

What is the benefit of breaking up a continuous predictor variable?

I'm wondering what the value is in taking a continuous predictor variable and breaking it up (e.g., into quintiles), before using it in a model. It seems to me that by binning the variable we lose ...
0
votes
1answer
3k views

Random effects in mixed models

I am not very familiar with mixed models --- I have been reviewing the various tutorials on the web but still am not sure how to specify my model, even as a starting point for comparison. I have a ...
1
vote
1answer
132 views

Low intensity Poisson estimation

I have a collection of Poisson processes each with an unknown $\lambda$. I would like to estimate $\lambda$ for each process. for each process I could take either the total number of event over the ...
4
votes
0answers
2k views

Comparing SEM software apps: AMOS vs Stata vs Mplus vs R -— opinions please? [closed]

I have AMOS and Stata, and I have a colleague who has Mplus (and I'd rather not ask the colleague to run my data for me). I recently discovered that Stata has SEM capabilities. I looked online, but I ...
-1
votes
2answers
75 views

What can be inferred from this data?

Suppose you ask 100 people, would you purchase this item for $1000? You get responses that fall into one of three categories, Yes, No or Maybe. Assuming you get a non-trivial number of each replies, ...
4
votes
1answer
2k views

auto.arima and prediction

I'm really new to ARIMA methods and am trying to forecast electricity load. I've integrated: electricity load, temperature, weekday (dummy), public holidays, and school holidays. My model tries to ...
7
votes
2answers
2k views

Conditional vs unconditional expectation

I'm having trouble understanding the calculation of conditional versus unconditional expectations in this case: \begin{array}{c c c c} & \quad &X=1\quad &X=-1\quad &X=2 \\ ...
3
votes
0answers
199 views

Plotting fit for binomial lme

I've been asked by a reviewer on a manuscript to provide plots of a model fit for a binomial lme which is specified as follows: ...
3
votes
1answer
2k views

How to calculate the Bayesian posterior probability from observations?

Let's say I give you a 10-sided die and tell you that it is biased in some way. I then let you roll the die 5 times and you observe, let's say, 8, 8, 8, 2, and 8. Lastly, I ask you what your belief is ...
3
votes
0answers
138 views

Predicting for month in R

I'm trying to understand some concepts related to predictive modeling. So let's say that I have the following data sample and am trying to regress sales on ...
4
votes
1answer
1k views

Stochastic inequality with product of binomial distributions

I am interested in modeling the following experiment: A binomial trial with $n$ Bernoulli experiments is run. For the k positive outcomes a second (independent) binomial trial with $k$ runs with a ...
-2
votes
1answer
107 views

Can we never have a normal distribution?

Since the population variance can never be estimated, will we always have a t-distribution instead of normal?
0
votes
1answer
310 views

Bootstrapping Average Treatment Effects and Confidence Intervals for multinomial logit models

I have a multinomial logit model with three outcome categories and a variety of (primarily binary) independent variables. How would I go about bootstrapping the average treatment effect and its ...
1
vote
1answer
169 views

Sentiment Analysis with respect to subject

I'm familiar with the bag of words/Naive bayes sentiment analysis for text (e.g. http://streamhacker.com/2010/05/10/text-classification-sentiment-analysis-naive-bayes-classifier/) I was curious to ...

15 30 50 per page