# All Questions

6 views

### What is the name of this validation procedure?

I have a set of classified data. In order to test the precision of several algorithms- I split the data into train and test sets. For the test set I choose at random 30% of the data and the rest is ...
11 views

### Sample size calculation for proportion

I am in a doubt about using Absolute and Relative precision for sample size calculations. Suppose if I want to conduct a study to asses the prevalence of Hypertension in a general population, which ...
14 views

### Multiple regression - how to deal with mixed linear and non-linear variables

Say I have a bunch of explanatory variables to predict a continuous independent variable. Below, a simple toy example: I think it would be easiest to do a log-log transform and proceed with linear ...
8 views

### Need to clarify certain doubts regarding Testing Hypotheses

I have some doubts in Hypothesis Testing which I would want to clarify. We know, by Neyman Pearson Lemma that the test which rejects $H_0$ when $\dfrac{f_1(x)}{f_0(x)}>\dfrac{k}{1}$ minimises the ...
6 views

### Confidence intervals of fitted Weibull survival function?

I'm implementing a Weibull survival analysis fitter, and have successfully estimated the parameters and their standard errors. I can also produce the fitted survival curve. My question is how can I ...
9 views

### Fourier/trigonometric interpolation revisited

Earlier, a user posted the Fourier transform equations of Epstein (1991), and now I am about to use these formulas to produce daily means from given monthly means. Could anyone explain why the last ...
10 views

### Wilcoxon Signed- Rank Test vs Kruskal Wallis vs One Way Anova? Which one to use

I am conducting research to rank the importance of certain elements within a property management framework. Would like to know which test is best to help me rank the items. The methodology is as ...
24 views

### What is variance of an election poll?

I was working through Harvard data science course: I came across this question: Assume you have $M$ polls with sample sizes $n_1,\ldots,n_M$. If the polls are independent, what is the average of the ...
4 views

### SPSS Multinomial Logistic Regression Interaction Term Reference

I am building a multinomial logistic regression model in SPSS. The dependent variable has 4 levels. There are two categorical factors; one factor has 4 levels, the other factor has 5 levels. I am also ...
14 views

### Best practices for documenting a data-science pipeline

Even though I try to keep it as simple as possible, the pipelines for some of my data science projects get rather complex. At some point it becomes necessary to document this pipeline so that someone ...
4 views

### Generating function from the infinitesimal generator of a continuous-time markov chain?

Take a generic $N$-state, continuous-time markov chain with infinitesimal generator $\mathbb{Q} \in \mathbb{R}^{N\times N}$. Then the evolution of the probability distribution of the $N$ states is ...
35 views

### Will two distributions with identical 5-number summaries always have the same shape?

I know that if I can have two distributions with the same mean and variance be different shapes, because I can have a N(x,s) and a U(x,s) But what about if their min, Q1, median, Q3, and max are ...
12 views

### Hierarchical Dirichlet Processes in topic modeling

I think I understand the main ideas of hierarchical dirichlet processes, but I don't understand the specifics of its application in topic modeling. Basically, the idea is that we have the following ...
3 views

### Multi-Level within factors in repeated measures ANOVA in SAS

This is a brief sketch of design containing both within and between subject factors. There are two groups. (each 10 subjects). Each subject responded many (over 600) trials. The trials are one of ...
5 views

### Comparable Mutual Information and Conditional Mutual Information values

Let's assume we have three continuous or discrete variables, X, Y and Z, for which I want to calculate Mutual Information (MI) and Conditional Mutual Information (CMI). The formulas to calculate this ...
6 views

### Apply Cross Validation to a prediction linear regression model

Hello I am slightly confused on how to apply a k-fold cross validation to a prediction linear regression model. Also how can i look at the data that the prediction model outputs before i apply cross ...
6 views

### SAS Fisher test p values for large sample sizes

I'm trying to calculate some odds ratios and significance forsomething that can be out into a 2x2 table. The problem is the Fisher test in Sas is taking a long time. I already have the cell counts. ...
9 views

### Multivariate discrete distribution

Let $\mathbf{x}=(x_{1}, \dots, x_{m})$ be a vector of discrete variable; $x_{j} \in \mathbb{Z}^+$. Which distribution can I use over $\mathbf{x}$ that allows to have dependence between each couple ...
8 views

### Class densities w/ SVM (LibSVM)

I have a question regarding number of class samples for training and testing with an SVM. For my situation, my training and testing samples both have the same number of class 1 and class 2 ...
6 views

### paired t-test for pairs each having more than one sample

I have a large and a small data set (the small data set is a subset of the large data set) with 2 columns(lets say var1 and var2). Now, based on var1 in the data set, I have sorted the large data set ...
12 views

### trouble plotting ARIMA [on hold]

I'm having trouble plotting a basic ARIMA model. I understand I can do a plot(fitted(modelname)), but the book I'm working through says the command herein is possible. The book actually suggests ...
17 views

### Testing and correctly interpreting the significance of nested random effects

I'm building a series of relatively simple random effects models where we repeatedly measure a water quality variable, say conductivity (cond), in different watersheds (ws) and streams (st). Here, ...
13 views

### Checking assumptions of a mixed model

I was wondering how to check the assumptions of a mixed model in R. Suppose I fitted a model with lmer. Is this what I should do? ...
11 views

### Predictive Model - Increase Pediction Accuracy for Less Likely Events

I am trying to build a model that predicts the which binary category a respondent belongs to (0 or 1). I have demographic variables (all categorical) and a few 10 point questions. I have built a few ...
11 views

### comparison behavior of time series [on hold]

I have three sets of time series data and I want to compare compare pro-all with two others. I want to validate our simulation output trend (c=0.1 and c=0.2) with real word trend (pro-all). I use ...
12 views

### Plotting a polynomial regression with its confidence interval of 95% in R

I have been trying for a while plotting a polynomial regression using R. I have read several libraries, as ggplot2, qplot, etc, with no succeed. The next are my data: I normally use the R GUI called ...
25 views

### R: Interpreting mlogit coefficients

Edit: The subset of the dataframe I provided way giving a different error, so I've replaced the pastebin entry with the full data frame. Here's the top of the data frame, then read into mlogit format ...
25 views

### How can I sample multivariate binary variables such that sum of them follows a gamma distribution?

Edit: Since the original question was confusing as whuber pointed out, let me rephrase the question with a Poisson distribution instead of a gamma distribution. The energy term of a Poisson ...
28 views

### machine learning with linear regression algorithm

I'm noob in machine learning, but I'm trying to know more about it. I have a question about a prediction model (predict for every page when the number of click) how can I do with no data in some ...
37 views

### Sum of iid random variables

Let $X_1, X_2,...,X_n$ be iid random variables. Let $Z_1, Z_2, Z_3$ be defined as $X_1, X_1+X_2, X_1+X_2+X_3$ respectively. Are $Z_1, Z_2$ and $Z_3$ also iid's? The question is based on renewal ...
40 views

### Model checking in bayesian stats considered “virtually illegal” in the 90's (Andrew Gelman's quote)

In this post, Andrew Gelman says: Bayesian inference can make strong claims, and, without the safety valve of model checking, many of these claims will be ridiculous. To put it another way, ...
11 views

23 views

### Can I add more cases and/or predictors to existing set of data?

If I have a set of data with 40 cases and 3 predictors can I add more cases and/or predictors later to the existing set of cases if I want to explore the effect of more possible predictors? The data ...
6 views

### What is a case where including random correlation parameters decreases the error on the fixed effects?

I'm wondering about the effect of true correlations among random effects on the standard error of my fixed effects in lme4::lmer models in R. My assumption is ...
8 views

### LIBLINEAR in C\C++ [migrated]

I want to use LIBLINEAR (http://www.csie.ntu.edu.tw/~cjlin/liblinear) directly in my C++ sources. While it seems simple using it in language like MATLAB/JAVA, in C seems very hard; for example, ...
7 views

### Looking for application to help predict win/loss based on previous wins/loses

I play in a basketball league that has 8 teams in rotation. Scores are kept so wins/loses are recorded along with number of points for/against. They don't post the final game scores, just an aggregate ...
34 views

### How to compute the probability of $P(a|b,e)$ given $P(a|b), P(a|e)$ and $A$ independent of $B$?

The following is given: \begin{align} P(b) &= P(e) = 0.1\\ P(a|e) &= 0.2\\ P(a|b) &= 0.95\\ P(a|\neg b, \neg e) &= 0\\ B &{\perp\!\!\!\perp} E \end{align} Is there any way ...
21 views

### How does caret handle factors?

I have been testing conditional trees and random forests with caret, and I've noticed it does something weird with factors. So, for example, a ctree using the base dataset 'chickwts': ...
22 views

### Popular methods for outlier detection (right skewed distribution)

What are the popular methods for outlier detection in univariate data, which do not assume normal distribution?
539 views

### When to use simulations?

So this is a very simple and stupid question. However, when I was in school, I paid very little attention to the whole concept of simulations in class and that's left me a little terrified of that ...
I've read that centering two normal (or symmetrical) variables $X$ and $Z$ affects correlation of centered $X$ with interaction term $X\cdot Z$ in such way, that this correlation $cor(X-EX, X\cdot Z)$ ...