0
votes
0answers
6 views

What is the name of this validation procedure?

I have a set of classified data. In order to test the precision of several algorithms- I split the data into train and test sets. For the test set I choose at random 30% of the data and the rest is ...
0
votes
1answer
11 views

Sample size calculation for proportion

I am in a doubt about using Absolute and Relative precision for sample size calculations. Suppose if I want to conduct a study to asses the prevalence of Hypertension in a general population, which ...
1
vote
2answers
14 views

Multiple regression - how to deal with mixed linear and non-linear variables

Say I have a bunch of explanatory variables to predict a continuous independent variable. Below, a simple toy example: I think it would be easiest to do a log-log transform and proceed with linear ...
0
votes
0answers
8 views

Need to clarify certain doubts regarding Testing Hypotheses

I have some doubts in Hypothesis Testing which I would want to clarify. We know, by Neyman Pearson Lemma that the test which rejects $H_0$ when $\dfrac{f_1(x)}{f_0(x)}>\dfrac{k}{1}$ minimises the ...
1
vote
0answers
6 views

Confidence intervals of fitted Weibull survival function?

I'm implementing a Weibull survival analysis fitter, and have successfully estimated the parameters and their standard errors. I can also produce the fitted survival curve. My question is how can I ...
1
vote
0answers
9 views

Fourier/trigonometric interpolation revisited

Earlier, a user posted the Fourier transform equations of Epstein (1991), and now I am about to use these formulas to produce daily means from given monthly means. Could anyone explain why the last ...
0
votes
0answers
10 views

Wilcoxon Signed- Rank Test vs Kruskal Wallis vs One Way Anova? Which one to use

I am conducting research to rank the importance of certain elements within a property management framework. Would like to know which test is best to help me rank the items. The methodology is as ...
0
votes
0answers
24 views

What is variance of an election poll?

I was working through Harvard data science course: I came across this question: Assume you have $M$ polls with sample sizes $n_1,\ldots,n_M$. If the polls are independent, what is the average of the ...
0
votes
0answers
4 views

SPSS Multinomial Logistic Regression Interaction Term Reference

I am building a multinomial logistic regression model in SPSS. The dependent variable has 4 levels. There are two categorical factors; one factor has 4 levels, the other factor has 5 levels. I am also ...
1
vote
0answers
14 views

Best practices for documenting a data-science pipeline

Even though I try to keep it as simple as possible, the pipelines for some of my data science projects get rather complex. At some point it becomes necessary to document this pipeline so that someone ...
0
votes
0answers
4 views

Generating function from the infinitesimal generator of a continuous-time markov chain?

Take a generic $N$-state, continuous-time markov chain with infinitesimal generator $\mathbb{Q} \in \mathbb{R}^{N\times N}$. Then the evolution of the probability distribution of the $N$ states is ...
3
votes
3answers
35 views

Will two distributions with identical 5-number summaries always have the same shape?

I know that if I can have two distributions with the same mean and variance be different shapes, because I can have a N(x,s) and a U(x,s) But what about if their min, Q1, median, Q3, and max are ...
0
votes
0answers
12 views

Hierarchical Dirichlet Processes in topic modeling

I think I understand the main ideas of hierarchical dirichlet processes, but I don't understand the specifics of its application in topic modeling. Basically, the idea is that we have the following ...
0
votes
0answers
3 views

Multi-Level within factors in repeated measures ANOVA in SAS

This is a brief sketch of design containing both within and between subject factors. There are two groups. (each 10 subjects). Each subject responded many (over 600) trials. The trials are one of ...
0
votes
0answers
5 views

Comparable Mutual Information and Conditional Mutual Information values

Let's assume we have three continuous or discrete variables, X, Y and Z, for which I want to calculate Mutual Information (MI) and Conditional Mutual Information (CMI). The formulas to calculate this ...
0
votes
0answers
6 views

Apply Cross Validation to a prediction linear regression model

Hello I am slightly confused on how to apply a k-fold cross validation to a prediction linear regression model. Also how can i look at the data that the prediction model outputs before i apply cross ...
1
vote
1answer
6 views

SAS Fisher test p values for large sample sizes

I'm trying to calculate some odds ratios and significance forsomething that can be out into a 2x2 table. The problem is the Fisher test in Sas is taking a long time. I already have the cell counts. ...
0
votes
0answers
9 views

Multivariate discrete distribution

Let $\mathbf{x}=(x_{1}, \dots, x_{m})$ be a vector of discrete variable; $x_{j} \in \mathbb{Z}^+$. Which distribution can I use over $\mathbf{x}$ that allows to have dependence between each couple ...
0
votes
1answer
8 views

Class densities w/ SVM (LibSVM)

I have a question regarding number of class samples for training and testing with an SVM. For my situation, my training and testing samples both have the same number of class 1 and class 2 ...
0
votes
0answers
6 views

paired t-test for pairs each having more than one sample

I have a large and a small data set (the small data set is a subset of the large data set) with 2 columns(lets say var1 and var2). Now, based on var1 in the data set, I have sorted the large data set ...
0
votes
0answers
12 views

trouble plotting ARIMA [on hold]

I'm having trouble plotting a basic ARIMA model. I understand I can do a plot(fitted(modelname)), but the book I'm working through says the command herein is possible. The book actually suggests ...
1
vote
0answers
17 views

Testing and correctly interpreting the significance of nested random effects

I'm building a series of relatively simple random effects models where we repeatedly measure a water quality variable, say conductivity (cond), in different watersheds (ws) and streams (st). Here, ...
0
votes
0answers
13 views

Checking assumptions of a mixed model

I was wondering how to check the assumptions of a mixed model in R. Suppose I fitted a model with lmer. Is this what I should do? ...
0
votes
1answer
11 views

Predictive Model - Increase Pediction Accuracy for Less Likely Events

I am trying to build a model that predicts the which binary category a respondent belongs to (0 or 1). I have demographic variables (all categorical) and a few 10 point questions. I have built a few ...
0
votes
0answers
11 views

comparison behavior of time series [on hold]

I have three sets of time series data and I want to compare compare pro-all with two others. I want to validate our simulation output trend (c=0.1 and c=0.2) with real word trend (pro-all). I use ...
0
votes
0answers
12 views

Plotting a polynomial regression with its confidence interval of 95% in R

I have been trying for a while plotting a polynomial regression using R. I have read several libraries, as ggplot2, qplot, etc, with no succeed. The next are my data: I normally use the R GUI called ...
0
votes
0answers
25 views

R: Interpreting mlogit coefficients

Edit: The subset of the dataframe I provided way giving a different error, so I've replaced the pastebin entry with the full data frame. Here's the top of the data frame, then read into mlogit format ...
1
vote
0answers
25 views

How can I sample multivariate binary variables such that sum of them follows a gamma distribution?

Edit: Since the original question was confusing as whuber pointed out, let me rephrase the question with a Poisson distribution instead of a gamma distribution. The energy term of a Poisson ...
0
votes
1answer
28 views

machine learning with linear regression algorithm

I'm noob in machine learning, but I'm trying to know more about it. I have a question about a prediction model (predict for every page when the number of click) how can I do with no data in some ...
0
votes
1answer
37 views

Sum of iid random variables

Let $X_1, X_2,...,X_n$ be iid random variables. Let $Z_1, Z_2, Z_3$ be defined as $X_1, X_1+X_2, X_1+X_2+X_3$ respectively. Are $Z_1, Z_2$ and $Z_3$ also iid's? The question is based on renewal ...
1
vote
0answers
40 views

Model checking in bayesian stats considered “virtually illegal” in the 90's (Andrew Gelman's quote)

In this post, Andrew Gelman says: Bayesian inference can make strong claims, and, without the safety valve of model checking, many of these claims will be ridiculous. To put it another way, ...
2
votes
0answers
11 views

Dynamics of birth-death process with discouraged arrivals (alternatively, M/M/1 queue with balking customers)

Take a continuous-time birth-death process, where $k \in \{0,1,\ldots\}$ is the count and the arrival rate of death is $\mu \geq 0$ for $k = 1, 2, \ldots$ the arrival rate of births is $\alpha_k ...
0
votes
0answers
15 views

ggplot2, change title size [migrated]

I would like to have my main title and axis title have the same font size as the annotated text in my plot. i used theme_get() and found that text size is 12, so I did that in my theme statement - ...
1
vote
1answer
28 views

Using B-splines within a linear mixed-effects model in R

I am using linear mixed-effect model (run with the lme() function in the nlme package in R) that has one fixed effect, one ...
0
votes
0answers
14 views

Uniformly choosing from a list of samples which are normally distributed?

What kind of distribution do we get if I have a list of let's say 100 numbers which were generated by a normal distribution [mean$=0$, variance$=1$], and I now choose $k$ times uniformly from this ...
0
votes
0answers
3 views

How do I make sure numbers are numeric from a .txt? [migrated]

I'm setting up a script to extract the thickness and voltages from a single column text file and perform a Weibull distribution on it. When I try to use fitdistr() I get an error stating "'x' must be ...
1
vote
1answer
20 views

Why not use always a binomial exact test to compare two proportions instead of chi square?

I am trying to figure out what test I should use in the following scenario: I know that there is a lot of room for improvement in a specific area at work - being extremely critical, let's say that ...
0
votes
0answers
11 views

t-test : how to recover alpha when only have t-stat and Dof

In a t-test, is it possible to recover the value of alpha when you only have the degrees of freedom and the t-statistic? e.g. qt(0.05/2,100) = -1.983972 If I know that dof=100 and that ...
0
votes
1answer
17 views

Multiple regression with raw and derived variables as predictors

Extending my previous question ( Multiple regression with correlated variables ), can I do multiple regression with height, weight, waist, BMI (body mass index) and BSA (body surface area) as ...
0
votes
0answers
13 views

Who first suggested to approximate phases from a time series via marker events?

A rather simple approach to approximating an instantaneous (unwrapped) phase $φ$ from a time series is as follows: Define some a appropriate marker events (e.g., upwards zero crossings) $t_0 < … ...
1
vote
1answer
23 views

Can I add more cases and/or predictors to existing set of data?

If I have a set of data with 40 cases and 3 predictors can I add more cases and/or predictors later to the existing set of cases if I want to explore the effect of more possible predictors? The data ...
1
vote
0answers
6 views

What is a case where including random correlation parameters decreases the error on the fixed effects?

I'm wondering about the effect of true correlations among random effects on the standard error of my fixed effects in lme4::lmer models in R. My assumption is ...
0
votes
0answers
8 views

LIBLINEAR in C\C++ [migrated]

I want to use LIBLINEAR (http://www.csie.ntu.edu.tw/~cjlin/liblinear) directly in my C++ sources. While it seems simple using it in language like MATLAB/JAVA, in C seems very hard; for example, ...
0
votes
0answers
7 views

Looking for application to help predict win/loss based on previous wins/loses

I play in a basketball league that has 8 teams in rotation. Scores are kept so wins/loses are recorded along with number of points for/against. They don't post the final game scores, just an aggregate ...
1
vote
0answers
34 views

How to compute the probability of $P(a|b,e)$ given $P(a|b), P(a|e)$ and $A$ independent of $B$?

The following is given: $$ \begin{align} P(b) &= P(e) = 0.1\\ P(a|e) &= 0.2\\ P(a|b) &= 0.95\\ P(a|\neg b, \neg e) &= 0\\ B &{\perp\!\!\!\perp} E \end{align} $$ Is there any way ...
0
votes
1answer
21 views

How does caret handle factors?

I have been testing conditional trees and random forests with caret, and I've noticed it does something weird with factors. So, for example, a ctree using the base dataset 'chickwts': ...
1
vote
1answer
22 views

Popular methods for outlier detection (right skewed distribution)

What are the popular methods for outlier detection in univariate data, which do not assume normal distribution?
11
votes
6answers
539 views

When to use simulations?

So this is a very simple and stupid question. However, when I was in school, I paid very little attention to the whole concept of simulations in class and that's left me a little terrified of that ...
0
votes
0answers
3 views

Formula behind variables importance in linear model with caret library

The caret library can measure the importance of predictors http://topepo.github.io/caret/varimp.html The importance for a Linear Models is computed as: 'the absolute value of the t-statistic for ...
4
votes
1answer
51 views

centering two variables X and Z makes cov (X,XZ) = 0

I've read that centering two normal (or symmetrical) variables $X$ and $Z$ affects correlation of centered $X$ with interaction term $X\cdot Z$ in such way, that this correlation $cor(X-EX, X\cdot Z)$ ...

15 30 50 per page