0
votes
0answers
4 views

Which model of lm/lme to believe?

I have a data set of height values for several individuals of several species in 2 different conditions. Not all the species are found in the 2 conditions. I want to compare height values between ...
0
votes
0answers
3 views

How to loop through elements in a large list, make NA variables for holes in data, and create new variable in another dataframe in R?

I am trying to find a way to avoid writing 78 lines of code by using some sort of loop in R. Say I have the following: ...
0
votes
0answers
15 views

Caclulating probabilty of drawing vector from multivariate normal distribution in R

given vector of mean $\mu$ and covariance matrix $\sigma$ as parameters of multivariate normal distribution, how can I calculate the probability of drawing $k$-dimensional vector from that ...
0
votes
0answers
4 views

Compute TFIDF matrix based on another TFIDF matrix using R 'tm' package

Using the R package tm, I'd like to compute a TFIDF matrix (tfidf_matrix_b) based on a previously computed TFIDF matrix (...
1
vote
0answers
5 views

Confidence interval on point biserial correlation coefficient

How can I calculate the confidence of a point biserial correlation coefficient? I'm calculating the point biserial correlation between a continues variable (score on a test), and a dichotomous ...
0
votes
0answers
5 views

Linear Mixed Effects

I am looking for seasonal influence on how effective a herbicide treatment is. I have data over ten years of several polygons that have received the treatment multiple times at varying intervals. ...
0
votes
0answers
12 views

Where can I find a data set of US Individuals? [on hold]

I'm looking for a data set of US citizens/residents, etc. With data on things such as age, geographic location, gender, income, marital status, etc. The more the better. Where can I find an ...
0
votes
0answers
7 views

Classification in real time without prior knowledge of the number of classes

Is there an implemented algorithm (with python/R or java in preference) that can classify incoming data from an unknown generator with absolutely no prior knowledge or assumption.
0
votes
0answers
18 views

Help logistic regression [on hold]

I have to predict a binary variable with logistic regression. The idea is to classify a number of subjects each either sick or not sick. Therefore, I have 11 risk factories for each person and have to ...
0
votes
1answer
10 views

Lmer v. lm with Dummy Variables. Where Does the Math Differ in a Simple Example?

I am trying to understand the concept of mixed effect along with the R syntax in lme4 with simple scenarios out of the dataset ...
0
votes
0answers
8 views

Which statistical test should i perform find performance diffecence in two sample?

I am trying to find if there is any deference in performance withing two groups of sample. ...
1
vote
0answers
6 views

Single-factor random effect ANOVA and components of variance

I've needed to learn some DOE for work and I've been going through Montgomery's book using R to help with the calculations. The fixed-effect models have been straightforward, but I have some trouble ...
0
votes
0answers
8 views

Multinomial Logistic Question

Using PROC Logistic with the glogit link, I am attempting to classify records according to 1 of 3 responses (0, 1-2, 3+). After cleaning and running multiple models, I landed on what I thought was a ...
0
votes
0answers
10 views

Which normalization procedure is suitable for Principal Component Analysis (PCA)?

I have a data-set that has features with different ranges, scales and measurement units so as proposed in literature I'm using Principal Component Analysis (PCA) with "correlation matrix". In this ...
0
votes
0answers
14 views

Interpretation of Saturated Model vs. Model with Interaction and One Main Effect

Say that I have two regressions: 1) $Y_i = \alpha_0 + \alpha_1 X_i + \alpha_2 X_i*Z_i + \epsilon_i$ 2) $Y_i = \beta_0 + \beta_1 X_i + \beta_2 Z_i + \beta_3 X_i*Z_i + \epsilon_i$ $X_i$ and $Z_i$ are ...
0
votes
0answers
5 views

Fitted Confidence Intervals Forecast Function R

I am using the forecast function in R written by Rob Hyndman. I am trying to determine the best way to determine confidence intervals for the historical fitted values. I would like to plot how often ...
0
votes
0answers
9 views

All of the series used in a model must be stationary at the same order of differencing

While practicing VAR analysis, all of the series used in the model must be stationary at the same order of differencing. Is this correct? For example, let $X$~$I(1)$ and $Y$~$I(2)$. Can I use these ...
0
votes
0answers
11 views

Factoring Fourth degree polynomial for invertible ARMA process

I have to represent a $MA(4)$ process as an $AR(\infty)$. In this regard, I need to factorize the polynomial $(1-\theta_1L-\theta_2L^2-\theta_3L^3-\theta_4L^4)$ to have a representation $(1-z)^{-1}$. ...
0
votes
0answers
5 views

model selection in quantile regression

I am quite new in quantile regression, and I am using it in R (quantreg). I have around 17 predictors,and I would like to select the best subset of predictors. Then, I was wondering if it would be OK, ...
0
votes
0answers
7 views

On choosing the size of test set for an (already collected) dataset

When I search online for criteria to select the fraction of a dataset that should be set aside as test set, invariably the answer boils down to "it depends" (on the type of data, the type of study, ...
0
votes
0answers
3 views

Caret - Changing tuneGrid gives different results for same parameters

I'm having an issue training a GBM model in caret. I'm quite new to all of this, but I'll try to explain things as best as possible, but please let me know if you need any further info. My code looks ...
0
votes
0answers
9 views

Abstraction in neural networks

Without running through all possible scenarios, can a neural network learn that 4 or more dots in a row, anywhere, is a line? I've been looking for an existing method, however "abstract" comes up in ...
1
vote
1answer
25 views

What is the duality relationship between eigensystems of X X' vs X' X?

I want to know the relationship between the eigensystems of two non-negative-definite (covariance-like) matrices. Both are derived from X which is a T-by-K real matrix (wlog say K > T). I avoid ...
1
vote
1answer
6 views

Filtering to avoid the Slutsky-Yule effect in a moving window average?

Can anyone suggest a generally accepted method of filtering time series data to avoid the appearance of artifactual oscillations when smoothing the data using a moving window average? Is it simply a ...
0
votes
1answer
23 views

Daily data, different dates

I have downloaded two different daily data sets, both for the same period. Together I want to use them to construct data for another variable. However, the problem is that the dates do not exactly ...
0
votes
0answers
12 views

How to Schedule R Activity [on hold]

I am having following requirement, 1. Run R regression techniques on mainframe Data 2. Do visualization and save it for business users My questions are, What is the best way to integrate R with ...
0
votes
0answers
5 views

Training an Elman Recurrent Neural Network?

I have few doubts related to training Elman RNN using Backpropagation Through Time Algorithm. Assume, I present a sequence to the network and the network adapts the parameters based on the error ...
2
votes
1answer
10 views

Metropolis : Set first sample value instead of randomly generate an arbitary value

According to Metropolis-Hasting algorithm, the first sample is an arbitrary value generated randomly at the Initialization step. ( http://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm ) ...
2
votes
1answer
17 views

What are some standard bimodal distributions?

I have plotted this curve using the default kernel density estimate function in R I am looking for some standard distribution which could be close to this. Is there any standard bimodal distribution ...
1
vote
1answer
8 views

Does VECM use the stationary series or the originals ones?

I have some cointegrated series and I decided to build a VECM model. (I differentiated them twice in order to get stationary series and that led me to believe that they might be cointegrated - I ...
1
vote
1answer
15 views

What is the right way of computing baseline hazard rate

What is the right way of computing baseline hazard rate? To my understanding hazard rate at time t is simply #events / #AtRisk ...
4
votes
1answer
25 views

Why exactly is normality an assumption in a T-test but not a Wilcox rank sum/Mann Whitney U test when obtaining a p-value?

In addition to the question, please do correct anything below that I have incorrect. I understand how the test statistics for a t-test and WRS/MWU test are calculated. In a t-test, $t=\frac{\bar ...
3
votes
1answer
19 views

Stein's estimator vs James-Stein estimator

I read a lot of sources concerning stein's estimator and James-Stein estimator. Unfortunately, a lot of sources do not write the correct formulas of each estimator. And so I am now confused!! Kindly, ...
0
votes
0answers
17 views

Using Dirac Delta functions for estimating a probability distribution

I'm having some trouble understanding this slide. It's mentioned in the context of gaussian distributions. I sort of understand the Dirac delta "function". The main difficulty I'm having is with ...
0
votes
0answers
6 views

Calculating the covariance of effect sizes for multiple-treatment multiple-enpoint studies

My question is somewhat related to Patrick's question . However, I'm interested in finding the covariance of effect sizes for multiple-treatment multiple-outcome design given the scenario below. ...
1
vote
0answers
8 views

online bin packing and machine learning

I am trying to optimize scheduling disk allocation in data centers. In nature it is an online bin packing problem. But, since each customer has it's own behavior or kind of workload I was wondering, ...
0
votes
0answers
7 views

Obtaining wide Confidence Intervals from a fitting of a model with 2x2 Crossover trial data. Where am I going wrong?

I am doing an analysis of a 2x2 crossover experiment with ordinal output. And it seem that the effect of the treatment (trt) is statistically significant. However my CI are very wide [1.05, 364.23]. ...
1
vote
0answers
11 views

How to treat different (multiple) measurements on the same subject over time?

Let's say I am interested in some microscopic measurements on a subject, for example, the pore sizes on my hand. I want to see how putting my hand under a humidifier changes the pore size over time, ...
0
votes
0answers
6 views

Signal detection theory: optimal criterion with unequal variances

Is there a formula for the optimal cut-off for signal detection theory with unequal variances? To put this in another way, if I assume two approximately normal distributions with different means and ...
0
votes
1answer
20 views

Clustering methods based off set membership

So most of the clustering algorithms I've looked at are based on the distance between points. I was wondering if anyone knows any simple clustering algorithms based off points making up a set. ...
1
vote
1answer
18 views

Why pnorm function and sum of dnorm function differs for same input?

Data is height of men in 'inches'. Mean is 70. SD is 3.3 I wanted to compute the probability of men whose height is in between 69 inches and 74 inches. The following methods gave me different ...
0
votes
0answers
4 views

Determining a fit test for a interupted stochastic data

I have 10s of thousands of linear data tracks (DNA sequence abundances) which I am attempting to classify by their adherence to a theoretical model. My first pass approach was quite naive, defining ...
3
votes
1answer
35 views

Simulating non-normal correlated data for Bayesian regression

I'm interested in generating data for three separate datasets where each contains three IVs and a single DV that are correlated with one another based on meta-analytic data. For example, I would like ...
0
votes
0answers
6 views

Where is the Vowpal Wabbit SGD objective function specified? [on hold]

I am new to Vowpal Wabbit, and have been reviewing everything I can find on it. I am interested in using SGD to optimize a fairly large problem with a few hundred parameters several hundred prediction ...
0
votes
1answer
18 views

AIC values “Without Covariates” in R [on hold]

I've been using SAS but I'm now transitioning to R. When doing cox-regression in SAS (proc phreg) the output included AIC values "Without Covariates" and "With Covariates": The difference between ...
0
votes
0answers
8 views

interpretation negative binomial

I have a question towards an analysis of purchasing decisions. I have a data set where I investigate the amount of previous purchases last week (predictor) on the amount of purchases today (response ...
3
votes
1answer
19 views

R lmer confint: theta values not the same as summary values

I've been using lmer's confint procedure to compute bootstrapped confidence intervals for random effects. I noticed that extracting the theta values using "getME" produces estimates that are slightly ...
0
votes
0answers
10 views

Estimating transition probabilities from the sequences without repeated states

Is it possible to estimate transition probabilities of the continuous process if we observe only the first occurence of the state? For example, if the real transition sequence was 1 - 5 seconds; 2 - ...
2
votes
3answers
19 views

Non-normal univariate distribution — how to define “low” to “high” subsets of values?

Sorry for the dumb question: it's been a million years since I took basic stats. I have a set of 48, non-normally distributed values ranging from 1 to 100. I'd like to be able to apply a (valid) ...
0
votes
0answers
13 views

Negative coefficient for expected return, what does it mean?

I'm estimating a set of parameters in a utility function $U=\alpha\ \text{payment} -\beta\ \text{investCost}+ \text{error}$ The payment and investCost are expected return (payment) and cost, and ...

15 30 50 per page