All Questions

25 views

statistical careers where you get to move around?

I'm about to get my PhD. My background is in engineering and applied mathematics. My research in statistical & machine learning methods for biological applications (without being too specific). ...
7 views

What is the “scale” parameter in “continuous autoregressive model” in cts package?

I am trying to use the "car" command in "cts package" in R program and I see the "scale" parameter there. I wonder whether this can be assumed to be equivalent to time intervals for time series ...
5 views

Sample Weights for classification using Gradient-Boosted trees?

How can "weights" be given to different samples according to their relative importance while using Gradient boosted decision trees for classification? How does the ...
20 views

When is beta distribution bell-shaped?

Is there some restriction to parameters $( \alpha , \beta)$ that make the beta distribution concave down?
8 views

GLM binomial regression in R with a mix of categorical and continuous predictors

I have three predictor variables: year (2009, 2010, or 2011), day-of-year (DOY, measured as # of days from jan 1), and vegetation type (forest or nonforest). Veg and year can be seen as categorical ...
21 views

How to test for a significant difference between several dependent correlation coefficients?

I have a group of correlation coefficients (more than two). They are all dependent on one variable A in the form of r_A1, r_A2, r_A3....r_Ak, where 1, 2 ...k denote other variables; they all have the ...
10 views

Feature boosting via rescaling in logistic regression and linear SVMs

If I were expressing a problem in terms of binary features, all encoded as {0,1}, could I boost some features by encoding them as {0,2}? Would the effect change based on whether I used either of the ...
10 views

Standardizing variables at item or index level: Does it make a difference?

I'm running some multiple group CFA models comparing covariance structure by race/ethnicity and have survey data from 6th, 8th, 10th and 12th graders. My supervisor has told me to combine 6th and 8th ...
25 views

In R, why doesn't “rm(ls())” clear the environment? Why add “list=” [on hold]

I get that rm(ls()) doesn't work to clear the working environment. I understand that rm(list=ls()) does work. I'm looking for ...
10 views

How to choose the training, cross-validation, and test set sizes for small sample-size data?

Assume I have a small sample size, e.g. N=100, and two classes. How should I choose the training, cross-validation, and test set sizes for machine learning? I would intuitively pick Training set ...
19 views

Event centric clustering on R (Sports news)

i'm about to start a project on R and my goal is cluster Soccer news by event. in other words, each cluster must contain all news about a specific event from multiple sources. SO, my questions are: ...
20 views

Proof for Sufficient and complete statistic (Shao)

Please can you help me, with this question: Let $X$ be a random variable with a distribution $P_\theta$ in $\{P_\theta : \theta \in \Theta\}$, $f_\theta$ be the p.d.f of $P_\theta$ w.r.t a measure ...
14 views

Is there a kind of random walk process for wich the variance is a harmonic mean of the variances of compound processes?

Context: The variance of a sum of independent random walks is a sum of their variances: $\sigma^2 = \sigma_1^2 + \sigma_2^2$. In case of a dependent random walks with bivariate normal distribution it ...
12 views

Can collaborative filtering be cast as a classical regression problem?

Having the Netflix challenge in mind: collaborative filtering is typically presented as a matrix dimension reduction. My question is how does the problem relate to classical regression (supervised ...
23 views

Principal component Regression Using R

Can anyone explain principal component regression with the help of an example and the code in R? How to interpret the result of a principal component regression? How to find the individual effect of ...
11 views

Accounting for Different Levels of Twin Relatedness in SAS

I have a dataset with a number of identical twin pairs and fraternal twin pairs. I want to examine the relationship between two variables (let's call them INDEPENDENT and DEPENDENT). However, I ...
13 views

Statistical demand forecasting

How is batch demand forecasting done in retail like in Walmart where number of products to forecast are very large in number and products are short lived i.e have less than 36 months of historical ...
15 views

lm weights and the standard error

I am fairly new to R, but am experienced in other languages and also in data analysis in the physical sciences. I have a problem and will illustrate with a straight line fit using ...
3 views

Measured moderator, which is influenced by an independent variable

I have a measured variable, which is found to be influenced by an independent variable (i.e., experimental conditions). I will report this in the first part of the result section. Also, as an ...
5 views

VECM “goodness of fit” in R

I'm using ca.jo in R to perform the Johansen test on a given dataset. I obtain my VECM coefficients, cointegration rank, etc. However, it does not seem to give any notion of "strength of ...
42 views

How to model a count dependent variable with upper limitation

I have a dependent variable, which have 0, 1, 2, or 3 for its value. I asked participants to choose three items and coded 1 if it is in a certain category and 0 otherwise. I add the three binary ...
9 views

Include nesting factor as fixed effect in a GLMM

I have the following GLMM: success ~ age + gender + group\task + (1 + group\task|school\subject), family = binomial I want to know whether participants' probability to succeed in certain ...
21 views

Three Events Problem Interpretation

So I am having trouble understanding what this problem is asking. (I am not asking someone to solve it but simply help interpret what the problem wants me to do.) The problem goes like this: "Let ...
11 views

regression analysis for dataset with few predictors and few samples

I have a small pilot dataset containing experimental measurements for 12 samples, 4 numeric predictors and 1 numeric outcome variable. The goal is to obtain a first rough estimate on the extent to ...
8 views

Zero inflated model problem: system is computationally singular

I'm using R.After getting an error asking me to provide starting values for a glm (poisson family), I took a look at my data and realized I had quite a bit of zeroes. So, I tried zeroinfl from pscl. I ...
26 views

Cantelli's inequality proof

I am trying to prove the following inequality: EDIT: Almost immediately after I posted this question, I discovered that the inequality I am being asked to prove is called Cantelli's inequality. When ...
7 views

Estimating Probabilities from Stratified Table

Say I have data stratified by smoking status, where $Y=1$ is has lung cancer, $Y=0$ doesn't have lung cancer, and finally $D=1$ drinks alcohol, and $D=0$ doesn't drink alcohol. How do I estimate ...
23 views

Identifying What Causes a Variable to Increase

Say I have a dataset with several continuous and categorical variables, and I want to identify what variables (values or properties of these variables) may cause one of the continuous variables to ...
23 views

Jaynes' Derivation of Herschel-Maxwell for Normal Distribution

I am reading the following paper: http://www-biba.inrialpes.fr/Jaynes/cc07s.pdf and cannot seem to figure out how Jaynes is deriving (P2) and below (specifically the log arithmetic ...
24 views

Treating Categorical Data as nominal or ordinal?

There is a four category classification system for individuals with HIV, labeled Stages 1, 2, 3, and 4, with higher stages indicating more advanced disease. Cell count is a continuous variable, and ...
7 views

computing EP messages in factor graph

I'm using infer.net for implementing a dynamic Bayesian network model. every node in each layer is dependent on all nodes in previous layer (we want to train wji, the weights of edges between each ...
12 views

AR models on non stationary data

i am currently reading Diebold and Li's 2006 paper: Forecasting the term structure of government yields where the authors fit, albeit simple, AR(1) models on clearly non stationary data. Why is this ...
13 views

what are the alternative open source tools for PredictionIO?

PredictionIO is good to be enough for content discovery and recommendation but it seems it does not support classification. Then I should use a different tool then Prediction IO for my prediction ...
16 views

Formulating a Z-test (or any other suitable test ) for survey data

I have a particular survey data related to a particular job. Out of a population of 90 people in this job, 36 people are in the lowest income group. 19 out of 36 people want to quit that job. I want ...
22 views

PCA scores and cluster analysis

I have obtained PCA scores (Anderson-Rubin) and thought to use them in cluster analysis, but have got confused how to make interpretations based on them or as what type of variable they should be ...
118 views

Does fixing coefficients in a regression make sense, and if so how to do it?

I have a generic question about whether it might sometimes make sense to fix specific regression coefficients to predetermined values. And if this makes sense in particular cases, how do you best go ...
16 views

Cannot get the p values from afex package [on hold]

According to this website: http://www.psychologie.uni-freiburg.de/Members/singmann/R/afex I have run: ...
19 views

Beta and Alpha Interpretation [on hold]

I have been given some outputs as part of a group project. The model is a deterministic time-varying coefficient model weighted using a Gaussian kernel. I am ok for most of the output although the ...
8 views

F1 score for biased binomial data

I am applying a Bayesian classifier and would like to find out the f1 score. I determined the TP, TN, FP, TP. Unfortunately I had to find out that in my cross-validation almost in all test scenarios ...
41 views

Linear regression of 0/1 response (Fig. 2.1 of The elements of statistical learning)

In chapter 2 ESL book authors write: Let's look at example of linear model in a classification context They fit a simple linear model $g = 0.3290614 -0.0226360\cdot x_1 + 0.2495983 \cdot x_2 + e$, ...
9 views

beta differs from overall subgroup

I want to know whether the beta of a subgroup differs from the beta of the overall group. Both variables are dummy variabes. The overall group variable has 52 out of 267 obseverations where it takes ...
17 views

Kalman Filter to correct model simulation bias

I am working with a large scale deterministic model, which attempts to simulate CO2 emissions in different regions. When compared to historic data, the model output suffers from systematic biases. ...
8 views

In R and INLA, how do I graph results of 'rw2' model?

I'm working with model 2 of INLA's Zambia example in R: ...
21 views

How to include noise in clustering evaluation?

When evaluating clustering methods which do have a definition for noise points (like dbscan), how noise will affect evaluation? Consider a clean dataset like well known Iris dataset. There is no ...
18 views

prediction interval of a new prediction out of a weighted linear regression model

This question is based on the example 11.1 out of the book Applied linear statistical models of Kutner, Nachtsheim, Neter and Li. You can find the data here. First they calculate a simple linear ...
27 views

What is the difference between the Mann-Whitney and Wilcoxon rank-sumtest? [duplicate]

First of all, what is the difference between the Mann-Whitney and Wilcoxon rank-sum test? How do I choose between the two? I know the latter is implemented in R, is there a way to do the Mann-Whitney ...
32 views

How to customize a link function to perform a logistic regression?

My data was collected using Randomized Response Technique. So I have additional variability into the data. I need to use a GLM (logistic regression) but I should customize a logit link function to ...
21 views

A question about warning that performing k-fold CV with caret

I am trying to createFolds function in caret to use k-fold cross-validation in R. But I came across this warning: ...