Tagged Questions

0
votes
0answers
26 views

Compare the predictive power of a model between datasets

I have two sets of continuous response data for the same group of species, but in different areas (area a and area b). I am building a model for each area separately, to predict the area-specific ...
0
votes
0answers
54 views

How to perform step() when n < p in R?

I am trying to perform stepwise regression for variable selection in R. In matlab, the stepwisefit function is able to work in ...
1
vote
1answer
99 views

Interactions terms and higher order polynomials

If I were interested in fitting two-way interactions between a linear explanatory variable $a$ and another explanatory variable $b$ that has a quadratic relationship with the dependent variable $y$, ...
2
votes
1answer
80 views

Curvature terms and model selection

I am running a model selection analysis with a continuous dependent variable and a variety of continuous and categorical explanatory variables. For two of my continuous explanatory variables I am ...
0
votes
0answers
55 views

When using lmer is a random intercept being estimated more than once if specified in seperate grouping factors?

I know there are a slew of lmer specification questions already floating around. Please let me know if this is a duplicate, or if it is deemed off-topic, and I'll delete it. I am using a forward ...
2
votes
1answer
141 views

Regression with 3 measurement points (in R)

I have a regression in which I try to understand how much variance of the metric dependent variable each of the regressors explains. I use the package R relaimpo (Grömping, 2006) for that purpose, ...
0
votes
0answers
51 views

Test and train datatasets with prediction models and TunePareto package

I'm trying to chose the best prediction/classification model for a concrete problem. The methodology I've been asked to follow is this: Separate the data into test and training. Run a concrete model ...
4
votes
2answers
425 views

Relative variable importance with AIC

I am confused and just need some confirmation about calculating the relative variable importance value for the co-variates I used in AIC model selection procedures. I know that there is this one ...
0
votes
0answers
153 views

Can dredge() in R package MuMIn deal with global model objects generated by gls() in nlme?

I am trying to use the function dredge() in the package MuMIn to compare AIC model-selection statistics for models of all ...
4
votes
2answers
621 views

Problem calculating, interpreting regsubsets and general questions about model selection procedure

I want to select models using regsubsets(). I have a dataframe called olympiadaten (data uploaded: http://www.sendspace.com/file/8e27d0). I first attach this ...
2
votes
2answers
242 views

R-code question: model selection based on individual significance in regression?

I'm trying to generate an R function that keeps relevant variables based on their absolute t-value (or p, whichever is easier in code). Basically what I want is to run one regression (1), retain all ...
3
votes
2answers
244 views

Improvement of regression model

I am just learning R. I have developed a regression model with six predictor variables. While developing it, I found the relationships are not very linear. So, maybe because of this the predictions of ...
0
votes
0answers
196 views

How to set up a non-linear mixed effects model with random effects in R using nlme?

I have some data with predictor variables, A and B and response variable C. I have a grouping factor SITE. ...
0
votes
1answer
197 views

Good model vs. AIC

Suppose I run a bidirectional stepwise in R with the model: step(glm(y ~ a + b + c + d, poisson)) And the result may be: ...
5
votes
2answers
489 views

What are chunk tests?

In answer to a question on model selection in the presence of multicollinearity, Frank Harrell suggested: Put all variables in the model but do not test for the effect of one variable adjusted ...
7
votes
1answer
712 views

Is it possible to calculate AIC and BIC for lasso regression models?

Is is possible to calculate an AIC or BIC values for lasso regression models and other regularized models where parameters are only partially entering the equation. How does one determine the degrees ...
2
votes
0answers
107 views

How to learn fitting models?

I am new to R(learned the basics) In my current job I have to solve the following problem(the following is an example) Suppose we have 10, 000 users For each user we have his expected ...
4
votes
3answers
579 views

Model evaluation and comparison for selecting the best model

When comparing results obtained with different models in R, what should I look for to select the best one? If I use for example the following 4 models applied to the same presence/absence sample ...
19
votes
1answer
509 views

Appropriate residual degrees of freedom after dropping terms from a model

I am reflecting on the discussion around this question and particularly Frank Harrell's comment that the estimate for variance in a reduced model (ie one from which a number of explanatory variables ...
1
vote
2answers
1k views

How to perform model selection in GEE in R

I would like to do model selection for generalized estimating equations (GEE). Pan (2001) is most frequently cited for developing a method using QIC. I am wondering if anyone knows of a way to do this ...
1
vote
0answers
184 views

Bayesian variable selection in R

There are at least three R packages providing some functions to perform a Bayesian selection variable in linear Gaussian regression model: LearnBayes, mombf and BMA. I would be glad to know some ...
7
votes
1answer
673 views

How to calculate sample size for comparing the area under the curve of two models?

Because I would like to calculate the sample size for comparing the area under the curve (AUC) of 2 models (cross-sectional study, predictor = continuous variable). Can you point me which function in ...
1
vote
1answer
587 views

Comparing two GLMs using cross validation

Does anybody know how to arrange a cross validation so I can compare two models (negative binomial with quasi-Poisson)? I know some theory beyond cross validation, but don't know what kind of cross ...
2
votes
2answers
664 views

GLM model validation

When performing model validation and we are dropping the least significant explanatory variables until we find the optimum model where all remaining variables are significant, how does one go about ...
4
votes
4answers
1k views

Compare R-squared from two different Random Forest models

I'm using the randomForest package in R to develop a random forest model to try to explain a continuous outcome in a "wide" dataset with more predictors than samples. Specifically, I'm fitting one RF ...
4
votes
3answers
561 views

Is it valid to select a model based upon AUC?

I have plot ROC for several models. These models were used to classify my samples into 2 classes. Using these commands, I can obtain sensitivity vs. specificity plots for each model: ...
4
votes
2answers
315 views

Variable selection for time covariate

I'm fitting a linear model where the response is a function both of time and of static covariates (i.e. ones that are independent of time). The ultimate goal is to identify significant effects of the ...
3
votes
2answers
461 views

Automatisation of GLM analysis with negative binomial errors

I am new to R and some help would be of great use to me. Basically, I need to perform a GLM analysis with negative binomial errors and with fixed factors, no covariates and no random effects. My ...
3
votes
1answer
383 views

Model performance metrics for ordinal response

I'm interested in assessing model performance on data with an ordinal categorical dependent variable. For my use case, the ideal metric would: 1) Not assume equal intervals between classes or that ...
2
votes
3answers
426 views

How to do model selection in dynamic linear model?

I am trying to use DLM to model a time series. Candiate model includes local level, local trend and local trend with seasonal part. I do not know how to do model selection. Can AIC be calculated? I ...
5
votes
1answer
553 views

Test equivalence of non-nested models

Let's say $y$ is a linear function of $x$ and a dummy $d$. My hypothesis is that $d$ itself is like a hedonistic index of a vector of other variables, $Z$. I have support for this in a $MANOVA$ of $Z$ ...
4
votes
2answers
2k views

Regression selection using all possible subsets selection and automatic selection techniques

Given the dataset cars.txt, we want to formulate a good regression model for the Midrange Price using the variables Horsepower, Length, Luggage, Uturn, Wheelbase, and Width. Both: using all possible ...
6
votes
2answers
2k views

How to measure/rank “variable importance” when using CART? (specifically using {rpart} from R)

When building a CART model (specifically classification tree) using rpart (in R), it is often interesting to know what is the importance of the various variables introduced to the model. Thus, my ...
3
votes
4answers
327 views

Automating model selection criteria production

I want to perform model comparison according to several criteria using R. My dataframe's name is df ...
12
votes
4answers
899 views

Comparing mixed effect models with the same number of degrees of freedom

I have an experiment that I'll try to abstract here. Imagine I toss three white stones in front of you and ask you to make a judgment about their position. I record a variety of properties of the ...
4
votes
4answers
913 views

Algorithm for choosing the number of clusters when using pam in R?

I am clustering a dataset using the pam command (from {cluster} package), and I wish to decide on the number of clusters to use. I was able to implement The_Elbow_Method in R (see wiki) for doing ...