Questions tagged [predictor]

Refers to the variables used in a model to predict a response. This tag can also be used for $X$ variables in explanatory & descriptive modeling, not just predictive modeling. This same construct goes by many names in different contexts, including: independent variable, explanatory variable, regressor variable, covariate, etc. This tag can be used for any of these synonymous terms.

Filter by
Sorted by
Tagged with
41
votes
4answers
45k views

Should covariates that are not statistically significant be 'kept in' when creating a model?

I have several covariates in my calculation for a model, and not all of them are statistically significant. Should I remove those that are not? This question discusses the phenomenon, but does not ...
18
votes
3answers
22k views

Continuous dependent variable with ordinal independent variable

Given a continuous dependent variable y and independent variables including an ordinal variable X1, how do I fit a linear model in R? Are there papers about this ...
19
votes
3answers
29k views

How to handle ordinal categorical variable as independent variable

I am using a logit model. My dependent variable is binary. However I have an independent variable which is categorical and contains the responses: ...
33
votes
3answers
49k views

Regression coefficients that flip sign after including other predictors

Imagine You run a linear regression with four numeric predictors (IV1, ..., IV4) When only IV1 is included as a predictor the standardised beta is +.20 When you ...
29
votes
4answers
10k views

Independent variable = Random variable?

I'm slightly confused if an independent variable (also called predictor or feature) in a statistical model, for example the $X$ in linear regression $Y=\beta_0+\beta_1 X$, is a random variable ?
51
votes
6answers
50k views

What is the difference between estimation and prediction?

For example, I have historical loss data and I am calculating extreme quantiles (Value-at-Risk or Probable Maximum Loss). The results obtained is for estimating the loss or predicting them? Where can ...
8
votes
1answer
7k views

Coding for an ordered covariate

I am performing ordinal regression, I have 5 response categories and several predictors both continuous and categorical. I would like to add a predictor which is categorical but ordered (1, 2, 3, 4). ...
18
votes
2answers
8k views

In a Poisson model, what is the difference between using time as a covariate or an offset?

I recently discovered how to model exposures over time using the log of (e.g.) time as an offset in a Poisson regression. I understood that the offset corresponds to having time as covariate with ...
12
votes
2answers
13k views

Forecasting hourly time series with daily, weekly & annual periodicity

Major edit: I would like to say big thanks to Dave & Nick so far for their responses. The good news is that I got the loop to work (principle borrowed from Prof. Hydnman's post on batch ...
38
votes
2answers
43k views

When and how to use standardized explanatory variables in linear regression

I have 2 simple questions about linear regression: When is it advised to standardize the explanatory variables? Once estimation is carried out with standardized values, how can one predict with new ...
19
votes
4answers
87k views

Maximum number of independent variables that can be entered into a multiple regression equation

What is the limit to the number of independent variables one may enter in a multiple regression equation? I have 10 predictors that I would like to examine in terms of their relative contribution to ...
15
votes
3answers
952 views

Do we really need to include “all relevant predictors?”

A basic assumption of using regression models for inference is that "all relevant predictors" have been included in the prediction equation. The rationale is that failure to include an important real-...
17
votes
1answer
11k views

Converting standardized betas back to original variables

I realise this is probably a very simple question but after searching I can't find the answer I am looking for. I have a problem where I need to standardize the variables run the (ridge regression) ...
4
votes
1answer
2k views

Should quantitative predictors be transformed to be normally distributed?

I am always struggling with normality testing for quantitative predictors (no factors) and transforming them to normality. If I am running a GLMM and my predictors are really non-normal, should I ...
10
votes
1answer
29k views

Getting negative predicted values after linear regression

I'm using linear regression to predict a price which is obviously positive. I have only one feature which is gross_area. I standardized it (z-score) I got this kind of value: ...
7
votes
1answer
6k views

How should I handle a left censored predictor variable in multiple regression?

I have a dataset (N=350) for which I would like to regress a neuropsychological test score (continuous) on age, education, symptom severity (continuous), and diagnosis (binary). Symptom severity is ...
5
votes
2answers
129 views

Explanatory variables may bias predictions

I' m asking this question out of sheer curiosity, my teacher was not able to explain it. If I'm using logistic regression with categorical variables they are coded like {1,2,3}. I guess it wouldn't ...
2
votes
2answers
1k views

Predictor transformation in logistic regression

In linear regression, I've seen (granted, not many) situations where basic transformations to some of the predictors can significantly improve the fit and stability of the model, and often a ...
7
votes
2answers
1k views

Is there a multiple testing problem when performing t-tests for multiple coeffcients in linear regression?

This question comes from a discussion on the recent post by @rvl It's all in the family; but do we include the in-laws too? Here's a common scenario that I've seen many times. A researcher runs a ...
10
votes
3answers
14k views

Can independent variables with low correlation with dependent variable be significant predictors?

I have eight independent variables and one dependent. I have run a correlation matrix, and 5 of them have a low correlation with the DV. I have then run a stepwise multiple regression to see whether ...
7
votes
3answers
5k views

Logistic regression performance with high number of predictors

I'm trying to understand the behavior of logistic regression in high dimensional problems (i.e. when you are fitting a logistic regression to data with a high number of predictor variables). Every ...
11
votes
3answers
42k views

Why would one use age-squared as a covariate in a genetic association study?

Why would one use age and age-squared as covariates in a genetic association study? I can understand the use of age if it has been identified as a significant covariate, but I am at a loss as to the ...
2
votes
1answer
112 views

Getting lagged values of indep. variables to model contemporaneous values of the dep. variable

I am trying to forecast the variable, oenb_dependent: My current sample data looks like that: ...
31
votes
7answers
5k views

In Regression Analysis, why do we call independent variables “independent”?

I mean some of those variables are strongly correlated among themselves. How / why / in what context do we define them as independent variables?
13
votes
4answers
3k views

Comparing importance of different sets of predictors

I was advising a research student with a particular problem, and I was keen to get the input of others on this site. Context: The researcher had three types of predictor variables. Each type ...
3
votes
1answer
5k views

Do I throw away a variable that is not statistically significant?

I am running various models in R for sake of prediction. If I run a model and a specific variable is showing itself to be insignificant (say, at the alpha=0.05 level), would I want to simply discard ...
3
votes
2answers
556 views

Hypothesis testing: Why is a null model that fits the data well better than one that doesn't?

Let's say we have two models: a null model, $M_0$, and an alternative model $M_1$. The only difference between them is that, in $M_0$ one parameter is fixed at $0$ and in $M_1$, that parameter is ...
3
votes
1answer
6k views

How to specify mixed ANOVA with multiple repeated measures and covariate in R

Working in R, how can I specify a mixed ANOVA with multiple between- and within-subjects factors in such a way that it's amenable to adding a covariate in a subsequent analysis? Also, ideally, I would ...
6
votes
1answer
237 views

When the effect size of a covariate is high and yet not significant

I was reading this answer to the question on whether all covariates should be kept in the model or just those that are statistically significant, and I noticed the point number 2: The effect size ...
5
votes
0answers
166 views

Why shouldn't I standardize my predictors when putting them into a regression model?

There are multiple reasons for applying standardisation/mean centre for predictors before putting them into a regression model. However, in the literature, some people do not do so or even argue ...
5
votes
1answer
6k views

Difference between: Offset and Weights?

I´d like to know the difference between these parameters when I am using GLM/GLMM/GAMLSS/BETAREG. I have observed a lot of published studies using offset, weights and covariates, however I am not sure ...
4
votes
3answers
11k views

Econometrics - choosing the best model when removing variables

So I am looking into a regression model that is supposed to predict the value of a house based on numerous independent variables. What I don't quite understand is how to select the "best" model when ...
4
votes
2answers
2k views

Predictor variables sum up to 1 but not necessarily correlated - is it a problem? [closed]

I am trying to fit hierarchical mixture model (using ML and MCMC, but this shouldn't matter) where the linear predictor part contains 17 independent variables. These are habitat variables: for each ...
3
votes
1answer
5k views

Relative importance of predictors in logistic regression

I would like to calculate an estimate (even a very rough one if it is the best I could get) of the relative importance of predictors in a logistic regression, something which can let me tell a common ...
3
votes
1answer
1k views

Change in order of predictors breaks logistic model estimation (glm, R)

I am fitting a binomial logistic regression in R using glm. By chance, I have found out that if I change the order of my predictor variables, glm fails to estimate the model. The message I get is ...
1
vote
0answers
323 views

Unequal sample sizes in generalized linear modeling

I would appreciate your advice. I have a number of outcome variables that follow Normal, Gamma or Poisson distribution. The predictor is categorical with two levels, as follows: Category 1: N=321, 9....
1
vote
1answer
798 views

Ordinal Dependent Variable and Ordinal Independent Variable. Which test to perform analysis with?

I have a question regarding Ordinal data. I have measured 'Attitude towards a product' using Likert scale and I have measured 'Purchase Intentions to buy the product' also using Likert scale. Now I ...
1
vote
1answer
52 views

Sum of all covariables value per patient is 1

I have a database with only 27 patients, and each patient was analyzed for the more than 119 different bacterial species. We test the percentage of each bacterial species among all the 119 species for ...
0
votes
1answer
3k views

Main Drawbacks of stepwise regression [duplicate]

People typically prefer the Lasso or other methods to stepwise regression. What are the main problems in stepwise regression which makes it unreliable specifically the problems with forward selection ...
6
votes
2answers
1k views

When to transform predictors in regression when response may be quadratic?

I am analyzing data from an experiment in which treatment levels increase quadratically, e.g. the treatment levels are $0, 1, 4, 9$. When analyzing the response using regression, would it make sense ...
4
votes
2answers
122 views

Bayesian Statistical Conclusions: We Implicitly Condition On the Known Values of Any Covariates, $x$?

My Bayesian data analysis textbook says the following: Bayesian statistical conclusions about a parameter $\theta$, or unobserved data $\tilde{y}$, are made in terms of probability statements. ...
4
votes
1answer
68 views

Is there a way to optimise the spread of explanatory variables in experimental design?

I am designing an experiment to test density effects of interacting species. The real example is quite complicated to explain, so I'll give a simplified example - I want to test whether the density ...
3
votes
0answers
589 views

Right-censored independent variable in Cox/logistic regression

I have a right-censored continuous independent variable that I want to include in a Cox regression. The variable is a physiologic test which is capped at a certain time, say 120 seconds, due to safety ...
3
votes
1answer
361 views

Overdispersion tests dependence on used covariates in Poisson model

One of the shortcomings of the Poisson regression model is that the mean, conditional on the independent variables should equal the conditional variance. If the observed variance is larger (smaller) ...
3
votes
1answer
612 views

Is Random Forest the only algorithm to measure the importance of input variables …?

I have three time series say (Stock price open, Stock price high, Stock price low) and one output (Stock price close) and I need to know which of the 3 inputs has a greater effect on my output. R's ...
2
votes
1answer
127 views

How to Include an Independent Variable with one-half 0s, one-half non-0 values

I am running a negative binomial regression. One of my independent variables is a measure of distance traveled - half of the observations are 0 because they do not travel, while the other half have a ...
2
votes
2answers
348 views

Relative importance of predictors in a model

A question that frequently comes up while presenting the findings of a predictive model to a business audience (with non-statistical background) is: which variable/predictor is most important in ...
1
vote
0answers
2k views

Testing to Compare for “Impact” of Independent Variable on Dependent Variable

How can I perform a statistical test to judge impact of an independent variable on a dependent variable given multiple regression output? In the example below, how would I test if gender or work ...
1
vote
0answers
427 views

Comparing predictors based on ROC AUC and cross-validation error

I am analysing how well some continuous variables (e.g. weight, height) predict the occurrence of a given disease after surgery. I have computed the area under the curve of the receiver-operator ...
1
vote
1answer
360 views

Interpolating between models in ROC space

Suppose I have two models $A$ an $B$ that predict class labels. If these give binary predictions, these will appear as pairs of (false positive rate, true positive rate) in the ROC space. We should be ...