Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

6
votes
3answers
302 views

What is the null hypothesis for the individual p-values in multiple regression?

I have a linear regression model for a dependent variable $Y$ based on two independent variables, $X1$ and $X2$, so I have a general form of a regression equation $Y = A + B_1 \cdot X_1 + B_2 \cdot ...
1
vote
0answers
10 views

Why would someone use regression analysis to compute weighted variables?

I stumbled upon a measure that uses multiple regression analysis to compute weighted variables, instead of factorial analysis or other more common methods. I have a feeling this is just bad statistics ...
2
votes
1answer
48 views

Proof of variance of point-estimate in simple linear regression

In the case of simple linear regression, I understand the math behind the variance of the estimates: $$ \operatorname{Var}(\widehat{\beta}_0) = s^2 \bigg(\frac{1}{n} + \frac{\bar{x}^2}{S_{xx}} \bigg) ...
1
vote
0answers
44 views

Logistic Regression Just Predicts 1

I am a 10th grade student working on a science fair project that involves making predictions about adherence given patient data. I have separated the week into 21 time slots, three for each time of ...
1
vote
2answers
41 views

Regarding glm.nb() and my parameter

I have been doing a negative binomial regression model using the following code My my estimate here comes out as 3.48. (the exponential of the intercept). The data was taken randomly (with set seed) ...
3
votes
1answer
90 views

Formulating quantile regression as Linear Programming problem?

How do I formulate quantile regression as a Linear Programming problem? When looking at the median quantile problem I know it is \begin{align} \text{minimize } & \sum_{i=1}^n |\beta_0 + X_i \...
0
votes
0answers
16 views

Biased coefficient estimates when using logistic regression with unbalanced classes?

I'm aware of the fact that probability estimates can be biased in logistic regression when dealing with unbalanced classes. When looking at the log-likelihood function... ℓ(β)= ∑ 𝑦𝑖 *log 𝑝(𝑥i)+(1−...
0
votes
0answers
15 views

caret chooses non-optimal RMSE?

I run a linear regression via caret / glmnet method with "RMSE" as metric. In the final model, caret tells me which values of the tuning parameters alpha and lambda were selected to minimize RMSE. If ...
1
vote
0answers
53 views

Independence relation between observation and error [duplicate]

Having a simple model like: Y = a + bX + u where u represent the error term, if I know that the observation of X and Y are i.i.d, also the error term is i.i.d?
1
vote
0answers
12 views

Magnitude of the effects using confidence interval [closed]

I have an assignment (doing regression) and one of the tasks is "Check the magnitude of the effects using confidence intervals.". How can I check this and how to report it in a paper? Using SPSS.
3
votes
0answers
15 views

How sensitive is $L_p$ regression to initialisation?

Consider that I wish to solve a linear regression in the $L_p$ framework That is, the optimisation problem that I wish to solve is $$ \mathbf{w} = \text{argmin}_{\mathbf{w}} ||\mathbf{w}^T\mathbf{x} -...
1
vote
0answers
33 views

Does it matter whether the predictor or fitted value goes on the x axis in a residual plot?

For example, say you fit the model: $$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon $$ where the residuals are $e_i = y_i - \hat y_i$. I often see residual plots of $e$ vs $\hat y$ as so: ...
0
votes
1answer
14 views

Conditional logistic regression for calculation odds ratios

I want to calculate the crude and adjusted odds ratios for exposure to occupational risk factors such as aluminum and fossil fuels in my case control study. My cases are 180 demented patients and I ...
1
vote
0answers
10 views

What is the confidence of the solution to a system of linear equations with associated correlation coefficients? [duplicate]

I am writing a high school statistics question that requires students to calculate the reliability of a solution to a system of linear equations by first creating lines of best fit using two data sets....
1
vote
0answers
36 views

Interpretation of parameter stability of regression coefficients and stability of predictions when multicollinearity is present

It is known that when the predictor variables are highly correlated with each other (e.g, correlation coefficient is 0.9) the regression coefficients are unstable as they have high standard errors. I ...
1
vote
1answer
32 views

Proof of contemporaneous exogeneity, and its implications for an AR(1) model

It can be shown by contradiction that exogeneity fails to hold for an AR(1) model. Is there any proof that contemporaneous exogeneity does not fail to hold? All I've come across is assuming it does ...
1
vote
0answers
11 views

Efficiently producing Partial Residual Plots in R [migrated]

I have a large glm (4Gb in size) for which I would like to display the partial residual plots using crPlots(myGLM) Currently RStudio hangs on displaying the first plot. The machine I'm using is ...
0
votes
1answer
43 views

Variable Selection for Negative Binomial Regression

First off I apologize, that I cannot share the code or details about the variables for this project. I am new to statistics and am working on a project using count data. I want to make sure I am going ...
1
vote
1answer
66 views

How to prove that the robust F statistic is asymptotically chi squared distributed?

The linear model is $$y_{i}= x_{i}'\beta+u_{i}$$ When written in vector notation such that $y_{i}$ is a $1$ x $1$ matrix of outcomes, $x_{i}'$ is a $1$ x $k$ matrix of control variables, $\beta$ ...
0
votes
1answer
52 views

How to interpret the results from a binary model done in R? [duplicate]

This is a binary model that my colleagues and I developed that regards a connection between the distance (dependent variable) and overall satisfaction with public transport (covariate). I don't know ...
1
vote
0answers
33 views

How are confidence intervals calculated for lm in R using predict?

Here, a simple linear model, given x = 98, yields a predicted value of 24.47 with 95% confidence interval [23.97, 24.96]. ...
0
votes
0answers
25 views

Poisson log linear Regression: using either R or python

I was hoping someone can help me with this problem. I posted a similar question earlier but it's not the same. I have the following: A 2x2 matrix of structural connectivity values between brain ...
2
votes
1answer
92 views

Linear Relationship vs Correlation

I am new to machine learning, and I'm trying to cover some of the basics. One of the assumptions of linear regression is a linear relationship. However on Reddit I was told today that no machine ...
0
votes
1answer
41 views

Implications of strict exogeneity for OLS in time series

Zero Conditional Mean (ZCM), or Strict Exogeneity, is given by: $E[u|X]=0$ Equivalently, $E[u_t|X]=0, t=1,...,T$ Is it true that this implies: Zero Unconditional Mean: $E[u_t]=0, \forall t$ ...
0
votes
0answers
14 views

Multivariate linear regression - optimizing one coefficient at a time

I have a few questions about solving the multivariate linear regression problem: What is the most popular numerical method used to get the coefficients from multivariate linear regression? I assume ...
1
vote
0answers
25 views

How to get a predicted value from a gaussian regression model [closed]

I have a set of the times where the trash level is greater than 60 % during 6 days and i want to predict the time in the 7th day. I choosed to work with gaussian regression and after training the ...
2
votes
0answers
32 views

Consistency of Adaptive LASSO

I'm reading the paper on Adaptive LASSO estimator (Zou, 2006). In one of the presented numerical simulation examples (Model 0 (Inconsistent lasso path), page 6 (1423)) they claim the following: To ...
1
vote
1answer
32 views

Why are t-tests rather than z-tests used in linear regression? [duplicate]

Do the explanatory variables need to have normal distribution in linear regression? Why are there z-tests rather than t-tests in logistic regression?
0
votes
2answers
46 views

k-fold cross validation: Force at least m instances in each fold

I'm dealing with a multi-output regression problem (~ 800 dependent variables, ~ 1300 observations). My current approach is to train a single model for each output. To select an "optimal" lambda I ...
0
votes
1answer
43 views

Jarque-Bera test for Normality

Which test should I consider if by JB-test result I have heteroscedasticity and by the result of two others no. $JB JB-Test (multivariate) data: Residuals of ...
1
vote
1answer
20 views
0
votes
2answers
67 views

Neural Networks: What activation function should I choose for hidden layers in regression models?

I am experimenting with Neural networks for regression tasks. I know some theory, and how to choose the activation function for the output layer. What is not clear to me is: how to choose activation ...
0
votes
0answers
15 views

regression with percentage DV and multiple dummy IV

I am running a market analysis about e-mail responses for my marketing class. I have the following data entries: Observations: 400 different sequences of e-mails where sent to different target ...
0
votes
1answer
21 views

VAR estimation-How to interpret the results?

I have these results when I estimate a VAR with two variables:Growth and Debt and p=2.How to interpret the result for each equation? Thank you. VAR Estimation Results: ...
3
votes
0answers
35 views

Standardizing qualitative variables in R to perform glm's, glm.nb's and lm's [closed]

I want to standardize the variables of a biological dataset. I need to run glm's, glm.nb's and lm's using different response variables but the same explanatory variables. The dataset contains counts ...
0
votes
0answers
21 views

When is it appropriate to use Support Vector Regression?

I'm reading The Elements of Statistical Learning by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie. In their chapter on support vector machines, they make a brief mention of support vector ...
0
votes
1answer
18 views

Is it possible to include an interaction term in logistic model if the interaction term had no observation in some category?

I tried to do a binary logistic regression. y was a binary outcome x1 was an independent variable by 5 categories x2 was an independent variable by 2 categories I found that x1 and x2 had the ...
0
votes
0answers
13 views

Exploring and analyzing date variables and address variables

I have a dataset that contains date variables, quantitative and qualitative predictor variables, and a binary dependent variable. The goal of my analysis is find the percent of success in CORRECT and ...
0
votes
0answers
7 views

Model for hormone levels over tissue cells

I have a certain type of biological data and I am unsure about how to model it. The data represent the amount of 3 hormones detected along 20 consecutive cells of a certain plant tissue. I think there ...
0
votes
1answer
13 views

Does 0-sum game violate linearity in linear regression? [closed]

I have a dataset that is derived in the fashion of 0-sum game (RNA-Seq data: the total amount of reads is fixed, inclusion of one read belonging to one feature means the exclusion of another read ...
8
votes
1answer
100 views

Does correlation between variables in an interaction matter?

Suppose you fit a model $y = x_1 + x_2 + x_1\times x_2$. Are there any practical implications for estimation of the interaction effect if $x_1$ and $x_2$ are correlated? I understand there could be ...
0
votes
0answers
9 views

Normality of residuals in a VAR model

If I have these differents results for the normal distribution of the residuals, I should consider them normal or not? $JB JB-Test (multivariate) data: ...
1
vote
0answers
24 views

PCA used for regression

Can anybody point me to a thorough example where PCA was used for predictive purposes and it compared favorably to regular linear regression (i.e., perhaps a lower RMSE using the PCA components ...
0
votes
0answers
6 views

How to correctly summarize a confidence interval in a sub--sample using linear regression?

This is probably a stupid question but I'll ask it anyway. So here it goes. Consider an ordinary linear regression setting. Assume you are using a relevant set of regressors, there is no sign of ...
1
vote
1answer
56 views

In GLMs are the Scale and Dispersion parameters the same?

Given a data set and a genralized linear model I am asked to find the estimation of the scale parameter obtained with the Pearson statistic. But I am a bit confused: I know that ${\rm Var}(Y)=\phi\...
1
vote
0answers
34 views

Are all classification models appropriate for estimation of probabilities?

I try to estimate the probability that a tennis player will win based on several predictors (such as skill, form, surface, weather etc.). Can I use every classification method to estimate a ...
2
votes
2answers
63 views

How do I forecast time series for which the range of residuals is increasing over time?

I have gathered 25 years worth of monthly timeseries data. The value of Y (dependent variable) has seasonality of 10 months. I have used polynomial equation to model seasonality cycle. The trend ...
2
votes
1answer
41 views

How to run a regression analysis in python on a nonlinear, strange dataset

I can't figure out what type of regression analysis or extrapolation technique to use in order to come up with an equation for the data I have plotted. For a school project, I've been testing diodes ...
0
votes
0answers
11 views

Panel Data: correlation between individual effects and variables

Is it possible to see if there is a correlation between any variable and the individual effects just by looking at the coefficients and standard-errors of an OLS, RE and FE functions?
0
votes
0answers
13 views

Regression model when stratified randomization is used

I have been going through articles on stratified randomization but I haven't seen one that elaborate regression function generated and also elaboration of strata effect on the outcome. I will ...