Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
2
votes
0answers
30 views

Name for spurious linear Regression Plots [duplicate]

Yesterday I was at a medical conference in which a lot of plots of Point Clouds with linear fits were shown. In many cases the fit seemed (at least to me and colleagues) to be influenced mostly by ...
-2
votes
1answer
1k views

How to explain model? [duplicate]

I am building model in R. Can you help me how to interpret this results. For example what is Pr(>|t|) and others...Thanks! ...
1
vote
1answer
2k views

Multiple Linear Regression with small dataset

I have a dataset of 30 social variables such as Facebook Likes, Posts, Comments, etc. I would like to see if these variables predict Website Views. MY problem is I only have 3 months of data- or 3 ...
-2
votes
0answers
35 views

SVM or neural network? [closed]

Why does a Support Vector Machine generalize better than a normal single layer neural network? I think it's because of maximal margins of SVM. Can anyone help me to understand why SVM is better than ...
0
votes
0answers
14 views

Change in the coefficient of determination (R2) when multiplying dependent variables by the independent one in linear regression

I have three independent variables {x_1,x_2,x_3} that I use to fit to a dependent variable y using an OLS regression. The coefficients of determination (R2) of the variables in relation to y are: R2(...
0
votes
0answers
11 views

Adjusting for Covariates: Not Enough Sometimes

I am aware that when the difference in the distributions of a covariate between groups (predictors) is too wide, a simple model adjustment (in regression) may not be enough. And, this may often lead ...
0
votes
0answers
5 views

Explanatory variables (exposures) that can be thought of as additive - best practices over multiple models?

I am modeling an exposure against an outcome (not COVID-19 related). The intended purpose of my analysis is to better understand (from a descriptive perspective) the nature of this exposure in regard ...
6
votes
2answers
193 views

Confusion in classification and regression task exception

The distinction between classification and regression accounts for a model output. I know that classification models have discrete and regression continuous outputs. I want to focus on a taxonomy ...
3
votes
0answers
168 views

When can Autocorrelation of Residuals be ignored?

One assumption of OLS regression is that residuals are idependent, so that there is no autocorrelation. When I checked the assumption, I noticed that autocorrelation is present. Now here are two ...
0
votes
0answers
24 views

Seasonal spikes in residual ACF plot for a Fourier regression ARIMA error model

I have recently fit a Fourier regression ARIMA error model to some time series data, which has weekly and yearly seasonality. The model is of this form (except there is another sum since I have ...
2
votes
2answers
24 views

Appropriate statistic for describing differences between nested regression models?

I have run a series of nested binary logistic and negative binomial regression models in SPSS examining the impact of an intervention on re-offending outcomes. For example: Model A = Individual ...
0
votes
1answer
23 views

Interaction Plot error: The lines do not meet

Hello I wanted to develop an alternative regression equation just to prove that my first regression equation is a better fit. I want to show the possibility that my independent variables my be ...
0
votes
0answers
17 views

Modelling Unbalanced Panel Data and Time-Invariant Explanatory variables

I am trying to create a regression model analogous to the following conditions: My dependent variable, $Y$, is unbalanced panel data (Investment flows over varying years for almost all countries) My ...
0
votes
1answer
35 views

Regression with percentage (%race/gender) as a predictor variable

I have data on hospital admission rates for 5 years at zip code level. I also have percentages on each of the 3 race categories, and percentages of gender for each of the 5 years at the zip code ...
0
votes
0answers
11 views

Python Libraries for different regression methods [closed]

What is the best library/functions to perform regressions like standard least squre regression (SR), Inverted standard least squre regression (ISR) , orthogonal regression (OR), general orthogonal ...
0
votes
0answers
15 views

Can I interpret the signs of coefficients in a VAR model?

I estimated a VAR-model. I checked the time series for stationarity and after estimating the model the residuals and all is fine. I know that it makes no sense to directly interpret the coefficients ...
1
vote
2answers
125 views

Pearl's Causal Inference In Statistics: Study Question 1.5.1

Problem Statement: Suppose we have the following Structural Causal Model (SCM). Assume all exogenous variables ($U$) are independent identically distributed standard normals. \begin{align*} V&=\{...
0
votes
0answers
9 views

Differing AAPC and Confidence Intervals Using the Joinpoint Regression Trend Analysis Software vs R

Let's say I have a data-set with trend data looking at an adjusted rate by year from 1980-2000, with a standard error associated with it: ...
1
vote
1answer
38 views

norm of ridge regression estimator

is there a characterization or an upper bound on the norm of the ridge regression estimator (coefficients)? As the Tikhonov regularization attempts to regularize these coefficients as part of the ...
0
votes
0answers
14 views

How to test whether group membership type 1 predicts group membership type 2?

I am planning an experiment in which I will categorize participants into groups based on performance on two tasks. Based on task 1, participants will be categorized into a "high performer" ...
2
votes
1answer
30 views

Theoretical Justification for Zellner's g Prior

What is the theoretical justification for Zellner's g prior for linear regression? I cannot see how it is possible to justify from a purely Bayesian perspective, in which probabilities are epistemic, ...
0
votes
0answers
12 views

Beta regression and post hoc test correct linear connectors?

I am trying to run a beta regression on my percent data which can be found here. My research question: How is percent cover changing in each transect over each event? Beta regression was choosen based ...
1
vote
1answer
24 views

Topic modeling for regression

Is there a way to influence the way topics are created with topic modelling in the sense that the topics also reflect their influence on the target variable of a machine learning problem? I have a ...
2
votes
1answer
61 views

Should we use regression or classification for recommendation systems?

Lets say that based on some user available ratings i want to predict the ratings of unrated products for some specific user. I totally understand the fact that classifier models are for categorical ...
0
votes
0answers
9 views

Generated regressor problem if using point estimate in second regression

I have a non-standard regression setup in which I run: $y = \beta x +\varepsilon $ to estimate $\beta$. I then want to use the estimated $\hat{\beta}$ in a second stage regression: $c = \gamma \cdot \...
0
votes
0answers
9 views

Conceptual difference between R squared and variance score

I am looking to gain conceptual understanding of the difference between these terms. Note that I do know the formula for both. These have been presented clearly in: What is the difference between $R^2$...
0
votes
0answers
15 views

Why does changing quarter to months of time series change the R-squared value?

When I use plot the average value month on month Vs quarter on quarter, I get different R-squared value. What does this mean for my regression? Do I pick month / quarter based on a higher R-squared ...
1
vote
0answers
13 views

I need a concrete example of collinearity and multicollinearity in linear regression [duplicate]

Consider a set of examples $x_1, x_2, x_3, \ldots, x_N$ where each $x_n \in \mathbb{R}^D$ I form a design matrix $X$, defined as $X = \begin{bmatrix} x_1^T \\ x_2^T \\ \vdots \\ x_N^T \end{bmatrix} \...
1
vote
0answers
9 views

Confidence interval bands in non linear quantile regression plot [closed]

I am working with the quantreg package in R, and I am using a non linear regression to fit my data: ...
0
votes
1answer
15 views

Controlling for Exam Change in Regression Model?

I am analyzing exam data collected from participants who applied to an academic program. The data was collected from 2014-2018, and applicants only took the exam once when applying to the program (...
0
votes
1answer
21 views

Interpretation of GAMM with Factor level Predictors

I'm running the following model in R using the package mgcv: ...
1
vote
0answers
15 views

How to deal with multicollinearity on categorical dataset with ordinal and nominal variables in ordinal Logistic regression to get odds ratios?

My dataset consists of ordinal and nominal variables for independent variables and an ordinal variable for the dependent variable. I have been trying to get odds ratios using the coefficients of ...
1
vote
1answer
17 views

Can I directly apply regression to stacked data?

I have a dataset generated from a survey. Each respondents mentioned several friends and how they interacted with these friends. Below is a simplified example. I am interested in knowing how ...
0
votes
0answers
13 views

Transforming data with a range of values

What is the best way to compute descriptive statistics from a wage_range variable collected as follows and use it for OLS purposes or should this variable be completely ignored from empirical analyses:...
0
votes
1answer
20 views

Interpreting a regression following PCA

I am looking for some help and I am struggling to find any answers. For my course i have completed Principal Component Analysis on my data (a health survey with 8 different variables - smoking, ...
1
vote
0answers
12 views

Intermittent Electricity Output - Causal Effects

I am working on modeling the electricity output of a single power plant. More specifically, I am trying to compute causal effects of a variable prop on output. My model would look something like this $...
5
votes
2answers
147 views

Prove that the OLS estimator of the intercept is BLUE

Consider the simple linear regression model $$y_i = \alpha + \beta x_i + u_i$$ with classic Gauss-Markov assumptions. In proving that $\hat{\beta}$, the OLS estimator for $\beta$, is the best linear ...
1
vote
0answers
22 views

Variable importance in random forest regression: scaled or unscaled

If I want to use the original random forest variable importance measure Mean Decrease in Accuracy [1] (which can be the Increase in MSE, for example) when applying random forest regression, should I ...
0
votes
1answer
35 views

Bayesian regularization intuition - what is a distribution of weights?

I understand the motivation for regularization, as well as its more "conventional" definition as a penalty in the loss function, including how ridge, LASSO and ElasticNet work. But I do not ...
0
votes
0answers
7 views

Linear regression regularization cost penalty magnitude vs dimension

If overfitting is a consequence of parameter dimension being too high, i.e., too close to the number of data points, why does the penalty applied to the cost function penalize $|\textbf{w}|$ (or $(\...
1
vote
2answers
287 views

How to simulate the outcome in a simple linear regression given X and R-squared?

Suppose that we have a fixed $R^2$ and one predictor $X$, sample size = $n$. How can we simulate an outcome variable that follows a normal distribution with $\epsilon \sim N(0,1)$ so that in a simple ...
0
votes
1answer
136 views

Using percentage change as a dependent variable?

If I use percentage change as my dependent variable what is the correct modeling method? I am trying to see how the size of a company affects its losses due to COVID-19. I use quarter over quarter ...
0
votes
0answers
9 views

How to interpret decreasing trend in residual vs fitted value plot

Below is my residual vs predicted value graph. It can be seen that it has a decreasing trend. I am using a multi-layer perceptron for regression analysis with 100 hidden layers and 1000 features. How ...
0
votes
0answers
4 views

Mediation analysis

What conclusions should be drawn if the indirect effect of variable X on Y(outcome) is significant via mediator variable M but the direct effect of X on Y is insignificant?
0
votes
0answers
4 views

Joint Test for Seasonal Forecasting Model using Dummy Variables

I recently created a seasonal dummy regression model in R given a dataset beginning Jan 2016 and ending May 2020. Given the results below there appears to be statistically significant seasonality in ...
0
votes
0answers
12 views

Continuous transition between additive and multiplicative operator?

Question What is the best way to create a function/operator which can smoothly transition between addition and multiplication? More specifically, is there an alternative to just calculating the ...
1
vote
0answers
5 views

Statistical test for two samples and unreplicated blocked data

I used the Friedman test with more than two samples and unreplicated blocked data. However, in my situation, I have only two samples and unreplicated blocked data. What statistical test you think ...
0
votes
2answers
47 views

How to describe the relationship between these two variables?

I would like to quantitatively describe the relationship between the two variables shown in this plot, but I am not sure what would be the correct way to do so. More specifically, my aim would be to ...
1
vote
0answers
23 views

Distribution of quadratic form?

Let $\mathbf{y = X \boldsymbol{\beta} + \epsilon}$, where $\mathbf{X} \in \mathcal{R}^{N\times p}$, $\boldsymbol\beta \in \mathcal{R}^p$ and $\boldsymbol{\epsilon} \sim N(0,\sigma^2 \mathbf{I}_N)$. ...