Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

When can we use fixed design regression results for the random design setting?

Suppose I have an independent vector $X$ and a dependent scalar random variable $Y$ and I wish to construct a regression model to predict $Y$ using $X$ given data $\{(x_i,y_i)\}_{i=1}^{n}$. For ...
0
votes
0answers
8 views

R-squared and sample size

I was wondering if R-squared is affected by the sample size? Is adjusted R-squared also affected? The reason behind this though is, that i have run a multiple linear regression on two samples. The R^...
0
votes
0answers
6 views

Formula for type III sum of squares of the intercept term in linear multiple regression

assume we have the regression model: $$Y = b_0 + b_1 x_1 + \dots + b_k x_k + \varepsilon $$ I know the formulas for all type III sum of squares for the regression terms except the formula for SS of ...
1
vote
1answer
15 views

Detrending time series for regression

I want to perform a regression between 3 variables [x1,x2,x3] that have no trend and no seasonality across their time observations and a variable [Y] that has trend and seasonality. For [Y] I've ...
0
votes
1answer
14 views

How to represent gender or any non numerical data for a Logistic regression classifier?

I've coded a logistic regression classifier trained on the Kaggle Titanic dataset to predict whether or not a passenger will survive. I've noticed that depending on how I represent the gender the ...
2
votes
2answers
114 views

Regression when x and y each have uncertainties

I have a set of $N$ points $(x_i,y_i)$. $X$ and $Y$ both have some noise associated with them due to measurement inaccuracy however the relationship of the underlying true values (i.e. if we could ...
0
votes
1answer
39 views

Simple linear regression swap x and y, why t/F don't change?

I am playing with the simple linear regression (y=ax+b both x and y are scalars). And I notice that after I swap x and y, the coefficient changes(which we can find a nice answer here). But the t-stat ...
0
votes
1answer
13 views

find subset of n variables (aggregated with OR) most correlated with target [on hold]

I have 25 variables and I'm trying to find a subset of the variables that maximize the correlation between (X1 OR X2 OR.. Xk) and the target variable. The variables are of the type 'in how many ...
0
votes
0answers
21 views

Problem with significance of results from regression on subset of data

I have performed a hedonic regression with a database where all the variables are very significant and according to the calculations made in R with a squared R = 0.6123. I have performed the same ...
0
votes
2answers
45 views

how to deal with panel data using linear regression

i have a data set that with several columns about the top 5 managers in 100 firms from 2009 to 2018, some managers still the same through years and some had been changed for some firms in some years ...
0
votes
0answers
15 views

Derive the expression for the standard errors of the intercept in the simple regression via MLE

I am trying to derive the expression for the variance/standard errors of the alpha parameter in the simple regression framework. The model is: $$y_i = \alpha + \beta x_i + \epsilon_i$$ for $i=1,...,...
0
votes
1answer
12 views

Is it necessary to scale the dependent variable in k-NN regression?

I want run kNN analysis to predict Y (continuous variable). I know that it is necessary to normalize all of the Xs. My question: is it also necessary to normalize Y values?
1
vote
0answers
23 views

Violation of Independence of Observations - Logistic Regression

Apologies if this seems a basic question but I can't find enough detailed information about what exactly constitutes a violation of independent observations. The regression I'm wanting to run is ...
0
votes
0answers
13 views

What is the difference between moderation and adjustment

What is the difference between moderation and adjustment in regression analysis? How can I conduct each of them in SPSS? I would appreciate any type of help. Thanks..
1
vote
0answers
17 views

Regression with sparse data - binning or other approaches?

A couple years ago, I built a GAM to predict game sales regressing away team, sales to date, event month, and days to event against final event sales using smoothing parameters over days to event and ...
0
votes
0answers
16 views

Consistency under mild endogenity

Assume the usual linear model: $$Y_i = X_i\beta + \varepsilon_i, \quad 1\leq i \leq n$$ whit $E(\varepsilon_i)=0, Cov(\varepsilon_i, \varepsilon_j) = \sigma^2 \delta_{ij}$ and $Cov(X_i , \...
1
vote
0answers
30 views

Correct error estimation for linear fit

This may be a simple problem, but I want to be thorough in setting up my problem as I'd like to know why I should proceed in one of two ways (or another if someone thinks it is suitable), so please ...
1
vote
0answers
4 views

How to measure explained variance in y in the case of regression dilution bias?

I would like to find the explained variance in y by x when both y and ...
1
vote
0answers
12 views

What is the interpretation of polynomial terms for ordered factors in Ordinal Logisitic Regression

I am creating an OLR model using R with the polr function in the MASS package. All of my predictors are also ordinal data, all of the data is the integers from 1 to 5 coming from a customer survey. I ...
1
vote
1answer
15 views

Regression with paired, repeated measures design

I have a large population of books. Each book is either a hardback or softback (thus hardback and softback books are paired with one another by title), and can fall into two categorical genres - ...
-1
votes
0answers
14 views

what is the best approach to generate a synthetic data set and get a good prediction?

I'm new to ML and I want to try to predict sales starting from my data set. Considering that my dataset is small (46 observations) I thought of creating random observations (those that in the ...
2
votes
2answers
27 views

Regression with multiple dependent variables controlling for age and gender [on hold]

I have multiple dependent variables (interval) and one independent variable (interval). I would also like to control for age (interval) and sex (categorical). Is this possible in SPSS? Moreover, my ...
1
vote
0answers
8 views

Hausman Test when doing Fixed Effects Panel Regression

I'm currently doing a Fixed Effects panel regression on my data and would like to know whether it would be useful to carry out Hausman tests. From what I understood from lectures, the Hausman test ...
2
votes
0answers
19 views

Low Rank Gaussian Process vs Bayesian Linear Regression

A main benefit of Gaussian Process Regression is, that we not only get a prediction, but also a variance that we might use as indication of the prediction confidence. While bayesian linear regression ...
3
votes
0answers
20 views

Is explained variance a valid measure for weighted regression?

How should we calculate the R^2 statistics in weighted linear regression? Should we also weight each residual to decrease the sum of squares of the residuals? If ...
2
votes
0answers
12 views

What model to use to evaluate whether the “variability” of one variable is determined by the “variability” of other variables?

I have observations taken for four subjects at a number of points/locations. The data is formatted similarly to as below: I am not directly interested in the impact that a predictor has on the ...
0
votes
0answers
15 views

Discrepancies in p-values between t.test vs regression after random forest analysis [duplicate]

I was trying to analyze two groups of samples for multiple variables. I first used Boruta (random forest analysis) test to determine the importance of variables in my data. ...
1
vote
0answers
72 views

Loss functions for Regression task

I am trying to understand the idea of Loss functions For Regression Task perfectly. I have read many textbooks and articles, and I came up with questions related to this subject. Several different ...
1
vote
1answer
35 views

Regression terminology, predictor vs IV vs?

A reviewer has objected to our use of the term "predictor" in a multiple regression analysis using observational data. They argue that because the model is intended to be explanatory and not ...
-1
votes
0answers
6 views

Multiple orthogonal regression (aka multiple total least square) in excel [on hold]

I am trying to make a multiple orthogonal regression in excel. I need to do it strictly in excel - cannot use programming languages. I did not find any free addins that calculate it for me, ...
0
votes
0answers
26 views

Adjust for confounder in calculating explained variability in cox-regression

My question is closely related to this one. I am interested in the proportion of variability which is explained by a certain covariate X in a cox-model. So I have the cox-model “outcome ~ X”, for ...
0
votes
0answers
12 views

Simulated Regression - getting the population like data

I try to evaluate measures for Feature Importance for a regression. I have data set with highly correlated features. So the betas have a variance. This makes it difficult to estimate the "true betas" (...
1
vote
2answers
51 views

regression with multiple independent variables vs multiple regressions with one independent variable

For example, we want to use age and IQ to predict GPA. Of course we can do a multiple linear regression, i.e. regress GPA on age and IQ. My question is: can we do two simple regressions instead? ...
0
votes
0answers
5 views

Significance of hyper parameters in the DHR model in R forecast package

The Dynamic Harmonic Regression model in R requires the input of parameters K, the length of which depends on the number of seasonality in the forecast data. According to https://otexts.com/fpp2/dhr....
0
votes
0answers
13 views

What is the difference between lasso and WOE encoding in logistic regression?

I know lasso is one of the best method to select important variables and make variables sparse. But WOE encoding does the same thing, making variable smooth. I would like to know what is the ...
0
votes
0answers
9 views

Question about evidence approximation when maximise wrt alpha (bishop chapter 3.5.2)

I'm reading 3.5.2 of PRML book and am confused about the result of taking derivative of marginal likelihood function with regard to alpha. In particular regarding this function: And according to the ...
0
votes
1answer
26 views

interpretation of Linear regression

I was reading about linear regressions on wikipedia and came across the mean and predicted response. I just wanted to clarify somethings. So suppose we have a simple linear regression model, is the ...
2
votes
1answer
37 views

How to predict by hand in R using splines regression? [on hold]

The R package splines allows one to fit a non linear model using splines. For instance, ...
3
votes
0answers
30 views

If parallel trends assumption is not met for diff-in-diff analysis, why not add more interaction terms to regression model?

The literature presents a violation of the parallel trends assumption as a major obstacle when performing a difference-in-differences analysis. Why can't we model non-parallel trends by simply ...
0
votes
1answer
27 views

Comparing multiple lm results created in ggplot2 [on hold]

I have the following example plot: Created via: ...
1
vote
0answers
19 views

Why prefer poly() to I() ? Are they different? [duplicate]

A post "Fitting Polynomial Regression in R" used two ways to model the polynomial regression: (a) poly(..., ...); (b) I(...). ...
0
votes
0answers
21 views

Can MAE be interpreted as the average standard deviation around the true value of a prediction?

MAE is defined as the Mean Absolute Error, that is how far on average the prediction (derived from some prediction model) lies from the real value. The standard deviation is usually interpreted as ...
0
votes
0answers
18 views

How to generate b-splines that are orthogonal to the corresponding variables in non-linear regression?

I want to fit a non-linear regression model of the type $$y_i = \alpha_0 + x_i\alpha_1 + s_i^T\beta + e_i,$$ $i=1,\dots,n$, $\alpha_0,\alpha_1\in{\mathbb R}$, $\beta\in {\mathbb R}^p$. I am only ...
0
votes
0answers
9 views

Flexible Regression Specification for Panel Data in Stata

I am currently looking to estimate a somewhat complex regression model to analyze the market value of banks. The model can be found in Calomiris and Nissim (2007), and is pictured below. The model is ...
0
votes
0answers
45 views

How can I try to do logistic regression of time series? [on hold]

I'm trying to get some information using my data. My data set is below. ...
0
votes
0answers
17 views

best model for geopgraphical distribution of a characteristic prevalence

I've data about the presence of a specific allele E4 among different geographical region populations (e.g. nord,...
0
votes
0answers
23 views

Subtract independent variable from target in linear regression

I have a simple linear regression problem where: $y = a*x_1 + b*x_2 + c$ In my problem my target variable is more conveniently defined as: $y - x_1 = b'*x_2 + c'$ I still get the same coefficients ...
2
votes
1answer
34 views

Plotting of residuals by fitted values vs. sorted by fitted values

My question concerns two methods for plotting regression residuals against fitted values. The standard method: You make a scatterplot with the fitted values (or regressor values, etc.) on one axis (...
0
votes
0answers
25 views

What am I looking at here in this regression analysis? [closed]

I have no idea what to do with this information or how it helps me determine if education level influences tolerance towards minorities. Control Variables are age, gender, province, and if they ...
0
votes
1answer
21 views

Measuring R-Squared by category

It makes sense to look at metrics like recall/precision by category when performing classification. sklearn has classification_report for this purpose. But what if I want to look at error by category ...