Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

0
votes
1answer
10 views

Regression shows high multicollinearity even after a PCA

I am comparing men and women using a measure of personality which has 24 variables. I did a oblimin rotated PCA for women and men separately so that they are truly representative of the population and ...
11
votes
3answers
5k views

In a GLM, is the log likelihood of the saturated model always zero?

As part of the output of a generalised linear model, the null and residual deviance are used to evaluate the model. I often see the formulas for these quantities expressed in terms of the log ...
0
votes
0answers
2 views

What kind of discrete limited dependent variable regression do I need? Or other?

I have a sample of 78 scientific articles plus 385 software and data resources used by these papers. Each paper has between 1-26 such resources. Sometimes, the creators of software/data will request ...
1
vote
2answers
63 views

Should this regression be tested for stationarity?

This is a fairly basic question I am struggling with. I am using the oil prices I have currently to forecast the number of cars bought 8 months from now using the regression below. Would this ...
0
votes
2answers
23 views

ISLR - Interpretation of Confidence Interval in Linear Regression

I'm reading about Linear Regression in Introduction to Statistical Learning (Chapter 3) I see the confidence interval defined as A 95% confidence interval is defined as range of values such that ...
0
votes
1answer
61 views

removing collinarity of resulting dummy variable columns

I am currently reviewing some work which uses 'dummy variable encoding' as described here: ...
1
vote
0answers
263 views

feature embedding for categorical features

I'm training a model and among the features, I have the language of the users. Right now I have done one-hot encoding on the language feature. But I think it would make more sense to have the language ...
0
votes
0answers
32 views

MultiTaskLasso vs. Lasso with dummies

I am trying to do a Lasso regression, where one of the features is a categorical string e.g. suppose we have Price,Year,Make for a car. One option would be to use one-hot encoding for Make, and do ...
0
votes
0answers
115 views

Decision Tree vs Regression for Multiple Categorical Inputs

I have a problem with multiple categorical inputs. These categories do not intuitively map to integers, while preserving their adjacent relationship. Does it make more sense to us a Decision Tree ...
0
votes
1answer
42 views

Linear Regression and High Dimensional Categorical Data

I've read that mean encoding is useful for classification tasks with high dimensional categorical data. My question: What kinds of encodings are effective for high dimensional categorical data in ...
0
votes
0answers
1 view

Feature-construction for regression on samples consisting of varying numbers of sub-samples

I recently came across the following regression problem: The goal is to predict some value (e.g. the duration) of an event. One such event consists of multiple sub-events. The amount of sub-events ...
4
votes
2answers
319 views

Using pairwise differences as variable in regression

I have a dataset consisting of 72 data points, with 1 dependent variable $y$ and 37 predictors $X_j$. The typical way to perform a linear regression would be to model $y$ as $$y = XB + \epsilon$$ ...
-1
votes
0answers
18 views

Can I regress a country-level variable with a firm-level variable?

Can I regress a country-level variable with a firm-level variable, for example, the country's GDP and the profitability of the firm? If so, then in this case all companies from the same country would ...
1
vote
0answers
29 views

Regressing on residuals of linear mixed model

So for a linear mixed model like this: $y=X\beta+Zu+e$ I can estimate the BLUP for $u$: $\hat{u}=GZ^TV^{-1}(y-X\hat{\beta})$ Where $G$ is the co-variance matrix of $u$ and $V=ZGZ^T+R$, where $R$ ...
0
votes
4answers
50 views

Machine learning models for regression on small data sets

What are the "best" models to be used for simple regression of 1 numerical variable using only a small data set of e.g. 250 samples and up to 10 features? I understand that the data set is super ...
1
vote
1answer
51 views

Clear explaination of dummy variable trap [duplicate]

I have a confusion in multiple regression about dummy variable trap, so far I had seen tutorials explaining about dummy variable trap and multicollinearity but I'm unable to understand it fully.
0
votes
0answers
15 views

Translate data into parameter coefficients - Bayesian regression

I Have a data set of accident rates from a population in which I'm attempting to identify which factors have the most effect on how many injuries occur from each accident. Since I am trying to ...
0
votes
0answers
22 views

Why Normality is considered as an important assumption for dependent and independent variables? [on hold]

While going through one kernel on Kaggle regarding Regression in that it was mentioned that the data should look like a normal distribution. But I am not getting why? I know this question might be ...
0
votes
0answers
12 views

Method for fusing multiple repeating parameters into a single none-repeating parameter space

I have a system with an input (x) and multiple outputs (A, B, C, etc.) that are all measured on the same scale (y). What I want to do is take a measurement of y to obtain my x value, the challenge is ...
0
votes
2answers
48 views

Only the intercept is significant in regression model (with dummy variable?)

This is a similar question to Intercept significant but not the variables in GLM, but in more detail: my model's dependent variable is change in population density of a state, and the independent ...
1
vote
0answers
13 views

Can I use standardized residuals for outlier detection when I have population data?

I have a data set containing the entirety of the population I'm interested in, not just a sample. However, there is an abundance of outliers in this data, which we have determined is due to a lot of ...
2
votes
1answer
18 views

Label encoding vs Dummy variable/one hot encoding - correctness?

i understand that when label encoding is used, the numeric number can be interpreted to have an order and a model could assume a linear relationship. However shouldn't this be a problem when there are ...
1
vote
1answer
282 views

Using estimated parameters as dependent variables

I'm doing a project on gold mining firms. I've taken the daily prices of the stocks of the 10 biggest gold mining companies in the last 15 years (I'm using the returns in order to make the data ...
2
votes
2answers
43 views
0
votes
1answer
14 views

Use z-scores as measure for regression analysis/ANOVA

I want to calculate the difference between a certain value associated with a decision alternative and the value associated with the objectively correct alternative as a measure of decision accuracy in ...
6
votes
2answers
4k views

Should we standardize the data while doing Gaussian process regression?

I am performing Gaussian process regression (GPR) and optimizing over hyper-parameters. I am using minFunc to perform all optimizations. My question is should we (...
4
votes
2answers
2k views

Difference between correlation and covariance: is covariance only useful if the relation is linear?

I'm trying to understand better the difference between covariance and correlation, besides the fact that the correlation coefficient is a dimensional and has values between $-1$ and $1$. One ...
0
votes
0answers
6 views

Model for Predicting Sales Drop in Drop Ship Model E-commerce [on hold]

I am trying to build a rule based model to predict the sales drop at product level on a particular week...I have data at week level for about 8 weeks and the independent variable that I am having are ...
1
vote
0answers
12 views

Finding the exponents of a multiple power law: is linear regression valid?

I want to fit a multiple power law equation of the form $y = {x_1}^{\alpha_1} {x_2}^{\alpha_2}$ where I have many examples of $y, x_1, x_2$. (Note there is no intercept.) Is it possible for me to ...
-1
votes
2answers
461 views

Auto.arima is not fitting the data well

I have two variables speed and vibration and you can see that speed causes vibration. I am trying to fit this using auto.arima. But when i plot the fitted model, it gives bad result ...
71
votes
3answers
16k views

Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
-1
votes
1answer
53 views

Machine Learning Lottery [on hold]

I have lottery data and I want to use Scikit-Learn to create several different models starting with Regression(One vs All). I know the lottery is random, and impossible to predict, but I want to use ...
0
votes
0answers
31 views

Converting orthogonal coefficients to monomial form using specific data [on hold]

Using mtcars dataset and its first 2 variables, mpg and disp, can someone show me how to convert orthogonal coefficients made with poly 3 to monomial so I can evaluate using “basic” equation. I am ...
0
votes
0answers
46 views

Interpretation of logit model without reference category

I have developed a logit model with binary dependent variable in R using the glm function. Overall, the independent variables include seven nominal, 23 ordinal, and two numeric variables. The nominal ...
0
votes
0answers
16 views

Difference-in-differences analysis with multiple time periods

I am looking at the effects of a law on listed companies. I have a control group and a treatment group, but instead of having a pre treatment period and a treatment period I have 3 time periods. I ...
0
votes
0answers
9 views

2x2 crossover design linear mixed model [on hold]

I have a 2x2 crossover design study data. I am looking to find the effect of a particular diet on the study participants. My factors are Diet- (Chicken and Pork), Intervention ( Pre and Post), Sex (...
0
votes
0answers
16 views

What is difference between interrupted time series and regression discontinuity design

Say that one has data over time, t, on an outcome, y. There is an event that happens at t==0....
2
votes
1answer
15 views

Pattern in residual plot

I am still learning about regression and I am currently trying to perform some basic analysis on my experimental data but I do not know how to deal with or interpret this obvious linear line in my ...
1
vote
1answer
50 views

Writing up findings from a summary given for a linear regression model

I am new to linear regression and came across the following paper when looking for examples to practice: https://www.maths.cam.ac.uk/sites/www.maths.cam.ac.uk/files/pre2014/postgrad/mathiii/pastpapers/...
12
votes
2answers
3k views

Cross validation and ordinal logistic regression

I am trying to understand cross-validation for ordinal logistic regression. The aim of the game is to validate the model used in an analysis... I first construct a toy data set: ...
2
votes
1answer
11 views

Differences between probabilistic regression + threshold and classification?

Working on probabilistic models, we often end up thresholding the result to decide if we should take some action or not. This method allow simple and explicit decisions, while being adaptative to our ...
0
votes
0answers
7 views

Ordinal regression on probabilities

Say I have a data set X, and a binary output y. I use a probabilistic model to determine the probability of y being one. Then for practical reasons I put my data in ordered buckets, with simple or ...
0
votes
0answers
7 views

How to interpret the ratio between a coefficient on a dummy and the coefficient of a log income variable?

I was reading this paper by Kahneman & Deaton: Kahneman, D., & Deaton, A. (2010). High income improves evaluation of life but not emotional well-being. Proceedings of the national academy of ...
0
votes
0answers
16 views
+100

Proportion log-regression with constant population and events

I have a population n, which is subject to event e. I classify my population into buckets $n_1,...,n_m$ with associated number of event $e_1,...e_m$. The goal of the classification is to distinguish ...
0
votes
0answers
6 views

feeding distribution over time into regression model - many cooks spoil the broth

This is a simplified/contrived example. Let us say I want to model the overall time of a meal including cooking time and prep time depending on features of the dish and the types of cooks. Let us say ...
1
vote
0answers
6 views

Moran's eigenvector spatial regression vs autocorrelation term

Comparing two methods for spatial regression: Moran's Eigenvector-based spatial regression Regression with a term for spatial autocorrelation e.g. where $Y = \beta X + \gamma MY + \epsilon$ for ...
2
votes
0answers
22 views

How to fit the following model in R? [duplicate]

'Rwml' is the monthly log return So the first column is clear, I got nearly the same values, at least the same magnitude. But: If I regress on the variance, my input values are way too low to get a ...
36
votes
4answers
70k views

Difference between forecast and prediction?

I was wondering what difference and relation are between forecast and prediction? Especially in time series and regression? For example, am I correct that: In time series, forecasting seems to mean ...
2
votes
1answer
109 views

When to prefer PCA over regularization methods in regression?

When dealing with the curse of dimensionality, regularization methods seem to be clear in their intuition. All "regularization" methods can be seen as a "squeezing" of one's variables towards ...
2
votes
0answers
15 views

why Durbin Watson result could be so different from Ljung-box or Breusch–Godfrey test for OLS diagnostics

I have a residuals series from OLS regression (out.lm) where I do NOT have lagged dependent variable as a predictor. My residual series has about 1700 numbers. I ran ...