Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
0
votes
0answers
29 views

What does negative RMSE value mean? [closed]

I am using the this dataset. While training a ElasticNet. regression model I am getting negative RMSE value.I calculated the corresponding r2 score it almost had 99.98% indicating overfitting. What ...
1
vote
1answer
31 views

Why does a Least squares distribution look like a parabola?

For regression analysis, one often uses the least squares method to minimize the quadratic differences between data and a model function f as follows: $$\chi^2=\sum_i \frac{(\text{data}\, _i-f_i(p))^2}...
0
votes
0answers
8 views

How can i increase the r2 value on validation data? [closed]

I'm having a problem finding a model for my regression problem, I've tried various models with no success. I'm using 5 fold cross validation and optimizing for the r2 metric, but I get results similar ...
0
votes
1answer
30 views

Beta regression fitted values

I have a beta regression model in R, have generated predicted (fitted) values based on my data, and plotted lines of those fitted values on a scatter plot of the actual data. I'm most used to GLMMs, ...
2
votes
2answers
192 views

Can redundant/irrelevant features be called a Noise?

Let's say we want to predict job applicant' salary. We have a dataset with following features: {Age, Experience, Education, Astrological_Sign, Weather_Today} 5 features in total. In this set, ...
0
votes
0answers
23 views

How do I interpret a simple linear regression model when both dependent and independent variables are square root transformed? [duplicate]

Overview I built a simple linear regression model to understand if Universal Healthcare Index predicts suicides. My independent variable is Universal Healthcare Index (scale from 1 to 100). The ...
1
vote
1answer
20 views

Lewbel (1997)'s Higher Moments IV Approach for Multiplicative Model

I wonder how I can introduce Lewbel (1997)'s higher moments IV approach in a multiplicative / log-log model. Assume the following linear model: We know that e.g. $Y_t=5+1 X_{1t}+1 X_{2t}+1X_{3t}+\...
0
votes
0answers
29 views

Which regression method causes the ordered coefficients to decrease quickly in absolute value? [closed]

Which regression method causes the ordered coefficients to decrease quickly in absolute value? Lasso lets the coefficients shrink to zero, but I don't know if that has to be in the order of the ...
0
votes
1answer
131 views

Adjusting for age and gender in ANOVA

I am performing an ANOVA to compare the means of three groups. However, I need to adjust for the effects of age and gender in the ANOVA. I'm not quite sure how to go about it in R. Conventionally, I ...
0
votes
0answers
16 views

Choosing the most appropriate method for different data sets [closed]

I have five datasets with the following characteristics: Data Set 1: Many small but comparable coefficients, but no zeros (n=100, p=90) Data Set 2: Most of the coefficients are zeros (n=100, p=2001) ...
6
votes
2answers
2k views

Setting max_depth greater than the number of features in a Random Forest

I was using random forest regression to predict the price of a house. There are only 3 features in data set. Initially when I had set max_depth=2 the result was ...
0
votes
0answers
14 views

Constraining regression coefficient to non-zero

I have a regression problem where I don't want the coefficients to be negative. Is setting negative coefficients of OLS to zero the same as constraining the coefficient to be non-zero and solving it ...
3
votes
1answer
137 views

Generalized least squares error estimation

First of all, I have to admit that I am not statistician so some of my nomenclature could not be very rigorous and maybe a bit confusing; pleas ask me to clarify if necessary. The Problem Let's say ...
2
votes
1answer
120 views

Environmental variable vector in NMDS

I am using Non-metric MultiDimensional Scaling (NMDS) on a Bray-Curtis dissimilarity matrix. Then, I am trying to link the resulting NMDS axes (let's say "components") to environmental ...
0
votes
0answers
6 views

R Library for Ordinal Regression with Split-Plot (Cluster?, Repeated Measures?) Structure

I have a data set with the following structure. The response is ordinal. There is an experimental factor (with two treatment levels, each treatment level applied to a different sample of subjects). ...
120
votes
8answers
63k views

Why L1 norm for sparse models

I am reading the books about linear regression. There are some sentences about the L1 and L2 norm. I know them, just don't understand why L1 norm for sparse models. Can someone use give a simple ...
1
vote
1answer
44 views

Econometrics meaning of structural versus regression model

I want to make sure my understanding is correct. Particularly in econometrics, when authors write down a model: $Y_i = \beta_0 + \beta_1 X_i + \epsilon$ Can I think of this as a 'structural model'- or ...
1
vote
0answers
38 views

Appropriate goodness-of-fit test for the negative binomial regression

I have used the following Pearson $χ2$ test and the deviance test to assess the negative binomial regression using R as ######################################### ...
0
votes
1answer
28 views

Including a weighting variable in a linear regression

I'm looking at how temperature affects length. My length variable is the mean length calculated for every year, it is derived from ~10,000 data points. Not every year had the same sampling effort (e.g....
0
votes
0answers
8 views

Regression analysis before an optimization problem with an unkown function

I have data consisting of invested hours in different training seminars $T_{1},T_{2},...,T_{50}$ and the performance on an exam $P_{exam}$. Currently I know how much 1 hour of a training seminar costs ...
1
vote
0answers
19 views

Dummy variables and the weekend effect

I am running a regression to see the well documented "weekend effect" in the stock market. The weekend effect is a phenomenon in financial markets in which stock returns on Mondays are often ...
4
votes
1answer
144 views

How to model a conditional probability without estimating joint PDF?

I've around 3e3 two-dimensional data points, x, d 1 1, 0.1 2 3, 0.1 3 2, 0.2 4 1, 0.5 range(x) = [-600, 600], range(d) = [0, 1] We are trying to model ...
0
votes
0answers
6 views

What is the conditional variance of sample predictor on population predictor?

I am using the book introduction to Statistical learning with applications in R chapter 3. I've been able to find the conditional expectations, as well as the unconditional variance, but I've read ...
0
votes
0answers
15 views

Fitting a multivariate linear regression with different residual variance for each outcome (using a mixed effects model in R)

In a small simulation, I am fitting a multivariate normal model to predict two outcomes Y1 and Y2, while also modelling the covariance between them. This can be done through a mixed effects model (...
0
votes
1answer
131 views

Lasso and its dual: rates of regularisations

Let us consider the following lasso estimator: $$ \hat{\beta}_{L} = \arg\min \, \frac{1}{n}\sum_{i}^{n}||y_{i} - \textbf{x}_{i}\beta||_{2}^{2} + \frac{\lambda_{n}}{n}\sum_{j=1}^{p}|\beta_{j}| $$ For ...
27
votes
5answers
47k views

Why would anyone use KNN for regression?

From what I understand, we can only build a regression function that lies within the interval of the training data. For example (only one of the panels is necessary): How would I predict into the ...
16
votes
4answers
6k views

Updating linear regression efficiently when adding observations and/or predictors in R

I would be interested in finding ways in R for efficiently updating a linear model when an observation or a predictor is added. biglm has an updating capability when adding observations, but my data ...
3
votes
1answer
121 views

How to generate data that have given conditional mean and conditional quantile using R?

Suppose I want to generate independent data $(y_{i},x_{i})$ such that the conditional mean of $y_{i}$ given $x_{i}$ is a quadratic function in $x_{i}$ and the $.25$ conditional quantile of $y_{i}$ ...
0
votes
0answers
15 views

Can a range of priors being used for a linear regression be applied to a logistic regression?

I have trial level data from a study in which participants responded to a series of stimuli. I have a predictor of interest. For the sake of this example, let's call it the size of the stimulus. There ...
10
votes
1answer
2k views

Hidden state models vs. stateless models for time series regression

This is a quite generic question: assume I want to build a model to predict the next observation based on the previous $N$ observations ($N$ can be a parameter to optimize experimentally). So we ...
0
votes
2answers
30 views

Assumption of linearity between variables and log odds in logistic regression

I know that in logistic regression we assume a linear relationship between the independent variables and the logits. Can you explain why is this a reasonable assumption?
13
votes
4answers
8k views

What does linear stand for in linear regression?

In R, if I write lm(a ~ b + c + b*c) would this still be a linear regression? How to do other kinds of regression in R? I would appreciate any recommendation ...
1
vote
1answer
140 views

How to achieve linear relationship between predictors and logit of outcome?

Prior to conducting a logistic regression of the 0/1 likelihood of a nest hatching or failing as a function of 9 continuous predictors, I plotted each of the standardized (mean = 0, SD = 1) predictors ...
0
votes
1answer
27 views

Running a Regression… Sign on Coefficient seems to be the “wrong way”

I am looking at county level data to assess which variables are correlated with poverty. The dependent variable is poverty. As you can see, I am running a linear regression and here are the results. I ...
0
votes
0answers
14 views

Unstandardized estimated variance from regression

I am running a multivariate regression where both y1 and y2 are both predicted by x. For ...
17
votes
1answer
13k views

Fitting models in R where coefficients are subject to linear restriction(s)?

How should I define a model formula in R, when one (or more) exact linear restrictions binding the coefficients is available. As an example, say that you know that b1 = 2*b0 in a simple linear ...
1
vote
2answers
72 views

Homoscedasticity and independence of errors

In linear regression I often see homoscedasticity and independence of errors listed as assumptions (for example on wikipedia). But I would think that independence of errors would imply ...
0
votes
1answer
46 views

How do I determine when a (non-independent) time series approaches a horizontal asymptote?

I have time series data with many data points per subject over time. I want to determine the marginal time interval within which my dependent variable (dv) falls within given "equivalence" ...
2
votes
1answer
23 views

Adding a Dummy Variable to glm in R?

I'm running a glm in R with two categorical variables, one of which is binary, the other of which can take on five values. I would like it so that my model returns an intercept value that reflects the ...
3
votes
2answers
1k views

Regression Lines With Same Intercept

So, I struggle with Regression a lot. I just found out how to get 2 lines with the same slope, but I cannot manage to get 2 lines with the same intercept. I read about ANCOVA a lot (because I thought ...
0
votes
1answer
29 views

Does logit transformation of information entropy values make sense?

I have a vector of information entropy values that range between 0 and 1 which I want to explain with some explanatory variables. I realized that the distribution of the entropy values in my dataset ...
0
votes
0answers
14 views

What does this output tell me about my regression coefficients?

I've run a cross-lagged panel model with two variables in Mplus. I've received the following output and I'm not sure how to interpret the residuals (these are standardised). Are these too high to make ...
1
vote
2answers
91 views

Role of data spacing in the x-axis in linear calibrations by least-squares?

What is the role of data spacing along the $x$-axis in linear regression by least-squares? Is it an important criterion to consider in the experimental design of linear calibrations? Linear regression ...
0
votes
0answers
6 views

Given two equations that differ by one predictor, under ridge regression, which estimates are generally larger in magnitude?

Suppose we have two equations $$ Y=\alpha_1X_1+\alpha_3X_3 $$ and $$ Y=\beta_1X_1+\beta_2X_2+\beta_3X_3 $$ Suppose further that $X_1=X_2$, then would it usually be the case that $\hat{\alpha_1}$ or $\...
5
votes
1answer
162 views

In linear regression, we have 0 training error if data dimension is high, but are there similar results for other supervised learning problems?

P.S. I just posted this question on MathOverflow, as I didn't seem to get an answer here. Let's consider a supervised learning problem where $\{(x_1,y_1) \dots (x_n,y_n)\} \subset \mathbb{R}^p \times \...
0
votes
0answers
13 views

How do you interpret the instantaneous causality test of the causality()-function? [closed]

I have used the causality()-function to test a VAR(3)-model for granger causality. The test found that H0 could not be rejected for the Granger causality test, but for the instant causality test with ...
0
votes
1answer
12 views

Regression and interactions

I am trying to understand what is meant when I see something of the phrases like "full interactions between group a, b, and c, as well as variable x fully interacted with with each group" (...
0
votes
0answers
15 views

Multiple Linear Regression Dataset

Does anyone have any ideas of datasets which are suitable to fit a multiple linear regression model. However, I would like this specific dataset to have a binary variable which can be used for a two ...