Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Comparing incident rate ratios among dummy variables

The output below is from the Poisson example at https://stats.oarc.ucla.edu/stata/dae/poisson-regression/. The incident rate for program 2 is 2.96 times the incident rate for program 1 (the reference ...
  • 51
6 votes
2 answers
184 views

Coefficient of an Interaction Term when Regressors are Independent

Say we have an OLS regression of $y$ on $x_1$ and $x_2$, where both $x_1$ and $x_2$ are independent from each other, and create the following regression: $$ y= \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \...
0 votes
0 answers
3 views

How to determine reward point sensitivity at the customer level

I'm wanting to come up with a way to look at point sensitivity for a rewards program my company (grocery store chain) has at the customer level. The way the program works is, if the customer reaches ...
2 votes
1 answer
29 views

confidence interval for linear regression coefficient with error following a t distribution

I have this linear model $$𝑌=𝛽_0+𝛽_1𝑋+𝜖$$ where the error terms 𝜖 are iid from a student t-distribution with constant degrees of freedom k. I want to construct a 95% confidence interval for $\...
  • 21
7 votes
1 answer
133 views

Why does degrees of freedom = $\frac{\operatorname{Tr}(H'H)^2}{\operatorname{Tr}(H'HH'H)}$?

Wikipedia gives the following expression in "degrees of freedom" section, calling it the "Satterthwaite" approximation: \begin{equation*} \text{df}\approx \frac{\operatorname{Tr}(H'...
3 votes
1 answer
41 views

Generalized quantile regression? Transforming a conditional quantile like we transform conditional expected value

Linear models are models like $\mathbb E\left[Y\vert X\right]=X\beta$. Linear quantile regression models replace $\mathbb E\left[Y\vert X\right]$ with $Q_{\tau}\left(Y\vert X\right)$ for conditional ...
  • 35.7k
1 vote
0 answers
22 views

Is it possible to create a 0 intercept ridge regression model?

I am working on implementing ridge regression for market mix modeling where I wish to use my own create base(UCM) instead of intercept, I had been using linear regression for this purpose but now my ...
  • 21
0 votes
1 answer
16 views

Sample error terms relationship

In regression models, I just want to know why the sample error terms have such relationship as: $$\hat e_i=y_i-\hat\beta_0-\hat\beta_1 x_i=(y_i-\bar y)-\hat\beta_1(x_i-\bar x).$$ where $(\bar x, \bar ...
  • 137
0 votes
0 answers
13 views

Testing and training data for LMM/GLMM

I was reading this article on logistic regression and ML and noticed that they explicitly mention the need for using training sets of data and testing sets of data: However, I have never seen this ...
4 votes
1 answer
60 views

How to motivate the definition of $R^2$ in `sklearn.metrics.r2_score`?

TLDR: What motivates the definition of $R^2$ in the Python function sklearn.metrics.r2_score? DETAILS The Python machine learning package ...
  • 35.7k
0 votes
0 answers
13 views

Ridge and Lasso regression coefficient change when we change the scales of the variables

I am interested in the following question: suppose we run a ridge of a lasso model on a bunch of variables. Now if we multiple one of the variables $x_1$ by 2, what happens to the coefficients. Some ...
  • 213
2 votes
1 answer
29 views

Coefficient from Regressing the OLS Residual on X

Say we have an OLS residual $\hat{ϵ}$ from regressing $y$ on $X$. If we were to regress $\hat{ϵ}$ on the same $X$, what would the OLS coefficient be? If we rearrange the first regression equation for $...
user avatar
-1 votes
0 answers
13 views

Logistic regression(with Markov chain) on time series data?

I'm working with a biotech device's time series data to predict the replacement amount. The background is the battery of the device will die after the implant for a few years, and the battery will be ...
  • 1
0 votes
0 answers
4 views

Aggregation in (cross-sectional) factor model

Suppose I have a large factor model for security returns, i.e. I have a vector $\mathbf{Y}(t) \in \mathbb{R}^{P}$, with factor loadings $\mathbf{\beta} \in \mathbb{R}^{P \times K}$ over a set of $K$ ...
  • 1
0 votes
0 answers
21 views

Which regression models are appropriate for (ratios of) two different but dependent counts?

Consider a variable Y which is the ratio of the frequencies of two words/concepty (f,g) in a text: Y = f/g. Since the texts do not have equal length, the denominator when calculating the percentage of ...
2 votes
1 answer
30 views

Why does the logistic model for binary logistic regression return the probability that the outcome was in class/category 1?

In the context of simple binary logistic regression (https://en.wikipedia.org/wiki/Logistic_regression) we have $p(x)=\frac{1}{ 1 + e^{\beta x}}$, where $p(x)$ is interpreted as the probability that ...
0 votes
0 answers
12 views

How to interpret MSE RMSE in Randomforest and Xgboost regression [duplicate]

I apply Randomforest and Xgboost regression for my problem; data consist of 6236607. I feed the model the availability of machines, such as how many minutes the machine lasts at a practical start time....
0 votes
0 answers
22 views

Marginal Likelihood Computation for Bayesian Linear Model

Given a simple Bayesian linear model with $N$ observations $y = X\beta + \varepsilon \quad \quad \varepsilon \sim \mathcal{N}(0, \Sigma)$ with known error variance-covariance matrix $\Sigma$ and ...
  • 651
0 votes
1 answer
18 views

Should I use standardised or unstandardised beta coefficient in this scenario?

I am running a regression on Tax payment and investment. Tax payment is numerical and investment is categorical like, number of projects carried out. When I run the regression the a unstandardised ...
0 votes
0 answers
11 views

Finding coefficients that maximize Spearman Rank Correlation [closed]

I have a sample of observations on stocks across say 5 factors , and I want to find the linear coefficients on these factors which when multiplied with the factors and summed produce a signal with the ...
  • 1
6 votes
1 answer
58 views
+50

What do the terms "nearly-optimal rate", "near-minimax rate", "minimax optimal rate" and "minimax rate" mean in the context of posterior consistency?

Definition: A sequence $\epsilon_n$ is a posterior contraction rate at the parameter $θ_0$ if $$\Pi_n(θ: d(θ, θ_0) ≥ M_n \epsilon_n| X^{(n)}) → 0$$ in $P^{(n)}_{θ_0}$-probability, for every $M_n → ∞$. ...
0 votes
0 answers
7 views

anova/F-test in R [closed]

Assume that we have a model L1 nested into model L2, i.e. L2 is the bigger model (more complex). How do I compute the anova/F-test in R? Do I write the small model first or the large model in anova()? ...
0 votes
0 answers
14 views

How are regression residuals calculated for Panel data?

In cross-sectional data, the residuals are calculated along the individual observations. But in panel data there is individual and time observation. So how are the residuals calculated?
1 vote
0 answers
18 views

Individual slopes in linear mixed models vs individual linear regressions

I have a linear mixed model, from which I extracted the slopes for each individual participant (using coef(model)$participant). Then, out of curiosity, I ran a linear regression (equivalent in ...
0 votes
1 answer
19 views

GridSearchCV returns unrealistic AUC score with Logistic Regression

Long time lurker but I've just created an account because its the first time one of my questions has not actually been answered. I'm currently struggling with optimizing the hyperparameters of a ...
  • 13
0 votes
1 answer
32 views

Quantile regression necessarily has a solution with $r$ residuals equal to 0: why/how

Given data $\{(Y_i,X_{1,i},X_{2,i}\dots X_{p,i}):1\le i\le n\}$. let $\theta\in(0,1)$ and $\beta=(\beta_1,\beta_2\dots\beta_p)^T$. Then, the quantile regression problem $$\underset{\alpha,\beta}{\min}\...
  • 103
0 votes
0 answers
13 views

Data Analysis for a Time Series Clinical Trial Dataset

I have a Clinical Trial dataset where, I have the values of Blood haemoglobin level for 40 patients for 6 weeks, after a particular treatment. For drawing some meaningful insights from this data set, ...
  • 67
0 votes
0 answers
15 views

Testing equality of mean responses

In the context of linear regression, we know that $E(y|x_0)$ for a given new data $x_0$ is estimated by $\hat{y_0}=x_0^t\hat{\beta}$. If given another set of new observations $x_1$, is there a way to ...
7 votes
2 answers
515 views

What is the 'right' slope formula of a regression? deltas or Pearson?

this may be a silly question, but still: I've been told that the slope formula equals the rise/run ratio, like this: $$ m = \frac{rise}{run} = \frac{y_2 - y_1}{x_2 - x_1} $$ in which rise equals ...
0 votes
1 answer
22 views

do you need to specify the direction of b for a moderated regression

I conducted a moderated regression, but didn't specify the direction of moderation by M. that is I didnt specify whether the relationship between X and Y would be greater at lower or higher levels of ...
0 votes
0 answers
22 views

Approaches for running separate monthly regressions on a time series

I have a time series with daily granularity. The time series under consideration depends on an independent variable x (say). In order to account for seasonality effects - I run a separate regression ...
0 votes
0 answers
39 views

Interpretation of logistic regression results

I have a database that looks like this ...
1 vote
0 answers
22 views

how well would a robust mixed model fit these data? R (rlmer)

I want to investigate Y ~ X1 * X2 + (1|ID on this dataset (there's a plot of these data in that post too, it's the same dataframe) Y is a continuos outcome ...
0 votes
0 answers
9 views

Change versus level regression

I tried running a panel regression of y on contemporaneous x variables, and again using the change in y from year t-1 to year t. I am getting the opposite sign on my variable of interest (VOI) if I ...
0 votes
0 answers
9 views

How best to quantify how much one part (i.e., a group of items in a scale) contributes to the whole pool?

I am trying to determine how much specific types of stressors contribute to the overall pool of stress experienced, e.g., X type of stress contributes X% to the overall pool of stress, or be able to ...
1 vote
0 answers
18 views

Connection Between Bayesian Prior and Variable Selection in Lasso [duplicate]

I am interested in learning more about the Bayesian interpretation of the Lasso model. The Lasso model assumes a Laplace distribution of coefficients and the optimal coefficients maximize the ...
  • 213
0 votes
0 answers
17 views

Multivariate regression with dependent variables on an interval

I want to do a regression what the probability is of event being True (=P(E)). This dependends on three variables, let's say A, B and C. SO: P(E) = X1 * A + X2 * B + X3 * C My data consists of the ...
0 votes
0 answers
22 views

How to prove that L2 loss converges faster than L1 loss

I see many sources claim this result, however, I was not able to find a proof for it. I think this should be given in some paper or book. Can someone point me to some resources, or even better, show ...
0 votes
0 answers
18 views

How to improve the predictions of a model when we have too few predictor variables?

I tried to use a linear model to explain a variable "age" with two variables "x1" and "x2". I can clearly see a decreasing slope inside my scatterplot for age vs x1, or ...
  • 1
0 votes
0 answers
19 views

Regression with paired repeated measures design

I have the following data: ...
2 votes
2 answers
34 views

GLM: how to treat multiple variables that all measure a confounding aspect in a slightly different way?

For a response variable $y$ and predictor $x_0$, I have data for a number of additional variables $x_n$, $n = 1, ..., 7$. I would like to control for a confounder in my GLM, let's call it "size&...
  • 121
1 vote
0 answers
25 views

Violating assumptions of linear mixed model

I am currently analyzing a dataset using a linear mixed model. In the study - using a within-subjects design - participants had to rate the intensity of a stimulus (this is my dependent variable, DV) ...
  • 13
0 votes
1 answer
28 views

Logistic GLM vs GLMM diagnostic issues

Problem I am running through the diagnostics of two logistic regressions and two equivalent GLMMs with their only differences being crossed random effects (intercepts only). The output for the ...
1 vote
1 answer
32 views

determining the effect of the change in an independent variable using regression

I'm creating a regression model that predicts a customer's spending based on their income, while adjusting for age, gender , and region. The model looks as follows: ...
  • 11
0 votes
0 answers
75 views

The GMM estimator is reliable under the given conditions

Consider the linear model $$y_t= x_t'\beta +\epsilon_t$$ for $t=1,...,T$. where $x_t= ( x_{1t} \ \ x_{2t} \ \ ... \ \ x_{kt}) $ and $\beta$ is $(k\times 1)$ vector of unknown coefficients. Given $z_t= ...
  • 976
2 votes
0 answers
35 views

separate models vs joint model

My goal is to estimate the association between children BMI and distance to the nearest fast food restaurants. The hypothesis is that children BMI increases with increasing proximity of fast food ...
2 votes
1 answer
70 views

How to learn steep functions using neural network?

I am trying to use a neural network to learn the below function. In total, I have 25 features and 19 outputs. The above image shows the distribution of two features with respect to one of the outputs....
  • 175
0 votes
1 answer
27 views

Probit/Logit or Linear regression Model?

I have data about the occurence of a data breach at certain companies for the periode 2005-2018. Now I have a question about the model I should use. I have two options: Probit/Logit: I set the ...
user avatar
1 vote
1 answer
39 views

Robust standard errors with splines

I realize that large changes in model results between using robust and non-robust standard errors can suggest a misspecified model. My case refers to using a Cox regression and I have experimented ...
  • 335
1 vote
1 answer
20 views

Standard error logic

Obviously the closer the standard error to zero the better. However, what if the values the in which the standard errors are from are extremely large and the standard errors are much higher than zero ...
  • 11