# Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

21,143 questions
Filter by
Sorted by
Tagged with
3 views

### Best practice for pairing samples for linear regression

I am building linear regression models in R where the two distributions do not have ground truth or any obvious method for pairing samples from each. What is the best practice for this scenario? The ...
9 views

### What happens when I estimate too many univariate relationships?

Suppose we are in the context of OLS regression, and we have $n$ data points. If I run multivariate regression I can only estimate $n$ coefficients (including the intercept), and at that point the ...
7 views

109 views

### Logistic Regression Loss Function: Scikit Learn vs Glmnet

The loss function in sklearn is $$\min_{w,c}{\frac{1}{2}w^Tw+C\sum_{i=1}^N{\log(\exp(-y_i(X_i^Tw+c))+1)}}$$ Whereas the loss function in glmnet is \min_{\beta,\beta_0}{-\bigg[\frac{1}{N} \sum_{i=...
9 views

### Ridge regression on one predictor and the reduction of the risk of overfitting

In the book Hands-On Machine Learning with Scikit-Learn and TensorFlow, Chapter 1, the author stated that when doing Ridge Regression (for only one predictor), this regularization reduces the risk of ...
21 views

I’d like to understand a good experiment design for the following example: Say sales people are trying to find the ideal time (in days) to send a follow up phone call to increase conversion (a sale). ...
19 views

### If x is correlated with y, is there formula to use that alone to write x as a function of y, or to relate the two variables?

for example, if $\text{corr}(x,y)\not=0$, can you always decompose $x$ to be: $x={\text{cov}(x,y)/\text{var}(x)} + \epsilon$ where $\epsilon$ is just all other remaining parts of $x$ uncorrelated with ...
30 views

### How to fit the intercept

I'm practising using R and I'd like to do this task: So I fitted the model: ...
16 views

### How to check if there is a linear relationship for a logistic regression model

From what I understand logistic regression expects that there is a linear relationship between the log odds of the target and the feature. Fourth, logistic regression assumes linearity of ...
30 views

### Reformulation of logistic regression

I am given the question above and can't seem to get the form that it's asked for. I have tried working it backwards from the goal which gives me: ...
7 views

### Is there within and between R^2 for pooled cross section?

I am writing a referee report for a paper that reports within and between $R^2$ for pooled cross-section regression that includes year fixed effects (but no panel or other fixed effects)so the ...
8 views

### When calculating the variance of a linear combination of least squares estimators, what is C?

I am reading DeGroot and suddenly "C" comes out of nowhere. I am not sure where he got C from and how to calculate it. The context is linear regressions and calculating the variance of the ...
12 views

### In typical regression, if Y | X is normal, is Y itself normal?

I am reading DeGroot and we made the assumption that Y | X is normal and each Y | X has the same variance. However, in deriving the sampling distributions of b0 and b1, he says that Y is normal. Do ...
17 views

### Is there a way to solve for $\sigma^2$ in linear regression only using least squares estimation?

If we only use least squares estimation, is there a way to solve for the variance of the conditional distribution of Y | X? Or do we have to use Maximum Likelihood Estimation? Additionally, do most ...
27 views

### Interpreting Odds Ratio (in an Interaction)

I need advice on the correct interpretation of an odds ratio of an interaction term. Both the mixed-effect logistic regression output is below as well as the predicted odds values, which I calculate ...
15 views

### Is it better to build N models for each category of data?

I'm new to data science and I'm working on a challenge with some friends, I have a data set of 80 feature and around 4000 rows. The data is split into 180 category (A,B,C,D...etc), at first I tried ...
16 views

### Rationale for taking second derivative in least squares estimation?

I am reading DeGroot and he talks about how to derive the b0 and b1 coefficients using LS estimation. I understand everything except the last part where he talks about taking the second derivative of ...
2 views

### Why does one part of the interaction term turn insignificant when the interaction term is added to the model?

Suppose I am trying to figure out whether a bigger shoe-size makes you happy. In my model, I also control for gender. In model 1, there is no interaction term. All coefficients are significant. The ...
9 views

### Using Gaussian Process Regression in scikit-learn

I have a simple dataset with multiple trials of position over time, and I'm trying to fit a Gaussian Process over it. Here's a plot of all the raw data (6180 data points): My goal is to fit a ...
13 views

### While creating dummy variables with n values why cant we just create a single variable?

For a variable that can take on n values, we create n-1 dummy variables, and that part is perfectly understood by me, but why cant we just create a single variable and load it with n unique values. ...
16 views

### Breusch Pagan and White test always 0

I'm working on a machine learning algorithm and trying to evaluate the model based on a guide online. No matter how I change the features of the model, the results of both tests are always 0. Is there ...