Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Best practice for pairing samples for linear regression

I am building linear regression models in R where the two distributions do not have ground truth or any obvious method for pairing samples from each. What is the best practice for this scenario? The ...
1
vote
0answers
9 views

What happens when I estimate too many univariate relationships?

Suppose we are in the context of OLS regression, and we have $n$ data points. If I run multivariate regression I can only estimate $n$ coefficients (including the intercept), and at that point the ...
1
vote
0answers
7 views

How to block bootstrap in stata with set of dummy variables as controls

I want to estimate a Multiple multivariate regression of the type $$y_1=a_1+b_1*x+c_1*\text{countrydummies}+e_1 \\ y_2=a_2+b_2*x+c_2*\text{countrydummies}+e_2 \\ ... \\ y_N=a_N+b_N*x+c_N*\text{...
0
votes
0answers
10 views

Deleting outliers based on diagnostic plots is not working as intended (regression model) [closed]

I am trying to remove three problematic outliers which are troubling me in several diagnostic plots. I have a logical rationale for removing them (bad data quality for those points). Here goes the ...
1
vote
0answers
19 views

Can I use random forest and other machine learning models for inference?

I think the answer for this question is yes, but I'm still wondering how to do this. Here's the thing, I have a dataset with several products, their characteristics and the price a customer paid for ...
0
votes
1answer
17 views

R: regression analysis between two numeric variables stratified by groups

This was initially posted on Stackoverflow but based on a suggestion, I am posting it here: https://stackoverflow.com/questions/62028326/r-regression-analysis-between-two-numeric-variables-stratified-...
0
votes
0answers
5 views

How do I combine the weights of two predictor in a regression model with GRNN?

I am trying to build an algorithm that uses GRNN for regression, a model based on the formula: My csv files are looks like: ...
0
votes
0answers
8 views

Best score in SVM

i am new to machine learning and i took the house price dataset from kaggle.com to learn and understand SVM. for regression the best score would be 0.0 and for classification the best score ...
0
votes
0answers
15 views

Which model is a better fit?

Suppose I have 2 linear model such that: Linear model 1: ...
4
votes
1answer
30 views

What is the difference between a linear regression with a dummy variable and two separate regressions for each group?

I am interested in the connection between a multiple linear regression including a dummy variable (0/1) and two separated regressions split up by this dummy variable, i.e. two distinct regressions for ...
0
votes
0answers
19 views

Average treatment effect , binary outcome and odds ratio

I am struggling with ATE. I have two treatment group in a cohort (binary 0=treatment A; 1=treatment B). I want to know the odds of variable Y (binary 0=no event; 1=event) b/w two treatment groups ...
0
votes
0answers
5 views

What goes wrong in linear regression if we assume a Naive Bayes model when features aren't necessarily independent?

In the notes I'm working through, it says that in low dimensional models, it's often the case that we cannot get away with assuming that features are uncorrelated/independent i.e. that we can't use ...
0
votes
0answers
9 views

After converting a categorical variable to dummies, can I use all dummies in my decision tree and random forest?

When we learn about statistical models, we learn that we shouldn't have dummy variables that are complementary. For instance, if we have only men and women in the dataset, once we have a dummy ...
0
votes
0answers
9 views

Optimization method for gam when using restricted maximum likelihood (REML)

Im trying to get an overview of how parameter estimation is done in mgcv when using the restricted maximum likelihood REML. For ...
1
vote
0answers
13 views

Comparing significance of variables between each other

I would like to determine the significance of my variables in my model and compare them with each other. I have 6 explanatory variables. Here is my summary of the model: ...
0
votes
0answers
7 views

joint probability density function for multinomial [closed]

Let X and Y have the joint probability density function f(x, y) = {k xy , , 0 ≤ x ≤ y ≤ 1 0, otherwise } Find the value of k and compute the correlation coefficient.
0
votes
1answer
12 views

Design a feature map to achieve a perfect fit with linear regression

Suppose we have a dataset $D=\{(x_1,y_1),\dots,(x_n,y_n)\}$ with $x_i \in \{0,1\}^3$ and $y_i \in \{0,1\}$ and all of the feature vectors $x_i$ are linearly independent. If we attempt linear ...
0
votes
1answer
14 views

cross validation using the SPSE in R

We have implemented an cross validation method in R. But we are not sure whether the error that it puts out is correct. We want it to give us the sum of predicted squared errors (SPSE), but in our ...
0
votes
2answers
26 views

Simulate data based on linear regression and R squared

I have a small data set of 10 x,y points from which I can derive a simple linear regression. I'm looking to use this data set as a basis to simulate / predict additional "y" points as I have a much ...
0
votes
0answers
10 views

Cross validation in R for the SPSE [closed]

We have implemented an cross validation method in R. But we are not sure whether the error that it puts out is correct. We want it to give us the sum of predicted squared errors (SPSE), but in our ...
5
votes
1answer
48 views

Asymptotic t test question- regression when you do not assume normality of errors

Say you are running a regression: $Y_i$= $X_i$$\beta$ + $\eta_i$ And we are not assuming normality of $\eta_i$. My understanding is that as long as your sample size is relatively large (and i know ...
1
vote
1answer
25 views

How to check if a correlation exists between a continuous independent and a binary dependent variable

So this has been a headache for me. None of the resources I found could explain it in a manner I understand. Having two sets of data: ...
2
votes
0answers
14 views

How to deal with treatment variable which is determined by the outcome variable

I have a categorical treatment variable, MessageType, that has 12 different values. The outcome variable, Crash, sometimes determines these values. So more crashes lead to certain types of messages, ...
0
votes
0answers
6 views

wi (𝒘і) output from regression - what is this? [closed]

I have run a mixed effect model and want to describe the output can someone please explain what the 𝒘і is? thanks!!
0
votes
0answers
13 views

How to get Prophet to perform linear regression? [closed]

I am attempting to force Prophet to perform linear regression on normally distributed time series data. The reason is that this issue still exists when I try to make forecasts using real-world data. ...
0
votes
0answers
5 views

Training loss fluctuating in Multivariate Linear regression pytorch

I am trying to perform linear regression on multiple input dataset. There are 8 features and one Label. The training loss seems to be fluctuating at the same range and the predicted results are not ...
0
votes
0answers
13 views

Trying to choose between a LM or a GLM (family=negative binomial)?

I have data on captures per year (n=13) and I want to analyze the relation with years (is it increasing, decreasing or staying the same with time). Total captures ~ year Because is number of ...
0
votes
0answers
14 views
1
vote
1answer
7 views

How to choose what feature vector to plot in multivariate regression analysis?

I'm new to the field of machine learning and I have been having this doubt for a long time now. If we want to plot a scatter plot, we plot it as, x as a function of y where x is a 1-D array. But in ...
1
vote
1answer
38 views

If $\epsilon_i \sim \mathcal{N}(0, \sigma^2)$, why does this also imply $x_i|\beta \sim \mathcal{N}(0, \sigma^2)$

I have seen this stated in multiple sources, where if the errors in a linear model ($y_i = \beta x_i + \epsilon_i$) follow $\epsilon_i \sim \mathcal{N}(0, \sigma^2)$, then $x_i|\beta \sim \mathcal{N}(...
0
votes
1answer
109 views

Logistic Regression Loss Function: Scikit Learn vs Glmnet

The loss function in sklearn is $$\min_{w,c}{\frac{1}{2}w^Tw+C\sum_{i=1}^N{\log(\exp(-y_i(X_i^Tw+c))+1)}}$$ Whereas the loss function in glmnet is $$\min_{\beta,\beta_0}{-\bigg[\frac{1}{N} \sum_{i=...
0
votes
0answers
9 views

Ridge regression on one predictor and the reduction of the risk of overfitting

In the book Hands-On Machine Learning with Scikit-Learn and TensorFlow, Chapter 1, the author stated that when doing Ridge Regression (for only one predictor), this regularization reduces the risk of ...
1
vote
0answers
21 views

Experiment design advice

I’d like to understand a good experiment design for the following example: Say sales people are trying to find the ideal time (in days) to send a follow up phone call to increase conversion (a sale). ...
0
votes
0answers
19 views

If x is correlated with y, is there formula to use that alone to write x as a function of y, or to relate the two variables?

for example, if $\text{corr}(x,y)\not=0$, can you always decompose $x$ to be: $x={\text{cov}(x,y)/\text{var}(x)} + \epsilon$ where $\epsilon$ is just all other remaining parts of $x$ uncorrelated with ...
0
votes
0answers
30 views

How to fit the intercept

I'm practising using R and I'd like to do this task: So I fitted the model: ...
1
vote
1answer
16 views

How to check if there is a linear relationship for a logistic regression model

From what I understand logistic regression expects that there is a linear relationship between the log odds of the target and the feature. Fourth, logistic regression assumes linearity of ...
1
vote
1answer
30 views

Reformulation of logistic regression

I am given the question above and can't seem to get the form that it's asked for. I have tried working it backwards from the goal which gives me: ...
0
votes
0answers
7 views

Is there within and between R^2 for pooled cross section?

I am writing a referee report for a paper that reports within and between $R^2$ for pooled cross-section regression that includes year fixed effects (but no panel or other fixed effects)so the ...
0
votes
1answer
8 views

When calculating the variance of a linear combination of least squares estimators, what is C?

I am reading DeGroot and suddenly "C" comes out of nowhere. I am not sure where he got C from and how to calculate it. The context is linear regressions and calculating the variance of the ...
1
vote
0answers
12 views

In typical regression, if Y | X is normal, is Y itself normal?

I am reading DeGroot and we made the assumption that Y | X is normal and each Y | X has the same variance. However, in deriving the sampling distributions of b0 and b1, he says that Y is normal. Do ...
1
vote
0answers
17 views

Is there a way to solve for $\sigma^2$ in linear regression only using least squares estimation?

If we only use least squares estimation, is there a way to solve for the variance of the conditional distribution of Y | X? Or do we have to use Maximum Likelihood Estimation? Additionally, do most ...
1
vote
0answers
27 views

Interpreting Odds Ratio (in an Interaction)

I need advice on the correct interpretation of an odds ratio of an interaction term. Both the mixed-effect logistic regression output is below as well as the predicted odds values, which I calculate ...
2
votes
1answer
15 views

Is it better to build N models for each category of data?

I'm new to data science and I'm working on a challenge with some friends, I have a data set of 80 feature and around 4000 rows. The data is split into 180 category (A,B,C,D...etc), at first I tried ...
0
votes
0answers
16 views

Rationale for taking second derivative in least squares estimation?

I am reading DeGroot and he talks about how to derive the b0 and b1 coefficients using LS estimation. I understand everything except the last part where he talks about taking the second derivative of ...
0
votes
0answers
2 views

Why does one part of the interaction term turn insignificant when the interaction term is added to the model?

Suppose I am trying to figure out whether a bigger shoe-size makes you happy. In my model, I also control for gender. In model 1, there is no interaction term. All coefficients are significant. The ...
0
votes
0answers
9 views

Using Gaussian Process Regression in scikit-learn

I have a simple dataset with multiple trials of position over time, and I'm trying to fit a Gaussian Process over it. Here's a plot of all the raw data (6180 data points): My goal is to fit a ...
0
votes
1answer
13 views

While creating dummy variables with n values why cant we just create a single variable?

For a variable that can take on n values, we create n-1 dummy variables, and that part is perfectly understood by me, but why cant we just create a single variable and load it with n unique values. ...
0
votes
1answer
16 views

Breusch Pagan and White test always 0

I'm working on a machine learning algorithm and trying to evaluate the model based on a guide online. No matter how I change the features of the model, the results of both tests are always 0. Is there ...
0
votes
0answers
12 views

Regularization for regression vs classification [closed]

I'm following the great course of stanford on ML, I was just wondering why the regularization term is different when using regression or classification. In regression we must add the term (lambda/m)*...
0
votes
1answer
17 views

Within group variation for categorical dependent variable

I need some help. I am trying to test within group variations but my situation seems a little bit more complex and I’m yet to find a statistical test that caters to this situation. Or perhaps I ...

1
2 3 4 5
423