Questions tagged [regression]
Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.
30,254
questions
0
votes
0
answers
2
views
Estimating various outputs using PCA from various features
I have a situation where I am trying to come up with a function to predict different max altitude (in meters) of 5 seconds of various balls thrown by various players with different features such as ...
- 1
0
votes
0
answers
4
views
Verifying consistency of HAC SEs with a Monte Carlo Experiment
I'm trying to demonstrate the consistency of the default HAC standard errors given in R's sandwhich package via a Monte Carlo experiment. I'm using a linear model ...
- 11
2
votes
0
answers
22
views
Weighted linear regression -- how to interpret when weighting matrix is not diagonal
The classical case of weighted least squares problem$$(X^TWX)\hat{\beta} = X^T Wy$$ is solving a problem of minimizing $${\operatorname{arg\ min}}\, \sum_{i=1}^{n} w_i \left|y_i - \sum_{j=1}^{m} X_{ij}...
- 121
0
votes
0
answers
7
views
How to interpret a partial correlation where the independent variable codes experimental group?
Let's say I have 3 variables: Experimental group (A/B), attitude and age. I put them all into a regression and get the partial correlation with experimental group on the Y axis and attitude on the X ...
- 221
0
votes
0
answers
30
views
When does a line no longer approximate an exponential curve sufficiently well?
I have a somewhat complicated question and as someone with no data analysis or strong statistical science background, I'm searching for advice/recommendations for the best data analysis "tool&...
1
vote
1
answer
28
views
Contradicting interpretations when including factor variables into GAM
the model I built is like this:
...
- 171
0
votes
0
answers
9
views
Compare parameters of same model regressed to different data sets
I am looking for guidance on approaching a problem I formulated as: how to assess if different data sets possess different regressing parameters, considering the same model.
The context is an ...
- 21
1
vote
0
answers
14
views
Plotting effects in Dirichlet regression
I performed different regression analyses, including Dirichlet regression, on compositional data.
I want to create a plot similar to the one obtained when you use the package ...
- 11
1
vote
0
answers
5
views
Added variable plot and CCPR plot for categorical variable
Do the added variable plot and the CCPR plot make sense for categorical variables? The significance of the variable can be obtained from the partial F-test, and non-linearity only applies to ...
- 21
0
votes
0
answers
14
views
Residuals changes with Predicted values range in LGBM regressor
I am doing a regression problem where the target variable ranges between 0.01 to 0.15. The model gives the best value when the predicted value is around 0.1. Plotting the residuals seems to show ...
- 1
2
votes
1
answer
54
views
MSE gets better but $R^2$ gets worse
Consider the following small dataset (around 569 data points), where Uptake is the regression target:
As you can see, most of the variables are skewed, with some of them having only 2 or 3 data ...
2
votes
3
answers
93
views
Low $R^2$ puzzle
I conducted an experiment and using the data I fitted a basic linear regression model. The top chart displays the fit. The $R^2$ is abysmally low indicating almost no explanatory power (adjusted $R^2$ ...
- 4,675
0
votes
0
answers
24
views
A Method for Functional Misspecification
I have been thinking about unbiasedness recently. An idea for came to my mind . Imagine that the dataset is separated into three different parts as training, the data that will be added to training ...
2
votes
0
answers
37
views
Standard deviation of predicted value of a regression equation
In a typical regression setup with residuals are independent, the standard deviation of the new value of response variable would have been
${\sigma} \sqrt{ \left(1 + a^T { \left(X^T X \right)}^{-1} a\...
- 41
0
votes
2
answers
47
views
Question about regression [duplicate]
Let's assume that I have large samples of data for the price and quantity of a product. I regress this data and I find that the correlation is one. Since the residuals in a linear regression are 0 ...
6
votes
2
answers
412
views
When to use negative binomial and Poisson regression
When would one use a negative binomial regression and when would one use Poisson regression with respect to the mean and variance?
1
vote
0
answers
22
views
Some further explanation of Alex Smola's 1998 implementation of support vector regression
I am currently going through, and trying to implement the pseudo-code in Alex Smola's 1998 paper on support vector regression, particularly the one on sequential minimal optimization. (Section 4.6.3, ...
- 11
3
votes
2
answers
203
views
How to perform correlation & regression analysis using both 5-point likert scale and "Yes/No/Maybe" questionnaire?
I have collected a data for my thesis to analyze the degree of awareness among the individual taxpayers. For that purpose, i have distributed the questionnaire that contains questions for IV (audit, ...
0
votes
0
answers
9
views
How to interpret conditional effects for a orthogonal polynomial contrast coded categorical factor?
I understand that when estimating an interaction, the interpretation of lower order coefficients changes-- In the presence of an interaction, lower order coefficients are interpreted as conditional ...
- 1,113
1
vote
1
answer
27
views
What is in-sample vs out-of-sample in a multiple linear regression?
I was just thinking about what would be considered interpolation vs extrapolation for multiple linear regression, and realised I'm not sure exactly how it would be defined, nor could I find an answer ...
- 521
0
votes
1
answer
48
views
Estimation of model coefficients of ARIMA model
Let say I have below ARIMA model estimation in R
...
- 375
3
votes
2
answers
117
views
Deriving MSE($\hat{\beta}$) under Linear regression
I was able to derive the MSE, but there's a part of the derivation which I don't really get. Here's what I got:
Facts:
$\mathbb{E}(\hat{\beta})=\hat{\beta}\space$ (unbiased estimator)
$\text{Cov}(\...
- 33
1
vote
1
answer
29
views
Models with Uncertainty in Y
Let's say I have data with predictors x1, x2, x3...xn for a variable y. I have essentially imputed y using a Bayesian analysis, which means I have a posterior distribution for each value of y. To ...
- 107
1
vote
1
answer
46
views
Statistical Error in Simple Linear Regression
I want to start off this question by saying that I'm looking for more of a conceptual understanding of this term in a simple regression model, not a mathematical one.
In econometrics, simple linear ...
- 21
2
votes
2
answers
37
views
Does ceiling effect of outcome variable violate linearity assumption of linear regression
If there is a ceiling effect in the outcome variable, e.g. in my case the outcome variable is limited to a certain value and 25% of data points have that highest possible value, does this mean that ...
- 63
0
votes
0
answers
16
views
Change score model formulation and contrasts - simultaneous inference
Suppose we would like to compare pre- and post- intervention scores for several biomarkers (please note I am aware of the change from baseline vs- ANCOVA debate but my question is not about that).
Let ...
- 869
0
votes
0
answers
16
views
Unsure if spatial Autocorrelation is present or not
I built a classic linear model with 1000 data points for which I have coordinates. I thus checked for the presence of spatial autocorrelation with a bubble plot of the residuals and with Moran's I ...
- 1
8
votes
1
answer
420
views
Regression techniques for a “triangular” scatterplot
I am doing a regression analysis of environmental data, and I encounter some rather specific relationships between my predictors and the response variable. I am doubtful that a simple linear ...
0
votes
0
answers
16
views
Standard Errors of 2000 [closed]
I am currently doing a regression analysis, where the standard errors are around 2000, while the coefficient estimates are around 1.3-17.0. The sample size is 101 and the regression analysis is a ...
- 1
1
vote
0
answers
19
views
How can I model with random effect on the coefficient [closed]
The simple linear regression model is : $$y= a x + b + e $$
However, in real data:
there is random effect (error or variation);
when $x$ increases, the abs of random effect increase proportionally.
...
- 11
0
votes
0
answers
35
views
Categories automatically dropped in SPSS for my logistic regression [duplicate]
I have a problem with my logistic regression model. I use SPSS for analyzing the relationship between some categorical IV/2 numerical variables and customer churn (the well-known IBM Telco dataset). I ...
- 1
0
votes
0
answers
32
views
Fitting a regression line which passes through the anchor point
In our setting, we have data $X_1, \ldots, X_n$, which can be ordered as $X_{1,n}\leq X_{2,n}\leq \ldots \leq X_{n,n}$ and we have the points $(-\log (1-\frac{i}{n+1}), X_{i,n})$ for $i=1,\ldots,n$.
...
- 656
0
votes
0
answers
18
views
How can I get the main effects of a GEE model and pool across imputed datasets R
With mice I imputed 5 datasets, and stored these as a list in data_list. I want to predict outcomes stored in ...
Noa
3
votes
1
answer
283
views
What I have to do more to improve my regression model in r [closed]
I want to make beverage sales predicting model. I am doing regression analysis. All the column types are integer. The dimensions of the data are 15375 rows x 400 columns. The dependent variable $y$ is ...
- 33
0
votes
0
answers
17
views
Setting predictor variables with 3-levels in multilevel mode
I am working with a random intercept multilevel modeling.
I want to predict general health based on survey data. The survey uses nested data set on three levels: individual, county, and state.
I am ...
- 123
4
votes
2
answers
55
views
What does calibration mean when the outcome is not categorical?
In a situation where a binary variable is of interest and we want to predict the probability of either event (dog vs cat, say), it is common to talk about the calibration of the predictions, if the ...
- 65.6k
1
vote
1
answer
46
views
Should I use ordinal regression or multinomial regression for these variables?
I am very new to R Studio and statistics so I need some help. My dependent variable is a welfare scale with values
1 = Pro-welfare,
2 = Neither,
3 = Right wing
and my independent variables include a ...
Charlotte
1
vote
1
answer
22
views
Experimental condition with multilevel model
I am working with a survey experiment. The data is set at three levels: individual, county, and state.
The experimental condition was randomized at the individual level 1. That is, some individuals in ...
- 123
0
votes
0
answers
58
views
non-negative constraints and interactions in the ensemble model
In the context of prediction problems using regression models, suppose I have $K$ different models all trained (fitted) on the same targets (observations). These models are different - low correlation ...
- 1
0
votes
1
answer
51
views
Does including a variable for stepwise change and another variable for linear change in an ARIMAX model introduce issues of multicollinearity?
I am trying to perform interrupted time series analysis using an ARIMA model based on the following paper: Interrupted time series analysis using autoregressive integrated moving average (ARIMA) ...
3
votes
1
answer
62
views
Struggling to prove that the Hessian is PSD for simple linear regression least squares method
I am looking at the simplest case of minimizing the following over $b_0$ and $b_1$:
$$(y_1 - [b_0 + b_1 x_1])^2 + \ldots + (y_n - [b_0 + b_1 x_n])^2$$
To establish that this function is convex in $b_0$...
9
votes
3
answers
505
views
Can I retain the ordinal nature of a predictor while answering a question about it that is inherently binary?
As part of a collaboration, I've been asked to fit a model with a continuous response $Y$ and an ordinal predictor $X$ (levels 1 to 5). The dataset owner is after an answer that is inherently binary: ...
- 19.7k
1
vote
0
answers
25
views
Is data set enough for regression analysis? [closed]
I need to conduct regression analysis for my thesis (panel data), however, my data set turned out to be really small. I have only 6 companies to study (annual data for 11 years) and the sector i am ...
- 11
2
votes
1
answer
30
views
Analyzing the effect of satisfaction on transport mode preference using mixed logistic regression in R
I want to analyze the effect of satisfaction with different modes of transport (bus vs. train) on a specific route on stated preference. participants drove a specific route repeatedly with either bus ...
- 21
7
votes
2
answers
550
views
R Squared Causal Inference
I'm trying to know whether a low R-squared value would pose a problem when assessing the coefficients. My population is divided into two groups (A and B), and I want to assess if there's a significant ...
- 351
1
vote
1
answer
31
views
Should I be using a lower resolution in my LMM if it dosen't match my predictor variables?
I have a set of animal movement values which are sampled at three hours (e.g. 500m/3h)
I ran a Linear mixed model with the movement distance as the response variable, using sex, daytime (night and day)...
- 35
1
vote
1
answer
47
views
Potential Sign Issues in a Composite Performance Metric for Model Selection
I am analyzing the results of various machine learning models for a regression task, using four metrics: RMSE, MAE, MAPE, and $R^2$. My approach involves two types of analyses:
Individual Metric ...
- 13
0
votes
1
answer
27
views
interactions in non-negative least squares
In the context of regressions models, I have some features for which I have a prior view on the coefficient sign while others may not be useful on their own (expect small or zero coefficient and no ...
- 1
0
votes
1
answer
25
views
Standardized regression coefficient
I was wondering the following: Does the standardized coefficient on X1 in a regression of Y on X1, X2, ...,XN go to one, as the bivariate correlation of Y and X1 goes to one. If so, why?
Or is it ...
- 21
0
votes
1
answer
29
views
What scale do I need to consider when calculating a body condition index?
I'm trying to calculate a body condition index (scaled mass index [SMI]; Peig and Green 2009) for groups of frogs and I'm having trouble understanding what scale I need to consider when calculating a ...
- 3