Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
3
votes
1answer
194 views

Simple Regression: how to prove that adding an observation that exactly follows the regression line never decreases the magnitude of the correlation?

Suppose we fit by least square a regression line to $n$ pairs of $(x_i,y_i)$ observations, with $$\hat{y}_i = \hat{\beta}_0 + x_i \hat{\beta}_1$$ Now suppose we add a single observation $(x_{n+1}, y_{...
0
votes
0answers
6 views

Difference between two models in R programming language

I have a research about the effect of the predictors (A, B, C) on X (Note that the predictor A is a main predictor). I analyzed my data using mixed effect linear regression. The dependent variable is ...
0
votes
0answers
13 views

What model is appropriate for flow cytometry data with dependent variable as a percentage?

I have some flow cytometry data where my dependent variable is measured as a percentage (number of cells expressing marker/total number of cells measured in that sample). My independent variables are ...
1
vote
0answers
68 views

Should I use robust standard errors if I have ARCH effects?

Im estimating the carhart 4 factor model. Im testing for heteroskedasticity to see whether i need to use adjusted standard errors, but i am finding conflicted results. All but one test (ARCH) are ...
1
vote
1answer
23 views

Is the limit in probablity of an inverse matrix equal to the inverse of the limit in probability of the matrix?

Suppose $X_n$ is a random matrix, which converges in probability to a matrix of constants, $Y$. It seems intuitive that therefore $X_n^{-1} \xrightarrow{p} Y^{-1}$ - so the limit in probability of an ...
0
votes
0answers
10 views

Distribution of error terms in linear regression [duplicate]

In the assumptions of linear regression, it is mentioned that error terms should be normally distributed with mean 0 and standard deviation $\sigma$. Is it necessary for the error terms to be normally ...
0
votes
0answers
15 views

Is this Fitted vs Observed diagnostic plot strange?

I am running a linear regression model in R with generalized least squares gls() on my data to fix residuals with unequal variance. I seem to have achieved this; ...
2
votes
0answers
30 views

Confidence interval for difference between regression lines

Assume two linear regression models $$ Y_{b} = \beta_{0b} + x\beta_{1b} + \varepsilon_b \qquad \text{with} \qquad \varepsilon_b \sim N(0, \sigma_b^2) \\ Y_{r} = \beta_{0r} + x\beta_{1r} + \...
0
votes
1answer
26 views

Can we use Gradient Descent in the place of Ridge Regression in overfitting problem while doing linear regression problem?

What is the difference between Gradient Descent and Ridge regression? We use ridge regression for overfitting problem when the Mean Squared Error for test dataset is high. I think that we can use ...
3
votes
1answer
191 views

Weighted normal errors regression with censoring

I have some data which I would model via standard multiple regression except: There is censoring (left-censored, fixed but varying censoring points which are known) The errors are assumed independent ...
0
votes
0answers
45 views

Representer Theorem for Support Vector Regression

I would like to know what is the expression of the predictor function in terms of the Representer Theorem in the case of Support Vector Regression. For example, in the SVM binary classification case, ...
2
votes
2answers
1k views

Why does ARIMA not perform well?

I created a simple AR(1) process with a constant=1 and coefficient=0.5: ...
26
votes
4answers
10k views

Raw or orthogonal polynomial regression?

I want to regress a variable $y$ onto $x,x^2,\ldots,x^5$. Should I do this using raw or orthogonal polynomials? I looked at the question on the site that deals with these, but I don't really ...
3
votes
1answer
147 views

Daily Data Transfer Logs - Anomaly Detection

First time poster but I've lurked here quite a bit! I need a bit of guidance with regards to what approach I should use with the below problem: DATASET: 1 Master Log that records ~20 databases and ...
1
vote
0answers
37 views

Correcting linear regression for change in air temperature over time

Say I collect temperature data over 5 square kilometers over the course of an hour in the evening but I want to analyze the data as though they were collected all at the same time. Naturally the area ...
1
vote
1answer
32 views

Why does a Least squares distribution look like a parabola?

For regression analysis, one often uses the least squares method to minimize the quadratic differences between data and a model function f as follows: $$\chi^2=\sum_i \frac{(\text{data}\, _i-f_i(p))^2}...
2
votes
2answers
194 views

Can redundant/irrelevant features be called a Noise?

Let's say we want to predict job applicant' salary. We have a dataset with following features: {Age, Experience, Education, Astrological_Sign, Weather_Today} 5 features in total. In this set, ...
0
votes
0answers
24 views

How do I interpret a simple linear regression model when both dependent and independent variables are square root transformed? [duplicate]

Overview I built a simple linear regression model to understand if Universal Healthcare Index predicts suicides. My independent variable is Universal Healthcare Index (scale from 1 to 100). The ...
1
vote
1answer
20 views

Lewbel (1997)'s Higher Moments IV Approach for Multiplicative Model

I wonder how I can introduce Lewbel (1997)'s higher moments IV approach in a multiplicative / log-log model. Assume the following linear model: We know that e.g. $Y_t=5+1 X_{1t}+1 X_{2t}+1X_{3t}+\...
0
votes
1answer
145 views

Adjusting for age and gender in ANOVA

I am performing an ANOVA to compare the means of three groups. However, I need to adjust for the effects of age and gender in the ANOVA. I'm not quite sure how to go about it in R. Conventionally, I ...
6
votes
2answers
2k views

Setting max_depth greater than the number of features in a Random Forest

I was using random forest regression to predict the price of a house. There are only 3 features in data set. Initially when I had set max_depth=2 the result was ...
3
votes
1answer
137 views

Generalized least squares error estimation

First of all, I have to admit that I am not statistician so some of my nomenclature could not be very rigorous and maybe a bit confusing; pleas ask me to clarify if necessary. The Problem Let's say ...
2
votes
1answer
124 views

Environmental variable vector in NMDS

I am using Non-metric MultiDimensional Scaling (NMDS) on a Bray-Curtis dissimilarity matrix. Then, I am trying to link the resulting NMDS axes (let's say "components") to environmental ...
0
votes
0answers
7 views

R Library for Ordinal Regression with Split-Plot (Cluster?, Repeated Measures?) Structure

I have a data set with the following structure. The response is ordinal. There is an experimental factor (with two treatment levels, each treatment level applied to a different sample of subjects). ...
121
votes
8answers
63k views

Why L1 norm for sparse models

I am reading the books about linear regression. There are some sentences about the L1 and L2 norm. I know them, just don't understand why L1 norm for sparse models. Can someone use give a simple ...
1
vote
1answer
45 views

Econometrics meaning of structural versus regression model

I want to make sure my understanding is correct. Particularly in econometrics, when authors write down a model: $Y_i = \beta_0 + \beta_1 X_i + \epsilon$ Can I think of this as a 'structural model'- or ...
1
vote
0answers
39 views

Appropriate goodness-of-fit test for the negative binomial regression

I have used the following Pearson $χ2$ test and the deviance test to assess the negative binomial regression using R as ######################################### ...
0
votes
1answer
29 views

Including a weighting variable in a linear regression

I'm looking at how temperature affects length. My length variable is the mean length calculated for every year, it is derived from ~10,000 data points. Not every year had the same sampling effort (e.g....
0
votes
0answers
9 views

Regression analysis before an optimization problem with an unkown function

I have data consisting of invested hours in different training seminars $T_{1},T_{2},...,T_{50}$ and the performance on an exam $P_{exam}$. Currently I know how much 1 hour of a training seminar costs ...
4
votes
1answer
144 views

How to model a conditional probability without estimating joint PDF?

I've around 3e3 two-dimensional data points, x, d 1 1, 0.1 2 3, 0.1 3 2, 0.2 4 1, 0.5 range(x) = [-600, 600], range(d) = [0, 1] We are trying to model ...
0
votes
0answers
6 views

What is the conditional variance of sample predictor on population predictor?

I am using the book introduction to Statistical learning with applications in R chapter 3. I've been able to find the conditional expectations, as well as the unconditional variance, but I've read ...
0
votes
0answers
16 views

Fitting a multivariate linear regression with different residual variance for each outcome (using a mixed effects model in R)

In a small simulation, I am fitting a multivariate normal model to predict two outcomes Y1 and Y2, while also modelling the covariance between them. This can be done through a mixed effects model (...
0
votes
1answer
132 views

Lasso and its dual: rates of regularisations

Let us consider the following lasso estimator: $$ \hat{\beta}_{L} = \arg\min \, \frac{1}{n}\sum_{i}^{n}||y_{i} - \textbf{x}_{i}\beta||_{2}^{2} + \frac{\lambda_{n}}{n}\sum_{j=1}^{p}|\beta_{j}| $$ For ...
28
votes
5answers
47k views

Why would anyone use KNN for regression?

From what I understand, we can only build a regression function that lies within the interval of the training data. For example (only one of the panels is necessary): How would I predict into the ...
16
votes
4answers
6k views

Updating linear regression efficiently when adding observations and/or predictors in R

I would be interested in finding ways in R for efficiently updating a linear model when an observation or a predictor is added. biglm has an updating capability when adding observations, but my data ...
3
votes
1answer
122 views

How to generate data that have given conditional mean and conditional quantile using R?

Suppose I want to generate independent data $(y_{i},x_{i})$ such that the conditional mean of $y_{i}$ given $x_{i}$ is a quadratic function in $x_{i}$ and the $.25$ conditional quantile of $y_{i}$ ...
0
votes
0answers
15 views

Can a range of priors being used for a linear regression be applied to a logistic regression?

I have trial level data from a study in which participants responded to a series of stimuli. I have a predictor of interest. For the sake of this example, let's call it the size of the stimulus. There ...
10
votes
1answer
2k views

Hidden state models vs. stateless models for time series regression

This is a quite generic question: assume I want to build a model to predict the next observation based on the previous $N$ observations ($N$ can be a parameter to optimize experimentally). So we ...
0
votes
2answers
33 views

Assumption of linearity between variables and log odds in logistic regression

I know that in logistic regression we assume a linear relationship between the independent variables and the logits. Can you explain why is this a reasonable assumption?
13
votes
4answers
8k views

What does linear stand for in linear regression?

In R, if I write lm(a ~ b + c + b*c) would this still be a linear regression? How to do other kinds of regression in R? I would appreciate any recommendation ...
2
votes
1answer
141 views

How to achieve linear relationship between predictors and logit of outcome?

Prior to conducting a logistic regression of the 0/1 likelihood of a nest hatching or failing as a function of 9 continuous predictors, I plotted each of the standardized (mean = 0, SD = 1) predictors ...
0
votes
0answers
14 views

Unstandardized estimated variance from regression

I am running a multivariate regression where both y1 and y2 are both predicted by x. For ...
17
votes
1answer
13k views

Fitting models in R where coefficients are subject to linear restriction(s)?

How should I define a model formula in R, when one (or more) exact linear restrictions binding the coefficients is available. As an example, say that you know that b1 = 2*b0 in a simple linear ...
1
vote
2answers
75 views

Homoscedasticity and independence of errors

In linear regression I often see homoscedasticity and independence of errors listed as assumptions (for example on wikipedia). But I would think that independence of errors would imply ...
0
votes
1answer
47 views

How do I determine when a (non-independent) time series approaches a horizontal asymptote?

I have time series data with many data points per subject over time. I want to determine the marginal time interval within which my dependent variable (dv) falls within given "equivalence" ...
2
votes
1answer
24 views

Adding a Dummy Variable to glm in R?

I'm running a glm in R with two categorical variables, one of which is binary, the other of which can take on five values. I would like it so that my model returns an intercept value that reflects the ...
3
votes
2answers
1k views

Regression Lines With Same Intercept

So, I struggle with Regression a lot. I just found out how to get 2 lines with the same slope, but I cannot manage to get 2 lines with the same intercept. I read about ANCOVA a lot (because I thought ...
1
vote
1answer
31 views

Does logit transformation of information entropy values make sense?

I have a vector of information entropy values that range between 0 and 1 which I want to explain with some explanatory variables. I realized that the distribution of the entropy values in my dataset ...
0
votes
0answers
14 views

What does this output tell me about my regression coefficients?

I've run a cross-lagged panel model with two variables in Mplus. I've received the following output and I'm not sure how to interpret the residuals (these are standardised). Are these too high to make ...

1
3 4
5
6 7
468