Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
-2
votes
1answer
22 views

I am a beginning researcher. I need a textbook or any reading material for step by step research and data analysis [closed]

I want to learn the researching methods and data analysis step by step starting from scratch. starting from the basis statistics to the advance level that includes: Regression, Simple and Multiple. ...
1
vote
0answers
6 views

Regression methods for indicator function as covariate

I am looking for a regression method to fit the following model: $$Y=\beta_0 + \beta_1X+\beta_2 I(X>\beta_3)X + \varepsilon,$$ where $\varepsilon \sim N(0, \sigma^2)$, and $I$ is the indicator ...
0
votes
0answers
7 views

Out of distribution prediction, not sample

My dataset contains samples that are recorded time signals. I want to predict the start of some event in the time signal. Say I train a neural network with mean squared error as loss function to ...
1
vote
0answers
10 views

What does it mean to have a model fit via GLS with REML? Aren't GLS and REML two different methods of estimation?

As in the title. I am confused. We often read that a regression model was fit using the OLS, GLS, TLS or ML. But recently I found a text about the analysis of repeated data, where it was modelled ...
1
vote
1answer
982 views

Does the delta output from a k-fold cross validation indicate the estimated classification error?

I'm currently working on a logistic regression analysis and want to determine if my model validates well. I used the following R code using the "boot" package: ...
1
vote
1answer
904 views

Averaging data then fitting vs fitting then averaging in non-linear regression?

I have a very similar problem like in this question. The difference is that I am dealing with non-linear regression. Moreover, the answer to that question suggests that there should be no difference ...
0
votes
1answer
35 views
+50

Converting Categorical Data to Numerical by Sampling

Suppose I had sampled $1000$ individuals from a population in order to learn about two different questions, both of which had categorical, binary answers. (For the sake of this hypothetical, let's say ...
1
vote
0answers
949 views

Insignificant coefficients in Logistic Regression after LASSO variable selection

I am trying to use the LASSO technique to identify which variables to include in my model. I used cross validation to identify the value of lambda which minimizes the CV error. For this minimal value ...
3
votes
1answer
255 views

Nonparametric/nonlinear regression

I am looking for a survey/book on some state-of-the-art non-parametric (or nonlinear regression) methods, preferably with an inclination towards sequential data. Till date I have used ...
1
vote
1answer
375 views

Linear regression with depended predictor variables

Short story: Why is it important that the predictor variables of a linear regression model are independent? If I am not interested in the coefficient but only in the question, which predictor variable ...
0
votes
0answers
9 views

Unique Variance Explained by Regressors [closed]

I have a question that needs a bit of help: "The pearson correlation between x1 and y is r=0.4 and SSY=100. When a second variable, x2, will be added to the regression model, we get R^2=0.25. How ...
0
votes
1answer
15 views

Independent Variable effect's change Between Two Periods

I used the Cox proportional-hazards model, which investigates the relationship between the time of occurrence of an event and a set of explanatory variables in the presence of censored data. The goal ...
3
votes
2answers
2k views
+50

Testing predictive power of a set of features

This is perhaps a typical setup in Bioinformatics: we need to build a model to predict a dependent biological variable (say $y$), given a large set of (usually genomic) features $X$ (which might not ...
3
votes
1answer
190 views

Simple Regression: how to prove that adding an observation that exactly follows the regression line never decreases the magnitude of the correlation?

Suppose we fit by least square a regression line to $n$ pairs of $(x_i,y_i)$ observations, with $$\hat{y}_i = \hat{\beta}_0 + x_i \hat{\beta}_1$$ Now suppose we add a single observation $(x_{n+1}, y_{...
0
votes
0answers
6 views

Difference between two models in R programming language

I have a research about the effect of the predictors (A, B, C) on X (Note that the predictor A is a main predictor). I analyzed my data using mixed effect linear regression. The dependent variable is ...
0
votes
0answers
12 views

What model is appropriate for flow cytometry data with dependent variable as a percentage?

I have some flow cytometry data where my dependent variable is measured as a percentage (number of cells expressing marker/total number of cells measured in that sample). My independent variables are ...
1
vote
0answers
67 views

Should I use robust standard errors if I have ARCH effects?

Im estimating the carhart 4 factor model. Im testing for heteroskedasticity to see whether i need to use adjusted standard errors, but i am finding conflicted results. All but one test (ARCH) are ...
1
vote
1answer
18 views

Is the limit in probablity of an inverse matrix equal to the inverse of the limit in probability of the matrix?

Suppose $X_n$ is a random matrix, which converges in probability to a matrix of constants, $Y$. It seems intuitive that therefore $X_n^{-1} \xrightarrow{p} Y^{-1}$ - so the limit in probability of an ...
0
votes
0answers
10 views

Distribution of error terms in linear regression [duplicate]

In the assumptions of linear regression, it is mentioned that error terms should be normally distributed with mean 0 and standard deviation $\sigma$. Is it necessary for the error terms to be normally ...
0
votes
0answers
14 views

Is this Fitted vs Observed diagnostic plot strange?

I am running a linear regression model in R with generalized least squares gls() on my data to fix residuals with unequal variance. I seem to have achieved this; ...
2
votes
0answers
22 views

Confidence interval for difference between regression lines

Assume two linear regression models $$ Y_{b} = \beta_{0b} + x\beta_{1b} + \varepsilon_b \qquad \text{with} \qquad \varepsilon_b \sim N(0, \sigma_b^2) \\ Y_{r} = \beta_{0r} + x\beta_{1r} + \...
4
votes
2answers
177 views
+100

What core topics would all statisticians be required to know?

I would like to know what topics are considered 'core knowledge' for a statistician. Please keep in mind I know very little about statistics. At my university, I hear statistics students discuss ...
0
votes
1answer
23 views

Can we use Gradient Descent in the place of Ridge Regression in overfitting problem while doing linear regression problem?

What is the difference between Gradient Descent and Ridge regression? We use ridge regression for overfitting problem when the Mean Squared Error for test dataset is high. I think that we can use ...
0
votes
0answers
4 views

What happens if the weights are plot?

Binary Regression for Images I'm not very good at coding, so couldn't really test much this hypothesis. But if weights are a sort of "measure of importance" of a particular pixel, then if ...
3
votes
1answer
190 views

Weighted normal errors regression with censoring

I have some data which I would model via standard multiple regression except: There is censoring (left-censored, fixed but varying censoring points which are known) The errors are assumed independent ...
0
votes
0answers
30 views

Representer Theorem for Support Vector Regression

I would like to know what is the expression of the predictor function in terms of the Representer Theorem in the case of Support Vector Regression. For example, in the SVM binary classification case, ...
0
votes
0answers
7 views

Multi Target Techniques where Dependent Variables are Correlated

I have browser data that contains over 100 independent variables to predict customer spend. Instead of predicting total spend over a given time, let's say we want to predict the monthly spend for each ...
2
votes
2answers
1k views

Why does ARIMA not perform well?

I created a simple AR(1) process with a constant=1 and coefficient=0.5: ...
26
votes
4answers
9k views

Raw or orthogonal polynomial regression?

I want to regress a variable $y$ onto $x,x^2,\ldots,x^5$. Should I do this using raw or orthogonal polynomials? I looked at the question on the site that deals with these, but I don't really ...
3
votes
1answer
147 views

Daily Data Transfer Logs - Anomaly Detection

First time poster but I've lurked here quite a bit! I need a bit of guidance with regards to what approach I should use with the below problem: DATASET: 1 Master Log that records ~20 databases and ...
1
vote
0answers
17 views

Chaining/combining logistic and linear models

I have a analysis that is looking to predict the total customer value based on a customer's first purchase amount. I am noticing that a set of features predict whether the customer will purchase ever ...
1
vote
0answers
34 views

Correcting linear regression for change in air temperature over time

Say I collect temperature data over 5 square kilometers over the course of an hour in the evening but I want to analyze the data as though they were collected all at the same time. Naturally the area ...
0
votes
0answers
28 views

What does negative RMSE value mean? [closed]

I am using the this dataset. While training a ElasticNet. regression model I am getting negative RMSE value.I calculated the corresponding r2 score it almost had 99.98% indicating overfitting. What ...
1
vote
1answer
31 views

Why does a Least squares distribution look like a parabola?

For regression analysis, one often uses the least squares method to minimize the quadratic differences between data and a model function f as follows: $$\chi^2=\sum_i \frac{(\text{data}\, _i-f_i(p))^2}...
0
votes
0answers
8 views

How can i increase the r2 value on validation data? [closed]

I'm having a problem finding a model for my regression problem, I've tried various models with no success. I'm using 5 fold cross validation and optimizing for the r2 metric, but I get results similar ...
0
votes
1answer
30 views

Beta regression fitted values

I have a beta regression model in R, have generated predicted (fitted) values based on my data, and plotted lines of those fitted values on a scatter plot of the actual data. I'm most used to GLMMs, ...
2
votes
2answers
191 views

Can redundant/irrelevant features be called a Noise?

Let's say we want to predict job applicant' salary. We have a dataset with following features: {Age, Experience, Education, Astrological_Sign, Weather_Today} 5 features in total. In this set, ...
0
votes
0answers
23 views

How do I interpret a simple linear regression model when both dependent and independent variables are square root transformed? [duplicate]

Overview I built a simple linear regression model to understand if Universal Healthcare Index predicts suicides. My independent variable is Universal Healthcare Index (scale from 1 to 100). The ...
1
vote
1answer
20 views

Lewbel (1997)'s Higher Moments IV Approach for Multiplicative Model

I wonder how I can introduce Lewbel (1997)'s higher moments IV approach in a multiplicative / log-log model. Assume the following linear model: We know that e.g. $Y_t=5+1 X_{1t}+1 X_{2t}+1X_{3t}+\...
0
votes
0answers
29 views

Which regression method causes the ordered coefficients to decrease quickly in absolute value? [closed]

Which regression method causes the ordered coefficients to decrease quickly in absolute value? Lasso lets the coefficients shrink to zero, but I don't know if that has to be in the order of the ...
0
votes
1answer
129 views

Adjusting for age and gender in ANOVA

I am performing an ANOVA to compare the means of three groups. However, I need to adjust for the effects of age and gender in the ANOVA. I'm not quite sure how to go about it in R. Conventionally, I ...
0
votes
0answers
15 views

Choosing the most appropriate method for different data sets [closed]

I have five datasets with the following characteristics: Data Set 1: Many small but comparable coefficients, but no zeros (n=100, p=90) Data Set 2: Most of the coefficients are zeros (n=100, p=2001) ...
6
votes
2answers
2k views

Setting max_depth greater than the number of features in a Random Forest

I was using random forest regression to predict the price of a house. There are only 3 features in data set. Initially when I had set max_depth=2 the result was ...
0
votes
0answers
14 views

Constraining regression coefficient to non-zero

I have a regression problem where I don't want the coefficients to be negative. Is setting negative coefficients of OLS to zero the same as constraining the coefficient to be non-zero and solving it ...
3
votes
1answer
137 views

Generalized least squares error estimation

First of all, I have to admit that I am not statistician so some of my nomenclature could not be very rigorous and maybe a bit confusing; pleas ask me to clarify if necessary. The Problem Let's say ...
2
votes
1answer
120 views

Environmental variable vector in NMDS

I am using Non-metric MultiDimensional Scaling (NMDS) on a Bray-Curtis dissimilarity matrix. Then, I am trying to link the resulting NMDS axes (let's say "components") to environmental ...
0
votes
0answers
6 views

R Library for Ordinal Regression with Split-Plot (Cluster?, Repeated Measures?) Structure

I have a data set with the following structure. The response is ordinal. There is an experimental factor (with two treatment levels, each treatment level applied to a different sample of subjects). ...
0
votes
0answers
11 views

Diagnostics of a parametric Survival Regression in R

I am doing a Survival Analysis in R with the "survival" package and I don't know how to do any plots of the results for diagnostic purposes. Here is my model (I have given the variables self-...