Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Linear regression with multiple observations for each input point: WLS approach

Consider a dataset $X, y$ where each $y$ is distinct, but many of the rows of $X$ are repeated. Suppose that there are indices $\{ i_j: j = 1 \dots J \}$ such that the consecutive rows $X_{i_j}=X_{...
user avatar
0 votes
0 answers
3 views

What is the scaling applied to Helmert contrasts in emmeans?

I'm trying to understand why the values under 'estimate' from an emmeans contrast function differ from those of the default 'Estimate' values from, say, 'summary.lm()' in R. As an example, let's use ...
user avatar
  • 101
0 votes
0 answers
8 views

Choose between residual sum of squares (RSS) and comfounded RSS?

In every course I have taken, I was taught to use the residual sum of squares as (part of) the loss function in regressions, either in simple OLS, lasso or other linear regression methods. Recently I ...
user avatar
0 votes
0 answers
16 views
+50

Choosing response and predictors

Information on the Girth (), Height () and Volume () for 31 cherry trees yielded the following correlations : $r_{12}$ = 0.519, $r_{23}$= 0.598 and $r_{13}$= 0.967. If you were asked to setup a ...
user avatar
  • 83
0 votes
1 answer
15 views

The correlation coefficient with a priori intercept and slope [closed]

What does the formula for the product moment correlation coefficient look like if the intercept and the slope are given a priori?
user avatar
  • 1
1 vote
0 answers
16 views

Combining Biological Replicates

I am doing an experiment where I track the behavior of larval zebrafish of 3 genotypes during a photomotor assay. Due to constraints with our tracking system, I can only run between 24-48 fish at a ...
user avatar
  • 11
2 votes
0 answers
27 views

A misspecification error with linear models that can complete reverse the direction of an effect, has this been described, has this a name?

Linear models are ubiquitous in economic, social, health and nutritional sciences and the starting point for much research and many articles. However, there is a problem with linear models. When the ...
user avatar
3 votes
1 answer
26 views

What is this one-hot-like encoding of ordinal variables called and is it feasible?

Common understanding seems to be that when we're doing machine learning (classification or regression) using linear models, there are basically two ways to encode ordinal variables: Ordinal scale: 1, ...
user avatar
  • 570
0 votes
0 answers
32 views

What could cause regression linear models to predict exactly the mean of train set while random forests perform worse?

Data set: I'm working on a linear regression problem where my train set $X$ is of shape $(703 557, 53)$. Each row is a client's features, which could be its age, its gender, how many calls we received ...
user avatar
  • 155
2 votes
0 answers
42 views

Is there a fundamental mathematical reason that ordered factors are represented as orthogonal polynomials in linear regression?

At least for R, Chambers/Hastie write in their book "Statistical Models in S" in chapter 2.3.2 "Coding Factors by Contrasts": Ordered factors are coded so that individual ...
user avatar
  • 139
3 votes
1 answer
30 views

Extreme Heteroskedasticity - Multiplicative Model - Strange residuals

I absolutely need your help with my research. When I checked for heteroskedasticity I obtained a weird result from the white test (p value = 0). When I plot the residuals, these are the results: ...
user avatar
  • 31
0 votes
0 answers
20 views

Fitting models to test revenue performance

I have a dataset containing different survey models and their participant score. When a participant scores, we have a value 1 and 0 otherwise. Therefore, we have a binary classification for each model....
user avatar
0 votes
0 answers
30 views

Using multiple mean tests instead of linear model

First, I would like to apologize because I'm a beginner in statistics and I'm surely confused on some points. This is one of my first statistic work doing for university. I would like to explain a ...
user avatar
  • 11
1 vote
0 answers
24 views

Assessing a single predictor with linear modelling [closed]

I have data that involves countries in and out of Europe, life expectancy, social support and expenditure per GDP. I want to assess how successful a single country in terms of life expectancy given ...
user avatar
  • 113
3 votes
1 answer
45 views

When is least squares better than reduced major axis?

Consider two linear regression methods: least squares regression (LSR) reduced major axis (RMA) I know the definitions of both regression methods but I would like to know when is the LSR better than ...
user avatar
0 votes
2 answers
28 views

why the p value changes when I run a linear mixed model instead of a simple regression? and how random effects affect the output?

I ran two models, a linear regression model and a linear mixed model, I did this because I was suspecting that there were some levels or hierarchy in my data, specifically in my subjects and ...
user avatar
3 votes
1 answer
72 views

How to deal with crossed and nested factors at the same time in a linear mixed model?

I've recently started analysing the data for a project using linear mixed models but am not sure how to deal with crossed and nested factors at the same time. In my study, each participant reported ...
user avatar
1 vote
3 answers
86 views

Regression coefficients do not match conditional means

In a nutshell, I want the regression coefficients of a model to match several differences in conditional means. You can download the data from this repo. I have a data set that has a dependent ...
user avatar
0 votes
0 answers
13 views

variance of Y based on a simple linear regression of X in which the slope and intercept are not constants

Consider a linear regression $y = aX + b$, where mean(a) = 5, SE(a) = 0.5; and mean(b) = 3, SE(b) = 0.1. When a and b are constants, $Var(Y) = a^2 Var(X)$. Do the SE's make a and b non-constants? If ...
user avatar
  • 51
1 vote
0 answers
50 views

Difference between y against x, x^2 and sqrt(y) against x in a linear model

Suppose that we have a response variable y which is known to have a quadratic relationship with a predictor variable x. What are the differences between fitting a linear model of y against x and x^2, ...
user avatar
0 votes
0 answers
20 views

build a linear regressor with labels in different scales

I just ran into this linear regression problem where the labels are in entirely different range for example for 25% of the samples, the labels are in [0.001,0.01], then for another 25 % of the samples,...
user avatar
  • 1,586
0 votes
0 answers
9 views

EDA with Data Modeling

After I read R for DataScience and ggplot2: elegant graphics for data analysis, I am learning how use modeling techniques to improve my EDA. I applied this on two notebooks (https://www.kaggle.com/...
user avatar
0 votes
0 answers
12 views

Out of Sample Regression errors

I am trying to compute $\text{R}^2$ and $ {delta RMSE} $ from an Out of Sample Linear Model in R. $ e _{ N }$ is the vector of rolling OOS errors from the historical mean model $ e_{A}$ is the vector ...
user avatar
  • 1
0 votes
1 answer
26 views

Interpretation problems of linear model with no predictors

Let $Y=X\beta$, where $X=(\begin{matrix}1 & 1 & 1\end{matrix})^T$, $Y=(\begin{matrix}6 & 5 & 4\end{matrix})^T$ and $\beta=(\begin{matrix}\beta_0 \end{matrix})$. Now $X\beta=(\begin{...
user avatar
  • 125
2 votes
1 answer
51 views

Regressing on nuisance features as pre-processing

A friend recently told me about a technique to remove the effects of an unwanted feature $x$ in a response variable $y$. He mentioned an example from genetics. By regressing $y$ on $x$ (or a ...
user avatar
3 votes
0 answers
49 views

A faster way of finding unbiased estimators for this linear model

No access to computers or calculators is available for this problem. Consider the following linear model $$Y_1 =\theta_1 + \theta_2 + \theta_3 + \theta_4 + \theta_5 + \theta_6 + \epsilon_1\\ Y_2 =\...
user avatar
  • 155
0 votes
0 answers
15 views

How do I obtain or calculate the value of theta in an Ornstein-Uhlenbeck PGLS model?

I'm using phylolm in R to run a phylogenetic least squares regression with an Ornstein-Uhlenbeck model of evolution. The model summary gives me the values for alpha and sigma squared but not theta (...
user avatar
  • 143
1 vote
1 answer
87 views

Categorical linear-model coefficients from a pairwise competition experiment

I am presenting a question for which there may be a simple statistical answer, but I have prefaced it with perhaps a longer explanation, to err on the side of caution, in hopes that the data make more ...
user avatar
3 votes
1 answer
51 views

Re-using computations in several least squares problems

I have $K$ least squares problems of the form $Y_k = X_k\beta_k$ for $k = 1, \dots, n$. If the matrix $X_k$ is the same for each index $k$, we can rewrite the problem as $Y_k = X \beta_k$. How can I ...
user avatar
0 votes
0 answers
17 views

Have my assumptions been met?

I am running assumptions for multiple regression and scatterplots are a real bane of mine. Can anyone advise as to whether the following scatterplot provides a linear or non linear relationship ...
user avatar
0 votes
0 answers
18 views

lme4/lmerTest nested mixed effect model output missing fixed effect with consistently significant penultimate t-value

I have a mixed effect model with the following fixed effect categories: Error: Y/N Groups: A/B And a random effect for each subject. I am interested in the difference in response variables due error ...
user avatar
1 vote
1 answer
30 views

How to calculate residuals from a given linear model

So I have been given a linear model, with the Beta0 = 0 as well as given equations. I have done several calculations needed in order to calculate residuals, as shown in my work below: I also went ...
user avatar
  • 65
1 vote
1 answer
27 views

Binary variable in regression model (interview question)

I got this question, it may be easy but for some reason for me is not: you are given a binary variable $b\in [0,1]$, that has no predictive power on $y$, but has some on $X$. How would you use it to ...
user avatar
  • 155
1 vote
0 answers
11 views

How to derive least squares estimators from normal equations with work shown

My last question was closed, so I made a new one that is more relevant and has my full calculations thus far. I have a scenario where I have been given some information, such as normal equations from ...
user avatar
  • 65
0 votes
0 answers
4 views

Using a spatial instrument in IV regression under spatial heterogeneity

I am wondering what the implications are of using a spatial (or group-level) instrument on identification of a coefficient in a standard linear model. For convenience sake assume: $Y = \alpha + X\beta ...
user avatar
1 vote
2 answers
88 views

How to obtain least squares when $X^TX$ cannot be inverted

This work is all theoretical and for school, so we were only provided this information to work with, no actual y values. I have a simple linear model I have been asked to translate into a matrix, ...
user avatar
  • 65
2 votes
1 answer
43 views

How can I include multiple random effects in my linear mixed effects model?

I am trying to apply a linear mixed effects model using the R package 'lme4'. I am wondering how I can incorporate two random effects in my model rather than just one. I have a dataset where I am ...
user avatar
1 vote
0 answers
28 views

Linear Regression: Minimizing proposed changes to the response variable with a goal of statistical insignificance?

I am performing a linear regression for a pay equity analysis. That regression will typically look something like this (where yearsOfExperience is a stand-in for one or many controls that may be ...
user avatar
  • 11
0 votes
0 answers
11 views

Could a Pearson's Correlation Coefficient of 0 (or close to it) suggest the presence of quadratic regression or another type of regression?

If you plot data, and receive a R value of 0, or close to it, could it mean that the model follows quadratic regression instead of linear regression? This what Im getting at, see the two graphs below ...
user avatar
  • 11
0 votes
0 answers
27 views

How to interpret this problem question?

My course problem booklet (mathematics BSc, second-year module in statistical inference and modelling, unpublished) has a question, A runner completes two laps of a course. He is timed over each lap ...
user avatar
  • 265
2 votes
1 answer
40 views

Does bagging work for OLS to improve prediction?

In Elements of Statistical Learning, section 8.7, the author states that The bagged estimate will differ from original estimate only when the latter is a nonlinear or adaptive function of the data I ...
user avatar
  • 101
2 votes
1 answer
70 views

Logistic Regression applied to biased dataset

I have collected a binary classification dataset in a somewhat biased way: I have thousands of unlabeled samples. A small percentage of these samples belong to the positive class. I know for a fact ...
user avatar
  • 23
1 vote
0 answers
51 views

Least square estimate expected value and variance of linear model

I am practice some exercises. Here it goes. "Assume we fit the simple model \begin{equation} \hskip 5cmy=X_1\beta_1+\epsilon \hskip 5cm (1) \end{equation} however the true model is \begin{...
user avatar
1 vote
2 answers
61 views

Interpreting Interaction Variables With Numerous Variables

I have a model that has the following results: Is it possible for me to interpret a white man who received Degree1 and went into Profession1 as having a wage as $15,230.05 ($12,917.22 + $357.53 + $2,...
user avatar
1 vote
2 answers
54 views

Why does removing intercept not change predicition of linear model in the precence of factor predictors? [duplicate]

In a linear model that predicts birth rate (TFR) per country from per capita GDP, the country is encoded in "treatment coding", and there are several measurements (different years) per ...
user avatar
  • 2,956
0 votes
1 answer
20 views

Correlated linear survival model

I'm trying to model a survival situation like the following: Say I have 100 data points, and each will survive some number of days before dying at some point. In my problem I know that over a span of ...
user avatar
1 vote
1 answer
30 views

correlation range between y and two variables x1 and x2

What is the correlation coefficient (or range) $corr(y,\hat{y})$ for the regression $\hat{y}=ax_1+bx_2+c$ given that the correlation between $x_1$ and $y$ is 0.5 and the correlation between $x_2$ and $...
user avatar
  • 11
0 votes
1 answer
49 views

Is there an equivalent for an interaction term in linear models for non-linear regressions?

I’m working on how different temperatures affect the scaling of metabolic rate and gill area with body mass. It is commonly accepted that both metabolic rate and gill surface area increase with body ...
user avatar
  • 25
0 votes
0 answers
14 views

Underdetermined system is fit with constant residuals

I am observing a strange (constant) residual error in my linear model. PROBLEM: I fitted a linear model (LM) $y=Ax$, where $A \in \mathbb{R}^{n \times p}$, with $n < p$. Hence, $A$ is ...
user avatar
  • 123
0 votes
1 answer
49 views

Dealing with bimodal residuals

I want to run linear models to understand the effect of single predictors on an outcome. This is quite straightforward, but I am facing a situation where my residuals appear to be bimodal. I can't ...
user avatar
  • 45

1
2 3 4 5
45