Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
1
vote
0answers
32 views

Problem with calculating a confidence interval

I'm working on Exercise 3.2 from Elements of Statistical Learning. It asks to find a $95\%$ confidence interval for a linear regression prediction (ordinary least squares are used) using two different ...
1
vote
1answer
31 views

Screening candidate models before AIC comparison?

I am interested in identifying the best of 3 physiologically reasonable models that fits my continuous data. Data is some measure derived from neurons recorded from 3 adjacent regions of brain tissue (...
2
votes
0answers
18 views

Linear regression: an input variable as a multiplication/addition of other input variables

We have ways to identify collinearity in a multiple regression (using input variables' correlation matrix) and remove collinearity by dropping some of the collinear input variables. But what can be ...
1
vote
0answers
11 views

Using likelihood ratio test to compare 2 nested simple linear models instead of anova

Is it valid to use a likelihood ratio test to compare 2 nested linear models instead of anova? I'm trying to assess whether the quadratic model is a better fit to the data. I know anova seems to be ...
0
votes
0answers
10 views

Price elasticity for n data points

So let's say I have 100 data points which contains the price and sales of a product. Just to start I assume that the relationship is of the form Q = a - b*P where P ...
2
votes
0answers
13 views

Identifiability of parameters in a linear model when covariates are random

Suppose we have a linear model (in $\mathbb{R}^n$, say), $$y = X\beta + \epsilon $$ where $\bf{\epsilon}$ is Gaussian with mean $0$ and covariance matrix $\Sigma(\theta)$ where $\theta$ is an unknown ...
3
votes
2answers
49 views

LMM: Non-independence of observations sharing a single fixed-effect value

I am interested in the effect of monkey's stress levels on the pitch of their calls. Each stress measurement is associated with a bout of calling (that is, multiple calls), and in some cases I have ...
0
votes
0answers
52 views
+50

When can we use fixed design regression results for the random design setting?

Suppose I have an independent vector $X$ and a dependent scalar random variable $Y$ and I wish to construct a regression model to predict $Y$ using $X$ given data $\{(x_i,y_i)\}_{i=1}^{n}$. For ...
0
votes
0answers
16 views

Consistency under mild endogenity

Assume the usual linear model: $$Y_i = X_i\beta + \varepsilon_i, \quad 1\leq i \leq n$$ whit $E(\varepsilon_i)=0, Cov(\varepsilon_i, \varepsilon_j) = \sigma^2 \delta_{ij}$ and $Cov(X_i , \...
0
votes
0answers
78 views

Adjust for confounder in calculating explained variability in cox-regression

My question is closely related to this one. I am interested in the proportion of variability which is explained by a certain covariate X in a cox-model. So I have the cox-model “outcome ~ X”, for ...
0
votes
0answers
18 views

How to generate b-splines that are orthogonal to the corresponding variables in non-linear regression?

I want to fit a non-linear regression model of the type $$y_i = \alpha_0 + x_i\alpha_1 + s_i^T\beta + e_i,$$ $i=1,\dots,n$, $\alpha_0,\alpha_1\in{\mathbb R}$, $\beta\in {\mathbb R}^p$. I am only ...
0
votes
0answers
26 views

What am I looking at here in this regression analysis? [closed]

I have no idea what to do with this information or how it helps me determine if education level influences tolerance towards minorities. Control Variables are age, gender, province, and if they ...
0
votes
0answers
49 views

How do I analyze this linear regression residual plot?

I need help interpreting the residual plot and model diagnostics. I built a model for number of ticket sales for an event. so the dependent variable is a continuous variable. Below is how the ...
0
votes
0answers
12 views

Change in Residual Sum of Squares [duplicate]

I need to show that $R^2$ will never fall when a variable is added to the regression. For the proof, the book I'm referring to (Econometric Analysis- William H. Greene) considers this step: $e'_1e'_1-...
2
votes
2answers
68 views

Can I use multiple regression on a ranked response variable as a significance test for multiple covariates?

This blog post illustrates the relationship between inference tests on groups (t-test, ANOVA, etc.) and equivalent linear models. It also claims that for reasonable sample size, regression of a ranked ...
0
votes
1answer
31 views

What is the special name for linear regression when there are many parameters in y instead of one?

I rememeber reading about something that is exactly a linear regression A x = y Except that y for each x, is not just one point but rather a vector. Can any one please remind me what's that?
0
votes
0answers
10 views

Regression model for matched case-control study with continuous outcome

I have a matched case-control study design, where the outcome is continuous (and highly skewed), and there are multiple confounding variables. I have found a lot of literature discussing the use of ...
0
votes
0answers
6 views

Why a weight vector can be expressed as a linear combination of the training examples?

I'm digging into SVM's, and there is a certain step which is not all clear to me, and it is the part of representing $W$ as a linear combination of the training examples. How can we suppose that this ...
0
votes
0answers
15 views

Why my Deming Regression line change so much when switching variables? If they seem to be a linear relationship betwen them?

I am trying to fit a line that best predicts the production of energy Y given the speed of wind X, a typical Y = xm + b , using deming regression. I am looking for the slope and the intercept of that ...
1
vote
0answers
37 views

A case of multiple hypothesis testing?

I am trying to understand whether the use of several linear models using the same variable as a response variable (but differing in one of the specified predictors) constitutes a case of multiple ...
0
votes
1answer
33 views

Best Way To Illustrate Survey Data Correlation

I have a question regarding the validity of the methods behind this graph I created. On the Y-axis we have objectively measured food intake (daily energy intake) and on the X-axis we have "hunger ...
1
vote
0answers
40 views

What is the most appropriate way to aggregate two identical linear models fit with different data?

Suppose I have a linear model y ~ Xb, and I split my observations into multiple X's X1, X2, X3 etc. What is the most appropriate way to aggregate the separate models y1 ~ X1b1, y2 ~ X2b2 to produce ...
0
votes
0answers
29 views

Multiple Linear regression: inverse the y value

I have a dataset of 3500 samples where delay (dependent variable) depends on multiple system variables,such as cpu, memory, etc. I can use a multiple regression model and predict the delay against a ...
0
votes
1answer
36 views

Algebraic equations for mixed linear models and when to use constraints on parameters

My issue relates to Question 4a. of Paper 1. The corresponding solution gives the algebraic equation of the fitted model as $Y_{ijk} = \mu + \tau_i + b_{ij} + \epsilon_{ijk}$ and imposes a ...
1
vote
1answer
44 views

Survey Data in General linear Models

I have survey data and a collection of covariates. I have a few questions. I am wanting to predict a dependent continuous variable with GLM (Ancova) Is it necessary for this dependent variable to ...
0
votes
0answers
14 views

How do you compute the slope for weighted least squares? [duplicate]

You can compute the slope of linear regression (without weights) by: cor(x, y) * sd(y) / sd(x) If we have add weights (w) in here as well - how do you compute the slope then?
0
votes
0answers
17 views

Checking Heteroskedasticity in ANCOVA models : Breusch-Pagan test?

I would like to check the heteroskedaticity in an ANCOVA model : ...
0
votes
0answers
24 views

Orthogonal principal component of submatrix

Consider a data matrix $M_{n ~ x ~Features}$ and its top two principal components ($PC_1$ and $PC_2$). For any of submatrix $S_{n ~x ~F}$, where $F \subset Features$ (i.e. S contains all samples but a ...
0
votes
0answers
8 views

How to fit linear with Multimodal indepdndent varibale [duplicate]

How can i fit linear regression if my dependent variable is log normally distributed and independent variable is bimodal or multimodal?
0
votes
0answers
18 views

Fitting a regression model with spurious variables: do they vanish with large samples?

I am fitting the usual linear regression model $$y_j = x_j^T\beta + e_j,$$ where the errors $e_j$ are iid normal with unknown variance. If the vector of covariates $x_j$ contain spurious variables, ...
0
votes
1answer
11 views

contrast to find dependence on continuous variable within a group?

I have a linear model like Y ~ group * X, where X is a continuous variable and group is a ...
0
votes
1answer
26 views

Linear Combinations of Least Square Estimator

I come across a problem about finding the least square estimator of A$\beta$, where $\beta$ is the parameter vector in linear model ($Y=X\beta+\epsilon$). My question is, would the least square ...
0
votes
0answers
26 views

Am I doing the right model? lm or lmer?

I am looking at the effect of land cover (tree species, grass, woodland) on soil carbon at 3 depths. I have site as a random factor and biomass a covariate. I ran a ranova which revealed there was no ...
2
votes
2answers
1k views

Why does linear regression use “vertical” distance to the best-fit-line, instead of actual distance? [duplicate]

Linear regression uses the "vertical" (in two dimensions) distance of (y - ŷ). But this is not the real distance between any point and the best fit line. I.e. - in the image here: you use the ...
0
votes
1answer
14 views

biased prediction on output variable

Consider the linear regression model, y = Xβ + e, where as usual y and e are of dimension n × 1, X is n × k and β is k × 1. Additionally, the error term is correlated with the data such that E(e|X) = ...
0
votes
0answers
13 views

Covariance correction for linearly mixed signals

Consider the simple linear mixing model: $$ X = AS + v $$ where: $X$ is N-by-T, $A$ is N-by-M, $S$ is M-by-T, and noise $v\sim\mathcal{N}(0,\Sigma_v)$. Assume that we know the matrices $X, A,...
0
votes
0answers
26 views

How do I solve a linear inequality system ($X\beta+b<0$)?

Given a low-dimension linear regression problem $\mathbf{y}=\mathbf{X}\beta + \epsilon$, we can easily estimate $\beta$ with $(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^Ty$. However, the problem seems ...
3
votes
2answers
98 views

How does R's lm algorithm handle factors [closed]

I thought they use discriminant analysis as discribed e.g. in chapter 4.4. in James et. al. "An Introduction to Statistical Learning with Applications in R". But after input from this article and ...
0
votes
1answer
52 views

Linear model for repeated-measures regression

I have two independent variables $y_{mi},z_{mi}$ ($z$ is measured in fasting, so it is the basal state), measured with two different methods $m$ (m=2 is the reference method) in the same subjects $i$, ...
0
votes
1answer
21 views

Different price elasticity results

According to this article, calculating elasticity of demand for different models is: I generate data for 5% reduction in prices with a corresponding 10% increasing in sales: price elasticity = (+10%/-...
0
votes
0answers
16 views

Dealing with multiple independent variables with the sum of Linear Regressions

Suppose I want to predict a quantity (in week $t$): $V_t := \sum\limits_{i=1}^{10} V_{i,t}$. We do this by (simple) linear regression on each of the individual quantities that make up the sum, with ...
0
votes
0answers
10 views

Regression with non-frequent variables

I have a dataset in which I am regressing an output variable which changes daily to 4 independent variables. 3 of these variables changes on a daily frequency, but the other one changes on a quarterly ...
1
vote
1answer
30 views

Checking for significance in several variables independently

I believe this question is very simple, but I can't seem to google it right. I have data on the efficiency o vehicles (Km/L) for several different routes, vehicles, models and several other ...
1
vote
0answers
17 views

Why picking several times of same instances generally converge faster than going through instance by instance using Stochastic Gradient Descent?

I am reading Hands-on Machine Learning with Scikit-Learn & TensorFlow by Aurelien Geron. In chapter 4: Training models page 122, where it is explaining linear regression using SGD, it says that ...
0
votes
1answer
34 views

If logistic regression is a linear classifier why does it fail on linearly separable data?

Logistic regression is a linear model, decision boundary generated is linear. If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
0
votes
0answers
14 views

Understanding influence of correlated predictors on target variable

I am working on a problem where I have a "target" variable $Z$ that I know for sure is influenced by a "predictor" $Y$. I also have a second predictor $X$ that is correlated with $Y$ (about -.3), and ...
1
vote
0answers
5 views

Does the initial distribution of data have any affect on which regularization parameter can work well?

In scenarios when we want to know why performance of a predicting linear regression model when using L1 regularization has outperformed with the case that we have used L2 regularization, I wonder ...
0
votes
1answer
61 views

Am I using the right linear mixed model design for my data?

I want to move from using repeat measure ANOVAs to linear mixed models (LMM). However, where I have good intuitions about ANOVAs, LMMs are new to me. I'm using python's StatsModels as my package. Here'...
1
vote
0answers
15 views

Finding the exponents of a multiple power law: is linear regression valid?

I want to fit a multiple power law equation of the form $y = {x_1}^{\alpha_1} {x_2}^{\alpha_2}$ where I have many examples of $y, x_1, x_2$. (Note there is no intercept.) Is it possible for me to ...
0
votes
1answer
27 views

Evidence for heteroscedasticity from unordered valued

I'm fitting a linear regression model on a dataset about how many upvotes a certain post will get based on its views, its author's reputation ecc. To satisfy the normality assumptions I performed a ...