New answers tagged

0

Fitting the model on the residuals from your control variables is the same as fitting the model all together after orthogonalizing the study variables with respect to controls (it's the same model, but partly rotated). Not the best thing by interpretability standards, but absolutely legit all in all. Compared to standard OLS, you have two pieces of the same ...


0

Since the most uncorrelated new stock is what you want (1) and not which whose variance is best explained by another stock's variance (2) the first route should be followed since the second plan requires at least 400 regressions. Calculate the correlation matrix for the existing 100 stocks and 400 candidates together, while remembering that the first 100 row ...


0

I want to do this in an entirely data-driven way If you are interested solely in prediction then there are many different approaches including partial least squares and regularization. However, I have 40 treatment variables, and I am interested to find out which ones are related to my dependent variable implies that you are thinking causally. If so, you ...


0

To find out which variables have the most correlation with a dependent variable, the simplest way I can think of is to perform a Parial Least Squares (PLS) analysis in that data with your dependent variable defined as output. Then, you can obtain the VIP (Variable Importance in Projection ref, ref) values, oe even simpler to analyze that, the VIP plot, and ...


0

There is a battery of problems with stepwise regression, which also includes best subset regression. Multiple regression is fine if $n\gg p$, otherwise you may be better off using regularization. Namely, a regularized model (like the LASSO you mentioned), restricts the total size of the parameter estimates. Perhaps somewhat surprisingly, introducing this ...


2

It depends on your modelling goals: if you are thinking of causal inference then in the first model, you are either estimating the effect of the average results in other subjects while controlling for the possible confounder (or competing exposure) Gender; or you are estimating the effect of Gender on the outcome, while controlling for the possible ...


2

The issue here, is that you are fitting an interaction. The interpretation of the main effects changes in the presence of an interaction. Without an interaction, each of the main effects has the meaning of: the association of a 1 unit change in that variable, with a change in the outcome, leaving the other variable unchanged With an interaction, when ...


0

I had a look at the simpler case of $ Y = aX+b $ To minimise $ E[(Y-aX-b)^2] $ take the partial derivative wrt $ a $ and $ b $. $ \implies E[(Y-aX-b)X] = 0 $ $ \implies E[XY] = aE[X^2] + bE[X] $ Also, $ E[Y-aX-b] = 0 $ $ \implies E[Y] = aE[X] + b $ $ \implies $ $ \hat a = (E[XY] - E[X]E[Y])/(E[X^2]-E[X]^2) $ $ \hat b = E[Y] - aE[X] $ Do the same for $ E[(Y-...


1

I must add another answer. If you like proofs, let me agree with you that a single example, and a single picture, doesn't prove anything :) Your example depends on the number of soft/hard candies (there are much more hard candies than soft candies), on their position etc. Building another example where the overall regression matches the regression for soft ...


1

I found a very easy example that show the effect. The overall regression (including hard and soft candy) indicates a decreasing trend. But if you only look at the soft candy you will realize that the regression shows a positive trend.


0

I agree with Robby the Belgian and I only wish to add an example.[1] y = number of deaths in London from Dec 1 to Dec 15, 1952 x = smog (mg/m3) z = sulfur dioxide (parts per million) > airpoll y x z 1 112 0.30 0.09 2 140 0.49 0.16 3 143 0.61 0.22 4 120 0.49 0.14 5 196 2.64 0.75 6 294 3.45 0.86 7 513 4.46 1.34 8 518 4.46 1.34 9 430 1.22 ...


1

No, they aren't interchangeable. It may help you to read my answer to: What is the difference between linear regression on y with x and x with y? For an overview of the case with logistic regression, it might be worth reading my answer to: Relationship between regressing Y on X, and X on Y in logistic regression. In the linear regression case, the slopes ...


4

This is not uncommon. It indicates that fruity correlates with another variable. Essentially, having taken all other variables into account, fruity will correlate positively with winpercent. For example, maybe hard correlates with fruity. Say, we have the following table: hard | fruity | # disliked | # liked ------------------------------------ no | no ...


1

There are two issues here You would, ideally, use weights inversely proportional to the variance of the individual $Y_i$. So says the Gauss-Markov Theorem. You don't know the variance of the individual $Y_i$ If you have deterministic weights $w_i$, you are in the situation that WLS/GLS are designed for. One traditional example is when each observation is ...


2

Your question is a bit unclear to me. However, as far I can understand... if the generalized linear model is $y(x_1,x_2)=b_0+b_1x_1+b_2x_2+b_3x_1x_2+b_4x^2_1+b_5x^2_2$, and there are, say, four data points, then ...then your model matrix $X$ is a $4\times 6$ matrix: $$\begin{bmatrix} 1 & x_{11} & x_{21} & x_{11}x_{21} & x_{11}^2 & x_{21}...


3

As a general rule, regression models with penalties are reasonably good at variable selection. (Better than the bad old days of stepwise procedures anyway!) Penalty models usually have some consistency properties that ensure accurate selection of variables for large samples under certain conditions on the penalties. The goal of these models is to ...


1

The answer depends on whether you are restricting yourself to the class of linear models, which I will define as something with the form: \begin{align} y_i &\sim \mu_i \\ g(\mu_i) &= X_i\beta. \end{align} Further, let's denote the sample size by $n$ and the number of predictors/variables by $p$. Case 1: Linear model If you have a large sample, then ...


2

The outcome graduated is binary, so you need a model for binary data such as a logistic model. There appears to be repeated measures within id so you need to account for correlations within id. You could use a model with random effects for id to do so. To answer your research questions you can fit a model with fixed effects for time_with_new_teach and ...


2

First off, I think it is useful to start looking at generalizations of models rather than their exact implementations. A polynomial regression is a regression just with a polynomial expansion (feature engineering) and multiple linear regression is a simple linear regression just with a different input matrix size. It's really all the same "model" ...


1

Prediction versus Understanding Broadly speaking, your goal when fitting a model is either to be able to make predictions, or to understand the relationships between your variables. All kinds of models can be used for both purposes, but some are better suited to prediction (but are hard to understand or interpret), while others are better for understanding (...


2

pyplot defaults to '-' for linestyle argument, which means a line that connects through all of the dots, so to produce a scatter plot, whose linestyle are data points instead write plt.plot(X_test[0], y_hat, '.')


2

Generally plotting functions connects the points that are consecutive in the array. So, first sort those points and then plot.


1

I'll answer your question first, but then I'm going to ask you a question too! Yes, you're on the right track with the dummy variable, but it sounds like you think that there may be effect modification from the number of days a promotion is offered, e.g. promotion $X$ might generate many sales the first day or two, but then fewer sales the longer it goes on? ...


1

This isn't really a 'discrete' variable, it's a categorical variable. Only the intercept is the odds of a woman (in the reference level party) being elected. The other coefficients are odds ratios. You multiply those odds ratios times the odds in the intercept to get the odds of a woman in the, say, green party being elected (in that case the odds is ....


1

I saw something like this in a data set once. The graph was of charitable contributions (Y) vs income (X). There was a distinct linear pattern with a slope of 0.10 that looked different from the rest of the data scatter. These were (I strongly believe) the tithers. The variable "tither", a 0/1 variable, is a type of latent variable mentioned by ...


3

I get an output that only lists group A but not group B for some reason. Presumably, group B is the reference level, and since R uses contrast coding by default, B will be included in the intercept and the estimate for Group will be the estimated difference in the outcome variable between A and B The difference between your two models is that the first one ...


1

What do you expect with 1000 variables and 30 observations? There is nothing wrong with your code, with fewer obs than variables you can always find a perfect fit with least squares (the solution is not unique, but using ginv you try to get the solution with smallest norm.) glmnet does not do this, it does not compute matrix inverses (generalized or not), it ...


3

There are clearly latent clusters in your data. Do you have any other variables, especially categorical variables, that might account for the different bands? If so, there is an interaction between that (those) categorical variable(s) and something else. In general, you should look at your data before you fit a model. You don't want to be surprised by ...


1

There's several possibilities: They intend you to fit an ordinary cubic regression spline with 4 df total. (In the context of this chapter, I think this is probably what they meant.) The answer to "how did you choose the knots" is just "I didn't -- there's no d.f. left for any". They meant "with 4 knots" rather than with 4df. ...


3

It's not a valid way to do it. Among other things, $x_1$ and $x_2$ can be correlated. Here is a simple simulation (coded in R): set.seed(9684) # makes this perfectly reproducible x1 = c(rnorm(20), rnorm(20, mean=1)) x2 = rep(0:1, each=20) cor(x1, x2) # [1] 0.4715828 these are ...


5

If you start with $$ Y\mid b \sim\mathcal N(X\beta + Zb, \Sigma) \\ b \sim \mathcal N(0, D) $$ then you can use the tower property and law of total variance, along with the fact that convolutions of Gaussians are Gaussian, to work out the marginal distribution of $Y$. Once you do this, you'll have a Gaussian RV with some mean and covariance. Then compare ...


4

This is a good question. The confusion stems from the "assumption" of no multicollinearity. From the Wikipedia page on multicollinearity: Note that in statements of the assumptions underlying regression analyses such as ordinary least squares, the phrase "no multicollinearity" usually refers to the absence of perfect multicollinearity, ...


1

"Partial moderation" is not a thing because you can either detect a statistically significant moderating effect or you fail to do so -- no partial moderations. Now, assuming moderating effect exists, the coefficient and p-value of X could change depending on the level of the moderator -- which depends on how you code / where you center your ...


1

Hopefully things become a bit easier if you look at the linear model as a linear regression. Your fit equation can be read as a linear model, in which the log of index is to be computed as a sum of some constant, a value for one of the levels of a and a value for each of the levels of group. summary(lm(log(index) ~ a + group))) a not being significant from ...


3

Yes, you are correct that a multivariable regression model is not qdequate here due to non-independence of measures within subjects. That is, measures within one subject are more likely to be similar to each other than to measures in other subjects. to account for these correlations, you can fit a model with random intercepts for subjects. Such a model is ...


3

The two regression produce numerically equivalent estimates of $\beta$, but do not lead to the same predicted values of the outcome, nor do they share the same residuals. The standard errors are identical as well (Proof ommitted but sufficient to show the estimates are identical). Proof (This follows closely from Davidson and McKinnon Ch. 2.4 on the FWL ...


2

Note sure this is necessarily a regression question. You have IQ scores on the same individuals, before and after an intervention, is that right? If so, you could calculate the percentage change for each person $i$, namely, $PctChange_i = 100(IQ_{i2} - IQ_{i1})/IQ_{i1}$. Then apply the usual types of techniques to the $PctChange_i$ data, such as point and ...


1

There is no issue in centering (or standardising preferably) your data before regression, but the percentage changes should be calculated from original IQ scores (if that makes sense anyway). Besides, you may have some numerical issues if you do it after centering, because somebody might have $0$ IQ.


-1

There are a number of issues here are you even certain you are doing a logistic regression? Assuming that is R, you’ve not specified the link function and i don’t know your data types if you are doing correlation, do correlation. If you are doing glm, do glm. Each method has accepted practice of follow up comparisons (regression = regression contrasts, glm ...


0

With respect to AIC, the Wikipedia page says: AIC can be used to form a foundation of statistics that is distinct from both frequentism and Bayesianism. Convincing a reader that you can use the AIC to choose among non-nested models depends on the reader's appreciating what AIC means.* Someone with that level of statistical sophistication will probably ...


2

Not necessarily. Search around on feature selection and model selection. Model selection is not a solved problem and it is unlikely to be solved since it is NP-hard. In my own experience, I have seen the LASSO sometimes select poor or even insanely wrong models. That is not restricted to the LASSO. Ridge regression, stepwise selection methods, searches using ...


5

Are there any mathematical drawbacks with this idea? Yes, you will lose statistical power as well as running into multiple testing problems. Don't split the dataset. How do I compare the resulting n models to my other model? I am unsure on how to interpret the n resulting metrics (i.e., a cross-validated r squared score) to those of my other model. You ...


1

What are the problems following this up with this If there are any problems, I think they are scientific. In the first model, what you are modelling is the outcome, let's call it $y$, given the two factors $x$ and $z$. The data generating process would be Draw jointly $x$ and $z$. You claim they are independent so this is the same as drawing $x$ then $z$...


0

This can occur with a disordinal interaction, AKA a cross-over interaction. This answer below covers the case where the variables aren't actually "interacting," as the four groups are simple randomly drawn normal distributions, but the interaction term is very significant. The answer also covers the case of a hypothetical real-world interaction, as ...


2

Welcome to Cross Validated! Here are some thoughts: Because correlation is not the same as causation, without knowing a priori that there is a single, known causal direction, there is no reason why cannot consider any variable as the response variable. Only further experimentation can tell you if there is a distinct causal direction. p = 0.23 does NOT mean ...


7

As mentioned in the comments to the question, you are interested in inference, not prediction - that is, you are interested in how the explanatory variables influence the outcome. Any kind of stepwise procedure, and any other automated method of variable selection (eg the lasso or other regularised procedure) will not work as it cannot account for bias due ...


1

So I had an opportunity to recreate the raw dataset and run the logistic regression. It does in fact, run in R and SAS, but you have a problem with what is known as "quasi-complete separation of data points." This happens when a linear combination of predictor variables completely determines or separates the outcome variable, and so the maximum ...


7

For intuition, consider that the $F$ test assesses whether all three variables B, C, and the interaction B:C collectively "explain" the variance of the response, whereas the t-test for any single coefficient considers only that coefficient (after the effects of the other coefficients have been accounted for). The F-test has to account for the ...


0

I notice that for these sort of questions there is always a lot of pedantry in the community about the use of the term "correlation". Us non-statisticians use the term to generally mean "relationship", but some people might not get that. So like others have told you, you can't compute the correlation coefficient for a non-linear ...


0

Like @Noah mentioned, there's no way to establish a causal link between an individual and whether they moved or not. This sounds like an interesting problem! Here's one approach. If you can group the latitude and longitude information of the present location into regions, you could try and predict that (see multinomial regression). If you include the past ...


Top 50 recent answers are included