New answers tagged

2

The first step usually taken for non-linear relationships like this is by adding a polynomial term. A linear relationship is y ~ x, but you can use a quadratic approximation by doing y ~ x + x ^ 2. This is like x interacting with itself: the relationship between x and y depends on the value of x. It is still a linear model, since you are adding up ...


2

But I do not understand what it means by "correct test for significance". Can someone explain what he's referring to? If I were you I would post a comment to that answer by @EdM, otherwise, unless they actually see this question and answers themself, we can only make an informed guess. Having said that, what I think is meant by that statement, is that the ...


3

I'm afraid I don't know Stata (at all...), so I'm making some guesses here. Your real data differ somehow, or Stata is doing something I can't divine, because you have yhats for two patients with missing responses. Using complete case analysis, I replicated the logistic regression model in R. Your yhats are predicted probabilities from a standard ...


0

Does this mean that the score has a higher validity than the binary variables on their own? Precision and validity are not the same thing. I can estimate something very precisely and have it be invalid. You'll need to determine what your hypothesis actually is before you select a regression model. Are you interested in how the composite score affects the ...


2

First, the title of the question is so broad that it would need a book or two to answer. But in the text, you seem to be concerned mostly with collinearity and want to use correlations to weed out some variables. I'll just answer about that. Correlations are not a good tool for diagnosing collinearity. You can have problematic collinearity with very low ...


0

Multiple imputation of the missing data is one accepted way to approach this problem. There are nearly 400 questions on this site tagged as multiple-imputation. That's valid provided that the data are missing at random in a specific way, often a reasonable assumption. Instead of simply replacing missing values individually with means or medians or modes you ...


0

I would suggest you fix the number of data points before creating any model. You either remove all the data points which contain missing values or replace the missing values with the mean/median of that variable. You can also look for advanced methods like EM-algorithm to fill the missing values. Once you are done with that, you can proceed with creating ...


1

The interaction term you propose would give you a predictor that is 0 for values of $x$ less than or equal to 0.5 and equal to $x$ for values above that. In principle there's nothing wrong with that formulation of a predictor variable in your regression. That wouldn't, however, test your hypothesis that "$x$ is a significant predictor of $y$ but only when $...


2

The output is a linear combination of the inputs. Form of the input features does not matter; for example, $y=\beta_0+\beta_1\sin e^x$ can still be a linear regression model. We can always call our features with different names, e.g. let $u_1=xt,u_2=x^2$, then your model becomes: $$y=\beta_0+\beta_1u_1+\beta_2u_2=u^T\beta$$ which is linear. Edit: An example ...


8

Another issue is that the relationship between the IV and the DV (here, BP and heart attack risk) might not be linear. I would think this sort of nonlinearity would be quite common in medical fields. Indeed, this is sometimes given as a reason for categorizing the continuous variable (albeit into more than two categories). But this isn't good. A better ...


14

Dichotomizing a continuous covariate is ill-advised, as has been noted by other users. One strategy I employ is to rescale the predictor to something more reasonable. 1 mmHg may not be a very meaningful scale on which to interpret changes in BP. But, if you rescale the predictor so that a difference of 1 unit represents say a difference 10 mmHg then ...


12

Similar to EdM's answer, a marginal effects plot is a useful way to showcase the relationship between a clinical measurement and outcome while holding other variables constant. These plots are helpful because they show the relationship between the predictor and outcome, so if the outcome is nonlinear, physicians can easily see this and interpret it ...


2

The multilevel model allows you to simultaneously model within- and between-plot variation. Consider a simplified multilevel model, with the Xs representing within-plot variables and W representing a between-plot variable. The within-plot model: $y_{ij}$ = $\beta0_j$ + $\beta1_jX_{1ij}$ + $\beta2X_{2ij}...$ + $e_{ij}$ Here you can include as many X ...


9

For digestibility, use examples from representative situations: in your example, maybe comparing risks at BP of 160 versus BP of 120. For an approach that can take into account the multiple predictors typically important in clinical studies, use a nomogram. It provides a graphical tool to show how predictor values affect outcomes. The rms package in R ...


0

If you are a complete beginner as you state, run the multiple linear regression with the happiness score (if it is indeed a continuous and not a categorical variable) for each country and look at the coefficients and whether or how they change. This answers the question "how was country x1's happiness score in 2015, 2016 and 2017, controlled for GDP and ...


2

For a video I'd recommend the MIT 18.650 Statistics for Applications course see lectures 21-24. Also the Princeton website for their courses on the subject https://data.princeton.edu/wws509


2

The importance measures here are unlikely to be adapted into a scale that can be compared. Rather, the best way to compare these models is to compare the rank order of the input/predicting variables. It is also worth noting that the linear/lasso/ridge models are fairly different in what they represent compared to boosted tree ensemble. In particular, the ...


7

The multivariate regression can take into account potential dependence between the two dependent (response) variables. Running two regressions separately cannot.


0

Multiple regression model always capture effect et ceteris paribus for each variable, meaning when every other variables are held constant. I think this property is enough to justify my first intuition, that is: In Equation $(1)$: $\hat{\beta_a}$ accounts for $e_1 + e_2$ In Equation $(2)$: $\hat{\beta_a}$ accounts only for $e_2$; $\hat{\beta_b}$ accounts ...


0

It probably means PLS outperforms OLS even in the presence of unrelated variables. I would definitely check regression coefficients of PLS model to see whether the the unrelated variables have relatively low absolute magnitudes. However, if the unrelated variables are related to each other, then the regression coefficients can be deceiving. The lower RMSE ...


2

Suppose we have a linear regression model $$y=y_{12\ldots p-1}+\varepsilon_{12\ldots p-1}\,,$$ where $y_{12\ldots p-1}=\beta_0+\beta_1 x_1+\beta_2x_2+\cdots+\beta_{p-1}x_{p-1}$ is the part of $y$ explained by $(x_1,x_2,\ldots,x_{p-1})$ and $\varepsilon_{12\ldots p-1}$ is the unexplained part. Parameters $(\beta_0,\beta_1,\ldots,\beta_{p-1})$ are estimated ...


2

The thing that strikes me about your example data is the low proportion (about 4%) of Fractures. That may very well be legitimate, but it does mean your data contains very little information. In the example data there are only 25 persons with a fracture. With such a tiny sample size it is no surprise that the model becomes highly unstable, especially in a ...


2

You ask a couple of questions here so let's unpack. 1.) "If you have a variable you want to investigate but you realize you should probably control for a number of variables, is it okay to do this for every variable you find interesting?" Normally, you would use the literature to determine which variables you should examine that need to be controlled for. ...


0

As a guide to the analysis of correlation relationships in your database, I refer one to this comment on the proper use (or misuse) of Pearson’s correlation coefficient, to quote: If a parametric test of the correlation coefficient is being used, assumptions of bivariate normality and homogeneity of variances must be met. We give several examples where ...


3

Since you say : However, the relationship between $\textbf{y}$ and $\textbf{X}$ is not strictly linear over the entire domain, but could be better modeled as such within several subgroups $g$ (and coefficients are more meaningful if defined for each subgroup): it sounds to me very much like a mixed effects model (of which multilevel and hierarchical ...


2

There is nothing weird about the correlation. It answers one question only: how well could these data be summarized by a straight line? You get a bonus answer, the sign of the slope of that best fit line. The scatter plot makes it clear: no straight line could work well to summarize the entire relationship. The grouping of points is interesting and important ...


4

You need to decide what tests you need to make, because although the models are the same, the tests that the software automatically conducts will differ. To see why, consider the simplified version of the situation you originally proposed, where there are two regressors $x_1, x_2$ and their interaction $x_1x_2$. Let $\xi_i$ be the corresponding ...


0

My recommendation is to apply a Box-Cox analysis of transformation using the two-parameter option (as the modeling will suggest the proper power transform and additive constant). Here is an easy discussion source and also at here. More advanced is this paper and also the discussion in Wikipedia, especially the first example noting plots (b) and (c) which ...


2

I would not do this. The problem is that what you choose to add to make x positive is arbitrary and can have a huge effect on the parameter estimates. First, let's set up x and y and the model: set.seed(1234) #Sets a seed x <- rnorm(100, -10, 1) #Normal mean = -10, sd = 1 y <- 3*x + rnorm(100) Now, we'll adjust x to be positive so that logs can be ...


0

Considering you're comfortable with the idea of making subjective assumptions, and it's not a model that will determine if someone lives or dies, I would suggest a Bayesian model might be a good option. This would allow you to state your assumptions explicitly and visualize them in a graph, so it can be audited and interpreted in a fairly straightforward ...


2

The approach that is currently used is wrong for many reasons, see Algorithms for automatic model selection for a nice summary. As for alternatives, there are two (three) main ones, depending on your goal: As you suggested, use domain knowledge to filter the data and then run one model, in which case inference holds and can be reported for all variables. ...


0

If you are targeting on getting better accuracy instead of interpretability, DO NOT do feature selection. Use all features and add regularization. In general, more features means more information, in an extreme case, the feature has nothing to do with the prediction target, the fit will get the coefficient to be 0 automatically. So, more feature will not ...


2

In regression equation $$ y = \alpha + \beta x + \varepsilon $$ the $\beta$ parameter is about the slope of the regression line, while $\alpha$ is about moving it vertically along $y$-axis. Since a picture is worth a thousand words, you can see this on a picture below showing $x$, and shifted variants of it, on $x$-axis, and $0.56 x$ on $y$-axis. As you ...


3

The intercept $\beta_0$ is interpreted as the predicted value of $y$ when $x$ is zero. The p-value corresponds to the test of $H_0: \beta_0 = 0$. Sometimes this is an interesting test, and other times it isn't. In general, you might ask the question, "For a given value of $x$, does the predicted value for $y$ differ from zero?". The test of $\beta_0$ answers ...


2

The standard error, p-value, and confidence intervals about regression constants represent the uncertainty in the estimated value of the constant. Let's take bivariate OLS linear regression as an example. One of the primary questions we answer with linear regression is what straight line* best predicts the trend in mean values of $y$ over values of $x$? ...


0

The computational cost of gradient descent depends on the number of iterations it takes to converge. But according to the Machine Learning course by Stanford University, the complexity of gradient descent is $O(kn^2)$, so when $n$ is very large is recommended to use gradient descent instead of the closed form of linear regression. source: https://www....


0

Yes, there is an advantage to $R^2$: It has a direct interpretation as the proportion of variance in the dependent variable that is accounted for by the model. Adjusted $R^2$ does not have this interpretation. Also, you write that adjusted $R^2$ "penalizes the model for useless variables". That is true but incomplete. First, almost no variable is totally ...


5

A bit late to the party but I wanted to clarify that the intercepts in your models have different interpretations, as follows. model1 <- lm(Totalpoints~m400 + m1500 + m100 + m110hurdles, data = samp1) The intercept for model1 represents the total points predicted by the model for a randomly selected athlete from your target population of athletes for ...


6

Linear regression is $$ y = \text{intercept} + \text{coefficients} \times \text{features} + \varepsilon $$ If $\text{coefficients} \times \text{features}$ is high, then the intercept would need to be low enough to align the result with the predicted values, and if $\text{coefficients} \times \text{features}$ is small, the intercept would need to be bigger. ...


0

With no data, I will attempt to take a stab at this: The first part of the problem suggests that you have a strong correlation between two markers, but the second part of your question indicates that there is no strong evidence that surgery can be determined by either marker, but needs both to be significant. This sounds like multicollinearity a ...


2

What else should I do to improve it? Maybe the association is nonlinear. Did you plot the data first ? Should I add more variables, such as number of # of visitors, into regression and why? If you have other variables that are potentially associated with the outcome then yes. Why ? Because it could improve the model. But first try to look at all the ...


0

I don't think this question has a really good/canonical answer. Because of the low level of replication within cities, you don't have a lot of information to distinguish within- and between-city variation. You could: fit a multi-level model and hope for the best, i.e. that you don't run into numeric problems or singular fits pretend that the within-city ...


2

First, you would expect that the mutation and no mutation groups would have the same BMI since the presence of the mutation is completely independent from BMI and all other variables in the data. The fact that you got a significant difference using the first comparison is a fluke; if you use a different seed, (e.g., 9999), you get a nonsignificant difference....


1

Mixed models are used to account for the correlations in an outcome variable in grouped or clustered data. Like in your case, scores coming from the same participant will be correlated. However, there may also be other sources of correlations, e.g., scores from the same location could also be correlated. Mixed models do allow for such complex correlation ...


0

You have two options: Option A) Try to figure out a confidence band for each observation in y, given its corresponding value of x (or x's, in case of multiple regressors).Keep on doing this independently for each observation, as if the other predictions did not exist, and you get your red-band. Note, the band has some non-zero width since the predicted ...


2

For the question: "How much of a dependent variable is explained by each of a bunch of independent variables?", the short answer is the so-called adjusted R-squared. It is a modified version of R-squared that has been adjusted for the number of predictors (degrees of freedom) in the model. The adjusted R-squared increases only if the new variable actually ...


6

The important thing is that the influence that each predictor has depends on which other predictors are in the model. So the effect of each predictor depends on the model you're using, and which other variables are in it. For example, you might have two highly collinear variables that both have a large $R^2$ when used in a bivariate linear model, but, ...


0

In addition to @EdM 's excellent answer, I'd like to address your question: why it can be directly known that it should be reduced to 5th order model? If you go strictly by p values, then all the terms of higher order are nonsignificant. But that's not really a good test. If you had similar effect sizes with a very large N, then the 6th or 7th or ...


1

With orthogonal polynomial terms as predictors in linear regression (orthogonality is enforced by the poly() function in the formula), the lack of correlations between predictors means that when you choose to remove a high-order term from the model you don't face the problem with omitted-variable bias that occurs when you remove a predictor associated both ...


1

Although more sophisticated optimizers do well in general for solving highly non-convex problems, using vanilla gradient descent may very well suffice for a convex one such as this (i.e. linear regression): model.compile(loss='mean_squared_error', optimizer='sgd') A small touch on your plotting: plt.plot(y_train, model.predict(X_train), 'x', color='blue', ...


Top 50 recent answers are included