New answers tagged

3

If your response variable is a (continuous) proportion, why not model it using a beta regression model. The beta regression model is not a GLM model but shares some similarities with GLM models. For GLM models, the effects package in R provides ways to visualize the model effects on various scales (including the response scale). For example, if the GLM ...


0

I'll outline a few ways to do this. My favorite way is to plot the coefficients and the confidence intervals. I know that business people are not always statistically savvy, so while in my opinion this is the way to present regression model results it may go over your partner's heads. Another way may be to just plot some different lines under different ...


2

One classic way to deal with count data that takes multiple factors into account is Poisson regression. That's a type of generalized linear model. In a way you've already started down that path by looking at the logarithms of the counts; that's the standard link function for Poisson regression. Poisson regression can be considered an extension of chi-square ...


-1

Maybe you can try ks.test > ks.test(rnorm(50),rnorm(30)) Two-sample Kolmogorov-Smirnov test data: rnorm(50) and rnorm(30) D = 0.18, p-value = 0.5272 alternative hypothesis: two-sided > ks.test(rnorm(50),runif(30)) Two-sample Kolmogorov-Smirnov test data: rnorm(50) and runif(30) D = 0.56, p-value = 6.303e-06 alternative hypothesis: two-...


0

Heteroskedasticity means quite general that the residual variance is not homogenous, i.e. depends on groups or some other variable. Note that, according to this definition, if you find that var(res) ~ fitted, you clearly have heteroskedasticity However, if you don't find a relationship between var(res) ~ fitted, you could still have heteroskedasticity, for ...


6

The reference in NYT appears to be to the authors of the paper Carson, Miller and Sassler 2018. Among other things, this paper gives analysis of the statistical relationship between division of household work and relationship trouble. I have been unable to find the descriptive data for the interface of these two variables in the paper (and there are not ...


2

Linear regression against time ASSUMES a model form and uncorrelated residuals to actually test the significance of estimated parameters. Your series might be adequately described with a local time trend (NOT GLOBAL) and a few level shifts and possible pulses and a possible memory component (arima) but only your data knows for sure. Post your actual data and ...


1

If you want to know the direction point to point, I believe that you can evaluate easily the slope: $\frac{Y_{t+1}-Y_{t}}{x_{t+1}-x_t}$. If this result is positive, it is going upwards. On the other hand, if it is negative it is going downward. If it is zero, it is going sideway. If your question is about to know the average direction, you should estimate a ...


3

What do you exactly mean by upwards, downwards or sideways? You could fit a linear regression model on the data. You will get a coefficient for the slope and a confidence interval. If the coefficient is not significant then you could say that there is no trend.


0

In my opinion "examining fitted vs residuals" is a graphical alternative to the Box-Cox test When (and why) should you take the log of a distribution (of numbers)? which can help to evaluate the hypothesis that the variability of residuals is or is not linearly related (sympathetic) to the level i.e.the fitted values of the series. IFF you are analyzing ...


1

If you want to include the testing results on a data set whose test results were used for optimization (your "validation" set), I'd recommend to do that in a third, separate lift curve (same for any other figure of merit or diagram). Yes, optimization is part of the model training and the model did learn from these results. We expect the optimization lift ...


0

The Elbow Method can sometimes provide a very clear answer, but often it's hard to discriminate the elbow or bend that reveals the best clusters. In the absence of a clear elbow, one can then turn to the Silhouette Method, which involves examining the average Silhouette Coefficient (SC) and finding the number of clusters k that maximize this value (with the ...


0

What this plot is saying is that you only have two predicted values from your model. One way that this could happen is that you only have one independent variable and it has only two levels. So, for instance, if you are trying to model weight based on sex, this sort of thing could happen. It is also showing that the residuals are much bigger than the ...


1

This is a histogram with superimposed kernel density for two variables, with no overlap in the data. If you are looking for a name for the plot, I would suggest "Histogram and KDE of Predicted Classification Probabilities". (You will need to add axis labels and a legend to your plot so that it makes sense.)


0

That looks a lot like the efficient frontier from modern portfolio theory. The efficient frontier is a set of attainable portfolios, the x-axis being the scale of risk with higher risk to the right, and y-axis being the scale of return with higher return being higher. The frontier line is made up of theoretical portfolios of all risky assets.


0

One way to account for (or "control for") a covariate in a regression is to "take it out of the model" by regressing all other variables against that covariate and retaining only the residuals from those regressions. See https://stats.stackexchange.com/a/46508/919 for an illustrated explanation and https://stats.stackexchange.com/a/113207/919 for a more ...


1

You're right: Boxplots are a good way to show data, but when there are few data it is recommended to just plot the actual data themselves. Further, you are most interested in showing the relationship within the pairings (probably the difference, but possibly their ratio, etc.), not the raw data, although they could be plotted in the background. For a ...


0

For such data I would start out with visualization, in this case a Bland-Altman plot (or Tukey mean-difference plot.) For an example, see Bland & Altman plot for repeated measures using one measuring device. In this case, maybe plotting on the x axis the IQbefore, and on the y-axis the change score IQafter-IQbefore. Then color the points according to ...


0

I finally found out (via this README file) that the mentioned Shiny app by Matthew Leonawicz can be run locally in R with install.packages("devtools") devtools::install_github("leonawicz/snapapps") library(snapapps) snapp("rv4") (The description for accessing it online does not work, however.) I also liked this "Distribution calculator", which is online, ...


2

Choice of plots depends on what you want to communicate to the reader, so it is always a contextual issue. Assuming that your goal is simply to give the reader a sense of the distribution of the file sizes, a histogram of the data is perfectly reasonable. Once you add clear axis titles and a plot title, the histogram you have constructed ought to be ...


2

As you already mentioned the Kruskal-Wallis test is a test of significance based on the ranks. In my opinion however, plotting the ranks isn't really that helpful for the reader in order to understand the underlying response variable. Instead, what I would do is to plot the individual data points (including the median for descriptive purposes) plus the ranks ...


1

Partial dependence doesn't tell you the full story. You are looking at the marginal effect of one predictor on the response variable. It's just like the marginal distribution doesn't tell you the full story of the joint distribution. The graphs that motivate individual conditional expectation here (https://blogs.sas.com/content/subconsciousmusings/2018/06/12/...


Top 50 recent answers are included