5
$\begingroup$

I have data that goes from negative to positive. When plotted in an histogram it looks like this

histogram

The data is the "error".

If I have another set of data I want to learn how to judge which data is "better" (the smaller the error the better the data).

I have been advised to convert everything from frequency to density to standardize the plots and be able to compare. How can I do this?

And what other plots I can use to judge data in this case?

I am trying box plots and I get something like

box plot1

which has a lot of outliers; eliminating those

boxplot2

I suppose that I can compare somehow these with boxplots of other data, but I am not really sure how.

Any other ideas how to compare and evaluate data are welcomed.

$\endgroup$
1
  • 1
    $\begingroup$ The title tells "Plots to judge data that is a bit asymmetric", but from the body text it is not very clear to me what you exactly try to compare. $\endgroup$ Commented Oct 3, 2023 at 13:39

1 Answer 1

7
$\begingroup$

There are several distinct questions here arising directly or indirectly.

How to calculate and show density?

Any good statistical software allows showing histograms with a density scale and also other methods of working with density, such as kernel density estimation. The details will necessarily depend on your software.

Outliers

Box plots such as you have drawn show points more than 1.5 IQR away from the nearer quartile as (indeed) points on the graph. This is not a criterion for data that should be ignored or can be omitted. It's primarily a way to show fine structure in the tails.

Symmetry

You can pair off points,

minimum and maximum,

second smallest and second largest,

and so on, and -- noting that in a perfectly symmetrical distribution the means of each pair would be identical -- plot those against their differences, maximum $-$ minimum and so on.

Here is a simple example. In this case, the last-mentioned plot doesn't add much information to what is apparent from a histogram, quantile plot or cumulative distribution plot, but for your datasets it could provide more subtle comparison.

You can superimpose histograms with care (watch that the bars of any set don't occlude the bars of any other set; transparency is often needed). It is easier to superimpose a few quantile or cumulative distribution plots.

enter image description here

$\endgroup$
3
  • $\begingroup$ Could you explain the second point(about outliers)? What information can I get from the outliers as shown in the plot? $\endgroup$ Commented Oct 2, 2023 at 12:11
  • $\begingroup$ An additional easy piece of advice is to make a simple table of summary stats (e.g. mean, median, different quantiles, min/max). In python (to the OP) it will be something like data['residual'].describe(), or more familiar to Nick in Stata summarize residual, detail. I only say that as personally I find that much quicker/easier to spot gross differences in distributions. Plotting of two variables with different ranges on the same graph can be tricky. $\endgroup$
    – Andy W
    Commented Oct 2, 2023 at 12:18
  • 3
    $\begingroup$ Sorry, but I don't know what you find unclear about outliers. You need a really good reason to omit outliers, such as establishing that their values are wrong and can't be corrected. $\endgroup$
    – Nick Cox
    Commented Oct 2, 2023 at 12:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.