I would like to know what the best significance tests and metrics are to determine the fit of distributions in the following experiment:
Given a product feature value x, participants in a study are supposed to estimate the value of product feature y under one of 3 conditions (positive, negative, neutral). These estimations are the lower bound and the upper bound of an interval of values for which the participants would consider the condition to be fair / adequate. Taking a laptop seller's product review as an example:
"Given the tiny size of 12 inch of the laptop, it is positive that it has a comparably large battery of 80 Wh."
Here, x is the small size of the laptop, y is the battery capacity (80 Wh), the condition is positive.
A negative condition would be:
"Given the large size of 17 inch of the laptop, it is disappointing that it has a small battery of 40 Wh."
In the experiment, the battery capacity value (80 Wh) is masked and the participants task is to select an upper bound and a lower bound for the interval where the positive condition would be adequate, which would be e.g. 70 - 99 Wh. In contrast to this, for a battery capacity of 10 - 30 Wh, the positive condition would not be adequate. The opposite applies to the negative condition.
Given the examples, I collected interval estimations for all three conditions (positive, negative, neutral) for a certain number of participants.
In order to assess the influence of the condition on the estimated value, I performed an Anova significance tests between the three conditions' distributions of the lower bound and the upper bound separately. For this, I paired (pos, neut) (neut, neg) and (pos, neg) with the latter showing of course the largest difference in both lower and upper bound distributions. The results perfectly agree with my expectations.
Question 1: Is there a better way of evaluating the significance of the distributional differences? Should I compare the centers of the intervals (lower + upper / 2) instead of the upper and lower bounds separately (so only the distribution of one value per condition needs to be compared)?
Now, I would like to evaluate the fit of the estimated intervals with real values from a domain-specific database (technical data about laptops).
In the group of 12 inch laptops, there is a distribution over the feature battery capacity. This distribution has an average avg and the standard deviation std.
Question 2:
I would like to determine how well the distributions of the negative, the neutral and the positive conditions match with the average value in the database as well as the average +/- the standard deviation.
This means that for every condition (pos, neg, neut), we take the center of the intervals (lower bound + upper bound / 2), and compare the respective distribution for pos, neg and neut to the values avg+std, avg-std and avg respectively.
What is the best way to evaluative that fit?
