Characterizing the distribution of manufacturing data I have some data from manufacturing concerning the variability of certain geometric dimensions of a given machine. These are real numbers (diameters, angles, lengths, etc.), so the variables are continuous. I need to perform an uncertainty quantification study of the performance of the machine: I want to propagate the manufacturing process variability across the chain of codes which are used to compute the performance of the machine, in order to estimate the uncertainty in the machine performance. I thus need to perform a Monte Carlo analysis (or Polynomial Chaos, or Gaussian Process - pick your favourite tool). In a way or the other, I need to be able to sample from a distribution. The distribution is not normal, nor it can reasonably be approximated by a normal distribution: see
preceding discussion
Anderson-Darling test for normality with estimated parameters
I've been advised to try to characterize the data distribution. How do I do that? Is there a standard procedure, or can you suggest useful references? I'm not an expert (as you probably can tell), so I need something simple. I can code in R, I am familiar with the Monte Carlo standard error and I can run hypothesis tests (for example, I understand when to reject normality based on the result of an Anderson-Darling test). I don't know about more advanced stuff, but I'm a good learner, though :)  
PS I imagine that characterizing the data distribution means to fit different distributions to my data and find the best fitting one. However, I've heard that this is a kind of data-dredging (I guess because fitting would favor distributions with more parameters), so I imagine there's more to this problem.
 A: After failing to fit the distribution that theory suggests the data should follow then you must ask why does this data not fit.  For example, the angle on your machine may follow more of an exponential distribution, since they will target a more acute angle and bend until it hits the low end of the tolerance.  This is because a plate is much easier to bend than to straighten.  If you still cannot find a distribution, then maybe your data is correlated to some other variable outside of your dataset.  If this still does not yield an answer then I would let the data determine the distribution.
This is the process I would follow after the above has been exhausted:


*

*Find the range of the variable

*Divide the range into equal sections

*Calculate the frequency of each section

*Plot the frequencies in a histogram


This should give you a visual representation of the shape of the distribution.  Compare this image to several distribution pdfs.  Once you have found one that looks like a match then you can test the goodness of fit by using the Chi-squared test.  Here is some guidance on the test:
https://onlinecourses.science.psu.edu/stat414/node/259
When checking a distribution, calculate the mean and variance and other parameters of the proposed distribution using your dataset; so you can generate a sample distribution to test.  For every parameter you estimate from your data, you must subtract one degree of freedom from your Chi-squated test statistic.  Wikipedia is a good source for calculating the parameters of a distribution.
