I am running a large number of simulations in which I have a 3D parameter space, and for each set of parameters (point in the 3D space) I run 100 simulations. I then use 14 measures to quantify the results of the simulations, so for each point in parameter space I have the mean and standard deviation of each of these 14 measures.

I want to test which of the 14 measures are most useful for discriminating between my 3 initial parameters, so I'm using Principal Component Analysis. I can discriminate pretty well between all the mean values of my measures using 2 or 3 principal components. My question is, how do I convert the standard deviations on each of the measures at each point into principal component space so I can tell if they are well spaced enough to differentiate between the different parameters?

Also, I'm an astrophysicist, so please keep your maths clear and explanations free of jargon/unexplained symbols.

  • $\begingroup$ In mainstream statistics, parameters are unknown constants you are estimating, which does not sound like your situation, so this might be as hard for a non-astrophysicist to understand as what statistical people might say in return. $\endgroup$ – Nick Cox Apr 14 '16 at 14:20
  • $\begingroup$ Well actually later in my program they are unknows that I am estimating by comparing a real event to the simulations I have and seeing which simulations it resembles (using the PCA) so I can see what parameters best describe the real event. But if my terminology is confusing, I'm happy to edit it. What term would you suggest? $\endgroup$ – FJC Apr 14 '16 at 14:45
  • $\begingroup$ I can't be very constructive here because I couldn't follow your story. I see already a better response. $\endgroup$ – Nick Cox Apr 14 '16 at 14:59

I'm not sure to understand the specifics of your problem so I am gonna explain in the general context of sensometrics (in which we often have to estimate a variance-like parameter to see if products/descriptors are well discriminated in the space)

Consider a matrix $X$ of dimensions $r\times c$, you can create a space out of the eigenvectors of $X$ and have your $r$ points in it. What you can do to estimate "variance" for those $r$ points is use a resampling method like bootstrap. With that method you can generate $n$ matrices $X^{(i)}$ with $i \in [1, n]$. If you project those matrices onto your space (as supplementary individuals/variables) you can have for each $r$ points a cloud of $n$ points.

Something we do in sensometrics to decide if two points (from the initial $X$ matrix) are well-discriminated is this : plot for each of the $r$ points an ellispoid in which there is at least 95% (or any other value) of the $n$ points associated. If two ellispoids are overlaping then it may be a sign that they are not that different.

Either way, you generated a $n + 1$ sample for each of the $r$ points so you can estimate mean and variance and conduct comparison test such as Student or Hotelling.

Hope this helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.