1
$\begingroup$

I have an ordinary differential equation based model for a system which depends on 16 parameters (all continuous and positive).

I have 10000 random sets of parameters where each set has 12 elements. Within each set, the value of the parameters are randomly sampled between 0.1 to 10x of a predefined value.

For example if the predefined set is [1, 2, 3, 4] (showing only 4 elements for simplicity), then each random set will have values from a uniform distribution in the ranges [0.01, 10], [0.02, 20], [0.03, 30] and [0.04, 40], respectively. Assuming, for simplicity, that any value lower then the predefined value is "low" and vice versa, then each set will have a random combination of high and low values of different parameters.

Now. I used these randomly sampled sets for simulating the ODE and calculate a metric — Q. With each parameter set, the ODE simulation returns a distinct value of Q. Finally, I get a distribution of Q for different parameter sets.

In the scatter plot shown below, I have retained the region of interest. The y-axis denotes the value of Q. The points highlighted in red denote "high" values of Q and those in black denote "low" values.

                             enter image description here

I know the parameter sets that generate the red dots and those that generate black dots. I want to see if there are specific patterns of parameter values that generate red dots that are distinct from those that generate black dots. I am assuming that this problem can be addressed using clustering.

I am not asking for software or tools that can be used for clustering. I wish to know if clustering can be at all used for this kind of analysis and if so, how should I proceed? How do I make a comparison between the "red sets" and the "black sets"?

I agree that this question is a little broad but I am fine with answers that point me to techniques/methods that are used to solve these kind of problems. I do not have much experience in this area and any clue about how to proceed would be helpful.


Edit:

Would binomial logistic regression be appropriate in this case?

$\endgroup$
6
  • $\begingroup$ What kind of space do the parameters belong to? How many parameters are there? $\endgroup$ – shadowtalker Jul 17 '15 at 19:45
  • $\begingroup$ @ssdecontrol The parameters are continuous positive numbers. There are, in total 16 parameters. $\endgroup$ – WYSIWYG Jul 18 '15 at 4:14
  • $\begingroup$ Is each axis in the plot a parameter? $\endgroup$ – shadowtalker Jul 18 '15 at 4:15
  • $\begingroup$ @ssdecontrol no. The plot depicts two metrics (on each axis) that are dependent on these parameters. I want to see what parameter patterns produce the subset of the results marked in red. I have a one to one map between parameter sets and the metrics. I just have to find out how different are the parameters that produce the different sets of output (And also find a pattern). $\endgroup$ – WYSIWYG Jul 18 '15 at 4:24
  • $\begingroup$ How complicated are those metrics? What kinds of functions are they? $\endgroup$ – shadowtalker Jul 18 '15 at 11:35
1
$\begingroup$

This is kind of a boilerplate answer. Hopefully someone with experience in similar problems will come along with some more insight.

Logistic regression would work fine as a first approximation, but keep in mind that linear regression can only capture linear effects. I would say to start there. Since you're looking for relative importance of the parameters, you could use regularization (eg ridge, lasso, or elastic net regression) to get a more principled idea of which parameters have the biggest effect on class. Otherwise you would have to run a Wald test on each parameter, or on groups of parameters, and control for multiple testing. If the results are understandable, you're done.

However this might also be a good place to look into a nonlinear or nonparametric approach, like a generalized additive model, or another spline model like MARS. Since they are based on splines, they're best used for interpolation instead of prediction, but since your "features" here are parameters selected from a grid, that's just fine. Another alternative would be a decision tree grown by an algorithm like CART, or even an ensemble of trees like in the Random Forest algorithm. These are difficult to use for determining the relative importance of predictors, but usually you can efficiently fit each model leaving out one predictor at a time, and then determine importance according to how much worse the model is when that predictor is excluded. A lot of programs for fitting these models include built-in functions/routines for this kind of thing. Another issue with these nonparametric models is that they usually need to be tuned in some way (eg with cross validation). That's why, if the regression results are satisfying, it's not worth digging into these methods

$\endgroup$
4
  • $\begingroup$ Thanks for the answer. If possible can you please provide some links to analyses in which the techniques you mention have been used to solve these kind of problems? $\endgroup$ – WYSIWYG Jul 19 '15 at 8:55
  • $\begingroup$ @WYSIWYG sadly no, because I don't have any experience with it myself. Like I said, it's a boilerplate answer based on what general knowledge I have $\endgroup$ – shadowtalker Jul 19 '15 at 16:15
  • $\begingroup$ Logistic regression says that no parameter is significant!! Perhaps the effects are non-linear. $\endgroup$ – WYSIWYG Jul 20 '15 at 12:14
  • $\begingroup$ @WYSIWYG they probably are, seeing as the metrics are nonlinearly related to the effects. Maybe an alternative strategy would be to better understand how each metric depends on the parameters, since you already know how the class depends on the two metrics. $\endgroup$ – shadowtalker Jul 20 '15 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.