Skip to main content
deleted 153 characters in body
Source Link
russellpierce
  • 19k
  • 17
  • 77
  • 104

A quick glance suggests that this is over my head... but here I go anyway.

In terms of options the docs make it seem like if you had a prior for Hstart things might go a little faster. Things also look like they might go faster if binned, but I get the sense that binning is forced off for if ncol(dat) > 4. However at four (as in your text, but not the purported author's code) it looks like you could turn on binning. The consequences of this are entirely unknown to me.

Manually debugging the code... I saw that most of the time seems to be lost in Hscv (at least for k < 6 (where my computer will finish this month)) during calls in that function to Hpi, gamse.scv, and ultimately nlm. nlm is being passed around as an optimization function. No nasty loops or easily paralleled apply statements jumped out at me (which didn't/doesn't mean much). Both nlm and optim (the other choice mentioned in the docs) are .Internal, so I don't think we'll speed them up by much.

Trying Rprof: Poking way beyond my level of familiarity, I tried using Rprof in utils to profile the code. I think that in the package ks the function dmvnorm.deriv But, I got nowhere.sum is taking up a fair bit Most of time... probably doingthe crunching that is happening is related to matrix multiplication. If I'm reading the code right this line of code in dmvnorm.deriv.sum...

<wrong line>

might benefit from being turned into a parallel function across multiple cores. I haven't used it myself yet, but I understand that R 2.14 has 'parallel' built in with parApply, parCapply, and parRapply functions. Perhaps If you can find a high enough order non-serial loop or apply statement, perhaps replacing the existing calls to apply with thosebe parallel would help. However, and not just load you down with overhead may wipe away the advantage.

Perhaps someone with a beefer computer than mine, experience in R 2.14, and/or more experience profiling can tell you more.

A quick glance suggests that this is over my head... but here I go anyway.

In terms of options the docs make it seem like if you had a prior for Hstart things might go a little faster. Things also look like they might go faster if binned, but I get the sense that binning is forced off for if ncol(dat) > 4. However at four (as in your text, but not the purported author's code) it looks like you could turn on binning. The consequences of this are entirely unknown to me.

Manually debugging the code... I saw that most of the time seems to be lost in Hscv (at least for k < 6 (where my computer will finish this month)) during calls in that function to Hpi, gamse.scv, and ultimately nlm. nlm is being passed around as an optimization function. No nasty loops or easily paralleled apply statements jumped out at me (which didn't/doesn't mean much). Both nlm and optim (the other choice mentioned in the docs) are .Internal, so I don't think we'll speed them up by much.

Trying Rprof: Poking way beyond my level of familiarity, I tried using Rprof in utils to profile the code. I think that in the package ks the function dmvnorm.deriv.sum is taking up a fair bit of time... probably doing matrix multiplication. If I'm reading the code right this line of code in dmvnorm.deriv.sum...

<wrong line>

might benefit from being turned into a parallel function across multiple cores. I haven't used it myself yet, but I understand that R 2.14 has 'parallel' built in with parApply, parCapply, and parRapply functions. Perhaps replacing the existing calls to apply with those would help. However, overhead may wipe away the advantage.

Perhaps someone with a beefer computer than mine, experience in R 2.14, and/or more experience profiling can tell you more.

A quick glance suggests that this is over my head... but here I go anyway.

In terms of options the docs make it seem like if you had a prior for Hstart things might go a little faster. Things also look like they might go faster if binned, but I get the sense that binning is forced off for if ncol(dat) > 4. However at four (as in your text, but not the purported author's code) it looks like you could turn on binning. The consequences of this are entirely unknown to me.

Manually debugging the code... I saw that most of the time seems to be lost in Hscv (at least for k < 6 (where my computer will finish this month)) during calls in that function to Hpi, gamse.scv, and ultimately nlm. nlm is being passed around as an optimization function. No nasty loops or easily paralleled apply statements jumped out at me (which didn't/doesn't mean much). Both nlm and optim (the other choice mentioned in the docs) are .Internal, so I don't think we'll speed them up by much.

Trying Rprof: Poking way beyond my level of familiarity, I tried using Rprof in utils to profile the code. But, I got nowhere. Most of the crunching that is happening is related to matrix multiplication. I haven't used it myself yet, but I understand that R 2.14 has 'parallel' built in with parApply, parCapply, and parRapply functions. If you can find a high enough order non-serial loop or apply statement, perhaps replacing the existing calls to be parallel would help and not just load you down with overhead.

Perhaps someone with a beefer computer than mine, experience in R 2.14, and/or more experience profiling can tell you more.

Source Link
russellpierce
  • 19k
  • 17
  • 77
  • 104

A quick glance suggests that this is over my head... but here I go anyway.

In terms of options the docs make it seem like if you had a prior for Hstart things might go a little faster. Things also look like they might go faster if binned, but I get the sense that binning is forced off for if ncol(dat) > 4. However at four (as in your text, but not the purported author's code) it looks like you could turn on binning. The consequences of this are entirely unknown to me.

Manually debugging the code... I saw that most of the time seems to be lost in Hscv (at least for k < 6 (where my computer will finish this month)) during calls in that function to Hpi, gamse.scv, and ultimately nlm. nlm is being passed around as an optimization function. No nasty loops or easily paralleled apply statements jumped out at me (which didn't/doesn't mean much). Both nlm and optim (the other choice mentioned in the docs) are .Internal, so I don't think we'll speed them up by much.

Trying Rprof: Poking way beyond my level of familiarity, I tried using Rprof in utils to profile the code. I think that in the package ks the function dmvnorm.deriv.sum is taking up a fair bit of time... probably doing matrix multiplication. If I'm reading the code right this line of code in dmvnorm.deriv.sum...

<wrong line>

might benefit from being turned into a parallel function across multiple cores. I haven't used it myself yet, but I understand that R 2.14 has 'parallel' built in with parApply, parCapply, and parRapply functions. Perhaps replacing the existing calls to apply with those would help. However, overhead may wipe away the advantage.

Perhaps someone with a beefer computer than mine, experience in R 2.14, and/or more experience profiling can tell you more.