Timeline for mlogit + logitr packages fail to recover true estimates of mixed logit random coefficient model
Current License: CC BY-SA 4.0
18 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 18, 2023 at 10:57 | history | edited | JediKnight | CC BY-SA 4.0 |
deleted 16 characters in body
|
| Sep 6, 2023 at 17:38 | comment | added | JediKnight | I replied in the logitr Github issue. | |
| Sep 5, 2023 at 10:21 | comment | added | jhelvy |
I'd like to see the exact code you're using to stress test logitr with different numbers and types of draws, etc. Continuing to probe there will help address the draws issue. Then I'd like to look at convergence criteria to see if that is related.
|
|
| Sep 5, 2023 at 10:20 | comment | added | jhelvy | github.com/jhelvy/logitr/issues/49 | |
| Sep 5, 2023 at 10:20 | comment | added | jhelvy |
Really puzzling why both mlogit and logitr aren't converging as quickly as Stata's cmmixlogit at even 100 draws. This might be a combination of 1) how draws are being simulated and 2) how the optimization algorithms determine a model has converged. For logitr I use NLopt, so it's not a custom optimizer. My guess is the Stata package has a hand-written optimizer made specifically to solve this problem. In any case, I'd like to get to the bottom of this, and Stackexchange really doesn't facilitate this too well. Would you mind moving some of the detail to the related logitr issue?
|
|
| Sep 3, 2023 at 12:02 | comment | added | JediKnight | Regarding the WTP estimates: Ok, I get your point regarding the distribution of the scale parameter. I will turn to that once I have figured out what is driving the bias in the preference space. | |
| Sep 3, 2023 at 12:00 | comment | added | JediKnight |
Referring to the textbooks by Cameron & Trivedi (Chapter 15.7.1) and Train (Chapter 10.5.1) my understanding is that the MSL estimator should be consistent as long as N – or in the panel setting NT – and the number of draws both converge to infinity. Additionally, if the number of draws converges faster than the square root of N / NT than the MSL estimator is also efficient. Increasing just NT only reduces the simulation noise, which I think can be seen by the density plots above. Still the open question remains why cmmixlogit achieves unbiasedness much faster than mlogit and logitr?
|
|
| Sep 3, 2023 at 11:58 | comment | added | JediKnight |
I also had a look how Stata’s cmmixlogit performs if I force it to just use 100 draws of either the Hammersley sequence (the default) or the Halton sequence. Both are unbiased than mlogit or logitr variants even for this number of draws.
|
|
| Sep 3, 2023 at 11:58 | comment | added | JediKnight |
Thanks for your reply @jhelvy! As requested, I additionally run simulations with number of draws equal to 100 and 200 and compared (i) mlogit with pseudo random numbers draws (the default) to mlogit with halton draws and (ii) logitr with halton draws to logitr with sobol draws. For number of draws equal to 200 there is no real difference detectable in the estimates among the two groups. For number of draws equal to 100, there is again no difference for the mlogit estimators but the sobol draw variant of logitr performs slight less than the halton variant.
|
|
| Aug 30, 2023 at 10:40 | comment | added | jhelvy | On the WTP estimates, I suspect this still may be the draws for the most part. For the ones where you're getting crazy bias levels on the SD parameters (~800%), I suspect that it is because you're modeling the scale parameter with a normal distribution, which really should never be done. The scale parameter must be strictly positive, and if some of the draws are negative then the model won't converge. I should probably not even allow that - I could just insert a check where the user cannot use 'n' for the randScale argument. | |
| Aug 30, 2023 at 10:33 | comment | added | jhelvy | Thanks for extending the analysis. I ran my code and also still got some bias at 200 draws, so clearly something is off. I still suspect this may have to do with the draws. I also expected that Halton would perform better (not worse) than pseudo-random numbers, but perhaps either my implementation of Halton is off or they just don't do as good a job as people thought. There is an option to use sobol draws in logitr - could you try changing drawType = 'sobol' and replicate your results for logitr? | |
| Aug 28, 2023 at 12:03 | history | edited | JediKnight | CC BY-SA 4.0 |
added 2 characters in body
|
| Aug 28, 2023 at 11:44 | history | edited | JediKnight | CC BY-SA 4.0 |
added 3213 characters in body
|
| Aug 24, 2023 at 13:57 | history | edited | JediKnight | CC BY-SA 4.0 |
added 2178 characters in body
|
| Aug 21, 2023 at 18:05 | answer | added | jhelvy | timeline score: 2 | |
| Aug 21, 2023 at 11:12 | comment | added | JediKnight | Just to add I am using R version 4.2.3 and the following package versions: mlogit (1.1.1), logitr (1.1.0) and EnvStats (2.8.0). | |
| S Aug 18, 2023 at 12:45 | review | First questions | |||
| Aug 18, 2023 at 13:06 | |||||
| S Aug 18, 2023 at 12:45 | history | asked | JediKnight | CC BY-SA 4.0 |