Topic modeling (LDA) gives different outputs I am using Topic Modeling Tool which is based on Mallet and using latent dirichlet allocation (LDA). When I ran the tool multiple times, with the same input (a folder of 200-500 short text files), and set Topics = 10 with the default of 10 words printed per each topic, the generated 10 topics and the words in the output changes.
So am i supposed to reran it multiple times and how can I pick one?
Does it matter which tool of topic modelling I use? I found a few other tools in my search including Stanford Topic Modeling Toolbox, Natural Language Toolkit, etc.
 A: As Cliff notes in comments, the objective function for LDA is non-convex, making it a multimodal problem. That is, you can expect any given run to be locally optimal; you cannot expect that any given run would outperform some other run from different starting points. 
To choose between multiple runs, consider this from another paper on variational methods by LDA creators David Blei and Michael Jordan (emphasis mine):

Practical applications of variational methods must address initialization of the variational distribution. While the algorithm yields a bound for any starting values of the variational parameters, poor choices of initialization can lead to local maxima that yield poor bounds. We initialize the variational distribution by incrementally updating the parameters according to a random permutation of the data points. (This can be viewed as a variational version of sequential importance sampling). We run the algorithm multiple times and choose the final parameter settings that give the best bound on the marginal likelihood.

They were writing about Dirichlet process mixtures, but the principle of selecting the run that best predicts the data carries. Also consider selecting based on perplexity of a held-out test set.
A: In the c implementation of D. Blei hyperparameters are optimized using an Expectation maximization algorithm that is initialized by n random document seeds.
This could produce slightly different results...try to set the hyperparameters fixed and see whether you get the same results... 
Regarding re-running the algorithm several times (with slightly different settings of the hyperparameters):
You can measure the similarity between the topics in the different iterations in order to measure "how much" the topics differ from each others. If in 2 consecutive iterations you find 4 pairs of topics with a good similarity (~90%?) you can consider the topics consistent.
Choosing the right similarity metric is another question....
In principle you can use any similarity (or dissimilarity) function between histograms (Jeffrey divergence?), or be elengant: http://www.machinelearning.org/proceedings/icml2007/papers/105.pdf
