# Questions tagged [perplexity]

Perplexity is a measure uses to evaluate how well a probability model predicts a given test set. It is closely related to cross-entropy.

19 questions
Filter by
Sorted by
Tagged with
5 views

### Likelihood ratio test for language modeling

I am trying to use a likelihood ratio test to evaluate whether one language model is significantly better than another. (Note: In the example below, the language models should not be very different, ...
9 views

### Normalizing by root N in perplexity

In Chapter 3 N-gram Language Models by Jurafsky & Martin (pg 8), perplexity is defined as: The perplexity (sometimes called PP for short) of a language model on a test set is the inverse ...
43 views

### Perplexity for short sentences

I have a model that outputs short sentences and want to compare the quality of its outputs for different configurations by computing their perplexities using another model. I tried to use the gpt-2 ...
17 views

### Metrics used in evaluation of topic models

I know that perplexity requires a held out corpus. I am new to coherence measures. What are intrinsic and extrinsic coherence measures? Which one of them require an external corpus like Wikipedia ...
47 views

### Why does perplexity change with different ranges of k?

I ran a 5-fold cross-validation in R to calculate LDA perplexity for k = 2:9 using a 10% sample of my data. The output was: ...
523 views

### Why does larger perplexity tend to produce clearer clusters in t-SNE?

Why does larger perplexity tend to produce clearer clusters in t-SNE? By reading the original paper, I learned that the perplexity in t-SNE is $2$ to the power of Shannon entropy of the conditional ...
81 views

### Perplexity formula in the t-SNE paper vs. in the implementation

The perplexity formula in the official paper of t-SNE IS NOT the same as in its implementation. In the implementation (MATLAB): ...
101 views

### Perplexity and accuracy

I am a little bit unsure of the results I am getting from my skip-gram model, as it says that I have a very high perplexity (>100k) but that the accuracy is also more than 50%. It seems a bit ...
284 views

### Intuition behind perplexity parameter in t-SNE

While reading Laurens van der Maaten's paper about t-SNE we can encounter the following statement about perplexity: The perplexity can be interpreted as a smooth measure of the effective number of ...
2k views

### LDA and test data perplexity [closed]

I've performed Latent Dirichlet Analysis on a training set of documents. At the ideal number of topics I would expect a minimum of perplexity for the test dataset. However, I find that the ...
234 views

### Perplexity of a Non-Statistical Language Model

I have a piece of software that, given a input phrase, returns an ordered list of the next most likely words (entire vocab is ordered 1 to n). This is essentially an Language Model with the exception ...
595 views

### Perplexity calculation with neural nets

I am having troubles understanding which formula to use to calculate perplexity of a neural language model. Various places online on the forums people suggest using 2^(cross-entropy) measure, which is ...
448 views

### Why do I get weird results when using high perpexity in t-SNE?

I played around with the t-SNE implementation in scikit-learn and found that increasing perplexity seemed to always result in a torus/circle. I couldn't find any ...
5k views

### Inferring the number of topics for gensim's LDA - perplexity, CM, AIC, and BIC

I am confused as to how to interpret the LDA's perplexity fluctuations with different numbers of topics, in the endeavour of determining the best number of topics. Additionally, I would like to know ...
2k views

### How should perplexity of LDA behave as value of the latent variable k increases?

When increasing the value of the latent variable k for LDA (latent Dirichlet allocation), how should perplexity behave: On the training set? On the testing set?
1k views

### Language Model compare probability scores between Length varying sentence

My question is : How can I compare Language Model(LM) score for two sentences with different lengths ? Probabilities are < 1...
5k views

### Perplexity and cross-entropy for n-gram models

Trying to understand the relationship between cross-entropy and perplexity. In general for a model M, Perplexity(M)=2^entropy(M) . Does this relationship hold for all different n-grams, i.e. unigram, ...
Lets say I have, though some unspecified means, created a model that gives me a continuous distribution with a p.d.f $f_X$, for some variable $X$ I am modelling. And let us say that I have a set of ...