Number-of-topics criteria of Arun et al. (2010) and Deveaud et al. (2014)

Two criteria from the literature on LDA, computed as in the ldatuning package (Nikita 2020), which made them popular in the social sciences:

Arun et al. (2010): symmetric Kullback-Leibler divergence between the singular values of the topic-word matrix and the topic proportions weighted by document lengths. The authors argue it is minimal at the "natural" number of topics. Lower is better.
Deveaud et al. (2014): average divergence between pairs of topics (symmetrised Kullback-Leibler, as computed by ldatuning). Higher is better: topics are more distinct.

The criterion of Cao et al. (2009), the third of ldatuning, is the mean cosine similarity between topics: see similarity_mean in model_metrics() and topic_similarity().

These criteria were designed for LDA, with weaker justification than held-out likelihood or human validation, and are often noisy: treat them as secondary evidence.

Usage

ldatuning_metrics(model, documents, ...)

Arguments

model: A fitted stm model (or a list with beta$logbeta[[1]] and vocab).
documents: The documents the model was fitted on (stm format).
...: Not used: an argument renamed in the development version of mintR gives an error naming its replacement.

Value

A one-row data.table with arun_2010 and deveaud_2014.

References

Arun, R., Suresh, V., Veni Madhavan, C. E. and Narasimha Murthy, M. N. (2010). On finding the natural number of topics with latent Dirichlet allocation: Some observations. Advances in Knowledge Discovery and Data Mining (PAKDD 2010), 391-402.

Deveaud, R., SanJuan, E. and Bellot, P. (2014). Accurate and effective latent concept modeling for ad hoc information retrieval. Document numérique, 17(1), 61-84. doi:10.3166/dn.17.1.61-84

Nikita, M. (2020). ldatuning: Tuning of the Latent Dirichlet Allocation models parameters. R package. https://CRAN.R-project.org/package=ldatuning