Skip to contents

mintR

mintR helps researchers go from a collection of texts to a topic model they can defend. It is built around the structural topic model (STM) of the stm package, a method that finds recurring themes in a large corpus (articles, speeches, archives, social media posts) and measures how much attention each theme receives, for instance over time or across authors. Fitting such a model is the easy part. The hard questions come before and after: which words and documents should the model see, how many topics are reasonable, do the topics mean something to a human reader, and would they come back if the model were fitted again?

The package covers this whole path. It prepares texts for the model, fits and compares many candidate models in parallel, computes the quality measures proposed in the literature, helps you read the topics, and organises tests in which human coders check them. It was written for research corpora of millions of words, so the text-processing functions are fast and economical with memory. Everything returns plain tables (data.table) or ggplots, so that results can be inspected, saved and reported like any other data.

Installation

You can install mintR from Codeberg:

# install.packages("remotes")
remotes::install_git("https://codeberg.org/agoutsmedt/mintR")

A first look

The example below uses the inaugural addresses of US presidents (from the quanteda package). Each address is cut into paragraphs, which become the documents of the model. extract_ngrams() counts the words and pairs of words of each paragraph, filter_vocabulary() drops the rare ones, and tokens_to_stm() turns the result into the input that stm() expects.

library(mintR)
library(stm)
library(data.table)

corpus <- quanteda::corpus_reshape(quanteda::data_corpus_inaugural, to = "paragraphs")
paragraphs <- data.table(quanteda::docvars(corpus), text = as.character(corpus))
paragraphs[, id := sprintf("%d_%s_%03d", Year, President, seq_len(.N)), by = .(Year, President)]
paragraphs <- paragraphs[stringi::stri_count_words(text) >= 40]

tokens <- extract_ngrams(paragraphs, ngrams = 2L, document_col = "id")
tokens <- filter_vocabulary(tokens, document_col = "id", min_documents = 5)

stm_data <- tokens_to_stm(
  tokens,
  meta = paragraphs[, .(id, Year, President)],
  document_col = "id"
)

Once the model is fitted with stm(), with the topic proportions allowed to change with the year, topic_labels() gives each topic a short name made of its most characteristic words:

fit <- stm(stm_data$documents, stm_data$vocab, K = 8,
           prevalence = ~ s(Year), data = stm_data$meta, seed = 1)
topic_labels(fit)
#>    topic                          label
#> 1:     1   1: citizens, country, fellow
#> 2:     2  2: government, people, public
#> 3:     3       3: law, government, laws
#> 4:     4      4: war, union, government
#> 5:     5     5: nation, america, people
#> 6:     6      6: nations, policy, peace
#> 7:     7 7: government, people, country
#> 8:     8       8: world, peace, freedom

Eight topics is an arbitrary choice here, and the guides below show how to justify one.

Guides

Five guides, written around the same example, follow the steps of a study. Each explains the idea behind the step before showing the code.

  • From texts to topics explains what a topic model is and how to prepare texts for it: how to split a corpus into documents, count words and expressions, weight them with tf-idf, choose a vocabulary, fit a model and read its topics.
  • Building the vocabulary goes through every choice that decides which words the model sees (documents, stop words, multiword expressions, frequency and tf-idf filters), with what the literature says about each and how to check their effect.
  • Choosing a topic model deals with the decisions that the data do not settle by themselves: the number of topics, the vocabulary, the stability of the results. It presents the usual quality measures and shows how to read them together rather than one at a time.
  • Validating topics with humans shows how to build intrusion tests, in which coders must spot the word or the topic that does not belong, how to hand them out, and how to score the answers, including the agreement between coders.
  • Reading a topic model shows how to explore a fitted model: labels and top words, representative documents, words of interest, correlations between topics, prevalence by group and over time, and models with a content covariate.

What is in the package

Preparing texts. clean_texts() removes links, handles and OCR hyphenation, detect_language() finds the language of each text (stop words, from get_stop_words(), are language-specific), and find_duplicate_texts() finds exact and near-duplicates. chunk_texts() cuts long documents (books, OCRed PDFs) into pieces of comparable size, following their paragraphs, and pool_texts() joins short ones (posts of the same thread or author).

From texts to tokens. extract_ngrams() splits texts into words and sequences of words, compute_tf_idf() tells which of them are specific to a document or a group of documents, filter_group_quantile() keeps the most specific ones, filter_vocabulary() applies the usual frequency rules and reports what each removes, and tokens_to_stm() builds the input of the model. Expressions such as “monetary policy” can be counted as single tokens rather than on top of their words: score_collocations() scores the candidate n-grams, select_collocations() applies a rule, and extract_ngrams(compounds = ) rewrites the texts with them.

Choosing a model. evaluate_topic_models() fits a whole grid of models, varying the vocabulary, the number of topics and the random start, and computes their quality measures. The fits run in parallel in background R processes, a failed fit does not stop the others, and an interrupted evaluation can resume where it stopped. The measures themselves are available one by one (topic_coherence(), topic_exclusivity(), topic_diversity(), topic_similarity(), topic_stability()), and plot_model_metrics() and plot_k_frontier() draw them to help you compare the candidates.

evaluation <- evaluate_topic_models(
  corpora = list(strict = stm_strict, loose = stm_loose),  # e.g. two vocabularies
  K = seq(20, 100, by = 10),
  prevalence = ~ s(year), max.em.its = 150,   # settings of the final model
  n_workers = 3,                  # models fitted at the same time: set it from memory
  output_dir = "data/evaluation"  # optional: save each fit and resume after a crash
)
plot_model_metrics(evaluation)
plot_k_frontier(evaluation)       # coherence against exclusivity

With one corpus and the default measures, evaluate_topic_models() returns the same statistics as stm::searchK() (this is tested), but fits the models in parallel. Peak memory is about n_workers times that of one fit.

Reading topics. A topic is a list of words, and several rankings can describe it: the most probable words, the most exclusive ones, and so on. topic_terms() and topic_labels() compute them, topic_documents() finds the documents to read for each topic, and topic_effects() with plot_topic_prevalence_over_time() shows how the attention to each topic evolves.

Validating with humans. make_word_intrusion(), make_word_set_intrusion() and make_topic_intrusion() create the questions, write_intrusion_sheets() prepares one sheet per coder, and read_intrusion_responses() and score_intrusion() turn the answers into scores with confidence intervals and measures of agreement.

The reference lists every function.

Speed

The text and topic-term functions were optimised and checked against their previous versions. bench/benchmark_functions.R compares them on 2,664 articles of the American Economic Review (JSTOR) and checks that results are unchanged (bench/results.csv). bench/benchmark_search_k.R compares parallel fitting (evaluate_topic_models()) with stm::searchK() (bench/results_search_k.csv): on 1,000 articles, K = 10 to 50 and 4 workers, 137 s against 169 s for searchK(cores = 4) and 352 s for searchK(), with 5-9% less peak memory than searchK(cores = 4) (measured with search_k(), the predecessor of evaluate_topic_models(), which had the same parallel design).

Licence: CC BY 4.0.