Skip to contents

Representative documents of each topic

Source: R/stm_helpers.R
topic_documents.Rd

The documents to read to understand a topic. Two methods:

  • "top": the documents with the highest proportion of the topic (what stm::findThoughts() gives). They show the topic in its purest form, but over-select documents with a narrow, repetitive vocabulary (templates, bots, lists).

  • "sampled": a random draw among the documents dominated by the topic (it is their largest topic, with a proportion of at least min_theta). They show what an ordinary clear case of the topic looks like. Read them together with "top".

Two filters make the list more informative, especially on social media or press corpora: text_col drops duplicated texts (same words once links, digits and punctuation are removed, see find_duplicate_texts()), and unique_col keeps at most one document per author (or outlet, institution...), so that a prolific author does not fill the list.

Usage

topic_documents(
  theta,
  meta = NULL,
  n = 5L,
  by = NULL,
  document_col = NULL,
  topics = NULL,
  method = "top",
  text_col = NULL,
  unique_col = NULL,
  min_theta = 0.3,
  seed = 1L,
  ...
)

Arguments

theta

Document-topic matrix (model$theta, documents in rows), or a fitted model.

meta

Optional metadata in the order of the rows of theta.

n

Integer scalar, number of documents per topic (and group and method).

by

Optional column name of meta to rank within.

document_col

Optional column name of meta identifying documents. Documents are always identified by their row (document_index).

topics

Integer vector of topics (default: all).

method

"top", "sampled", or both.

text_col

Optional column of meta with the texts: duplicated texts are kept once, and the text is returned.

unique_col

Optional column of meta (e.g. the author): at most one document per value, per topic (and group).

min_theta

Numeric scalar in [0, 1], minimal proportion of the dominant topic of a sampled document.

seed

Integer seed of the "sampled" draw (the global random state is restored after).

...

Not used: an argument renamed in the development version of mintR gives an error naming its replacement.

Value

A data.table with topic, by (if given), method, rank, document_index (row of theta), document_col (if given), theta and text_col (if given).

When to use it

To label and describe topics, read the top words (topic_labels(), topic_terms()) and documents: words can look coherent while the documents show that the topic mixes two themes, or is a genre (greetings, announcements) rather than a subject. Read at least the "top" documents of every topic before naming it, with "sampled" ones for the large topics; use by to rank within types of documents (so that short abstracts and long reports are not ranked together); quote these documents in a paper as examples of a topic.

Examples

theta <- matrix(c(0.9, 0.1, 0.2, 0.8, 0.6, 0.4), ncol = 2, byrow = TRUE)
topic_documents(theta, n = 2)
#>    topic method  rank document_index theta
#>    <int> <char> <int>          <int> <num>
#> 1:     1    top     1              1   0.9
#> 2:     1    top     2              3   0.6
#> 3:     2    top     1              2   0.8
#> 4:     2    top     2              3   0.4