Detect the language of texts — detect

Stop words, collocations and topics are language-specific: a corpus that mixes languages gives topics made of the other languages (the Spanish, German and French speeches of a central bank corpus each formed a topic). detect_language() gives the language of each text, to filter the corpus or to give each text the stop words of its language (language_col of extract_ngrams()).

Usage

detect_language(
  data,
  text_col = "text",
  method = c("cld2", "stopwords"),
  min_words = 3L,
  ...
)

Arguments

data: A character vector of texts, or a table with text_col.
text_col: Character scalar, the text column when data is a table.
method: "cld2" (Google's Compact Language Detector 2, package cld2: fast, 80+ languages, needs some words) or "stopwords": the language whose snowball stop words make up the largest share of the text's words (15 European languages, no extra package; reliable on texts of a few dozen words, not on short posts).
min_words: Integer scalar: texts with fewer words get NA.
...: Not used: an argument renamed in the development version of mintR gives an error naming its replacement, any other one an error.

Value

A character vector of ISO 639-1 codes (NA when undecided), one per text. Links and handles are removed before detection.

Examples

detect_language(c(
  "The central bank raised its policy rate to curb inflation.",
  "La banque centrale a relevé son taux directeur pour freiner l'inflation."
), method = "stopwords", min_words = 3)
#> [1] "en" "es"