With the option politecollect.log_file set, polite_request() writes one
row per request sent: time, site, method, URL (keys removed), status,
duration and reason of the failure. read_request_log() reads that file and
request_log_summary() counts requests by site and kind of failure.
Value
read_request_log(): a data.table with one row per request and a
column kind (failure_kind(), "ok" for a success).
request_log_summary(): a data.table with domain, kind, n and the
median duration in seconds.
Details
A log is the answer to two questions that come up late in a project: what was requested, when, and with what result (the provenance of the collection); and was the server treated properly (the number of requests per site and of errors received).
See also
Other requests:
failure_kind(),
normalise_url(),
polite_get()
Examples
log <- tempfile(fileext = ".csv")
data.table::fwrite(
data.table::data.table(
at = "2026-11-09 10:00:00", domain = "https://a.org", method = "GET",
url = c("https://a.org/1", "https://a.org/2"), status = c(200L, 404L),
duration_s = c(0.5, 0.2), reason = c(NA, "HTTP 404")
),
log
)
request_log_summary(log)
#> domain kind n median_duration_s
#> <char> <char> <int> <num>
#> 1: https://a.org ok 1 0.5
#> 2: https://a.org permanent 1 0.2