Prepare a rate-limited API request

Builds an httr2::request() for a scholarly API, with its rate, retries and timeout. It is not sent: give it to polite_request() with robots = FALSE, or to httr2::req_perform_parallel() for APIs that allow several requests at once. The clients of the package (crossref_request(), scopus_request(), openalex_request()) are built on it.

Usage

api_request(
  base_url,
  path = NULL,
  query = list(),
  headers = list(),
  secret_headers = character(),
  rate = NULL,
  realm = NULL,
  max_tries = 6L,
  backoff = function(i) min(60, 2^i),
  transient = c(429L, 500L, 502L, 503L, 504L),
  timeout = 180,
  user_agent = getOption("politecollect.user_agent")
)

Arguments

base_url

Character. Root of the API, or a complete URL.

path

Character or NULL. Route appended to base_url.

query

Named list of query parameters. NULL elements are dropped.

headers

Named list of headers.

secret_headers

Character. Names of the headers holding keys, which httr2 hides when printing.

rate

Numeric or NULL. Maximum requests per second, spaced evenly.

realm

Character or NULL. Throttling group; by default the host.

max_tries

Integer. Attempts per request.

backoff

Function of the attempt number returning the seconds to wait.

transient

HTTP codes worth retrying, or a function of a response.

timeout

Numeric. Seconds before the request is abandoned.

user_agent

Character. By default the option politecollect.user_agent.

Value

An httr2_request.

Details

The throttle is a bucket of one token, refilled every 1 / rate seconds, so that requests are evenly spaced. httr2's default bucket of 60 seconds would allow a burst of 60 * rate requests at once.

See also

Other APIs: api_paginate(), crossref_request(), ia_search(), openalex_api_key(), scopus_api_key(), sparql_query()

Examples

req <- api_request(
  "https://api.example.org", "works",
  query = list(q = "money"), rate = 2, user_agent = "my_project (me@example.org)"
)
req$url
#> [1] "https://api.example.org/works?q=money"