Crawl API

Endpoint

GET query.php?q=<host-or-url>&page=<number>&languages=<csv-iso639-3>

The parameter q can be a hostname (e.g., example.com) or a full URL; the host part will be used for lookup. The optional page parameter (default 1) returns results in pages of 50. The optional languages parameter accepts a comma-separated list of three-letter ISO 639-3 codes (e.g., eng,deu,fra) and filters results to rows where content_languages contains at least one of the provided codes. If omitted, no language filter is applied.

Example:

Curl:

The response is JSON containing matching index rows. For convenience, this demo decodes and shows the response below.

Status

Response

Click "Try the API" to fetch a live response from the server.

(no response yet)

Notes

Response includes warc_content Base64-encoded; this page does not decode it automatically.
You can filter by language using languages, e.g., ?languages=eng,deu. If omitted, all languages are returned.
Errors are returned as JSON with { "error": "..." } and an appropriate HTTP status.