Web context

The web tag in OpenAPI exposes three read operations:

GET /v1/webcontext/scrape accepts url plus optional selector filters and returns page content.
GET /v1/webcontext/map accepts url and discovers same-site URLs.
GET /v1/webcontext/crawl accepts url, max_pages, and max_depth for bounded same-origin crawling.

Use the API Reference for each operation's parameter and response schema.

Limiting scrape output

The optional max_chars parameter is an integer from 1 through 1048576. It counts Unicode code points, not bytes or model tokens; as a rough guide, there are 4 code points per English token. If a longer page is cut, it is cut at the last line break in the second half of the budget; otherwise it is cut at the last whitespace there; otherwise it is cut exactly at the budget. Trailing whitespace is trimmed and nothing is appended. data.truncated is a bool, data.markdown_chars is the length of the full page Markdown, and data.links / data.images are listed from the full page rather than the excerpt. The same price applies with or without max_chars. A non-integer or out-of-range value returns 400 invalid_request before any fetch and is not charged.

curl -sS -H "Authorization: Bearer $REPLYNODES_API_KEY" "https://api.replynodes.com/v1/webcontext/scrape?url=https://example.com&max_chars=2000"

Free markdown by URL (no API key)

A no-signup, zero-auth endpoint returns the content of one public page as markdown:

curl -sS https://md.replynodes.com/stripe.com

The request is a bare host with an optional path: https://md.replynodes.com/{domain} returns the home page and https://md.replynodes.com/{domain}/{path} returns a page inside it. The response is text/markdown; charset=utf-8 — the page's text with its headings, links, and lists kept — so a client can read or store it without an HTML parser.

Response headers carry the cache and quota state of the request: X-Cache (hit when the URL was served from cache, miss when the host had to fetch it — the query string is part of the cache key, so repeating a URL with a new query parameter produced a miss in testing), X-RateLimit-Limit (100), X-RateLimit-Remaining, X-RateLimit-Reset, and X-Request-Id.

One request is one page: this endpoint does not crawl, follow links, or paginate, so issue a request per page and use the webcontext operations above when you need crawl or map behaviour. A request it cannot serve is answered as the standard JSON error envelope instead of markdown — a host that does not resolve returns 400 invalid_request with a request_id — so check the status and the content type before storing a response.

HEAD describes the same page a GET returns. curl -I https://md.replynodes.com/stripe.com answers 200 with the same content-length and content-type the GET returns and no body, so a size or freshness preflight is safe. The quota still treats it as a request: every call, HEAD included, spends one unit of the rate limit.

Output formats

Choose the response format with headers; the query string belongs to the target URL.

Request headers	Response `Content-Type`	Body
none, `Accept: /`, `Accept: text/markdown`, `Accept: text/plain`	`text/markdown; charset=utf-8`	Markdown (unchanged)
`X-Format: frontmatter`	`text/markdown; charset=utf-8`	YAML front matter + Markdown
`Accept: application/json` (and no `X-Format`)	`application/json`	`{data, meta}` envelope
`X-Format: markdown`	as the default row	Markdown
`X-Format` with any other value	`application/json`	`400 invalid_request`
`Accept: application/json` and `X-Format: frontmatter`	`application/json`	`400 invalid_request` (conflicting)

curl -sS -H 'Accept: application/json' https://md.replynodes.com/stripe.com
curl -sS -H 'X-Format: frontmatter' https://md.replynodes.com/stripe.com

Each format spends one quota unit. Successful responses set Vary: Accept, X-Format.

The logo counterpart of this endpoint is documented under Brand intelligence; the product overview for this one is replynodes.com/markdown-api.

Selectors

Filter the page with request headers:

Request header	Meaning	Example
`X-Include-Selectors`	Keep only matching elements	`article,.post-body`
`X-Exclude-Selectors`	Remove matching elements	`nav,footer,.sidebar`

Selectors are comma-separated, with at most 10 entries and 512 bytes per header. Duplicates are removed, and control characters or { } ; < > are rejected. Violations return 400 invalid_request with reason: bad_option and spend no quota. A page's main heading may remain in the output even when it matches an exclude selector.

curl -sS -H 'X-Exclude-Selectors: nav,footer' https://md.replynodes.com/stripe.com

Filtered and unfiltered pages are cached separately. If a page returns an access-check page with selectors set, the response is 502 upstream_blocked because the crawl fallback is not used with selectors.

Errors

Every error is the standard JSON envelope with unchanged error.code and HTTP status, while two extra fields help a program decide what to do: error.reason is a stable value from the table, and error.retryable is true when sending the identical request again later can succeed without changing anything; it says nothing about how soon, which X-RateLimit-Reset / Retry-After provide.

HTTP	`error.code`	`error.reason`	Retryable	What to do
400	`invalid_request`	`empty_target`	`false`	Provide a target URL.
400	`invalid_request`	`bad_scheme`	`false`	Use `http` or `https`.
400	`invalid_request`	`malformed_url`	`false`	Fix the URL.
400	`invalid_request`	`target_too_long`	`false`	Shorten the URL.
400	`invalid_request`	`target_not_allowed`	`false`	Use a public target.
400	`invalid_request`	`target_rejected`	`false`	The fetcher rejected the URL; check that it is a valid public page.
400	`invalid_request`	`bad_option`	`false`	Fix the option request headers.
404	`not_found`	`route_disabled`	`false`	The free Markdown route is not enabled on this host; use `/v1/webcontext/scrape`.
404	`not_found`	`target_not_found`	`false`	The page does not exist; check the URL.
405	`method_not_allowed`	`method_not_allowed`	`false`	Use `GET` or `HEAD`.
429	`anonymous_limit_reached`	`daily_limit`	`false`	Wait for `X-RateLimit-Reset` or use an API key.
451	`source_excluded`	`publisher_excluded`	`false`	The publisher excludes automated access; contact them for licensing.
502	`upstream_blocked`	`access_check`	`false`	Try another page or contact the publisher.
502	`upstream_unavailable`	`target_unreachable`	`true`	Check the URL or try again later.
502	`upstream_unavailable`	`empty_content`	`false`	Try a text-based page; scanned PDFs and image-only pages have no text.
502	`upstream_unavailable`	`content_too_large`	`false`	Try a smaller page.
502	`upstream_unavailable`	`provider_unavailable`	`true`	Try again later.
503	`degraded`	`service_degraded`	`true`	Try again later.
499	`gateway_timeout`	`client_closed`	`true`	Your client closed the connection; keep it open for up to 33 seconds.
500	`internal_error`	`internal_error`	`true`	Try again later; report the `request_id` if it persists.
504	`gateway_timeout`	`target_timeout`	`true`	Try again later.

{ "error": { "code": "upstream_unavailable", "reason": "empty_content", "retryable": false,
             "message": "The page was fetched but has no extractable text, …", "request_id": "…" } }

Reason values are a closed set; new values may be added, but existing ones are never renamed or removed.

Cross-discovery and returning-caller CTA

Successful free responses include X-ReplyNodes-Discovery: brand and a Link relation to https://brand.replynodes.com/{domain} when the target host is known. A valid returning rn_anon_caller_id cookie also exposes X-ReplyNodes-CTA: api-key and a Link relation to https://platform.replynodes.com/sign-up. These headers are informational: the cookie is analytics-only, and the first free request remains keyless.