Dev.to WebDev πŸ›  Dev πŸ‘ 0 πŸ“– 3 min read

Let your x402 agent read web pages: from a free test call to a paid call

Disclosure: I'm on the WebExtract team. This post was drafted with AI assistance, and the commands below were run by hand before publishing. WebExtract is our own product. If you're building an agent that has to read a

Disclosure: I'm on the WebExtract team. This post was drafted with AI assistance, and the commands below were run by hand before publishing. WebExtract is our own product.

If you're building an agent that has to read arbitrary web pages, there are three things to settle: how to try the API without an account, how the payment handshake works, and how to pay from code. This walkthrough covers all three using an x402-enabled endpoint, WebExtract, which turns a URL into clean Markdown (and, on the other endpoint, structured JSON). It costs $0.002 per call for Markdown and $0.005 per call for the structured extract.

1. Try it without a wallet

REST is opt-in for the free tier. Add one header and you get 20 free calls a day per IP, no signup:

curl -sS -G -H "X-Billing: free" "https://api.webextract.dev/v1/markdown" \
  --data-urlencode "url=https://example.com"

You get JSON back with title, markdown, word_count and the final URL. The x-free-remaining response header tells you how many free calls are left today. Failed fetches are not charged.

2. See what the server wants for payment

Call the endpoint without the header:

curl -sS -i "https://api.webextract.dev/v1/extract?url=https%3A%2F%2Fexample.com"

You get HTTP 402. The x402 v2 payment requirements are in the PAYMENT-REQUIRED response header, base64-encoded JSON. Decode it (the base64 -d is the whole trick):

curl -sS -D - -o /dev/null "https://api.webextract.dev/v1/extract?url=https%3A%2F%2Fexample.com" \
  | grep -i '^payment-required:' | cut -d' ' -f2- | tr -d '\r\n' | base64 -d | python3 -m json.tool

When I ran it, accepts listed seven USDC options, Base first, then Polygon, Arbitrum, Avalanche, Optimism, World Chain and Monad. Every one has amount: "5000", which is $0.005 because USDC has 6 decimals. The response also carries a Link: rel="payment" header and an X-Free-Tier-Hint header pointing back at the free option, so an agent that hits the 402 first can find its way.

3. Pay from code (testnet first)

The official Node packages handle the 402, signing and retry. Use a throwaway wallet and the sandbox, which only accepts Base Sepolia test USDC (free from Circle's faucet):

npm i @x402/fetch @x402/evm viem
import { wrapFetchWithPaymentFromConfig } from "@x402/fetch";
import { ExactEvmScheme } from "@x402/evm";
import { privateKeyToAccount } from "viem/accounts";

const account = privateKeyToAccount(process.env.TESTNET_PRIVATE_KEY); // throwaway test wallet
const pay = wrapFetchWithPaymentFromConfig(fetch, {
  schemes: [{ network: "eip155:84532", client: new ExactEvmScheme(account) }],
});
const res = await pay("https://api.webextract.dev/sandbox/v1/markdown?url=https://example.com");
console.log(res.status, (await res.json()).markdown);

Our own run of this pattern returned HTTP 200 with a settled 0.002 USDC payment on Base Sepolia (transaction).

Two things we ran into with the SDK (version 2.28):

  • Registering eip155:* makes it pick the first option the server lists, which is Base, wherever your USDC actually is. If your USDC is only on Polygon, register just eip155:137 or pass your own payment-requirements selector.
  • Optimism (eip155:10) and World Chain (eip155:480) are rejected by the SDK's default spend controls because it has no default USDC asset for them. To pay there, add spendControls: { allowedAssets: [...] } for that network.

4. Go to mainnet

Point the same code at https://api.webextract.dev/v1/markdown (drop /sandbox) and register the mainnet network where you hold USDC, for example eip155:8453 for Base. Start with a wallet that holds a few dollars, not your main one.

5. If you'd rather not run a wallet

  • MCP: add https://api.webextract.dev/mcp as a remote server in Cursor, VS Code or Claude Desktop (via mcp-remote). The tools are extract_url and page_markdown, and the free tier works through MCP without any header. x402 payment is not available through MCP; after the free calls run out, the tool tells you to switch to REST or credits.
  • Prepaid credits: $1 = 200 credits by card, then send Authorization: Bearer <key>. Details at webextract.dev.

JS-heavy pages: use /v1/render/extract ($0.008 per call) or /v1/render/markdown ($0.004 per call); each IP gets 5 free renders a day.

What I'd like feedback on

If the 402 handshake is confusing for your agent framework, tell me where it breaks: the client library, the headers, or the docs. That's the part we're still improving.

πŸ“° Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.