Scraping OLX Indonesia past Akamai: architecture notes and a 25-row sample dataset

OLX Indonesia (olx.co.id) is one of the largest classified marketplaces in Southeast Asia, but it is also one of the harder Indonesian sources to collect reliably. Search and listing pages sit behind an Akamai interstitial, and the data you want is not in the initial HTML — it is fetched by the page's own JavaScript after the challenge is cleared.

This article documents the architecture that worked for me, the normalized schema I settled on, and a small real sample dataset you can inspect. It is written for data engineers who need Indonesian marketplace data and for anyone evaluating whether a scraper is worth building.

Why plain HTTP and headless browsers fail

The first attempts were the obvious ones:

• plain fetch / requests — returned the interstitial or an empty shell;
• server-side rendered parsing — the listing JSON was not in the document;
• headless Chrome without a real browsing context — the challenge sometimes passed, but the listing request was never captured correctly.

The pattern is familiar: the challenge is not the final boss. The final boss is the JSON request the page makes after the challenge, and that request only exists if the page is allowed to run like a normal browser.

What worked: run a real browser, intercept the response

The working approach was:

1. Launch a real browser context with a normal desktop fingerprint.
2. Navigate to the public search URL.
3. Let the page solve the interstitial naturally — no captcha-solving service, no login.
4. Intercept the JSON response the page's own frontend fetches.
5. Normalize the response into a flat record.

A simplified sketch:

page.on('response', async (response) => {
  const url = response.url();
  if (!url.includes('/api/') && !url.includes('listing')) return;
  const type = response.headers()['content-type'] || '';
  if (!type.includes('application/json')) return;

  const payload = await response.json();
  // map payload -> normalized listing
});

The important part is not a secret endpoint. It is the order of operations: real browser → challenge → page JavaScript → response interception. A proxy alone does not fix this; a different user agent does not fix this; parsing harder does not fix this.

The schema

I normalized each listing into a flat record. The exact fields depend on the marketplace, but this shape has been stable:

Two design choices worth calling out:

• Keep the marketplace ID as the stable key. Titles change, prices change, and listings get republished. adId is what lets you deduplicate across runs.
• Separate the display price from the numeric price. priceDisplay is useful for QA; price is what your pipeline should aggregate.

The sample dataset

I published a real 25-row snapshot (not synthetic) collected on 3 November 2026 from a public search for mechanical keyboard:

• JSON: https://gist.githubusercontent.com/FauziMatakupan/0b420c5dea9a55677db65cff79dac196/raw/olx_indonesia_sample_2026-11-03.json
• CSV: https://gist.githubusercontent.com/FauziMatakupan/0b420c5dea9a55677db65cff79dac196/raw/olx_indonesia_sample_2026-11-03.csv
• Schema notes: https://gist.github.com/FauziMatakupan/0b420c5dea9a55677db65cff79dac196

The sample omits seller names and exact coordinates. Descriptions are seller-written and may contain typos. Prices are asking prices, not transaction prices. It is a schema sample, not a representative market sample.

Limitations and ethics

• Public data only. No login, no private messages, no seller contact details.
• Respect the platform. Read the terms of service and applicable data-protection rules (UU PDP in Indonesia). Do not republish personal data.
• Expect drift. Classified sites change their frontend often. A schema that survives a redesign is worth more than a clever selector.
• Small samples lie. Twenty-five rows can validate a schema; they cannot estimate a market.

What I packaged

I turned this architecture into an Apify Actor so the browser lifecycle, challenge handling, pagination, and normalization are reusable:

https://apify.com/matakupanlabs/olx-indonesia-listings-scraper

Disclosure: this is my Actor and it is pay-per-listing. The sample dataset and schema notes are free. If you are building something similar, I am happy to compare notes on fields, deduplication, or handling large runs.

What would you add?

For marketplace price monitoring, the interesting question is time: is createdAt enough, or do you need first-seen / last-seen fields to build a real price time series? I would like to hear how other people model republished listings and seller churn.