Beyond Single-Command Recon: Building Async Multi-Stage Pipelines

Running one tool once isn't recon. Here's the asynchronous, multi-stage pipeline I use to turn a domain into structured, triage-ready telemetry.

A hard truth about bug bounty and attack-surface management: running subfinder once, piping it into httpx once, and calling it a day is not reconnaissance. The low-hanging fruit a single-command scan finds already has a thousand bots hitting it within 30 seconds of an asset coming online.

Your edge is in multi-stage, context-aware pipelines — chaining crawling, parameter mining and reflection analysis into one structured stream instead of ten messy text files.

The 4-stage engine

[ target domain ]
      |
      v
1. Async crawler (katana / headless BFS)
   endpoints, inline JS bundles, forms, webpack manifests
      |
      v
2. Parameter discovery (arjun)
   unadvertised GET/POST/JSON params and query keys
      |
      v
3. Context + reflection engine (dalfox / XSStrike-go)
   probes reflections, parses DOM sinks and filter behaviour
      |
      v
4. Structured telemetry (Pydantic schema)
   emits a machine-readable recon_report.json

The tools are all open source and Go/Python-native:

• katana: https://github.com/projectdiscovery/katana
• arjun: https://github.com/s0md3v/Arjun
• dalfox: https://github.com/hahwul/dalfox
• XSStrike-go: https://github.com/Sakura-501/XSStrike-go

Why monolithic scanners miss what pipelines catch

1. Single-page apps. Plain HTTP scrapers don't execute client-side JavaScript. A headless crawler pulls routes that only load via chunk.js and webpack runtime manifests.

2. Hidden parameters. Developers ship debug flags and feature toggles (?debug=true, ?preview_token=) that never appear in the UI. A parameter miner finds them before you send a single payload.

3. Context-aware injection. Firing a classic alert(1) payload at every input just trips the WAF. A good pipeline first checks where the input reflects:

• Inside an HTML attribute such as value="..."? You need to break out of the quotes first (a crafted " onfocus= sequence).
• Inside a script block? You need a string escape or a tag close.
• Inside a JSON response? You need quote and brace escaping.

The takeaway

Don't write one-off bash that dumps noise into ten files. Build small, modular pipelines that parse real models — Page, Form, JSFile, Finding. When your recon output is clean and structured, finding the bug becomes methodology instead of luck.

How do you orchestrate recon — chained CLI tools, or a unified Python/Go controller? Tell me in the comments.