π― ffuf Content Discovery: Cut the False Positives, Prove the Hits
You point ffuf at a target with a decent wordlist, and thirty seconds later the terminal is a wall of green: hundreds of 200 OK lines. /admin, /backup, /config, /.git. It looks like the jackpot. Often it isn't. Often ev
You point ffuf at a target with a decent wordlist, and thirty seconds later the terminal is a wall of green: hundreds of 200 OK lines. /admin, /backup, /config, /.git. It looks like the jackpot.
Often it isn't. Often every one of those lines is the same page: a single-page app's shell, a custom "not found" template served with status 200, or a login redirect. The tool did exactly what you asked. You just asked the wrong question.
This article is the ffuf workflow I'd teach someone on day one: how content discovery works, why it produces so much noise, how to filter that noise with ffuf's own options, and how to prove that a hit is real before it goes anywhere near a report.
Only run this against applications you own or are explicitly authorized to test, and respect the program's rate limits. Every example uses stand-ins like app.example.com.
What content discovery is
Content discovery (also called forced browsing or directory brute-forcing) means requesting paths that aren't linked anywhere, to find what the server will serve anyway: old backups, admin panels, debug endpoints, API versions nobody removed. OWASP's testing guide covers it under WSTG-CONF-04, reviewing old, backup and unreferenced files.
ffuf is a fast web fuzzer written in Go. You put the keyword FUZZ wherever you want words from a list to go, and ffuf sends one request per word:
ffuf -u https://app.example.com/FUZZ \
-w ~/wordlists/SecLists/Discovery/Web-Content/common.txt
FUZZ can go in the URL, in a header (-H) or in POST data (-d), which is why the same tool also does parameter and virtual-host fuzzing. The wordlist here is common.txt from SecLists, a good small starting list.
Why you get so many fake hits
By default ffuf keeps any response whose status code is in this list: 200-299, 301, 302, 307, 401, 403, 405, 500. That's in both the README's usage section and the option parser in the source. It's a sensible default for a classic server that returns a real 404 for missing paths. Modern apps often don't:
-
Catch-all routes. Single-page apps return the same
index.htmlwith status 200 for every path, so the client-side router can decide what to show. - Soft 404s. A custom "page not found" template, served with 200.
-
Blanket 403s and redirects. A WAF or reverse proxy that blocks everything it doesn't recognize, or an auth layer that sends every unknown path to
/login.
In all three cases, the status code tells you nothing. What tells you something is the shape of the response: its size, word count and line count. If 900 different words all produce a 200 with exactly the same number of words and lines, that's one page, 900 times.
Filter the noise
Find the baseline by hand
Before tuning anything, request a path that can't exist and look at what comes back:
curl -s -o /dev/null -w '%{http_code} %{size_download}\n' \
https://app.example.com/this-does-not-exist-8f3a1c
If that says 200 4242, you know what "not found" looks like on this target.
Filter on size, words or lines
ffuf has a filter for each property of the response:
| Flag | Filters out responses with⦠|
|---|---|
-fc |
these status codes (-fc 404,429) |
-fs |
this size in bytes (-fs 4242) |
-fw |
this many words |
-fl |
this many lines |
-fr |
a body matching this regex (-fr "Page not found") |
Each accepts a comma-separated list, and the numeric ones accept ranges. Word and line counts are often better than size: if the catch-all page reflects the requested path back into the HTML, its size changes with every word, but its word and line counts usually don't.
ffuf -u https://app.example.com/FUZZ -w common.txt -fw 312
Or let ffuf calibrate itself
-ac turns on auto-calibration. Before the run, ffuf requests a handful of random paths built to be nonexistent (random strings, and variants like admin or .htaccess followed by random characters), looks at what the server returns, and adds a filter for it. Reading the calibration code, it tries the most specific match first: if the calibration responses share a size, it filters that size; if not, the word count; then the line count.
ffuf -u https://app.example.com/FUZZ -w common.txt -ac
Two cautions. First, calibration only works if "not found" responses look alike; a page that changes its word count per request will slip through, so you're back to -fr. Second, check what it decided: run with -v, or spot-check a few filtered words by hand. If you're scanning several hosts in one run, -ach calibrates per host, since each one has its own idea of "not found".
Treat 403 carefully
A 403 on /admin and a 403 on /zzzz mean different things. If everything is 403, filter it (-fc 403), or use -sf, which stops the run when more than 95% of responses are 403. If only a few paths return 403, keep them: they're leads. A 403 on its own still isn't a finding.
A run you can defend
Here's a full command for an authorized test, with the options I'd actually reach for:
ffuf -u https://app.example.com/FUZZ \
-w common.txt -ic \
-e .bak,.old,.zip \
-ac \
-t 10 -rate 20 \
-recursion -recursion-depth 2 \
-o ffuf-app.json -of json
-
-icignores comment lines in the wordlist. -
-e .bak,.old,.zipadds an extra request per word for each extension, appended to the word (include the dot). Backups and archives are exactly what forced browsing is for. -
-t 10 -rate 20drops ffuf's default of 40 concurrent threads and caps it at 20 requests per second. A test that knocks the target over isn't a test. Use whatever the program's rules say. -
-recursionstarts a new scan inside each directory ffuf finds. By default it only treats a hit as a directory when the server redirects/backupto/backup/, and-recursion-depth 2stops it from going deeper than two levels. (-recursion-strategy greedyrecurses into every match instead, which gets loud fast.) -
-o ffuf-app.json -of jsonsaves the results so you can triage them properly instead of scrolling.
If you'd rather inspect hits in Burp, -replay-proxy http://127.0.0.1:8080 sends only the matched requests through your proxy, so your HTTP history isn't buried under thousands of misses.
Triage: look for the odd ones out
Even after filtering, sort by how unusual each result is. In ffuf's JSON output, every result has url, status, length, words, lines and content-type. This short script groups results by shape and lists the rare ones first:
import json
import sys
from collections import Counter
with open(sys.argv[1]) as f:
results = json.load(f)["results"]
# Group hits by response "shape". A shape shared by many words is almost
# always a catch-all page; the rare shapes are the ones worth a look.
shape = lambda r: (r["status"], r["words"], r["lines"])
counts = Counter(shape(r) for r in results)
for r in sorted(results, key=lambda r: counts[shape(r)]):
n = counts[shape(r)]
tag = "noise?" if n > 5 else "LOOK"
print(f'{tag:6} x{n:<4} {r["status"]} {r["length"]:>7}B '
f'{r["content-type"]:<24} {r["url"]}')
Against a local mock app with a catch-all route, the output looks like this:
LOOK x1 403 9B text/plain http://127.0.0.1:8765/admin
LOOK x1 301 0B http://127.0.0.1:8765/backup
noise? x6 200 118B text/html http://127.0.0.1:8765/about
noise? x6 200 118B text/html http://127.0.0.1:8765/blog
...
The six identical 200s are the app shell. The two odd ones are where you spend your time. The threshold of 5 is arbitrary; adjust it to the size of your wordlist.
Prove the hit before you report it
A line in ffuf's output is a lead, not a finding. "/backup/app.zip returned 200" can still be the catch-all page. Detection isn't proof. Before you write anything up, a hit should pass three checks:
- It's different from "not found". Request the hit and a path next to it that can't exist, and compare. Same status and same body means it's the catch-all.
-
It's what its name says. A file called
app.zipshould start with the ZIP signaturePK\x03\x04(PKWARE's format spec calls it the local file header signature). A.bakof source code should look like source code. A.git/hit should serve Git's files, not HTML. - It's reachable without the access it shouldn't need. Request it with no cookies or tokens. If it's only reachable when you're logged in as an admin, it's probably working as intended.
This script does checks 1 and 2 in one go. It sends no cookies, so it covers check 3 too, and it reads at most 4 KB of each response, so you never pull down a whole backup:
import secrets
import sys
import urllib.error
import urllib.request
def fetch(url):
req = urllib.request.Request(url, headers={"User-Agent": "authorized-test"})
try:
with urllib.request.urlopen(req, timeout=10) as r:
return r.status, r.headers.get("Content-Type", ""), r.read(4096)
except urllib.error.HTTPError as e:
return e.code, e.headers.get("Content-Type", ""), e.read(4096)
def confirm(hit_url):
base = hit_url.rsplit("/", 1)[0]
control_url = f"{base}/{secrets.token_hex(8)}" # a path that can't exist
hit = fetch(hit_url)
control = fetch(control_url)
same = hit[0] == control[0] and hit[2] == control[2]
print(f"hit {hit[0]} {len(hit[2]):>6}B {hit[1]} {hit_url}")
print(f"control {control[0]} {len(control[2]):>6}B {control[1]} {control_url}")
print("verdict " + ("catch-all, not a finding" if same
else f"distinct response, first bytes: {hit[2][:16]!r}"))
confirm(sys.argv[1])
On the same local mock:
$ python confirm.py http://127.0.0.1:8765/backup/app.zip
hit 200 64B application/zip http://127.0.0.1:8765/backup/app.zip
control 200 121B text/html http://127.0.0.1:8765/backup/c695c3437f322ce8
verdict distinct response, first bytes: b'PK\x03\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
$ python confirm.py http://127.0.0.1:8765/login
hit 200 121B text/html http://127.0.0.1:8765/login
control 200 121B text/html http://127.0.0.1:8765/d8870273aa8f0068
verdict catch-all, not a finding
One caveat: if the catch-all page includes something random per request (a CSRF token, a request ID), the bodies will never match exactly and everything will look "distinct". In that case compare word and line counts instead, or strip the changing part before comparing.
Then stop
Once you've shown that a backup exists, is a real archive and is publicly reachable, you have the finding. Don't download and unpack it to "show impact", and don't go looking for credentials inside it. Report the URL, the request, the first bytes, the Content-Length and the fact that no authentication was needed, and say what such a file could contain. Let the owner check the rest. A finding isn't real until it's proven, and that proof is small enough to reproduce in one request.
Try it yourself
These are built for exactly this, and legal to hit:
- ffuf.me, by Adam Langley, is a set of lessons made for practicing ffuf. It's also available as a Docker image if you'd rather run it locally.
- PortSwigger's Source code disclosure via backup files lab hides a backup in a directory. Solve it once the intended way, then try finding the directory with a wordlist and a rate limit instead.
- PortSwigger's Information disclosure in version control history lab exposes a
.gitdirectory, a classic content-discovery find.
For each one, run ffuf once with no filters and once with -ac, and compare how many results you get. Then run the confirm script on the hits that survive.
Conclusion
ffuf is fast, which is the easy part. The skill is in what you do with its output: learn what "not found" looks like on the target, filter on the shape of the response rather than the status code, sort for the odd ones out, and prove each hit with a control request and a few bytes of content before you call it a finding. Most of the green lines in a raw run are the same page. The one that isn't is what you're there for.
Further reading
- ffuf on GitHub: README, usage and the wiki
- OWASP WSTG-CONF-04: Review Old Backup and Unreferenced Files for Sensitive Information
- SecLists: Discovery/Web-Content wordlists
- PortSwigger Web Security Academy: Information disclosure
I'm Krishna, a pentester and the founder of Xalgorix, an open-source AI pentester that only reports vulnerabilities it has actually exploited.
π If this was useful, follow me for more hands-on web & API security write-ups.
π Connect with me on LinkedIn: https://www.linkedin.com/in/xalgord/
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.