Check security headers for a list of websites in Python: HSTS, CSP, X-Frame-Options and an A-F grade (2026)
Security headers are the cheapest hardening a website can get: a handful of response headers that tell the browser to stay on HTTPS, refuse to run injected scripts, refuse to be framed by another site and stop guessing c
Security headers are the cheapest hardening a website can get: a handful of response headers that tell the browser to stay on HTTPS, refuse to run injected scripts, refuse to be framed by another site and stop guessing content types. They're also easy to forget, and they quietly disappear when someone swaps a CDN, a load balancer or a framework.
Checking one site by hand is fine (curl -sI https://example.com). Checking a client list, every subdomain you own, or a few hundred prospects isn't. This post builds one script that takes a list of URLs and returns, for each one, the final URL after redirects, an A-F grade, a 0-100 score and a plain-English list of what's missing.
What the script checks
| Header | Points | What "good" looks like |
|---|---|---|
Strict-Transport-Security |
25 |
max-age of at least 180 days |
Content-Security-Policy |
25 | restricts scripts (script-src or default-src); minus points for 'unsafe-inline' without a nonce or hash, and for 'unsafe-eval'
|
| Clickjacking | 15 |
X-Frame-Options or CSP frame-ancestors
|
X-Content-Type-Options |
15 | nosniff |
Referrer-Policy |
10 | present |
Permissions-Policy |
5 | present |
| Cookies | 5 | every Set-Cookie has Secure
|
A final page on plain HTTP scores zero. 90+ is an A, 75+ a B, 60+ a C, 45+ a D, anything lower an F. The weights are my own judgement, not a standard, so change them to match what you care about.
The script
# Usage: python3 security_headers.py urls.txt
# Bulk security-headers audit: fetches each URL (following redirects by hand), scores HSTS, CSP,
# clickjacking, X-Content-Type-Options, Referrer-Policy and Permissions-Policy, grades A-F,
# and writes headers_report.csv. Standard library only.
import csv, sys, re, time, http.client, ssl
from urllib.parse import urlsplit, urljoin
from concurrent.futures import ThreadPoolExecutor
UA = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0 Safari/537.36'
CTX = ssl.create_default_context()
def fetch(url, max_hops=10):
chain = []
for _ in range(max_hops):
u = urlsplit(url)
Conn = http.client.HTTPSConnection if u.scheme == 'https' else http.client.HTTPConnection
kw = {'context': CTX} if u.scheme == 'https' else {}
c = Conn(u.hostname, u.port, timeout=15, **kw)
c.request('GET', (u.path or '/') + ('?' + u.query if u.query else ''), headers={'User-Agent': UA, 'Accept': '*/*'})
r = c.getresponse(); r.read(200_000); c.close()
chain.append(f'{r.status} {url}')
loc = r.getheader('Location')
if r.status in (301, 302, 303, 307, 308) and loc:
url = urljoin(url, loc); continue
# header names are case-insensitive; Set-Cookie can repeat
h = {k.lower(): v for k, v in r.getheaders()}
h['set-cookie'] = [v for k, v in r.getheaders() if k.lower() == 'set-cookie']
return url, r.status, h, chain
raise RuntimeError('too many redirects')
def grade(final_url, h):
score, notes = 0, []
m = re.search(r'max-age=(\d+)', h.get('strict-transport-security', ''))
if m and int(m.group(1)) >= 180 * 86400: score += 25
else: notes.append('HSTS missing or max-age < 180 days')
csp = h.get('content-security-policy', '')
if 'script-src' in csp or 'default-src' in csp:
score += 25
if "'unsafe-inline'" in csp and not re.search(r"'(nonce|sha\d+)-", csp): score -= 8; notes.append("CSP allows 'unsafe-inline'")
if "'unsafe-eval'" in csp: score -= 4; notes.append("CSP allows 'unsafe-eval'")
elif csp: score += 5; notes.append('CSP does not restrict scripts')
else: notes.append('no CSP')
if h.get('x-frame-options') or 'frame-ancestors' in csp: score += 15
else: notes.append('no clickjacking protection')
if h.get('x-content-type-options', '').lower() == 'nosniff': score += 15
else: notes.append('no X-Content-Type-Options: nosniff')
if h.get('referrer-policy'): score += 10
else: notes.append('no Referrer-Policy')
if h.get('permissions-policy'): score += 5
else: notes.append('no Permissions-Policy')
insecure = [c.split('=')[0] for c in h['set-cookie'] if 'secure' not in c.lower()]
if not insecure: score += 5
else: notes.append('cookies without Secure: ' + ', '.join(insecure[:3]))
if not urlsplit(final_url).scheme == 'https': score = 0; notes.append('final page is plain HTTP')
letter = 'A' if score >= 90 else 'B' if score >= 75 else 'C' if score >= 60 else 'D' if score >= 45 else 'F'
return score, letter, notes
def check(raw):
url = raw if '://' in raw else 'https://' + raw
try:
final, status, h, chain = fetch(url)
score, letter, notes = grade(final, h)
return {'input': raw, 'final_url': final, 'status': status, 'grade': letter, 'score': score,
'hops': len(chain) - 1, 'server': h.get('server', ''), 'issues': '; '.join(notes)}
except Exception as e:
return {'input': raw, 'error': f'{type(e).__name__}: {e}'[:200]}
if __name__ == '__main__':
urls = list(dict.fromkeys(l.strip() for l in open(sys.argv[1]) if l.strip() and not l.startswith('#')))
t = time.time()
with ThreadPoolExecutor(10) as ex: rows = list(ex.map(check, urls))
cols = ['input', 'final_url', 'status', 'grade', 'score', 'hops', 'server', 'issues', 'error']
with open('headers_report.csv', 'w', newline='') as f:
w = csv.DictWriter(f, cols); w.writeheader(); w.writerows(rows)
for r in sorted(rows, key=lambda r: -r.get('score', -1)):
print(f"{r.get('grade', 'ERR'):3} {r.get('score', ''):>3} {r['input']:22} {r.get('issues') or r.get('error', '')}"[:150])
print(f'{len(urls)} URLs in {time.time() - t:.1f} s -> headers_report.csv')
Two details matter. First, redirects are followed by hand with http.client instead of letting urllib do it, because the headers you care about are the ones on the final page, and the redirect chain itself (http:// to https:// to www.) is worth seeing. Second, header names are lowercased, since HTTP headers are case-insensitive and servers disagree on capitalization.
Results
$ python3 security_headers.py urls.txt
A 100 bbc.co.uk
A 95 github.com no Permissions-Policy
A 95 stripe.com no Permissions-Policy
B 83 mozilla.org CSP allows 'unsafe-inline'; CSP allows 'unsafe-eval'; no Permissions-Policy
C 73 apify.com CSP allows 'unsafe-inline'; CSP allows 'unsafe-eval'; no Referrer-Policy; no Permissions-Policy
D 50 php.net no CSP; no X-Content-Type-Options: nosniff; no Referrer-Policy
D 45 python.org no CSP; no X-Content-Type-Options: nosniff; no Referrer-Policy; no Permissions-Policy
F 30 wikipedia.org no CSP; no clickjacking protection; no X-Content-Type-Options: nosniff; no Referrer-Policy; no Permissions-Policy
F 5 example.com HSTS missing or max-age < 180 days; no CSP; no clickjacking protection; ...
ERR expired.badssl.com SSLCertVerificationError: certificate has expired
10 URLs in 0.4 s -> headers_report.csv
A low grade isn't automatically a vulnerability. Wikipedia serves almost no user-specific content to anonymous readers and has its own reasons for its setup, and example.com is a placeholder page. The grade is a checklist, not a pentest. Where it earns its keep is on your own sites and your clients' sites, where a missing CSP or a dropped HSTS header after a migration is a real regression.
The most common gap in this sample is Permissions-Policy, which almost nobody sets. The most common CSP weakness is 'unsafe-inline', which keeps the policy from blocking injected inline scripts. The fix there is nonces or hashes, as the BBC's 'strict-dynamic' policy does.
Gotchas I hit
- Expired or invalid certificates throw before you see any headers. The script reports them as errors, which is usually what you want. If you still want to grade the headers, retry with an unverified SSL context and mark the row F.
- Redirect loops exist. Cap the hops (10 here) or one misconfigured site hangs a thread.
-
Content-Security-Policy-Report-Onlydoesn't enforce anything. It's a testing mode. Don't count it as a CSP; at most give it partial credit. -
Check the HTTP-to-HTTPS redirect separately. HSTS only helps after the first HTTPS visit, so request
http://too and confirm it redirects. The script above starts onhttps://to keep it short. -
Some headers are deprecated.
X-XSS-ProtectionandExpect-CTare ignored by modern browsers, so don't award points for them, andFeature-Policywas renamedPermissions-Policy. - Some sites serve different headers to bots. Send a browser User-Agent or you may be grading a bot-protection page.
- Don't point it at internal addresses. If you run this as a service that accepts user input, block private and reserved IPs, or it becomes an SSRF tool.
Running it at scale
For a weekly cron over your own sites the script is enough. If you'd rather not handle the HTTP redirect check, invalid certificates, cookie flags, security.txt and SSRF guards yourself, I also published it as a cloud tool: Security Headers Checker on Apify (disclosure: I built it). It records the full redirect chain with timings, grades sites with broken TLS instead of skipping them, checks security.txt, and can return only sites below a grade you choose. It costs $1 per 1,000 URLs checked; unreachable URLs and invalid inputs are free.
Either way, it takes a few seconds to know which of your sites quietly lost a header, and that's worth a look before someone else finds out.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.