Deep Dive: Implementing RFC 9110 HTTP Content Negotiation for Semantic Web in GitLab Pages

GitLab Pages powers static websites for millions of developers and open-source projects. For years, requesting an extensionless URI (such as GET /alice or GET /docs/ontology) resolved only to .html or index.html.

For standard web publishing, that works fine. But in the Semantic Web and Linked Data world (RDF, Turtle, N-Triples, JSON-LD, OWL), URIs represent abstract real-world resources or ontologies. A single URI is expected to serve different representations based on what the client asks for in the HTTP Accept header:

• A browser gets text/html.
• A SPARQL endpoint or semantic crawler gets text/turtle or application/rdf+xml.
• A web application gets application/ld+json.

In this article, I walk through how we implemented RFC 9110 HTTP Content Negotiation in GitLab Pages (MR !1384 addressing Issue #1135), and how we solved subtle performance, security, and caching pitfalls along the way.

1. The Challenge

When a client makes a request to GitLab Pages:

GET /resource HTTP/1.1
Host: example.gitlab.io
Accept: text/turtle;q=0.9, application/rdf+xml;q=0.8, text/html;q=0.5

The server needs to:

1. Parse and rank the client's Accept header by quality value (q), specificity, and client order.
2. Map preferred media types to candidate file extensions (.ttl, .rdf, .html, etc.).
3. Probe the virtual file system (VFS) in preference order.
4. Fall back cleanly to .html if no semantic variant is found.
5. Set the appropriate HTTP Vary: Accept caching headers so CDNs and browsers don't serve cached HTML to RDF clients or vice versa.

2. Parsing the Accept Header (RFC 9110 Section 12.5.1)

RFC 9110 specifies that Accept headers contain comma-separated media ranges with optional parameters: