Content Negotiation Comes to GitLab Pages: Serving Semantic Web Formats with Standard HTTP Headers

Static websites are often treated as HTML-only applications. But the web has always supported multiple representations of the same resource: HTML for browsers, JSON for applications, RDF for linked-data clients, and Turtle or N-Triples for semantic web tools.

Until now, GitLab Pages primarily resolved extensionless URLs to HTML files. That meant a request such as:

GET /alice
Accept: text/turtle

could still return an HTML document, even when a Turtle representation was available on disk.

This limitation made it difficult to publish semantic web content using clean, extensionless URLs.

The Problem

A project might contain several representations of the same resource:

public/resource.html
public/resource.rdf
public/resource.ttl
public/resource.jsonld

Different clients may prefer different representations.

A browser may request HTML:

Accept: text/html

A linked-data client may request RDF/XML:

Accept: application/rdf+xml

Another client may prefer Turtle:

Accept: text/turtle

Without content negotiation, all these requests resolve to the same default representation or fail to find the requested format.

The goal was to let GitLab Pages select the appropriate representation based on the client's HTTP Accept header, without requiring extensions in the URL.

Implementing Accept Header Negotiation

GitLab Pages now parses the HTTP Accept header and uses the client's preferences when resolving extensionless paths.

Supported formats include:

Supporting quality values

HTTP content negotiation allows clients to express preferences using quality values (q).

For example:

Accept: text/turtle;q=0.9, application/rdf+xml;q=0.8

In this example, the Turtle representation is preferred over RDF/XML when both files exist.

The parser also considers media-type specificity:

Accept: text/turtle, text/*, */*

An exact media type takes precedence over a type wildcard, which takes precedence over the global wildcard.

This makes representation selection follow standard HTTP semantics instead of relying on a fixed extension preference.

Safe Fallback Behavior

Content negotiation is designed to preserve existing GitLab Pages behavior.

If the requested representation does not exist, GitLab Pages falls back to the HTML representation:

/resource.ttl  -> resource.ttl
/resource      -> resource.html

Direct requests containing an explicit extension continue to bypass negotiation.

The feature is also protected by a feature flag:

FF_ENABLE_CONTENT_NEGOTIATION

This allows the behavior to be enabled progressively without changing the default behavior for existing deployments.

Correct Caching with Vary

Different clients can receive different representations for the same URL. Shared caches must know that the response depends on the Accept header.

Negotiated responses therefore include:

Vary: Accept

This prevents a cached Turtle response from accidentally being served to a client requesting HTML.

The implementation also keeps variant-specific ETags distinct so that conditional requests do not incorrectly return 304 Not Modified for a different representation.

This is an important detail: content negotiation is not complete if the server selects the correct representation but caching does not distinguish between variants.

Better Acceptance Testing

The acceptance tests now use the repository's shared fixture convention instead of creating ZIP files at runtime.

The fixture contains:

shared/pages/group/content-negotiation/public/
├── resource.html
├── resource.rdf
├── resource.ttl
├── resource.jsonld
└── dir/
    ├── index.html
    └── index.ttl

The GitLab API stub registers the domain through the normal apiResponses map, matching the conventions used by the rest of the repository.

This makes the tests:

• Easier to understand
• More deterministic
• Closer to real GitLab Pages behavior
• Free from temporary archive servers
• Safer because test assertions do not run inside background HTTP handlers

Using shared fixtures also makes it easier to extend the acceptance tests when additional representations or directory-index scenarios are introduced.

Validation

The implementation was validated with:

go test ./internal/serving/disk ./test/acceptance -short -count=1
go test ./... -short -count=1
go vet ./internal/serving/disk ./test/acceptance ./test/gitlabstub

All targeted tests and the short repository test suite passed.

Why This Matters

Content negotiation makes GitLab Pages more useful for projects that publish structured and linked data while preserving the familiar extensionless URL experience.

Authors can publish multiple representations:

/alice

And let clients request the format they understand:

Accept: text/html
Accept: text/turtle
Accept: application/ld+json

This brings standards-based representation selection to GitLab Pages without requiring application code, routing frameworks, or custom servers.

GitLab Pages remains static, simple, and fast — while becoming more capable for semantic web publishing.

References

• Merge Request: gitlab-org/gitlab-pages!1384

Suggested blog tags: GitLab, GitLab Pages, Open Source, HTTP, Content Negotiation, Semantic Web, RDF, Developer Experience

Author: Manish Kumar

GitLab: @manishpatel00