Using AWS to Manage AI Spec-Driven Development Artifacts

My Pain Point

Doing spec-driven development with AI agents, I quickly ran into a problem: managing all the artifacts they produce. Brainstorming notes, architecture designs, specifications and contracts, results of architecture and code reviews, troubleshooting notes, etc. Even on a small project you quickly end up with hundreds of markdown files. On a team project, thousands to tens of thousands.

The basic approach is to store them in a folder in the repo (e.g. docs/), with an index.md file listing each file with a short description of its content. This approach has several issues:

• Documentation pollutes your code base
• Publishing an artifact goes through the same PR review as code, which makes code reviews harder and sharing artifacts between team members have to wait the PR review
• It is unrealistic to capture the content of an ADR or a spec in a 1-2 sentence description in an index file, so agents can't rely on it to find the right information
• You can't share information across teams (e.g. the agents of an application developer need access to the architecture designed by the networking or security team in another repo)

How I Used Just 3 AWS Services to Solve it

This is why I came up with the idea of using AWS services behind an MCP server to store, manage, share and search these artifacts. The architecture is pretty simple:

• S3 stores all the markdown artifacts
• S3 Vectors stores an embedding of each H2 section of every artifact. This gives semantic search at a more granular level than the whole document, plus filtering on the vector metadata (e.g. on the artifact type)
• A Bedrock embedding model (Titan Text Embeddings v2) creates these embeddings
• An MCP server exposes tools to store, manage and search artifacts, a few resources, and an MCP App to browse the artifacts visually
• The MCP server uses the developer's own AWS credentials (e.g. an AWS CLI profile)
• The artifacts follow the Open Knowledge Format (OKF) for their frontmatter. The artifact sources and relationships (work in progress), as well as the commit references, are stored in S3 object annotations. This way they can be updated without having to replace the S3 object, since S3 objects and their metadata are immutable.
• In an enterprise, a single S3 bucket and a single vector index can store the artifacts of all teams and projects, each in its own prefix ("subfolder"). A team can then mark some of its artifacts as shared to make them visible to other projects and teams.

Architecture Diagram

The Result

Combining just these 3 AWS services gave birth to the Arkeology project with the following features:

• Immediate write-then-read consistency: an artifact written in one agent call is searchable in the next
• Semantic search with metadata filtering: retrieve artifacts by meaning, not just by keywords
• Section-level search precision: a query matches the specific section of an artifact that is relevant, not an average of the whole document
• Cross-team knowledge sharing: canonical artifacts such as ADRs can be made discoverable by the agents of other projects or teams. Nothing is shared by accident: only permanent artifacts explicitly marked as shared are visible outside their project
• Rich, filterable metadata: every S3 object and every vector carries structured metadata, and S3 objects also carry annotations
• Commit-to-artifact traceability: artifacts carry git commit SHA references, stored in S3 object annotations and mirrored into the vector index so you can filter on them
• Arkeology Studio: calling it opens a visual artifact browser directly inside any MCP host that supports MCP Apps (Claude Desktop, claude.ai, VS Code Copilot...)
• Full artifact lifecycle: archive, delete and purge artifacts as projects evolve
• Migration: a skill and server tools to import the documentation of an existing project
• And more

Arkeology Sudio MCP APP Preview

Caveats

Team Sharing and IAM Permissions
The sharing of artifacts is enforced by the MCP server, but anyone with IAM read permissions on the whole bucket can bypass the MCP server and, for example, use the AWS CLI to read any data. Least-privilege permissions restricting each team to its own prefix in the S3 bucket must be enforced in IAM. That said, IAM can't do the same inside a shared vector index: any team with query access to the index can read the metadata (titles, descriptions, tags) and embeddings of all teams' artifacts. So only share a vector index with teams you trust.

Arkeology Studio MCP App
It requires an MCP host that supports MCP Apps (Claude Code, for example, doesn't), or using the MCP Inspector, which is a bit of a pain as a developer. I am currently thinking of building an optional frontend on AWS, to have a web interface to access, search and read the artifacts.

Lessons Learned

Vector metadata limits
Vector metadata is limited to 2 KB filterable and 40 KB total per vector. This is why we can't put sources and relationships in the vector metadata and use them to search and filter artifacts. It is also why only the 20 most recent commit references go into the vector index, so you can still find the artifacts linked to recent commits; the full list stays in the S3 object annotations.

Annotations
S3 user-defined object metadata is capped at 2 KB and immutable like the object. Annotations hold up to 1 MiB and can be updated without rewriting the object. But there is a catch: overwriting the object deletes its annotations, so every rewrite must read them first and put them back. And if that read fails, it must raise an error rather than return nothing, or you end up writing empty values over good data, with no other copy to recover from.

Using annotations for sources and relationships comes with some limitations:

• It is not possible to do a search like "give me all artifacts referencing artifact XYZ"
• Search results don't include these fields: an agent has to read or list the artifact to see them

In other words, agents can only navigate these links, not query them.

S3 Vectors
QueryVectors doesn't return the distance unless you ask for it (returnDistance: true). Our code defaulted the missing distance to 0, so every result got a perfect score of 1.0 (score = 1 − distance). The search ranking was silently broken while all unit tests passed.

Conclusion

With just 3 basic AWS services I was able to build an MCP server that basically solves my pain points with managing all the artifacts generated through AI spec-driven development. Improvements to make the tool really functional at an enterprise level would be:

• Host the MCP server on AWS
• Enforce least privilege, so nobody can bypass the MCP server to read the whole bucket content
• Host a web frontend on AWS, to get around the MCP App limitations

The code is open source and available at github.com/amanoxsolutions/arkeology.

(Note: I used AI only to proofread this article, improve readability and draw the architecture diagram)