How I Taught a Security Agent to Remember with Hindsight

When I started building SentinelOps, I kept coming back to one question: what happens when a security team sees the same kind of attack twice?

An analyst might spend time investigating suspicious PowerShell activity, identify what happened, and document how the incident was handled. A few days or weeks later, a similar alert may appear on another machine. The team has already learned something useful, but an AI assistant that only sees the new alert may not have access to that earlier investigation.

I wanted to build something that could carry that knowledge forward. That became SentinelOps, an AI-assisted incident response application that uses Google Gemini for analysis and Hindsight for persistent organizational memory.

The goal is not to let an AI make security decisions on its own. I wanted it to give an analyst a better starting point by bringing relevant previous findings into a new investigation.

Building the application

SentinelOps is designed around a SOC analyst's investigation workflow. The interface includes an overview dashboard, an incident center, an AI investigation view, an organizational memory section, and learning analytics.

I built the frontend with React and TypeScript. The backend uses Node.js and Express, and it coordinates the incident workflow and the calls to the AI and memory services. Gemini analyzes the current alert. Hindsight stores and retrieves information from earlier investigations. Incident records are currently stored in local JSON files.

Keeping the integrations on the backend was important. It means API credentials can be loaded from environment variables instead of being exposed in the browser. It also gives the application one place to validate incoming alerts, prepare investigation context, and process analyst feedback.

Why I added Hindsight

At first, it might seem enough to send an alert to an LLM and ask it what to do. That can help with the alert in front of the analyst, but it does not automatically give the model access to the team's earlier investigations.

I did not want to solve that by attaching the entire incident history to every request. I wanted a separate memory layer that could keep useful outcomes and return information relevant to a new alert.

That is where Hindsight fits into SentinelOps. I use three operations in the workflow:

• Retain saves useful information from a completed investigation, including the analyst's verdict, root-cause notes, and response actions.
• Recall searches for relevant information when a new incident is investigated.
• Reflect helps bring together information across stored memories when the question is broader than one incident.

The separation of responsibilities is useful: Hindsight provides historical context, while Gemini reasons about the current alert using the evidence and context supplied by the application.

You can explore the project behind this memory layer through the Hindsight GitHub repository, the Hindsight documentation, and Vectorize's explanation of agent memory.

Connecting the services

I kept the Hindsight integration in a separate file, server/hindsightService.ts, rather than mixing memory-service code into the incident routes. The project uses the @vectorize-io/hindsight-client package, with the endpoint, API key, and bank configuration supplied on the server.

The client setup follows this pattern:

this.client = new HindsightClient({
  baseUrl: this.config.baseUrl,
  apiKey: this.config.apiKey
});

When an analyst submits feedback, the backend prepares a summary of the incident and sends it to Hindsight. The retain call follows this structure:

const response = await this.client.retain(
  this.config.bankId,
  incidentSummary,
  {
    context: 'SOC Incident Resolution & Analyst Feedback',
    metadata: stringifiedMetadata
  }
);

I kept the context and metadata with the retained information so that the memory is associated with the kind of work it came from. The actual values and response should be checked in the running application when demonstrating the integration; a code snippet by itself does not prove that a cloud request succeeded.

Recall before asking Gemini

The most important part of the integration is the order of operations. When an analyst starts an investigation, the backend first builds a query from the alert and asks Hindsight for relevant memories. It then passes the available memories to Gemini along with the current alert.

The core flow looks like this:

const hindsightResult =
  await hindsightService.recall(recallQuery);

let recalledMemories = hindsightResult.memories;

const analysis =
  await geminiService.analyzeIncident(
    incident.alert,
    recalledMemories
  );

This is a shortened excerpt to show the sequence, not the complete route. The surrounding code also handles incident data and fallback behavior.

One implementation detail is worth being clear about: SentinelOps has a local-memory fallback for cases where Hindsight does not return memories. That can help the demo continue, but it also means the source of a result matters. A memory returned by Hindsight and a match found in the local incident store should not be presented as if they were the same thing. I want the interface and logs to make that distinction obvious.

The FIN-SRV-04 and FIN-SRV-07 example

The main demonstration follows two related incidents.

The first alert is on FIN-SRV-04. It involves suspicious PowerShell activity associated with a WMIC parent process. The analyst investigates it and records the verdict, root cause, and containment actions. SentinelOps then sends the incident outcome to Hindsight through Retain.

Later, a similar pattern appears on FIN-SRV-07. Instead of treating this alert as completely unrelated to the earlier one, SentinelOps builds a recall query from the new alert. If Hindsight returns the earlier incident as relevant, that information can give Gemini additional context: what the team previously observed, which account was involved, and which response steps were taken.

That does not mean the second alert should automatically receive the same verdict. The analyst still needs to review the new evidence. The value is that the earlier investigation can help guide the next questions and recommendations, rather than forcing the team to rediscover every detail.

For the demo, I plan to show the actual Retain and Recall activity as well as the application screens. A memory card in the UI is not enough on its own to establish that a live Hindsight request took place. I also avoid treating a model-generated confidence value as measured detection accuracy; it is an estimate produced during analysis, not a result from an independent evaluation.

Keeping the analyst in the loop

The workflow does not end when Gemini returns an answer. An analyst needs to decide whether the alert is malicious, benign, or still unresolved, and then add the findings that should be useful later.

In SentinelOps, the intended cycle is: receive and validate an alert, recall relevant history, analyze the current evidence with Gemini, present the result to the analyst, collect feedback, and retain the outcome. That feedback is what gives a later investigation something useful to recall.

I have kept the application advisory-only. It can provide investigation findings and response recommendations, but it should not independently perform destructive actions such as disabling accounts or deleting files. Those decisions need human review and the appropriate operational controls.

What I learned and what I would improve

The biggest lesson for me was that adding a memory service is not the same as building a system that learns from previous work. The application has to decide what is worth retaining, when to recall it, and how to pass the retrieved information into the reasoning step.

I also learned that it is important to keep the sources of information clear. The current alert, a memory retrieved from Hindsight, a local fallback match, and Gemini's conclusion are four different things. An analyst should be able to tell which information came from where.

There is still work to do before I would consider SentinelOps ready for a production SOC. Incident records are stored in local JSON files rather than a database designed for concurrent use. Authentication, authorization, audit logging, error handling, and broader integration tests also need more attention. The fallback path needs to remain visible so that a service failure is not mistaken for a successful memory retrieval.

My next steps are to test the workflow with more types of alerts, improve the visibility of memory provenance, and make it easier to inspect exactly which historical details influenced a recommendation. I also want to evaluate the system with a wider set of incidents rather than relying only on the demonstration scenario.

For me, the point of SentinelOps is simple: an investigation should leave behind useful knowledge. When a similar alert appears again, the next analyst should not have to start from zero. Hindsight gives me a way to make that history available to the application, while keeping the analyst responsible for the final decision.