Building RecallOps: An AI Maintenance Agent That Learns from Past Repairs Using Hindsight

By Rahul R | HackwithBangalore 4.0

The Problem: Machines Fail, but Why Do We Keep Starting from Zero?

Imagine a factory pump suddenly starts overheating. A maintenance technician checks the bearings, inspects the motor, and tries different repairs before finally identifying the problem.

The machine is repaired, and production resumes.

A few weeks later, another pump develops similar symptoms. Unfortunately, the technician investigating this new failure may not know what happened previously.

The entire troubleshooting process starts again.

This made me think about an important problem: factories generate valuable maintenance experience every day, but how much of that experience is actually reused?

Traditional maintenance software can store repair records. However, simply storing information doesn't mean that an AI system can learn from it.

I wanted to build something that could remember previous repair experiences, understand their outcomes, and use that knowledge during future investigations.

That is how our project, RecallOps, was developed.

What Is RecallOps?

RecallOps is an AI-powered industrial maintenance assistant that helps technicians investigate machine failures using both AI reasoning and historical repair experiences.

A technician can report symptoms such as excessive vibration, overheating, or abnormal machine noise.

RecallOps analyzes the problem, retrieves relevant historical incidents, and suggests possible causes and diagnostic steps.

But the most interesting part comes after the repair.

When a technician records and verifies the outcome, RecallOps can retain that experience and use it when investigating similar failures in the future.

The idea is simple:

An AI agent should remember not only what was attempted, but also what actually worked and what failed.

Why We Chose Hindsight

The main theme behind our project is building an AI agent that learns using hindsight.

A typical AI chatbot can provide troubleshooting suggestions based on the information supplied in the current conversation.

However, it doesn't automatically remember every repair performed in a particular factory.

We needed a way to give our agent long-term memory.

This is where Hindsight Cloud became central to our architecture.

Hindsight allows RecallOps to retain previous maintenance experiences and retrieve relevant information when new problems occur.

For example, a stored experience may include:

• The machine and reported symptoms.
• The suspected cause of the failure.
• The repair action performed.
• Whether the repair succeeded or failed.
• Information recorded during outcome verification.

These details help our agent distinguish between a repair that was merely attempted and one that actually resolved the problem.

We aren't retraining the language model every time a technician completes a repair.

Instead, we're using memory retrieval to provide better historical context for future AI reasoning.

A Simple Example of How RecallOps Learns

Suppose a factory pump starts vibrating excessively.

During the first incident, a technician suspects damaged bearings and replaces them. Unfortunately, the vibration continues.

The unsuccessful repair is recorded.

Later, another technician investigates a similar problem and discovers that the pump's shaft is misaligned.

After correcting the alignment, the vibration decreases and the repair is verified as successful.

Now consider a third pump experiencing similar vibration symptoms.

When a new investigation begins, RecallOps retrieves the relevant historical experiences through Hindsight.

It can see that replacing bearings alone failed in one previous case, while correcting shaft alignment helped resolve another.

The agent can therefore prioritize checking shaft alignment among its recommendations.

It doesn't mean shaft misalignment must always be the cause. The technician still needs to inspect the machine.

But the AI now has useful experience that wasn't available during the first investigation.

This is the difference between generating an answer and reasoning with remembered outcomes.

How We Built the System

We developed RecallOps as a full-stack web application with a React frontend, a Python backend, AI inference through Groq, and long-term memory through Hindsight.

Frontend: React, Vite, and Tailwind CSS

Our frontend is built using React, Vite, and Tailwind CSS.

We designed a dark-themed industrial dashboard that allows technicians to access the main features without navigating through complicated menus.

The application contains six sections:

1. Overview: Displays maintenance activity and allows technicians to start investigations.
2. AI Investigations: Shows possible diagnoses, safety warnings, recommended steps, and historical evidence.
3. Learning Benchmark: Demonstrates how memory-informed recommendations can differ from baseline recommendations.
4. Hindsight Memory: Displays retained maintenance experiences.
5. Incidents: Organizes reported failures and their statuses.
6. Machines: Provides a view of equipment and related maintenance activity.

Our goal was to make the interface understandable even to someone who isn't familiar with AI systems.

Backend: Python and FastAPI

The backend is developed using FastAPI.

It handles incident creation, investigation requests, repair records, outcome verification, and communication with our AI services.

SQLite stores the prototype's operational incident records.

Each reported incident follows a structured workflow, allowing the application to track its progress from reporting to verification.

AI Reasoning: Groq

We integrated Groq to generate structured AI investigations using the "openai/gpt-oss-120b" model.

When a technician reports an issue, the backend combines the current symptoms with relevant historical information retrieved through Hindsight.

The AI then generates a response containing a possible diagnosis, recommended diagnostic steps, safety precautions, and supporting historical precedents.

This helps make the investigation more useful than a general troubleshooting response.

Long-Term Memory: Hindsight Cloud

Hindsight is responsible for retaining and retrieving repair experiences.

When an outcome is verified, RecallOps can store the experience for future investigations.

When another incident occurs, the agent retrieves relevant memories and uses them as context.

This is the most important part of our architecture because it connects past outcomes with future decisions.

Our Complete Maintenance Workflow

We designed the application around six main steps.

Step 1 — Report a failure

The technician selects a machine and enters the symptoms.

Step 2 — Retrieve relevant memories

RecallOps searches Hindsight for similar previous incidents and repair outcomes.

Step 3 — Generate an investigation

Groq analyzes the current symptoms along with retrieved experiences and returns a structured diagnosis.

Step 4 — Record the repair

The technician performs the necessary physical inspection or repair and records the action.

Step 5 — Verify the result

The repair outcome is recorded and verified, distinguishing successful repairs from unsuccessful ones.

Step 6 — Retain the experience

The verified experience is retained through Hindsight so that it can help with future investigations.

This creates a continuous learning cycle:

Report → Investigate → Repair → Verify → Remember → Improve Future Recommendations

The agent's knowledge can become more useful as relevant verified experiences are added.

How We Demonstrated Learning

One challenge was demonstrating that Hindsight memory actually makes a difference.

Simply showing an AI-generated diagnosis wouldn't prove that our agent was using previous experiences.

So we created a controlled synthetic experiment comparing two conditions.

Condition A: Without additional historical memory

The agent investigates a machine problem without the relevant verified repair experience.

Condition B: With Hindsight-informed context

The agent investigates using an additional verified historical experience.

In our predefined three-scenario benchmark, the baseline condition achieved 0 out of 3 target-aligned prioritizations, while the memory-informed condition achieved 3 out of 3.

This demonstrated how adding relevant memory could change the agent's diagnostic priorities.

These are controlled synthetic results, not real-world diagnostic accuracy measurements. A much larger evaluation with actual industrial maintenance data would be necessary to establish performance in real factory environments.

Still, the experiment helped us demonstrate the core mechanism behind our project.

Testing Our Deployed Application

We wanted RecallOps to be more than a frontend prototype.

After connecting the services, we deployed the frontend on Vercel and the FastAPI backend on Render.

We then performed automated testing across browser interactions, API endpoints, integration workflows, and responsive layouts.

Our reported quality-assurance audit completed 56 automated checks successfully.

One end-to-end browser test followed the full technician workflow: creating an incident, generating an AI investigation, retrieving Hindsight precedents, recording a repair, and verifying the outcome.

The verified experience was successfully retained through Hindsight.

These tests gave us confidence that the main demonstration workflow was functioning, although they don't guarantee that the application is free from every possible defect.

Challenges We Faced

One major challenge was connecting multiple services while keeping the workflow consistent.

The frontend needed to communicate with FastAPI, the backend needed to call Groq and Hindsight, and the returned information needed to appear correctly in the investigation interface.

Another challenge was deciding what should count as a learned experience.

We didn't want every repair attempt to automatically become a successful precedent.

That's why outcome verification became an important part of our design.

We also had to make sure the application clearly distinguished between historical evidence, AI-generated recommendations, and controlled benchmark results.

The current prototype still has limitations. It depends on the quality of technician-provided information, and its benchmark uses synthetic cases rather than large-scale industrial data. The free hosting environment also doesn't provide durable storage for all local SQLite records.

These are areas we would improve before considering real industrial deployment.

What's Next for RecallOps?

There are several ways we could extend the project.

We would like to connect real-time IoT sensor data, including temperature and vibration readings, so technicians don't always need to enter symptoms manually.

We also want to expand the memory system with larger verified maintenance datasets and evaluate performance across different machine categories.

Other future improvements include stronger access controls, a durable production database, organization-specific memory, and more detailed evidence tracking.

Our long-term vision is to help maintenance teams preserve valuable repair knowledge and make it available whenever similar problems occur.

Final Thoughts

Building RecallOps helped us explore an important idea in AI agent development.

A language model can generate recommendations, but a memory-enabled agent can also consider what happened during previous attempts.

By combining FastAPI, Groq, and Hindsight Cloud, we built a prototype that connects machine failure investigation with verified repair experiences.

The most valuable lesson from this project is that learning from failure is just as important as remembering success.

An unsuccessful repair isn't necessarily wasted effort if the experience helps someone avoid the same mistake in the future.

That's what RecallOps is designed to achieve.

Our goal is simple: help machines get repaired faster by making sure valuable maintenance experience isn't forgotten.

Explore RecallOps

Live Application: https://recallops-lake.vercel.app

GitHub Repository: https://github.com/Rahul-2006ra/recallops

Built With: React, Vite, Tailwind CSS, Python, FastAPI, Groq, Hindsight Cloud, SQLite, Vercel, and Render.

Developed for HackwithBangalore 4.0.