Realty Check: an agent that fact-checks real estate listings (and why we never let it judge)
This is a submission for the MLH x DEV Writing Challenge What We Built A listing is written by the person who wants your money. Twelve good photos, a paragraph of adjectives, a bedroom count, a square footage
This is a submission for the MLH x DEV Writing Challenge
What We Built
A listing is written by the person who wants your money. Twelve good photos, a paragraph of adjectives, a bedroom count, a square footage. You rent, buy or sign a lease based on it, and nobody checks it.
We, Raunak Choudhary and Rohan Gore, built Realty Check, a Chrome extension backed by a custom agent harness that checks a property listing against the record. You click the address on any listing page. About a minute later, a drawer shows every claim that doesn't match: what the listing says, what the record says, a confidence score, and the source that decided it. The full report opens the live 3D scan at the exact spot that settles the claim, lists what the listing left out, and drafts questions to ask before you commit.
The truth about a building usually already exists. It is just spread across places a renter never looks:
- CoStar property records: floor plans, unit counts, year built, amenities
- Matterport 3D scans of the actual units, with labeled rooms and measured floor area
- Vexcel aerial imagery and elevation data
- County open records
No single source settles much on its own. Together they do: a floor count is settled by roof height measured from the air and confirmed against the records, and a listed unit size is settled by the area measured inside the 3D scan.
How it works
listing page β Chrome extension β POST /check (FastAPI, streamed)
β orchestrator (LangChain deepagents)
ββ inside β Matterport scan
ββ outside β Vexcel aerial imagery
ββ paper β CoStar + county records
β judge (plain Python rules) β drawer + full report
The extension reads the page with no per-site selectors (regex, JSON-LD and one model call), so it works on any listing site. The orchestrator reads up to 40 claims of 11 kinds, matches the address to one building, and sends three subagents out in parallel. Then a judge written in plain Python decides every verdict.
What broke (a lot)
Our agent was a confident liar. The first full test across 10 real listing sites produced 20 red "mismatch" rows, and most were false. "1 bedroom, but the scan shows 2", because an unnamed claim got compared with whichever floor plan's scan happened to load. "5 bedrooms" in a 2-bed unit, because Matterport room labels are tags people type, not facts. We rewrote the rules: evidence about one plan can only confirm an unnamed claim, room labels alone can never contradict, and near misses (215 vs 216 units, built 2019 vs 2020) count as "close", not "wrong". Re-judging the same saved runs: 20 false alarms became 2.
3D scans are smaller than the ad. Across 89 floor plans, the area measured inside the scan was a median 11% smaller than the advertised size, because scans measure the walkable floor. Our first 5% rule called almost every apartment a liar. Scan measurements now get their own tolerance.
The vision model miscounted floors. It looked at our demo building from above and said 7 floors. The building has 6. Photos no longer decide floor counts. Floors come from the measured roof height ("23.1 m means about 6 floors") and the records.
It checked the wrong building. On some sites, the extension grabbed the management company's office address from the footer. Every address on the page is now scored by where it sits and how it is labeled, and the drawer says which building it is checking, with a "Not this? Pick another" link.
It was slow. One check on a Los Angeles building with 15 floor plans took 147 seconds. Parallel subagents, one less model turn, fetching aerial photos once per run, and capping proof images brought it to 56 seconds.
What we learned
An agent is good at finding evidence and bad at being the judge. Most of building a useful harness is deciding what the model is not allowed to do: which tools it doesn't get, how many calls it can make, when the clock runs out, and which decisions belong to ten lines of boring Python instead.
Demo
- π₯ Video: https://www.youtube.com/watch?v=lBtpzvgc_eg
- π» Code: https://github.com/rohan-g0re/Realty-Check
- π Devpost: https://devpost.com/software/realty-check-kruw17
The side panel streams each step of the run, then shows the verdict with a confidence score and citations:
"Listed vs. in reality": the 3D scan opens at the spot that settles the claim:
What the listing left out, and questions to ask before you commit:
Partner Technologies
LangChain deepagents is the core of the harness, and we changed almost every default.
-
One orchestrator, three named subagents. We built it with
create_deep_agent. The subagents areinside(Matterport),outside(Vexcel), andpaper(CoStar and county). Each has its own tools, and the orchestrator calls them in parallel through thetasktool. - We took tools away. A harness profile switches off the file tools, the to-do list, and the general-purpose subagent. The agents can call our six tools and nothing else.
- Middleware for hard limits. It caps a run at 14 model calls and 96 seconds, and ends the run as soon as verdicts exist. If the agent runs out of time, a model-free closing step makes any source call it skipped, so a cut-off run still ends in a complete answer.
- A custom chat model. We wrote a LangChain chat model that drives Claude headless through the Claude Code CLI, with tool calling and image input handled in our own adapter. The project holds no API key.
-
The model never judges.
judge_claimsis plain rule code: counts must match exactly, written sizes within 5%, and a verdict with no citation becomes "couldn't check". The tool even refuses to run once if a needed helper wasn't asked yet.
The experience: DeepAgents gave us planning, subagents, and parallel task calls for free, which let us spend the weekend on the hard part, the rules. The defaults are built for open-ended agents that write files and keep to-do lists. A fact-checker needs the opposite: a narrow agent on a short leash, and the profile and middleware hooks made that easy to do without forking anything.
The sponsor data platforms were just as important:
-
Matterport Model API (GraphQL) for rooms, floor plans, and measured area, and the Showcase SDK for the in-report viewer. Gotcha: our SDK key accepted
localhostbut rejected127.0.0.1with HTTP 400 "referrer is not allowed", which is the host the extension opened. So the viewer silently never moved to the proof spot until we added a redirect. - Vexcel ortho and elevation API for roof height, footprint, and top-down and angled photos. Every aerial finding carries its photo date, because the photo can be older than the listing.
- CoStar property data, event-only and kept out of the repo.
Hackathon Experience
This was Raunak's second HackPrinceton and Rohan's third. We built Realty Check there (Fall '26, Nov 13 to 15) for CoStar's "Inside and Out: AI for the Built World" track.
We keep coming back to hackathons for two reasons. The first is the people. We love meeting builders, hearing what they're stuck on at 2 am, and leaving with friends we didn't have on Friday. The second is the clock. Every developer has a creative person sitting somewhere inside, and a hackathon is where you get to find out how much of that creativity survives a 36-hour deadline. For us, hackathons are the soul of being a developer.
The part we'll remember is how the project came together. We teamed up once again, and for the first half of the weekend we built two versions in parallel. Raunak's was a "Before You Sign" check that started from one question, can a small child walk from inside this house straight to the pool through a door?, and grew into a general claims checker for scanned houses. Rohan's was Realty Check. On Saturday afternoon we put the two side by side, compared them honestly, and went all in on the Realty Check harness. Raunak moved to the demo video, the architecture diagram, and the write-up, and brought over ideas from the other version, like citations and confidence scores on every finding. Killing your own project halfway through a hackathon stings for about ten minutes. Then you realize the team just got twice as fast.
We didn't win a prize. We're saying that plainly because we think it matters. The CoStar mentors liked what we built, and every judge we pitched to gave us positive feedback. Winning and losing are both part of this ecosystem. What stays is what you learned, and we learned more about building agents in this one weekend than in months of reading about them. We're only looking forward from here.
Thanks to the CoStar, Matterport, and Vexcel teams for opening up real data for a weekend, and to the HackPrinceton organizers and MLH for making space for all of it.
If you've ever opened a door to a room smaller than the photos, tell us in the comments what you'd want checked first π
I'm a software engineer (NYU CS MS) who builds backend systems and AI agents, and I'm open to full-time roles. If your team works on problems like this one, I'd love to talk: [email protected] Β· GitHub.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.


