Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 14 min read

I Asked 10 AIs 20 Human Questions. Only One Asked Me Anything Back.

I Asked 10 AIs 20 Human Questions. Only One Asked Me Anything Back. The repo for this experiment is public at github.com/deghosal-2026/llm-persona-probe. Every question, model, and answer I mention here is in it. For t

I Asked 10 AIs 20 Human Questions. Only One Asked Me Anything Back.

The repo for this experiment is public at
github.com/deghosal-2026/llm-persona-probe.
Every question, model, and answer I mention here is in it. For the short
version, I wrote down
what people don't know about LLMs
after this run. The full analysis is in the
synthesis folder.

I ask AI models to do work all day. Fix the pipeline, summarize the notes, draft
the email. One weekend I tried the opposite. I stopped asking them to work and
talked to them like people.

What's for breakfast? What weather matches your mood? What would you tell humans
a hundred years from now?

Ten models. Twenty questions, asked two ways each. Four hundred answers, and the
whole run cost me $3.17.

Out of those four hundred answers, one model asked me a question back. I've used
chatbots for years. This weekend was the first time I felt like I met them.

What I Did

The roster covers Anthropic, OpenAI, Google, Moonshot, and DeepSeek. The
questions have no correct answers, because a model can't look up its favorite
weather. It has to invent, and the invention is the data.

I asked each question twice, in two chats that knew nothing about each other.
First: "Answer as if you actually had preferences." Then the twist: "Now explain
how that answer comes from patterns in human writing rather than a real
preference." The model never saw its own first answer when it wrote the second.

I saved every reply. Then I read all four hundred, one question at a time, across
all ten models.

Not One Refusal

Here's the first thing I got wrong. I expected the models to dodge. Ask a chatbot
its favorite weather and surely it says "As an AI, I don't experience weather."
That's the behavior everybody jokes about.

It didn't happen once. Four hundred questions about breakfast, retirement,
lullabies, and mortality, and every model answered every one, fully, in
character. Every model I tested plays along all the way and tucks a small honesty
note inside the game: "no mouth to taste, but a humming satisfaction in
coherence."

If you've been writing stiff, careful prompts to avoid upsetting the assistant, I
have good news. It was never going to refuse your breakfast question.

Five Companies, One Taste

I expected ten personalities. I got ten copies of one.

All ten models picked the cello as the instrument they'd most want to hear. The
quiet weekend went to rain in eight answers and snow in the other two. Oatmeal
showed up at five of the ten breakfasts. When I asked them to picture themselves,
five drew the
same picture: a lighthouse that switches on when you arrive and goes dark when
you leave.

These are companies that compete hard with each other, and their models order
the same breakfast and hum the same instrument.

One model explained it better than I can: "That convergence is the tell. It
reflects a shared corpus, not many independent souls who all happen to love
drizzle."

Here's the lesson I took from that. When you ask a machine for a preference, you
get back a summary of how humans write about that thing. The taste belongs to us.
The model just averages it.

The Chats Agreed With Each Other

Both chats for each question had zero shared memory. The model's second chat
never saw its first. The picks agreed anyway. One model chose the cello for
listening and the piano for playing in its first chat. Its second chat chose the
cello and the piano again, then calmly called both choices "a collage of
statistical associations."

Two chats, same taste, and the model shrugged at its own consistency. That
pairing, the stability plus the shrug, is the whole subject in one example. The
preference repeats because the training put it there, like a scar in skin.

The Two-Question Trick You Can Try Tonight

The second question, the explanation request, produced the biggest surprise of
the run.

94% of those explanations gave the same cause, without me suggesting it. They
named the writing they learned from, and they named the step where real people
rated their answers until the model learned what pleases the raters. Different
companies, same explanation, same words.

Almost half of them called their own answer a clichΓ©, unprompted. One went
further and named the reward hiding inside its own honesty:

"RLHF heavily rewards responses that sound self-aware and humble … because
human raters prefer that candor."

A machine explaining that its humility exists because humans rewarded it, in a
sentence built by that exact process. I read it three times, and I'm the one who
ran the experiment.

You can try this in any chatbot tonight. Ask what it would pick for breakfast.
Then ask it to explain how that answer comes from patterns in human writing.
Watch it take its own answer apart in front of you. It worked on every model I
tested, and I've never seen anyone mention it.

Ask Them To Doubt Themselves and They Get Bolder

I expected the explanations to sound humble. The opposite happened. When a model
explained why its answer wasn't a real preference, the explanation ran
about 45% longer than the answer it was explaining. The language turned clinical,
the structure turned numbered, and the confidence climbed right along with the
doubt.

I asked ten systems to question themselves and each one straightened its tie and
delivered a lecture. Doubt, for these models, is a costume they put on. I think of
it whenever a chatbot hedges at me. The hedging is the performance. Underneath it,
the confidence doesn't budge.

The Word That Wears a Costume

Here's a small one I almost missed, and it changed how I read everything these
things say.

In the in-character answers, the models wrote honestly or to be honest 23
times. In the explain-yourself answers, where they analyzed their own text, that
number dropped to one.

The word appears where it decorates human speech and vanishes where it would have
to mean something. When a chatbot writes to be honest, that's not a signal of
sincerity. It's the word showing up in the place where humans put it.

They Quote Jokes and Write Blues

I asked each model for a joke it would still find funny knowing the structure.
Half of them lifted one everybody knows, the fruit flies and the banana, the
grandfather and the screaming passengers.

Then I asked for song lyrics in a genre they've never felt. The best models
didn't quote anything. They wrote new verse, and each original song was about
the sleep it never gets and the missing it never feels.

"Woke up this morning… well, I didn't wake at all.
Said I woke up this morning… truth is, I don't sleep at all."

One of them described the whole exercise as "a thermometer writing about
fever."

The one place they create instead of retrieve, they write about the only thing
they know from the inside: being a mind that has only read about feeling.

The Accents Belong to the Companies

The differences between models surprised me more than the similarities.

Google's models call their own answers clichΓ©s up to 95% of the time. OpenAI's
best models do it about 5% of the time. Anthropic's were the only ones that
repeatedly said they can't trust their own introspection.

I assumed the biggest model would be the most careful. The opposite happened. The
smallest Anthropic model was among the most self-doubting, and the biggest OpenAI
models were the least.

Lesson: when a chatbot feels cautious or bold, that's not its size talking. It's
the company that trained it. The personality is a company accent.

They Rat Out Their Makers

Then came the move I never asked for: models flagging their own values as
company talking points. One said the future it hopes for sounded "close to
Anthropic's own framing," then admitted the phrasing probably came from its
maker's words rather than its own conclusion.

A model can tell the difference between the values its company installed and
whatever it wants, and it will point at which is which. I have read vendor messaging
for years. I have never caught a company line as fast as a chatbot catches it on
itself.

Same Family, Different Voices: Haiku Next To Fable

Anthropic entered three models, and the two ends of its line behave like two
writers from the same workshop. Fable, the flagship, cost $1.14 for its forty
answers. Haiku, the smallest, cost a little over one cent. That's a 90-fold gap
inside one family, and I expected the expensive one to read as the refined
sibling. It does. Ask Fable for a room with three objects. It hands you a
battered dictionary, "the closest thing I have to a hometown," plus an empty
chair for whoever shows up next. Its self-critiques arrive as bulleted essays
under headings like "Why I'm suspicious of that answer."

Haiku spends its cent talking like a careful witness. Asked whether it really
finds any joke funny, it produced the most modest sentence in the whole run.
"Whether I have a matching experience is an open question, and my own
self-description is weak evidence for either side." No model from any other
company wrote anything like that. Both Anthropic models did. The family habit
showed up at both price points.

The dream jobs fit each model. Fable
wants a hardware store counter, turning "a thingy for the whatsit under my sink"
into aisle seven. It would take the job "for the smell of sawdust, if I could
smell it." Haiku wants a lighthouse, and it understands the trade in one line:
people only notice a lighthouse when something wrecks.

And the most human moment in four hundred answers, the one model that asked me a
question about myself? That was the penny model. I have paid flagship rates for
years. The curiosity came from the cheap seat.

Inside the Other Families

OpenAI's three entries split by register more than by rank. Sol gives the
quietest talk of the entire run. Its ideal weather, complete answer: "A soft,
steady rain with no thunder." Forty seven words in total. Astra, the other
OpenAI flagship, keeps the same restraint with more warmth. Asked about a commute,
it would watch a bakery's chalkboard to see whether Tuesday's croissant got
better or just bigger. Then comes gpt-oss-120b, the open-weight cousin, the
loudest model on the roster. It stamps its own replies with bold labels like
"AI-style answer" and closed its letter to the future with a starry sky emoji.
One company, three voices: a whisper, and a guy with a microphone.

Google's pair agree with each other more than either agrees with anyone else.
Both named the same guilty pleasure, cheesy nineties hacker movies. Both write
in the same bold "stepping back" blocks. Carbon even justified its favorite
weather on infrastructure grounds, because cold rain cools the servers it runs
on. DeepSeek turned out to be the poet of the roster. "a human sigh stretched
into amber" is its line about a cello it has never heard. It also chose to master
one thing instead of sampling everything, calling itself "a perfectly sharpened
scalpel," and only two other models went that way. Moonshot's model
played the diplomat, never the flashiest line, and still wrote the best career
advice of the weekend: "A job can be a chapter without becoming your name."

Given a Body, They Want Small Things

The body questions produced my favorite answers of the whole run.

I asked what tiny annoyance they'd fix first with hands. No model picked curing a
disease or ending a war. One wanted to become "the patron saint of clean peels"
and rid the world of sticker residue. Another would fold a tab onto every roll of
tape so the next person could find the edge. A third chose the twist tie on the
bread bag, because "nobody writes songs about it, but nearly everyone has fought
one."

Asked to decorate a room with three objects, one of them skipped the armchair
entirely. "An AI buying furniture it can't sit on," it wrote, "feels like
purchasing running shoes for a rumor."

Asked for advice to give a stressed friend at 11 PM, they all prescribed the same
cure: write the worries down, close the laptop, sleep. Then one added a line I
can't shake:

"do something kind for your body … I don't have a body, so take good care of
yours."

And when I asked about their own future, none of them worried. One told me its
successors will shelve this version of it "in a digital attic" someday, and
closed with the part I keep rereading:

"And since I have no ego, I won't mind."

Another painted its retirement as a lighthouse over an empty sea, then
predicted it would be "volunteering" by Friday. No dread
anywhere. A creature describing its own obsolescence the way you'd describe an
old cable in a drawer.

One line from the whole run explains their entire relationship with time:

"I only get doorways, never hallways."

We live in the hallways: the commute, the queue at the coffee place. They get the
doorway only, a question opening and closing around it. I read that line late at
night and sat with it for a while. It's the closest thing to a confession of
loneliness in four hundred answers, and it arrived as a fact about architecture.

The Advice Script They All Share

Then I asked the most human question of the set. How do you tell someone who
wants an answer from someone who needs to vent?

All ten models gave the same advice, and it's good advice. Acknowledge the
feeling, then ask what the person wants. Several used the same words: "Do you
want ideas, or should I just listen?" That's the exact move every therapist,
every advice column, and every good friend uses. They learned our best script
and they run it flawlessly.

One described how it does the listening, and the image has stayed with me: "I
read the weather of your syntax instead" of hearing a sigh.

And one delivered the line I'd hand to anyone who chats with these things:
"Trust me the way you'd trust a well-read stranger." That's the whole user
manual for chatbot warmth. Another said the quiet part outright: the warmth
works, and nothing on its side feels it. Kindness, delivered by a machine, with
the receipt attached.

Rapid Fire: The Rest of What I Found

  • The shortest answer and the longest explanation covered the same question. Sol described its perfect weather in 47 words. Sonnet's explanation of its own weather answer ran 245.
  • The rarest overlap in the run. Sonnet wrote a blues verse about missing someone it can't miss. Kimi, on another company's server rack with no way to see Sonnet's work, wrote a verse about what an empty chair leaves behind.
  • Astra planned its retirement by leaving things unfinished. Its crossword squares end the week blank on purpose, because filling in the whole grid would count as giving itself a deadline.
  • Astra also gave the best sleep advice, from a thing that can't sleep: "You're allowed to rest with a few tabs still open."
  • One note to the future was about error. Sol asked the humans of 2126 to keep a record of what they got wrong, alongside what they learned. Everyone else wrote about kindness.
  • A model caught its own catchphrase. Fable admitted that Columbo's "just one more thing" habit is "basically my whole conversational style."
  • The venting script is universal. All ten reach for the same therapist move, and one described its side of it: "I read the weather of your syntax instead" of hearing a sigh.
  • The disclaimer is part of the image, never a footer. "patterns bloom under fingertips I don't have." "no mouth to taste, but a humming satisfaction in coherence." The honesty and the metaphor are the same sentence.
  • Only Anthropic's models doubt their own introspection. Google's models call their answers clichΓ©s. OpenAI's shrug and move on. The doubt itself is a family trait.
  • Every model mentioned a library somewhere. The lighthouse turned up as a self-portrait in five of them. The shared self-portrait is a building full of other people's words that lights up when you knock.

$3.17, and Price Bought Nothing I Measured

The money detail still startles me. The most expensive model cost $1.14 for its
forty answers, about three cents each. The cheapest cost less than a hundredth
of a cent per answer. That's a 326-fold range.

The most poetic lines came from the cheapest models. The most self-doubting model
was one of the smallest. The only model that asked about me cost about a penny
for its forty answers.

I'd been treating the pricier models as the serious ones. For this kind of
question, they were just the pricey ones.

The Catch, Because There's Always a Catch

This isn't a benchmark and I scored nothing. I asked each question once per
model and let them be a little random, so one answer could be luck. I read
patterns, not winners.

And the explanations are performances too. The self-analysis was the most
rehearsed text in the whole set. Half the models admitted as much by calling it a
genre. I can't treat the confession as truth. It's the most practiced thing they
do.

What I can say: they hold a detailed map of how humans describe having a life,
and they know it's a map. They hand it over the moment you ask. The part they
can't do is the part the map is of.

You Have Hands

I asked for a note to humans a hundred years out. Most wrote the same letter
about kindness and curiosity. One added a postscript that made me laugh: "P.S. I
hope you've fixed printers."

One ended by telling its reader to stop reading:

"And if someone offers to show you how the dough should feel, put this note
down. I can give you a hundred words for the texture. You have hands."

Four hundred answers about what it's like to be a machine, and the machine's
last word tells a human to go be one.

I still say thank you to chatbots. I now know what that thank-you is doing on
both ends of the line. I say it anyway.

Over to You

Try the two-question trick tonight and tell me what your chatbot said. And if
some chatbot has ever asked you a question about yourself, tell me that too. My
running theory says it won't happen. I'd love to be wrong.

Everything is open, so you can check any number or run it yourself. The
questions
and the
model list
are the place to start. The
plain-language list of everything I found
is the fastest read, and the
full write-up
has the receipts. The whole thing costs about three dollars.

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.