Voicemail Detection Is a Classifier. Plan for Both Ways It Is Wrong.

Most of the voice agents I build answer the phone. Someone dialed the business, so a human is on the line, and the hard problems are latency, transfers and not making things up.

Outbound flips that. Appointment reminders, speed-to-lead callbacks, lead reactivation: every one of those campaigns dials people who did not ask for a call right now, and a big share of those dials never reach a person. They reach a voicemail greeting, a carrier's default message, call screening, or a phone tree.

The first time you build outbound, voicemail feels like an edge case. It is not. On a lot of lists it is the most common outcome. And the thing that decides "person or machine" is a classifier, which means it will be wrong some of the time, in both directions. Once I started treating it that way, most of the design fell out on its own.

What goes wrong when the agent cannot tell

If the agent treats a greeting like a person, three things happen.

It talks to the recording. The greeting says "Hi, you've reached Mike, leave a message," and the agent cheerfully replies "Hi Mike, is now a good time?" Mike later gets half a conversation in his inbox.

It burns minutes. Every second spent pitching a machine is billed.

It wrecks the numbers. If voicemails get logged as answered calls, the conversion rate looks terrible and the client blames the agent, which barely spoke to a human all day.

None of these is a prompting problem.

Detection is a classifier, so think in two error types

Retell has a voicemail detection setting for outbound calls. It listens to the start of the call and decides whether it reached a person or a machine, and you choose what happens on a machine: hang up, or leave a message. Option names change as Retell ships updates, so check the current dashboard instead of trusting an old screenshot.

The way it decides is roughly how you would. A person says a short "hello?" and stops. A machine talks for a while without pausing, then beeps. That means it needs a few seconds of audio before it commits, and it can be fooled:

• Someone who answers with a long "Hello, this is Mike speaking, how can I help you?" sounds a lot like a greeting.
• Some voicemail greetings are a single word.
• Call screening and phone trees behave like neither.

So there are two ways to be wrong, and they cost different things.

When you see it as a table, the design question changes from "how do I make detection perfect" (you cannot) to "which mistake is cheaper for this campaign, and how do I make both mistakes survivable."

Make the opening line work for both answers

The single most useful change is the first few seconds.

Let the other side speak first. If the agent opens with a pitch, it talks over the greeting while detection is still listening, and it talks over a human who said "hello." If it waits, a person says something short and a machine starts its greeting. Either way detection gets clean audio.

I will be honest here: when I went back through my own agent exports, every one of them has start_speaker set to agent, the outbound ones included. That is exactly the setting I now tell people to change for outbound. It is easy to miss, because a test call where you pick up your own phone and say hello works fine either way.

Keep the first line short. "Hi, is this Mike?" is fine whether a person or a misclassified machine hears it. A thirty second opener is not.

Hang up or leave a message is a product decision

Every one of my exported agent configs uses the same voicemail action:

"voicemail_option": { "action": { "type": "hangup" } }

That is a reasonable default, but it should be a decision, not a default. It depends on the campaign:

• Appointment reminders: leave a short message. The customer expects the business to call.
• Speed-to-lead callbacks: leave a short message and send a text. The lead just asked to be contacted.
• Lead reactivation: hang up on the first attempt, leave a message on the last one.
• Anything medical, legal or financial: minimal message or none. Voicemail is heard by whoever holds the phone.

If the agent leaves a message, it should be a fixed script, not something the model improvises. Improvised voicemails ramble. My rules: under fifteen seconds, business name in the first sentence, one reason for the call, one callback number said twice, nothing sensitive, approved word for word by the client.

And because plenty of people never check voicemail at all, I pair it with an SMS through n8n or GoHighLevel when the contact has consented to texts. In practice the text often does more work than the voicemail. The plumbing for that is the same post-call webhook I described in An AI Voice Agent Is Just Two Webhooks.

Every call ends in exactly one outcome

This is the part I care about most as an engineer, because it is where the reporting either tells the truth or does not.

In VoiceDash, the portal we built for agencies to hand their clients, each call is stored with an end reason taken straight from Retell:

endReason: call.disconnection_reason || call.call_status,

That is useful for debugging and the analytics page charts it, but a raw disconnection reason is not a business outcome. "The agent hung up" can mean the booking succeeded, or it can mean the agent heard a beep and left. The client needs a different field.

So I define a closed list of outcomes and make every call land in exactly one:

// Illustrative: the shape I aim for, not code from a specific repo
type CallOutcome =
  | "connected_goal_met"
  | "connected_goal_not_met"
  | "voicemail_message_left"
  | "voicemail_no_message"
  | "no_answer_or_busy"
  | "phone_tree_ended"
  | "wrong_or_disconnected";

Retell's post-call analysis can extract a structured field from the transcript, and my agent configs already use it for things like qualified and appointment_type. Adding an outcome enum next to those is cheap. The point is that it is an enum, not free text, so the funnel can be computed instead of eyeballed: dials, connects, voicemails, and conversions among the people you actually reached.

A campaign with a modest connect rate and a strong conversion rate among connects is a good campaign. If you only show "calls made" and "appointments booked," you hide that, and the agent gets blamed for the list.

Retries are the same problem in a different hat

If a call hits voicemail, something has to decide whether and when to try again. My defaults:

1. Two or three attempts per contact per campaign, never unlimited.
2. Different days and different times of day.
3. One voicemail per contact, not one per attempt. Three identical messages from the same number is how a business gets reported as spam.
4. Any opt-out removes the contact from every campaign for that client, immediately.
5. Calls only in the contact's local daytime hours.

Outbound calling rules vary by country and US state, and AI voices are a specific focus for regulators. I am an engineer, not a lawyer, so the consent question goes to the client and their counsel before any list is loaded.

Test against real greetings, not your own phone

A start_speaker setting like mine sails through if you only test by calling your own phone. So before a campaign goes live, call a small set of numbers that cover the classifier's weak spots:

• A personal voicemail greeting
• The default carrier greeting
• A phone with call screening turned on
• A business line with a phone tree
• A real person who answers with a long, formal greeting

Then listen to every recording. Did the agent wait? Did detection classify correctly? Did the message start after the beep, not halfway through the greeting? Did the outcome field land in the right bucket? Phone calls are hard to reproduce after the fact, which I wrote about in There Is No Repro for a Phone Call, so this is cheaper done before the list goes live than after.

Outbound clients mostly want to know what the robot is saying to their customers, which is why VoiceDash puts recordings in front of them. A clinic owner can play the exact fifteen seconds the agent left and stop wondering.

What I would tell myself before the first outbound build

Voicemail detection is a classifier. Decide which of its two mistakes is cheaper per campaign, make the opening line survive both, script the message, store one outcome per call as an enum, cap the retries, and test against greetings that are not your own.

I wrote the longer, agency-facing version of this on the VoiceDash blog: Retell AI voicemail detection.

If you run outbound voice agents, how do you handle the "person sounds like a machine" case? Do you lean toward hanging up or toward talking? I am curious where other people landed.