Scope chat agents with citations and multi-pass search

Limit the agent to the conversation that opened the task, keep a citation on every message you use, and search in passes before you summarize. For a project catch-up, Jaime DeLanghe described one keyword spinning off five related searches, then a cluster of interest, then a pared-down set, then the summary. Recency and engagement find the slice. Correctness still depends on the citation.

Jaime DeLanghe is Chief Product Officer at Slack. The episode aired in September 2026.

How do you scope which conversations an agent can read?

Start from the conversation that opened the task; membership limits the catch-up fan-out, and broader access still depends on installation and permissions.

Someone mentions the agent in a conversation. From there it can call APIs that people in the workspace do not have and open a channel for the work. People from that conversation can join the development environment the agent prepared. The channel winds down when the task is done.

Put the originating conversation id on the job and default every search to that id. A project catch-up is the exception. The fan-out then stays inside channels that person already belongs to. She said Slack can hold both broad shared context and specific grounded context, depending on the constraints you set.

You have more of an agent operating in the loop with humans. So it's really easy to create these loops inside of a channel. The notification system inside of Slack is kind of built for loops, human loops, in the past and now agent loops today.

Jaime DeLanghe, Chief Product Officer at Slack, on Chain of Thought ep 71

Slack Wants to Be the Context … - Chain of Thought | AI Agents, Infrastructure & Engineering - Apple Podcasts

Podcast Episode · Chain of Thought | AI Agents, Infrastructure & Engineering · September 2 · 55m

podcasts.apple.com

How do you cite a chat answer so a bad source stays visible?

Rank by recency and engagement, and keep a message id on every claim you return.

Search APIs are the ones they use most, and the ones they were focused on making work for agents at scale. Agents call them to ground themselves in company conversation before they act. That company context is what makes an in-company assistant different from a generic one. Slackbot, the agent that comes with Slack, is built to cite. It very rarely just tells you something. She had seen an uncited reply once or twice. They tried to ground the system prompt in Slack context as much as possible. More connected MCPs ground the same agent in more sources.

Files and other work land in chat even when nobody planned a knowledge base. She called the result an accidental knowledge system. It is strong on what just happened and pretty good at what drew engagement, which mostly yields what was accurate to the moment. It is not always the best at knowing what is correct. The most recent item is much more likely to be right than an outdated knowledge base. Show the id beside the sentence so a person can throw the hit out.

... it's really good at knowing what's most recent. And it's pretty good at knowing what's most engaged with. And if you put those things together, you mostly get what's most accurate to the moment.

Jaime DeLanghe, Chief Product Officer at Slack, on Chain of Thought ep 71

How do you run a project catch-up across many channels?

Run the search in passes in your application code, narrow the hits, and only then call the model.

Her own version, as CPO, is a request to catch her up on a project. There is no single channel. The search looks across channels she is a member of, finds project-related text, and follows nearby terms. A keyword turns into related keywords: five searches, a cluster of interest, a pared-down set, then the summary.

Slackbot does that multi-path search in the application layer, so the model sees a chosen subset. Third-party agents hit the search API and still do not get that multi-pass path. Slack is working on doing more of that data handling for agents in the second half of the year, including rate limits and context-window management. She said the raw data hose stays for developers who want to manage their own context window.

Until that helper exists, own the passes.

• Cap the related terms.
• Search only member channels.
• Drop duplicates.
• Rank by recency and engagement.
• Pass text and ids to the model.

A minimal sketch to adapt, not a drop-in library.

def project_catchup(project, member_channels, search_messages, call_model):
    terms = related_keywords(project)[:5]
    hits = []
    for term in terms:
        hits.extend(search_messages(member_channels, term))
    chosen = pare_down(rank_recent_engaged(dedupe(hits)))
    cited = [{"text": item["text"], "cite": item["id"]} for item in chosen]
    return call_model("Summarize only the cited messages.", cited)

related_keywords, pare_down, and rank_recent_engaged are stand-ins. The slice of five matches the related-search fan-out she described. search_messages receives only channels the requester is already in. The list passed to call_model still carries an id.

So it finds a keyword, and then it'll spin off five searches that have related keywords that sort of, like, give it a cluster of interest, and then it'll pare down what's in that, and then it'll use that to build the summary.

Jaime DeLanghe, Chief Product Officer at Slack, on Chain of Thought ep 71

FAQ

Which conversation should the first search use?
The one the task started in. Widen a catch-up only into channels that person already belongs to; other access still follows installation and permissions.

How many searches make a catch-up?
One keyword leads to five related searches, then a narrower set, then a summary. With only the search API, rate limits and context-window size stay with the developer.

Can I trust the highest-ranked message?
Treat the top hit as a candidate. Recency and engagement mostly surface what was accurate to the moment. She also said Slack is not always the best at knowing what is correct. The citation is how a person overrides the rank.

Takeaway

• Install the agent, then mention it in the conversation that owns the task.
• Default search to that conversation. On a project catch-up, stay inside channels the user is already in.
• Keep a citation on every message the model is allowed to use.
• Search in passes: a keyword, related terms (she described five), a narrower set, then a summary.
• Show the citation beside each claim so a person can throw out a bad hit.

The full conversation, with the transcript, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.