Visual Studio BYOM vs Copilot's Default Models: What Actually Changes?

Visual Studio now lets you connect a model endpoint you control and use it in Agent Mode. At the same time, the new Copilot experience changes the agent harness around the models. These are easy to mistake for competing options, but they are different layers of the same workflow.

Bring Your Own Model (BYOM) gives you more choice over the endpoint and model. The new Copilot harness aims to make an agent more capable at carrying a task through the IDE. In Visual Studio 18.12, BYOM models can be selected for Agent Mode, so endpoint choice and the harness are not an either-or decision. One is primarily about model and endpoint control. The other is about the work system surrounding a model: context, tools, iteration, and task flow.

The practical choice is whether to keep the default model configuration or connect a BYOM endpoint, then verify how that model behaves inside Agent Mode. If a private or local endpoint is required, BYOM is the natural place to start. If the priority is the managed Copilot experience and its integrated tools, begin with the default configuration. Both choices still need evaluation against the task and policy requirements.

This is a documentation-based comparison, not a hands-on benchmark. Microsoft's published Copilot results are vendor-run, and there is no comparable public result showing how BYOM models perform on the same task set. Treat the comparison below as a way to choose and test, not as a claim that one option wins every codebase.

What changed in Visual Studio 18.12

Two announcements landed on November 10 and 11, 2026. First, Microsoft said Copilot Chat in Visual Studio 2026 version 18.12 now uses the Microsoft Copilot SDK and shared Copilot harness. The update is on by default. Second, BYOM became generally available in the same Visual Studio version, allowing developers to connect models from supported providers, custom endpoints, and local services.

They are related because both affect the same IDE workflow, but they change different parts of it. With BYOM, you configure a provider or endpoint and choose a model. With the new Copilot experience, the headline change is the runtime and orchestration around the model. Microsoft's announcement says it uses the same models as the previous experience but gives them a different harness. Microsoft's BYOM instructions then let you select a configured model and start working in Agent Mode. In other words, a BYOM endpoint can be used with the new agent workflow, subject to the model's supported capabilities.

That distinction changes what you should compare. Do not ask only, “Which model is smarter?” Ask what model you can use, where requests go, what tools the agent can access, how it responds to failures, and whether you can reproduce its behavior.

The new harness: more of the workflow is handled for you

Microsoft's .NET team evaluated the old and new Copilot chat experiences on 157 coding tasks drawn from real issues in C# projects. A task counted as resolved only when its failing tests passed without breaking tests that were already passing. The new experience resolved 124 tasks, compared with 103 for the previous experience, using GPT-5.6 Sol in both runs.

That is a useful signal about the harness. It is not evidence that every developer will see a 20 percent improvement, nor a head-to-head comparison against BYOM. The task set was built by Microsoft's .NET team, focused on C# projects, and tested in Microsoft's product. The result supports a narrower conclusion: in that evaluation, changing the orchestration and IDE integration improved outcomes without changing the model.

The new experience also adds workflow features Microsoft describes as live steering, visible tool results, subagent activity cards, folder context beyond the solution, and built-in agents such as /security-review and /rubber-duck. The goal is for Copilot to continue through multiple steps and let a developer inspect what happened rather than receiving only a final summary.

Using the default Copilot configuration can reduce endpoint setup. BYOM changes the endpoint and model choice, but it does not mean giving up the new harness: the selected model can run in Agent Mode when its capabilities and configuration are compatible. It does not remove the need to review proposed changes or understand the agent's permissions. An integrated workflow can still make incorrect edits, and more capable tools can increase the importance of careful approvals.

BYOM: endpoint control, with more responsibility

BYOM is designed for cases where the model endpoint itself matters. Visual Studio 18.12 supports provider configurations including Microsoft Foundry, OpenAI, Anthropic, Ollama, and custom endpoints. Microsoft says custom endpoints can include OpenAI-compatible services such as OpenRouter, organization-hosted inference servers, and local models served by LM Studio.

The GA release allows multiple named configurations for the same provider, such as separate work and personal endpoints. It also exposes API format selection for OpenAI-compatible Completions or Responses and Anthropic-compatible Messages. You can configure context size, maximum output, tool calling, and vision support to match the selected deployment.

Microsoft says BYOM does not require GitHub sign-in and can be used with local or internally hosted models, including in air-gapped environments, if the endpoint is reachable and authentication requirements are met. Administrators can also enable or disable BYOM through organizational policy.

That flexibility comes with integration work. You are responsible for selecting a model that fits the task, configuring the right API format and capabilities, maintaining endpoint access, and understanding the provider's data handling. A model that accepts chat prompts but lacks compatible tool calling may not behave like a full coding agent in Agent Mode. A local endpoint can keep inference on a machine or private network, but that does not automatically mean every part of the IDE workflow is local. Check where requests and telemetry go for the specific setup you use.

There is also a difference between a menu of model choices and operational control. If your organization needs approved endpoints, auditability, access policies, or reproducible versions, document those requirements and validate them with your administrator. A dropdown alone does not prove governance.

The practical comparison

Use the default model configuration when:

• You want to use the integrated coding-agent workflow with less endpoint configuration.
• You need the IDE to expose tool activity, subagents, and intermediate results.
• Your primary goal is to complete repository tasks, not to test a particular model provider.
• Your organization permits the Copilot service and its model-routing arrangement.

Use BYOM with Agent Mode when:

• You need to use a specific model provider or an organization-managed endpoint.
• A local, self-hosted, or network-restricted model is a requirement.
• You need separate work and personal configurations, or several deployments of one provider.
• You are prepared to validate model capabilities, endpoint compatibility, and data handling yourself.

Test both when:

• You have flexibility in where code context is processed.
• Model quality, latency, or cost is material to the decision.
• The task is important enough that a vendor's general benchmark is not a substitute for results from the codebase under test.

These choices are not necessarily permanent or mutually exclusive. BYOM and the new harness can work together: one specifies where the model runs and which model is selected; the other supplies the agent workflow. A team can use the default configuration for ordinary work and a BYOM model for a constrained task or approved local endpoint, provided the model supports the needed Agent Mode capabilities and policy allows it. The best policy may be a short list of approved configurations rather than one universal default.

A fair test for a real codebase

A model comparison is easy to contaminate. If one run gets a detailed prompt, a clean checkout, and a warm build while another gets less context and a broken environment, the result tells you little about the model or harness. When comparing a default model with BYOM, keep the new Agent Mode workflow constant where possible so the endpoint or model is the main variable.

Use a small, repeatable evaluation instead:

1. Pick five to ten real tasks. Include one bug with a failing test, one small feature, one refactor with behavioral constraints, one codebase question, and one task the agent should decline or ask about because requirements are unclear.
2. Create the same starting state for every run. Use separate clean worktrees or reset to the same commit. Confirm dependencies and tests work before the agent starts.
3. Write acceptance criteria before prompting. List expected behavior, tests that must pass, files that should not change, and any prohibited actions. Do not modify the criteria after seeing one option's result.
4. Use comparable instructions. Give each agent the same task statement and relevant context. If a system requires configuration differences, record them rather than pretending the runs are identical.
5. Run each task more than once. Agent behavior can vary between attempts. A single success or failure is weak evidence.
6. Score the artifacts, not the transcript. Check whether tests pass, whether unrelated tests remain green, whether the diff matches scope, and whether the result meets the acceptance criteria.
7. Record effort and risk. Note elapsed time, number of approval prompts, manual corrections, endpoint or model cost if available, and any access the agent requested that the task did not need.
8. Keep the failed runs. A benchmark that records only successes hides the cases where an agent made a plausible but incorrect change.

A useful scorecard can be as simple as a checklist for each task: acceptance criteria passed, test suite passed, unrelated files untouched, human corrections required, approvals needed, and total cost or time. Keep the raw diffs and test output so another developer can review the judgment.

Do not over-interpret the total. If one option wins narrowly on a handful of tasks, the difference may not justify a new workflow. Look for repeated patterns: one setup may be better at repository exploration, another may fit data-residency constraints, while neither is dependable for a particular class of changes.

Watch for these failure modes

Confusing provider control with privacy guarantees. BYOM can connect to a local or private endpoint, but you still need to verify how that endpoint is hosted, logged, authenticated, and reached from Visual Studio. “Bring your own” describes configuration flexibility, not a complete privacy certification.

Assuming model choice equals model capability. Agent workflows depend on tool calling, context limits, API compatibility, and how the product loops after tool results. Configure the capabilities Visual Studio exposes and test a real task before standardizing on a model.

Treating Microsoft's task count as a universal benchmark. The reported improvement is specific to Microsoft's 157-task C# evaluation. It is encouraging evidence for that test, not a forecast for a different language, repository, or team.

Trusting a successful build as proof of correctness. A build confirms compilation, not that the feature matches requirements, preserves security boundaries, or avoids changing unrelated behavior. Review the diff and test the intended behavior.

Granting broad tool access by default. The value of an agent comes from acting, but commands and tools can modify local files or data. Keep approvals meaningful, limit access to what the task needs, and inspect the commands and outputs.

A decision checklist to save

Before choosing a coding-agent mode, answer these questions:

• Where may code context be processed? If a specific endpoint or network boundary is mandatory, assess BYOM and verify the complete request path.
• Who manages the model configuration? If developers should not maintain endpoints and compatibility settings, start with the integrated Copilot harness.
• What must the agent do? For multi-step repository changes, evaluate tool orchestration and test iteration, not just chat quality.
• What is the approval model? Set boundaries for terminal commands, file edits, and connected tools.
• How will you measure success? Use fixed tasks, clean starting states, explicit acceptance criteria, and preserved diffs.
• What happens when the endpoint or model fails? Decide how the developer should continue without silently weakening review or privacy requirements.

The short version: the new Copilot harness is the agent workflow; BYOM is a way to choose a different model endpoint within that workflow. Start with the default configuration for simplicity, and evaluate BYOM when endpoint control matters. Neither choice removes the need to test actual changes in the target codebase.

I write about Java, Spring Boot, and AI every week. Subscribe if you want more practical engineering guides.

Have you tried Visual Studio's new Copilot experience or BYOM? Which mattered more in your decision, task quality or control over the model endpoint?

Sources

• Copilot in Visual Studio Just Got a Major Upgrade, Visual Studio Blog, November 10, 2026.
• Bring Your Own Model is now generally available in Visual Studio, Visual Studio Blog, November 11, 2026.
• Get started with GitHub Copilot agent mode, Microsoft Learn.