Let the user overrule the built-in search guess

Whether an endpoint really runs web search on its own servers is inferred
from the model id, and a gateway can make that inference wrong in either
direction. Both directions turned up within a day of shipping it: an
OpenAI-shaped `/responses` proxy in front of Claude got a server tool it
forwards as an ordinary function, and DeepSeek's Anthropic-shaped
endpoint — which does implement the tool — got refused for not being
Claude. A guess that fails both ways needs a switch, so Web Search grows
one. On by default; off sends every model through the backend, which
behaves the same on every endpoint.

The switch made it worth separating the two questions that were tangled
together. Whether the protocol has anywhere to *put* a server tool is a
hard fact about the request body — Chat Completions does not, and no
setting overrides that, because overriding it yields a request with no
search tool of any kind. Whether the endpoint runs search *for this
model* is the guess. They now have separate names.

DeepSeek is also a first party on the Anthropic shape. Its `/anthropic`
guide lists `server_tool_use` and `web_search_tool_result` as supported
content blocks while marking `code_execution_tool_result` and
`mcp_tool_use` unsupported in the same table, so that list is real rather
than copied from Anthropic's. The family rule was reading "not
Anthropic's own model" as "no native search", which refused a pairing
that works. It is about implementation, not branding. The tool
*declaration* name is the one thing that page does not document; sending
Anthropic's `web_search_20250305` is an assumption, and an unverified
one.

`DuckDuckGo` gains the `(Local)` suffix `Bing` already had. Both are the
same thing — a browser-shaped request whose answer is a page Termind
parses itself — so marking one and not the other was distinguishing
nothing. Applied to both it separates the two backends that have no key
and no contract from the three that do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kaede AkinoandClaude Opus 5 committed 2026-08-26 01:24:49 +08:00
1 parent 42d03d4d98
commit c1af3a189f
7 files changed
+158 -32

No files matched your search

+5 -1
View File
@@ -295,7 +295,11 @@ final class AppModel {
model: AIModel, provider: ModelProvider
) -> (isNative: Bool, local: WebSearchClient?) {
guard SettingsKey.isWebSearchEnabled else { return (false, nil) }
if provider.supportsNativeWebSearch(model: model) { return (true, nil) }
if SettingsKey.isBuiltInWebSearchEnabled,
provider.supportsNativeWebSearch(model: model)
{
return (true, nil)
}
let defaults = UserDefaults.standard
let backend = SearchBackend(
rawValue: defaults.string(forKey: SettingsKey.webSearchBackend) ?? ""
+17
View File
@@ -46,6 +46,23 @@ nonisolated enum SettingsKey {
static var isWebSearchEnabled: Bool {
UserDefaults.standard.object(forKey: webSearchEnabled) as? Bool ?? defaultWebSearchEnabled
}
/// Let a provider run the search on its own servers where it can.
///
/// On by default, because where it works it beats a snippet API: the
/// provider reads whole pages and the model simply knows things. Off is the
/// compatibility position — every model then goes through the backend
/// below, which behaves identically on every endpoint.
///
/// The switch exists because whether a given endpoint really runs the
/// search is a *guess*: `ModelProvider.supportsNativeWebSearch(model:)`
/// infers it from the model id, and a gateway can make that inference wrong
/// in either direction. This is the escape hatch for when it is.
static let webSearchBuiltIn = "search.builtIn"
static let defaultWebSearchBuiltIn = true
static var isBuiltInWebSearchEnabled: Bool {
UserDefaults.standard.object(forKey: webSearchBuiltIn) as? Bool
?? defaultWebSearchBuiltIn
}
/// A `SearchBackend` raw value. Unrecognised falls back to the default.
static let webSearchBackend = "search.backend"
/// Blank means the backend's own default, exactly as a model provider's
+14 -11
View File
@@ -12,14 +12,12 @@ import Foundation
nonisolated enum SearchBackend: String, Codable, Sendable, CaseIterable, Identifiable {
/// Reads `bing.com/search` the way a browser would, with no account and no
/// key. Ported from Kelivo's `BingSearchService`, down to the `b_algo`
/// selectors, and named "(Local)" for the same reason Kelivo names it that:
/// it is the user's own machine making an ordinary web request, not an API
/// with a contract behind it.
/// selectors.
///
/// The default, and reachability is why. Bing answers from mainland China;
/// `.duckDuckGo` does not, and a default that times out for a whole country
/// is not a default. Every other candidate needs a key, so this is the only
/// one that can be switched on out of the box everywhere.
/// The default, and reachability is why: it answers from networks where
/// `.duckDuckGo` is blocked, and a default that times out for a large part
/// of the world is not a default. Every other candidate needs a key, so
/// this is the only one that can be switched on out of the box everywhere.
///
/// Its relevance is the weak point, and it is worth knowing before trusting
/// it. Bing degrades an anonymous request to roughly first-token matching:
@@ -37,7 +35,7 @@ nonisolated enum SearchBackend: String, Codable, Sendable, CaseIterable, Identif
/// Markedly better results than `.bing` where it is reachable: against the
/// live endpoints, "CVE-2024-3094 xz backdoor" returns the NVD record, the
/// CVE record and the Wikipedia article, where Bing returns none of them.
/// Worth switching to anywhere outside mainland China.
/// Worth switching to wherever the endpoint resolves.
case duckDuckGo
case brave
case tavily
@@ -48,10 +46,15 @@ nonisolated enum SearchBackend: String, Codable, Sendable, CaseIterable, Identif
nonisolated var id: String { rawValue }
/// "(Local)" marks the two that are not APIs: the request leaves this Mac
/// looking like a browser and the answer is a web page Termind reads
/// itself. Borrowed from Kelivo, and worth carrying because the difference
/// is one a user is entitled to know — no key, no contract, and it stops
/// working the day the markup moves.
var displayName: String {
switch self {
case .bing: "Bing (Local)"
case .duckDuckGo: "DuckDuckGo"
case .duckDuckGo: "DuckDuckGo (Local)"
case .brave: "Brave Search"
case .tavily: "Tavily"
case .exa: "Exa"
@@ -120,9 +123,9 @@ nonisolated enum SearchBackend: String, Codable, Sendable, CaseIterable, Identif
var detail: String {
switch self {
case .bing:
String(localized: "No account or key, and reachable from mainland China. Results for an anonymous request are only loosely related to the query; prefer DuckDuckGo or a keyed backend where you can reach one.")
String(localized: "No account or key. Reachable almost anywhere, but an anonymous request comes back with only loosely related results — prefer DuckDuckGo or a keyed backend where one works for you.")
case .duckDuckGo:
String(localized: "No account or key, and better results than Bing — but not reachable from mainland China.")
String(localized: "No account or key, and markedly better results than Bing. Not reachable from every network.")
case .brave:
String(localized: "Subscription token from the Brave Search API dashboard.")
case .tavily:
+33 -17
View File
@@ -191,15 +191,27 @@ nonisolated struct ModelProvider: Identifiable, Hashable, Sendable {
return !(usesVertex && kind == .google)
}
/// Whether this endpoint accepts a *server-side* search tool in its request
/// body — the kind the provider runs itself and never tells the agent
/// runtime about.
/// Whether this wire protocol has anywhere to *put* a server-side search
/// tool. A hard fact about the request body, not a guess.
///
/// Chat Completions is excluded on purpose. OpenAI's `web_search_options`
/// exists there only on the two `*-search-preview` SKUs, while `.openAI` is
/// also every compatible gateway this app talks to — OpenRouter, DeepSeek,
/// Groq, Together, Ollama. Sending them an option they have never heard of
/// turns a working setup into a 400 for a feature nobody asked for.
/// Chat Completions has nowhere. OpenAI's `web_search_options` exists there
/// only on the two `*-search-preview` SKUs, while `.openAI` is also every
/// compatible gateway this app talks to — OpenRouter, DeepSeek, Groq,
/// Together, Ollama. Sending them an option they have never heard of turns
/// a working setup into a 400 for a feature nobody asked for. Nothing
/// overrides this, including the user, because overriding it would produce
/// a request with no search tool in it at all.
var acceptsNativeWebSearchTool: Bool {
switch kind {
case .anthropic, .google: true
case .openAI: usesResponsesAPI
}
}
/// Whether the *model* is one this endpoint runs search for.
///
/// A guess, unlike `acceptsNativeWebSearchTool`, and `WebSearchMode` exists
/// so the user can overrule it either way.
///
/// The wire protocol alone is not enough, which is the part that had to be
/// learned. A server tool exists at the intersection of *this vendor's API*
@@ -210,22 +222,26 @@ nonisolated struct ModelProvider: Identifiable, Hashable, Sendable {
/// has one registered — because native mode deliberately withholds the
/// local tool — and the run reports `Unknown tool: web_search`.
///
/// So the model family has to agree with the endpoint. A Claude id on an
/// OpenAI-shaped endpoint means a translator sits in between, and the right
/// answer there is the local `web_search` tool, which works everywhere.
/// DeepSeek is the case that shows the rule is about *implementation*
/// rather than branding. It ships both shapes and implements the server
/// tool in each: `/responses` takes `{"type": "web_search"}` and answers
/// with `web_search_call` items, while `/anthropic` documents
/// `server_tool_use` and `web_search_tool_result` as supported content
/// blocks — marking `code_execution_tool_result` and `mcp_tool_use`
/// unsupported in the same table, so that list is real and not copied. It
/// is a first party on both, and reading "not Anthropic's own model" as "no
/// native search" would refuse a pairing that works.
func supportsNativeWebSearch(model: AIModel) -> Bool {
guard model.abilities.contains(.webSearch) else { return false }
guard model.abilities.contains(.webSearch), acceptsNativeWebSearchTool else {
return false
}
let id = model.id.lowercased()
switch kind {
case .anthropic:
return id.contains("claude")
return id.contains("claude") || id.contains("deepseek")
case .google:
return id.contains("gemini")
case .openAI:
// DeepSeek belongs here: its `/responses` implements the identical
// `{"type": "web_search"}` server tool, so it is a first party for
// this purpose even though it is reached OpenAI-shaped.
guard usesResponsesAPI else { return false }
return id.contains("gpt") || id.contains("deepseek")
|| id.range(of: #"(^|[-_/])o[3-9]([-._/]|$)"#, options: .regularExpression) != nil
}
@@ -12,6 +12,8 @@ struct WebSearchSettings: View {
@AppStorage(SettingsKey.webSearchEnabled) private var isEnabled =
SettingsKey.defaultWebSearchEnabled
@AppStorage(SettingsKey.webSearchBuiltIn) private var usesBuiltIn =
SettingsKey.defaultWebSearchBuiltIn
@AppStorage(SettingsKey.webSearchBackend) private var backendID = SearchBackend.default.rawValue
@AppStorage(SettingsKey.webSearchBaseURL) private var baseURL = ""
@AppStorage(SettingsKey.webSearchCredentialRef) private var credentialRef = ""
@@ -39,10 +41,18 @@ struct WebSearchSettings: View {
Section {
Toggle("Enable Web Search", isOn: $isEnabled)
Toggle("Prefer Built-in Web Search", isOn: $usesBuiltIn)
.disabled(!isEnabled)
Text(
"Models that can search on their own do so directly, and no backend below is "
+ "used for them. Everything else gets a web_search tool backed by the "
+ "backend below. Either way the assistant reads a result page with fetch."
usesBuiltIn
? "Providers that run search on their own servers do it themselves, and "
+ "read whole pages rather than snippets. Anything else falls back to "
+ "the backend below. Whether an endpoint really does is inferred from "
+ "the model, and a gateway can make that wrong — turn this off if "
+ "search fails or the model reports an unknown tool."
: "Every model uses the backend below, which behaves the same on every "
+ "endpoint."
)
.font(.caption)
.foregroundStyle(.secondary)
@@ -291,6 +291,41 @@ struct ModelCapabilityResolverTests {
#expect(google.supportsNativeWebSearch(model: gemini))
}
/// DeepSeek ships both shapes and implements the server tool in each, so
/// the family rule has to be about implementation rather than branding.
/// Reading "not Anthropic's own model" as "no native search" would refuse a
/// pairing that works.
@Test
func deepSeekIsAFirstPartyOnBothShapes() {
var model = AIModel(id: "deepseek-v4-pro")
model.abilities = [.toolCall, .webSearch]
// https://api.deepseek.com/anthropic — `server_tool_use` and
// `web_search_tool_result` are documented as supported there.
var anthropicShaped = ModelProvider(name: "DeepSeek via Anthropic")
anthropicShaped.kind = .anthropic
anthropicShaped.baseURL = "https://api.deepseek.com/anthropic"
#expect(anthropicShaped.supportsNativeWebSearch(model: model))
// https://api.deepseek.com/v1/responses — `{"type": "web_search"}`.
var responsesShaped = ModelProvider(name: "DeepSeek via Responses")
responsesShaped.baseURL = "https://api.deepseek.com/v1"
responsesShaped.usesResponsesAPI = true
#expect(responsesShaped.supportsNativeWebSearch(model: model))
// Chat Completions has no search on any shape, and the older models
// never get the ability in the first place.
var chat = ModelProvider(name: "DeepSeek Chat")
chat.baseURL = "https://api.deepseek.com/v1"
#expect(!chat.supportsNativeWebSearch(model: model))
var legacyModel = AIModel(id: "deepseek-chat")
legacyModel.abilities = ModelCapabilityResolver.resolve(
model: AIModel(id: "deepseek-chat"), provider: responsesShaped
).model.abilities
#expect(!legacyModel.abilities.contains(.webSearch))
}
@Test
func searchIsNotInferredForModelsThatCannotChat() {
let provider = ModelProvider(name: "OpenAI")
+41
View File
@@ -359,6 +359,47 @@ struct WebSearchTests {
#expect(with.contains { $0.descriptor.name == "web_search" })
}
// MARK: - The built-in switch
/// The escape hatch, and the one thing it must not do is leave a model with
/// no search at all: turning built-in off has to hand the work to the
/// backend, not drop it.
@Test
func turningOffBuiltInSendsEverythingThroughTheBackend() {
var model = AIModel(id: "gpt-5.4")
model.abilities = [.toolCall, .webSearch]
var provider = ModelProvider(name: "OpenAI")
provider.usesResponsesAPI = true
// With it on, this pairing is the one case that goes native.
#expect(provider.supportsNativeWebSearch(model: model))
// The switch is read in `AppModel.webSearchPlan` ahead of this call, so
// what it guards is exactly this predicate — and the local client is
// keyless by default, which is what makes the fallback always there.
#expect(WebSearchClient(backend: .default).isConfigured)
}
/// Chat Completions has nowhere in the request body to put a server tool,
/// which is a fact about the protocol rather than a guess about the model —
/// so it sits on its own property and no setting overrides it.
@Test
func protocolSupportIsSeparateFromTheModelGuess() {
var provider = ModelProvider(name: "Gateway")
#expect(!provider.acceptsNativeWebSearchTool)
provider.usesResponsesAPI = true
#expect(provider.acceptsNativeWebSearchTool)
var anthropic = ModelProvider(name: "Anthropic")
anthropic.kind = .anthropic
#expect(anthropic.acceptsNativeWebSearchTool)
var google = ModelProvider(name: "Google")
google.kind = .google
#expect(google.acceptsNativeWebSearchTool)
}
// MARK: - Provider-native search must stay out of the transcript
/// The bug this pins produced, on the *second* request of a conversation: