From 42d03d4d98550267b5f080eb018ea4a8c1c2cea6 Mon Sep 17 00:00:00 2001 From: Kaede Akino Date: Wed, 26 Aug 2026 00:57:38 +0800 Subject: [PATCH] Let the agent find a page, not just read one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The agent had thirteen tools and no way to search. `fetch` reads a URL it is handed; nothing produced one, so looking up a CVE, a changelog or an error string meant the model guessing an address. Provider-native search was not wired either: every kind sent only local function declarations, and the parsers knew `text`, `tool_use` and `function_call` and nothing else. Two paths now, and exactly one is live per turn. A model with the `webSearch` ability on an endpoint that actually implements the server tool gets the provider's own search; everything else gets a `web_search` tool backed by a backend configured once in Settings. Offering both would put two tools in front of the model for one job and let its guess decide whether the search appears as a card beforehand or a record afterwards. Chat Completions is excluded. `web_search_options` exists there on two `*-search-preview` SKUs, while `.openAI` is also every compatible gateway this app talks to, and an unknown tool type is a 400 on every message rather than a missing feature. The wire protocol alone turned out not to be enough. A server tool exists where a vendor's API and that vendor's model meet, and a gateway breaks the pairing: an OpenAI-shaped `/responses` endpoint pointed at Claude accepts `{"type": "web_search"}` and forwards it as an ordinary function declaration. The model calls it, native mode has deliberately withheld the local tool, and the run reports `Unknown tool: web_search`. So the model family has to agree with the endpoint before native is chosen. A native search never reaches the tool pipeline, so nothing would be approved, recorded or shown, and the assistant would cite pages the transcript never mentions reaching. `recordNativeWebSearch` raises an ordinary card for it after the fact — but that card is a record, not a turn. Appending it to the model-facing transcript replays a `function_call_output` whose call id matches nothing, which OpenAI answers with `No tool call found for function call output`, and only on the *second* request, because the first is what plants the orphan. `priorMessages` is persisted before the request goes out, so an affected conversation keeps the orphan across restarts; incoming history is now filtered for tool results that answer no call, which repairs it. Anthropic's `server_tool_use` streams `input_json_delta` exactly like `tool_use`. Read as a tool call it produces one with an empty name, and `AgentRuntime` rejects that — so a single native search failed the whole assistant turn. `AnthropicStreamState` exists for this. Bing is the keyless default, and reachability is the reason: it answers from mainland China and DuckDuckGo does not. Its relevance is the weaker half of that trade. Probed live, `mkt` is not optional — without it Bing returns a well-formed page, correctly titled with the query, of results for something else entirely, which is the worst failure available here because the parse succeeds and the model is handed confident nonsense. Even with it, an anonymous request degrades to roughly first-token matching: "CVE-2024-3094 xz backdoor" returns cve.org's landing page. Browser headers, a cookie jar, `form=` and `setlang`/`cc` moved none of it. DuckDuckGo returns the NVD record, the CVE record and the Wikipedia article for the same query and is offered for anyone who can reach it. Snippets only, from every backend. Each can return page text, and taking it would make search a second way to read the web — one whose destination is picked by the engine rather than classified by us. Leaving it out keeps `fetch` the single door, with its address classification, redirect budget and private-network escalation still on the path. `web_search` is `.locallyReadOnly` for a public backend to match what `fetch` became, and escalates for exactly the destinations `fetch` escalates for. DeepSeek v4 needed no code: its `/responses` implements the identical tool type and returns identical `web_search_call` items, so it only had to be recognised. Recognition is a regex over the model id, applied outside the reported-abilities guard on purpose — no `/models` payload carries server-side search and Pi's catalog has no field for it, so a source listing abilities never means it ruled search out. Co-Authored-By: Claude Opus 5 (1M context) --- Termind/App/AppModel.swift | 42 +- Termind/App/SettingsKey.swift | 19 + .../Application/Agent/AgentBuiltInTools.swift | 64 +- Termind/Application/Agent/AgentRuntime.swift | 94 ++- .../Agent/CommandComposerService.swift | 7 +- .../Agent/ConversationTitleService.swift | 7 +- .../Application/Agent/WebSearchClient.swift | 766 ++++++++++++++++++ Termind/Core/LLM/AgentProviderClient.swift | 272 +++++-- .../Core/LLM/ModelCapabilityResolver.swift | 41 + Termind/Core/Models/ModelProvider.swift | 47 ++ Termind/Domain/Agent/AgentTypes.swift | 43 + Termind/Features/AI/ConversationModel.swift | 6 + .../Features/Settings/SettingsPageView.swift | 13 +- .../Features/Settings/WebSearchSettings.swift | 151 ++++ TermindTests/AIProviderPersistenceTests.swift | 101 ++- TermindTests/AgentProviderAdapterTests.swift | 303 ++++++- TermindTests/WebSearchTests.swift | 478 +++++++++++ 17 files changed, 2394 insertions(+), 60 deletions(-) create mode 100644 Termind/Application/Agent/WebSearchClient.swift create mode 100644 Termind/Features/Settings/WebSearchSettings.swift create mode 100644 TermindTests/WebSearchTests.swift diff --git a/Termind/App/AppModel.swift b/Termind/App/AppModel.swift index 1f2cf84a..8ef05c90 100644 --- a/Termind/App/AppModel.swift +++ b/Termind/App/AppModel.swift @@ -209,9 +209,11 @@ final class AppModel { } let mode = ai.resolve(conversation: conversation.permissionMode) + let searchPlan = webSearchPlan(model: chat.model, provider: chat.provider) let model = AgentProviderClient( provider: chat.provider, model: chat.model, secret: secret, - reasoning: conversation.reasoningEffort + reasoning: conversation.reasoningEffort, + webSearch: searchPlan.isNative ) let reviewer: (any SecurityReviewing)? = { guard mode == .approveForMe, let review = ai.resolve(.securityReview) else { return nil } @@ -251,7 +253,10 @@ final class AppModel { var tools: [AnyAgentTool] = [] if chat.model.abilities.contains(.toolCall) { - tools = AgentBuiltInTools.make(hosts: agentHosts, connections: agentConnections) + tools = AgentBuiltInTools.make( + hosts: agentHosts, connections: agentConnections, + search: searchPlan.local + ) let servers = ai.mcpServers.filter(\.isEnabled).map { server in (server: server, bearerToken: ai.secret(for: server.credentialRef)) } @@ -274,6 +279,39 @@ final class AppModel { scheduleTitle(for: conversation) } + /// Which of the two web-search paths this turn gets, resolved once so the + /// request body and the tool list cannot disagree. + /// + /// Native wins where it exists. It searches better than a snippet API can, + /// costs no second key, and needs no round trip through Termind — the model + /// simply knows things. What it gives up is visibility, which + /// `AgentRuntime.recordNativeWebSearch` buys back after the fact. + /// + /// Exactly one path is ever live. Offering both would put two tools in + /// front of the model for one job and leave it to guess which, and the + /// guess would decide whether the search shows up as a card beforehand or a + /// record afterwards. + private func webSearchPlan( + model: AIModel, provider: ModelProvider + ) -> (isNative: Bool, local: WebSearchClient?) { + guard SettingsKey.isWebSearchEnabled else { return (false, nil) } + if provider.supportsNativeWebSearch(model: model) { return (true, nil) } + let defaults = UserDefaults.standard + let backend = SearchBackend( + rawValue: defaults.string(forKey: SettingsKey.webSearchBackend) ?? "" + ) ?? .default + let client = WebSearchClient( + backend: backend, + baseURL: defaults.string(forKey: SettingsKey.webSearchBaseURL) ?? "", + apiKey: ai.secret( + for: defaults.string(forKey: SettingsKey.webSearchCredentialRef) ?? "" + ) ?? "" + ) + // A keyed backend with no key registers nothing rather than a tool that + // fails on first use — see `AgentBuiltInTools.make(search:)`. + return (false, client.isConfigured ? client : nil) + } + /// Arms the one-shot summary for this conversation. /// /// Titling is deliberately once per conversation, not per run: a chat list diff --git a/Termind/App/SettingsKey.swift b/Termind/App/SettingsKey.swift index 29f08593..86e46980 100644 --- a/Termind/App/SettingsKey.swift +++ b/Termind/App/SettingsKey.swift @@ -34,4 +34,23 @@ nonisolated enum SettingsKey { static let sftpPerHostTransferLimit = "sftp.transfers.perHostLimit" static let defaultSFTPGlobalTransferLimit = 3 static let defaultSFTPPerHostTransferLimit = 2 + + /// Let the assistant search the web. + /// + /// On by default, which it can only be because the default backend needs no + /// account: `SearchBackend.default` reads a results page the way a browser + /// would. A default that required a key would be a switch that did nothing + /// until the user went and got one, which is the same as off. + static let webSearchEnabled = "search.enabled" + static let defaultWebSearchEnabled = true + static var isWebSearchEnabled: Bool { + UserDefaults.standard.object(forKey: webSearchEnabled) as? Bool ?? defaultWebSearchEnabled + } + /// A `SearchBackend` raw value. Unrecognised falls back to the default. + static let webSearchBackend = "search.backend" + /// Blank means the backend's own default, exactly as a model provider's + /// blank base URL means its kind's default. + static let webSearchBaseURL = "search.baseURL" + /// Where the search key is filed in the Keychain. Never the key itself. + static let webSearchCredentialRef = "search.credentialRef" } diff --git a/Termind/Application/Agent/AgentBuiltInTools.swift b/Termind/Application/Agent/AgentBuiltInTools.swift index 45ef5846..789153fa 100644 --- a/Termind/Application/Agent/AgentBuiltInTools.swift +++ b/Termind/Application/Agent/AgentBuiltInTools.swift @@ -337,10 +337,17 @@ actor AgentConnectionPool { } nonisolated enum AgentBuiltInTools { + /// `search` is optional, and nil means the tool is not registered at all + /// rather than registered and always failing. A model shown a `web_search` + /// it cannot use spends a turn discovering that; a model not shown one + /// reaches for `fetch` or asks, which is the behaviour we want when no + /// backend is configured. It is also nil whenever the model has provider + /// search of its own — see `AppModel.webSearchPlan`. static func make( hosts: AgentHostResolver, connections: AgentConnectionPool, - fetcher: RestrictedHTTPFetcher = RestrictedHTTPFetcher() + fetcher: RestrictedHTTPFetcher = RestrictedHTTPFetcher(), + search: WebSearchClient? = nil ) -> [AnyAgentTool] { [ listHosts(hosts), sshExec(connections), terminalSuggest(), @@ -348,7 +355,7 @@ nonisolated enum AgentBuiltInTools { sftpWrite(connections), sftpMkdir(connections), sftpMove(connections), sftpChmod(connections), sftpDelete(connections), fetch(fetcher), requestUserInput(), requestUserSecret(), - ] + ] + (search.map { [webSearch($0)] } ?? []) } private static func listHosts(_ hosts: AgentHostResolver) -> AnyAgentTool { @@ -627,6 +634,59 @@ nonisolated enum AgentBuiltInTools { ) } + /// Finding a URL, as the counterpart to `fetch` reading one. + /// + /// Snippets only, deliberately. Every backend here can also return page + /// text, and taking it would make this a second way to read the web — + /// one whose destination is chosen by the search engine rather than + /// classified by us. Leaving it out keeps `fetch` the single door, with its + /// address classification, redirect budget and private-network escalation + /// all still on the path. + /// + /// `.locallyReadOnly` for a public backend, matching `fetch`: a query is an + /// outbound request, not a change to anything. It escalates for the same + /// destinations `fetch` escalates for, which is what stops a search backend + /// configured at a LAN address from being reached without a word. + static func webSearch(_ search: WebSearchClient) -> AnyAgentTool { + let tool = descriptor( + "web_search", + "Search the public web. Returns titles, URLs and snippets only; use fetch to read a " + + "result page in full.", + properties: ["query": string(max: 1_024), "count": integer(min: 1, max: 10)], + required: ["query"], target: .network, safety: .locallyReadOnly + ) + return AnyAgentTool( + descriptor: tool, + preflight: { invocation in + _ = try Arguments(invocation).string("query") + guard try await search.classification() == .restricted else { + return AgentToolPreflight(invocation: invocation, safety: .locallyReadOnly) + } + return AgentToolPreflight( + invocation: invocation, + safety: .requiresAuthorization, + reasons: [ + "The configured search backend is on a private or otherwise non-public " + + "network: \(search.resolvedURL)" + ] + ) + }, + execute: { invocation, _ in + let arguments = try Arguments(invocation) + let rows = try await search.search( + arguments.string("query"), count: arguments.optionalInt("count") ?? 5 + ) + return result(invocation, .array(rows.map { row in + .object([ + "title": .string(row.title), "url": .string(row.url), + "snippet": .string(row.snippet), + "published_at": row.publishedAt.map { .string($0.ISO8601Format()) } ?? .null, + ]) + })) + } + ) + } + private static func requestUserInput() -> AnyAgentTool { AnyAgentTool(descriptor: descriptor( "request_user_input", "Pause and ask the user for text or one structured choice.", diff --git a/Termind/Application/Agent/AgentRuntime.swift b/Termind/Application/Agent/AgentRuntime.swift index e00ccc05..4d4fff2b 100644 --- a/Termind/Application/Agent/AgentRuntime.swift +++ b/Termind/Application/Agent/AgentRuntime.swift @@ -17,6 +17,10 @@ nonisolated enum AgentModelStreamEvent: Sendable { id: String, providerItemID: String?, name: String, arguments: String ) case providerItem(AgentJSONValue) + /// A search the provider ran itself. Distinct from `providerItem`, which is + /// the *replay* channel and must stay byte-shaped the way its provider + /// sent it; this one is normalized for display and is never sent back. + case webSearch(AgentWebSearchActivity) case finished(AgentStopReason) } @@ -268,12 +272,44 @@ actor AgentRuntime { return result } + /// Drops tool results that answer no call in the same history. + /// + /// A `tool` message only means anything beside the assistant turn that + /// asked for it. One left on its own is rejected outright — OpenAI answers + /// `No tool call found for function call output with call_id ...`, and the + /// other two reject the equivalent — so a single orphan makes a + /// conversation permanently unusable rather than degrading it. + /// + /// Applied to incoming history rather than inside each provider's encoder, + /// which is what makes it a repair and not just a guard: `priorMessages` + /// seeds the snapshot that gets persisted *before* the request goes out, so + /// filtering here also stops a stored orphan from being written forward + /// into the next run. + /// + /// Messages with no call id at all are kept. Google addresses a tool result + /// by name rather than by id, so absence here is not evidence of an orphan. + static func withoutOrphanedToolResults( + _ messages: [AgentTranscriptMessage] + ) -> [AgentTranscriptMessage] { + let answered = Set( + messages.lazy + .filter { $0.role == .assistant } + .flatMap { $0.toolCalls.map(\.id) } + ) + return messages.filter { message in + guard message.role == .tool, let id = message.toolCallID, !id.isEmpty else { + return true + } + return answered.contains(id) + } + } + private func run(_ request: AgentRunRequest) async { var snapshot = AgentRunSnapshot( conversationID: request.conversationID, state: .running, permissionMode: request.permissionMode, - messages: request.priorMessages + messages: Self.withoutOrphanedToolResults(request.priorMessages) ) activeRunID = snapshot.id let runID = snapshot.id @@ -367,6 +403,54 @@ actor AgentRuntime { finish(runID: runID) } + /// Raises an ordinary tool card for a search the provider already ran. + /// + /// Proposed and finished in one breath, with no approval between, because + /// by the time this event exists there is nothing left to authorize — the + /// search happened on the provider's servers during the turn. The card is a + /// record rather than a gate, which is the honest thing to show and better + /// than an answer that cites pages the transcript never mentions reaching. + /// + /// The result content deliberately carries no `` wrapper. + /// That wrapper is addressed to a model, and this content is never sent to + /// one: it exists only for the card and for run history. + private func recordNativeWebSearch(_ activity: AgentWebSearchActivity) { + guard let runID = activeRunID else { return } + let invocation = AgentToolInvocation( + runID: runID, + call: AgentToolCall( + id: "provider-web-search-\(UUID().uuidString)", + name: "web_search", + arguments: .object(["query": .string(activity.query)]) + ) + ) + let descriptor = AgentToolDescriptor( + name: "web_search", + summary: String(localized: "Web search run by the model provider."), + inputSchema: .object([ + "type": .string("object"), + "properties": .object(["query": .object(["type": .string("string")])]), + "required": .array([.string("query")]), + "additionalProperties": .bool(false), + ]), + target: .network, + safety: .locallyReadOnly + ) + let sources = AgentJSONValue.array(activity.sources.map { source in + .object(["title": .string(source.title), "url": .string(source.url)]) + }) + emit(.toolProposed(invocation, descriptor)) + emit(.toolFinished(invocation, AgentToolResult( + callID: invocation.call.id, + content: sources.encodedString, + // The flag every reader of this event needs: `callID` here names + // nothing the provider ever asked for, so anything that replays + // tool results must leave this one out. See + // `AgentToolResult.isProviderNative`. + metadata: [AgentToolResult.providerNativeKey: .bool(true)] + ))) + } + private func assistantTurn( systemPrompt: String, messages: [AgentTranscriptMessage] @@ -375,6 +459,10 @@ actor AgentRuntime { var pending: [String: PendingToolCall] = [:] var order: [String] = [] var stopReason = AgentStopReason.completed + // One card per distinct query. Gemini reports its grounding metadata on + // more than one chunk of the same stream, and a card per chunk would + // read as the model having searched four times for one thing. + var recordedSearches: Set = [] emit(.messageStarted(message.id)) let request = AgentModelRequest( @@ -416,6 +504,10 @@ actor AgentRuntime { } else { message.providerItems.append(item) } + case .webSearch(let activity): + if recordedSearches.insert(activity.query).inserted { + recordNativeWebSearch(activity) + } case .finished(let reason): stopReason = reason } diff --git a/Termind/Application/Agent/CommandComposerService.swift b/Termind/Application/Agent/CommandComposerService.swift index 514aa3d4..c9784fe5 100644 --- a/Termind/Application/Agent/CommandComposerService.swift +++ b/Termind/Application/Agent/CommandComposerService.swift @@ -56,9 +56,14 @@ actor CommandComposerService { text = snapshot case .toolCallDelta, .toolCallSnapshot: throw CommandComposerError.invalidResponse - case .providerItem: + case .providerItem, .webSearch: // Some reasoning providers require opaque output items in agent // turns. This one-shot, tool-free request never replays them. + // + // A search is ignored rather than rejected for the same reason: + // this client never asks for one, so its only source is a + // gateway that searches regardless, and failing to compose a + // command over that would help nobody. continue case .finished(let reason): stopReason = reason diff --git a/Termind/Application/Agent/ConversationTitleService.swift b/Termind/Application/Agent/ConversationTitleService.swift index eb3cf062..e0264496 100644 --- a/Termind/Application/Agent/ConversationTitleService.swift +++ b/Termind/Application/Agent/ConversationTitleService.swift @@ -65,9 +65,14 @@ actor ConversationTitleService { text = snapshot case .toolCallDelta, .toolCallSnapshot: throw ConversationTitleError.invalidResponse - case .providerItem: + case .providerItem, .webSearch: // Some reasoning providers require opaque output items in agent // turns. This one-shot, tool-free request never replays them. + // + // A search is ignored rather than rejected for the same reason: + // this client never asks for one, so its only source is a + // gateway that searches regardless, and losing the conversation + // title over that would help nobody. continue case .finished(let reason): stopReason = reason diff --git a/Termind/Application/Agent/WebSearchClient.swift b/Termind/Application/Agent/WebSearchClient.swift new file mode 100644 index 00000000..04fe3c93 --- /dev/null +++ b/Termind/Application/Agent/WebSearchClient.swift @@ -0,0 +1,766 @@ +import Foundation + +/// Which search endpoint's wire protocol Termind speaks. +/// +/// Four, and for the same reason `ModelProviderKind` is three: these are +/// *shapes*, not vendors. Unlike chat completions there is no de-facto standard +/// one of them can stand in for — Bing answers a GET with a web page, Brave +/// with `web.results`, Tavily with `results[].content`, Exa with +/// `results[].text` — so each shape genuinely costs a case. A self-hosted or +/// compatible endpoint is reached by overriding `baseURL` on whichever shape it +/// imitates, which is what keeps this list from growing a case per vendor. +nonisolated enum SearchBackend: String, Codable, Sendable, CaseIterable, Identifiable { + /// Reads `bing.com/search` the way a browser would, with no account and no + /// key. Ported from Kelivo's `BingSearchService`, down to the `b_algo` + /// selectors, and named "(Local)" for the same reason Kelivo names it that: + /// it is the user's own machine making an ordinary web request, not an API + /// with a contract behind it. + /// + /// The default, and reachability is why. Bing answers from mainland China; + /// `.duckDuckGo` does not, and a default that times out for a whole country + /// is not a default. Every other candidate needs a key, so this is the only + /// one that can be switched on out of the box everywhere. + /// + /// Its relevance is the weak point, and it is worth knowing before trusting + /// it. Bing degrades an anonymous request to roughly first-token matching: + /// verified live, "nginx 502 bad gateway upstream timeout" returns + /// nginx.org's home page and download page, and "CVE-2024-3094 xz backdoor" + /// returns cve.org's landing page — with the query echoed correctly in the + /// page title and the terms appearing nowhere in the results. A full set of + /// browser headers, a cookie jar, `form=`, `setlang`/`cc` and `mkt` in + /// every combination moved none of it; only `mkt` mattered, and only to + /// stop the results being unrelated *and* in the wrong language. + case bing + /// Reads DuckDuckGo's HTML endpoint the way a browser would, also with no + /// account and no key. + /// + /// Markedly better results than `.bing` where it is reachable: against the + /// live endpoints, "CVE-2024-3094 xz backdoor" returns the NVD record, the + /// CVE record and the Wikipedia article, where Bing returns none of them. + /// Worth switching to anywhere outside mainland China. + case duckDuckGo + case brave + case tavily + case exa + + /// One place to change what an unset or unrecognised preference means. + static let `default` = SearchBackend.bing + + nonisolated var id: String { rawValue } + + var displayName: String { + switch self { + case .bing: "Bing (Local)" + case .duckDuckGo: "DuckDuckGo" + case .brave: "Brave Search" + case .tavily: "Tavily" + case .exa: "Exa" + } + } + + var requiresAPIKey: Bool { + switch self { + case .bing, .duckDuckGo: false + case .brave, .tavily, .exa: true + } + } + + /// Whether the answer is a web page rather than an API payload. The two + /// that are share a byte budget, a browser user agent, and the caveat that + /// they break when markup moves. + var readsMarkup: Bool { + switch self { + case .bing, .duckDuckGo: true + case .brave, .tavily, .exa: false + } + } + + /// Carries whatever version segment the vendor's own documentation prints, + /// and none where it prints none — the same rule + /// `ModelProviderKind.defaultBaseURL` follows, for the same reason: the + /// user pastes what they were given. + var defaultBaseURL: String { + switch self { + case .duckDuckGo: "https://html.duckduckgo.com" + case .bing: "https://www.bing.com" + case .brave: "https://api.search.brave.com/res/v1" + case .tavily: "https://api.tavily.com" + case .exa: "https://api.exa.ai" + } + } + + var path: String { + switch self { + case .duckDuckGo: "/html/" + case .bing: "/search" + case .brave: "/web/search" + case .tavily, .exa: "/search" + } + } + + /// Redirects are refused outright for the keyed backends: their request + /// carries an API key in a header, and following a redirect would hand that + /// key to whatever host the response named. The keyless two carry no + /// credential and do bounce through region and consent hops, so they get a + /// small budget. + var maximumRedirects: Int { + readsMarkup ? 3 : 0 + } + + /// A search-results page is two orders of magnitude larger than a JSON + /// answer, and truncating one mid-markup loses results rather than trimming + /// them. + var responseByteLimit: Int { + readsMarkup ? 4 * 1_024 * 1_024 : 512 * 1_024 + } + + /// Where to get a key, shown under the field. Every keyed backend here is + /// bring-your-own — Termind holds no search credential of its own, exactly + /// as it holds no model credential of its own. + var detail: String { + switch self { + case .bing: + String(localized: "No account or key, and reachable from mainland China. Results for an anonymous request are only loosely related to the query; prefer DuckDuckGo or a keyed backend where you can reach one.") + case .duckDuckGo: + String(localized: "No account or key, and better results than Bing — but not reachable from mainland China.") + case .brave: + String(localized: "Subscription token from the Brave Search API dashboard.") + case .tavily: + String(localized: "API key from the Tavily dashboard.") + case .exa: + String(localized: "API key from the Exa dashboard.") + } + } +} + +nonisolated struct WebSearchResult: Hashable, Sendable { + var title: String + var url: String + var snippet: String + var publishedAt: Date? +} + +/// One search backend, configured once in Settings. +/// +/// Shares `RestrictedHTTPFetcher`'s address classifier: the resolved endpoint is +/// re-resolved immediately before the request, so a mistyped self-hosted base +/// URL cannot quietly aim at something on the user's own network. The +/// classification is repeated here rather than trusted from preflight because +/// DNS can change between the two, and it is what makes an unspecified or +/// multicast address a throw rather than a connection. +/// +/// Whether a *restricted* result is allowed is not decided here — `web_search` +/// escalates to approval in preflight for exactly the destinations `fetch` +/// does, so a search backend configured at a LAN address is disclosed rather +/// than silently reached. +nonisolated struct WebSearchClient: Sendable { + typealias Classifier = @Sendable (URL) async throws -> RestrictedNetworkScope + + var backend: SearchBackend + /// Blank means `backend.defaultBaseURL`. + var baseURL: String + /// Empty for backends that need none. `requiresAPIKey` decides whether that + /// is a configuration error or the ordinary case. + var apiKey: String + var timeout: TimeInterval + /// Only Bing reads it. Sent so a user in a Chinese or Japanese locale gets + /// their own results rather than an American default, which for a search + /// engine is a correctness matter and not a preference. + var acceptLanguage: String + /// Bing's market, `language-REGION`. + /// + /// Not optional, and not a nicety. Verified against the live endpoint: with + /// no `mkt`, Bing answers an anonymous request with a page that is + /// well-formed, correctly titled with the query, and full of `b_algo` + /// results for something else entirely — a search for "swift 6 concurrency" + /// came back as Chinese-language threads about ChatGPT. Sending a cookie + /// jar made it worse rather than better. + /// + /// That is the worst failure this feature could have. The request + /// succeeds, the parse succeeds, and the model is handed confident + /// nonsense with no signal that anything went wrong. `mkt` is the whole + /// difference between that and results that match the query. + var market: String + var classify: Classifier + + init( + backend: SearchBackend, + baseURL: String = "", + apiKey: String = "", + timeout: TimeInterval = 20, + acceptLanguage: String = WebSearchClient.preferredLanguages, + market: String = WebSearchClient.preferredMarket, + classify: @escaping Classifier = PublicAddressValidator.classify + ) { + self.backend = backend + self.baseURL = baseURL + self.apiKey = apiKey + self.timeout = timeout + self.acceptLanguage = acceptLanguage + self.market = market + self.classify = classify + } + + /// A browser user agent, because these two requests really are that: an + /// ordinary page load. Both engines serve a stripped page to anything that + /// does not look like a browser, and the stripped page has none of the + /// result markup the parsers below read. + /// + /// Safari on macOS specifically. The request is coming from a Mac over + /// Apple's own networking stack, so claiming Chrome on Windows would make + /// the user agent the one part of it that disagrees with every other part. + static let browserUserAgent = + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 " + + "(KHTML, like Gecko) Version/18.5 Safari/605.1.15" + + static var preferredLanguages: String { + let codes = Locale.preferredLanguages.prefix(3) + guard !codes.isEmpty else { return "en-US,en;q=0.9" } + return codes.enumerated() + .map { index, code in index == 0 ? code : "\(code);q=\(1.0 - 0.1 * Double(index))" } + .joined(separator: ",") + } + + /// Falls back to `en-US` a component at a time rather than all at once, so + /// a Mac with a language but no region still searches in its own language. + static var preferredMarket: String { + let locale = Locale.current + let language = locale.language.languageCode?.identifier ?? "en" + let region = locale.region?.identifier ?? "US" + return "\(language)-\(region)" + } + + var resolvedBaseURL: String { + let typed = baseURL.trimmingCharacters(in: .whitespaces) + let value = typed.isEmpty ? backend.defaultBaseURL : typed + return value.hasSuffix("/") ? String(value.dropLast()) : value + } + + var resolvedURL: String { resolvedBaseURL + backend.path } + + /// True when this backend is usable as configured. A keyless backend is + /// always ready, which is what lets Web Search work the moment it is + /// switched on. + var isConfigured: Bool { + !backend.requiresAPIKey || !apiKey.isEmpty + } + + /// Where the query would actually go, for the tool's preflight check and + /// for the caption under the Settings fields. + func classification() async throws -> RestrictedNetworkScope { + guard let url = URL(string: resolvedURL) else { throw WebSearchError.invalidEndpoint } + return try await classify(url) + } + + func search(_ query: String, count: Int) async throws -> [WebSearchResult] { + let trimmed = query.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { throw WebSearchError.emptyQuery } + guard isConfigured else { throw WebSearchError.missingKey } + let limit = min(max(count, 1), 10) + + let request = try buildRequest(query: trimmed, count: limit) + guard let url = request.url else { throw WebSearchError.invalidEndpoint } + _ = try await classify(url) + + let delegate = SearchRedirectDelegate( + maximumRedirects: backend.maximumRedirects, classify: classify + ) + let configuration = URLSessionConfiguration.ephemeral + configuration.timeoutIntervalForRequest = timeout + configuration.timeoutIntervalForResource = timeout + configuration.waitsForConnectivity = false + configuration.httpCookieStorage = nil + configuration.urlCredentialStorage = nil + configuration.requestCachePolicy = .reloadIgnoringLocalAndRemoteCacheData + let session = URLSession( + configuration: configuration, delegate: delegate, delegateQueue: nil + ) + defer { session.invalidateAndCancel() } + + let (bytes, response) = try await session.bytes(for: request) + guard let http = response as? HTTPURLResponse else { + throw WebSearchError.invalidResponse + } + var data = Data() + for try await byte in bytes { + if data.count >= backend.responseByteLimit { break } + data.append(byte) + } + if (300..<400).contains(http.statusCode) { throw WebSearchError.blockedRedirect } + guard (200..<300).contains(http.statusCode) else { + throw WebSearchError.http( + http.statusCode, String(decoding: data.prefix(2_048), as: UTF8.self) + ) + } + return try Self.decode(data, backend: backend, limit: limit) + } + + /// DuckDuckGo's region, which is `mkt` with its halves swapped and + /// lowercased: `en-US` here is `us-en` there. + var duckDuckGoRegion: String { + let parts = market.split(separator: "-") + guard parts.count == 2 else { return "wt-wt" } + return "\(parts[1].lowercased())-\(parts[0].lowercased())" + } + + func buildRequest(query: String, count: Int) throws -> URLRequest { + switch backend { + case .duckDuckGo: + guard let url = URL(string: resolvedURL) else { throw WebSearchError.invalidEndpoint } + var request = URLRequest(url: url) + request.httpMethod = "POST" + request.setValue( + "application/x-www-form-urlencoded", forHTTPHeaderField: "Content-Type" + ) + request.setValue("text/html", forHTTPHeaderField: "Accept") + request.setValue(acceptLanguage, forHTTPHeaderField: "Accept-Language") + request.setValue(Self.browserUserAgent, forHTTPHeaderField: "User-Agent") + var form = URLComponents() + form.queryItems = [ + URLQueryItem(name: "q", value: query), + URLQueryItem(name: "kl", value: duckDuckGoRegion), + ] + request.httpBody = Data((form.percentEncodedQuery ?? "").utf8) + return request + + case .bing: + guard var components = URLComponents(string: resolvedURL) else { + throw WebSearchError.invalidEndpoint + } + components.queryItems = + (components.queryItems ?? []) + + [ + URLQueryItem(name: "q", value: query), + URLQueryItem(name: "mkt", value: market), + ] + guard let url = components.url else { throw WebSearchError.invalidEndpoint } + var request = URLRequest(url: url) + request.httpMethod = "GET" + request.setValue("text/html", forHTTPHeaderField: "Accept") + request.setValue(acceptLanguage, forHTTPHeaderField: "Accept-Language") + request.setValue(Self.browserUserAgent, forHTTPHeaderField: "User-Agent") + return request + + case .brave: + guard var components = URLComponents(string: resolvedURL) else { + throw WebSearchError.invalidEndpoint + } + components.queryItems = + (components.queryItems ?? []) + + [ + URLQueryItem(name: "q", value: query), + URLQueryItem(name: "count", value: String(count)), + ] + guard let url = components.url else { throw WebSearchError.invalidEndpoint } + var request = URLRequest(url: url) + request.httpMethod = "GET" + request.setValue("application/json", forHTTPHeaderField: "Accept") + request.setValue(apiKey, forHTTPHeaderField: "X-Subscription-Token") + return request + + case .tavily: + return try postRequest( + headers: ["Authorization": "Bearer \(apiKey)"], + body: ["query": query, "max_results": count] + ) + + case .exa: + // `contents.text` with a character cap is what keeps this a snippet + // rather than a page fetch. Reading a page in full stays the job of + // `fetch`, which is the one path with a redirect budget and address + // classification on the *target* rather than on the search API. + return try postRequest( + headers: ["x-api-key": apiKey], + body: [ + "query": query, "numResults": count, + "contents": ["text": ["maxCharacters": 600]], + ] + ) + } + } + + private func postRequest( + headers: [String: String], body: [String: Any] + ) throws -> URLRequest { + guard let url = URL(string: resolvedURL) else { throw WebSearchError.invalidEndpoint } + var request = URLRequest(url: url) + request.httpMethod = "POST" + request.setValue("application/json", forHTTPHeaderField: "Content-Type") + request.setValue("application/json", forHTTPHeaderField: "Accept") + for (name, value) in headers { request.setValue(value, forHTTPHeaderField: name) } + request.httpBody = try JSONSerialization.data(withJSONObject: body) + return request + } + + // MARK: - Decoding + + static func decode( + _ data: Data, backend: SearchBackend, limit: Int + ) throws -> [WebSearchResult] { + guard backend.readsMarkup else { + return try decodeJSON(data, backend: backend, limit: limit) + } + let html = String(decoding: data, as: UTF8.self) + return backend == .bing + ? bingResults(html, limit: limit) + : duckDuckGoResults(html, limit: limit) + } + + /// Lenient on purpose. Every field but the URL is optional on the wire, and + /// a row missing its URL is dropped rather than thrown on: one malformed + /// result should not lose the other nine. + private static func decodeJSON( + _ data: Data, backend: SearchBackend, limit: Int + ) throws -> [WebSearchResult] { + guard let root = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { + throw WebSearchError.invalidResponse + } + let rows: [[String: Any]] + switch backend { + case .brave: + let web = root["web"] as? [String: Any] + rows = web?["results"] as? [[String: Any]] ?? [] + case .tavily, .exa: + rows = root["results"] as? [[String: Any]] ?? [] + case .bing, .duckDuckGo: + rows = [] + } + return rows.prefix(limit).compactMap { row in + guard let url = row["url"] as? String, !url.isEmpty else { return nil } + let snippet: String? = + switch backend { + case .brave: row["description"] as? String + case .tavily: row["content"] as? String + case .exa: row["text"] as? String + case .bing, .duckDuckGo: nil + } + let published: String? = + switch backend { + case .brave: (row["page_age"] ?? row["age"]) as? String + case .tavily: row["published_date"] as? String + case .exa: row["publishedDate"] as? String + case .bing, .duckDuckGo: nil + } + return WebSearchResult( + title: plainText(row["title"] as? String ?? ""), + url: url, + snippet: plainText(snippet ?? ""), + publishedAt: published.flatMap(date) + ) + } + } + + // MARK: - DuckDuckGo result page + + /// Reads `result__a` for the link and `result__snippet` for the summary, + /// skipping any block whose class marks it an ad. + /// + /// Scanned with regular expressions rather than parsed, because this app + /// ships no HTML parser and adding one for three selectors is not a trade + /// worth making. That is a real limitation: this returns an empty list when + /// the markup moves, which reads to the model as "no results" rather than + /// as a failure, and is why the keyed backends are worth configuring. + static func duckDuckGoResults(_ html: String, limit: Int) -> [WebSearchResult] { + var results: [WebSearchResult] = [] + for block in markupBlocks(html, opener: duckDuckGoBlockPattern, classGroup: 1) { + guard results.count < limit else { break } + // `result--ad` and `result--sponsored` wear the same result markup. + guard !block.classes.contains("result--ad"), + !block.classes.contains("result--sponsored") + else { continue } + guard let link = firstMatch(duckDuckGoLinkPattern, in: block.body), link.count >= 3 + else { continue } + let url = unwrapDuckDuckGoRedirect(plainText(link[1])) + guard url.hasPrefix("http") else { continue } + let snippet = firstMatch(duckDuckGoSnippetPattern, in: block.body)?.dropFirst().first + results.append(WebSearchResult( + title: plainText(link[2]), + url: url, + snippet: plainText(snippet ?? ""), + publishedAt: nil + )) + } + return results + } + + /// DuckDuckGo sometimes serves links direct and sometimes through + /// `//duckduckgo.com/l/?uddg=`. Both shapes appear on the + /// same endpoint, so neither can be assumed. + static func unwrapDuckDuckGoRedirect(_ raw: String) -> String { + guard raw.contains("/l/"), raw.contains("uddg=") else { return raw } + let absolute = raw.hasPrefix("//") ? "https:" + raw : raw + guard let components = URLComponents(string: absolute), + let target = components.queryItems?.first(where: { $0.name == "uddg" })?.value, + target.hasPrefix("http") + else { return raw } + return target + } + + private static let duckDuckGoBlockPattern = try? NSRegularExpression( + pattern: #"])[^>]*\bclass="([^"]*\bresult\b[^"]*)"[^>]*>"#, + options: [.caseInsensitive] + ) + private static let duckDuckGoLinkPattern = try? NSRegularExpression( + pattern: #"]*\bclass="[^"]*\bresult__a\b[^"]*"[^>]*\bhref="([^"]+)"[^>]*>(.*?)"#, + options: [.caseInsensitive, .dotMatchesLineSeparators] + ) + private static let duckDuckGoSnippetPattern = try? NSRegularExpression( + pattern: #"\bclass="[^"]*\bresult__snippet\b[^"]*"[^>]*>(.*?)"#, + options: [.caseInsensitive, .dotMatchesLineSeparators] + ) + + // MARK: - Bing result page + + /// Reads the same three selectors Kelivo does — `li.b_algo`, `h2 > a`, and + /// `.b_algoSlug` / `.b_caption p`. + /// + /// Scanned with regular expressions rather than parsed, because this app + /// ships no HTML parser and adding one for three selectors is not a trade + /// worth making. That is a real limitation and the reason a keyed backend + /// is worth configuring: this returns an empty list when the markup moves, + /// which reads to the model as "no results" rather than as a failure. + static func bingResults(_ html: String, limit: Int) -> [WebSearchResult] { + var results: [WebSearchResult] = [] + for block in markupBlocks(html, opener: bingBlockPattern, classGroup: 1) { + guard results.count < limit else { break } + guard let link = firstMatch(bingLinkPattern, in: block.body), link.count >= 3 else { + continue + } + let url = unwrapBingRedirect(plainText(link[1])) + guard url.hasPrefix("http") else { continue } + let snippet = + firstMatch(bingSlugPattern, in: block.body)?.dropFirst().first + ?? firstMatch(bingParagraphPattern, in: block.body)?.dropFirst().first + results.append(WebSearchResult( + title: plainText(link[2]), + url: url, + snippet: plainText(snippet ?? ""), + publishedAt: nil + )) + } + return results + } + + // MARK: - Shared markup scanning + + nonisolated struct MarkupBlock: Sendable { + var classes: String + var body: String + } + + /// Each organic result runs from its own opening tag to the next one. + /// + /// Slicing on the next opener rather than on a matching close tag is what + /// makes this survive the nesting both engines use inside a result, which a + /// regular expression cannot balance. The trailing slice runs to the end of + /// the document and so carries the page footer with it — harmless, because + /// every selector below is anchored to result markup that appears once. + private static func markupBlocks( + _ html: String, opener: NSRegularExpression?, classGroup: Int + ) -> [MarkupBlock] { + guard let opener else { return [] } + let range = NSRange(html.startIndex..., in: html) + let matches = opener.matches(in: html, range: range).compactMap { + match -> (classes: String, start: String.Index)? in + guard let whole = Range(match.range, in: html) else { return nil } + // `range(at:)` traps rather than returning nil when the pattern has + // no such group, so the bound is checked rather than assumed: a + // pattern edited to drop its capture would otherwise take the app + // down on the next search. + let classes = + classGroup < match.numberOfRanges + ? Range(match.range(at: classGroup), in: html).map { String(html[$0]) } ?? "" + : "" + return (classes, whole.upperBound) + } + return matches.enumerated().map { index, match in + let end = index + 1 < matches.count ? matches[index + 1].start : html.endIndex + return MarkupBlock(classes: match.classes, body: String(html[match.start..`. + /// Handing that to the model is handing it a URL that says nothing about + /// the source, so the real one is recovered where it is recoverable. + static func unwrapBingRedirect(_ raw: String) -> String { + guard raw.contains("/ck/a"), + let components = URLComponents(string: raw), + let encoded = components.queryItems?.first(where: { $0.name == "u" })?.value, + encoded.hasPrefix("a1") + else { return raw } + var base64 = String(encoded.dropFirst(2)) + .replacingOccurrences(of: "-", with: "+") + .replacingOccurrences(of: "_", with: "/") + while base64.count % 4 != 0 { base64 += "=" } + guard let data = Data(base64Encoded: base64), + let url = String(data: data, encoding: .utf8), + url.hasPrefix("http") + else { return raw } + return url + } + + /// `(?=[\s>])` after the tag name, or `])[^>]*\bclass="([^"]*\bb_algo\b[^"]*)"[^>]*>"#, + options: [.caseInsensitive] + ) + private static let bingLinkPattern = try? NSRegularExpression( + pattern: #"]*>.*?]*\bhref="([^"]+)"[^>]*>(.*?)"#, + options: [.caseInsensitive, .dotMatchesLineSeparators] + ) + private static let bingSlugPattern = try? NSRegularExpression( + pattern: #"]*\bclass="[^"]*\bb_algoSlug\b[^"]*"[^>]*>(.*?)

"#, + options: [.caseInsensitive, .dotMatchesLineSeparators] + ) + private static let bingParagraphPattern = try? NSRegularExpression( + pattern: #"]*>(.*?)

"#, + options: [.caseInsensitive, .dotMatchesLineSeparators] + ) + + /// Returns the whole match followed by its capture groups, so callers index + /// them the way the pattern reads. + private static func firstMatch( + _ regex: NSRegularExpression?, in text: String + ) -> [String]? { + guard let regex else { return nil } + let range = NSRange(text.startIndex..., in: text) + guard let match = regex.firstMatch(in: text, range: range) else { return nil } + return (0..`, and Bing's + /// snippets are raw page markup. The model reads these as text, so the tags + /// are noise that costs tokens and invites a snippet to be mistaken for + /// instructions in markup. + static func plainText(_ raw: String) -> String { + numericEntitiesDecoded( + raw.replacingOccurrences(of: "<[^>]+>", with: "", options: .regularExpression) + ) + .replacingOccurrences(of: " ", with: " ") + .replacingOccurrences(of: "<", with: "<") + .replacingOccurrences(of: ">", with: ">") + .replacingOccurrences(of: """, with: "\"") + .replacingOccurrences(of: "'", with: "'") + // Last, or an escaped entity such as `&lt;` decodes twice. + .replacingOccurrences(of: "&", with: "&") + .trimmingCharacters(in: .whitespacesAndNewlines) + } + + /// Decodes `·` and `'` generally, rather than listing the ones + /// seen so far. + /// + /// Bing writes its date separator as `· ` and its apostrophes as + /// `'`, so a fixed list of named entities leaves raw `·` sitting + /// in the snippet the model reads. There is no bound on which code points a + /// result page will use, so this handles the form rather than the instances. + private static func numericEntitiesDecoded(_ text: String) -> String { + guard text.contains("&#") else { return text } + var output = "" + var rest = Substring(text) + while let marker = rest.range(of: "&#") { + output += rest[.. Date? { + let fractional = ISO8601DateFormatter() + fractional.formatOptions = [.withInternetDateTime, .withFractionalSeconds] + if let value = fractional.date(from: raw) { return value } + if let value = ISO8601DateFormatter().date(from: raw) { return value } + let day = DateFormatter() + day.locale = Locale(identifier: "en_US_POSIX") + day.timeZone = TimeZone(secondsFromGMT: 0) + day.dateFormat = "yyyy-MM-dd" + return day.date(from: raw) + } +} + +/// Bounds redirects and re-classifies every hop. +/// +/// `RestrictedHTTPFetcher` has its own, which also carries the caller's custom +/// headers across origins and escalates private destinations to approval. None +/// of that applies here: a search request has no user-supplied headers, and a +/// backend that redirects onto a private address is a thing to refuse rather +/// than a thing to ask about. +nonisolated private final class SearchRedirectDelegate: NSObject, URLSessionTaskDelegate, + @unchecked Sendable +{ + private let lock = NSLock() + private let maximumRedirects: Int + private let classify: WebSearchClient.Classifier + private var redirectCount = 0 + + init(maximumRedirects: Int, classify: @escaping WebSearchClient.Classifier) { + self.maximumRedirects = maximumRedirects + self.classify = classify + } + + func urlSession( + _ session: URLSession, task: URLSessionTask, + willPerformHTTPRedirection response: HTTPURLResponse, + newRequest request: URLRequest, + completionHandler: @escaping @Sendable (URLRequest?) -> Void + ) { + let allowed = lock.withLock { () -> Bool in + redirectCount += 1 + return redirectCount <= maximumRedirects + } + guard allowed, let url = request.url else { + completionHandler(nil) + return + } + Task { [classify] in + do { + guard try await classify(url) == .publicInternet else { + completionHandler(nil) + return + } + completionHandler(request) + } catch { completionHandler(nil) } + } + } +} + +nonisolated enum WebSearchError: LocalizedError, Sendable { + case emptyQuery, missingKey, invalidEndpoint, invalidResponse, blockedRedirect + case http(Int, String) + + var errorDescription: String? { + switch self { + case .emptyQuery: String(localized: "The search query is empty.") + case .missingKey: String(localized: "Web Search has no API key configured.") + case .invalidEndpoint: String(localized: "The Web Search endpoint URL is invalid.") + case .invalidResponse: + String(localized: "The Web Search endpoint returned an unsupported response format.") + case .blockedRedirect: + String(localized: "The Web Search endpoint redirected somewhere Termind will not follow.") + case .http(let status, let body): + String(localized: "The Web Search endpoint returned \(status): \(body)") + } + } +} diff --git a/Termind/Core/LLM/AgentProviderClient.swift b/Termind/Core/LLM/AgentProviderClient.swift index 169c8c68..1145aa92 100644 --- a/Termind/Core/LLM/AgentProviderClient.swift +++ b/Termind/Core/LLM/AgentProviderClient.swift @@ -7,6 +7,13 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { private let model: AIModel private let secret: String private let reasoning: ReasoningEffort + /// Ask the provider to run web search on its own servers this turn. + /// + /// Off by default so the short, tool-free features — titling, the command + /// composer, Security Review — cannot acquire it by accident. Only the + /// Chat/Agent path sets it, and only after `AppModel` has checked both the + /// model's ability and `ModelProvider.supportsNativeWebSearch`. + private let webSearch: Bool private let session: URLSession init( @@ -14,12 +21,14 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { model: AIModel, secret: String, reasoning: ReasoningEffort = .medium, + webSearch: Bool = false, session: URLSession? = nil ) { self.provider = provider self.model = model self.secret = secret self.reasoning = reasoning + self.webSearch = webSearch self.session = session ?? Self.shared } @@ -42,7 +51,7 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { let task = Task { do { var chatCallIDs: [Int: String] = [:] - var anthropicCallIDs: [Int: String] = [:] + var anthropicState = AnthropicStreamState() var responseCallIDs: [String: String] = [:] var responseCallNames: [String: String] = [:] var urlRequest = try buildRequest(request, streaming: true) @@ -78,7 +87,7 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { if event.data == "[DONE]" { break } for item in parse( event, chatCallIDs: &chatCallIDs, - anthropicCallIDs: &anthropicCallIDs, + anthropicState: &anthropicState, responseCallIDs: &responseCallIDs, responseCallNames: &responseCallNames ) { @@ -177,24 +186,28 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { let reasoningValue = reasoningResolution.wireValue(for: normalizedReasoning) switch provider.kind { case .openAI where provider.usesResponsesAPI: - let tools = request.tools.map(Self.openAIToolObject) + var tools: [[String: Any]] = request.tools.map(Self.openAIToolObject).map { tool in + var responseTool = tool + responseTool["type"] = "function" + return responseTool + } + if webSearch { tools.append(["type": "web_search"]) } var body: [String: Any] = [ "model": model.id, "stream": streaming, "instructions": request.systemPrompt, "input": request.messages.flatMap(Self.responsesMessages), ] - if !tools.isEmpty { - body["tools"] = tools.map { tool in - var responseTool = tool - responseTool["type"] = "function" - return responseTool - } - } + if !tools.isEmpty { body["tools"] = tools } if let reasoningValue { body["reasoning"] = ["effort": reasoningValue] } return body case .openAI: + // No search here, whatever the model claims. `web_search_options` + // exists on two `*-search-preview` SKUs, while this branch is also + // every OpenAI-compatible gateway Termind talks to — see + // `ModelProvider.supportsNativeWebSearch`, which is what actually + // prevents it reaching this far. let tools = request.tools.map(Self.openAIToolObject) var messages: [[String: Any]] = [["role": "system", "content": request.systemPrompt]] messages += request.messages.flatMap(Self.chatMessages) @@ -205,29 +218,40 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { if let reasoningValue { body["reasoning_effort"] = reasoningValue } return body case .anthropic: - let tools = request.tools.map(Self.providerToolObject) + var tools: [[String: Any]] = request.tools.map(Self.providerToolObject).map { tool in + ["name": tool["name"]!, "description": tool["description"]!, "input_schema": tool["parameters"]!] + } + if webSearch { + tools.append([ + "type": "web_search_20250305", "name": "web_search", + "max_uses": Self.nativeWebSearchMaxUses, + ]) + } var body: [String: Any] = [ "model": model.id, "stream": streaming, "max_tokens": model.maxOutputTokens ?? 4096, "system": request.systemPrompt, "messages": request.messages.flatMap(Self.anthropicMessages), ] - if !tools.isEmpty { - body["tools"] = tools.map { tool in - ["name": tool["name"]!, "description": tool["description"]!, "input_schema": tool["parameters"]!] - } - } + if !tools.isEmpty { body["tools"] = tools } if let reasoningValue { body["output_config"] = ["effort": reasoningValue] } return body case .google: - let tools = request.tools.map(Self.providerToolObject) + let declarations = request.tools.map(Self.providerToolObject) + // Two entries, not two keys in one entry. Gemini's `tools` is a + // list of tool *groups*, and putting `google_search` beside + // `functionDeclarations` inside one object is the arrangement it + // rejects. + var tools: [[String: Any]] = [] + if !declarations.isEmpty { tools.append(["functionDeclarations": declarations]) } + if webSearch { tools.append(["google_search": [String: Any]()]) } var body: [String: Any] = [ "systemInstruction": ["parts": [["text": request.systemPrompt]]], "contents": request.messages.map(Self.googleMessage), ] - if !tools.isEmpty { body["tools"] = [["functionDeclarations": tools]] } + if !tools.isEmpty { body["tools"] = tools } if let reasoningValue { body["generationConfig"] = [ "thinkingConfig": ["thinkingLevel": reasoningValue] @@ -237,6 +261,11 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { } } + /// A ceiling on searches per turn, which only Anthropic's tool takes as a + /// parameter. Eight is enough for a model to refine a query a few times and + /// low enough that a loop costs a bounded number of billed searches. + static let nativeWebSearchMaxUses = 8 + private func authorize(_ request: inout URLRequest) async throws { switch provider.kind { case .openAI: request.setValue("Bearer \(secret)", forHTTPHeaderField: "Authorization") @@ -253,7 +282,7 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { private func parse( _ event: SSEEvent, chatCallIDs: inout [Int: String], - anthropicCallIDs: inout [Int: String], + anthropicState: inout AnthropicStreamState, responseCallIDs: inout [String: String], responseCallNames: inout [String: String] ) -> [AgentModelStreamEvent] { @@ -272,7 +301,7 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { ) case .openAI: return Self.parseChat(root, callIDs: &chatCallIDs) case .anthropic: - return Self.parseAnthropic(eventType, root, callIDs: &anthropicCallIDs) + return Self.parseAnthropic(eventType, root, state: &anthropicState) case .google: return Self.parseGoogle(root) } } @@ -297,14 +326,14 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { // framing the already-buffered body rather than feeding `data: ...` // into JSONSerialization and surfacing Foundation's opaque error. var chatCallIDs: [Int: String] = [:] - var anthropicCallIDs: [Int: String] = [:] + var anthropicState = AnthropicStreamState() var responseCallIDs: [String: String] = [:] var responseCallNames: [String: String] = [:] var output: [AgentModelStreamEvent] = [] for event in SSEStream.events(in: data) where event.data != "[DONE]" { output += parse( event, chatCallIDs: &chatCallIDs, - anthropicCallIDs: &anthropicCallIDs, + anthropicState: &anthropicState, responseCallIDs: &responseCallIDs, responseCallNames: &responseCallNames ) @@ -511,6 +540,9 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { { return [.providerItem(value)] } + if item["type"] as? String == "web_search_call" { + return responsesWebSearch(item) + } guard item["type"] as? String == "function_call", let sourceID = (item["id"] ?? item["call_id"]) as? String, let name = item["name"] as? String ?? callNames[sourceID] @@ -554,6 +586,8 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { switch item["type"] as? String { case "reasoning": if let value = jsonValue(item) { events.append(.providerItem(value)) } + case "web_search_call": + events += responsesWebSearch(item) case "message": for content in item["content"] as? [[String: Any]] ?? [] { if content["type"] as? String == "output_text", @@ -629,12 +663,16 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { static func parseCompletedAnthropic(_ root: [String: Any]) -> [AgentModelStreamEvent] { guard root["type"] as? String == "message" else { return [] } var events: [AgentModelStreamEvent] = [] + // The buffered body has both halves of a search in hand, so the query + // is matched to its results here rather than parked across events the + // way the streaming parser has to. + var queries: [String: String] = [:] for block in root["content"] as? [[String: Any]] ?? [] { - if block["type"] as? String == "text", let text = block["text"] as? String { - events.append(.textDelta(text)) - } else if block["type"] as? String == "tool_use", - let name = block["name"] as? String - { + switch block["type"] as? String { + case "text": + if let text = block["text"] as? String { events.append(.textDelta(text)) } + case "tool_use": + guard let name = block["name"] as? String else { continue } let data = block["input"].flatMap { try? JSONSerialization.data(withJSONObject: $0) } ?? Data("{}".utf8) @@ -642,6 +680,16 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { id: block["id"] as? String ?? UUID().uuidString, name: name, arguments: String(decoding: data, as: UTF8.self) )) + case "server_tool_use": + guard let id = block["id"] as? String else { continue } + queries[id] = (block["input"] as? [String: Any])?["query"] as? String ?? "" + case "web_search_tool_result": + let query = (block["tool_use_id"] as? String).flatMap { queries[$0] } + events.append(.webSearch(AgentWebSearchActivity( + query: query ?? "", sources: anthropicSources(block["content"]) + ))) + default: + continue } } events.append(.finished(events.containsToolCall ? .toolCalls : .completed)) @@ -649,42 +697,127 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { } static func parseAnthropic(_ type: String, _ root: [String: Any]) -> [AgentModelStreamEvent] { - var callIDs: [Int: String] = [:] - return parseAnthropic(type, root, callIDs: &callIDs) + var state = AnthropicStreamState() + return parseAnthropic(type, root, state: &state) } static func parseAnthropic( - _ type: String, _ root: [String: Any], callIDs: inout [Int: String] + _ type: String, _ root: [String: Any], state: inout AnthropicStreamState ) -> [AgentModelStreamEvent] { - if type == "content_block_start", let block = root["content_block"] as? [String: Any], - block["type"] as? String == "tool_use" - { - let index = root["index"] as? Int ?? 0 - let id = block["id"] as? String ?? "index-\(index)" - callIDs[index] = id - return [.toolCallDelta( - id: id, - name: block["name"] as? String, arguments: "" - )] + let index = root["index"] as? Int ?? 0 + + if type == "content_block_start", let block = root["content_block"] as? [String: Any] { + switch block["type"] as? String { + case "tool_use": + let id = block["id"] as? String ?? "index-\(index)" + state.callIDs[index] = id + return [.toolCallDelta( + id: id, + name: block["name"] as? String, arguments: "" + )] + case "server_tool_use": + // The provider's own call, not the model's. Claiming the index + // here is the whole point: `server_tool_use` streams + // `input_json_delta` exactly like `tool_use` does, and letting + // those deltas fall through to the tool-call branch below + // manufactures a call with an empty name — which `AgentRuntime` + // rejects with `invalidToolCall`, failing the entire turn. + state.serverBlocks[index] = AnthropicStreamState.ServerBlock( + id: block["id"] as? String ?? "", name: block["name"] as? String ?? "" + ) + return [] + case "web_search_tool_result": + state.serverBlocks[index] = AnthropicStreamState.ServerBlock() + // Anthropic sends the query and its hits as two sibling blocks. + // The query was parked at the previous block's stop so one card + // can carry both. + let query = (block["tool_use_id"] as? String) + .flatMap { state.pendingQueries.removeValue(forKey: $0) } + return [.webSearch(AgentWebSearchActivity( + query: query ?? "", sources: anthropicSources(block["content"]) + ))] + default: + return [] + } } + if type == "content_block_delta", let delta = root["delta"] as? [String: Any] { + if state.serverBlocks[index] != nil { + if let json = delta["partial_json"] as? String { + state.serverBlocks[index]?.input += json + } + return [] + } if let text = delta["text"] as? String { return [.textDelta(text)] } if let json = delta["partial_json"] as? String { - let index = root["index"] as? Int ?? 0 return [.toolCallDelta( - id: callIDs[index] ?? "index-\(index)", name: nil, arguments: json + id: state.callIDs[index] ?? "index-\(index)", name: nil, arguments: json )] } } + + if type == "content_block_stop", + let block = state.serverBlocks.removeValue(forKey: index), !block.id.isEmpty + { + state.pendingQueries[block.id] = anthropicQuery(block.input) + return [] + } + if type == "message_stop" { return [.finished(.completed)] } return [] } - static func parseGoogle(_ root: [String: Any]) -> [AgentModelStreamEvent] { - guard let candidate = (root["candidates"] as? [[String: Any]])?.first, - let content = candidate["content"] as? [String: Any] - else { return [] } + /// A completed Responses search, as both a replay item and a card. + /// + /// Two events for one item, deliberately. The verbatim `web_search_call` + /// has to go back to OpenAI on the next turn or the reasoning that produced + /// it is orphaned, and that is what `providerItem` is for. The card wants + /// the query, which lives at `action.query` and would mean nothing to the + /// API. Neither can stand in for the other. + /// + /// Sources are absent here on purpose: Responses reports them as + /// `url_citation` annotations on the message rather than on this item. + private static func responsesWebSearch(_ item: [String: Any]) -> [AgentModelStreamEvent] { var events: [AgentModelStreamEvent] = [] + if let value = jsonValue(item) { events.append(.providerItem(value)) } + let action = item["action"] as? [String: Any] + let query = action?["query"] as? String ?? item["query"] as? String ?? "" + events.append(.webSearch(AgentWebSearchActivity(query: query))) + return events + } + + private static func anthropicQuery(_ rawInput: String) -> String { + guard let data = rawInput.data(using: .utf8), + let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any] + else { return "" } + return object["query"] as? String ?? "" + } + + /// Tolerates the error shape. A failed search replaces the array of hits + /// with a single `web_search_tool_result_error` object, so the cast is what + /// turns "the search failed" into an empty source list rather than a crash. + private static func anthropicSources(_ content: Any?) -> [AgentWebSearchActivity.Source] { + (content as? [[String: Any]] ?? []).compactMap { row in + guard let url = row["url"] as? String, !url.isEmpty else { return nil } + return AgentWebSearchActivity.Source( + title: row["title"] as? String ?? "", url: url + ) + } + } + + static func parseGoogle(_ root: [String: Any]) -> [AgentModelStreamEvent] { + guard let candidate = (root["candidates"] as? [[String: Any]])?.first else { return [] } + var events: [AgentModelStreamEvent] = [] + // Grounding hangs off the candidate, beside `content`, not inside + // `parts` — and it is read before the guard below, because a chunk can + // carry the metadata for a search without carrying any content. + if let grounding = candidate["groundingMetadata"] as? [String: Any] { + events.append(.webSearch(googleGrounding(grounding))) + } + guard let content = candidate["content"] as? [String: Any] else { + if candidate["finishReason"] != nil { events.append(.finished(.completed)) } + return events + } for part in content["parts"] as? [[String: Any]] ?? [] { if let text = part["text"] as? String { events.append(.textDelta(text)) } if let call = part["functionCall"] as? [String: Any], let name = call["name"] as? String { @@ -695,10 +828,57 @@ nonisolated struct AgentProviderClient: AgentModelStreaming, Sendable { if candidate["finishReason"] != nil { events.append(.finished(events.containsToolCall ? .toolCalls : .completed)) } return events } + + /// Gemini reports every query of the turn in one `webSearchQueries` array + /// and its sources in `groundingChunks`, with no mapping between them. The + /// queries are joined rather than split into a card each, because inventing + /// a source-to-query assignment the payload does not state would be worse + /// than showing them together. + private static func googleGrounding( + _ metadata: [String: Any] + ) -> AgentWebSearchActivity { + let queries = (metadata["webSearchQueries"] as? [String] ?? []) + .filter { !$0.isEmpty } + let sources = (metadata["groundingChunks"] as? [[String: Any]] ?? []).compactMap { + chunk -> AgentWebSearchActivity.Source? in + guard let web = chunk["web"] as? [String: Any], + let url = web["uri"] as? String, !url.isEmpty + else { return nil } + return AgentWebSearchActivity.Source( + title: web["title"] as? String ?? "", url: url + ) + } + return AgentWebSearchActivity( + query: queries.joined(separator: ", "), sources: sources + ) + } } extension AgentProviderClient: AgentModelCompleting {} +/// Per-stream Anthropic block bookkeeping. +/// +/// Anthropic streams two kinds of block through the same delta events: the +/// model's own `tool_use`, which Termind must execute, and the provider's +/// `server_tool_use`, which it must not. Knowing which index is which is the +/// only way to tell them apart mid-stream, and getting it wrong is not a +/// cosmetic error — see `parseAnthropic`. +nonisolated struct AnthropicStreamState: Sendable { + fileprivate struct ServerBlock: Sendable { + var id: String = "" + var name: String = "" + var input: String = "" + } + + fileprivate var callIDs: [Int: String] = [:] + fileprivate var serverBlocks: [Int: ServerBlock] = [:] + /// `server_tool_use.id` → query, held from that block's stop until the + /// sibling `web_search_tool_result` arrives, so one card carries both. + fileprivate var pendingQueries: [String: String] = [:] + + init() {} +} + nonisolated private extension Array where Element == AgentModelStreamEvent { var containsToolCall: Bool { contains { diff --git a/Termind/Core/LLM/ModelCapabilityResolver.swift b/Termind/Core/LLM/ModelCapabilityResolver.swift index 87139752..7734fb9c 100644 --- a/Termind/Core/LLM/ModelCapabilityResolver.swift +++ b/Termind/Core/LLM/ModelCapabilityResolver.swift @@ -170,6 +170,16 @@ nonisolated enum ModelCapabilityResolver { if record.reasoning { result.abilities.insert(.reasoning) } } } + // Outside every branch above, and that is the point. Server-side search + // is not treated as a headline capability by anyone: no `/models` + // payload reports it and Pi's catalog has no field for it, so "the + // source listed abilities" never means "the source ruled search out". + // Gating this on silence about *other* abilities would leave it off for + // exactly the well-described models most likely to have it. + // + // Additive only. It can turn the ability on from the model id; it never + // takes away one the payload actually asserted. + if Fallback.hasWebSearch(model.id) { result.abilities.insert(.webSearch) } if result.contextLength == nil { result.contextLength = record.contextWindow } if result.maxOutputTokens == nil, let maxOutputTokens = record.maxOutputTokens { result.maxOutputTokens = maxOutputTokens @@ -457,9 +467,18 @@ nonisolated enum ModelCapabilityResolver { if tools.matches(id) { result.abilities.insert(.toolCall) } if reasoning.matches(id) { result.abilities.insert(.reasoning) } } + // Outside the guard, for the reason given in `fill`. It sits after + // the embedding and image-generation returns above rather than + // before them, so neither can pick up a search ability on its way + // out. + if webSearch.matches(id) { result.abilities.insert(.webSearch) } return result } + static func hasWebSearch(_ modelID: String) -> Bool { + webSearch.matches(modelID.lowercased()) + } + private static let embedding = Pattern( #"(^|[-_/])embed(ding)?s?([-._/]|$)|text-embedding"# ) @@ -475,6 +494,28 @@ nonisolated enum ModelCapabilityResolver { + #"|grok-[3-9]|kimi-k[2-9]|qwen-?[3-9]|glm-[4-9]|minimax-m[2-9]|mistral|llama-[34]"# + #"|deepseek-(chat|v[3-9]|reasoner)|command-r|nova-(lite|pro)|step-3|internlm|intern-s1"# ) + /// Deliberately the narrowest of these patterns. + /// + /// The others answer "can this model probably do X" and an over-guess + /// costs a greyed-out tick box. This one decides whether a *server + /// tool* goes into the request body, and an over-guess costs a 400 on + /// every message. Only families with a shipped, current-shape search + /// tool are listed: Gemini 1.5's `googleSearchRetrieval` is a different + /// object and is excluded rather than approximated. + /// + /// DeepSeek v4 belongs here despite being reached as an OpenAI-shaped + /// endpoint: its `/responses` accepts the identical + /// `{"type": "web_search"}` and answers with the identical + /// `web_search_call` items, so it needs no code of its own — only to be + /// recognised. `ModelProvider.supportsNativeWebSearch` still requires + /// the Responses toggle, which is what keeps the older + /// `deepseek-chat`/`deepseek-reasoner` chat-completions path out. + private static let webSearch = Pattern( + #"gpt-4o|gpt-4\.1|gpt-[5-9]|(^|[-_/])o[3-9]([-._/]|$)"# + + #"|claude-(opus|sonnet|haiku)-[4-9]|claude-3[-.][57]"# + + #"|gemini-(2\.[05]|[3-9])"# + + #"|deepseek-v[4-9]"# + ) private static let reasoning = Pattern( #"gpt-[5-9]|gpt-oss|(^|[-_/])o[1-9]([-._/]|$)|gemini-(2\.5|[3-9])"# + #"|claude-(opus|sonnet|haiku)-[4-9]|claude-3[-.]7"# diff --git a/Termind/Core/Models/ModelProvider.swift b/Termind/Core/Models/ModelProvider.swift index 9c108351..c3aaf906 100644 --- a/Termind/Core/Models/ModelProvider.swift +++ b/Termind/Core/Models/ModelProvider.swift @@ -191,6 +191,46 @@ nonisolated struct ModelProvider: Identifiable, Hashable, Sendable { return !(usesVertex && kind == .google) } + /// Whether this endpoint accepts a *server-side* search tool in its request + /// body — the kind the provider runs itself and never tells the agent + /// runtime about. + /// + /// Chat Completions is excluded on purpose. OpenAI's `web_search_options` + /// exists there only on the two `*-search-preview` SKUs, while `.openAI` is + /// also every compatible gateway this app talks to — OpenRouter, DeepSeek, + /// Groq, Together, Ollama. Sending them an option they have never heard of + /// turns a working setup into a 400 for a feature nobody asked for. + /// + /// The wire protocol alone is not enough, which is the part that had to be + /// learned. A server tool exists at the intersection of *this vendor's API* + /// and *this vendor's model*, and a gateway breaks that pairing: point an + /// OpenAI-shaped `/responses` endpoint at Claude and it will accept + /// `{"type": "web_search"}` and forward it as an ordinary function + /// declaration. The model then calls `web_search` as a tool, nothing here + /// has one registered — because native mode deliberately withholds the + /// local tool — and the run reports `Unknown tool: web_search`. + /// + /// So the model family has to agree with the endpoint. A Claude id on an + /// OpenAI-shaped endpoint means a translator sits in between, and the right + /// answer there is the local `web_search` tool, which works everywhere. + func supportsNativeWebSearch(model: AIModel) -> Bool { + guard model.abilities.contains(.webSearch) else { return false } + let id = model.id.lowercased() + switch kind { + case .anthropic: + return id.contains("claude") + case .google: + return id.contains("gemini") + case .openAI: + // DeepSeek belongs here: its `/responses` implements the identical + // `{"type": "web_search"}` server tool, so it is a first party for + // this purpose even though it is reached OpenAI-shaped. + guard usesResponsesAPI else { return false } + return id.contains("gpt") || id.contains("deepseek") + || id.range(of: #"(^|[-_/])o[3-9]([-._/]|$)"#, options: .regularExpression) != nil + } + } + /// What the role pickers and the conversation's model menu offer. An /// embedding model in a chat-model picker is noise. var chatModels: [AIModel] { @@ -320,6 +360,11 @@ nonisolated struct AIModel: Identifiable, Hashable, Sendable { nonisolated enum Ability: String, Codable, Sendable, CaseIterable, Identifiable { case toolCall, reasoning + /// The provider runs the search on its own servers when asked, rather + /// than the model calling one of Termind's tools. Whether that offer is + /// actually taken up also depends on the provider kind — see + /// `ModelProvider.supportsNativeWebSearch`. + case webSearch nonisolated var id: String { rawValue } @@ -327,6 +372,7 @@ nonisolated struct AIModel: Identifiable, Hashable, Sendable { switch self { case .toolCall: String(localized: "Tools") case .reasoning: String(localized: "Reasoning") + case .webSearch: String(localized: "Web Search") } } @@ -334,6 +380,7 @@ nonisolated struct AIModel: Identifiable, Hashable, Sendable { switch self { case .toolCall: "wrench.and.screwdriver" case .reasoning: "brain" + case .webSearch: "magnifyingglass" } } } diff --git a/Termind/Domain/Agent/AgentTypes.swift b/Termind/Domain/Agent/AgentTypes.swift index 196a13df..5c5ad803 100644 --- a/Termind/Domain/Agent/AgentTypes.swift +++ b/Termind/Domain/Agent/AgentTypes.swift @@ -120,12 +120,55 @@ nonisolated struct AgentToolPreflight: Hashable, Sendable { var executionMetadata: [String: AgentJSONValue] = [:] } +/// One search the *provider* ran on its own servers, normalized out of three +/// different vendor shapes. +/// +/// This exists because a native search is invisible to the tool pipeline: no +/// tool call reaches `AgentRuntime`, so nothing would be approved, recorded, or +/// shown, and the assistant would cite pages the transcript never mentions +/// fetching. Carrying the query back lets the runtime raise an ordinary tool +/// card for it after the fact. +/// +/// `results` is empty when the provider reports the search without its hits — +/// OpenAI's `web_search_call` carries the query and puts the sources in message +/// annotations instead. The query alone is still the part worth showing. +nonisolated struct AgentWebSearchActivity: Hashable, Sendable, Codable { + nonisolated struct Source: Hashable, Sendable, Codable { + var title: String + var url: String + } + + var query: String + var sources: [Source] = [] +} + nonisolated struct AgentToolResult: Hashable, Sendable, Codable { var callID: String var content: String var isError: Bool = false var isTruncated: Bool = false var metadata: [String: AgentJSONValue] = [:] + + /// Marks a result the provider produced inside its own turn. + static let providerNativeKey = "provider_native" + + /// Whether this is a *record* of something that already happened rather + /// than a result the model is still owed. + /// + /// Load-bearing, not cosmetic. A provider-native web search never was a + /// tool call, so a transcript that carries it as one replays a + /// `function_call_output` / `tool_result` whose call id matches nothing — + /// and every provider rejects that. OpenAI answers + /// `No tool call found for function call output with call_id ...`, which + /// only surfaces on the *second* request of a conversation, because the + /// first is the one that puts the orphan in the history. + /// + /// Read from `metadata` rather than stored as its own property so the flag + /// survives persistence through the existing `metadataData` column instead + /// of costing a schema version. + var isProviderNative: Bool { + metadata[Self.providerNativeKey]?.boolValue == true + } } nonisolated enum AgentRunState: String, Hashable, Sendable, Codable { diff --git a/Termind/Features/AI/ConversationModel.swift b/Termind/Features/AI/ConversationModel.swift index aebb555e..a20f2afe 100644 --- a/Termind/Features/AI/ConversationModel.swift +++ b/Termind/Features/AI/ConversationModel.swift @@ -478,6 +478,12 @@ final class ConversationModel: Identifiable { toolCards[index].result = result toolCards[index].state = result.isError ? .failed : .completed } + // A card, but not a turn. A provider-native search happened inside + // the assistant's own turn and was never a tool call here, so + // adding it to the model-facing transcript would replay a tool + // result the provider has no matching call for — which it rejects + // on the *next* request. See `AgentToolResult.isProviderNative`. + guard !result.isProviderNative else { break } transcript.append(AgentTranscriptMessage( role: .tool, text: result.content, toolCallID: result.callID, toolName: invocation.call.name, isError: result.isError diff --git a/Termind/Features/Settings/SettingsPageView.swift b/Termind/Features/Settings/SettingsPageView.swift index f6d72a2d..0118bf7d 100644 --- a/Termind/Features/Settings/SettingsPageView.swift +++ b/Termind/Features/Settings/SettingsPageView.swift @@ -4,7 +4,7 @@ import SwiftUI enum SettingsSection: String, CaseIterable, Identifiable { case app, startup, sidebar case terminalAppearance, sftp - case providers, modelRoles, mcp + case providers, modelRoles, webSearch, mcp case iCloud var id: String { rawValue } @@ -18,6 +18,7 @@ enum SettingsSection: String, CaseIterable, Identifiable { case .sftp: "SFTP" case .providers: "Providers" case .modelRoles: "Model Roles" + case .webSearch: "Web Search" case .mcp: "MCP" case .iCloud: "Sync" } @@ -32,6 +33,7 @@ enum SettingsSection: String, CaseIterable, Identifiable { case .sftp: "arrow.up.arrow.down" case .providers: "network" case .modelRoles: "person.2.badge.gearshape" + case .webSearch: "magnifyingglass" case .mcp: "wrench.and.screwdriver" case .iCloud: "icloud" } @@ -146,6 +148,13 @@ struct SettingsPageView: View { SettingsFormPage(title: "Model Roles", subtitle: "Choose which model handles each assistant task.") { ModelRoleSettings() } + case .webSearch: + SettingsFormPage( + title: "Web Search", + subtitle: "How the assistant finds pages before it reads them." + ) { + WebSearchSettings() + } case .mcp: SettingsCollectionPage(title: "MCP", subtitle: "Remote tool servers available to the assistant.") { MCPSettings { id, isNew in route = .mcpServer(id, isNew: isNew) } @@ -200,7 +209,7 @@ private struct SettingsSidebar: View { LazyVStack(alignment: .leading, spacing: Metrics.Space.xl) { group("GENERAL", sections: [.app, .startup, .sidebar]) group("TERMINAL", sections: [.terminalAppearance, .sftp]) - group("AI", sections: [.providers, .modelRoles, .mcp]) + group("AI", sections: [.providers, .modelRoles, .webSearch, .mcp]) group("iCloud", sections: [.iCloud]) } .padding(.horizontal, Metrics.Space.m) diff --git a/Termind/Features/Settings/WebSearchSettings.swift b/Termind/Features/Settings/WebSearchSettings.swift new file mode 100644 index 00000000..1c56598f --- /dev/null +++ b/Termind/Features/Settings/WebSearchSettings.swift @@ -0,0 +1,151 @@ +import SwiftUI + +/// The "Web Search" section: how the assistant finds a URL, as the counterpart +/// to `fetch` reading one. +/// +/// A form rather than a collection, because there is one backend rather than a +/// list of them. Two backends configured at once would raise a question — which +/// one runs? — that nothing here would be able to answer better than the user +/// picking. +struct WebSearchSettings: View { + @Environment(AppModel.self) private var app + + @AppStorage(SettingsKey.webSearchEnabled) private var isEnabled = + SettingsKey.defaultWebSearchEnabled + @AppStorage(SettingsKey.webSearchBackend) private var backendID = SearchBackend.default.rawValue + @AppStorage(SettingsKey.webSearchBaseURL) private var baseURL = "" + @AppStorage(SettingsKey.webSearchCredentialRef) private var credentialRef = "" + + @State private var apiKey = "" + @State private var probe: ProbeState = .idle + + private enum ProbeState: Equatable { + case idle + case running + case succeeded(Int) + case failed(String) + } + + private var backend: SearchBackend { + SearchBackend(rawValue: backendID) ?? .default + } + + private var client: WebSearchClient { + WebSearchClient(backend: backend, baseURL: baseURL, apiKey: apiKey) + } + + var body: some View { + Group { + Section { + Toggle("Enable Web Search", isOn: $isEnabled) + + Text( + "Models that can search on their own do so directly, and no backend below is " + + "used for them. Everything else gets a web_search tool backed by the " + + "backend below. Either way the assistant reads a result page with fetch." + ) + .font(.caption) + .foregroundStyle(.secondary) + .fixedSize(horizontal: false, vertical: true) + } header: { + Text("Web Search") + } + + Section { + Picker("Backend", selection: $backendID) { + ForEach(SearchBackend.allCases) { option in + Text(option.displayName).tag(option.rawValue) + } + } + + TextField("Base URL", text: $baseURL, prompt: Text(backend.defaultBaseURL)) + + LabeledContent("Full URL") { + Text(client.resolvedURL) + .font(.caption.monospaced()) + .foregroundStyle(.secondary) + .textSelection(.enabled) + .multilineTextAlignment(.trailing) + .fixedSize(horizontal: false, vertical: true) + } + + if backend.requiresAPIKey { + SecureField("API Key", text: $apiKey, prompt: Text("Required")) + .onChange(of: apiKey) { _, value in store(key: value) } + } + + Text(backend.detail) + .font(.caption) + .foregroundStyle(.secondary) + .fixedSize(horizontal: false, vertical: true) + + if backend.requiresAPIKey { + Text("The key is stored in this Mac's Data Protection Keychain.") + .font(.caption) + .foregroundStyle(.secondary) + .fixedSize(horizontal: false, vertical: true) + } + } header: { + Text("Backend") + } + .disabled(!isEnabled) + + Section { + // A real query against the real endpoint. The offline checks + // cannot tell a working key from a wrong one, and for the + // keyless backend they cannot tell working markup from markup + // that moved — which is exactly what this button is for. + Button("Test Search") { runProbe() } + .disabled(probe == .running || !isEnabled || !client.isConfigured) + + switch probe { + case .idle: + EmptyView() + case .running: + Label("Searching…", systemImage: "ellipsis.circle") + .font(.caption).foregroundStyle(.secondary) + case .succeeded(let count): + Label( + count == 0 + ? String(localized: "Reached the backend, but it returned no results.") + : String(localized: "\(count) results."), + systemImage: count == 0 ? "exclamationmark.triangle" : "checkmark.circle" + ) + .font(.caption) + .foregroundStyle(count == 0 ? .orange : .green) + case .failed(let reason): + Label(reason, systemImage: "xmark.octagon") + .font(.caption) + .foregroundStyle(.red) + .fixedSize(horizontal: false, vertical: true) + } + } + .disabled(!isEnabled) + } + .onAppear { apiKey = app.ai.secret(for: credentialRef) ?? "" } + .onChange(of: backendID) { _, _ in probe = .idle } + } + + private func store(key: String) { + guard !key.isEmpty else { + app.ai.setSecret(nil, for: credentialRef) + credentialRef = "" + return + } + if credentialRef.isEmpty { credentialRef = AIStore.makeCredentialRef("search") } + app.ai.setSecret(key, for: credentialRef) + } + + private func runProbe() { + probe = .running + let client = client + Task { + do { + let results = try await client.search("termind web search test", count: 3) + probe = .succeeded(results.count) + } catch { + probe = .failed(error.localizedDescription) + } + } + } +} diff --git a/TermindTests/AIProviderPersistenceTests.swift b/TermindTests/AIProviderPersistenceTests.swift index 5042db3e..4252726e 100644 --- a/TermindTests/AIProviderPersistenceTests.swift +++ b/TermindTests/AIProviderPersistenceTests.swift @@ -204,12 +204,104 @@ struct ModelCapabilityResolverTests { #expect(ModelCapabilityResolver.catalogMetadata?.modelCount == 1_312) #expect(resolution.catalogSource == .init(provider: "openai", modelID: "gpt-5.4")) #expect(resolution.model.input == [.text, .image]) - #expect(resolution.model.abilities == [.toolCall, .reasoning]) + // `webSearch` is the one ability that does come from the model id, and + // it has to: Pi's catalog has no field for server-side search, so this + // record says nothing about it either way. The rule the rest of this + // test guards is unchanged — a record that *does* list abilities is + // taken verbatim, and only a record silent on them is filled in. + #expect(resolution.model.abilities == [.toolCall, .reasoning, .webSearch]) #expect(resolution.model.contextLength == 272_000) #expect(resolution.model.maxOutputTokens == 128_000) #expect(resolution.supported == [.off, .low, .medium, .high, .xHigh]) } + /// Nobody publishes server-side search as a capability: no `/models` + /// payload carries it and Pi's catalog has no field for it. So "the + /// endpoint reported its abilities" must not be read as "the endpoint said + /// there is no search" — otherwise the ability stays off for exactly the + /// well-described models most likely to have it. + @Test + func webSearchIsInferredEvenWhenTheEndpointReportedItsAbilities() { + let provider = ModelProvider(name: "OpenAI") + var reportedModel = AIModel(id: "gpt-5.4") + reportedModel.abilities = [.toolCall] + + let resolution = ModelCapabilityResolver.resolve( + model: reportedModel, provider: provider, reported: [.abilities] + ) + + // The reported list is still honoured — reasoning is absent because the + // endpoint did not claim it — and search is added on top. + #expect(resolution.model.abilities == [.toolCall, .webSearch]) + } + + /// DeepSeek v4 reaches `/responses` with the identical + /// `{"type": "web_search"}` OpenAI uses, so it needs recognising rather than + /// any code of its own. + @Test + func deepSeekV4IsRecognisedAsSearchCapable() { + var provider = ModelProvider(name: "DeepSeek") + provider.baseURL = "https://api.deepseek.com/v1" + provider.usesResponsesAPI = true + + let resolution = ModelCapabilityResolver.resolve( + model: AIModel(id: "deepseek-v4-pro"), provider: provider + ) + #expect(resolution.model.abilities.contains(.webSearch)) + #expect(provider.supportsNativeWebSearch(model: resolution.model)) + + // The older chat-completions models are excluded by the provider gate + // even where the id might tempt the pattern. + var legacy = ModelProvider(name: "DeepSeek Chat") + legacy.baseURL = "https://api.deepseek.com/v1" + #expect(!legacy.supportsNativeWebSearch(model: resolution.model)) + } + + /// The failure this pins reads `Unknown tool: web_search` at runtime. + /// + /// An OpenAI-shaped `/responses` gateway pointed at Claude accepts + /// `{"type": "web_search"}` and forwards it as an ordinary function + /// declaration. The model then calls `web_search` as a tool — and native + /// mode deliberately withholds the local one, so nothing can run it. + /// A server tool only exists where the vendor's API and the vendor's model + /// meet. + @Test + func nativeSearchNeedsTheModelFamilyToMatchTheEndpoint() { + var claude = AIModel(id: "anthropic/claude-sonnet-4") + claude.abilities = [.toolCall, .webSearch] + + var gateway = ModelProvider(name: "OpenRouter") + gateway.baseURL = "https://openrouter.ai/api/v1" + gateway.usesResponsesAPI = true + #expect(!gateway.supportsNativeWebSearch(model: claude)) + + // The same model on Anthropic's own API does have it. + var anthropic = ModelProvider(name: "Anthropic") + anthropic.kind = .anthropic + #expect(anthropic.supportsNativeWebSearch(model: claude)) + + // And a Gemini id on an Anthropic-shaped endpoint is the same mistake + // in the other direction. + var gemini = AIModel(id: "gemini-2.5-pro") + gemini.abilities = [.toolCall, .webSearch] + #expect(!anthropic.supportsNativeWebSearch(model: gemini)) + + var google = ModelProvider(name: "Google") + google.kind = .google + #expect(google.supportsNativeWebSearch(model: gemini)) + } + + @Test + func searchIsNotInferredForModelsThatCannotChat() { + let provider = ModelProvider(name: "OpenAI") + for id in ["text-embedding-3-large", "dall-e-3"] { + let resolution = ModelCapabilityResolver.resolve( + model: AIModel(id: id), provider: provider + ) + #expect(!resolution.model.abilities.contains(.webSearch), "\(id)") + } + } + @Test func bundledOverrideEnrichesCodexAutoReview() { let provider = ModelProvider(name: "OpenAI") @@ -279,7 +371,12 @@ struct ModelCapabilityResolverTests { #expect(resolution.model.kind == .other) #expect(resolution.model.input == [.audio]) #expect(resolution.model.output == [.image]) - #expect(resolution.model.abilities.isEmpty) + // Reported abilities still win — `toolCall` and `reasoning` stay off + // despite the catalog claiming both for gpt-5.4. `webSearch` is the + // exception, deliberately: an empty array is the endpoint listing what + // it knows about, and no endpoint knows about server-side search. See + // `webSearchIsInferredEvenWhenTheEndpointReportedItsAbilities`. + #expect(resolution.model.abilities == [.webSearch]) #expect(resolution.model.contextLength == 7) #expect(resolution.model.maxOutputTokens == 8) #expect(resolution.supported == [.off]) diff --git a/TermindTests/AgentProviderAdapterTests.swift b/TermindTests/AgentProviderAdapterTests.swift index 86a3aac5..94830e5d 100644 --- a/TermindTests/AgentProviderAdapterTests.swift +++ b/TermindTests/AgentProviderAdapterTests.swift @@ -336,18 +336,18 @@ struct AgentProviderAdapterTests { @Test func anthropicRetainsNativeToolUseIDAcrossPartialJSON() { - var ids: [Int: String] = [:] + var state = AnthropicStreamState() let start = AgentProviderClient.parseAnthropic( "content_block_start", ["index": 1, "content_block": [ "type": "tool_use", "id": "toolu_123", "name": "sftp_read", ]], - callIDs: &ids + state: &state ) let delta = AgentProviderClient.parseAnthropic( "content_block_delta", ["index": 1, "delta": ["partial_json": "{\"path\":\"/tmp\"}"]], - callIDs: &ids + state: &state ) #expect(start.firstTool?.id == "toolu_123") #expect(delta.firstTool?.id == "toolu_123") @@ -366,6 +366,298 @@ struct AgentProviderAdapterTests { #expect(events.firstTool?.name == "list_hosts") #expect(events.containsFinish == .toolCalls) } + + // MARK: - Provider-native web search + + @Test + func nativeSearchToolIsSentOnlyWhenAsked() throws { + var anthropic = ModelProvider(name: "Anthropic") + anthropic.kind = .anthropic + + let off = Self.body(anthropic, webSearch: false, tools: [Self.lookupTool]) + let offTools = try #require(off["tools"] as? [[String: Any]]) + #expect(offTools.count == 1) + #expect(!offTools.contains { $0["type"] as? String == "web_search_20250305" }) + + let on = Self.body(anthropic, webSearch: true, tools: [Self.lookupTool]) + let onTools = try #require(on["tools"] as? [[String: Any]]) + #expect(onTools.count == 2) + let server = try #require(onTools.first { $0["type"] as? String == "web_search_20250305" }) + #expect(server["name"] as? String == "web_search") + #expect(server["max_uses"] as? Int == AgentProviderClient.nativeWebSearchMaxUses) + } + + /// The guard that used to be skipped: `tools` was only set when the local + /// list was non-empty, so a chat with no tools got no search either. + @Test + func nativeSearchSurvivesAnEmptyLocalToolList() throws { + var anthropic = ModelProvider(name: "Anthropic") + anthropic.kind = .anthropic + let anthropicTools = try #require( + Self.body(anthropic, webSearch: true, tools: [])["tools"] as? [[String: Any]] + ) + #expect(anthropicTools.count == 1) + #expect(anthropicTools[0]["type"] as? String == "web_search_20250305") + + var responses = ModelProvider(name: "Responses") + responses.usesResponsesAPI = true + let responseTools = try #require( + Self.body(responses, webSearch: true, tools: [])["tools"] as? [[String: Any]] + ) + #expect(responseTools.count == 1) + #expect(responseTools[0]["type"] as? String == "web_search") + } + + /// Chat Completions is also every OpenAI-compatible gateway Termind talks + /// to, and an unknown tool type there is a 400 on every message. + @Test + func chatCompletionsNeverAsksForNativeSearch() throws { + let provider = ModelProvider(name: "Gateway") + var capable = AIModel(id: "gpt-5.4") + capable.abilities = [.toolCall, .webSearch] + #expect(!provider.supportsNativeWebSearch(model: capable)) + + let body = Self.body(provider, webSearch: true, tools: [Self.lookupTool]) + let tools = try #require(body["tools"] as? [[String: Any]]) + #expect(tools.count == 1) + #expect(tools[0]["type"] as? String == "function") + #expect(body["web_search_options"] == nil) + } + + @Test + func googleSendsSearchAsItsOwnToolGroup() throws { + var provider = ModelProvider(name: "Google") + provider.kind = .google + + let tools = try #require( + Self.body(provider, webSearch: true, tools: [Self.lookupTool])["tools"] + as? [[String: Any]] + ) + // Two entries, not two keys in one entry — the arrangement Gemini + // rejects is `functionDeclarations` and `google_search` as siblings + // inside a single object. + #expect(tools.count == 2) + #expect(tools.contains { $0["functionDeclarations"] != nil }) + #expect(tools.contains { $0["google_search"] != nil }) + #expect(tools.allSatisfy { $0.count == 1 }) + } + + /// The regression that costs a whole turn. + /// + /// `server_tool_use` streams `input_json_delta` exactly like `tool_use` + /// does. Read as a tool call it produces one with an empty name, which + /// `AgentRuntime` rejects with `invalidToolCall` — so a single native + /// search would fail the entire assistant turn. + @Test + func anthropicServerToolBlocksNeverBecomeToolCalls() { + var state = AnthropicStreamState() + var events: [AgentModelStreamEvent] = [] + + events += AgentProviderClient.parseAnthropic( + "content_block_start", + [ + "index": 0, + "content_block": [ + "type": "server_tool_use", "id": "srvtoolu_1", "name": "web_search", + ], + ], + state: &state + ) + for chunk in [#"{"query""#, #": "swift 6""#, "}"] { + events += AgentProviderClient.parseAnthropic( + "content_block_delta", + ["index": 0, "delta": ["type": "input_json_delta", "partial_json": chunk]], + state: &state + ) + } + events += AgentProviderClient.parseAnthropic( + "content_block_stop", ["index": 0], state: &state + ) + + #expect(events.firstTool == nil) + #expect(events.isEmpty) + } + + @Test + func anthropicPairsItsSearchQueryWithItsResults() throws { + var state = AnthropicStreamState() + var events: [AgentModelStreamEvent] = [] + + events += AgentProviderClient.parseAnthropic( + "content_block_start", + [ + "index": 0, + "content_block": [ + "type": "server_tool_use", "id": "srvtoolu_1", "name": "web_search", + ], + ], + state: &state + ) + events += AgentProviderClient.parseAnthropic( + "content_block_delta", + [ + "index": 0, + "delta": ["type": "input_json_delta", "partial_json": #"{"query": "swift 6"}"#], + ], + state: &state + ) + events += AgentProviderClient.parseAnthropic( + "content_block_stop", ["index": 0], state: &state + ) + events += AgentProviderClient.parseAnthropic( + "content_block_start", + [ + "index": 1, + "content_block": [ + "type": "web_search_tool_result", "tool_use_id": "srvtoolu_1", + "content": [ + ["type": "web_search_result", "url": "https://swift.org", "title": "Swift"] + ], + ], + ], + state: &state + ) + // The model's own text still streams normally afterwards. + events += AgentProviderClient.parseAnthropic( + "content_block_delta", + ["index": 2, "delta": ["type": "text_delta", "text": "Swift 6 ships"]], + state: &state + ) + + let activity = try #require(events.firstWebSearch) + #expect(activity.query == "swift 6") + #expect(activity.sources.map(\.url) == ["https://swift.org"]) + #expect(events.firstText == "Swift 6 ships") + #expect(events.firstTool == nil) + } + + /// A failed search replaces the array of hits with a single error object. + @Test + func anthropicSearchErrorYieldsNoSources() throws { + var state = AnthropicStreamState() + let events = AgentProviderClient.parseAnthropic( + "content_block_start", + [ + "index": 0, + "content_block": [ + "type": "web_search_tool_result", "tool_use_id": "srvtoolu_1", + "content": ["type": "web_search_tool_result_error", "error_code": "max_uses_exceeded"], + ], + ], + state: &state + ) + let activity = try #require(events.firstWebSearch) + #expect(activity.sources.isEmpty) + } + + /// Two events for one item: the verbatim call has to go back to OpenAI next + /// turn, and the card wants the query, which the API would not understand. + @Test + func responsesSearchCallIsBothReplayedAndShown() throws { + let events = AgentProviderClient.parseResponses( + "response.output_item.done", + [ + "item": [ + "type": "web_search_call", "id": "ws_1", "status": "completed", + "action": ["type": "search", "query": "swift 6 release date"], + ] + ] + ) + + let activity = try #require(events.firstWebSearch) + #expect(activity.query == "swift 6 release date") + // Sources are absent on purpose: Responses reports them as + // `url_citation` annotations on the message, not on this item. + #expect(activity.sources.isEmpty) + + let item = try #require(events.firstProviderItem) + #expect(item.objectValue?["type"]?.stringValue == "web_search_call") + #expect(item.objectValue?["id"]?.stringValue == "ws_1") + #expect(events.firstTool == nil) + } + + @Test + func responsesSearchCallReplaysInTheNextRequest() throws { + let item = try #require( + AgentProviderClient.parseResponses( + "response.output_item.done", + [ + "item": [ + "type": "web_search_call", "id": "ws_1", "status": "completed", + "action": ["type": "search", "query": "swift 6"], + ] + ] + ).firstProviderItem + ) + var provider = ModelProvider(name: "Responses") + provider.usesResponsesAPI = true + let body = AgentProviderClient( + provider: provider, model: AIModel(id: "gpt-test"), secret: "test" + ).body(AgentModelRequest( + systemPrompt: "test", + messages: [AgentTranscriptMessage(role: .assistant, providerItems: [item])], + tools: [] + )) + + let input = try #require(body["input"] as? [[String: Any]]) + #expect(input.contains { $0["type"] as? String == "web_search_call" }) + } + + @Test + func googleGroundingMetadataBecomesOneSearchCard() throws { + let events = AgentProviderClient.parseGoogle([ + "candidates": [[ + "content": ["parts": [["text": "Swift 6 shipped."]]], + "groundingMetadata": [ + "webSearchQueries": ["swift 6 release"], + "groundingChunks": [ + ["web": ["uri": "https://swift.org/blog", "title": "swift.org"]], + ["web": ["title": "no uri"]], + ], + ], + "finishReason": "STOP", + ]] + ]) + + let activity = try #require(events.firstWebSearch) + #expect(activity.query == "swift 6 release") + #expect(activity.sources.map(\.url) == ["https://swift.org/blog"]) + #expect(events.firstText == "Swift 6 shipped.") + #expect(events.containsFinish == .completed) + } + + /// Gemini can report grounding on a chunk that carries no `content` at all. + @Test + func googleGroundingIsReadFromAContentlessChunk() throws { + let events = AgentProviderClient.parseGoogle([ + "candidates": [[ + "groundingMetadata": ["webSearchQueries": ["swift 6"]], + ]] + ]) + #expect(events.firstWebSearch?.query == "swift 6") + } + + // MARK: - Helpers + + private static let lookupTool = AgentToolDescriptor( + name: "lookup", summary: "Look something up.", + inputSchema: .object([ + "type": .string("object"), + "properties": .object(["query": .object(["type": .string("string")])]), + "required": .array([.string("query")]), + "additionalProperties": .bool(false), + ]), + target: .local, safety: .locallyReadOnly + ) + + private static func body( + _ provider: ModelProvider, webSearch: Bool, tools: [AgentToolDescriptor] + ) -> [String: Any] { + AgentProviderClient( + provider: provider, model: AIModel(id: "model-test"), secret: "test", + webSearch: webSearch + ).body(AgentModelRequest(systemPrompt: "test", messages: [], tools: tools)) + } } private extension [AgentModelStreamEvent] { @@ -401,4 +693,9 @@ private extension [AgentModelStreamEvent] { for case .providerItem(let item) in self { return item } return nil } + + var firstWebSearch: AgentWebSearchActivity? { + for case .webSearch(let activity) in self { return activity } + return nil + } } diff --git a/TermindTests/WebSearchTests.swift b/TermindTests/WebSearchTests.swift new file mode 100644 index 00000000..0cdbb76a --- /dev/null +++ b/TermindTests/WebSearchTests.swift @@ -0,0 +1,478 @@ +import Foundation +import Testing + +@testable import Termind + +@Suite +struct WebSearchTests { + + // MARK: - Configuration + + @Test + func keylessBackendIsConfiguredWithoutAKey() { + let bing = WebSearchClient(backend: .bing) + #expect(bing.isConfigured) + #expect(!SearchBackend.bing.requiresAPIKey) + + let brave = WebSearchClient(backend: .brave) + #expect(!brave.isConfigured) + #expect(WebSearchClient(backend: .brave, apiKey: "token").isConfigured) + } + + @Test + func blankBaseURLResolvesToTheBackendDefault() { + #expect(WebSearchClient(backend: .bing).resolvedURL == "https://www.bing.com/search") + #expect( + WebSearchClient(backend: .brave, apiKey: "k").resolvedURL + == "https://api.search.brave.com/res/v1/web/search" + ) + // A pasted URL very often carries a trailing slash, and every path + // starts with one. + #expect( + WebSearchClient(backend: .tavily, baseURL: "https://proxy.example/", apiKey: "k") + .resolvedURL == "https://proxy.example/search" + ) + } + + // MARK: - Requests + + @Test + func bingRequestLooksLikeSafariOnAMac() throws { + let request = try WebSearchClient(backend: .bing, market: "en-US") + .buildRequest(query: "swift concurrency", count: 5) + + let url = try #require(request.url?.absoluteString) + #expect(url.hasPrefix("https://www.bing.com/search?")) + #expect(url.contains("q=swift%20concurrency") || url.contains("q=swift+concurrency")) + #expect(request.httpMethod == "GET") + + let agent = try #require(request.value(forHTTPHeaderField: "User-Agent")) + #expect(agent.contains("Macintosh")) + #expect(agent.contains("Safari")) + #expect(!agent.contains("Windows")) + #expect(request.value(forHTTPHeaderField: "Accept-Language") != nil) + } + + /// Verified against the live endpoint: with no `mkt`, Bing answers an + /// anonymous request with a well-formed page of results for an unrelated + /// query. The parse succeeds and the model is handed confident nonsense, + /// so this parameter is load-bearing rather than a locale nicety. + @Test + func bingAlwaysCarriesAMarket() throws { + let url = try #require( + WebSearchClient(backend: .bing, market: "ja-JP") + .buildRequest(query: "swift", count: 5).url?.absoluteString + ) + #expect(url.contains("mkt=ja-JP")) + + // Derived per component, so a Mac with a language but no region still + // searches in its own language rather than falling all the way to en-US. + let market = WebSearchClient.preferredMarket + #expect(market.contains("-")) + #expect(market.split(separator: "-").count == 2) + } + + @Test + func keyedBackendsCarryTheirOwnAuthenticationHeader() throws { + let brave = try WebSearchClient(backend: .brave, apiKey: "brave-token") + .buildRequest(query: "q", count: 3) + #expect(brave.value(forHTTPHeaderField: "X-Subscription-Token") == "brave-token") + + let tavily = try WebSearchClient(backend: .tavily, apiKey: "tavily-token") + .buildRequest(query: "q", count: 3) + #expect(tavily.value(forHTTPHeaderField: "Authorization") == "Bearer tavily-token") + #expect(tavily.httpMethod == "POST") + + let exa = try WebSearchClient(backend: .exa, apiKey: "exa-token") + .buildRequest(query: "q", count: 3) + #expect(exa.value(forHTTPHeaderField: "x-api-key") == "exa-token") + } + + @Test + func exaAsksForCappedSnippetsRatherThanWholePages() throws { + let request = try WebSearchClient(backend: .exa, apiKey: "k") + .buildRequest(query: "q", count: 3) + let payload = try #require(request.httpBody) + let body = try #require( + JSONSerialization.jsonObject(with: payload) as? [String: Any] + ) + let contents = try #require(body["contents"] as? [String: Any]) + let text = try #require(contents["text"] as? [String: Any]) + // Reading a page in full stays `fetch`'s job, because that is the path + // with address classification on the target. + #expect(text["maxCharacters"] as? Int == 600) + } + + // MARK: - Bing result page + + /// Trimmed from a real results page: the `b_algo` list item, the `h2 > a` + /// title link, and the `b_algoSlug` snippet Kelivo's selectors read. + private static let bingPage = """ +
    +
  1. +

    Swift Documentation

    +
    +

    The Swift language guide & reference.

    +
    +
    2. +
  2. +

    Example Page

    +

    A second result.

    +
    3. +
  3. +

    Third

    +
    4. +
+ """ + + @Test + func bingPageYieldsTitleURLAndSnippet() { + let results = WebSearchClient.bingResults(Self.bingPage, limit: 10) + + #expect(results.count == 3) + #expect(results[0].title == "Swift Documentation") + #expect(results[0].url == "https://swift.org/documentation/") + // Tags stripped and entities decoded: the model reads this as prose, + // and markup here costs tokens and invites confusion with instructions. + #expect(results[0].snippet == "The Swift language guide & reference.") + // A result with no snippet is still a result. + #expect(results[2].title == "Third") + #expect(results[2].snippet.isEmpty) + } + + @Test + func bingRedirectWrapperIsUnwrappedToTheRealURL() { + let results = WebSearchClient.bingResults(Self.bingPage, limit: 10) + // Left wrapped, the model would be handed a bing.com/ck/a URL that says + // nothing about the source it is about to trust. + #expect(results[1].url == "https://example.com/page") + #expect(results[1].title == "Example Page") + + // Anything that is not the `u=a1` form is left alone. + let plain = "https://example.com/ck/a?u=not-base64" + #expect(WebSearchClient.unwrapBingRedirect(plain) == plain) + } + + @Test + func bingResultsRespectTheRequestedCount() { + #expect(WebSearchClient.bingResults(Self.bingPage, limit: 2).count == 2) + } + + /// Bing writes its date separator as `· `, so a fixed list of + /// named entities leaves raw numeric references in the text the model reads. + @Test + func numericEntitiesAreDecoded() { + #expect( + WebSearchClient.plainText("Jul 10, 2026· At Swift, we're open") + == "Jul 10, 2026·\u{20}At Swift, we're open" + ) + #expect(WebSearchClient.plainText("café & bar") == "café & bar") + // A stray `&#` is left alone rather than swallowing the rest of the + // text up to whatever semicolon comes next. + #expect(WebSearchClient.plainText("A &# B; C") == "A &# B; C") + } + + @Test + func markupThatMovedYieldsNoResultsRatherThanFailing() { + // The honest failure mode of scraping, and the reason a keyed backend + // exists: an empty list reads to the model as "nothing found". + #expect(WebSearchClient.bingResults("no results here", limit: 5).isEmpty) + } + + // MARK: - DuckDuckGo result page + + private static let duckDuckGoPage = """ + +
+
+

+ NVD - CVE-2024-3094 +

+ Malicious code was discovered in xz tarballs. +
+
+
+
+

Sponsored Thing

+ An advert. +
+
+
+
+

XZ Utils backdoor

+
+
+ + """ + + @Test + func duckDuckGoPageYieldsResultsAndSkipsAds() { + let results = WebSearchClient.duckDuckGoResults(Self.duckDuckGoPage, limit: 10) + + #expect(results.count == 2) + #expect(results[0].url == "https://nvd.nist.gov/vuln/detail/CVE-2024-3094") + #expect(results[0].title == "NVD - CVE-2024-3094") + #expect(results[0].snippet == "Malicious code was discovered in xz tarballs.") + // An advert wears the same result markup, and handing one to the model + // as a source is worse than returning one result fewer. + #expect(!results.contains { $0.url.contains("sponsor.example") }) + } + + @Test + func duckDuckGoRedirectWrapperIsUnwrapped() { + let results = WebSearchClient.duckDuckGoResults(Self.duckDuckGoPage, limit: 10) + #expect(results[1].url == "https://en.wikipedia.org/wiki/XZ") + + // Both shapes appear on the same endpoint, so a direct link must survive + // the unwrapper untouched. + let direct = "https://example.com/page" + #expect(WebSearchClient.unwrapDuckDuckGoRedirect(direct) == direct) + } + + @Test + func duckDuckGoIsKeylessAndPostsItsQuery() throws { + #expect(!SearchBackend.duckDuckGo.requiresAPIKey) + let request = try WebSearchClient(backend: .duckDuckGo, market: "en-US") + .buildRequest(query: "swift 6", count: 5) + + #expect(request.httpMethod == "POST") + #expect(request.url?.absoluteString == "https://html.duckduckgo.com/html/") + let body = String(decoding: try #require(request.httpBody), as: UTF8.self) + #expect(body.contains("q=swift")) + // DuckDuckGo's region is Bing's market with its halves swapped. + #expect(body.contains("kl=us-en")) + } + + // MARK: - JSON backends + + @Test + func braveResponseDecodesIntoResults() throws { + let payload = Data(""" + {"web": {"results": [ + {"title": "Brave Hit", "url": "https://a.example", + "description": "First & best", "page_age": "2026-01-02T03:04:05Z"}, + {"title": "No URL"} + ]}} + """.utf8) + + let results = try WebSearchClient.decode(payload, backend: .brave, limit: 10) + + // The row with no URL is dropped rather than thrown on: one malformed + // result should not lose the others. + #expect(results.count == 1) + #expect(results[0].title == "Brave Hit") + #expect(results[0].snippet == "First & best") + #expect(results[0].publishedAt != nil) + } + + @Test + func tavilyAndExaReadTheirOwnSnippetFields() throws { + let tavily = try WebSearchClient.decode( + Data(""" + {"results": [{"title": "T", "url": "https://t.example", + "content": "tavily snippet", "published_date": "2026-02-03"}]} + """.utf8), + backend: .tavily, limit: 10 + ) + #expect(tavily[0].snippet == "tavily snippet") + #expect(tavily[0].publishedAt != nil) + + let exa = try WebSearchClient.decode( + Data(""" + {"results": [{"title": "E", "url": "https://e.example", "text": "exa snippet"}]} + """.utf8), + backend: .exa, limit: 10 + ) + #expect(exa[0].snippet == "exa snippet") + } + + @Test + func nonJSONFromAKeyedBackendIsAnError() { + #expect(throws: WebSearchError.self) { + try WebSearchClient.decode(Data("nope".utf8), backend: .brave, limit: 5) + } + } + + // MARK: - The tool + + @Test + func webSearchToolIsReadOnlyForAPublicBackend() async throws { + let client = WebSearchClient(backend: .bing) { _ in .publicInternet } + let tool = AgentBuiltInTools.webSearch(client) + + #expect(tool.descriptor.name == "web_search") + #expect(tool.descriptor.target == .network) + #expect(tool.descriptor.safety == .locallyReadOnly) + + let preflight = try await tool.preflight(invocation(query: "swift")) + #expect(preflight.safety == .locallyReadOnly) + #expect(preflight.reasons.isEmpty) + } + + @Test + func aBackendOnAPrivateNetworkAsksFirst() async throws { + // Matching `fetch`: a search backend configured at a LAN address is + // disclosed rather than silently reached, even though a query changes + // nothing. + let client = WebSearchClient( + backend: .tavily, baseURL: "https://searx.lan", apiKey: "k" + ) { _ in .restricted } + let preflight = try await AgentBuiltInTools.webSearch(client) + .preflight(invocation(query: "swift")) + + #expect(preflight.safety == .requiresAuthorization) + #expect(preflight.reasons.count == 1) + #expect(preflight.reasons[0].contains("https://searx.lan/search")) + } + + @Test + func aQueryIsRequired() async { + let client = WebSearchClient(backend: .bing) { _ in .publicInternet } + await #expect(throws: AgentToolError.self) { + try await AgentBuiltInTools.webSearch(client).preflight( + AgentToolInvocation( + runID: UUID(), + call: AgentToolCall(id: "1", name: "web_search", arguments: .object([:])) + ) + ) + } + } + + @Test @MainActor + func theToolIsAbsentUntilABackendIsConfigured() { + let hosts = AgentHostResolver(store: HostStore()) + let pool = AgentConnectionPool( + configuration: { _ in throw AgentToolError.invalidHost }, + routeFingerprint: { _ in "" }, + openExec: { _ in throw AgentToolError.invalidHost }, + openSFTP: { _ in throw AgentToolError.invalidHost }, + sweepInterval: nil + ) + + let without = AgentBuiltInTools.make(hosts: hosts, connections: pool) + #expect(!without.contains { $0.descriptor.name == "web_search" }) + + // A model shown a tool it cannot use spends a turn discovering that. + let with = AgentBuiltInTools.make( + hosts: hosts, connections: pool, search: WebSearchClient(backend: .bing) + ) + #expect(with.contains { $0.descriptor.name == "web_search" }) + } + + // MARK: - Provider-native search must stay out of the transcript + + /// The bug this pins produced, on the *second* request of a conversation: + /// + /// 400 No tool call found for function call output with call_id + /// provider-web-search-11328F4A-... + /// + /// A native search is a card, not a turn. The provider ran it inside its own + /// assistant turn and never asked Termind for anything, so a transcript that + /// carries it as a tool result replays a `function_call_output` matching no + /// `function_call`. The first request looks fine because it is the one that + /// puts the orphan into the history; the next one carries it and is rejected. + @Test @MainActor + func providerNativeSearchMakesACardButNotATranscriptTurn() { + let conversation = ConversationModel() + let invocation = self.invocation(query: "swift 6") + let descriptor = AgentToolDescriptor( + name: "web_search", summary: "Web search run by the model provider.", + inputSchema: .object(["type": .string("object")]), + target: .network, safety: .locallyReadOnly + ) + + conversation.handle(.toolProposed(invocation, descriptor)) + conversation.handle(.toolFinished(invocation, AgentToolResult( + callID: invocation.call.id, + content: "[]", + metadata: [AgentToolResult.providerNativeKey: .bool(true)] + ))) + + #expect(conversation.toolCards.count == 1) + #expect(conversation.transcript.isEmpty) + // And so nothing reaches the next request's history. + #expect(conversation.pendingPriorMessages.isEmpty) + } + + /// The other half: an ordinary tool result still becomes a turn, or the + /// model never learns what its own tool returned. + @Test @MainActor + func ordinaryToolResultsStillEnterTheTranscript() { + let conversation = ConversationModel() + let invocation = AgentToolInvocation( + runID: UUID(), + call: AgentToolCall( + id: "call-1", name: "web_search", + arguments: .object(["query": .string("swift 6")]) + ) + ) + let descriptor = AgentToolDescriptor( + name: "web_search", summary: "Search the public web.", + inputSchema: .object(["type": .string("object")]), + target: .network, safety: .locallyReadOnly + ) + + conversation.handle(.toolProposed(invocation, descriptor)) + conversation.handle(.toolFinished(invocation, AgentToolResult( + callID: "call-1", content: "\n[]\n" + ))) + + #expect(conversation.transcript.count == 1) + #expect(conversation.transcript[0].role == .tool) + #expect(conversation.transcript[0].toolCallID == "call-1") + } + + /// Repairs conversations already damaged by the bug above. + /// + /// A run persists its snapshot before the request goes out, so the failing + /// request's own history was written to disk. Without this, an affected + /// conversation keeps carrying the orphan forward and keeps being rejected, + /// restart or not. + @Test + func orphanedToolResultsAreDroppedFromIncomingHistory() { + let history: [AgentTranscriptMessage] = [ + AgentTranscriptMessage(role: .user, text: "look it up"), + AgentTranscriptMessage( + role: .assistant, text: "", + toolCalls: [AgentToolCall( + id: "call-1", name: "ssh_exec", arguments: .object([:]) + )] + ), + AgentTranscriptMessage( + role: .tool, text: "ok", toolCallID: "call-1", toolName: "ssh_exec" + ), + AgentTranscriptMessage( + role: .tool, text: "[]", + toolCallID: "provider-web-search-11328F4A", toolName: "web_search" + ), + AgentTranscriptMessage(role: .assistant, text: "Done."), + ] + + let repaired = AgentRuntime.withoutOrphanedToolResults(history) + + #expect(repaired.count == 4) + #expect(!repaired.contains { $0.toolCallID?.hasPrefix("provider-web-search") == true }) + // The answered one survives, or the model loses what its tool returned. + #expect(repaired.contains { $0.toolCallID == "call-1" }) + } + + @Test + func toolResultsWithoutACallIDAreKept() { + // Google addresses a tool result by name, not by id, so an absent id is + // not evidence of an orphan. + let history = [ + AgentTranscriptMessage(role: .tool, text: "ok", toolName: "list_hosts") + ] + #expect(AgentRuntime.withoutOrphanedToolResults(history).count == 1) + } + + private func invocation(query: String) -> AgentToolInvocation { + AgentToolInvocation( + runID: UUID(), + call: AgentToolCall( + id: "call-1", name: "web_search", + arguments: .object(["query": .string(query)]) + ) + ) + } +}