Ask a group of decisions at once, not one at a time

`request_user_input` took one question. A turn that needed three raised them
one after another, each hidden until the one before it was answered — and a
second question you cannot see is a second question that cannot inform the
first. It now takes up to three, drawn as one form with one Continue.

The earlier message on `2ec0e12` gave the wrong reason for not doing this.
It said several questions per call "would need a queue and a
multi-question card", as though the runtime's single `pendingUserInput`
forbade it. It never did: a call carrying three questions is still one
blocking request and one continuation, which is exactly what a queue is
not. That was a preference dressed as a constraint. The real cost is the
two things below — per-question drafts, and a form that can outgrow its
strip — and neither was a reason to refuse.

The queue that *did* arrive in `22258e3` does not cover it either, for a
reason worth writing down. `request_user_input` is `.sequential`, and a
batch runs in parallel only when every call in it is, so two questions in
one turn execute one after the other and `pendingUserInputs` never holds
more than one. The queue and its `1 of N` are live for approvals and
purely defensive for questions. Sequential is right for this tool —
racing a question against other calls means output arriving under
something still being read — so batching inside one call is the only way
several questions reach a person together. The queue stays as it is.

A question may be left unanswered, and the tool reports that as a null
rather than an error. That needs saying in the tool's own description or
a model reads the null as a failure and asks the same thing again, which
puts the user in a loop they answered their way out of once already. So
the description now says plainly: a null is the user declining to decide
that one — choose, say which assumption you used, and do not ask again.
Continue needs one answer, not all of them; it needs at least one because
answering nothing is Cancel, and reporting that as success would hand the
model a result whose only content is that it was not refused.

Drafts become per question. Sharing one text field and one selection set
across a form meant typing into the third question cleared the first, and
the mutual exclusion between an option and a typed answer — which is
per-question by definition — reached across all of them. `AgentUserQuestion`
carries the id the model chose, so answers travel under the model's own
names rather than an index we assign, and every question is reported back
whether answered or not: an absent key would read as a bug where a null
reads as a decision.

Two things fell out of rendering it. A bare `ScrollView` capped at the
form's maximum height is greedy, so a one-question card was handed 320pt
of box for 120pt of question; `ViewThatFits` prefers the natural stack and
falls back to scrolling only when the questions genuinely overflow. And
`reported` is not private, because that shape is the contract with the
model and a test should be able to hold it still — including that a
secret, which `request_user_input` never asks for, is reported as skipped
rather than echoed if one ever arrives there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kaede AkinoandClaude Opus 5 committed 2026-08-26 09:52:21 +08:00
1 parent bfbebc7b80
commit da3082aff8
7 files changed
+761 -263

No files matched your search

+155 -61
View File
@@ -715,77 +715,115 @@ nonisolated enum AgentBuiltInTools {
AnyAgentTool(descriptor: descriptor(
"request_user_input",
"""
Ask the user to decide something you cannot decide yourself, and wait for the answer. \
Ask the user to decide things you cannot decide yourself, and wait for the answers. \
Use it only when the answer is genuinely theirs: a preference, a tradeoff with no \
defensible default, or a fact only they hold — which host is production, where a \
credential lives. If the repository, the request, or an obvious convention already \
answers it, choose, say which assumption you made, and keep going; a question the \
user has to stop and think about costs more than a decision you make and flag. Never \
use it to ask whether to proceed. Give 2-4 options, each with a description saying \
what it costs rather than restating the label, and put your recommendation on the \
option itself instead of in its wording. A free-form field is offered alongside them \
unless you set allow_custom false, so never write an "Other" option yourself. Omit \
options entirely for a question no menu can answer, such as a URL or a name.
use it to ask whether to proceed. Ask a whole group of related decisions in one \
call, up to three, rather than one at a time — a second question raised only after \
the first is answered was hidden at the moment it would have informed it. One \
question is still the common case; three is a ceiling, not a target. Give each \
question 2-4 options with a description saying what it costs rather than restating \
the label, and put your recommendation on the option itself instead of in its \
wording. A free-form field is offered alongside them unless you set allow_custom \
false, so never write an "Other" option yourself. Omit options entirely for a \
question no menu can answer, such as a URL or a name. A question may come back with \
a null answer: that is the user declining to decide that one, not an error and not \
a reason to ask again — decide it yourself and say which assumption you used.
""",
properties: [
"prompt": string(max: 4_096),
"header": string(max: 24),
"options": array(
"questions": array(
items: object(
properties: [
"id": string(max: 64),
"label": string(max: 120),
"description": string(max: 400),
"recommended": boolean(),
"header": string(max: 24),
"prompt": string(max: 4_096),
"options": array(
items: object(
properties: [
"id": string(max: 64),
"label": string(max: 120),
"description": string(max: 400),
"recommended": boolean(),
],
required: ["id", "label"]
),
max: 4
),
"multi_select": boolean(),
"allow_custom": boolean(),
"custom_placeholder": string(max: 120),
],
required: ["id", "label"]
required: ["id", "prompt"]
),
max: 4
),
"multi_select": boolean(),
"allow_custom": boolean(),
"custom_placeholder": string(max: 120),
], required: ["prompt"], target: .user, safety: .locallyReadOnly
max: AgentUserInputLimits.maxQuestions
)
], required: ["questions"], target: .user, safety: .locallyReadOnly
)) { invocation, context in
let arguments = try Arguments(invocation)
let options = try arguments.userInputOptions()
let request = AgentUserInputRequest(
prompt: try arguments.string("prompt"),
kind: options.isEmpty
? .text
: .choice(
AgentUserInputChoice(
options: options,
allowsCustomAnswer: arguments.optionalBool("allow_custom") ?? true,
allowsMultipleAnswers: arguments.optionalBool("multi_select") ?? false,
customPlaceholder: arguments.optionalString("custom_placeholder") ?? ""
)
),
header: arguments.optionalString("header") ?? ""
let questions = try Arguments(invocation).userQuestions()
let response = await context.userInteraction.request(
AgentUserInputRequest(questions: questions)
)
switch await context.userInteraction.request(request) {
case .text(let answer):
return result(invocation, .object(["answer": .string(answer)]))
guard case .answered(let answers) = response else { throw AgentToolError.cancelled }
return result(invocation, .object([
"answers": .array(reported(questions, answers))
]))
}
}
/// Every question, answered or not, in the order it was asked.
///
/// Reporting the skipped ones rather than omitting them is what makes a
/// null legible: a model reading its own list back finds each entry where
/// it put it, and an absent key would read as a bug rather than as the user
/// declining that one.
///
/// Not private, because this shape *is* the contract with the model and a
/// test should be able to hold it still.
static func reported(
_ questions: [AgentUserQuestion], _ answers: [AgentUserInputAnswer]
) -> [AgentJSONValue] {
let byID = Dictionary(answers.map { ($0.questionID, $0.value) }) { first, _ in first }
return questions.map { question in
switch byID[question.id] {
case .choice(let selectedIDs, let custom):
let labels = selectedIDs.compactMap { id in
options.first { $0.id == id }?.label
question.options.first { $0.id == id }?.label
}
return result(invocation, .object([
return row(
question.id,
// The canonical human answer, whichever route produced it,
// so a model that reads only one field still reads the
// right one.
"answer": .string(custom ?? labels.joined(separator: ", ")),
"selected_option_ids": .array(selectedIDs.map(AgentJSONValue.string)),
"custom_text": custom.map(AgentJSONValue.string) ?? .null,
]))
// so a model that reads one field reads the right one.
answer: custom ?? labels.joined(separator: ", "),
selected: selectedIDs, custom: custom
)
case .text(let typed):
return row(question.id, answer: typed, selected: [], custom: typed)
case .secret:
throw AgentToolError.invalidArguments("Secret input must use request_user_secret.")
case .cancelled:
throw AgentToolError.cancelled
// Unreachable — `request_user_input` never builds a secret
// question. Reported as skipped rather than echoed, so that if
// one ever does arrive here the plaintext still does not reach
// the model.
return row(question.id, answer: nil, selected: [], custom: nil)
case nil:
return row(question.id, answer: nil, selected: [], custom: nil)
}
}
}
private static func row(
_ questionID: String, answer: String?, selected: [String], custom: String?
) -> AgentJSONValue {
.object([
"question_id": .string(questionID),
"answer": answer.map(AgentJSONValue.string) ?? .null,
"selected_option_ids": .array(selected.map(AgentJSONValue.string)),
"custom_text": custom.map(AgentJSONValue.string) ?? .null,
])
}
private static func requestUserSecret() -> AnyAgentTool {
AnyAgentTool(descriptor: descriptor(
"request_user_secret", "Collect a secret in SecureField and return a one-shot bound handle.",
@@ -803,12 +841,27 @@ nonisolated enum AgentBuiltInTools {
let hostID = try arguments.uuid("host_id")
let toolName = try arguments.string("consumer_tool")
let purpose = try arguments.string("purpose")
// One question, always. A secret is not a decision among options and
// has no business sharing a form with one.
let request = AgentUserInputRequest(
prompt: try arguments.string("prompt"), kind: .secret,
questions: [
AgentUserQuestion(
id: "secret", prompt: try arguments.string("prompt"), kind: .secret
)
],
purpose: purpose, hostID: hostID
)
switch await context.userInteraction.request(request) {
case .secret(let secret):
case .answered(let answers):
guard case .secret(let secret)? = answers.first?.value else {
// A secret must come back through `SecureField` and the
// broker. Anything else means it took the ordinary-answer
// path, where it would have been echoed on screen and kept
// in plain state.
throw AgentToolError.invalidArguments(
"The secret prompt returned ordinary text."
)
}
let handle = await context.secretBroker.issue(
secret,
binding: SecretBinding(
@@ -818,11 +871,6 @@ nonisolated enum AgentBuiltInTools {
return result(invocation, .object([
"secret_handle": .string(handle.id.uuidString), "single_use": .bool(true),
]))
case .text, .choice:
// A secret must come back through `SecureField` and the broker.
// Anything else means it took the ordinary-answer path, where it
// would have been echoed on screen and kept in plain state.
throw AgentToolError.invalidArguments("The secret prompt returned ordinary text.")
case .cancelled: throw AgentToolError.cancelled
}
}
@@ -1022,13 +1070,59 @@ nonisolated struct Arguments: Sendable {
object[key]?.arrayValue?.compactMap(\.stringValue) ?? []
}
/// The `options` array of `request_user_input`.
/// The `questions` array of `request_user_input`.
///
/// Malformed rows are rejected rather than dropped. A menu silently missing
/// the option the user wanted is a worse failure than an error the model
/// can see and retry.
func userInputOptions() throws -> [AgentUserInputOption] {
guard let value = object["options"], value != .null else { return [] }
/// Everything malformed is rejected rather than dropped. A form silently
/// missing the question — or the option — the user needed is a worse
/// failure than an error the model can see and retry.
func userQuestions() throws -> [AgentUserQuestion] {
guard let rows = object["questions"]?.arrayValue else {
throw AgentToolError.invalidArguments("questions must be an array.")
}
guard !rows.isEmpty else {
throw AgentToolError.invalidArguments("questions must contain at least one question.")
}
guard rows.count <= AgentUserInputLimits.maxQuestions else {
throw AgentToolError.invalidArguments(
"At most \(AgentUserInputLimits.maxQuestions) questions may be asked at once."
)
}
var seen: Set<String> = []
return try rows.map { row in
guard let fields = row.objectValue,
let id = fields["id"]?.stringValue, !id.isEmpty,
let prompt = fields["prompt"]?.stringValue, !prompt.isEmpty
else {
throw AgentToolError.invalidArguments("Each question requires id and prompt.")
}
guard seen.insert(id).inserted else {
throw AgentToolError.invalidArguments("Question ids must be unique: \(id).")
}
let options = try Self.options(in: fields)
return AgentUserQuestion(
id: id,
header: fields["header"]?.stringValue ?? "",
prompt: prompt,
kind: options.isEmpty
? .text
: .choice(
AgentUserInputChoice(
options: options,
allowsCustomAnswer: fields["allow_custom"]?.boolValue ?? true,
allowsMultipleAnswers: fields["multi_select"]?.boolValue ?? false,
customPlaceholder: fields["custom_placeholder"]?.stringValue ?? ""
)
)
)
}
}
private static func options(in question: [String: AgentJSONValue]) throws
-> [AgentUserInputOption]
{
guard let value = question["options"], value != .null else { return [] }
guard let rows = value.arrayValue else {
throw AgentToolError.invalidArguments("options must be an array.")
}
+64 -11
View File
@@ -201,30 +201,83 @@ nonisolated struct AgentUserInputChoice: Hashable, Sendable {
var customPlaceholder: String = ""
}
nonisolated struct AgentUserInputRequest: Identifiable, Hashable, Sendable {
nonisolated enum AgentUserInputLimits {
/// Codex's ceiling, and for the same reason: past three, a form stops being
/// a decision and starts being a questionnaire.
static let maxQuestions = 3
}
/// One thing the agent needs decided.
nonisolated struct AgentUserQuestion: Identifiable, Hashable, Sendable {
nonisolated enum Kind: Hashable, Sendable {
case text
case choice(AgentUserInputChoice)
case secret
}
var id: UUID = UUID()
var prompt: String
var kind: Kind
/// Named by the model, unique within its request. It wrote the questions
/// and it reads the answers back by these names, so they are its own
/// vocabulary rather than an index we assign.
var id: String
/// Short label naming the decision, shown above the prompt.
var header: String = ""
var prompt: String
var kind: Kind
/// The choices this question offers, or none for a free-form one. Saves
/// every reader of a question from re-matching on `kind` to find out.
var options: [AgentUserInputOption] {
guard case .choice(let choice) = kind else { return [] }
return choice.options
}
}
/// A set of questions put to the user at once.
///
/// Several rather than one because a group of decisions is usually a group —
/// asking about storage, then about migrations only once storage is answered,
/// hides the second question at the moment it would have informed the first.
/// The tool's calls are `.sequential`, so asking twice in a turn already meant
/// answering twice in a row; this makes it one form instead.
nonisolated struct AgentUserInputRequest: Identifiable, Hashable, Sendable {
var id: UUID = UUID()
var questions: [AgentUserQuestion]
var purpose: String = ""
var hostID: UUID?
}
/// What has been filled into one question so far.
nonisolated struct AgentUserInputDraft: Hashable, Sendable {
/// The "something else" field, and the whole answer for a free-form
/// question.
var text = ""
var selection: Set<String> = []
var trimmedText: String { text.trimmingCharacters(in: .whitespacesAndNewlines) }
}
nonisolated struct AgentUserInputAnswer: Hashable, Sendable {
nonisolated enum Value: Hashable, Sendable {
case text(String)
/// Selected option ids in the order the question listed them, plus
/// whatever the user typed instead of — or, when several answers are
/// allowed, alongside — them. Kept structured rather than flattened to
/// a string so the tool can hand the model ids it chose itself.
case choice(selectedIDs: [String], custom: String?)
case secret(String)
}
var questionID: String
var value: Value
}
nonisolated enum AgentUserInputResponse: Hashable, Sendable {
case text(String)
/// Selected option ids in the order the request listed them, plus whatever
/// the user typed instead of — or, when several answers are allowed,
/// alongside — them. Kept structured rather than flattened to a string so
/// the tool can hand the model ids it chose itself.
case choice(selectedIDs: [String], custom: String?)
case secret(String)
/// Only the questions the user actually answered.
///
/// An absent question was skipped, and skipping is itself an answer — the
/// user declining to decide that one. So there is no `.skipped` case: the
/// absence carries it, and the tool reports it to the model as a null.
case answered([AgentUserInputAnswer])
case cancelled
}
+7
View File
@@ -176,6 +176,13 @@ nonisolated enum Metrics {
/// draft instead; these two bound the result.
static let composerMinHeight: CGFloat = 40
static let composerMaxHeight: CGFloat = 132
/// How tall the agent's question form may grow before it scrolls.
///
/// Three questions with four described options each runs past 500pt,
/// and this card sits between the transcript and the composer — past
/// this it would push the conversation it is about off the screen.
static let userInputMaxHeight: CGFloat = 320
}
enum TransferCenter {
+74 -43
View File
@@ -218,10 +218,12 @@ final class ConversationModel: Identifiable {
var pendingApproval: AgentApprovalRequest? { pendingApprovals.first }
var pendingUserInput: AgentUserInputRequest? { pendingUserInputs.first }
/// The free-form answer, for a text question and for the "something else"
/// field beside a set of options.
var userInputDraft = ""
var userInputSelection: Set<String> = []
/// What has been filled in so far, keyed by question. One request can now
/// carry several questions, so the drafts are per question rather than one
/// pair of fields shared by whichever one happened to be on screen.
private(set) var userInputDrafts: [String: AgentUserInputDraft] = [:]
/// Kept out of `userInputDrafts` on purpose: a secret does not belong in
/// the same dictionary as text that is echoed back to the model.
var secretInputDraft = ""
/// The model-facing history, which is not what `messages` holds.
///
@@ -477,64 +479,94 @@ final class ConversationModel: Identifiable {
refreshRunState()
}
/// Nothing is preselected, so this is what stops Continue from returning an
/// answer the user never gave.
var canSubmitUserInput: Bool {
guard let request = pendingUserInput else { return false }
switch request.kind {
func userInputDraft(for questionID: String) -> AgentUserInputDraft {
userInputDrafts[questionID] ?? AgentUserInputDraft()
}
/// Whether this one question has been answered. Skipping is allowed, so
/// this drives a count rather than a validation error.
func isUserInputAnswered(_ question: AgentUserQuestion) -> Bool {
switch question.kind {
case .secret: return !secretInputDraft.isEmpty
case .text: return !trimmedUserInputDraft.isEmpty
case .choice: return !userInputSelection.isEmpty || !trimmedUserInputDraft.isEmpty
default:
let draft = userInputDraft(for: question.id)
return !draft.selection.isEmpty || !draft.trimmedText.isEmpty
}
}
func toggleUserInputOption(_ id: String) {
guard case .choice(let choice)? = pendingUserInput?.kind else { return }
var answeredUserInputCount: Int {
pendingUserInput?.questions.filter(isUserInputAnswered).count ?? 0
}
/// Nothing is preselected, so this is what stops Continue from returning an
/// answer the user never gave.
///
/// One answer is enough — a question may be left alone, and the tool
/// reports that as a null the model has to decide for itself. Zero is not
/// enough: answering nothing is Cancel, and dressing it as success would
/// hand the model a result that says only that it was not refused.
var canSubmitUserInput: Bool { answeredUserInputCount > 0 }
func toggleUserInputOption(_ optionID: String, in questionID: String) {
guard case .choice(let choice)? = question(questionID)?.kind else { return }
var draft = userInputDraft(for: questionID)
if choice.allowsMultipleAnswers {
if userInputSelection.contains(id) {
userInputSelection.remove(id)
if draft.selection.contains(optionID) {
draft.selection.remove(optionID)
} else {
userInputSelection.insert(id)
draft.selection.insert(optionID)
}
} else {
userInputSelection = userInputSelection.contains(id) ? [] : [id]
draft.selection = draft.selection.contains(optionID) ? [] : [optionID]
// One answer means one answer: a half-typed "something else" left
// sitting under a chosen radio would be a second, contradicting one.
userInputDraft = ""
draft.text = ""
}
userInputDrafts[questionID] = draft
}
/// The other half of that rule — typing clears the radio it would otherwise
/// silently disagree with. Left alone in a multi-answer question, where the
/// two are additive rather than exclusive.
func setUserInputCustomAnswer(_ text: String) {
userInputDraft = text
guard case .choice(let choice)? = pendingUserInput?.kind,
func setUserInputCustomAnswer(_ text: String, for questionID: String) {
var draft = userInputDraft(for: questionID)
draft.text = text
if case .choice(let choice)? = question(questionID)?.kind,
!choice.allowsMultipleAnswers, !text.isEmpty
else { return }
userInputSelection.removeAll()
{
draft.selection.removeAll()
}
userInputDrafts[questionID] = draft
}
func submitUserInput() {
guard let request = pendingUserInput, canSubmitUserInput else { return }
let response: AgentUserInputResponse
switch request.kind {
case .secret:
response = .secret(secretInputDraft)
case .text:
response = .text(userInputDraft)
case .choice(let choice):
let custom = trimmedUserInputDraft
response = .choice(
// Ordered by the request, not by whatever order the user
// clicked in — the model wrote the list and reads the answer
// against it.
selectedIDs: choice.options.map(\.id).filter(userInputSelection.contains),
custom: custom.isEmpty ? nil : custom
)
let answers = request.questions.compactMap { question -> AgentUserInputAnswer? in
guard isUserInputAnswered(question) else { return nil }
let draft = userInputDraft(for: question.id)
switch question.kind {
case .secret:
return AgentUserInputAnswer(
questionID: question.id, value: .secret(secretInputDraft)
)
case .text:
return AgentUserInputAnswer(questionID: question.id, value: .text(draft.text))
case .choice(let choice):
let custom = draft.trimmedText
return AgentUserInputAnswer(
questionID: question.id,
// Ordered by the question, not by whatever order the user
// clicked in — the model wrote the list and reads the
// answer against it.
value: .choice(
selectedIDs: choice.options.map(\.id).filter(draft.selection.contains),
custom: custom.isEmpty ? nil : custom
)
)
}
}
advanceUserInput()
inputWaiters.removeValue(forKey: request.id)?.resume(returning: response)
inputWaiters.removeValue(forKey: request.id)?.resume(returning: .answered(answers))
}
func cancelUserInput() {
@@ -543,8 +575,8 @@ final class ConversationModel: Identifiable {
inputWaiters.removeValue(forKey: request.id)?.resume(returning: .cancelled)
}
private var trimmedUserInputDraft: String {
userInputDraft.trimmingCharacters(in: .whitespacesAndNewlines)
private func question(_ id: String) -> AgentUserQuestion? {
pendingUserInput?.questions.first { $0.id == id }
}
/// Retires the answered question and hands a clean sheet to the next one.
@@ -556,8 +588,7 @@ final class ConversationModel: Identifiable {
private func clearUserInputDrafts() {
secretInputDraft = ""
userInputDraft = ""
userInputSelection = []
userInputDrafts = [:]
}
/// What the transcript shows, from what is actually outstanding.
@@ -258,71 +258,121 @@ private struct QueueDepthLabel: View {
}
}
/// The agent asking the user to decide something.
/// The agent asking the user to decide one or more things.
///
/// A form rather than a run of cards. A turn's questions are usually one group
/// of decisions, and revealing the second only once the first is answered hides
/// it at the moment it would have informed the first.
///
/// Options are a list rather than a `Picker` because each one carries a
/// sentence explaining what it costs, and a pop-up menu has nowhere to put
/// that. Nothing is preselected — including the recommendation, which is a
/// label and not a default, so that an answer always takes a deliberate click.
/// A question may also be left alone: the tool reports that as a null and the
/// model decides it itself, so the count in the footer is a count and not a
/// validation error, and an unanswered question is marked in no way at all.
private struct AgentUserInputCard: View {
let request: AgentUserInputRequest
let conversation: ConversationModel
private var isForm: Bool { request.questions.count > 1 }
var body: some View {
@Bindable var conversation = conversation
VStack(alignment: .leading, spacing: Metrics.Space.s) {
HStack {
if !request.header.isEmpty {
Text(request.header)
.font(.caption.weight(.semibold))
.foregroundStyle(.secondary)
VStack(alignment: .leading, spacing: Metrics.Space.m) {
if conversation.pendingUserInputs.count > 1 {
HStack {
Spacer()
QueueDepthLabel(remaining: conversation.pendingUserInputs.count)
}
Spacer()
QueueDepthLabel(remaining: conversation.pendingUserInputs.count)
}
Text(request.prompt)
.font(.callout.weight(.medium))
.fixedSize(horizontal: false, vertical: true)
switch request.kind {
case .secret:
SecureField("Secret", text: $conversation.secretInputDraft)
.textFieldStyle(.roundedBorder)
case .choice(let choice):
choiceList(choice)
case .text:
TextField("Answer", text: $conversation.userInputDraft)
.textFieldStyle(.roundedBorder)
}
HStack {
Button("Cancel") { conversation.cancelUserInput() }
Spacer()
Button("Continue") { conversation.submitUserInput() }
.buttonStyle(.borderedProminent)
.disabled(!conversation.canSubmitUserInput)
// Three questions with four described options each runs past 500pt,
// and this card sits between the transcript and the composer. But a
// bare `ScrollView` is greedy: capped at the same height it would
// hand a one-question card 320pt of box for 120pt of question. So
// the natural stack is preferred and the scrolling one is the
// fallback, which is what `ViewThatFits` is for.
ViewThatFits(in: .vertical) {
questionStack
ScrollView { questionStack }
.scrollBounceBehavior(.basedOnSize)
}
.frame(maxHeight: Metrics.Chat.userInputMaxHeight)
footer
}
.padding(Metrics.Space.l)
}
private func choiceList(_ choice: AgentUserInputChoice) -> some View {
VStack(alignment: .leading, spacing: Metrics.Space.xs) {
ForEach(choice.options) { option in
optionRow(option, choice: choice)
private var questionStack: some View {
VStack(alignment: .leading, spacing: Metrics.Space.l) {
ForEach(request.questions) { question in
questionBlock(question)
}
if choice.allowsCustomAnswer {
customRow(choice)
}
.frame(maxWidth: .infinity, alignment: .leading)
}
private var footer: some View {
HStack(spacing: Metrics.Space.m) {
Button("Cancel") { conversation.cancelUserInput() }
Spacer(minLength: 0)
if isForm {
Text(
"\(conversation.answeredUserInputCount) of \(request.questions.count) answered"
)
.font(.caption.monospacedDigit())
.foregroundStyle(.secondary)
}
Button("Continue") { conversation.submitUserInput() }
.buttonStyle(.borderedProminent)
.disabled(!conversation.canSubmitUserInput)
}
}
@ViewBuilder
private func questionBlock(_ question: AgentUserQuestion) -> some View {
@Bindable var conversation = conversation
VStack(alignment: .leading, spacing: Metrics.Space.s) {
if !question.header.isEmpty {
Text(question.header)
.font(.caption.weight(.semibold))
.foregroundStyle(.secondary)
}
Text(question.prompt)
.font(.callout.weight(.medium))
.fixedSize(horizontal: false, vertical: true)
switch question.kind {
case .secret:
SecureField("Secret", text: $conversation.secretInputDraft)
.textFieldStyle(.roundedBorder)
case .choice(let choice):
choiceList(choice, in: question.id)
case .text:
TextField("Answer", text: customBinding(for: question.id))
.textFieldStyle(.roundedBorder)
}
}
}
private func optionRow(_ option: AgentUserInputOption, choice: AgentUserInputChoice)
-> some View
{
let isSelected = conversation.userInputSelection.contains(option.id)
private func choiceList(_ choice: AgentUserInputChoice, in questionID: String) -> some View {
VStack(alignment: .leading, spacing: Metrics.Space.xs) {
ForEach(choice.options) { option in
optionRow(option, choice: choice, in: questionID)
}
if choice.allowsCustomAnswer {
customRow(choice, in: questionID)
}
}
}
private func optionRow(
_ option: AgentUserInputOption, choice: AgentUserInputChoice, in questionID: String
) -> some View {
let isSelected = conversation.userInputDraft(for: questionID).selection.contains(option.id)
return Button {
conversation.toggleUserInputOption(option.id)
conversation.toggleUserInputOption(option.id, in: questionID)
} label: {
HStack(alignment: .firstTextBaseline, spacing: Metrics.Space.s) {
Image(systemName: marker(isSelected: isSelected, choice: choice))
@@ -358,31 +408,29 @@ private struct AgentUserInputCard: View {
/// The escape hatch, sitting in the list as one more choice rather than
/// below it as an afterthought — an answer the model did not think of is
/// not a lesser kind of answer.
private func customRow(_ choice: AgentUserInputChoice) -> some View {
HStack(alignment: .firstTextBaseline, spacing: Metrics.Space.s) {
Image(
systemName: marker(
isSelected: !conversation.userInputDraft.isEmpty, choice: choice
)
)
.foregroundStyle(
conversation.userInputDraft.isEmpty
? AnyShapeStyle(.secondary) : AnyShapeStyle(.tint)
)
.frame(width: Metrics.Sidebar.rowIcon)
private func customRow(_ choice: AgentUserInputChoice, in questionID: String) -> some View {
let typed = conversation.userInputDraft(for: questionID).text
return HStack(alignment: .firstTextBaseline, spacing: Metrics.Space.s) {
Image(systemName: marker(isSelected: !typed.isEmpty, choice: choice))
.foregroundStyle(typed.isEmpty ? AnyShapeStyle(.secondary) : AnyShapeStyle(.tint))
.frame(width: Metrics.Sidebar.rowIcon)
TextField(
choice.customPlaceholder.isEmpty
? String(localized: "Something else…") : choice.customPlaceholder,
text: Binding(
get: { conversation.userInputDraft },
set: { conversation.setUserInputCustomAnswer($0) }
)
text: customBinding(for: questionID)
)
.textFieldStyle(.roundedBorder)
}
}
private func customBinding(for questionID: String) -> Binding<String> {
Binding(
get: { conversation.userInputDraft(for: questionID).text },
set: { conversation.setUserInputCustomAnswer($0, for: questionID) }
)
}
private func marker(isSelected: Bool, choice: AgentUserInputChoice) -> String {
if choice.allowsMultipleAnswers {
isSelected ? "checkmark.square.fill" : "square"
+5 -1
View File
@@ -176,7 +176,11 @@ struct AgentApprovalQueueTests {
conversation.handle(.reviewStarted(request))
#expect(conversation.runState == .reviewing)
conversation.present(AgentUserInputRequest(prompt: "which host?", kind: .text))
conversation.present(
AgentUserInputRequest(
questions: [AgentUserQuestion(id: "host", prompt: "which host?", kind: .text)]
)
)
#expect(conversation.runState == .waitingForUser)
// The review finishing must not announce that nothing is outstanding.
+351 -90
View File
@@ -7,79 +7,121 @@ import Testing
@Suite
struct AgentUserInputTests {
// MARK: - Parsing the tool's options
// MARK: - Parsing the tool's questions
@Test
func optionsCarryTheirDescriptionAndRecommendation() throws {
let options = try arguments([
"prompt": .string("Which database?"),
"options": .array([
func questionsCarryTheirOptionsDescriptionsAndRecommendations() throws {
let questions = try arguments([
"questions": .array([
.object([
"id": .string("postgres"), "label": .string("Postgres"),
"description": .string("JSONB, and it matches the existing schema."),
"recommended": .bool(true),
"id": .string("storage"),
"header": .string("Storage"),
"prompt": .string("Which database?"),
"options": .array([
.object([
"id": .string("postgres"), "label": .string("Postgres"),
"description": .string("JSONB, and it matches the existing schema."),
"recommended": .bool(true),
]),
.object(["id": .string("sqlite"), "label": .string("SQLite")]),
]),
]),
.object(["id": .string("sqlite"), "label": .string("SQLite")]),
]),
]).userInputOptions()
.object(["id": .string("base_url"), "prompt": .string("Which URL?")]),
])
]).userQuestions()
#expect(options.count == 2)
#expect(options[0].id == "postgres")
#expect(options[0].detail == "JSONB, and it matches the existing schema.")
#expect(options[0].isRecommended)
#expect(questions.count == 2)
#expect(questions[0].header == "Storage")
#expect(questions[0].options.count == 2)
#expect(questions[0].options[0].detail == "JSONB, and it matches the existing schema.")
#expect(questions[0].options[0].isRecommended)
// Absent is absent, not false-y noise.
#expect(options[1].detail.isEmpty)
#expect(!options[1].isRecommended)
}
#expect(questions[0].options[1].detail.isEmpty)
#expect(!questions[0].options[1].isRecommended)
@Test
func noOptionsMeansAFreeFormQuestion() throws {
#expect(try arguments(["prompt": .string("Which URL?")]).userInputOptions().isEmpty)
#expect(
try arguments(["prompt": .string("Which URL?"), "options": .null])
.userInputOptions().isEmpty
)
}
/// A malformed row is rejected rather than dropped: a menu quietly missing
/// the option the user wanted is worse than an error the model can retry.
@Test
func malformedOptionsAreRejectedRatherThanDropped() {
#expect(throws: AgentToolError.self) {
try arguments([
"prompt": .string("?"),
"options": .array([.object(["label": .string("No id here")])]),
]).userInputOptions()
// No options is a free-form question, not an empty menu.
guard case .text = questions[1].kind else {
Issue.record("expected a free-form question, got \(questions[1].kind)")
return
}
#expect(throws: AgentToolError.self) {
try arguments([
"prompt": .string("?"),
}
@Test
func choiceFlagsDefaultToOfferingAWayOut() throws {
let questions = try arguments([
"questions": .array([
.object([
"id": .string("q"), "prompt": .string("?"),
"options": .array([.object(["id": .string("a"), "label": .string("A")])]),
])
])
]).userQuestions()
guard case .choice(let choice) = questions[0].kind else {
Issue.record("expected a choice question")
return
}
#expect(choice.allowsCustomAnswer, "a way out has to be the default")
#expect(!choice.allowsMultipleAnswers)
}
/// Everything malformed is rejected rather than dropped: a form silently
/// missing the question the user needed is worse than an error the model
/// can retry.
@Test
func malformedQuestionsAreRejectedRatherThanDropped() {
let bad: [[String: AgentJSONValue]] = [
[:],
["questions": .array([])],
["questions": .object([:])],
["questions": .array([.object(["prompt": .string("no id")])])],
["questions": .array([.object(["id": .string("q")])])],
["questions": .array([
.object(["id": .string("q"), "prompt": .string("first")]),
.object(["id": .string("q"), "prompt": .string("duplicate id")]),
])],
["questions": .array([
.object([
"id": .string("q"), "prompt": .string("?"),
"options": .array([
.object(["id": .string("a"), "label": .string("A")]),
.object(["id": .string("a"), "label": .string("Also A")]),
]),
])
])],
["questions": .array([.object([
"id": .string("q"), "prompt": .string("?"),
"options": .array([.string("a bare string")]),
]).userInputOptions()
])])],
]
for object in bad {
#expect(throws: AgentToolError.self) { try arguments(object).userQuestions() }
}
}
@Test
func duplicateOptionIdsAreRejected() {
func askingMoreThanThreeQuestionsAtOnceIsRefused() {
let four = (1...4).map { index in
AgentJSONValue.object([
"id": .string("q\(index)"), "prompt": .string("Question \(index)?"),
])
}
#expect(throws: AgentToolError.self) {
try arguments([
"prompt": .string("?"),
"options": .array([
.object(["id": .string("a"), "label": .string("First")]),
.object(["id": .string("a"), "label": .string("Second")]),
]),
]).userInputOptions()
try arguments(["questions": .array(four)]).userQuestions()
}
#expect(throws: Never.self) {
try arguments(["questions": .array(Array(four.prefix(3)))]).userQuestions()
}
}
// MARK: - One answer, or several
// MARK: - One question
@Test
func nothingIsPreselectedEvenWhenAnOptionIsRecommended() {
let conversation = ConversationModel()
conversation.present(request(multiple: false))
#expect(conversation.userInputSelection.isEmpty)
#expect(conversation.userInputDraft(for: "storage").selection.isEmpty)
// A recommendation is a label, not a default — Continue stays off until
// the user actually answers.
#expect(!conversation.canSubmitUserInput)
@@ -90,12 +132,12 @@ struct AgentUserInputTests {
let conversation = ConversationModel()
conversation.present(request(multiple: false))
conversation.setUserInputCustomAnswer("MySQL")
conversation.setUserInputCustomAnswer("MySQL", for: "storage")
#expect(conversation.canSubmitUserInput)
conversation.toggleUserInputOption("postgres")
#expect(conversation.userInputSelection == ["postgres"])
#expect(conversation.userInputDraft.isEmpty)
conversation.toggleUserInputOption("postgres", in: "storage")
#expect(conversation.userInputDraft(for: "storage").selection == ["postgres"])
#expect(conversation.userInputDraft(for: "storage").text.isEmpty)
}
@Test
@@ -103,11 +145,11 @@ struct AgentUserInputTests {
let conversation = ConversationModel()
conversation.present(request(multiple: false))
conversation.toggleUserInputOption("postgres")
conversation.setUserInputCustomAnswer("MySQL")
conversation.toggleUserInputOption("postgres", in: "storage")
conversation.setUserInputCustomAnswer("MySQL", for: "storage")
#expect(conversation.userInputSelection.isEmpty)
#expect(conversation.userInputDraft == "MySQL")
#expect(conversation.userInputDraft(for: "storage").selection.isEmpty)
#expect(conversation.userInputDraft(for: "storage").text == "MySQL")
}
@Test
@@ -115,13 +157,13 @@ struct AgentUserInputTests {
let conversation = ConversationModel()
conversation.present(request(multiple: false))
conversation.toggleUserInputOption("postgres")
conversation.toggleUserInputOption("sqlite")
#expect(conversation.userInputSelection == ["sqlite"])
conversation.toggleUserInputOption("postgres", in: "storage")
conversation.toggleUserInputOption("sqlite", in: "storage")
#expect(conversation.userInputDraft(for: "storage").selection == ["sqlite"])
// Clicking the chosen one again takes the answer back.
conversation.toggleUserInputOption("sqlite")
#expect(conversation.userInputSelection.isEmpty)
conversation.toggleUserInputOption("sqlite", in: "storage")
#expect(conversation.userInputDraft(for: "storage").selection.isEmpty)
}
/// When several answers are allowed the two routes are additive, not
@@ -131,30 +173,111 @@ struct AgentUserInputTests {
let conversation = ConversationModel()
conversation.present(request(multiple: true))
conversation.toggleUserInputOption("postgres")
conversation.toggleUserInputOption("sqlite")
conversation.setUserInputCustomAnswer("and DuckDB")
conversation.toggleUserInputOption("postgres", in: "storage")
conversation.toggleUserInputOption("sqlite", in: "storage")
conversation.setUserInputCustomAnswer("and DuckDB", for: "storage")
#expect(conversation.userInputSelection == ["postgres", "sqlite"])
#expect(conversation.userInputDraft == "and DuckDB")
#expect(conversation.userInputDraft(for: "storage").selection == ["postgres", "sqlite"])
#expect(conversation.userInputDraft(for: "storage").text == "and DuckDB")
}
// MARK: - Several questions at once
@Test
func eachQuestionKeepsItsOwnAnswer() {
let conversation = ConversationModel()
conversation.present(threeQuestions())
conversation.toggleUserInputOption("postgres", in: "storage")
conversation.toggleUserInputOption("flyway", in: "migrations")
conversation.setUserInputCustomAnswer("staging-2", for: "host")
#expect(conversation.userInputDraft(for: "storage").selection == ["postgres"])
#expect(conversation.userInputDraft(for: "migrations").selection == ["flyway"])
#expect(conversation.userInputDraft(for: "storage").text.isEmpty)
#expect(conversation.userInputDraft(for: "host").text == "staging-2")
#expect(conversation.answeredUserInputCount == 3)
}
/// Single- and multi-answer questions sit in the same form and keep their
/// own rule; the mutual exclusion of one must not reach into the other.
@Test
func exclusionRulesApplyPerQuestionNotPerForm() {
let conversation = ConversationModel()
conversation.present(threeQuestions())
conversation.toggleUserInputOption("flyway", in: "migrations")
conversation.toggleUserInputOption("liquibase", in: "migrations")
#expect(
conversation.userInputDraft(for: "migrations").selection == ["flyway", "liquibase"],
"migrations allows several"
)
conversation.toggleUserInputOption("postgres", in: "storage")
conversation.toggleUserInputOption("sqlite", in: "storage")
#expect(
conversation.userInputDraft(for: "storage").selection == ["sqlite"],
"storage allows one"
)
#expect(
conversation.userInputDraft(for: "migrations").selection == ["flyway", "liquibase"],
"and choosing in one question must not disturb another"
)
}
@Test
func oneAnswerIsEnoughToContinueButNoneIsNot() {
let conversation = ConversationModel()
conversation.present(threeQuestions())
#expect(!conversation.canSubmitUserInput, "answering nothing is Cancel, not Continue")
conversation.toggleUserInputOption("postgres", in: "storage")
#expect(conversation.canSubmitUserInput)
#expect(conversation.answeredUserInputCount == 1)
}
// MARK: - What the tool gets back
@Test
func answerReportsIdsInTheOrderTheRequestListedThem() async {
func skippedQuestionsAreAbsentFromTheResponse() async {
let conversation = ConversationModel()
let pending = request(multiple: true)
let answering = Task { await conversation.request(pending) }
let answering = Task { await conversation.request(threeQuestions()) }
await settle(conversation)
conversation.toggleUserInputOption("flyway", in: "migrations")
conversation.submitUserInput()
#expect(
await answering.value
== .answered([
AgentUserInputAnswer(
questionID: "migrations", value: .choice(selectedIDs: ["flyway"], custom: nil)
)
])
)
}
@Test
func answerReportsIdsInTheOrderTheQuestionListedThem() async {
let conversation = ConversationModel()
let answering = Task { await conversation.request(request(multiple: true)) }
await settle(conversation)
// Clicked back to front; the answer still reads front to back, because
// the model wrote the list and reads the reply against it.
conversation.toggleUserInputOption("duckdb")
conversation.toggleUserInputOption("postgres")
conversation.toggleUserInputOption("duckdb", in: "storage")
conversation.toggleUserInputOption("postgres", in: "storage")
conversation.submitUserInput()
#expect(await answering.value == .choice(selectedIDs: ["postgres", "duckdb"], custom: nil))
#expect(
await answering.value
== .answered([
AgentUserInputAnswer(
questionID: "storage",
value: .choice(selectedIDs: ["postgres", "duckdb"], custom: nil)
)
])
)
}
@Test
@@ -163,28 +286,36 @@ struct AgentUserInputTests {
let answering = Task { await conversation.request(request(multiple: false)) }
await settle(conversation)
conversation.setUserInputCustomAnswer(" MySQL ")
conversation.setUserInputCustomAnswer(" MySQL ", for: "storage")
conversation.submitUserInput()
// Trimmed, because trailing whitespace is not part of anyone's answer.
#expect(await answering.value == .choice(selectedIDs: [], custom: "MySQL"))
#expect(
await answering.value
== .answered([
AgentUserInputAnswer(
questionID: "storage", value: .choice(selectedIDs: [], custom: "MySQL")
)
])
)
}
@Test
func cancellingReportsCancelledRatherThanAnEmptyAnswer() async {
let conversation = ConversationModel()
let answering = Task { await conversation.request(request(multiple: false)) }
let answering = Task { await conversation.request(threeQuestions()) }
await settle(conversation)
conversation.toggleUserInputOption("postgres", in: "storage")
conversation.cancelUserInput()
#expect(await answering.value == .cancelled)
#expect(conversation.pendingUserInput == nil)
#expect(conversation.userInputSelection.isEmpty)
#expect(conversation.userInputDraft(for: "storage").selection.isEmpty)
}
@Test
func submittingNothingIsRefusedRatherThanSentAsAnEmptyChoice() async {
func submittingNothingIsRefusedRatherThanSentAsAnEmptyAnswerList() async {
let conversation = ConversationModel()
let answering = Task { await conversation.request(request(multiple: false)) }
await settle(conversation)
@@ -192,27 +323,135 @@ struct AgentUserInputTests {
conversation.submitUserInput()
#expect(conversation.pendingUserInput != nil, "the request must still be waiting")
conversation.toggleUserInputOption("sqlite")
conversation.toggleUserInputOption("sqlite", in: "storage")
conversation.submitUserInput()
#expect(await answering.value == .choice(selectedIDs: ["sqlite"], custom: nil))
#expect(
await answering.value
== .answered([
AgentUserInputAnswer(
questionID: "storage", value: .choice(selectedIDs: ["sqlite"], custom: nil)
)
])
)
}
@Test
func aFreeFormQuestionStillAnswersAsPlainText() async {
let conversation = ConversationModel()
let answering = Task {
await conversation.request(AgentUserInputRequest(prompt: "Base URL?", kind: .text))
}
let pending = AgentUserInputRequest(
questions: [AgentUserQuestion(id: "base_url", prompt: "Base URL?", kind: .text)]
)
let answering = Task { await conversation.request(pending) }
await settle(conversation)
conversation.userInputDraft = "https://api.example.com"
conversation.setUserInputCustomAnswer("https://api.example.com", for: "base_url")
conversation.submitUserInput()
#expect(await answering.value == .text("https://api.example.com"))
#expect(
await answering.value
== .answered([
AgentUserInputAnswer(
questionID: "base_url", value: .text("https://api.example.com")
)
])
)
}
@Test
func aSecretAnswersThroughItsOwnDraftAndNotTheSharedOne() async {
let conversation = ConversationModel()
let pending = AgentUserInputRequest(
questions: [AgentUserQuestion(id: "secret", prompt: "Sudo password?", kind: .secret)]
)
let answering = Task { await conversation.request(pending) }
await settle(conversation)
#expect(!conversation.canSubmitUserInput)
conversation.secretInputDraft = "hunter2"
#expect(conversation.canSubmitUserInput)
conversation.submitUserInput()
#expect(
await answering.value
== .answered([
AgentUserInputAnswer(questionID: "secret", value: .secret("hunter2"))
])
)
// Never left behind in the plain drafts, which are what the transcript
// and the next question read from.
#expect(conversation.userInputDraft(for: "secret").text.isEmpty)
#expect(conversation.secretInputDraft.isEmpty)
}
// MARK: - What the model reads
/// The skipped question still appears, carrying a null. Omitting it would
/// read as a bug; a null says plainly that this one is now the model's call.
@Test
func everyQuestionIsReportedWhetherAnsweredOrNot() {
let questions = threeQuestions().questions
let rows = AgentBuiltInTools.reported(
questions,
[
AgentUserInputAnswer(
questionID: "migrations",
value: .choice(selectedIDs: ["flyway", "liquibase"], custom: "or none")
),
AgentUserInputAnswer(questionID: "host", value: .text("staging-2")),
]
)
#expect(rows.count == 3)
// Order follows the questions, not the answers.
#expect(field(rows[0], "question_id") == .string("storage"))
#expect(field(rows[0], "answer") == .null)
#expect(field(rows[0], "selected_option_ids") == .array([]))
#expect(field(rows[0], "custom_text") == .null)
// A typed answer wins the canonical `answer`, and the ids still travel.
#expect(field(rows[1], "answer") == .string("or none"))
#expect(
field(rows[1], "selected_option_ids")
== .array([.string("flyway"), .string("liquibase")])
)
#expect(field(rows[2], "answer") == .string("staging-2"))
}
/// With no typed answer the canonical `answer` is the labels, not the ids —
/// a model that reads one field should read something a human wrote.
@Test
func selectedLabelsBecomeTheCanonicalAnswer() {
let rows = AgentBuiltInTools.reported(
[storage(multiple: true)],
[
AgentUserInputAnswer(
questionID: "storage",
value: .choice(selectedIDs: ["postgres", "duckdb"], custom: nil)
)
]
)
#expect(field(rows[0], "answer") == .string("Postgres, DuckDB"))
#expect(field(rows[0], "custom_text") == .null)
}
/// Unreachable through `request_user_input`, which never builds a secret
/// question — but if one ever arrives, the plaintext must not be echoed.
@Test
func aSecretIsNeverReportedBackToTheModel() {
let rows = AgentBuiltInTools.reported(
[AgentUserQuestion(id: "secret", prompt: "Password?", kind: .secret)],
[AgentUserInputAnswer(questionID: "secret", value: .secret("hunter2"))]
)
#expect(field(rows[0], "answer") == .null)
#expect(field(rows[0], "custom_text") == .null)
}
// MARK: - Helpers
private func field(_ row: AgentJSONValue, _ key: String) -> AgentJSONValue? {
row.objectValue?[key]
}
private func arguments(_ object: [String: AgentJSONValue]) throws -> Arguments {
try Arguments(
AgentToolInvocation(
@@ -225,8 +464,31 @@ struct AgentUserInputTests {
}
private func request(multiple: Bool) -> AgentUserInputRequest {
AgentUserInputRequest(
prompt: "Which database?",
AgentUserInputRequest(questions: [storage(multiple: multiple)])
}
private func threeQuestions() -> AgentUserInputRequest {
AgentUserInputRequest(questions: [
storage(multiple: false),
AgentUserQuestion(
id: "migrations", header: "Migrations", prompt: "Which tools may we use?",
kind: .choice(
AgentUserInputChoice(
options: [
AgentUserInputOption(id: "flyway", label: "Flyway"),
AgentUserInputOption(id: "liquibase", label: "Liquibase"),
],
allowsMultipleAnswers: true
)
)
),
AgentUserQuestion(id: "host", prompt: "Which host is staging?", kind: .text),
])
}
private func storage(multiple: Bool) -> AgentUserQuestion {
AgentUserQuestion(
id: "storage", header: "Storage", prompt: "Which database?",
kind: .choice(
AgentUserInputChoice(
options: [
@@ -239,8 +501,7 @@ struct AgentUserInputTests {
],
allowsMultipleAnswers: multiple
)
),
header: "Storage"
)
)
}