* gateway: reasoning off goes to Gemini 3 at its lowest level where it has no minimal, and Google refusing minimal no longer stops the provider's thoughts until restart
With reasoning off, a request to Gemini's OpenAI-compatible API (AI
Studio's, or a proxy in front of it on this machine or the LAN) went with
reasoning_effort minimal, since Gemini 3 can't stop thinking.
gemini-3.8-flash has no minimal: models.dev gives its levels as low,
medium and high, and Google answers minimal with 400 "Thinking level is
unsupported: THINKING_LEVEL_MINIMAL". The message names thinking, so
refusesThinkingConfig took it for thinking_config turned away and marked
that field unfit for the provider. Until magpie restarted, nothing sent to
the provider asked for Gemini's thoughts again, on any model, and a client
that asked to see them got none. The turn itself was asked again with no
reasoning_effort, at the model's default level.
* Reasoning off goes at the model's lowest level where its levels are
known and have no minimal, and at minimal where they have it or aren't
known (Request.OffLevel, geminiOffLevel). gemini-3.8-flash on AI
Studio goes at low, in one request.
* Where the levels aren't known (a proxy's own name for the model),
Google's 400 for minimal is told apart from thinking_config turned
away (geminiMinimalRefused, which also matches gemini-3.1-pro-preview's
"thinking_level MINIMAL is not supported by this model" at
generateContent). It is remembered as the model refusing reasoning off
(offRefused, as for other vendors), and the model is asked again at its
lowest level, low when none are known, and so from then on.
thinking_config is kept, and the next turn that thinks gets the
model's thoughts.
* An effort of minimal set on the request, which fitFor keeps where the
levels aren't known, is turned away the same way and asked again at
low by the existing offEffort step, with its thinking_config.
Passthrough (a Chat client calling AI Studio itself) is unchanged: it fits
the effort to the model's levels where they're known, and has no retry for
a proxy whose levels aren't.
Tests, with what upstream's gateway gives:
* TestGeminiThinkingOffAtItsLowest: an Anthropic client's turn with
thinking disabled, then one thinking at high, then disabled again, to
a fake Gemini that answers minimal with the 400 above, in its literal
bytes. With the levels known, the first turn goes at [low]; not known,
at [minimal low]. Either way the thinking turn then sends
thinking_config {include_thoughts: true, thinking_level: high} and
gets a thinking block, and the last turn goes at low in one request.
Upstream sends [minimal <nil>] both ways.
* TestAIStudioThinking: "off at its lowest" sends reasoning_effort low.
Upstream sends minimal.
Each part was also broken on its own, and each break fails a test:
buildChat ignoring OffLevel ("reasoning_effort = "minimal", want "low"",
and [minimal minimal <nil>]), the loop never setting it ([minimal minimal
<nil>] both ways), no check for the refusal of minimal ([minimal <nil>]
with the levels not known), and geminiOffLevel ignoring the refusal
([minimal minimal <nil>]) or the known levels ([minimal low]).
Verified on 443f1cbc: gofmt -l, go vet ./..., go build, GOOS=linux go
build -tags nogui and GOOS=windows go build pass; go test -tags nogui
./... passes, and -race -count=20 on the tests above.
Live, with gemini-3.8-flash through a proxy on this machine in front of
Vertex AI's OpenAI-compatible API, the gateway in a temporary HOME (AI
Studio itself wasn't tried: there was no key for it). Upstream sent the
off turn at minimal, got the 400 and asked again with no
reasoning_effort; the thinking turn after it went as reasoning_effort high
with no thinking_config, and no thoughts came back. With this change the
off turn went at low in one request, the thinking turn with thinking_config
at high, and the last off turn at low. Under a name whose levels aren't
known, the first off turn went minimal, then low, and the next at low at
once. The thoughts reached the client as thinking with the next commit,
which reads how that endpoint marks them.
* gateway: Gemini's thoughts streamed from Vertex AI's OpenAI-compatible API reach the client as thinking, not as the start of the answer
A proxy on this machine or the LAN in front of Gemini is asked for the
model's thoughts as AI Studio is, in thinking_config. In front of Vertex
AI's OpenAI-compatible API, the thoughts don't come in AI Studio's
<thought> tags: each comes in a chunk of its own, the thought as the
delta's content, and the chunk is marked
"extra_content": {"google": {"thought": true}}. chatDecoder read only the
content, so the thoughts were streamed to the client as the first part of
the answer.
* A chunk marked as a thought is thinking (KThink).
A whole reply from that API carries its thoughts in <think> tags instead.
That isn't changed: the gateway streams from the upstream when it
translates, and a Chat client's own request is passed through as it is.
Tests, with what upstream's gateway gives:
* TestGeminiThoughtsMarkedInExtraContent: that API's stream for "你好"
at high with the thoughts asked for, in its literal bytes
(testdata/vertex_openai_thoughts.sse), to an Anthropic client: a
thinking block that starts "**Acknowledging Greeting**", then the
text "你好！很高兴与你交流。请问有什么我可以帮你的吗？". Upstream gives
thinking "" and the thought at the head of the text.
Verified on 443f1cbc: gofmt -l, go vet ./..., go build, GOOS=linux go
build -tags nogui and GOOS=windows go build pass; go test -tags nogui
./... passes, and -race -count=20 on the tests of this and the previous
commit.
Live, through the proxy and temporary HOME of the previous commit, a turn
thinking at high gave a thinking block of 238 characters, and the answer
apart from it.
forever
bed306ae60