mirror of
https://github.com/HeyPuter/puter.git
synced 2026-09-12 08:15:58 +00:00
fix(ai): route Mistral streamed thinking to the reasoning channel, unconditionally
Third triple-check round. Ten findings were put through independent skeptics first; six did not survive — pre-existing on main, inert, or resting on a false premise — and are not acted on here. Mistral streamed thinking now goes to `reasoning` on every path. The previous commit gated the split, which made this the only place in the repo where chain-of-thought reached the visible text channel, and made Mistral the only provider whose streamed chunk *types* depend on a response-format flag. Every other reasoning path routes thinking to `reasoning` unconditionally — ClaudeProvider's thinking_delta, the DeepSeek/OpenRouter rename, and this branch's own Responses summary-delta handler. Removing the gate restores that uniformity and makes the documented promise that streaming is unaffected by normalization true again; the two opposing tests collapse into one that runs the same fixture with and without `normalize` and asserts identical event streams. Docs stop claiming older models are unchanged. Four reasoning fields were made consistent across all models, ungated, and one of them removes a field: on non-streaming responses `message.reasoning_content` is now `message.reasoning`. chat.md gains a table naming all four so a caller reading `reasoning_content` learns why it disappeared, instead of reading that nothing changed for them. Adds the driver-level fallback test. Writing it surfaced that the invariant it was meant to assert is false and always was: the driver rewrites string `content` into text blocks in place on the caller's own messages (`normalize_single_message`, pre-existing) before any provider runs. The test now asserts what is true and load-bearing — both attempts receive the same array reference, and the reasoning artifacts survive attempt 1 so attempt 2 can still replay them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+16
-3
@@ -35,7 +35,7 @@ An object containing the following properties:
|
||||
- `tools` (Array) (Optional) - Function definitions the AI can call. See [Function Calling](#function-calling) for details.
|
||||
- `reasoning_effort` / `reasoning.effort` (String) (Optional) - Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Lower values give faster responses with less reasoning. OpenAI models and Meta's Muse Spark models only; Muse Spark always reasons, so `none` is ignored for it.
|
||||
- `verbosity` / `text.verbosity` (String) (Optional) - Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only.
|
||||
- `normalize` (Boolean) (Optional) - Controls the format of the non-streaming response. When `true`, the response is normalized to the OpenAI format regardless of the model's vendor: `message.content` is a string, tool calls appear as `message.tool_calls`, and `finish_reason` is one of `stop`, `length`, `tool_calls`, or `content_filter` — or the vendor's own stop reason, passed through unchanged when it has no OpenAI equivalent. When `false`, the response keeps the vendor's native format (for Anthropic models, an array of content blocks). When unset, `puter.ai.normalize` applies if you assigned it; otherwise **models released on or after September 1, 2026 return normalized (OpenAI-format) responses by default**, and older models keep their current behavior. Streaming responses are unaffected — chunks already share one format across vendors. See [Response normalization](#response-normalization).
|
||||
- `normalize` (Boolean) (Optional) - Controls the format of the non-streaming response. When `true`, the response is normalized to the OpenAI format regardless of the model's vendor: `message.content` is a string, tool calls appear as `message.tool_calls`, and `finish_reason` is one of `stop`, `length`, `tool_calls`, or `content_filter` — or the vendor's own stop reason, passed through unchanged when it has no OpenAI equivalent. When `false`, the response keeps the vendor's native format (for Anthropic models, an array of content blocks). When unset, `puter.ai.normalize` applies if you assigned it; otherwise **models released on or after September 1, 2026 return normalized (OpenAI-format) responses by default**, and for older models the default is unchanged — `message.content` keeps its vendor-native shape. (A handful of reasoning fields were made consistent across all models independently of this option; see [Reasoning fields on existing models](#reasoning-fields-on-existing-models).) Streaming responses are unaffected — chunks already share one format across vendors. See [Response normalization](#response-normalization).
|
||||
- `compaction` (Boolean | Object) (Optional) - Opt into inline context compaction for long conversations. Pass `true` to enable it with provider defaults, or `{ trigger_tokens: number }` to set the token threshold at which earlier context is summarized. When the model compacts, you receive a `compaction` chunk while streaming (or a `compaction` field on the result when not streaming) containing an opaque `encrypted_content` summary. Resend that item in `messages` on the next turn in place of the summarized history. The compaction chunk shape is identical across providers, so the same code works whether `model` is an OpenAI or Anthropic model. See [Compaction](#compaction).
|
||||
|
||||
#### `testMode` (Boolean) (Optional)
|
||||
@@ -119,7 +119,20 @@ We use different vendors for different models and try to use the best vendor ava
|
||||
|
||||
Most vendors respond in the OpenAI chat format, where `message.content` is a string and tool calls appear as `message.tool_calls`. Anthropic models historically respond in Anthropic's native format instead, where `message.content` is an array of content blocks such as `[{ type: "text", text: "..." }]`.
|
||||
|
||||
**Going forward, all models released on or after September 1, 2026 return responses in the OpenAI format**, no matter which vendor serves them — so the same response-handling code works across every new model. Models released before that date keep their historical behavior unless you opt in.
|
||||
**Going forward, all models released on or after September 1, 2026 return responses in the OpenAI format**, no matter which vendor serves them — so the same response-handling code works across every new model. For models released before that date, the `normalize` default does not change: leave the option unset and `message.content` keeps its vendor-native shape.
|
||||
|
||||
### Reasoning fields on existing models
|
||||
|
||||
Separately from the `normalize` default, four reasoning-related fields were made consistent across vendors. These apply to **every** model, including ones released before the cutoff, and are not affected by `normalize`:
|
||||
|
||||
| Field | Before | Now |
|
||||
| --- | --- | --- |
|
||||
| `message.reasoning_content` | Present on providers following the DeepSeek convention (DeepSeek, OpenRouter and others) | **Renamed to `message.reasoning`.** Read `reasoning` instead — `reasoning_content` is no longer present on non-streaming responses. |
|
||||
| `message.reasoning` on OpenAI Responses models | Always present as `null` | Absent when the model returned no reasoning summary; a string when it did. `if (msg.reasoning)` is unaffected; `'reasoning' in msg` changes. |
|
||||
| `message.reasoning_details` on OpenAI Responses models | Not present | Present when the model returned reasoning items, carrying their `id` and `encrypted_content` for replay. |
|
||||
| `finish_reason` on OpenAI Responses models | Always `"stop"` | `"tool_calls"` when the turn ended in tool calls, `"stop"` otherwise. |
|
||||
|
||||
If your code reads `message.reasoning_content` on a non-streaming response, that is the one change that removes a field — switch to `message.reasoning`.
|
||||
|
||||
You can control this per call with the `normalize` option:
|
||||
|
||||
@@ -150,7 +163,7 @@ One caveat. The release-date rule applies to the model that actually serves the
|
||||
|
||||
One thing to know about the release-date rule: a model's release date comes from the catalog of whichever provider serves it, and some providers report it from their own live listing. Models served through OpenRouter carry the date OpenRouter itself assigns, so a model newly listed there on or after September 1, 2026 is normalized by default without Puter shipping any change. Pin `normalize: false` if your code depends on a provider's native shape.
|
||||
|
||||
Streaming is unaffected by normalization: streamed [`ChatResponseChunk`](/Objects/chatresponsechunk) objects already share one format across all vendors.
|
||||
Streaming is unaffected by normalization: streamed [`ChatResponseChunk`](/Objects/chatresponsechunk) objects already share one format across all vendors, and the chunk types a model emits do not depend on the `normalize` option. Reasoning models stream their thinking as `reasoning` chunks on every provider, whether or not normalization applies.
|
||||
|
||||
## Function Calling
|
||||
|
||||
|
||||
Reference in New Issue
Block a user