mirror of
https://github.com/HeyPuter/puter.git
synced 2026-09-13 08:45:45 +00:00
404oopsandClaude Fable 5.1
d89dc347a2
OpenRouter and Together reject a request whose prompt plus max_tokens overflows the model's context window. Both providers retried by deleting max_tokens, which threw away the output cap the credit gate had sized to the caller's remaining balance and let the retry run to the model's full output limit with no second gate and no new hold. The retry now goes through a shared helper that sizes a new cap from the window and input count the rejection reports, falling back to the model's declared context and a doubled prompt estimate, and never exceeds the cap the gate set. When no window can be determined or no output fits, the original rejection is rethrown instead of retrying uncapped. The rejected params are copied rather than mutated, so the first attempt's record is not rewritten after the fact. Also corrects the estimator's own comment, which described the mean of two approximations as a deliberate halving. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
39 lines
1.5 KiB
TypeScript
39 lines
1.5 KiB
TypeScript
/*
|
|
* Copyright (C) 2024-present Puter Technologies Inc.
|
|
*
|
|
* This file is part of Puter.
|
|
*
|
|
* Puter is free software: you can redistribute it and/or modify
|
|
* it under the terms of the GNU Affero General Public License as published
|
|
* by the Free Software Foundation, either version 3 of the License, or
|
|
* (at your option) any later version.
|
|
*
|
|
* This program is distributed in the hope that it will be useful,
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
* GNU Affero General Public License for more details.
|
|
*
|
|
* You should have received a copy of the GNU Affero General Public License
|
|
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
*/
|
|
|
|
/**
|
|
* Tokens a run of text is worth.
|
|
*
|
|
* The v1 estimator: the mean of two cheap approximations — characters over
|
|
* four, words times four thirds. On prose the two agree and the mean lands
|
|
* within a few percent of the real count; on text with few whitespace
|
|
* boundaries the word term collapses and the mean runs about half low on JSON
|
|
* and code, and a fraction of the real count on base64, logs, or CJK prose.
|
|
* Shared by the chat credit gate, the chat unreported-stream backstop, and
|
|
* image-prompt pricing, so a calibration moves all of them at once.
|
|
*
|
|
* @see https://help.openai.com/en/articles/4936856
|
|
*/
|
|
export const estimateTextTokens = (text: string): number => {
|
|
if (!text) return 0;
|
|
return Math.floor(
|
|
(text.length / 4 + text.split(/\s+/).length * (4 / 3)) / 2,
|
|
);
|
|
};
|