mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-08 15:04:33 +00:00
c8b3d5fc8df5d3afe67b522ca10357ea73d8be8c
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Daniel Salazar
|
631b264c1c |
test(ai): every AI driver holds and meters at the AI cost factor (#4304)
Image, video, TTS, speech-to-text and voice-changer driver tests run one request at factor 1 and again with every `ai.cost.factor.*` hook set to 2, and check that both the credit hold and the recorded cost double. Chat and OCR already had this coverage. A driver that hands a provider the raw metering service instead of its factored view now fails its test. `withAiCostFactorHook` in integrationTestUtil keeps the hook in place for a callback, since providers record their usage unawaited. |
||
|
Daniel Salazar
|
d8f9aff3dd |
fix: follow-ups to #4161–#4203 (rate limits, broadcast, AI routing and billing, fs subtree walks) (#4205)
* fix(rate-limit): count the newest KV rate rows first The KV backend's rate check, peek and concurrent acquire list the key's rows with a limit + 1 cap that the store applies before dropping expired rows, so unswept rows from past windows (oldest keys) could fill the fetch and the live count read as zero. They now list newest first. * fix(broadcast): split webhook flushes under a byte budget, per-peer backlogs Flushes go out in bodies of at most `broadcast.outboundMaxBodyBytes` (default 4 MiB) instead of one POST, and each peer keeps its own backlog of what it missed, capped by count and four bodies' worth of bytes, oldest dropped first, so one failing peer can no longer grow every peer's body past their 50 MB limit. * fix(ai-chat): classify dropped upstream connections as provider failures A connection reset or refusal (SDK APIConnectionError, Mistral's ConnectionError, an undici reset on the cause) carries no status, so a chain of them fell through to 500 internal_error. Attempts now record connectionFailed; a chain of only resets and 5xx answers 400 upstream_provider_unavailable, and a reset counts as an upstream signal in a mixed chain. * fix(ai-chat): keep SDK retries on Anthropic count_tokens The Claude client is built with SDK retries off so the driver can fall back, but count_tokens has no fallback route. It now passes sdkRetryOptions, like the OpenAI moderation call, so a transient 429 or reset is retried. * fix(ai-chat): mark a route unhealthy for 60 s after a transient failure With SDK retries off, one 5xx, 429, timeout or dropped connection kept a route marked for 10 minutes, sending that model's traffic to resellers. Those failures now mark the route for 60 s; bad credentials, an exhausted account and other route-level failures keep 10 minutes. * fix(metering): don't memoise spent addons inside the force-exact window The spent-addons memo could cache a pre-grant record read from a lagging replica right after a credits-changed announcement, so overage skipped the new credit and requireCredits kept answering 402 for up to 15 s. A spent record is now only cached once the actor's force-exact window has ended. * fix(share): send the digest entries a flush claimed when another take fails Digest entries are taken concurrently, and a single rejected take threw away every entry the other takes had already removed from KV. The flush now logs the failed takes and sends what it claimed; an entry whose take failed stays queued for a later flush. * fix(fs): announce an overwriting move's replaced entry only when it took its name /move and v2 /fs/move looked up the entry at the destination name before moving and always announced its removal. A move into someone else's Trash renames instead of replacing, so the owner's desktop hid a live entry and legacy /move reported it as `overwritten`. Both routes now pass the target on only when the moved entry kept the name it was looked up by. * fix(ai): name the vendor in upstream error fields from media providers upstreamFetch now takes the provider id plus a display label, and UpstreamResponseError carries the id. translateProviderError copies a string `provider` off the error into `fields`, and the BytePlus video and Cloudflare image errors set one, so upstream_auth_failed and upstream_credits_exhausted alarms say which vendor failed again. * fix(teams): delete a provisioned seat whose home folders could not be created provisionUser logs a failed default-folder creation and still returns the account, so provisionAccount went on to add the membership, issue a password, mail the seat and emit team.account.created for an account with no home. It now cascade-deletes the new user and returns 500 internal_error before any of that. * fix(ai-chat): load model catalogs without holding up server start onServerStart awaited every gateway catalog (up to its 10s fetch timeout) while HTTP was already listening, so drivers started after it had no providers yet and chat requests found an empty model map. The first build now runs in the background; complete(), models() and token counting wait on it instead of failing. * fix(ai-chat): serve direct models while gateway catalogs load at boot The first model-map build now publishes a fresh map as each provider's catalog lands, so static catalogs are routable at once. complete() and token counting wait for the full first build only when the model, or the provider asked for, isn't in the map yet. Refreshes still swap once. * fix(ai-chat): split gateway-billed cost across input and output A completion billed at the cost OpenRouter, Infron or Neuralwatt reports was quoted to `ai.prompt.cost-calculated` as all input and no output. The ledger still records one `billedUsage` line; the reported per-key costs now spread that bill over the usage keys at the catalog's relative rates, in whole µ¢ that sum to it. Neuralwatt energy units take no share. * fix(ai-image): hold Replicate credits for the whole prediction The output and input-surcharge holds took the default 10-minute TTL, the same as the prediction window alone, so a slow prediction could outlive them once creation and input measurement were added. They now last 15 minutes, like the video job holds. * feat(ai-chat): report what a completion was charged as usage.cost Chat usage (the non-stream result and the stream's final usage line) now carries `cost`: the µ¢ the ledger records for the request after the cost factor, from the request's recorded charges and the gate's factor lookup. `usd_cents` is unchanged. A hooked recording now copies the usage object first, so keys the driver adds later never land in the ledger. * fix(ai): charge streamed speech once its first byte arrives Per-character TTS and the voice changer recorded their charge when the upstream's response headers arrived, so a body that failed before any audio was billed in full. A shared relay now charges on the first byte, or when the caller hangs up, and charges nothing for an upstream that fails or ends before sending audio. * fix(ai): release credit holds only after the request's usage lands A hold was released as soon as its operation returned, while the usage it stood for was still being written (behind the cost-factor lookup), so a concurrent request briefly saw neither. AI metering now tracks the request's usage writes and its holds (chat included) release once those settle, without delaying the response; a streamed charge chains its own. A failed write still releases the hold, and the hold's TTL backstops it. * test(ai-chat): wait for a stream's credit hold to come back after its usage lands * perf(fs): page subtree delete and copy by parent link, not by depth over the path range Each page of a recursive delete or copy sorted the whole remaining subtree by a computed depth, so a walk cost grew with the square of its size. It now pages the children of up to 100 sibling directories at a time by parent_uid, depth first, keyset on (parent_uid, id), so a page reads only its own rows; rows with no stored path are reached too. SQLite gets the parent_uid index the others have. * fix(fs): find children by parent_uid in the empty check and supplementary-name lookup Rows the old backend wrote carry parent_uid and no parent_id, so a non-recursive delete of a directory holding only such rows passed the empty check and orphaned them, and on MySQL the supplementary-character child lookup missed them, leaving their subtrees out of listings, counts and renames. Both now key on parent_uid; the empty check still probes parent_id too, which the delete cascade follows. * docs(fs): drop the single-POST note from the entry broadcast cap |
||
|
Daniel Salazar
|
f3134b2530 |
refactor(ai-media): shared upstream fetch, provider registry and bounded inputs for AI media drivers (#4188)
* refactor(ai): route TTS, STT and voice-changer REST calls through upstreamFetch upstreamFetch bounds a provider call with AbortSignal.timeout and throws an UpstreamResponseError carrying the upstream status and code, which the /drivers/call boundary maps like any SDK error. Replaces five hand-written status translators (ElevenLabs TTS, xAI TTS, Speechify, xAI STT, voice changer); caller-facing messages are now sanitized by the shared mapper. * perf(ai): bound input image fetches and share one data-URI parser fetchImageBytes now streams under a byte cap (30 MB by default) with a 30-second timeout, via a readBoundedBody helper shared with loadFileInput's web branch and Replicate's input measuring. assertInputImagesShape caps input_images at 16. Chat image inlining uses fetchImageBytes instead of buffering with no timeout. fileInput, imageInput and mediaParts parse data URIs through one parseDataUri. * refactor(ai-tts): one per-character metering path for TTS providers meterPerCharacter holds text length x price, runs the synthesis and meters the same amount once it succeeds; characterCostReport builds the per-character cost lines. Both sit on a new withAiCreditHold (reserve, run, release) that Gemini TTS uses directly. SAMPLE_AUDIO_URL is defined once. The TTSProvider base class is gone: its methods were stubs every provider overrode, and only Polly read its untyped providerConfig. * refactor(ai): one ProviderRegistry and key reader for the AI media drivers ProviderRegistry holds a driver's provider catalog (ids, default, aliases) and its configured instances: alias resolution, the not-found 400, default fallback and parallel 'all' listing. TTS, speech-to-text, OCR and the voice changer route through it. readProviderKey reads apiKey, secret_key, api_key and key in that order for every AI media driver, so providers.xai.secret_key now enables xAI TTS as it already did xAI image and speech-to-text. ElevenLabs caches its voice list for 10 minutes, as Polly does. * perf(ai): stream TTS and voice-changer audio instead of buffering it OpenAI, xAI and ElevenLabs TTS and the voice changer hand the upstream body straight to the caller through upstreamBodyStream; their costs are known before the call, so nothing waits on the download. A body that fails mid-stream ends the response early instead of raising an unhandled stream error. xAI speech-to-text keeps one copy of the audio (the upload Blob) instead of up to three. The voice changer and xAI STT hold credits through withAiCreditHold. * perf(ai-ocr): read Puter files only when the provider needs the bytes resolveFileInput resolves and access-checks a Puter file and defers the read to load(); loadFileInput is that plus the read. OCR resolves first: Textract reads the object straight from S3 and downloads it only if Textract can't reach the bucket, and only Mistral loads the bytes. A file over the size cap is refused from its recorded size before any read. countPdfPages inflates object streams on the libuv pool instead of the request thread. * perf(ai): build image and video model catalogs once and share output saving ModelCatalog holds what the public model routes serve (models, sorted names, cost lines), computed once after boot instead of on every listing. The video driver now awaits its model map before serving. resolveOutputPath and saveGeneratedMediaToFS replace the copies of puter_output_path handling in both drivers, using expandTildePath and normalizeAbsolutePath. The video driver normalizes a copy of the caller's args instead of mutating them, and its never-taken stream branch is gone. * fix(drivers): abort the response when a streamed driver result fails mid-body upstreamBodyStream now errors when the provider body fails instead of ending cleanly, and DriverController pipes stream results through pipeStreamResult (stream.pipeline): a failed source destroys the response so the client sees an aborted transfer, the failure is logged once, and nothing is left to raise an unhandled stream error. A client hanging up stops the source. * fix(drivers): drop a streamed driver result when the caller already hung up pipeline() throws synchronously on a closed destination, which surfaced as a 500 alarm on every early disconnect and left the upstream body open. * fix(ai): bound streamed upstream calls by time to first byte, not download time upstreamFetch takes a `stream` option: the timeout covers the request until the response (or a non-2xx error body) is read, then is cleared so a body piped to the client drains at the client's pace. ElevenLabs TTS, xAI TTS and the voice changer pass it; non-streamed calls keep the full-response bound. A stalled upstream body is still cut by undici's body idle timeout. * fix(drivers): type pipeStreamResult's response by the members it uses |
||
|
Daniel Salazar
|
aef0a32d72 |
fix(ai): hold credits for image and video generation (#4164)
Image and video providers checked the balance with hasEnoughCredits / getRemainingUsage and took no hold, so concurrent requests each saw the full balance until the first one recorded usage. They now reserve the estimate through the cost-factored reserveAiCredits and release it in a finally block once usage is recorded, like chat, speech and OCR. The video duration clamp now prices seconds at the model's cost factor and holds the clamped clip's cost for the job (poll window plus create time). Replicate holds the output-only estimate before fetching inputs and the input surcharge on top once it is measured. |
||
|
404oopsandClaude Opus 5.5
|
521812be02 |
refactor(ai): move the AI cost factor out of MeteringService
MeteringService knew about AI: it built the cost-factor facade, resolved the `ai.cost.factor.*` hook and, since the previous commit, offered `reserveAiCredits`. Metering stays agnostic to what it meters, so all of that now lives with the AI drivers in `drivers/util/aiCostFactor.ts`. `withAiCostFactor(metering, events, driver)` returns an `AiMeteringService`: the service with recording scaled by the factor, plus `costFactor` and `reserveAiCredits` for gates. The speech providers take that type. MeteringService itself has no AI methods left. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Filip KujundžićandClaude Opus 5.5
|
abd0d809d5 |
fix(ai): price speech credit gates at the charged cost and hold them in flight
Text-to-speech, speech-to-text and voice-changer providers checked affordability against the unscaled provider cost and never reserved it, so the gate ran low by the AI cost factor and concurrent requests could each spend the whole balance. They now go through `reserveAiCredits`, which scales the amount by the model's cost factor, checks it, and holds it until the request finishes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
922d203e18 |
fix(ai): send stable, non-sequential user identifiers to AI providers (#3856)
* fix(ai): send stable, non-sequential user identifiers to AI providers A precedence bug in the AI providers' identifier expression made every request send `user: ":undefined"` (the ternary bound the app-uid suffix to the whole `actor.user.id + actor.app?.uid` sum instead of just the suffix), or read `actor.user.id` on a missing user. The same expression also shipped the sequential internal user id, letting AI vendors correlate a single account across apps and sessions. All eight OpenAI-, Azure-, xAI-, Meta- and ZAI-style providers now build the identifier through one shared helper, `aiUserIdentifier()`: - `puter-<user-uuid>[-<app-token>]`: the random user UUID is always preserved in full; `maxLength` constrains only the app-bearing form - app attribution reads `effectiveApp`, so access-token requests name the issuing app instead of looking like direct user traffic - the app token is truncated to fit the budget, and omitted entirely when the remaining budget is below 8 chars, where a truncation could collide with another app's uid - nothing is sent for the system actor - Meta and ZAI keep a caller-supplied `safety_identifier` / `user_id` override, applied before the helper result `user` is deprecated by OpenAI; the SDK types direct callers to `safety_identifier` (abuse detection) and `prompt_cache_key` (cache-hit bucketing). The four OpenAI/Azure chat providers and MetaProvider now send `prompt_cache_key` as well, defaulting it to the same per-user identifier unless the caller supplies one; Azure's Grok branch drops both fields, matching its rejection of unknown args. The cap comment cites only verified limits: OpenAI's 64 for `safety_identifier` (from the SDK types) and Z.AI's 6-128 for `user_id` (from Z.AI's docs); Meta and xAI document none, so none is claimed. The xAI image `#edit` path now carries the identifier like generation, and takes a named-options param so `user` cannot be transposed with the adjacent same-typed `aspectRatio`. Tests share a four-actor matrix (`user` / `user+app` / `access token` / `system`) with `assertActorMatrixIdentifiers()` across the six OpenAI-style suites; the helper has exact-string and boundary coverage (size caps, zero-budget and sub-base cases, no dangling separator, UUID never truncated, collision guard); the Azure Grok assertions run under a real user actor so they cannot pass vacuously. 212 provider-suite tests pass; typecheck and ESLint are clean. * fix(ai): lock the vendor identifier down and keep vitest out of the test util Meta and Z.AI no longer let `custom` override the abuse identifier; it is Puter's attribution, not the caller's. The shared test util exposes pure field pickers instead of importing vitest into a file the production tsconfig compiles. The helper's length-cap comment now matches vendor docs (Meta does cap `safety_identifier` at 64), the redundant budget branch and the unused export are gone, and the per-user `prompt_cache_key` trade-off is stated once in the helper instead of five times in providers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: 404oops <me@404oops.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Daniel Salazar
|
116d6e6663 | tests: big test push for better coverage (#3490) | ||
|
Daniel Salazar
|
3ac6c8532e | fix: open ai cost (#3488) | ||
|
Daniel Salazar
|
8d5e9ef9cf |
fix: types for open ai driver (#2950)
* fix: types for open ai driver * fix: add extra time to tests * fix: test * fix: together ai tests * fix: tests |
||
|
Daniel Salazar
|
25fe9a3cdf |
fix: add coverage reports for changes (#2949)
* tests: ai integraton tests * fix: add coverage reports for this too |