108 Commits

Include renames

Author	SHA1	Message	Date
404oopsandClaude Opus 5.5	2425c7a30d	test(puter-js): fail credential lookups until the relay hits one ... The reconnect test made the next credential lookup fail once, but the SDK's own socket.io connection could take that failure before the relay's reconnect attempt did, so the relay never saw a failed lookup and the test failed in CI. Lookups now fail until the relay has refused an attempt on one, then recover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 00:45:48 -07:00
404oopsandClaude Opus 5.5	3a35ce39c0	fix(realtime): tell transient server failures apart from bad credentials ... A credential lookup that threw (a database or cache outage) was treated as no actor, so the session got `invalid_api_key` and a 1008 close. That is terminal for the SDK, which then ended an established conversation that only needed to reconnect. A failed lookup now gets `internal_error` and closes with 1011, which the SDK retries; missing, invalid and revoked credentials keep `invalid_api_key` and 1008. Nothing is cached for the failed lookup, and the cause is logged server-side only. A session whose usage records pile up behind metering now closes with 1013 instead of 1008, still with `rate_limit_exceeded`. Queued usage is still recorded and its holds released. An unexpected setup failure now sends `internal_error` before its 1011 close. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 00:45:48 -07:00
Daniel Salazar	23782dec12	feat(events): inline handlers in the user's own scope for onSchedule and onPersistent (#4487) ... * feat(events): schema for user-scope handlers Add event_user_handlers and event_subscriptions.handler_scope in all three dialects. event_handlers is unchanged. * feat(events): scope the handler and durable subscription stores Handler store methods take a scope (app, or a user's scope of an app) and read the matching table. App-scope behavior, SQL and cache keys are unchanged; user scopes get their own cache keys, a 20-handler cap, and the inline @<sha256> name. Adds user-scope listing, counting, deletion and unbound-handler reads. Durable subscriptions store and return handlerScope, and listByHandler filters by scope and holder. * fix(events): hold the user-scope handler cap under concurrency and drop all scope caches on app delete A create that lands past the cap removes itself and is refused. deleteForApp drops each holder's cached set in batches, here and on peers. Batch scans over user scopes and unbound handlers take limit and cursor. * feat(events): set a CPU limit on events worker deploys * test(events): cover redeploy and resend of an evicted events script * test(events): name the deploy-on-miss test for what it checks * fix(events): keep a deploy-on-miss call inside the delivery lease * feat(workers): cap rented units per plan and refuse only new ones * feat(events): publish handlers into a user scope, with inline create * test(events): cover user-scope publish, inline create and listing * test(workers): name the credentials gate without a vendor * feat(events): run user-scope handlers as the holder A user scope deploys one script per handler under the holder's own identity, throttled per scope, with an eager deploy after publish that skips an upload another holder already made. Delivery and schedule runs address the holder's handler, carry the holder on the invoke headers, batch calls, the miss handler and the rehydrate route, and stay owed while the flag is off. Suspensions of a user scope notify the holder, teardown (app delete, consent revoke, account delete) drops the scope, and a job deletes unbound handlers after a day. * test(events): cover user-scope deploy, address, notices, teardown and the sweep Runs an inline user-scope handler through the local worker runtime as its holder, and covers the holder on the wire, the flag-off path, revoke and account-delete teardown, and the unbound-handler sweep. * test(events): a lagging replica read does not suspend a user-scope schedule run * feat(puter-js): inline user-scope handlers and scope option for handlers and workers * docs(events): document inline and user-scope handlers, with playground examples Cover inline handlers on onSchedule and onPersistent, handlerScope, the scope option on handlers and workers, the user-scope limits and rented units, and the new error codes. Add six playground examples and pin the new limit constants in limits.test.ts. * feat(schedules): carry the handler scope on the schedule view * docs(events): describe user-handler cleanup, schedule handlerScope and suspension notices * fix(events): gate the rented-unit cap on user handlers and let an account session destroy its own user scope * fix(events): keep user-scope rows off app handlers across mixed versions and bound their deploys and reads Cached durable rows carry an explicit scope marker; an unmarked entry with a handler is resolved from the table and rewritten. User-scope rows are parked before credit checks, billing, client delivery and suspension while the flag is off. Delivery-time handler checks and addressing share the replica cache entry. Concurrent eager deploys of one user-scope script upload once. * fix(events): keep a user scope's age across handler churn for rent event_user_handlers.scope_since records when the scope first held a handler. A new row takes the scope's existing minimum, so replacing code does not make the scope look newer to the monthly sweep. The count uses it instead of the oldest remaining handler's created_at. * fix(events): drop cached entries no row backs; document account-session scope destroy * test(jobs): list the user-handler sweep among the system jobs * fix(events): let the user-handlers migration apply on its own * fix(events): fail a cached-row read whose scope lookup fails instead of returning app-scope rows * fix(events): touch a reused user-scope handler before binding so the sweep cannot take it * fix(events): bound the deploy-on-miss wait by the call's deadline	2026-11-03 23:26:04 -07:00
Daniel Salazar	d934a47c5e	feat: jobs and schedules (#4467) ... * feat(db): add nowMsExpr, the database clock in epoch milliseconds, and share the retry backoff * feat(config): type the jobs config as an augmentable IJobsConfig and resolve its namespace in one place * feat(jobs): add JobScheduler for fleet- and region-wide system jobs * refactor(jobs): run core's periodic sweeps as system jobs, and publish the generation bumps of reaped subscriptions * feat(events): seams for schedules run by an extension, with events_disabled where none is installed * feat(notifications): add notification.send_key, the deferred mailbox write and notify({ sendAt }) through the schedules contract * feat(puter.js): schedules, notify({ sendAt }) and notifications.cancel(), and treat a 503 events_disabled as final * docs(events): document schedules, sendAt and cancel(), with recipes * fix(events): stop resuming a holder's later pages once the credit sweep's signal aborts, and keep its checkpoint * fix(events): pre-release fixes for jobs, schedule seams and delayed sends - The delayed-send write is a field on the notifications driver, not a method, so /drivers/call can't reach it. - A delayed send checks for its own earlier write by uid, so it takes no gap lock that every other notification insert waits on. - The notification send_key index builds in place without locking writes. - Reaps re-check expiry and suspension on the primary, so a row resumed since the replica read is kept. - Revocation settles skip schedule rows they can't settle. - An empty handler set read from a replica is re-read on the primary. - One ended notice per settle, not one per page. - Sweeps that run once per fleet take more batches per run. - The global usage rate check keeps its 14-minute minimum window. - shutdown() waits for prepare hooks a drain already started. * fix(puter.js): quota refusals are final, and schedule error codes match the server Schedule, subscription and handler quota 429s no longer go through the generic retry. ScheduleErrorCode lists what the schedule methods throw, and the notify({ sendAt }) overload has type tests. * docs(events): schedule and delayed-send accuracy and style pass Spread and send-time wording match the runner, the per-user runs-at-once and rate-limited stored notification limits are published, and the new pages follow the docs writing rules. * docs(events): add the Change Data at a Set Time recipe A one-time schedule that changes data when it runs, cancelling and moving it, and a repeat-a-few-times variant with maxRuns. * fix(events): delayed sends recheck the app's access when they fire, and a handler set missing the run's handler is re-read on the primary Also stops job polling at drain start and marks replayable notification sends. * feat(events): batch handler invocations to the events dispatcher Invocations that come due within a few milliseconds go out as one /invoke-batch request (up to events.invokeBatchSize, default 60). The dispatcher runs a few at a time and streams a started line and an answer line per call, so a slow handler delays no other call's answer. A call's timeout runs from its started line, so time queued behind other calls is never the handler's; queue time comes out of the connection-wait share instead, and a call still queued past it is never started and fails as Puter's. A lone call still uses /invoke, and a dispatcher without the batch route gets single calls for five minutes before the batch route is tried again.	2026-11-02 23:48:35 -07:00
Daniel Salazar	dd144b6b64	feat(notifications): notify through KV shares, group replacement, fewer push prompts, and events/worker fixes (#4432) ... * feat(perms): add a pushNotifications resource to puter.perms.request notifications:push gets a resource like backgroundEvents, so it can be batched with other access behind one prompt. requestPermission() and hasPermission() ask through it. * fix(puter-js): read puter.gui_features once at load The desktop's feature list was read from location.search on each call, so an app whose router rewrote the URL lost it and enablePush() and the other desktop dialogs answered as if the desktop lacked them. * feat(gui): ask for notifications:push when an app calls enablePush without it enable() and enablePush() inside the desktop answered unavailable for an app without the push permission. They now show the permission dialog, whose Allow also starts the browser registration, and answer with the browser's push state once it settles, or off when denied. A browser that blocks notifications or has no push service still answers without a dialog. * fix(kv-share): charge mint and grant budgets only for share calls that change something A share call now spends the per-call mint budget and the per-recipient grant windows after it has looked up which recipients already hold the share. A call whose recipients all hold the share at the requested level and label spends neither. Recipients without the share, including unknown usernames and refused recipients, still spend a grant unit each. Every named recipient spends from a new 1,200/min lookup window before any lookup runs. * perf(events): skip the subject split for rows not made through a share handle kvHandleFromSubject checks the stored subject's kv:kvh- prefix before splitting it, so own-namespace and fs rows return null without parsing. * feat(notifications): resolve enable() on a website before the confirmation push enable() waited up to 30 seconds for the confirmation push, so a prompt the same click started next had lost its activation. On a website it now resolves once the browser answered and the server has the subscription, records the opt-in, and finishes the confirmation in the background. A failed confirmation goes to onPush's onError and the next load's resync tries again. disable() and sign-out stop a confirmation still waiting. The GUI keeps waiting, since it reads the registered device right after. * fix(notifications): show a page's own notifications only while push is on A hidden page with an onPush handler kept showing notifications after disable(), and its receipts kept claiming showsOwn, which held pushes back from the user's other devices. Both now need the user's opt-in, cached in the page, refreshed on enable, disable, sign-out and each sync, and re-read when another tab of the site changes it. * fix(events): leave subscriptions suspended for good out of the durable quota countForHolder no longer counts rows suspended with a reason that never resumes (permission_revoked), in both the pre-insert check and the recount after insert. Rows suspended for a resumable reason still count. The store now owns the list of terminal reasons, and isResumable reads it. listActiveForHolder with includeSuspended returns only rows suspended for a resumable reason, so its quota-sized read still covers every row a revocation has to settle. Each holder keeps at most 500 terminal rows; a subscribe past that deletes the oldest ones. * feat(notifications): add status() to read whether pushes reach this browser status() resolves 'on', 'off', 'blocked' or 'unavailable' without asking anything. On a website it reads the notification permission, this browser's subscription and the user's opt-in. Inside the desktop a new pushStatus IPC, advertised as push-status, answers the browser's push state, 'on' only while the app holds notifications:push. * feat(notifications): disable() inside the desktop revokes the app's push permission disable() did nothing inside the desktop. A new disablePush IPC, advertised as disable-push, revokes the calling app's notifications:push grant through the same route Settings uses, so its notifications stop going out as pushes and a later enable() shows the permission dialog again. * feat(notifications): allow @ in a notification group, up to 128 characters A group is an opaque display tag, hashed for the Web Push topic and used as the notification tag, so it can carry identifiers such as an email address. * docs(notifications): say when a page with an older Puter.js holds a push back * fix(notifications): let disable() win over an enable() still in flight An enable() whose subscription reached the server after a disable() went on to record the opt-in, so the next page load turned pushes back on. It now rejects with push_verification_canceled and records nothing. * docs(notifications): list push_verification_canceled and reword sentences that led with code * fix(events): carry KV values through a share handle only at read or write The delivery re-check now reads the share's level off the live handle along with whether it still holds. A row on a watch share loses its includeValue for that delivery, so its events name the key without the value; the owner's own rows on the handle keep theirs. Decided per delivery, so raising or lowering a share applies to subscriptions already made, and subscribing with includeValue at watch is still accepted. The live-updates recipe now shares at read, which its events need to carry the score. * fix(events): retry worker runs Puter could not complete without counting them Invocations that never reached the handler (transport and dispatcher errors, unmarked answers, failed or throttled deploys, no configured runtime, a bad invoke key or missing token) now report an 'unavailable' outcome. So does a handler that throws after one of its own puter.js calls got a 5xx, 429 or network error from Puter's API, which the events runtime reports as 'platform-error'. Those runs back off like retriable ones, honor the answer's retry-after up to the backoff ceiling, and no longer count toward the consecutive-failure suspension. A run past the invoke timeout still counts. * test(notifications): cover the pushNotifications resource and status() in the API suites * test(gui): Allow on a batch that includes notifications:push turns push on * fix(events): hold a share-handle row's backlog when its share can't be checked The share-handle delivery check now tells a share that is gone (handle not live, not the row's holder, identity gone, grant withdrawn) from one it can't check right now (a lookup that throws, an owner who is suspended). Only the first settles a queued row as permission_revoked and purges its backlog. The second delivers nothing, leaves the row in service with its backlog, and sends it to the back of the sweeper's line, so delivery resumes once the lookup recovers or the owner is unsuspended. A revocation settle skips such a row and leaves it to the delivery check. kvSharedRegionAuthorized returns null when the permission check throws, and that answer is not cached. * feat(events): update a persistent subscription's context in place POST /events/subscriptions/update replaces a durable row's context, with the same validation and 4 KB cap as subscribe, keeping its subId and backlog. The row's owner region cache is rebuilt from the table and the generation bumped so every region delivers the new value. It spends the subscribe call budget. puter.events.update(subId, { context }) wraps it; null clears the context, and a function handler this client registered for the subscription gets the new ctx. * fix(events): serve a fan-out's holders in turns and cap values per delivery #route now orders an event's candidate rows before any cap applies: the keyspace owner's rows first, then the other holders taking turns, one row per holder per round, starting from a random holder each event. The filter evaluation cap, the delivery cap and the gap markers all count in that order. Values no longer drop for every delivery once more than 128 rows match or the filter evaluation stops early. The first 128 deliveries that asked for the value carry it, in serving order, and later ones carry the key alone. EVENTS_KV_VALUE_OMIT_MATCHED_SUBSCRIPTIONS becomes EVENTS_KV_VALUE_DELIVERIES_PER_EVENT. The caps themselves are unchanged. * fix(email): file inbox mail as a delivery, not the request actor's write storeInboxMessage and copyIntoInbox now run their filesystem writes with a platformWrite context flag, and fs event dispatch leaves actingUserId unset under it. Mail filed into an account's inbox from inside that account's own request (an app mailing the signed-in user) is delivered with self: false, the same as external ingress. Handler depth and viaWorker are still carried from the actor. * feat(notifications): raise the default daily app message limit to 1000 The per-day default for `app.message` goes from 300 to 1000 per user and app, matching the daily push cap. The per-minute default stays at 30, and `notificationAppMessageLimits` still overrides both. * feat(notifications): look up an unseen notification by group `notification` gets a nullable `group_key` column, a sha256 hex digest of the group a row was written with, indexed on (user_id, app_uid, group_key). `NotificationStore.create` takes a `group`, and `getUnseenByGroup` returns the newest row an app wrote for a user under that group that is neither shown nor acknowledged. Rows written before the column exist never match. `updateValue` takes `onlyUnseen`, which also skips a shown row, and `refreshCreatedAt`, which dates the row to now. Migrations: sqlite 0098, mysql_mig_52, postgres_mig_41. * feat(notifications): replace an app's unseen message in the same group An `app.message` with a `group` now replaces the app's earlier notification to the same user with that group while it is neither shown nor acknowledged. The row keeps its uid and takes the new value and time. With no group, or once the earlier one was shown or dismissed, it is a new notification as before. A replacement spends a per-minute slot but no per-day slot. If the earlier row is marked between the lookup and the rewrite, the message is written as a new notification and spends a day slot then. `app.message` is groupable, so the replacement goes through `notifyUpdate`. For types kept off the desktop, `notifyUpdate` skips the desktop wire and dispatches the rewrite again under the same uid, so app subscribers and receipts see it, and `notif.written` carries `updated`. The driver's write path takes the recipient id rather than the actor, so rate limits key on the recipient. * feat(workers): report the app a worker runs as workers.create now returns app_uid and app_name for the app its token is bound to (null for a user-scoped worker), and list/get items carry app_name beside app_uid from the app rows they already load. The dev center workers table shows an App column, and the SDK typedefs and docs describe the fields and that a sandboxed worker's notifications and pushes come from its sandbox app. * feat(events): refuse handlers with free variables on REST publish POST /events/handlers/publish and /publishAll now run the SDK's own scanHandlerSource on each source before publishing and answer 400 events_handler_free_variable, naming the identifier. The backend imports the module from src/puter-js; the build marks dist/src/puter-js as an ES module package so the compiled backend can require it. Sources over the per-handler cap are left to the store's size refusal rather than tokenized. * docs(events): say Puter-side retries never suspend and stay within backlog caps * fix(events): redeploy an events worker that answers bad-key A script deployed before the internal secret rotated keeps the old invoke key, so every invocation answered bad-key and nothing redeployed it. A bad-key answer now goes through the same miss handler as a dispatcher miss (the deployer, with its hourly throttle) and is retried once with the deployed header. A throttled or failed redeploy stays 'unavailable' and backs off without counting. * perf(events): check the deploy throttle before reading handler sources EventsWorkerDeployer now reads the app's handler names and hashes first, answers no-handlers or stale from those, and checks the hourly deploy throttle before loading the sources and generating the worker. A throttled retry costs a hash read and a counter bump instead of a full source read and build, and a stale request still does not spend the throttle. The generated set is checked against the script again in case it moved between the two reads. * fix(workers): move a redeployed worker's row to the app it now runs as Redeploying an existing worker under a different app minted the new app's token but left subdomains.app_owner on the old one, so list/get and hot-reload kept naming and minting for the old app. The redeploy now sets app_owner to the binding #resolveWorkerAppBinding authorized (null for a user-scoped worker) through a new SubdomainStore.setAppOwner, since app_owner stays read-only to update patches. * fix(workers): revoke the old binding's worker session on a rebind A redeploy that binds a worker to a different app (or to none) now deletes the owner's worker sessions under that name other than the one for the new binding, through the same deleteWorkerSessionsByName path worker deletion uses (which also announces the revocation and drops access-token grants). SessionStore's deleteWorkerSessions takes an exceptAppUid filter for it, null meaning the user-scoped session. The app_owner move and the revocation now both run only after the new script deployed successfully, so a failed redeploy leaves the live script, its token and the row on the old binding. * feat(kv-share): store a notify flag on share handles kv_share_handles gains a notify column, off by default, so every existing share reads as not allowing notifications (sqlite 0099, mysql_mig_53, postgres_mig_42). The store reads it on every row, including the live handle cache and the holder listing, writes it on mint, and changes it through update(). * feat(kv-share): let the owner turn notifications on for a share POST /kv/shares and puter.kv.share() take notify, off by default. Leaving it out of a re-share keeps the stored value, and turning it on or off counts as a change, so it spends the mint budget. Turning it on for a holder who now refuses the owner's shares is refused, like a raise or a new label. The kv.share.received notice is sent only when a share starts carrying notify: a new share made with it, or an existing one switched on. Its hourly bound and the app-less namespace exclusion are unchanged. The share response, the owner's listing and the holder's listing report notify. * feat(kv-share): resolve the other party of a share for a notification KvShareService.resolveNotifyRecipient(actor, handle) returns the recipient user id, app uid and handle for a notification one party of a share sends the other through its handle. The handle must be live with an unsuspended owner, the caller must be its owner or holder acting from the app whose data it is, the share must have notify on, the holder's grant must still stand, and the recipient must not refuse shares from the sender. An unknown, retired or foreign handle, a caller who is not a party, and a caller in another app all get subject_does_not_exist. The account itself gets forbidden, a share without notify kv_share_notify_not_enabled, and a lookup that fails service_unavailable. Once every check passes it spends one unit from KV_SHARE_NOTIFY_LIMITS (30/min, 500/day per sender, recipient and app), peeking both windows first so a refusal spends nothing. * feat(notifications): hand a replaced notification to listeners again A replacement now records `replaced_at` (epoch ms) on its row, added to the same unreleased migrations as `group_key` (the sqlite file is renamed to 0098_notification-groups.sql). Its live event and its fetched copy both carry `replaced: true` and use that time as `ts`, so the pair versions the uid identically on both paths. `onNotification()` dedupes on uid plus that version: an original counts as version 0 and a replacement as its `ts`. A newer replacement is handed over again with `replaced: true`, while a copy already handed over, a replay, or an older state is dropped. Listeners started with `catchUp: false` still get replacements of what was already there. * fix(notifications): replace only the newest notification in a group The group lookup takes the newest row an app wrote under the group and returns nothing when that row was shown or dismissed, instead of falling back to an older row that is still unseen. The docs state the rule as it is: a grouped notification replaces the newest one in its group unless that was shown or dismissed. Puter.js marks neither, and the only thing that does is a tap on the push inside the Puter desktop, which dismisses it. * feat(notifications): notify the other party of a KV share through its handle `app.message` takes `to: { handle }`. The driver validates the message, then `to` must be an object with only a non-empty string `handle`, else 400. `KvShareService.resolveNotifyRecipient` checks the share and spends the pair limit, and the message is written for the share's other party in the share's app. The recipient's per-app limits and group replacement apply to their own rows, and the push is billed to them through `notif.written`. A call with `to` answers with `{ uid }` only, so nothing about the recipient's row reaches the sender. puter.js gets the `NotificationRecipient` type for `to`. * docs(notifications): document notifying the other side of a KV share notify.md covers `to: { handle }`: who can be reached, the errors, the recipient's limits plus the per-pair share limit, the uid-only return and that the recipient pays for pushes. push.md and rate-limits-and-quotas.md point at the share path. The chat notification recipe sends through each share's handle and keeps the per-member handler only for members who need to reach each other; the alert recipe says when its handler is the right fit. * fix(notifications): reveal nothing to the sender of a notification through a share A call with `to: { handle }` answers with an empty object, and puter.js resolves it with no value: a uid would show whether the recipient's last notification in a group was replaced. `notify()` gets one overload per form, and `ShareNotificationInput` is exported. Over the per-pair share limit or the recipient's own limits, such a call gets the same 429 `too_many_requests` with the same generic message, so the sender can't tell which limit it hit. * fix(notifications): serialize grouped app messages per group Grouped app.message sends take a Redis lock on (recipient, app, group) around the lookup and the replace-or-insert, so concurrent sends to an empty group write one row and spend one day slot. A lock still busy after the wait, or an unreachable Redis, sends unlocked as before. Sends without a group take no lock. * fix(events): attribute Puter API failures to the invocation that saw them The events worker runtime counted Puter API failures on the cached per-token client, so a concurrent invocation on the same token could answer platform-error for its own unrelated throw. A cached client now serves one invocation at a time and records failures and retry-after on that invocation; an invocation overlapping another on the same token gets its own client, and idle clients stay cached per token. * fix(events): drop KV values from a queued backlog once its share is at watch A queue delivery is stored with its value. Every hand-out of a stored entry (backlog drain to a socket or another region, the worker, and a declined delivery passed to the worker) now re-reads the share's level and removes the value at watch. A share that can't be checked still holds the backlog. * fix(events): return notifications replaced behind the fetch cursor The notification fetch cursor now holds the last row id and the last (replaced_at, id) it returned. A read from it merges two indexed reads: rows inserted after the id, and rows up to the id replaced after that mark. A replacement made while a client was away therefore comes back on its next catch-up. Adds idx_notification_replaced on (user_id, audience, app_uid, replaced_at).	2026-10-31 12:21:18 -07:00
Daniel Salazar	4851df716c	fix(kv): KV share lifecycle hardening (#4429) ... * fix(kv): harden the KV share lifecycle - Retire a deleted app's live shares before an origin app is registered again under the same uid. The retire runs before the app row exists and throws on failure, so a failed launch leaves no row and the next one retries. - Record deleted-app retires and uninstall cleanups (retire, then purge) as reserved items before they run. A lease keeps one node on a job; a failed run raises an alarm and backs off, and a run that hits its time budget or is cut short by a restart resumes on the next check. Each job is bounded to handles made before it was recorded. - Uninstalling with deleteData records the job and returns; the retire and purge run off the request. The purge checks whether the app was authorized again before deleting each page, so data written after that is kept. - Take a per-region lock when minting and re-check for a live handle under it, so concurrent shares of one region make one handle. - Retire the handles an account holds before the account is deleted, so the owners' subscriptions through them end instead of staying active. * fix(events): bound share-handle session rows by the per-connection cap The count across a holder's connections also counted connections that had dropped but weren't reaped yet (a crashed node, a deploy, a network blip the server hasn't noticed), so resubscribes after a reconnect could be refused with `events_subscription_limit`. Rows made through a share handle now count toward the 50-per-connection cap like any other, and with the open-connections cap per account that bounds what one holder can put in the owner's keyspace. The handle index stays for ending rows on retire. Rows are added to it with the row itself and taken out when the row is dropped, and members whose row is gone are cleared when a retire reads it. * fix(events): keep a retire's durable settle running if ending sessions fails Ending a retired handle's session subscriptions reads Redis. A failure there is now logged and the durable suspension that follows still runs, so those rows' backlogs are still purged. * perf(events): ask once per handle when a refresh screens rows, and share handle reads A socket refresh asks whether a share handle still stands once per handle, holder and app, rather than once per row: rows through one handle share its grant. Concurrent cache misses on one handle share a single read; a forget drops the shared read so later callers start a fresh one. * fix(permissions): chunk the exact-match subtree delete The holder-scoped subtree delete names every matched grant, so a large subtree could exceed an engine's limit on bound parameters. It now deletes 500 grants per statement. * fix(kv): give each holder a share of a namespace's handle budget One holder could spend a namespace's whole handle budget and starve the others. Each holder now gets at most 3,750 of the namespace's 15,000 units per 10 seconds, and the owner's own calls through a handle spend none. The holder's budget is checked before the grant lookups, so throttled retries cost none; the namespace's is only told to callers the grant admits. Multi-unit charges use a new counted claim that keeps two counters per key instead of a sorted-set member per unit. * fix(kv): refuse handles older than their app and claim cleanup jobs globally - A share handle created before its app row reads as gone: in the KV driver, in the handle cache, on the events subscribe path, and in the delegate implicator. An origin app registered again under a deleted app's uid can't reach its predecessor's shares, whatever created the row. The app-token launch no longer retires inline; it records a background retire and never waits on it or fails because of it. The clock-skew rule is shared with app sessions. - Cleanup job records now carry their lease, attempts and retry time, and a node claims a job by compare-and-set on its record. A job stays with the deployment that recorded or last ran it, and another one adopts it only after it has gone unclaimed well past due. - The job check pages through records from where the last look stopped, so waiting records don't hide due ones. After 10 failed runs a job stops retrying with a warning alarm, and its record expires after 30 days. - A mint that waits out the region lock looks for the share again and returns it, or goes ahead, instead of failing with too_many_requests. - The record-failure alarm dedups; an unfinished run is a log line. * fix(kv): hide shares older than their app from re-shares and listings After an app is registered again under a deleted app's uid, its predecessor's live handles stayed visible: sharing the same region again handed back the old, dead handle, and both sides' listings showed the old handles and labels. findLive and both listing queries now leave out a live handle created before its app row, the same rule handle resolution uses, so a re-share makes a new handle and the listings, totals included, skip the old ones until their retire lands. A new share also retires, grants and all, the handles its owner gave the same holder for the earlier app first, so withdrawing those later can't take the new grant with it. * fix(events): keep a loose cap on subscriptions through one share handle Without a count across connections, a holder could put up to 20,000 rows in the owner's keyspace (50 per connection, 400 connections), each one evaluated on every owner write. The cap is back at 1,000 per holder per handle, far above what connections that dropped without being reaped leave behind. The index is only read row by row once it's past the cap. * fix(kv): harden the share cleanup jobs - Job records are keyed by domain, so a deployment that shares the table but not the database or broadcast (staging beside production) never runs another's jobs. - A job whose record can't be written runs unrecorded instead of not at all, and a failed check for live handles records the job anyway so it retries. Failures that used to be swallowed are logged. - Recording a job again keeps its attempts and backoff; one that gave up gets one more attempt. - A system retire gives back the grant of a live share it doesn't cover inside a withdrawn region, and a share made under a reused uid grants again once its row stands, so a concurrent retire can't leave a live handle without its grant. * fix(kv): don't cache a live handle a retire overtook during its app lookup The live read decided whether it could cache before awaiting the app lookup, so a retire's forget that landed during that lookup was overwritten with the handle as live for up to 60 s. It's now checked after the lookup. * chore(kv): word the job key comment for any install	2026-10-31 12:12:29 -07:00
Daniel Salazar	2d5e037599	fix(ai): bill realtime responses the upstream drops before reporting usage (#4423) ... When the upstream realtime connection closes, for any reason, with a started response that never sent response.done, the relay now records that response at its credit hold (8k audio in + 4,096 audio out at the session model's rates, through the AI cost factor) instead of releasing the hold uncharged. The hold is released once the charge is recorded, and a warning alarm is still raised. Client conversation.item.retrieve events and MCP tools in session.update or response.create are refused with an invalid_event error and a 1008 close.	2026-10-30 23:48:58 -07:00
spacedandDevin AI	7ac2f2e247	feat(ai-chat): let requests opt out of provider fallback (#4343) ... * ai-chat: fallback: false fails on the first route instead of rerouting A chat request whose first provider fails is retried on up to two other providers that serve the same model, and the caller pays for whichever one answers. Callers who want one specific route, or who want to handle a vendor failure themselves, had no way to turn that off. Passing fallback: false (puter.js option, /drivers/call arg, and the OpenAI-compat and Anthropic-compat request bodies) makes the first attempt's failure final: the route is still marked unhealthy, and the error is classified the same way an exhausted chain would be. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ai-chat: cover fallback: false in the SDK suite Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ai-chat: fallback: false rejects a pinned provider with no route for the model Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: plainer wording for the fallback option Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-30 19:35:45 -07:00
Juan Fernando Castro	fea89c91b8	feat(puter-js)!: remove notifications.registerDevice (#4409) ... * docs(notifications): registerDevice takes ios or android * feat(puter-js)!: remove notifications.registerDevice	2026-10-30 19:31:30 -07:00
Daniel Salazar	4881264c88	feat(ai): realtime reconnect notice, context restore and retry hardening (#4415) ... * test(ai): end every realtime relay test's upstream before the next The fake upstream now answers response.create with response.created first, as OpenAI does, so closing those sessions no longer waits out the settle window. * feat(ai): ask realtime clients to reconnect before a restart Open realtime sessions get a puter.reconnect event when the drain starts, with close_in_ms set to the drain window, and new sessions are still refused until the 1012 close at shutdown. A host extension can announce a restart early by emitting serverRestartAnnounced; that only sends the notice, so a false alarm leaves the server working normally. A session gets the notice at most once a minute. onServerDrainStart now receives the drain duration. * fix(ai): retry realtime reconnects for about a minute and wait out offline browsers Five attempts gave up after about 15 seconds, shorter than a server restart. The conversation makes ten attempts with the backoff capped at 10 seconds, and while the browser reports being offline it waits for the online event, up to two minutes per reconnect, instead of spending attempts. close() ends either wait and removes its listener. * feat(ai): restore recent conversation context after a realtime reconnect The provider starts a fresh session on every reconnect, so the conversation lost all earlier items. The SDK now records finished items it can recreate from the provider's own events: user text, user speech with a transcript, assistant replies as text, and function calls with their outputs. After the new session acknowledges its settings, and before ready fires, it adds back the newest of them, up to 40 items and 48 KiB, which stays under the relay's upload burst. Assistant replies are cut to what the user saw or heard: the deltas shown before an interruption, and the share of the transcript matching the audio played before a truncation or a drop. Deleted items, unfinished replies, images, speech without a transcript and incomplete function calls are left out, as are function outputs whose call falls outside the window. Restored copies take over their new item IDs. A refused item is reported once as context_restore_failed without failing the reconnect. * feat(ai): move a realtime conversation to a new session on the restart notice On puter.reconnect the conversation waits until it is idle: no reply requested, generating or playing, no user turn being spoken or transcribed, no function call awaiting its output and no acknowledgment pending. Then it reconnects through the usual path, emitting reconnecting (code 1012) and ready with reconnected: true, without an error. Until then the session keeps working, and if the server closes it first the normal 1012 reconnect takes over. close() at any point stops the wait. * docs(ai): realtime restart notice, context restore and reconnect limits * test(ai): realtime reconnects against the local server The mock provider answers with item events and whole replies like OpenAI. New cases cover a dropped connection (cut through a TCP proxy so the client sees 1006), a host-announced restart, and a draining server refusing the reconnect; the first two check the provider receives the restored items before the next turn. The browser media test waits for captured audio and the reply instead of a fixed sleep. The live suite gains a tool and text context round trip across a restart notice. * ci: run the live realtime conversation tests with the AI provider suite They need the OpenAI key, which only this workflow has, and skip themselves without it. * fix(ai): open the replacement before leaving a realtime session on the restart notice The handover closed the current session and then reconnected, so every notice cost a second or two in which sends failed and microphone audio was dropped. On a single server that refuses new sessions while it drains, the retries ran out during the drain and the conversation closed sooner than if it had stayed. The conversation now opens a replacement session as soon as the notice arrives and switches only once it is open and the conversation is idle. A refused replacement is retried with backoff while the current session keeps working; when the server closes it first, the held replacement is used right away. The idle check also missed some in-progress work: - a reply created while the user was still speaking ended the user's turn; speech now lasts until its audio is committed or cleared, and a committed turn waits for the reply server turn detection creates - only the newest function call was checked; every finished call since the last user message needs its output, and calls cut off by a cancellation don't count - native session.update and conversation.item.create events the provider hadn't acknowledged could be lost; the switch waits for them, up to the acknowledgment timeout for items - only the newest transcription was waited on; every committed item is - clearing playback didn't recheck the switch Restored items stay internal: the new session's copies of them no longer reach listeners. At a reconnect, every queued item the user didn't hear is cut, not just the one playing. A notice from a failed reconnect attempt is dropped. * test(ai): realtime replay after split settings, and a draining server refusing the replacement The relay accepts the context replay right after two full-size settings updates. Against the local server, a draining relay refuses the replacement session while the current one keeps answering. * ci: build the SDK in the live realtime step so it stands on its own * fix(ai): a failed realtime reconnect attempt no longer poisons the next An attempt that dropped after sending its session.update but before the acknowledgment left that event ID queued, so the next attempt's acknowledgment was matched to it and timed out, and so on until the conversation closed. Turn state is now reset after each failed attempt. * fix(ai): bound the realtime handover's waits and stop retries that can't succeed - The first replacement opens within a randomized 3 seconds of the notice, so sessions notified together don't all open at once. - A replacement refused with a code retrying can't fix, such as insufficient_funds, ends the attempts; the server's close still falls back to the usual reconnect. - A held replacement that gets an error or its own server's notice is closed and replaced. - A held replacement used after a drop gets a 3 second acknowledgment timeout, since its socket may have died with the network. - close() during the switch closes the held replacement too. - The reply to a committed turn and each transcription hold the switch for at most 10 and 30 seconds, like app items already did, and an app item is released only by an added item of its type, role and call ID. * docs(ai): the realtime restart notice isn't guaranteed before every restart	2026-10-30 18:06:54 -07:00
3 people	c7c3f21e4b	feat(ai): puter.ai.convo - Preliminary Realtime API implementation (#4401) ... * feat(ai): relay OpenAI's Realtime API as puter.ai.assistant Adds a WebSocket relay at /puterai/openai/v1/realtime on the api host, one upstream OpenAI connection per client, frames passed through 1:1. The client socket is accepted before auth so socket.io's 1 s unhandled-upgrade timeout can't kill slow handshakes; failures arrive as OpenAI-shaped error events and a 1008 close. Usage is metered per response.done at the model's token rates, sessions are capped per user (realtime.maxConnectionsPerUser), and each node logs counters, event-loop delay and memory every 30 s. puter.js gets puter.ai.assistant.connect(), returning a session with send/on/off/close over the same OpenAI events. realtime.upstreamUrl points the relay at a mock for load tests; tools/realtime-stress has the mock upstream, a load generator with a JSON report, and an agent runbook. The relay README lists its quirks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ai): backpressure and upload budget on the realtime relay Neither direction applied backpressure: a client that stopped reading grew a queue up to 64 MiB, and a client streaming 16 MiB frames was read and forwarded as fast as it could send. A few such sockets pinned the event loop and pushed RSS past 1 GB. Each side now stops reading the other once 1 MiB is queued toward it, and a client that keeps the upstream paused for 30 s is closed. Client uploads are budgeted per session (1 MiB/s sustained, 16 MiB burst, realtime.maxClientBytesPerSecond), pre-open buffering pauses reads at 1 MiB instead of rejecting at 32 MiB, and clients get a 30 s ping so half-open sockets release their upstream and slot. Upstream frames use a zero mask and skip UTF-8 validation, which the relay never needed. The stats line gains elu. tools/realtime-stress gains abuse.mjs and a runbook phase for event-loop abuse. * fix(ai): drain, transcription metering and credential re-check on the realtime relay Shutdown now drains: a new onServerDrainStart hook (run when SIGTERM starts the drain, before the listener closes) makes the relay refuse new sessions with 1012, open ones keep running through the drain window, and at onServerPrepareShutdown each gets up to 5 s to finish an in-flight response before closing with 1012 so the client reconnects elsewhere. Input transcription is metered. The relay reads the type from each text frame's first 96 bytes and, after forwarding, parses response.done, input_audio_transcription.completed and session.created/updated (for the transcription model) on setImmediate, so metering never holds up frames. Unknown transcription models bill at the highest known rate. Credentials are re-checked: a successful auth is cached in the kv.js singleton for 60 s under a hash of the token, and a 15 s sweep re-authenticates sessions whose entry expired, closing revoked or suspended ones with invalid_api_key. Lookup errors keep the session and retry on the next sweep. * docs(ai): API shape proposal for puter.ai.conversation() across OpenAI, Gemini Live and xAI * feat(ai): add composable realtime conversations with bounded transport * fix(ai): realtime billing, subscription gate and conversation reconnects Relay: - Cancel and bill a response still generating when the client leaves, instead of cutting the upstream before response.done. - Record realtime usage through the AI cost-factor hook. - Drop the upstream byte and message caps; replies arrive faster than real time and tripping them closed sessions and skipped billing. - Price usage as it arrives, before any forwarding decision, so a closing session never drops a usage event. - Bill the model the session reports and refuse a session.update that changes the model. - Require a paid plan. SDK (puter.ai.convo): - Reconnect after a dropped connection with the same settings, keeping listeners and media; emits reconnecting and ready { reconnected }. - Errors reject only the operation they name; a failed send rejects only its own request. - configure() sends only the given settings and settles on its own ack; commitAudio follows turn detection set natively. - interrupt() right after sendText cancels the reply once it starts. - Microphone: capture at the device rate, drop chunks instead of stopping on stalls or a congested socket, no timeout on the permission prompt, permission_denied on refusal. - Speaker: 300 s playback bound without interrupting, played position accounts for output latency and gaps between chunks. - Audio control events keep headroom over queued audio. - Upgrade prompt on subscription_required / insufficient_funds. Tests that need the built SDK and Chromium move to the puter.js suite. * chore: remove stray node_modules symlinks * feat(ai): retire assistant API and document realtime beta * test(ai): wait for the matching realtime billing batch * test(kv): exclude health probes from warm read measurements * Revert "test(kv): exclude health probes from warm read measurements" This reverts commit 631003f755. * fix(ai): upstream message size, null usage, team 2FA and reconnect settings for realtime Relay: - Upstream messages may be up to 4 MiB; client messages stay at 64 KiB. An OpenAI event above 64 KiB (an echoed item near the client limit, a long reply, a retrieved audio item) closed the upstream before in-flight responses reported usage. - An upstream that closes with a response still in flight raises a warning alarm. - A null usage counts as none instead of closing the session. - Seats whose team requires 2FA need it for realtime too, at connect and on the credential recheck. SDK (puter.ai.convo): - A reconnect reapplies every acknowledged setting, native session.update events included, split to fit the message limit. - connect() during a reconnect settles with it. - Reconnect delays are jittered, and closing cancels a pending delay. The credential-recheck and browser worklet tests now fail when the behavior they cover breaks. * feat(ai): credit holds, attempt limits and late-response billing for realtime (#4407) Relay: - Each response holds 8,000 audio input and 4,096 audio output tokens of the account's budget, at the session model's rates, from response.created until its usage is recorded. A session opens only if one hold fits. - Session attempts are limited to 60 per minute per credential, checked before authentication. - A response the client requested that starts after the client leaves is cancelled and billed. - Input transcription is priced at the costlier of the transcription models in force at commit and at completion. - rate_limits.updated isn't forwarded, and provider organization IDs are removed from error messages. SDK (puter.ai.convo): - microphone.stop() during the permission prompt rejects the pending start() with session_closed. - speaker.start() rejects with { code, message } for browser exceptions. - connect() waits 20 seconds for the session, past the server's 15-second setup. - Convenience events fill IDs the provider leaves out; types cover turn.done, error, and client events for send(). --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-10-30 14:50:15 -07:00
Juan Castro	e8688d5025	Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications	2026-10-28 16:30:35 -04:00
Daniel Salazar	179ad990d6	feat(events): simplify onPersistent delivery (#4395) ... * feat(events): simplify onPersistent delivery Replace delivery/targets with a boolean single. Without single, every event runs the app's published handler in its events worker and is also sent to every connected client; the server no longer routes between the two. single keeps leasing each event to one consumer that acks. delivery: 'single' | 'broadcast' is still accepted and targets is ignored. Stored rows are read by their delivery column alone. * docs(events): compare onLocal, onPersistent and single * feat(events): rename single to queue onPersistent({ queue: true }) is the work-queue mode: each event goes to one consumer at a time and is offered again until acknowledged. delivery: 'single' is still accepted as an alias, and subscription objects report queue instead of single. The docs explain the three modes (onLocal, default onPersistent, queue) and each events recipe says which one it uses and why. * docs(events): retries are the only repeat for a queued event * feat(events): queue deliveries fall through to the worker on decline A queued delivery gets one socket attempt under a 15 s lease instead of two 60 s ones. A client with no handler for it, or whose handler fails, sends events.nack and the worker takes it without waiting for the lease. Declines are relayed to the region holding the lease like acks are, and a decline for a lease that is no longer the client's is ignored. * feat(events): drop thrown errors, retry only temporary ones A queued handler that throws now drops the event and counts one failure toward the pause. Network errors, timeouts, rate limits, 5xx responses and errors thrown with retry: true are retried with the existing backoff. One classifier is shared by puter.js and the events worker runtime, which inlines it at build time. A dropped event is billed once, like a completed one. ack() remains as an optional early release. * docs(events): queued events reach every tab with a handler	2026-10-28 12:58:51 -07:00
Juan Castro	352a2fbabd	Merge juancastro/put-2167-notification-written-hook into juancastro/put-2170-puterjs-notifications	2026-10-28 09:45:45 -04:00
Neal Shah	25a574b57d	GUI for custom domains (#4355) ... * PUT-2226 * modifications' * routing fixes * secondary custom domain verification	2026-10-27 15:54:02 -04:00
Juan Castro	5e376bc0c8	feat(puter-js): puter.notifications ... An app notifies the user it runs for, reads its own notifications back and follows new ones, hears clicks on them from the desktop, and asks for or checks the notifications:push grant. Puter's own apps also register, list and remove the user's devices.	2026-10-26 16:36:42 -04:00
Daniel Salazar	f850155411	KV share handles: hardening, higher limits, batch share, uninstall data deletion, account-deletion cleanup (#4344) ... * fix(kv): sign share handle ids and drop retired handles from every node's cache New handle ids carry a short HMAC (kvh-<uuid>.<mac>); an id whose mac does not verify is refused before any lookup. Unsigned ids keep working but no longer fall back to the primary on a replica miss. Retiring or changing a handle now broadcasts outer.pubsub.kv.share.forget so every node drops its cached row. Event subscribes and delivery rechecks refuse a handle whose owner is suspended, as the driver does. * fix(kv): an app its user removed stops reaching shares An app acting for a user exercises a share grant only while the user still has the app authorized: at least one user-to-app permission row, which every app token path leaves and revoke-user-app '*' removes. The share itself stays; authorizing the app again restores access. The answer rides the permission cache, keyed by the app-under-user generation the revoke bumps, so live subscriptions go quiet too. * feat(kv): list keys through a watch-level share handle list with as: 'keys' through a handle needs only watch, which already learns every key written in the region. Values and entries still need read. * feat(kv): filter share listings by app and by state GET /kv/shares and GET /kv/shares/with-me take an optional appUid: an app uid, or null for the app-less store. An account session may name any app; an app is refused any namespace but its own. GET /kv/shares also takes state=live|revoked|all, all by default. * feat(puter.js): appUid and state options on the KV share listings listShared() and listSharedByMe() pass appUid (null for the app-less store), and listSharedByMe() passes state. Docs cover the new options and that a watch share can list keys without their values. * feat(kv): indexes and store counts for share-handle quotas and retention Adds covering indexes for the owner's live count, the owner-holder pair, app-filtered listings and retired rows, plus store methods to count a pair's live handles and delete rows retired before a cutoff. The live handle cache holds up to 50,000 entries; the miss cache stays at 10,000. * feat(kv): higher share-handle caps, a per-holder cap, and sharing with several recipients Live share handles per account go to 25,000 paid / 5,000 free, and per app to 10,000 / 2,000. One owner may hold out 256 live handles to one holder on every plan, which bounds the rows a revoke or unshare walks. Re-sharing an existing region takes no slot and is never refused. POST /kv/shares takes recipients (1 to 50) as an alternative to recipient: one region, level and label for all. Caps and a new per-recipient grant budget (300/min, 2,000/hour) are checked for the whole batch up front; failures about one recipient are reported on its item. The single-recipient request and response are unchanged and run through the same code path. * feat(kv): delete share handles 30 days after they stop A daily job deletes retired share-handle rows older than 30 days, in batches of 500. One node runs it per day; it stops on shutdown. * feat(puter.js): share a KV prefix with several users in one call puter.kv.share() takes an array of recipients, positionally or as recipients in the options form, and resolves to one item per recipient: the share, or an error with its message and code. The single-recipient forms are unchanged. * feat(kv): delete an app's data on uninstall, and close account-deletion gaps Uninstall: - POST /auth/revoke-user-app takes an optional `deleteData` boolean, only with `permission: '*'` (400 otherwise). AuthService.uninstallApp revokes every grant, deletes the app's worker sessions, then awaits `app.uninstalled` { userId, userUuid, appUid, deleteData }. - With deleteData, KvShareService retires the handles the user shares out of that app's namespace (as the platform, in batches) and then empties the namespace in the background, 5,000 items per step, stopping on shutdown. Handles shared with the user are not touched. - SystemKVStore.purgeNamespace(namespace, { maxItems }) deletes up to maxItems items of one `v1:<uuid>:<appUid>` namespace, reading keys only, with no mutation events or usage, and reports `done` once it is empty. It refuses the system namespace and anything else. flush shares its page-delete path. - puter.perms.revokeApp(appUid, '*', { deleteData: true }); the desktop's uninstall dialog has an unchecked "Also delete this app's data" option. Account deletion: - EventsService settles each of a deleted account's apps on `user.delete` (they cascade without `app.changed`) and drops the owner's subscription keys. - The flat permission entries the account held are purged in pages. Suspension: - `user.suspended` / `user.unsuspended` events; the share-handle store drops cached rows owned by that user here and on other nodes. * fix(permission): await a peer's flat-permission deletes so emitAndWait sees them applied * fix(kv): strongly consistent purge pages, and an uninstall purge stops once the app is authorized again A stale empty page could end a purge as done while items remained. The background purge after an uninstall now checks before each step that the user hasn't authorized the app again, so data written after that is kept. * fix(kv): flush keeps eventually consistent page reads; only a purge reads strongly consistent * fix(user): emit user.suspended and user.unsuspended when the user store changes an account's suspension	2026-10-26 09:43:55 -07:00
Daniel Salazar	589a98b12c	feat(kv): share KV prefixes with other users at watch/read/write, and bound what apps can be granted (#4331) ... * feat(kv): share key-value prefixes with other users, and bound what apps can be granted - KV share handles move out of Events into KvShareService, served at /kv/shares (the /events/kv-handles routes stay as an alias). - Shares have watch, read and write levels, encoded in the permission root (kv-share, kv-share-read, kv-share-write) with write implying read and watch. - Every puter.kv method takes optConfig.handle; keys are relative to the shared prefix, cursors are sealed, and private entries stay hidden. - New puter.kv.share, unshare, listShared and listSharedByMe; optional label; kv.share.received notice in the app's mailbox only; recipient blocks apply. - puter.kv.set gains onlyIfAbsent. - fs and kv events gain viaWorker, and kv events report self from the writer. - /auth/grant-user-app only accepts canonical permissions from a fixed set of families; the consent dialog, puter.perms and docs follow. - Migrations: kv_share_handles gains revoked_by, label, and grantee and app indexes (mysql_mig_49, postgres_mig_38, sqlite 0095). * fix(kv): never name a private key in a cross-app or handle cursor, and bound flat-permission scans - Listings of another namespace (cross-app or through a handle) resume after the last entry they returned, and their cursors are sealed, bound to the namespace or handle, and padded to one length. Old plain cross-app cursors are still read when they name the target namespace; other namespaces get 400. - PermissionStore scans flat permissions a few roots at a time and skips roots nested under another root in the same call.	2026-10-25 08:16:19 -07:00
Daniel Salazar	8b8dd19eae	fix(teams): refuse to boot when teams_enabled has nothing registering Teams (#4300) ... * fix(teams): refuse to boot when teams_enabled has nothing registering Teams Core's seat checks (team 2FA, seat password reset, email and username changes, self-delete, the signup placeholder claim) read the team store and skip when it is absent. With `teams_enabled` on and no extension registering the store and service, the server now fails at boot instead of serving without those checks. - whoami no longer string-matches the extension proxy's "missing property" error: with the flag on, the store is always there. - Tests that turned the flag on without Teams drop it; the puter.js harness sets it only under the `teams` capability. - Drop a stale services/index.ts comment. * refactor(gui): drop the Teams console styles, which ship with the console now	2026-10-22 15:11:36 -07:00
Juan Fernando Castro	1627650abc	refactor: make teams an optional feature (#4248) ... * refactor: let core run without a teams registration Teams becomes optional: `services.team` and `stores.team` may be absent, and core stops assuming either is there. Seventeen call sites across share, signup, account deletion and the homepage now degrade instead of throwing: a team recipient that cannot be resolved is "no such recipient" rather than a fault, a group share folds in nothing, and account deletion stops asking what a seat was charged for. This generalises the pattern `gates.ts` and `SocketService` already used. Teams' own files take a private handle to their store and service, so the feature's code stays free of optionality that only core needs. No behaviour changes where Teams is registered. * refactor: put a contract between core and teams `core/teams.ts` holds the shapes core reads and the calls it makes, and the registries declare the pair as provided rather than built in. The service, store and controller are registered by whatever runs the feature. Typing them properly turned up eight AuthController call sites the previous commit could not see: the controller reads its stores from the built-in registry, so an absent `team` fell to an index signature and typed as `unknown` instead of erroring. They now go through the same optional path as the rest. The feature's own tests move with it. The cases in core suites that build a team to exercise a core rule still need relocating, so those suites are red here. * test: stub core's seat rules, rehome the rest with the feature A core suite can no longer build a team, so the rules that refuse a seat get `testFixtures/seatStub.ts` instead: it answers the contract, which is all those rules read, and puts the registry back when the test ends. Seventeen cases move that way. Four could not: group shares and a seat's 2FA enrolment over HTTP need the real feature and the `/teams` route, so they go with it, as does the whole of `GroupGrants.test.ts`. Two whoami cases asserted a seat's team through `extension.import`, which resolves the registry rather than the stores object. They now assert the absent case, and the behaviour they covered is asserted against `buildUserDetails`, which takes its stores as an argument. Suite: 12252 passed, 38 skipped. * refactor: take the teams console out of the built-in tabs `TabTeams`, `teamsConsole` and `teamActionIcons` are no longer built in. The dashboard already invites a tab through its `dashboard-will-open` event, so whatever provides Teams contributes the console the same way. Its 113 strings go with it and register into the shipped English dictionary at load, so `i18n('teams_*')` resolves as it did. Only `en.js` ever carried them, so no translation is lost. Three modules stay, because things other than the console use them: `teamBadge` (the dashboard shell), `orgSeat` (Home, Account, Security) and `shareTeams` (the share picker). They already render nothing when the account has no team. `teams_account_of` stays with `teamBadge` for the same reason, and the `share_*` and `plan_team_*` strings stay with their callers. The GUI builds clean with no console at all, and builds it in when the tab is contributed. * docs: seats past the owner's current limit get no allowance	2026-10-22 15:32:39 -04:00
Daniel Salazar	d22d419c4d	chore: move to Node 26 (#4295) ... * chore: move to Node 26 Docker image and CI run on Node 26 (bookworm-slim, same Debian base as before). Node 26's bundled undici reads the global dispatcher from the `undici.globalDispatcher.2` slot, so the guard's 10-minute timeout wrap on `.1` no longer reached plain fetch() and provider calls fell back to the 5-minute headers timeout. The guard now wraps every present slot and tracks wrapped dispatchers in a set, since compose() can return a Proxy that writes a marker property through to the original dispatcher. * test(puter-js): disable Node's web storage in test workers Node 26 defines its own localStorage global, undefined without a storage file, and vitest's jsdom environment doesn't replace globals that already exist, so jsdom tests saw localStorage as undefined.	2026-10-21 13:04:23 -07:00
Kailasnath TandDaniel Salazar	75385501a7	fix(fs): support offset and byte_count in read operations (#4266) ... * fix(fs): support offset and byte_count in read operations Allow partial content reads via offset and byte_count parameters in puter.fs.read and backend FS controllers. - In puter.js read operation, check options.offset and options.byte_count against null/undefined so offset: 0 is preserved in query params. - In handle_resp, accept any 2xx status code so 206 Partial Content responses resolve successfully. - In backend FS controllers (LegacyFSController and FSController), derive the HTTP byte range from query offset/byte_count parameters when an explicit Range header is not present. - Add unit and integration tests across controllers and SDK test suites. * fix(fs): strict range params, 416 past EOF, drop format churn - resolveReadRange accepts only decimal digit strings and safe integers - readContent maps S3 InvalidRange to 416 instead of a 500 - revert unrelated union-type reformatting, restore truncated test comment --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-10-20 21:20:51 -07:00
4 people	e83e27f306	feat(ai): edit a source video with txt2vid (Gemini Omni, FLUX 3) (#4272) ... * feat(ai): edit a source video with txt2vid on Gemini Omni txt2vid takes an `inputVideo` option (sent as `input_video`): a URL, data URI or Puter file path to an MP4, QuickTime or WebM video of up to 32 MiB. Gemini Omni receives it inline next to the prompt, which describes the edit or extension. Google enforces the 10-second limit. Models declare `videoInput` in the catalog, and the driver rejects a source video for any other model instead of letting the provider drop it. The credit hold adds 16,000 input tokens for a source video, since countTokens reports far fewer tokens than the reply bills. loadFileInput takes an optional `timeoutMs` for web URLs, and cancels the body of a failed response. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ai): edit a source video with FLUX 3 on Together AI FLUX 3 takes `input_video` as Together's `media.source_video`. A web URL goes through for Together to fetch; a data URI or Puter file is read with the caller's access, capped at 32 MiB, sniffed as a video and sent inline. Editing is estimated and billed at the 1080p rate of $0.54/s, since FLUX 3 jobs report no cost. Image inputs can't be combined with a source video. Seedance 2.0 ignores `source_video`, treats `reference_videos` as a loose reference rather than an edit, and Seedance 2.5 fails on reference videos, so neither declares `videoInput`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(ai): name FLUX 3 as a video editor in the AI overview Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ai): match Hailuo 02 to Together's price and document estimate billing Together quotes $0.56 for a Hailuo 02 clip; the catalog had $0.49. Together's video jobs no longer report what they cost, so every Together clip is billed at the catalog's highest published rate. The docs said the charge was the amount Together reports; they now describe the estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: enforce video edit durations and propagate source fetch cancellation * fix(ai): pass media inputs as Puter files and stream source videos Source videos were read whole into the API process, base64-encoded and sent twice to Gemini (countTokens and the interaction), which held about 128 MiB per in-flight edit for up to ten minutes. - Video inputs are a URL, a Puter file path or a data URI of at most 1 MiB. Together gets a Puter file as a 15-minute presigned S3 URL and fetches it itself; Gemini streams it into the request body with an exact Content-Length, so it's never held as one string. - Gemini counts only the prompt and image for a source video, and both providers read the source only after the credit hold. - txt2img and txt2vid image inputs accept Puter file paths, read within the caller's access right before the provider call. Existing URL, data URI and base64 inputs are unchanged. - puter.js writes a Blob, File or data URI over 1 MiB to a temporary file under .tmp, sends its path, and deletes it once the call settles. - FLUX 3 edits bill at the 720p editing rate, matching their 1280x704 output. * fix(ai): stage inputs after website sign-in, keep images working without a write - puter.js chose the temporary file path before signing a website visitor in, so it fell outside the app's data directory and the upload was refused. It now signs in first, then picks the path. - An image that can't be written (no access, storage full) is sent inline as before instead of failing the call. Videos still fail, since a large inline video isn't accepted. - Telling raw JPEG base64 from a path decodes only the first 24 characters instead of the whole input. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: 404oops <oops404@util.404oops.com> Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-10-19 23:33:03 -07:00
Filip Kujundžićand404oops	e950a87f29	Add Gemini Omni video generation with token metering (#4235) ... * Add Gemini Omni video generation with token metering * Keep explicit Gemini video model selection within provider * Settle reported Gemini video usage before validating output --------- Co-authored-by: 404oops <oops404@util.404oops.com>	2026-10-16 23:21:48 -07:00
3 people	1cb9dd756a	Add Gemini transcription with consistent usage settlement (#4234) ... * feat(ai): add Gemini unary transcription * Clarify transcription language results and omit pricing * style(ai): keep Gemini transcription sources prettier-clean Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ai-speech2txt): charge Gemini transcriptions that fail after Google answers Google bills an interaction it processed whether or not it completed, so an incomplete, failed or malformed reply was free to the caller and paid by Puter; audio whose transcript overran the output cap could be resubmitted at no cost. Usage is now recorded before the reply is checked, falling back to the counted audio tokens when the reported usage can't be read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Preserve transcript text with optional malformed annotations * Keep speech provider spacing consistent --------- Co-authored-by: 404oops <oops404@util.404oops.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-16 23:21:47 -07:00
404oops	7abd7b4417	Remove unsupported speaker reference option	2026-10-13 20:43:54 +02:00
404oops	29e0e1e2aa	Clarify speech catalog units and cover ignored transcription options	2026-10-13 20:43:54 +02:00
404oops	f9cde23937	Merge branch 'filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so' into filipkujundzic/put-1907-ai-streaming-buffering-and-error-shape-cleanup ... # Conflicts: # src/backend/drivers/ai-chat/utils/inlineImages.test.ts # src/backend/drivers/ai-chat/utils/inlineImages.ts # src/backend/drivers/util/fileInput.ts	2026-10-12 12:39:35 +02:00
Daniel Salazar	d8f9aff3dd	fix: follow-ups to #4161–#4203 (rate limits, broadcast, AI routing and billing, fs subtree walks) (#4205) ... * fix(rate-limit): count the newest KV rate rows first The KV backend's rate check, peek and concurrent acquire list the key's rows with a limit + 1 cap that the store applies before dropping expired rows, so unswept rows from past windows (oldest keys) could fill the fetch and the live count read as zero. They now list newest first. * fix(broadcast): split webhook flushes under a byte budget, per-peer backlogs Flushes go out in bodies of at most `broadcast.outboundMaxBodyBytes` (default 4 MiB) instead of one POST, and each peer keeps its own backlog of what it missed, capped by count and four bodies' worth of bytes, oldest dropped first, so one failing peer can no longer grow every peer's body past their 50 MB limit. * fix(ai-chat): classify dropped upstream connections as provider failures A connection reset or refusal (SDK APIConnectionError, Mistral's ConnectionError, an undici reset on the cause) carries no status, so a chain of them fell through to 500 internal_error. Attempts now record connectionFailed; a chain of only resets and 5xx answers 400 upstream_provider_unavailable, and a reset counts as an upstream signal in a mixed chain. * fix(ai-chat): keep SDK retries on Anthropic count_tokens The Claude client is built with SDK retries off so the driver can fall back, but count_tokens has no fallback route. It now passes sdkRetryOptions, like the OpenAI moderation call, so a transient 429 or reset is retried. * fix(ai-chat): mark a route unhealthy for 60 s after a transient failure With SDK retries off, one 5xx, 429, timeout or dropped connection kept a route marked for 10 minutes, sending that model's traffic to resellers. Those failures now mark the route for 60 s; bad credentials, an exhausted account and other route-level failures keep 10 minutes. * fix(metering): don't memoise spent addons inside the force-exact window The spent-addons memo could cache a pre-grant record read from a lagging replica right after a credits-changed announcement, so overage skipped the new credit and requireCredits kept answering 402 for up to 15 s. A spent record is now only cached once the actor's force-exact window has ended. * fix(share): send the digest entries a flush claimed when another take fails Digest entries are taken concurrently, and a single rejected take threw away every entry the other takes had already removed from KV. The flush now logs the failed takes and sends what it claimed; an entry whose take failed stays queued for a later flush. * fix(fs): announce an overwriting move's replaced entry only when it took its name /move and v2 /fs/move looked up the entry at the destination name before moving and always announced its removal. A move into someone else's Trash renames instead of replacing, so the owner's desktop hid a live entry and legacy /move reported it as `overwritten`. Both routes now pass the target on only when the moved entry kept the name it was looked up by. * fix(ai): name the vendor in upstream error fields from media providers upstreamFetch now takes the provider id plus a display label, and UpstreamResponseError carries the id. translateProviderError copies a string `provider` off the error into `fields`, and the BytePlus video and Cloudflare image errors set one, so upstream_auth_failed and upstream_credits_exhausted alarms say which vendor failed again. * fix(teams): delete a provisioned seat whose home folders could not be created provisionUser logs a failed default-folder creation and still returns the account, so provisionAccount went on to add the membership, issue a password, mail the seat and emit team.account.created for an account with no home. It now cascade-deletes the new user and returns 500 internal_error before any of that. * fix(ai-chat): load model catalogs without holding up server start onServerStart awaited every gateway catalog (up to its 10s fetch timeout) while HTTP was already listening, so drivers started after it had no providers yet and chat requests found an empty model map. The first build now runs in the background; complete(), models() and token counting wait on it instead of failing. * fix(ai-chat): serve direct models while gateway catalogs load at boot The first model-map build now publishes a fresh map as each provider's catalog lands, so static catalogs are routable at once. complete() and token counting wait for the full first build only when the model, or the provider asked for, isn't in the map yet. Refreshes still swap once. * fix(ai-chat): split gateway-billed cost across input and output A completion billed at the cost OpenRouter, Infron or Neuralwatt reports was quoted to `ai.prompt.cost-calculated` as all input and no output. The ledger still records one `billedUsage` line; the reported per-key costs now spread that bill over the usage keys at the catalog's relative rates, in whole µ¢ that sum to it. Neuralwatt energy units take no share. * fix(ai-image): hold Replicate credits for the whole prediction The output and input-surcharge holds took the default 10-minute TTL, the same as the prediction window alone, so a slow prediction could outlive them once creation and input measurement were added. They now last 15 minutes, like the video job holds. * feat(ai-chat): report what a completion was charged as usage.cost Chat usage (the non-stream result and the stream's final usage line) now carries `cost`: the µ¢ the ledger records for the request after the cost factor, from the request's recorded charges and the gate's factor lookup. `usd_cents` is unchanged. A hooked recording now copies the usage object first, so keys the driver adds later never land in the ledger. * fix(ai): charge streamed speech once its first byte arrives Per-character TTS and the voice changer recorded their charge when the upstream's response headers arrived, so a body that failed before any audio was billed in full. A shared relay now charges on the first byte, or when the caller hangs up, and charges nothing for an upstream that fails or ends before sending audio. * fix(ai): release credit holds only after the request's usage lands A hold was released as soon as its operation returned, while the usage it stood for was still being written (behind the cost-factor lookup), so a concurrent request briefly saw neither. AI metering now tracks the request's usage writes and its holds (chat included) release once those settle, without delaying the response; a streamed charge chains its own. A failed write still releases the hold, and the hold's TTL backstops it. * test(ai-chat): wait for a stream's credit hold to come back after its usage lands * perf(fs): page subtree delete and copy by parent link, not by depth over the path range Each page of a recursive delete or copy sorted the whole remaining subtree by a computed depth, so a walk cost grew with the square of its size. It now pages the children of up to 100 sibling directories at a time by parent_uid, depth first, keyset on (parent_uid, id), so a page reads only its own rows; rows with no stored path are reached too. SQLite gets the parent_uid index the others have. * fix(fs): find children by parent_uid in the empty check and supplementary-name lookup Rows the old backend wrote carry parent_uid and no parent_id, so a non-recursive delete of a directory holding only such rows passed the empty check and orphaned them, and on MySQL the supplementary-character child lookup missed them, leaving their subtrees out of listings, counts and renames. Both now key on parent_uid; the empty check still probes parent_id too, which the delete cascade follows. * docs(fs): drop the single-POST note from the entry broadcast cap	2026-10-12 01:10:54 -07:00
404oops	64db150a90	fix(ai): bound streaming buffers and normalize input errors	2026-10-11 21:14:16 +02:00
Daniel Salazar	360f6b1079	fix(events): bill reached deliveries, bound dispatch reads, seal cursors (PUT-1900) (#4146) ... * fix(events): bill reached deliveries only, bound dispatch reads, seal cursors - A broadcast socket copy is billed only when a connection for its holder exists on this node, anywhere in this region (connection count) or, for a durable row, in a region presence names. Without peers it bills as before. - Compiled match filters are held in a bounded LRU with a TTL, and dropped when another process reports a subscription ended. - GET /events/fetch reads the continuation from `cursor`, falling back to `after`. - Cursors from /events/fetch, /events/subscriptions and /events/kv-handles are encrypted with sealCursor/openCursor; plain cursors still read. - A durable subscribe counts again after its insert and removes its own row when over the cap; the generation bump for that removal is published. - Subscribe checks access before reading the anchor's owner, and an anchor gone in between answers with the subject the caller sent. - Dispatch reads at most the rows per token routing can use, scanning large token hashes; a removal still settles every row on the removed anchor. - puter.events.unsubscribe() stops routing after the server ends the subscription or reports it gone; fetch() accepts `cursor`. * fix(events): forward kv values only to regions with rows that ask - A region counts its session rows asking for kv values per token and announces the token's value variant (`v#<token>`) over the existing `watch` item on the first such row, withdrawing it with the last. Add, remove, reap, reanchor and refresh all keep it, reading `includeValue` off the row as it is dropped. - A kv write reads the value variants alongside the watch index and attaches the value only to forwards bound for a region that announced one. * fix(pagination): seal id-keyed cursors on every list endpoint Apps, subdomains, workers, team (members, directory, teams, audit, own audit), shares (inbound, outbound), the user-to-user permission audit and readdir/descendant listings now hand out cursors sealed with sealCursor and read them with openCursor, so a cursor no longer carries a row id. Plain cursors issued before still read. * test(events): pass socket ids to noteConnect in the billing tests	2026-10-09 00:08:13 -07:00
Juan Fernando Castro	d1d55a6bc3	SDK auth plumbing: cache scope, dedupe keys and the token listener (#4112) ... * fix: pin the token sender, scope the grant flag, and stop two stragglers Five of the SDK auth-plumbing items. **The global `puter.token` handler pinned origin but not source.** Its three siblings all pin both, and each says why: the GUI origin is shared by every frame on that domain, so origin alone identifies a domain, not a sender. `signIn` pins its popup and binds `msg_id`, the consent dialog pins its popup, and the reauth wait pins the embedder. This one pinned the embedder only when `env === 'app'`. Since `authenticateWithPuter` now hands off to `signIn`, which adopts the token itself, the embedder is the only sender this handler is left serving, so it pins in every environment. **`is_grant_user_app_permission` was a request-wide flag held across an await.** It tells the `app-root-dir` rewriter that a row is being written, and while it is set that rewriter resolves to a real `fs:` permission instead of `PERMISSION_FOR_NOTHING_IN_PARTICULAR`. Anything evaluating an `app-root-dir:` permission beside the grant -- the grant route runs one under `Promise.all` with fs work -- read the same flag. It now lives in a scope derived for that call, so what the write needs to say about itself is not also said to its neighbours. `runInDerivedContext` is new: `runWithContext` starts an empty store, which would drop the `actor` the rewriter reads. **`fs.stat` and `fs.readdir` deduped on parameters alone**, so two identities in one realm could share a result. They now key on origin and token as `os/user.js` does, which documents the reason. **`signIn`'s relay poll never stopped.** Closing the popup settles the promise; the loop kept asking `/login/wait` every second for the life of the page. **`triggerReauth`'s five-minute timeout was never cleared**, so a successful reauth left a timer to fire into a settled promise. The concurrency test parks the rewrite and reads the flag while it is held -- my first attempt read before the flag was even set and passed either way. * fix: scope the fs cache per identity, and retarget two paths that never ran The remaining SDK auth-plumbing items. **`puter._cache` outlived the identity that filled it.** `_clearAuthToken` dropped the token and the stored session but left the cache and `whoami` as they were, and its keys were raw paths -- `item:/alice/Documents` and a `~`-relative key alike, identical whoever asked. A GUI user switch without a reload could be served the previous user's stat or readdir. Keys now carry the origin and token they were read as, built in one place so the writers, the readers, the invalidations and the warm-up probes cannot drift apart, and signing out drops the cache and `whoami` with the token. **The driver permission prompt could not fire.** Both paths waited for a 200 carrying `{success:false, error:{code:'permission_denied'}}`. The driver route answers `success: true` or throws, so a refusal arrives as a 403 -- the shape the retry now reads. Its two tests synthesised the old envelope, so they passed against a response the API does not send; they use the real one now, and the prompt goes from never called to called once. This is a behaviour change: a driver call refused for permissions will now prompt, which is what the path was written to do. Happy to delete it instead if the prompt is not wanted. The half-written copy of the same idea in `utils.js` is gone -- it prompted for one hard-coded interface and its retry was a `// todo`. **The verification-gate single-flight was keyed on nothing**, so a phone-gated request in flight beside an email-gated one took the email answer, replayed, failed again and surfaced raw without ever asking. Keyed per gate: same gate still shares one dialog, a different gate gets its own. * test: ask the cache the same way the code does The API suites built `item:<path>` keys by hand, so they stopped finding anything once the real keys carried the identity they were read as. They go through `fsCacheKey` now, which is the point: a test that spells the key itself passes until the moment the spelling changes, and then fails for a reason that has nothing to do with the behaviour it is guarding. These run against a live server, so a plain `vitest run` does not reach them -- they were green locally and red in CI. * fix: the review's findings on this branch Eight, and three of them were regressions this branch introduced. **Pinning the token sender to `globalThis.parent` in every environment broke the case it was meant to protect.** On a top-level page `parent` is the page itself, and the token arrives from a popup, so the handler stopped accepting anything. The GUI posts `puter.token` to `window.opener` for the file pickers too, not only for sign-in, and this handler is their only consumer -- a site calling `showOpenFilePicker()` would have signed the user in, dropped the token and run the following read with none. It now pins the embedder when framed and a window this SDK opened when not; `UI.js` already tracked its picker popups for the same reason, so that set is shared rather than duplicated. **The GUI still spelled a cache key by hand.** `updateSubdomainsForItems` built `item:<path>`, which no longer matches what the SDK writes, so its `get` could not hit and its `set` wrote an orphan nothing reads and nothing evicts -- a published website's badge quietly stopped appearing on a cached render. It goes through `fsCacheKey` now, and that is the last hand-spelled key in the tree. **Retargeting the driver permission prompt to 403 was wrong and is reverted to the deletion the ticket asked for.** The route refuses with `forbidden`, not `permission_denied`, so the retarget still missed it; meanwhile a driver that raises `permission_denied` for its own reasons would have opened a prompt that had nothing to do with the refusal. And the prompt asks for `driver:<iface>:<method>` while the route checks `service:<driver>:ii:<iface>`, so granting it changes nothing. The path cannot work as written. The rest were pre-existing or latent: `cacheWhoami_` wrote its answer without checking the token it was issued under was still current, so a reauth landing mid-flight could repopulate `whoami` with the previous identity. `_dropIdentityCaches` also left `whoamiCache_`, which other call sites await. `setAuthToken` now drops the identity caches when the token actually changes: keys carry the token and the store is persisted with no TTL, so a rotation that did not go through `_clearAuthToken` orphaned a whole key space with nothing to reclaim it. The verification-gate entry was registered after the body could already have deleted it -- a synchronous throw stranded a resolved "not verified" in the map for the life of the page. It registers first and clears on a microtask. `signIn`'s poll is bounded as well as settle-checked: on a cross-origin isolated page opened from a gesture, nothing watches the popup, so closing it never settled the promise and the loop ran on exactly as before. `runInDerivedContext` says in its docblock that callee writes do not survive; only the one flag needs the scope, and a future caller should not expect to hand a value back through it. * fix: the review's findings on the SDK auth plumbing **The relay bound stopped asking without saying so.** The deadline left the `signIn` promise pending and its listener attached. On an isolated page opened from a gesture the relay is the only thing that can settle it -- nothing watches the popup there -- so a user who took longer than the deadline was left with a promise that never resolved, and `authenticateWithPuter` queues every later implicit-auth call behind it. It rejects now. **Pinning the token source left `signIn`'s own popup untracked.** Only the picker popups registered, so on a top-level page the global handler bailed on a sign-in it had opened itself: `puter.onAuth` never fired, and `puterAuthState.authGranted` was never set. `openAuthPopup` is the one place auth popups are opened, so they register there. **A closed popup is ours too.** Rejecting one that had already closed loses a message from a popup that posts and then closes itself, which is why `UI.js` tolerates the same case. Kept, with the set bounded instead. **The gate key carried the factors**, so one route naming them and another not opened two dialogs for the same gate. It keys on the gate, as `ctx.done` does. **`fsCacheKey` left `~` unresolved**, so a writer and an invalidation spelling the same place differently missed each other -- the drift the one key builder was meant to end. The dead `opts.permission` went with the path that used to read it. Four workerd-runner failures are left: `ai > chat stream adapter supports cancellation`, `kv > a GUI boot key written by another client reads fresh` and two cache-warmer ones. All four fail the same way on `origin/main`, and the node and browser runners pass. * chore: one-line the comments added here * chore: reflow the comments the removed retry cause left ragged	2026-10-08 15:04:49 -04:00
Filip Kujundžić	b6150d63f3	Merge pull request #4090 from HeyPuter/fix/ai-stream-cancellation ... fix(ai): propagate cancellation and stream termination failures	2026-10-08 13:33:50 +02:00
Daniel Salazar	2638e9ce93	fix(kv): batch-read GUI boot keys only when the SDK runs as the GUI (PUT-1899) (#4117) ... The boot-key batch now starts only when puter.env is 'gui'. In every other environment a get() of one of those names reads that key on its own, like any other key, instead of fetching all 14 and serving them for 4 s.	2026-10-07 22:34:22 -07:00
Daniel Salazar	fde323842e	fix(kv): SDK list rejections, boot-cache read-after-write, byte-accurate size checks (PUT-1899) (#4108) ... - puter.kv.list() no longer throws synchronously for invalid options; it returns a rejected promise, or for stream: true an iterator whose first next() rejects. Error codes unchanged. - Writes through puter.kv (set, del, incr, decr, add, remove, update, expire, expireAt, flush) stop the boot-key batch from serving the keys they touch, so the next get() reads the store. Untouched boot keys are still served from the batch. - Client-side key/value size checks measure UTF-8 bytes, and values are measured as their JSON encoding, matching the store. Same codes and messages.	2026-10-07 11:48:45 -07:00
Daniel Salazar	4a54beac6b	fix(fs): SDK uid stat cache, readdir page types, move lookup errors, UID-shaped paths (#4109) ... - stat: only path-addressed results are cached; uid-addressed stats no longer share one cache key. - readdir: overloads declare the page result for recursive + limit/offset and for the positional page/stream forms. - move: only a not-found destination lookup means "new path"; other lookup failures reject. A uid destination skips the lookup. - getAbsolutePathForApp always resolves paths. read/copy/move/delete use getAbsolutePathOrUidForApp, which keeps sending UID-shaped strings as uids. - Docs for read, copy, move, delete and readdir.	2026-10-07 11:46:22 -07:00
404oops	96f0c51d1e	fix(ai): propagate cancellation and stream termination failures	2026-10-06 20:24:19 +02:00
Daniel Salazar	a242e31343	feat(ai): carry the full Anthropic and OpenAI wire shapes through the chat driver ... The Anthropic-compatible route flattened requests into the OpenAI-style driver shape and rebuilt them for Claude, which dropped system blocks, thinking, tool_choice, cache_control, betas and stop reasons. Claude Code requests 400ed upstream and were silently served by a reseller. - normalized driver args, stream chunks and results gain tool choice, thinking, output format, stop sequences, caching, stop reasons, usage details, reasoning signatures, server-tool blocks and safeguard results - ClaudeProvider builds requests from those fields with a beta, tool and body-field allowlist; meters fast mode, web search and advisor calls, and no longer bills thinking tokens twice - fallback only on route-level failures; outages map to 502/529 - vendor error envelopes on the Anthropic and OpenAI routes - Anthropic route: native SSE with pings, count_tokens, /v1/models - OpenAI-family providers and routes map the same fields; Responses web search is metered; reasoning_effort gating fixed; /responses rejects org-scoped stateful params and forces store:false - puter.js: typedefs, new chat options, silent toString for new chunks	2026-10-05 03:35:54 -07:00
Juan Fernando CastroandDaniel Salazar	a46eb15d5e	Let the account's own API token reach the routes a privileged app needs (#4032) ... * feat: let the account's own API token reach the routes it should A privileged app is launched with the GUI session token today. Running one on a full-access personal access token instead costs much less when it leaks, but several routes those apps depend on refuse every access token, so the swap would break them. Admitted, each a read or a write the token's own HTTP reach already covers: - the realtime socket handshake, which refused access tokens outright; FS live updates and notifications stop without it - GET /get-dev-profile and POST /profile, both puter.js surface - GET /auth/list-permissions — a read of what was granted, not a grant - GET /share/shared-by-me/apps and GET /share/audit, over grants /share/shared-by-me already lists for the same credential - POST /rao, which puter.js calls on every setAuthToken Gated on `isAccountContext`, not the `full_access` claim alone, so a token with an app anywhere in its chain is still refused and a socket for one never joins the user room. Left session-only: mint, session, 2FA, grant, team and billing, plus two the list named. /share/blocks is a personal safety control whose only caller is the desktop's Blocked Senders window. /app-feedback is reachable only from our own GUI pages, and being unsubmittable programmatically on a user's behalf is the property it was built with. The new HTTP suite drives a real server with a minted token and fails on all six admissions without the change; the block routes' existing metadata check already pins their refusal. * test: sort the two token shapes at the events handshake * fix: carry the account-context invariant into the HTTP gate `requireUserActorGate` admitted on the raw `full_access` claim while the socket and /rao use `isAccountContext`, so one shape — a token carrying the claim with an app in its chain, or an actor built without `makeActor` — would have been refused in two places and admitted in the third. Unreachable today, since the mint and the auth-time read both drop the claim when `app_uid` is present, but it is the invariant this change states and it should hold at every enforcement point. The payout address goes back behind a plain session: `/get-dev-profile` is open to the account's own token now, and the rest of what it answers is a developer's own name and incentive-program status. Also drops a cross-reference in EventsService to the socket posture this change relaxed, and rewords the block-routes comment, which read as if access tokens were admitted there rather than refused. * fix: drop a revoked token's sockets, and read the dev-profile columns Two gaps the review found, both reachable only because this branch lets the account's own token hold a socket and call /get-dev-profile. `#revokeAccessTokenTail` soft-revoked the session row and told no one, so a revoked token kept its connection — and the account's `outer.gui.*` fan with it — until the five-minute reauth sweep. Sockets now join a per-token room and a new `auth.access-token.revoked` event drops exactly that room, so the account's other tabs stay up. Session revoke keeps its account-wide eviction. `/get-dev-profile` read `first_name`, `last_name`, `paypal` and the two incentive flags off the user row, which carries none of them: the columns have always been `dev_`-prefixed, so the endpoint answered nulls to everyone and `puter.apps.getDeveloperProfile()` has never returned anything. Reads the real columns, keeping the unprefixed name as a fallback for a deployment carrying both. The payout address stays behind a plain session, so opening the route to the account's token hands over a name and incentive status and nothing else. Both fail without the fix: the socket test waits five seconds on a connection that should already be gone, and the profile test reads back a seeded row. * fix: close /open_item to tokens, and stop the node runner leaking sockets `/open_item` carried `allowFullAccessToken`, but it grants an app a permission on the user's behalf and mints the app token to go with it — a grant and a mint, which stay session-only. Pre-existing, and missed by the first sweep here, which only asked whether routes lacking the flag should gain it and never re-read the ones that already had it. Refused at the top of the handler too, so a credential that gets past the gate cannot leave an ACL row behind. The node runner loads each SDK into a vm context and never closed it, so every test leaked a socket, and every socket holds a per-(user, origin) connection slot. Admitting the account's own token to the handshake pushed that past the cap, and the whole events block failed with `events_connection_failed` while the browser and workerd runners — which do not leak — passed. Closing the FS socket and the events channel after each test gives the slots back. * fix: announce only the revokes that can have a socket to drop Every access-token revoke broadcast an eviction cluster-wide, including the read-URL tokens `revokeReadUrl` retires constantly. A scoped token is refused at the handshake, so none of those broadcasts could ever reach a connection. The announcement now goes out only for a token the handshake would have admitted: full access, no app in the chain. `revokeOwnAccessToken` refuses full-access tokens outright, so that whole path is silent; a revoke by raw uuid says nothing about the token and still announces, which costs a no-op broadcast rather than leaving a revoked socket up. Fails without it: the test revokes a scoped token and a full-access one through the same entry point and counts the announcements. * test: pin the mint route against the account's own token `/auth/create-access-token` carries only `requireAuth`, so it never showed up in the sweep over `requireUserActor` routes — the refusal lives in `AuthService.createAccessToken`, which turns away any access-token actor outright. That holds today, and a leaked full-access token minting itself a sibling that survives revoking the original is exactly what the token swap is meant to rule out, so it is worth a test rather than a reading. * fix: let the recommended-apps test follow the list it tests Two commits on main rewrote `RECOMMENDED_APP_NAMES` without touching the test, which seeds `editor` and `camera` and asserts the result contains them. Neither is on the list any more, so the resolved set came back empty and `test (base)` has failed on every PR since. The test now seeds the first two names off the list itself and asserts on those, so the next edit to it cannot strand the test: prepending two unknown apps keeps all four cases green. * feat: let an extension waive a plan gate for one route or driver method A plan gate about to refuse a caller now emits `subscription.gate.<surface>` through `emitAndWait` first: `route.<method>.<path>` for a route, `driver.<iface>.<method>` for a driver method, `share.anyone` for link shares. The payload carries the actor, the request and the requirement, and a listener that sets `allow` waives the plan for that call. Callers the plan already covers never reach the hook, and a listener that throws leaves the refusal in place. * feat: mint godmode apps their own full-access token `/auth/get-user-app-token` answers a desktop launch of a godmode app (the `godmode` column) with a full-access token instead of an app token. Origin lookups, the sign-in popups for pages outside the desktop, keep getting ordinary app tokens, and only a web session may mint one. The token carries the app in `godmode_app_uid`, read into `accessToken.godmodeApp` for attribution only: `effectiveApp` stays null, so it keeps the account's reach and passes the same gates as a personal access token, including the vendor-compatible AI routes. It is refused once the app is gone or no longer godmode. Its row is an `access_token` session parented to the web session that asked, with `app_uid` set, so signing out, revoking that session or revoking all sessions takes it along, and the sessions list shows it under the app. Each mint re-signs onto the same row with a 12h `exp` and moves the row's expiry out, so the desktop keeps an open app alive while a copy taken earlier still expires on schedule. A full-access token may now mint scoped tokens (read URLs), parented to its own row; it still may not mint full access. Revoking a full-access token cascades to the tokens it minted. An access token whose session row has a parent now authenticates only while that parent is live, so a read URL a godmode app minted lapses with the app's token, and through it with the desktop session. * feat: launch godmode apps on their own token and renew it The desktop asks for a godmode app's token at launch and blocks the launch if it can't get one, as it does for other apps; a godmode app never runs on the desktop's session token. While the window is open the desktop re-mints the token once less than 6h of its 12h remain, checking on a timer and when the tab becomes visible, and answers the app's `reauth_required` with a fresh one, posted to the app's origin. puter.js keeps a godmode token in sessionStorage like a session token, never in localStorage. In app mode a 401 on a godmode token, background requests included, asks the desktop for a new token and replays, and a `puter.token` message is only taken from the embedding frame. The sessions list shows a godmode token row with its app's title and icon. * fix: keep a godmode token the same across renewals Apps hand their launch token to code that never sees a replacement (the terminal's Drive mount, a CLI's API key), so a token re-signed every renewal broke those copies once the first one expired. The JWT now carries no `exp` and a renewal moves only its row's expiry, so every copy keeps working while the desktop keeps the app alive. What still bounds it: - only the desktop's web session can extend it; using the token never does, and signing out or revoking that session revokes it; - it lapses 12h after the desktop last asked, and a lapsed row is never revived: the next launch gets a new token; - however often it is renewed, a token is replaced after 7 days, and the old one is revoked on the spot along with the tokens it minted and its sockets. The desktop no longer re-posts a renewal that kept the same token, so the app's sockets aren't rebuilt every few hours; an app that reports its token stopped working is always answered. * fix: reserve the feedback and no-reply mailbox names again Signup checks names against `util/reservedUsernames`, but `fbl`, `noreply` and `no-reply` were added to a second copy of the list in AuthController that nothing reads any more, so they became claimable. Move them into the live list and drop the copy. --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-10-05 01:10:29 -07:00
Filip KujundžićandClaude Opus 5.5	82bd7a7360	chore(ai): sync model catalogs with provider listings and deprecations ... Remove models providers have deprecated or retired, add newly released ones, and move provider defaults off retired models. Gemini video (Veo via the Gemini API) is removed; Infron and OpenRouter skip models their listings mark deprecated. xAI bills cached prompt tokens only at the cached rate. Claude Sonnet 4.6 max output raised to 128K. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-02 18:52:35 +02:00
Daniel Salazar	91c8b2385b	fix: run a broadcast persistent handler in the worker or the clients, not both (PUT-1889) (#4016)	2026-10-02 00:08:02 -07:00
Daniel Salazar	ed7b8f828a	fix: kv path errors and ttl hardening, events subscription end and race fixes (#4004)	2026-10-01 09:50:34 -07:00
Daniel Salazar	4448103ef0	fix: misc events and kv issues found during recipes (#3997) ... * fix: misc events and kv issues found during recipes * test: cover handler depth on narrowed actors and created notifications * test: fix stale legacy batch directory upload test and seat user-agent flake * test: wait for the refusal to be recorded before claiming its gap marker	2026-09-30 14:40:47 -07:00
404oops	0b1a4283f5	fix(ai): honor chat test mode	2026-09-29 12:17:33 +02:00
404oopsandClaude Opus 5.5	d36da7522b	fix(ai-ocr): address review: playground examples, Textract options, inline size ... Adds the Mistral and document-annotation img2txt examples to the playground, states that Textract takes no options, and drops the normalize mentions from the img2txt docs and types. The SDK now checks Mistral inline sources against the size a base64 upload can reach inside the 50 MB JSON body (36 MB) instead of the 10 MB Textract limit; 50 MB let 38-50 MB inputs through to a body-parser 413. Retired-model messages no longer cite vendor dates. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-25 18:56:29 +02:00
404oopsandClaude Opus 5.5	eef8258539	fix(ai-ocr): sync OCR models and prices, route by model, cap input size ... Model sync against Mistral's listing, docs and a live call per id: mistral-ocr-latest now serves OCR 4.1, so the catalog pins it (with mistral-ocr-4 and the retired-but-redirected 2503) to mistral-ocr-4-1, adds mistral-ocr-4-0 and OCR 3 (mistral-ocr-2512 / -3 / -3-0), and rejects the retired mistral-ocr-2505 before any upstream call. Mistral usage is metered per model at vendor rates: $4/1000 pages for OCR 4.x with $5 annotations, $2 and $3 for OCR 3, replacing the flat $1 and $3. Textract now calls DetectDocumentText instead of AnalyzeDocument with LAYOUT. The layout feature cost $4/1000 pages while we billed the $1.50 detect rate, and it only added text-less LAYOUT_* blocks that put blank lines into img2txt output. Multi-page PDFs and unsupported formats become a 400 that names Mistral as the alternative. Request shape: a model name alone now selects its provider, so calls differ only by model; an explicit provider still picks its default and must agree with the model. Mistral options gain tableFormat, extractHeader, extractFooter and documentAnnotationPrompt; annotation formats accept the REST (json_schema.schema) or SDK (jsonSchema.schemaDefinition) spelling and are validated before billing. Non-image documents such as DOCX go as document_url. Response shape: both providers return model, text and blocks with a 0-based page; Mistral also returns document_annotation, which was billed but dropped. img2txt resolves to that annotation when documentAnnotationFormat is set and to the text otherwise; the return is a string for every model, so normalize has no effect on it. Input size: OCR input is capped at the provider limit (10 MB Textract, 50 MB Mistral), and web inputs in loadFileInput are streamed under the cap instead of buffered whole, which also bounds speech2txt URLs. Docs: img2txt models, options, input limits, return value and errors; OCR limits in the quotas page. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-25 18:19:54 +02:00
Daniel Salazar	132cae9f6a	Bug bash fixes (PUT-1876, 1877, 1880, 1881, 1948, 1949, 1951) and KV event fan-out by plan (#3943) ... * fix(kv): keep disableSharing entries out of other apps' events (PUT-1876) Private kv keys and values are no longer delivered to cross-app subscriptions, including forwarded and durable deliveries. * fix(fs): refuse usernames whose home path still holds another account's rows (PUT-1877) Username claims (signup, save_account, change-username, OIDC, team provisioning) now also check for leftover rows under /<name>/, and the root listing only returns the actor's own home. * fix(puter-js): reconnect the events channel after a server-side disconnect (PUT-1881) A server hang-up is retried with a bounded backoff; persistent handlers survive it and get an optional onError when the connection can't be restored. * fix(backend): let the graceful shutdown finish before telemetry exits (PUT-1948) The telemetry preload no longer exits on signals; the entry point drains, tears down top-down, flushes telemetry last, then exits. DB pools stay open until teardown. * fix(fs): reserve quota for signed uploads in progress (PUT-1880) startWrite/startBatchWrite hold each upload's declared size in a per-owner cache reservation until it completes, aborts or expires, so a burst of starts can't all read the same committed usage. Caps in-flight uploads at 10,000 per account. * fix(fs): refuse to complete a signed upload whose object never arrived (PUT-1951) Completion now 400s when the object store reports the key missing (transient errors stay lenient), abort no longer deletes an object a live entry still uses, and a ghost entry is only removed after a second miss at its recorded location. * fix(metering): move usage to v2 keys with a small totals item, per-model detail shards and a 3-month TTL (PUT-1949) The billed item holds only totals; per-model detail lives in 100 hash-sharded items read in one batch and cached for 60s, and more than 5,000 distinct usage types in a month fold into other. Global and per-app aggregates store totals only, appTotals comes from a prefix listing, a refused path only skips the key that refused it, exact reads near the allowance are throttled per key, and every metering record expires after 3 months. September usage restarts at deploy; the monthly-charge claim stays on the v1 key through September so recurring charges don't fire twice. * test(events): let the worker backoff test tolerate a loaded run * feat(events): scale KV event fan-out with the key owner's plan A KV change delivers to up to 512 matching subscriptions (2,048 filter evaluations) for a paid key owner and 128 (512) otherwise, inline values are dropped once more than 128 match, and paid accounts may hold up to 512 share handles per app. The owner's plan is only looked up when a change has more candidates than any cap. * test(metering): seed the 150-app listing test directly so it fits under coverage	2026-09-24 16:00:52 -07:00
Daniel Salazar	b3fda186e7	fix: let apps read opted-in teams, scope worker listing, fix revokeReadURL (#3933) ... - GET /teams and /teams/:uid/members admit apps for teams whose owner opened the directory; apps get active members' usernames only - events.workers.list() from an app returns only its own worker - hide account-session-only SDK methods from the public types - unpublish the updateProfile docs page (redirects to getProfile) - revokeReadURL() works through a new /auth/revoke-own-access-token route	2026-09-23 12:06:56 -07:00
Daniel Salazar	1261976088	feat: user profiles in a system-owned store, public only for paid accounts (#3919) ... - ProfileService keeps each profile as /system/profiles/<uuid>.profile, admin-owned and served by the protected puter-profiles subdomain - GET/POST /profile; puter.auth.getProfile/updateProfile, with getProfilePicture reading through them - another user's profile is served only while its owner is on a paid plan (profileGate.enabled kill switch); the hosted file is gated through the new site.access.check hook the hosting middleware asks before streaming any file - SubdomainStore.create takes isProtected - the GUI reads and writes the profile through the SDK - docs: getProfile, updateProfile, UserProfile, limits PUT-1859 PUT-1860 PUT-1861 PUT-1862	2026-09-21 23:28:56 -07:00
404oopsandClaude Fable 5.1	027e9a71f3	fix(ai-image): refresh model catalogs, unify txt2img requests, expand docs (#3889) ... * fix(ai-image): sync image catalogs, unify request shape, document models Model sync against every vendor listing and a live generation sweep: Gemini stable ids replace the retired preview spellings (2.5 Flash Image delisted ahead of its 2026-10-02 shutdown), OpenAI gpt-image-1/-1-mini/-1.5 are delisted but routable until their shutdown dates with gpt-image-2 as the default, xAI gains grok-imagine-image-2.0 with its quality tiers, BytePlus gains the 3K/4K tiers and per-model pixel bounds, Cloudflare gains SDXL Lightning/Base and SD 1.5 Inpainting, Replicate gains a schema-driven catalog of 88 additional models with version-pinned community predictions, and every Together image route is excluded for the third-party data-sharing requirement. Request normalization: the driver collapses ratio/width/height/aspect_ratio into one imageSize with aspect-versus-pixel intent, validates prompt, quality and resolution once, resolves provider hints (short names or full driver ids) and aliases with exact ids winning over resellers, and hands each provider an immutable copy of the caller's args. Providers share prompt validation, aspect snapping, and content-sniffed data URIs so responses carry the right MIME type. Replicate predictions are created once, cancelled on abort or deadline, and bounded in input fan-out. SDK: txt2img copies caller options, rejects blank prompts with prompt_required in every call form, and documents that normalize has no effect because the image return shape is already uniform. Docs: txt2img options per provider, provider defaults and discovery, availability notes, a new image model catalog and pricing page, and the image-generation limits in the quotas page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(ai-image): sniff SVG outputs correctly, reject hex/exponent dimensions imageDataUri only decoded the first 24 base64 chars (18 bytes) before sniffImageMime, which cannot see an <svg> root behind an XML prolog - SVG outputs from recraft-v*-svg models were being labeled image/png. Decode enough of the payload to cover the 8 KB SVG sniff window. dimension() accepted string number literals ('0x10', '1e3') as if they were decimal dimensions; restrict to plain decimal notation. * docs(ai): link to the model directory instead of a static catalog The image model catalog and pricing page duplicated the always up to date model directory at developer.puter.com/ai/models. Link to the directory from txt2img-related docs and keep the Together data-sharing exclusion note self-contained. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-21 15:00:02 -07:00