75 Commits

Include renames

Author	SHA1	Message	Date
Daniel Salazar	d3869b7846	feat(notifications): dismissal across devices, high priority, buttons, schedule updates, docs fixes (#4501) ... - Docs: status(), browser limits, events worker fetch/ack, update() across pages, stored-notification retries, device removal, push delivery, queue coalescing, existing service workers and missing error codes match the code. - puter.notifications.markShown(uid) / dismiss(uid). - Dismissals reach the app's live feed (on('dismiss')); pages close the notification on screen, and a later page load closes ones already dismissed. The desktop closes its own copies on notif.ack. - Live notification postings are forwarded to sessions in other regions. - notify({ priority: 'high' }). - onPush() results accept requireInteraction and actions; clicks carry action. - replaced reaches service-worker onPush() handlers. - puter.events.updateSchedule(scheduleId, changes), with the invoker re-checking a schedule run's guard right before it is sent.	2026-11-05 00:17:27 -07:00
Daniel Salazar	91230697cd	fix(backend): shutdown hooks don't skip each other, and final flushes wait for in-flight work (#4499) (#4500) ... A throwing prepare/shutdown hook stopped the loop, so shutdown() skipped closeAllConnections and every later hook, including the final metering flush; each hook now runs and failures are logged. Broadcast, events forwarding and share digests joined nothing already in flight and could drop what was queued behind it; their final flushes now wait for it, ship the tail, and drain once more at shutdown. Local worker disposal and the OpenAI image metering call are awaited. settleWithin and FINAL_FLUSH_TIMEOUT_MS bound each final flush.	2026-11-04 21:47:35 -07:00
Juan Fernando Castro	a9a8df6409	feat(events): fetch can start at the newest notification, and the feed uses it (#4479) ... * feat(events): fetch can start from the newest event GET /events/fetch takes from=end: with no cursor it answers an empty page whose cursor sits at the newest row of the slice, so later reads return only rows created or replaced after it. puter.events.fetch() passes `from` through. Adds a (user_id, audience, app_uid, id) index on notification so the newest row of a slice is one seek. * fix(puter-js): onPush starts its feed from the newest notification A notifications listener started without catch-up asks for from: 'end' and keeps the cursor it gets back instead of reading every stored notification to mark them as seen. A server that ignores `from` answers with the oldest page, and the listener reads through it as before. * fix(events): read the newest notification id from the primary * fix(events): mark the end cursor at the slice's latest replacement The end cursor's replacement mark is the newest (replaced_at, id) in the slice rather than the serving node's clock, since replaced_at is stamped by whichever node wrote it. * docs(events): describe the end cursor's replacement mark	2026-11-04 10:15:14 -05:00
Daniel Salazar	23782dec12	feat(events): inline handlers in the user's own scope for onSchedule and onPersistent (#4487) ... * feat(events): schema for user-scope handlers Add event_user_handlers and event_subscriptions.handler_scope in all three dialects. event_handlers is unchanged. * feat(events): scope the handler and durable subscription stores Handler store methods take a scope (app, or a user's scope of an app) and read the matching table. App-scope behavior, SQL and cache keys are unchanged; user scopes get their own cache keys, a 20-handler cap, and the inline @<sha256> name. Adds user-scope listing, counting, deletion and unbound-handler reads. Durable subscriptions store and return handlerScope, and listByHandler filters by scope and holder. * fix(events): hold the user-scope handler cap under concurrency and drop all scope caches on app delete A create that lands past the cap removes itself and is refused. deleteForApp drops each holder's cached set in batches, here and on peers. Batch scans over user scopes and unbound handlers take limit and cursor. * feat(events): set a CPU limit on events worker deploys * test(events): cover redeploy and resend of an evicted events script * test(events): name the deploy-on-miss test for what it checks * fix(events): keep a deploy-on-miss call inside the delivery lease * feat(workers): cap rented units per plan and refuse only new ones * feat(events): publish handlers into a user scope, with inline create * test(events): cover user-scope publish, inline create and listing * test(workers): name the credentials gate without a vendor * feat(events): run user-scope handlers as the holder A user scope deploys one script per handler under the holder's own identity, throttled per scope, with an eager deploy after publish that skips an upload another holder already made. Delivery and schedule runs address the holder's handler, carry the holder on the invoke headers, batch calls, the miss handler and the rehydrate route, and stay owed while the flag is off. Suspensions of a user scope notify the holder, teardown (app delete, consent revoke, account delete) drops the scope, and a job deletes unbound handlers after a day. * test(events): cover user-scope deploy, address, notices, teardown and the sweep Runs an inline user-scope handler through the local worker runtime as its holder, and covers the holder on the wire, the flag-off path, revoke and account-delete teardown, and the unbound-handler sweep. * test(events): a lagging replica read does not suspend a user-scope schedule run * feat(puter-js): inline user-scope handlers and scope option for handlers and workers * docs(events): document inline and user-scope handlers, with playground examples Cover inline handlers on onSchedule and onPersistent, handlerScope, the scope option on handlers and workers, the user-scope limits and rented units, and the new error codes. Add six playground examples and pin the new limit constants in limits.test.ts. * feat(schedules): carry the handler scope on the schedule view * docs(events): describe user-handler cleanup, schedule handlerScope and suspension notices * fix(events): gate the rented-unit cap on user handlers and let an account session destroy its own user scope * fix(events): keep user-scope rows off app handlers across mixed versions and bound their deploys and reads Cached durable rows carry an explicit scope marker; an unmarked entry with a handler is resolved from the table and rewritten. User-scope rows are parked before credit checks, billing, client delivery and suspension while the flag is off. Delivery-time handler checks and addressing share the replica cache entry. Concurrent eager deploys of one user-scope script upload once. * fix(events): keep a user scope's age across handler churn for rent event_user_handlers.scope_since records when the scope first held a handler. A new row takes the scope's existing minimum, so replacing code does not make the scope look newer to the monthly sweep. The count uses it instead of the oldest remaining handler's created_at. * fix(events): drop cached entries no row backs; document account-session scope destroy * test(jobs): list the user-handler sweep among the system jobs * fix(events): let the user-handlers migration apply on its own * fix(events): fail a cached-row read whose scope lookup fails instead of returning app-scope rows * fix(events): touch a reused user-scope handler before binding so the sweep cannot take it * fix(events): bound the deploy-on-miss wait by the call's deadline	2026-11-03 23:26:04 -07:00
Daniel Salazar	d934a47c5e	feat: jobs and schedules (#4467) ... * feat(db): add nowMsExpr, the database clock in epoch milliseconds, and share the retry backoff * feat(config): type the jobs config as an augmentable IJobsConfig and resolve its namespace in one place * feat(jobs): add JobScheduler for fleet- and region-wide system jobs * refactor(jobs): run core's periodic sweeps as system jobs, and publish the generation bumps of reaped subscriptions * feat(events): seams for schedules run by an extension, with events_disabled where none is installed * feat(notifications): add notification.send_key, the deferred mailbox write and notify({ sendAt }) through the schedules contract * feat(puter.js): schedules, notify({ sendAt }) and notifications.cancel(), and treat a 503 events_disabled as final * docs(events): document schedules, sendAt and cancel(), with recipes * fix(events): stop resuming a holder's later pages once the credit sweep's signal aborts, and keep its checkpoint * fix(events): pre-release fixes for jobs, schedule seams and delayed sends - The delayed-send write is a field on the notifications driver, not a method, so /drivers/call can't reach it. - A delayed send checks for its own earlier write by uid, so it takes no gap lock that every other notification insert waits on. - The notification send_key index builds in place without locking writes. - Reaps re-check expiry and suspension on the primary, so a row resumed since the replica read is kept. - Revocation settles skip schedule rows they can't settle. - An empty handler set read from a replica is re-read on the primary. - One ended notice per settle, not one per page. - Sweeps that run once per fleet take more batches per run. - The global usage rate check keeps its 14-minute minimum window. - shutdown() waits for prepare hooks a drain already started. * fix(puter.js): quota refusals are final, and schedule error codes match the server Schedule, subscription and handler quota 429s no longer go through the generic retry. ScheduleErrorCode lists what the schedule methods throw, and the notify({ sendAt }) overload has type tests. * docs(events): schedule and delayed-send accuracy and style pass Spread and send-time wording match the runner, the per-user runs-at-once and rate-limited stored notification limits are published, and the new pages follow the docs writing rules. * docs(events): add the Change Data at a Set Time recipe A one-time schedule that changes data when it runs, cancelling and moving it, and a repeat-a-few-times variant with maxRuns. * fix(events): delayed sends recheck the app's access when they fire, and a handler set missing the run's handler is re-read on the primary Also stops job polling at drain start and marks replayable notification sends. * feat(events): batch handler invocations to the events dispatcher Invocations that come due within a few milliseconds go out as one /invoke-batch request (up to events.invokeBatchSize, default 60). The dispatcher runs a few at a time and streams a started line and an answer line per call, so a slow handler delays no other call's answer. A call's timeout runs from its started line, so time queued behind other calls is never the handler's; queue time comes out of the connection-wait share instead, and a call still queued past it is never started and fails as Puter's. A lone call still uses /invoke, and a dispatcher without the batch route gets single calls for five minutes before the batch route is tried again.	2026-11-02 23:48:35 -07:00
Daniel Salazar	5d34e803ff	fix(events): delivery path fixes for queued deliveries, handler failures and per-invocation database load (#4443) ... * fix(events): queue every row's single delivery before any handler runs A dispatch queued each matched single row's event and then waited on that row's hand-over, which can be a whole handler run, before queueing the next row's. An event matching several rows queued the later ones only after the earlier handlers finished, and lost them if the process stopped in between. A batched filesystem write did the same across its entries. The event is now queued in line and the hand-over runs detached; anything it doesn't finish is retried by the pending sweep. A hand-over that throws no longer raises the 'could not be queued and is lost' alarm, since the event is still queued. * perf(events): address an app's events worker from a cached handler set Every handler invocation read the app's handler set (names and hashes) from the primary database to work out which script to call. The delivery path now reads it through a Redis cache. Publishing, updating or removing a handler, and deleting the app's handlers, drop the entry on this node and every peer, so only a read that raced one of those writes can address the superseded script, for at most 30 seconds. An empty set isn't cached. Deploys and the events-worker listing still read the primary. * perf(events): skip leased queues without a database read, and sweep in parallel The pending sweep read each queue's subscription row from the primary database before finding out whether anything in the queue could be claimed. A queue whose head is in flight or waiting out a retry now goes to the back of the line on a Redis check alone; its row is read once the lease lapses. The backstop re-check of a queued row's access waits for that lapse too, and nothing is handed out in the meantime. The pass also worked through its subscriptions one at a time, so one slow handler run held up every queue behind it. It now works on up to ten at once. * fix(events): don't count a wait for a dispatcher connection as a handler timeout Invocations share one pool of connections to the events dispatcher per process, and the invoke timeout started when a call was made, not when it got a connection. With the pool busy, a call could spend its timeout waiting, be reported as a handler that ran past it, and count toward suspending a subscription whose handler never ran. A call now waits for a free connection before its timeout starts. A call that waits more than half the timeout for one fails as Puter's (unavailable): retried on the usual schedule and not counted, as the documented failure rules already say. Wait and run together stay inside the worker lease. * fix(events): end a background row whose consent a delivery finds withdrawn Withdrawing an app's events:background consent is settled by a listener on the node that handled the revoke. If that settle never ran or failed, the row stayed in service: every delivery found the consent gone, minted no token and was deferred, so the backlog was retried forever and the row was never suspended or reported. When the delivery path's consent check answers no, it now runs the same revocation settle (permission_revoked, backlog purged, holder told, worker session revoked), one at a time per holder and app. A consent check that throws still only defers the delivery. * perf(auth): record no session or user activity for events handler calls Every API call an events handler made authenticated through its worker session and touched the session and the user, so background handler runs wrote session and user activity rows on the primary database (each at most once a minute), and replaced the session's last IP and user agent with the worker runtime's. Calls made with an events handler token no longer touch either. A handler run isn't the user being active, and the handler session expires only when it is revoked, so nothing reads its activity to keep it alive. The handler session's last activity in the session list now reads as when it was created. * fix(events): settle withdrawn consent only on a fresh answer The delivery path's consent check reads through the permission scan cache and the cached user-to-app grants, which can trail a grant, for example on a node that cached the app's grants before the consent was given. A stale no then ended the subscription for good (permission_revoked, backlog purged, holder told, worker session revoked), where before it only deferred the delivery until the caches caught up. Before settling, the no is now asked again with a new `primary` scan option, which skips the scan cache and reads user-to-app grants from the primary database. Only a fresh no settles. A fresh yes, or a check that throws, leaves the delivery deferred. * test(events): assert no forwarded delivery, not no traffic at all The region's own connect announces presence to its peers on the forward queue's next flush. Now that a dispatch returns before the hand-over, the wait for the socket send can outlast that flush, and the test saw the presence announce as forwarding. * fix(events): budget the settles deliveries start for withdrawn consent A settle a delivery starts when it finds background consent withdrawn ends rows for good. If the permission check itself answered no wrongly, every row a delivery reached would be suspended, its backlog purged and its holder told, with nothing to bring it back. Each process now settles at most 20 such pairs a minute. Past that, the rest stay deferred, as a cached no always left them, and a warning alarm says so. A spent budget skips the fresh read too. A pair a fresh read just found consented isn't read again for a minute, so a stale cached no costs one primary read rather than one per delivery. Each settle is logged.	2026-10-31 14:52:06 -07:00
Daniel Salazar	dd144b6b64	feat(notifications): notify through KV shares, group replacement, fewer push prompts, and events/worker fixes (#4432) ... * feat(perms): add a pushNotifications resource to puter.perms.request notifications:push gets a resource like backgroundEvents, so it can be batched with other access behind one prompt. requestPermission() and hasPermission() ask through it. * fix(puter-js): read puter.gui_features once at load The desktop's feature list was read from location.search on each call, so an app whose router rewrote the URL lost it and enablePush() and the other desktop dialogs answered as if the desktop lacked them. * feat(gui): ask for notifications:push when an app calls enablePush without it enable() and enablePush() inside the desktop answered unavailable for an app without the push permission. They now show the permission dialog, whose Allow also starts the browser registration, and answer with the browser's push state once it settles, or off when denied. A browser that blocks notifications or has no push service still answers without a dialog. * fix(kv-share): charge mint and grant budgets only for share calls that change something A share call now spends the per-call mint budget and the per-recipient grant windows after it has looked up which recipients already hold the share. A call whose recipients all hold the share at the requested level and label spends neither. Recipients without the share, including unknown usernames and refused recipients, still spend a grant unit each. Every named recipient spends from a new 1,200/min lookup window before any lookup runs. * perf(events): skip the subject split for rows not made through a share handle kvHandleFromSubject checks the stored subject's kv:kvh- prefix before splitting it, so own-namespace and fs rows return null without parsing. * feat(notifications): resolve enable() on a website before the confirmation push enable() waited up to 30 seconds for the confirmation push, so a prompt the same click started next had lost its activation. On a website it now resolves once the browser answered and the server has the subscription, records the opt-in, and finishes the confirmation in the background. A failed confirmation goes to onPush's onError and the next load's resync tries again. disable() and sign-out stop a confirmation still waiting. The GUI keeps waiting, since it reads the registered device right after. * fix(notifications): show a page's own notifications only while push is on A hidden page with an onPush handler kept showing notifications after disable(), and its receipts kept claiming showsOwn, which held pushes back from the user's other devices. Both now need the user's opt-in, cached in the page, refreshed on enable, disable, sign-out and each sync, and re-read when another tab of the site changes it. * fix(events): leave subscriptions suspended for good out of the durable quota countForHolder no longer counts rows suspended with a reason that never resumes (permission_revoked), in both the pre-insert check and the recount after insert. Rows suspended for a resumable reason still count. The store now owns the list of terminal reasons, and isResumable reads it. listActiveForHolder with includeSuspended returns only rows suspended for a resumable reason, so its quota-sized read still covers every row a revocation has to settle. Each holder keeps at most 500 terminal rows; a subscribe past that deletes the oldest ones. * feat(notifications): add status() to read whether pushes reach this browser status() resolves 'on', 'off', 'blocked' or 'unavailable' without asking anything. On a website it reads the notification permission, this browser's subscription and the user's opt-in. Inside the desktop a new pushStatus IPC, advertised as push-status, answers the browser's push state, 'on' only while the app holds notifications:push. * feat(notifications): disable() inside the desktop revokes the app's push permission disable() did nothing inside the desktop. A new disablePush IPC, advertised as disable-push, revokes the calling app's notifications:push grant through the same route Settings uses, so its notifications stop going out as pushes and a later enable() shows the permission dialog again. * feat(notifications): allow @ in a notification group, up to 128 characters A group is an opaque display tag, hashed for the Web Push topic and used as the notification tag, so it can carry identifiers such as an email address. * docs(notifications): say when a page with an older Puter.js holds a push back * fix(notifications): let disable() win over an enable() still in flight An enable() whose subscription reached the server after a disable() went on to record the opt-in, so the next page load turned pushes back on. It now rejects with push_verification_canceled and records nothing. * docs(notifications): list push_verification_canceled and reword sentences that led with code * fix(events): carry KV values through a share handle only at read or write The delivery re-check now reads the share's level off the live handle along with whether it still holds. A row on a watch share loses its includeValue for that delivery, so its events name the key without the value; the owner's own rows on the handle keep theirs. Decided per delivery, so raising or lowering a share applies to subscriptions already made, and subscribing with includeValue at watch is still accepted. The live-updates recipe now shares at read, which its events need to carry the score. * fix(events): retry worker runs Puter could not complete without counting them Invocations that never reached the handler (transport and dispatcher errors, unmarked answers, failed or throttled deploys, no configured runtime, a bad invoke key or missing token) now report an 'unavailable' outcome. So does a handler that throws after one of its own puter.js calls got a 5xx, 429 or network error from Puter's API, which the events runtime reports as 'platform-error'. Those runs back off like retriable ones, honor the answer's retry-after up to the backoff ceiling, and no longer count toward the consecutive-failure suspension. A run past the invoke timeout still counts. * test(notifications): cover the pushNotifications resource and status() in the API suites * test(gui): Allow on a batch that includes notifications:push turns push on * fix(events): hold a share-handle row's backlog when its share can't be checked The share-handle delivery check now tells a share that is gone (handle not live, not the row's holder, identity gone, grant withdrawn) from one it can't check right now (a lookup that throws, an owner who is suspended). Only the first settles a queued row as permission_revoked and purges its backlog. The second delivers nothing, leaves the row in service with its backlog, and sends it to the back of the sweeper's line, so delivery resumes once the lookup recovers or the owner is unsuspended. A revocation settle skips such a row and leaves it to the delivery check. kvSharedRegionAuthorized returns null when the permission check throws, and that answer is not cached. * feat(events): update a persistent subscription's context in place POST /events/subscriptions/update replaces a durable row's context, with the same validation and 4 KB cap as subscribe, keeping its subId and backlog. The row's owner region cache is rebuilt from the table and the generation bumped so every region delivers the new value. It spends the subscribe call budget. puter.events.update(subId, { context }) wraps it; null clears the context, and a function handler this client registered for the subscription gets the new ctx. * fix(events): serve a fan-out's holders in turns and cap values per delivery #route now orders an event's candidate rows before any cap applies: the keyspace owner's rows first, then the other holders taking turns, one row per holder per round, starting from a random holder each event. The filter evaluation cap, the delivery cap and the gap markers all count in that order. Values no longer drop for every delivery once more than 128 rows match or the filter evaluation stops early. The first 128 deliveries that asked for the value carry it, in serving order, and later ones carry the key alone. EVENTS_KV_VALUE_OMIT_MATCHED_SUBSCRIPTIONS becomes EVENTS_KV_VALUE_DELIVERIES_PER_EVENT. The caps themselves are unchanged. * fix(email): file inbox mail as a delivery, not the request actor's write storeInboxMessage and copyIntoInbox now run their filesystem writes with a platformWrite context flag, and fs event dispatch leaves actingUserId unset under it. Mail filed into an account's inbox from inside that account's own request (an app mailing the signed-in user) is delivered with self: false, the same as external ingress. Handler depth and viaWorker are still carried from the actor. * feat(notifications): raise the default daily app message limit to 1000 The per-day default for `app.message` goes from 300 to 1000 per user and app, matching the daily push cap. The per-minute default stays at 30, and `notificationAppMessageLimits` still overrides both. * feat(notifications): look up an unseen notification by group `notification` gets a nullable `group_key` column, a sha256 hex digest of the group a row was written with, indexed on (user_id, app_uid, group_key). `NotificationStore.create` takes a `group`, and `getUnseenByGroup` returns the newest row an app wrote for a user under that group that is neither shown nor acknowledged. Rows written before the column exist never match. `updateValue` takes `onlyUnseen`, which also skips a shown row, and `refreshCreatedAt`, which dates the row to now. Migrations: sqlite 0098, mysql_mig_52, postgres_mig_41. * feat(notifications): replace an app's unseen message in the same group An `app.message` with a `group` now replaces the app's earlier notification to the same user with that group while it is neither shown nor acknowledged. The row keeps its uid and takes the new value and time. With no group, or once the earlier one was shown or dismissed, it is a new notification as before. A replacement spends a per-minute slot but no per-day slot. If the earlier row is marked between the lookup and the rewrite, the message is written as a new notification and spends a day slot then. `app.message` is groupable, so the replacement goes through `notifyUpdate`. For types kept off the desktop, `notifyUpdate` skips the desktop wire and dispatches the rewrite again under the same uid, so app subscribers and receipts see it, and `notif.written` carries `updated`. The driver's write path takes the recipient id rather than the actor, so rate limits key on the recipient. * feat(workers): report the app a worker runs as workers.create now returns app_uid and app_name for the app its token is bound to (null for a user-scoped worker), and list/get items carry app_name beside app_uid from the app rows they already load. The dev center workers table shows an App column, and the SDK typedefs and docs describe the fields and that a sandboxed worker's notifications and pushes come from its sandbox app. * feat(events): refuse handlers with free variables on REST publish POST /events/handlers/publish and /publishAll now run the SDK's own scanHandlerSource on each source before publishing and answer 400 events_handler_free_variable, naming the identifier. The backend imports the module from src/puter-js; the build marks dist/src/puter-js as an ES module package so the compiled backend can require it. Sources over the per-handler cap are left to the store's size refusal rather than tokenized. * docs(events): say Puter-side retries never suspend and stay within backlog caps * fix(events): redeploy an events worker that answers bad-key A script deployed before the internal secret rotated keeps the old invoke key, so every invocation answered bad-key and nothing redeployed it. A bad-key answer now goes through the same miss handler as a dispatcher miss (the deployer, with its hourly throttle) and is retried once with the deployed header. A throttled or failed redeploy stays 'unavailable' and backs off without counting. * perf(events): check the deploy throttle before reading handler sources EventsWorkerDeployer now reads the app's handler names and hashes first, answers no-handlers or stale from those, and checks the hourly deploy throttle before loading the sources and generating the worker. A throttled retry costs a hash read and a counter bump instead of a full source read and build, and a stale request still does not spend the throttle. The generated set is checked against the script again in case it moved between the two reads. * fix(workers): move a redeployed worker's row to the app it now runs as Redeploying an existing worker under a different app minted the new app's token but left subdomains.app_owner on the old one, so list/get and hot-reload kept naming and minting for the old app. The redeploy now sets app_owner to the binding #resolveWorkerAppBinding authorized (null for a user-scoped worker) through a new SubdomainStore.setAppOwner, since app_owner stays read-only to update patches. * fix(workers): revoke the old binding's worker session on a rebind A redeploy that binds a worker to a different app (or to none) now deletes the owner's worker sessions under that name other than the one for the new binding, through the same deleteWorkerSessionsByName path worker deletion uses (which also announces the revocation and drops access-token grants). SessionStore's deleteWorkerSessions takes an exceptAppUid filter for it, null meaning the user-scoped session. The app_owner move and the revocation now both run only after the new script deployed successfully, so a failed redeploy leaves the live script, its token and the row on the old binding. * feat(kv-share): store a notify flag on share handles kv_share_handles gains a notify column, off by default, so every existing share reads as not allowing notifications (sqlite 0099, mysql_mig_53, postgres_mig_42). The store reads it on every row, including the live handle cache and the holder listing, writes it on mint, and changes it through update(). * feat(kv-share): let the owner turn notifications on for a share POST /kv/shares and puter.kv.share() take notify, off by default. Leaving it out of a re-share keeps the stored value, and turning it on or off counts as a change, so it spends the mint budget. Turning it on for a holder who now refuses the owner's shares is refused, like a raise or a new label. The kv.share.received notice is sent only when a share starts carrying notify: a new share made with it, or an existing one switched on. Its hourly bound and the app-less namespace exclusion are unchanged. The share response, the owner's listing and the holder's listing report notify. * feat(kv-share): resolve the other party of a share for a notification KvShareService.resolveNotifyRecipient(actor, handle) returns the recipient user id, app uid and handle for a notification one party of a share sends the other through its handle. The handle must be live with an unsuspended owner, the caller must be its owner or holder acting from the app whose data it is, the share must have notify on, the holder's grant must still stand, and the recipient must not refuse shares from the sender. An unknown, retired or foreign handle, a caller who is not a party, and a caller in another app all get subject_does_not_exist. The account itself gets forbidden, a share without notify kv_share_notify_not_enabled, and a lookup that fails service_unavailable. Once every check passes it spends one unit from KV_SHARE_NOTIFY_LIMITS (30/min, 500/day per sender, recipient and app), peeking both windows first so a refusal spends nothing. * feat(notifications): hand a replaced notification to listeners again A replacement now records `replaced_at` (epoch ms) on its row, added to the same unreleased migrations as `group_key` (the sqlite file is renamed to 0098_notification-groups.sql). Its live event and its fetched copy both carry `replaced: true` and use that time as `ts`, so the pair versions the uid identically on both paths. `onNotification()` dedupes on uid plus that version: an original counts as version 0 and a replacement as its `ts`. A newer replacement is handed over again with `replaced: true`, while a copy already handed over, a replay, or an older state is dropped. Listeners started with `catchUp: false` still get replacements of what was already there. * fix(notifications): replace only the newest notification in a group The group lookup takes the newest row an app wrote under the group and returns nothing when that row was shown or dismissed, instead of falling back to an older row that is still unseen. The docs state the rule as it is: a grouped notification replaces the newest one in its group unless that was shown or dismissed. Puter.js marks neither, and the only thing that does is a tap on the push inside the Puter desktop, which dismisses it. * feat(notifications): notify the other party of a KV share through its handle `app.message` takes `to: { handle }`. The driver validates the message, then `to` must be an object with only a non-empty string `handle`, else 400. `KvShareService.resolveNotifyRecipient` checks the share and spends the pair limit, and the message is written for the share's other party in the share's app. The recipient's per-app limits and group replacement apply to their own rows, and the push is billed to them through `notif.written`. A call with `to` answers with `{ uid }` only, so nothing about the recipient's row reaches the sender. puter.js gets the `NotificationRecipient` type for `to`. * docs(notifications): document notifying the other side of a KV share notify.md covers `to: { handle }`: who can be reached, the errors, the recipient's limits plus the per-pair share limit, the uid-only return and that the recipient pays for pushes. push.md and rate-limits-and-quotas.md point at the share path. The chat notification recipe sends through each share's handle and keeps the per-member handler only for members who need to reach each other; the alert recipe says when its handler is the right fit. * fix(notifications): reveal nothing to the sender of a notification through a share A call with `to: { handle }` answers with an empty object, and puter.js resolves it with no value: a uid would show whether the recipient's last notification in a group was replaced. `notify()` gets one overload per form, and `ShareNotificationInput` is exported. Over the per-pair share limit or the recipient's own limits, such a call gets the same 429 `too_many_requests` with the same generic message, so the sender can't tell which limit it hit. * fix(notifications): serialize grouped app messages per group Grouped app.message sends take a Redis lock on (recipient, app, group) around the lookup and the replace-or-insert, so concurrent sends to an empty group write one row and spend one day slot. A lock still busy after the wait, or an unreachable Redis, sends unlocked as before. Sends without a group take no lock. * fix(events): attribute Puter API failures to the invocation that saw them The events worker runtime counted Puter API failures on the cached per-token client, so a concurrent invocation on the same token could answer platform-error for its own unrelated throw. A cached client now serves one invocation at a time and records failures and retry-after on that invocation; an invocation overlapping another on the same token gets its own client, and idle clients stay cached per token. * fix(events): drop KV values from a queued backlog once its share is at watch A queue delivery is stored with its value. Every hand-out of a stored entry (backlog drain to a socket or another region, the worker, and a declined delivery passed to the worker) now re-reads the share's level and removes the value at watch. A share that can't be checked still holds the backlog. * fix(events): return notifications replaced behind the fetch cursor The notification fetch cursor now holds the last row id and the last (replaced_at, id) it returned. A read from it merges two indexed reads: rows inserted after the id, and rows up to the id replaced after that mark. A replacement made while a client was away therefore comes back on its next catch-up. Adds idx_notification_replaced on (user_id, audience, app_uid, replaced_at).	2026-10-31 12:21:18 -07:00
Daniel Salazar	4851df716c	fix(kv): KV share lifecycle hardening (#4429) ... * fix(kv): harden the KV share lifecycle - Retire a deleted app's live shares before an origin app is registered again under the same uid. The retire runs before the app row exists and throws on failure, so a failed launch leaves no row and the next one retries. - Record deleted-app retires and uninstall cleanups (retire, then purge) as reserved items before they run. A lease keeps one node on a job; a failed run raises an alarm and backs off, and a run that hits its time budget or is cut short by a restart resumes on the next check. Each job is bounded to handles made before it was recorded. - Uninstalling with deleteData records the job and returns; the retire and purge run off the request. The purge checks whether the app was authorized again before deleting each page, so data written after that is kept. - Take a per-region lock when minting and re-check for a live handle under it, so concurrent shares of one region make one handle. - Retire the handles an account holds before the account is deleted, so the owners' subscriptions through them end instead of staying active. * fix(events): bound share-handle session rows by the per-connection cap The count across a holder's connections also counted connections that had dropped but weren't reaped yet (a crashed node, a deploy, a network blip the server hasn't noticed), so resubscribes after a reconnect could be refused with `events_subscription_limit`. Rows made through a share handle now count toward the 50-per-connection cap like any other, and with the open-connections cap per account that bounds what one holder can put in the owner's keyspace. The handle index stays for ending rows on retire. Rows are added to it with the row itself and taken out when the row is dropped, and members whose row is gone are cleared when a retire reads it. * fix(events): keep a retire's durable settle running if ending sessions fails Ending a retired handle's session subscriptions reads Redis. A failure there is now logged and the durable suspension that follows still runs, so those rows' backlogs are still purged. * perf(events): ask once per handle when a refresh screens rows, and share handle reads A socket refresh asks whether a share handle still stands once per handle, holder and app, rather than once per row: rows through one handle share its grant. Concurrent cache misses on one handle share a single read; a forget drops the shared read so later callers start a fresh one. * fix(permissions): chunk the exact-match subtree delete The holder-scoped subtree delete names every matched grant, so a large subtree could exceed an engine's limit on bound parameters. It now deletes 500 grants per statement. * fix(kv): give each holder a share of a namespace's handle budget One holder could spend a namespace's whole handle budget and starve the others. Each holder now gets at most 3,750 of the namespace's 15,000 units per 10 seconds, and the owner's own calls through a handle spend none. The holder's budget is checked before the grant lookups, so throttled retries cost none; the namespace's is only told to callers the grant admits. Multi-unit charges use a new counted claim that keeps two counters per key instead of a sorted-set member per unit. * fix(kv): refuse handles older than their app and claim cleanup jobs globally - A share handle created before its app row reads as gone: in the KV driver, in the handle cache, on the events subscribe path, and in the delegate implicator. An origin app registered again under a deleted app's uid can't reach its predecessor's shares, whatever created the row. The app-token launch no longer retires inline; it records a background retire and never waits on it or fails because of it. The clock-skew rule is shared with app sessions. - Cleanup job records now carry their lease, attempts and retry time, and a node claims a job by compare-and-set on its record. A job stays with the deployment that recorded or last ran it, and another one adopts it only after it has gone unclaimed well past due. - The job check pages through records from where the last look stopped, so waiting records don't hide due ones. After 10 failed runs a job stops retrying with a warning alarm, and its record expires after 30 days. - A mint that waits out the region lock looks for the share again and returns it, or goes ahead, instead of failing with too_many_requests. - The record-failure alarm dedups; an unfinished run is a log line. * fix(kv): hide shares older than their app from re-shares and listings After an app is registered again under a deleted app's uid, its predecessor's live handles stayed visible: sharing the same region again handed back the old, dead handle, and both sides' listings showed the old handles and labels. findLive and both listing queries now leave out a live handle created before its app row, the same rule handle resolution uses, so a re-share makes a new handle and the listings, totals included, skip the old ones until their retire lands. A new share also retires, grants and all, the handles its owner gave the same holder for the earlier app first, so withdrawing those later can't take the new grant with it. * fix(events): keep a loose cap on subscriptions through one share handle Without a count across connections, a holder could put up to 20,000 rows in the owner's keyspace (50 per connection, 400 connections), each one evaluated on every owner write. The cap is back at 1,000 per holder per handle, far above what connections that dropped without being reaped leave behind. The index is only read row by row once it's past the cap. * fix(kv): harden the share cleanup jobs - Job records are keyed by domain, so a deployment that shares the table but not the database or broadcast (staging beside production) never runs another's jobs. - A job whose record can't be written runs unrecorded instead of not at all, and a failed check for live handles records the job anyway so it retries. Failures that used to be swallowed are logged. - Recording a job again keeps its attempts and backoff; one that gave up gets one more attempt. - A system retire gives back the grant of a live share it doesn't cover inside a withdrawn region, and a share made under a reused uid grants again once its row stands, so a concurrent retire can't leave a live handle without its grant. * fix(kv): don't cache a live handle a retire overtook during its app lookup The live read decided whether it could cache before awaiting the app lookup, so a retire's forget that landed during that lookup was overwritten with the handle as live for up to 60 s. It's now checked after the lookup. * chore(kv): word the job key comment for any install	2026-10-31 12:12:29 -07:00
Daniel Salazar	279bb8beb6	fix(events): KV share handle subscription follow-ups (#4434) ... * fix(events): bound share-handle session rows by the per-connection cap The count across a holder's connections also counted connections that had dropped but weren't reaped yet (a crashed node, a deploy, a network blip the server hasn't noticed), so resubscribes after a reconnect could be refused with `events_subscription_limit`. Rows made through a share handle now count toward the 50-per-connection cap like any other, and with the open-connections cap per account that bounds what one holder can put in the owner's keyspace. The handle index stays for ending rows on retire. Rows are added to it with the row itself and taken out when the row is dropped, and members whose row is gone are cleared when a retire reads it. * fix(events): keep a retire's durable settle running if ending sessions fails Ending a retired handle's session subscriptions reads Redis. A failure there is now logged and the durable suspension that follows still runs, so those rows' backlogs are still purged. * perf(events): ask once per handle when a refresh screens rows, and share handle reads A socket refresh asks whether a share handle still stands once per handle, holder and app, rather than once per row: rows through one handle share its grant. Concurrent cache misses on one handle share a single read; a forget drops the shared read so later callers start a fresh one. * fix(events): keep a loose cap on subscriptions through one share handle Without a count across connections, a holder could put up to 20,000 rows in the owner's keyspace (50 per connection, 400 connections), each one evaluated on every owner write. The cap is back at 1,000 per holder per handle, far above what connections that dropped without being reaped leave behind. The index is only read row by row once it's past the cap.	2026-10-31 12:10:34 -07:00
Daniel Salazar	7f832ede43	fix(events): KV share handle subscription hardening (#4428) ... * fix(events): keep a share's prefix out of handle subscribe errors and limits A `kv:<handle>:<key>` pattern is now checked against the 256-character limit as the holder wrote it, before it's composed onto the shared prefix. The composed pattern is compiled with room for the longest prefix a grant can carry, at subscribe and at dispatch, so neither the limit nor its error depends on the prefix. A handle subscription's `match` is read off the holder's own subject instead of the stored composed pattern, so whether it's `null` no longer depends on how deep or long the prefix is. * fix(events): cap and end session subscriptions made through a share handle Session subscriptions made through a share handle are indexed under the handle in the owner's keyspace. A holder may keep 100 of them through one handle across all their connections; past that, subscribe fails with `events_subscription_limit`. The owner's own subscriptions through the handle don't count. Retiring a handle now ends its session subscriptions in the same region and sends `events.ended` with `permission_revoked`, alongside the existing durable settle. A socket refresh no longer renews a row whose handle was retired or whose grant is gone: it ends it the same way, which is how a retire made elsewhere reaches these rows. * fix(kv): read a forgotten share handle from the primary After a handle or its owner is forgotten, reads of that handle go to the primary for the next minute, so a replica that hasn't caught up can't put the old row back in the cache. A read that a forget lands on while it's in flight is answered but not cached. * fix(events): read a socket ref's subId after the last `|` A KV token carries the subscribed key, which can contain `|`, so splitting the ref at the second one cut the token short. A subscription on such a key couldn't be unsubscribed, and a reap left its row behind in the owner's keyspace. SubIds never contain `|`, so the ref is split at the last one.	2026-10-31 00:58:42 -07:00
Juan Fernando CastroandDaniel Salazar	c00f693ec6	feat(notifications): report how each live notification was received (#4403) ... * feat(notifications): report how each live notif.message was received SocketService.sendWithAck sends an event to rooms or sockets and waits, up to a timeout, for an acknowledgement from each socket that listed the event in its handshake (`auth.acks`); every other socket gets a plain emit. It lists sockets across the cluster with fetchSockets and asks the listed ones through the adapter's broadcast acks, and counts this node alone when the adapter cannot list the others. The emitting region now sends notif.message through it, with the window from `notificationAckTimeoutMs` (default 5000), and emits the node-local `notif.delivery.settled` with the outcome: received, unreceived, no-socket or remote-only (sockets only in a peer region), whether any socket there acknowledges, and each responder's visibilityState. Peer regions keep sending their own copy as before; the GUI event carries `region_delivered` so this region does not send it twice. The desktop and dashboard list notif.message in their handshake and answer its callback once with document.visibilityState. Nothing in core acts on the settled event. * fix(notifications): send live notif.message at once and report what the count covers sendWithAck now emits to the whole room straight away as a broadcast with ack, and lists the room's sockets alongside it only to fill in the counts, so nobody waits on the listing. Whether the adapter can count the cluster's servers is checked once; where it cannot, sends go out per socket on this node with a one-time warning. The result carries `complete`: false when only this node's sockets were counted, either for that reason or because the listing failed or took over 1 s, which is warned about at most once a minute. notif.delivery.settled gains `sendId`, `type`, `complete` and `peerRegions`. An incomplete count with no socket found settles as unreceived rather than no-socket, and the peer-region read runs alongside the send instead of after it. notif.written carries the same `sendId`, minted per send (an update gets a new one) and passed through notif.created to the socket adapter on the fold-in path. The comment on revoking sockets no longer says fetchSockets is unavailable with our Redis client; it works with ioredis cluster. * fix(notifications): keep a failed server count from ending the process socket.io awaits the adapter's serverCount without a catch on every broadcast with an ack, and the redis-streams adapter's count rejects as soon as one cluster node fails to answer. The count on the root namespace's adapter now never rejects: a failure reports the last good count (1 before any), and is warned about at most once a minute. A stale count only lets an ack wait run to its timeout. Whether sendWithAck takes the cluster path is no longer a one-shot check awaited before the emit. Until a count has succeeded, sends go out per socket on this node at once while a count is retried in the background, at most every 30 s (SocketService.COUNT_RETRY_MS); the first success switches to the cluster path for good. * fix(notifications): ask only acknowledging sockets, through an ack room A socket whose handshake lists an event in `acks` now joins `<room>:acks:<event>` for each of its rooms and its own id. sendWithAck sends the ordinary broadcast to the target rooms except those ack rooms, which keeps it on the adapter's stream, and a broadcast with an ack only to the ack rooms. Sockets that never acknowledge are no longer asked, so other nodes hold no pending ack for them and a send completes as soon as every acknowledging socket answered. The per-socket listing that matched socket.data.acks is gone; the ack-capable count comes from the room membership the listing already returns. The per-socket path stays only for a node whose adapter has not counted servers yet. A failed server count during a send now marks its counts incomplete, so a stale count can't make a listing that missed a node read as complete. With no count ever succeeded (an in-memory Redis), the failure is warned once per process instead of once a minute. * fix(notifications): say whether every socket on a live notification acknowledges it A tab on an older bundle beside one that acknowledges made the settle read as ack-capable, so a listener could not tell that the older tab may have shown the notification. notif.delivery.settled now carries everySocketAcks. * feat(notifications): app pages acknowledge the live copy of an app message NotificationService.appReceipt() asks an app's own sockets whether a notification reached a page, and settles it as notif.delivery.settled with appUid, the same outcomes as the desktop's. Callers ask only when the answer decides something, since it is a round trip to every node. - SocketService.askWithAck asks without listing sockets: sockets that don't acknowledge get an acked broadcast too, which can only time out, and that says whether an older client shares the target. - puter.js declares notif.receipt on its filesystem and events sockets. A page answers once one of its notification feeds received that uid, before any handler runs, and never when no feed on the page got it. - Docs describe the push route as "no page of the app received it live within 5 seconds". * fix(notifications): count a socket that acknowledges other events as a current client Every desktop page also holds puter.js's own socket, which never shows notif.message, so "every socket acknowledges notif.message" never held there. sendWithAck now reports `declaring`, the sockets that listed any event in auth.acks, and everySocketAcks on the desktop's settle means no socket here declares nothing. * fix(puter-js): fall back to the Continue dialog when a popup can't be opened Some browsers report user activation but still return null from window.open (iOS home-screen web apps, Safari after awaited work), so permission prompts, signIn, enablePush, and GUI action popups failed without showing anything. PuterDialog.launch() now owns the open-now-or-show-Continue decision for every popup flow and shows the consent dialog when the direct open returns null. On cross-origin-isolated pages signIn leaves settling to the /login/wait relay on both launch paths. * fix(notifications): show a live push when the page's window isn't focused onPush showed the handler's result as a system notification only when the tab was hidden. A visible tab in a window the user left showed nothing. It now also shows when the document doesn't have focus. Inside the Puter desktop the desktop's banner shows the notification, so an app's frame still never does. * build(puter-js): publish push-sw.js exactly as written The build emits dist/push-sw.js next to puter.js, but terser minified it along with the bundle. It is now marked as already minimized, so the file apps import into their service worker is the source unchanged. * fix(notifications): send live notif.message plainly and ask for receipts on request Every live notif.message used to be sent with an ack, list the user's sockets across the cluster (fetchSockets ships each socket's handshake between nodes) and wait out the receipt window, whether or not anything would act on it. notif.message is a plain send again. NotificationService.receipt() asks the desktop's sockets, or with appUid an app's own, the way app receipts already did: an ask with no socket listing, settled as notif.delivery.settled. A caller asks only when the answer changes what it does. - The desktop and dashboard sockets answer notif.receipt from the page's puter.js receipt log and note each live notif.message in it, so every socket on the page answers alike. A page whose puter.js has no log lists no acks. - SocketService.sendWithAck, the socket listing and region_delivered are gone. askWithAck counts this node's sockets when the adapter cannot count servers. - The send id stays on notif.written and the settle, and leaves the wire. * fix(socket): join only the ack room receipts are asked in A socket listing events in auth.acks joined an ack room beside each of its rooms and its own id. Receipts are only asked in the user's room, or the app's for an app socket, so that is the one ack room it joins. * docs(notifications): publish the push limits of 12 a minute and 1,000 a day Pushes are capped per user per app at 12 a minute and 1,000 a day, and at 6 a minute and 200 a day while an app is new. The rate limits table and the notification pages said one a minute and 100 a day. --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-10-29 18:57:36 -07:00
Juan Castro	c024e007aa	Merge juancastro/put-2258-push-from-app-subdomains into juancastro/put-2262-web-push	2026-10-28 16:30:45 -04:00
Juan Castro	10f10029c2	Merge juancastro/put-2168-let-users-decide-which-apps-can-send-them-pushes into juancastro/put-2167-notification-written-hook	2026-10-28 16:30:33 -04:00
Daniel Salazar	179ad990d6	feat(events): simplify onPersistent delivery (#4395) ... * feat(events): simplify onPersistent delivery Replace delivery/targets with a boolean single. Without single, every event runs the app's published handler in its events worker and is also sent to every connected client; the server no longer routes between the two. single keeps leasing each event to one consumer that acks. delivery: 'single' | 'broadcast' is still accepted and targets is ignored. Stored rows are read by their delivery column alone. * docs(events): compare onLocal, onPersistent and single * feat(events): rename single to queue onPersistent({ queue: true }) is the work-queue mode: each event goes to one consumer at a time and is offered again until acknowledged. delivery: 'single' is still accepted as an alias, and subscription objects report queue instead of single. The docs explain the three modes (onLocal, default onPersistent, queue) and each events recipe says which one it uses and why. * docs(events): retries are the only repeat for a queued event * feat(events): queue deliveries fall through to the worker on decline A queued delivery gets one socket attempt under a 15 s lease instead of two 60 s ones. A client with no handler for it, or whose handler fails, sends events.nack and the worker takes it without waiting for the lease. Declines are relayed to the region holding the lease like acks are, and a decline for a lease that is no longer the client's is ignored. * feat(events): drop thrown errors, retry only temporary ones A queued handler that throws now drops the event and counts one failure toward the pause. Network errors, timeouts, rate limits, 5xx responses and errors thrown with retry: true are retried with the existing backoff. One classifier is shared by puter.js and the events worker runtime, which inlines it at build time. A dropped event is billed once, like a completed one. ack() remains as an optional early release. * docs(events): queued events reach every tab with a handler	2026-10-28 12:58:51 -07:00
Daniel Salazar	40253ae12b	feat(notifications): onPush and enable with Web Push on the app's own origin ... puter.notifications.onPush(key, handler) handles an app's notifications for one channel: live over the events socket while the app is connected, and as a browser push otherwise. enable() asks for notification permission on the site's origin and subscribes through the site's worker, which imports push-sw.js (served at js.puter.com/v2/push-sw.js). A new subscription is confirmed by a challenge push the worker hands back to the page. disable() removes it. The GUI uses the same flow through /sw.js when the server publishes pushWebPushKey.	2026-10-28 01:05:36 -07:00
Daniel Salazar	3b4945f018	feat(events): isOnline presence check for a user and app	2026-10-28 00:53:28 -07:00
Juan Castro	fceaee6372	feat(notifications): node-local notif.written hook ... NotificationService emits notif.written after each non-silent write and after an in-place rewrite, on the node that wrote the row, so an extension can act on new notifications without a second bus round trip. It stays off the published event stream; notif.created carries the notification itself.	2026-10-26 16:34:18 -04:00
Daniel Salazar	f850155411	KV share handles: hardening, higher limits, batch share, uninstall data deletion, account-deletion cleanup (#4344) ... * fix(kv): sign share handle ids and drop retired handles from every node's cache New handle ids carry a short HMAC (kvh-<uuid>.<mac>); an id whose mac does not verify is refused before any lookup. Unsigned ids keep working but no longer fall back to the primary on a replica miss. Retiring or changing a handle now broadcasts outer.pubsub.kv.share.forget so every node drops its cached row. Event subscribes and delivery rechecks refuse a handle whose owner is suspended, as the driver does. * fix(kv): an app its user removed stops reaching shares An app acting for a user exercises a share grant only while the user still has the app authorized: at least one user-to-app permission row, which every app token path leaves and revoke-user-app '*' removes. The share itself stays; authorizing the app again restores access. The answer rides the permission cache, keyed by the app-under-user generation the revoke bumps, so live subscriptions go quiet too. * feat(kv): list keys through a watch-level share handle list with as: 'keys' through a handle needs only watch, which already learns every key written in the region. Values and entries still need read. * feat(kv): filter share listings by app and by state GET /kv/shares and GET /kv/shares/with-me take an optional appUid: an app uid, or null for the app-less store. An account session may name any app; an app is refused any namespace but its own. GET /kv/shares also takes state=live|revoked|all, all by default. * feat(puter.js): appUid and state options on the KV share listings listShared() and listSharedByMe() pass appUid (null for the app-less store), and listSharedByMe() passes state. Docs cover the new options and that a watch share can list keys without their values. * feat(kv): indexes and store counts for share-handle quotas and retention Adds covering indexes for the owner's live count, the owner-holder pair, app-filtered listings and retired rows, plus store methods to count a pair's live handles and delete rows retired before a cutoff. The live handle cache holds up to 50,000 entries; the miss cache stays at 10,000. * feat(kv): higher share-handle caps, a per-holder cap, and sharing with several recipients Live share handles per account go to 25,000 paid / 5,000 free, and per app to 10,000 / 2,000. One owner may hold out 256 live handles to one holder on every plan, which bounds the rows a revoke or unshare walks. Re-sharing an existing region takes no slot and is never refused. POST /kv/shares takes recipients (1 to 50) as an alternative to recipient: one region, level and label for all. Caps and a new per-recipient grant budget (300/min, 2,000/hour) are checked for the whole batch up front; failures about one recipient are reported on its item. The single-recipient request and response are unchanged and run through the same code path. * feat(kv): delete share handles 30 days after they stop A daily job deletes retired share-handle rows older than 30 days, in batches of 500. One node runs it per day; it stops on shutdown. * feat(puter.js): share a KV prefix with several users in one call puter.kv.share() takes an array of recipients, positionally or as recipients in the options form, and resolves to one item per recipient: the share, or an error with its message and code. The single-recipient forms are unchanged. * feat(kv): delete an app's data on uninstall, and close account-deletion gaps Uninstall: - POST /auth/revoke-user-app takes an optional `deleteData` boolean, only with `permission: '*'` (400 otherwise). AuthService.uninstallApp revokes every grant, deletes the app's worker sessions, then awaits `app.uninstalled` { userId, userUuid, appUid, deleteData }. - With deleteData, KvShareService retires the handles the user shares out of that app's namespace (as the platform, in batches) and then empties the namespace in the background, 5,000 items per step, stopping on shutdown. Handles shared with the user are not touched. - SystemKVStore.purgeNamespace(namespace, { maxItems }) deletes up to maxItems items of one `v1:<uuid>:<appUid>` namespace, reading keys only, with no mutation events or usage, and reports `done` once it is empty. It refuses the system namespace and anything else. flush shares its page-delete path. - puter.perms.revokeApp(appUid, '*', { deleteData: true }); the desktop's uninstall dialog has an unchecked "Also delete this app's data" option. Account deletion: - EventsService settles each of a deleted account's apps on `user.delete` (they cascade without `app.changed`) and drops the owner's subscription keys. - The flat permission entries the account held are purged in pages. Suspension: - `user.suspended` / `user.unsuspended` events; the share-handle store drops cached rows owned by that user here and on other nodes. * fix(permission): await a peer's flat-permission deletes so emitAndWait sees them applied * fix(kv): strongly consistent purge pages, and an uninstall purge stops once the app is authorized again A stale empty page could end a purge as done while items remained. The background purge after an uninstall now checks before each step that the user hasn't authorized the app again, so data written after that is kept. * fix(kv): flush keeps eventually consistent page reads; only a purge reads strongly consistent * fix(user): emit user.suspended and user.unsuspended when the user store changes an account's suspension	2026-10-26 09:43:55 -07:00
Daniel Salazar	589a98b12c	feat(kv): share KV prefixes with other users at watch/read/write, and bound what apps can be granted (#4331) ... * feat(kv): share key-value prefixes with other users, and bound what apps can be granted - KV share handles move out of Events into KvShareService, served at /kv/shares (the /events/kv-handles routes stay as an alias). - Shares have watch, read and write levels, encoded in the permission root (kv-share, kv-share-read, kv-share-write) with write implying read and watch. - Every puter.kv method takes optConfig.handle; keys are relative to the shared prefix, cursors are sealed, and private entries stay hidden. - New puter.kv.share, unshare, listShared and listSharedByMe; optional label; kv.share.received notice in the app's mailbox only; recipient blocks apply. - puter.kv.set gains onlyIfAbsent. - fs and kv events gain viaWorker, and kv events report self from the writer. - /auth/grant-user-app only accepts canonical permissions from a fixed set of families; the consent dialog, puter.perms and docs follow. - Migrations: kv_share_handles gains revoked_by, label, and grantee and app indexes (mysql_mig_49, postgres_mig_38, sqlite 0095). * fix(kv): never name a private key in a cross-app or handle cursor, and bound flat-permission scans - Listings of another namespace (cross-app or through a handle) resume after the last entry they returned, and their cursors are sealed, bound to the namespace or handle, and padded to one length. Old plain cross-app cursors are still read when they name the target namespace; other namespaces get 400. - PermissionStore scans flat permissions a few roots at a time and skips roots nested under another root in the same call.	2026-10-25 08:16:19 -07:00
Daniel Salazar	9ff224e22f	perf(backend): cut sequential round trips on auth, fs, driver, event and site paths (#4259) ... Pipeline - CORS preflights send Access-Control-Max-Age: browsers stop re-preflighting authed calls every 5 s. - `req.ua` is parsed on first read instead of on every request. - JSON `rawBody` reuses the parsed buffer instead of copying it. - `emitAndWait` returns early when nothing listens: no span or closure per unlistened driver call. - SQLBatcher sends a lone query on the next tick while fewer than 8 flushes are in flight, and waits `maxTimeInQueue` as before otherwise. Lone reads skip the 10/30 ms wait; batching under load is unchanged. Auth and permissions - Token-to-actor auth starts the user, session and app lookups together and checks them in the same order: fewer sequential Redis round trips per authed request. - get-user-app-token and check-app reuse the resolved app row. getOrCreateApp skips the re-select after its own insert and doesn't wait on the cache write for replica rows. - getOrCreateApp reads the race winner from the primary when its insert loses, so it no longer hands out a session that was never written. - whoami runs the email check and the OIDC/taskbar/team lookups in parallel; key order unchanged. - OIDC providers are memoized once every discovery succeeds, with one discovery fetch in flight per URL, and the Apple key is parsed once. - gui-token and sync-cookie use `req.actor.user` instead of re-reading the user. - listSessions batches app lookups. App icon probes run together, and the original-icon probe skips the lineage fallback. - Default-granted permissions are answered from config before the scan cache, and exploded options are deduped: one less Redis read per /drivers/call. - ACL resolves ancestors lazily (owners and full-access tokens skip them) and scans the node first, then the ancestors concurrently on a miss. Filesystem - Missing-path lineage lookups run 10 at a time: batch-upload path resolution drops from about 1.3 s to about 40 ms per 100 new files. - MySQL single-file inserts return the row built from the written values, falling back to the primary read: one less primary round trip per new file. - Batch thumbnails are stored with one UPDATE per chunk (200 rows or 4M chars), with a per-row fallback. - storage.quota.bonus listeners run concurrently next to the usage SUM, and the payload carries `userUuid`: quota checks and /df no longer serialize the listener reads. - readdir cache backfill is one pipeline instead of 3xN SETEX. - Profile reads skip the lineage fallback; the own profile is read alongside the user lookup. - Legacy /stat runs is_empty (new `hasChildrenByParentUid`), owner and thumbnail concurrently. - Downloads and the video proxy pipe through `pipeToResponse`: aborts release the storage stream at once with no extra 'close' listeners, and the video proxy no longer crashes on a mid-stream upstream error. - mkdir looks up target and parent together. - Upload completion reuses the session from the access check (status is still re-read before completing), and multipart signing reads it once. - Moving a shared file to Trash skips two lookups that always returned the entry itself. Drivers and events - Subscription resolvers run concurrently and are folded in registration order. - SubdomainDriver and WorkerDriver overlap independent lookups. - The events "anyone subscribed?" check is one Redis pipeline when the region is warm. - Handler publish overlaps the count with the size check. - Broadcast flushes peers concurrently over a keep-alive agent, so a slow peer no longer delays cross-region invalidations to the others. Sites, share and AI - Hosted sites resolve the owner first, then the app and root together. Site config loads only on 404s. The hosted-app query selects only the columns it uses and moves into AppStore. - shareFlags asks about the plan only when an item is shared by link only. - Share listings run the count, teams and plan reads alongside their page query. - Claude attachment cleanup no longer blocks the response. - New util `awaitLater` for promises started early and awaited later.	2026-10-17 12:03:35 -07:00
Daniel Salazar	ecef154630	refactor(services): page fs subtree walks, gate share fan-out, one fs ACL helper (#4190) ... * refactor(acl): read ancestor chains from FSEntryStore and share one fs ACL helper getAncestorChain (split a path, batch-read the entries) moves to FSEntryStore, with getAncestors for callers that want the rows. Services read it from the store instead of calling FSService sideways; FSService.getAncestorChain stays for drivers and extensions. ACLService gains fsDescriptor (memoized chain), assertFsAccess, assertFsCreate and refusal, plus the pure aclRefusal mapping. The eight hand-rolled descriptor and safe-error copies in FSService, ShareService, events/authorization, FSController, legacyFsHelpers, LegacyFSController, AuthController and WebDAVController use them. Each route keeps its status, legacyCode and message: fs routes answer access_denied on a 403, the legacy routes still hide refusals from apps as 404, share and events keep forbidden and their own 404 text. * perf(fs): page recursive deletes and batch their event dispatch A recursive delete read every descendant into memory, and an empty-check for a non-recursive one did the same. It now walks the subtree a page at a time in descending path order (children before the directory holding them, so the parent_id cascade never takes a row before it is handled), deleting objects and rows and publishing each page as it goes. A non-recursive delete probes with one LIMIT 1 query. Removals publish once per entry, after the row is gone (the same change as the open fix for the double emit), with one surviving-ancestor chain read at most once for the whole subtree. Events-API dispatch for a batch runs as one background task at bounded concurrency instead of a fire-and-forget promise per entry. FSEntryStore gains hasDescendants and listDescendantsInOrder (keyset paged by path, then id). * perf(fs): copy directories a level at a time A directory copy walked every descendant in memory and copied them one by one: an insert per directory and, per file, an S3 CopyObject, a batchCreateEntries of one and its read-back. It now pages the source in ascending path order (parents before children) and takes each page a depth at a time: rows for directories, links and empty files at bounded concurrency, then the depth's S3 copies at bounded concurrency and one batchCreateEntries for all of them. A single-file copy goes through the same path. A failed S3 copy still fails the call (a ghost file still answers 404), but only after the objects that did copy have their rows, so nothing is left in the bucket unreferenced. Copied entries publish per depth in one dispatch. * refactor(share): build masked share paths in one place sharePathMask exports maskedRoot(owner, uuid, name) and maskUnder(root, realPath). ShareService's maskedPathVia goes, its maskedSelfPath and shareDeepLink's maskedSharePath build on them, and the masker, maskUnderAnchor and learnShareRoots use maskedRoot. Output is byte-for-byte the same; tests pin it. * fix(share): send old_path on a recipient's item.moved The desktop's item.moved handler reads old_path; ShareService sent the former masked path only as from_path. Both are sent now. * perf(share): skip the audience lookup for owners with nothing shared Every fs write, create, rename, move and delete looked for share recipients: an ancestor walk plus two share queries, for an owner who in nearly every case has shared nothing. #sharesReaching now reads the chain through FSEntryStore.getAncestors (no third copy of the walk) and asks a per-owner "has anything shared" flag before the share queries. The flag is the per-user cache events already keeps for subscriptions (SubscriptionCache): one ShareStore.ownerHasShares read per owner per TTL, coalesced across a burst. Creating a share (user, team, claimed invite) or moving a subtree to a new owner bumps it locally and over outer.pubsub.share.ownerShared; the TTL covers a process that misses the bump. A failed lookup reads as shared, so the full lookup decides. * perf(share): batch the per-row reads behind share listings Share listings looked things up one row at a time: - teams by id: TeamStore.getByIdsIncludingDeleted (the single-id read calls it) for listSharedByMe, listSharesOf and #revokeGroupSharesBy - team shares per node and ancestor: ShareStore.listGroupOnFsentries, one query for the chain, in the same node-then-ancestor order - #liveGroupGrants: PermissionStore.queryIssuerGroupPermsByPrefixes reads every issuer/group pair under every prefix in one query - #revokeGroupSharesBy: the five canManagePermission checks per row run together instead of one after another - ShareNotificationService #unblocked: users via getByIds and blocks via UserBlockStore.blockersAmong, two queries instead of two per member - digest flush: entries are taken at bounded concurrency rather than one at a time * perf(events): ask background consent once per holder and app on a revoke settle Withdrawing events:background re-checked consent row by row, resolving the holder's actor and asking the permission service for every one of their background rows. Rows of one holder and app stand on one consent, so it is asked once and shared. * refactor(events): drop the registry's unused push fields notify, defaultDelivery, PushMessage, NotifyProjection and pushProjection were read only by registry.test. Every subject was unpushable and broadcast by default; the subscription's delivery class stays in DeliveryClass, which the stores use. * refactor(share): retire trashed shares through onEntryOwnerChanged onEntryTrashed was a copy of onEntryOwnerChanged: the same subtree share sweep, grant retire and row delete. * fix(fs): page subtree walks by depth, not path MySQL sorts a PAD SPACE string on its first max_sort_length (1024) bytes, so under utf8mb4_unicode_ci paths past about 512 characters tie and fall back to id. A file moved under a newer directory then sorted after it: the delete walk reached the directory first and the parent_id cascade took the file without its object delete or event, and the keyset skipped or repeated rows. Pages now order by segment count, then id, which no collation can reorder. * fix(share): cache an owner as shared when the bump arrives A bump only dropped the cached answer, so the next lookup could read a replica that hadn't applied the new share yet and cache "nothing shared" for another TTL. The bump now records "shared" outright. * fix(fs): probe a directory for children by parent_id, from the primary * fix(fs): keep the copied-size check in the batched directory copy A copied file records the size CopyObject actually duplicated; growth past the source row is charged against the allowance, and a copy that no longer fits is deleted and refused, as a single-file copy already does.	2026-10-11 21:54:13 -07:00
Daniel Salazar	360f6b1079	fix(events): bill reached deliveries, bound dispatch reads, seal cursors (PUT-1900) (#4146) ... * fix(events): bill reached deliveries only, bound dispatch reads, seal cursors - A broadcast socket copy is billed only when a connection for its holder exists on this node, anywhere in this region (connection count) or, for a durable row, in a region presence names. Without peers it bills as before. - Compiled match filters are held in a bounded LRU with a TTL, and dropped when another process reports a subscription ended. - GET /events/fetch reads the continuation from `cursor`, falling back to `after`. - Cursors from /events/fetch, /events/subscriptions and /events/kv-handles are encrypted with sealCursor/openCursor; plain cursors still read. - A durable subscribe counts again after its insert and removes its own row when over the cap; the generation bump for that removal is published. - Subscribe checks access before reading the anchor's owner, and an anchor gone in between answers with the subject the caller sent. - Dispatch reads at most the rows per token routing can use, scanning large token hashes; a removal still settles every row on the removed anchor. - puter.events.unsubscribe() stops routing after the server ends the subscription or reports it gone; fetch() accepts `cursor`. * fix(events): forward kv values only to regions with rows that ask - A region counts its session rows asking for kv values per token and announces the token's value variant (`v#<token>`) over the existing `watch` item on the first such row, withdrawing it with the last. Add, remove, reap, reanchor and refresh all keep it, reading `includeValue` off the row as it is dropped. - A kv write reads the value variants alongside the watch index and attaches the value only to forwards bound for a region that announced one. * fix(pagination): seal id-keyed cursors on every list endpoint Apps, subdomains, workers, team (members, directory, teams, audit, own audit), shares (inbound, outbound), the user-to-user permission audit and readdir/descendant listings now hand out cursors sealed with sealCursor and read them with openCursor, so a cursor no longer carries a row id. Plain cursors issued before still read. * test(events): pass socket ids to noteConnect in the billing tests	2026-10-09 00:08:13 -07:00
Daniel Salazar	dcfb1263de	fix(events): single-writer presence items, live-socket counts and region-wide leave grace (PUT-1759) (#4142) ... * fix(events): keep each region's presence item written only by that region - A region that receives a forward for a pair it holds no socket for retires its own presence item through the pin-ordered leave path, gated by a region-shared claim. The sender no longer retires another region's item, and the forward reply no longer carries noSocket. - Presence reads skip items for regions outside the peer list instead of retiring them. - removeConnection decrements and deletes the count in one script. - A failed join releases the join pin, and any connect that finds the pin free writes the join. - A single delivery treats a socket on any node of the region as local, using the region-wide connection count, and picks the next remote candidate by the regions already tried instead of by index. - Correct the connection-count TTL comment. * fix(events): count presence by live socket and honor the leave window region-wide - A region's presence for a pair is a sorted set of socket ids scored by last renewal under a new key, replacing the integer count. A socket counts while renewed within the concurrency-slot window; add, renew and remove each prune lapsed members and report the live count in one script, so a node that dies stops holding the pair. - The last socket going marks the pair as leaving region-wide for the grace window, in the same script; a connect clears it. A forward that lands on another node inside the window does not retire the item. - Add the events.presence.write counter, labelled by op (join, leave, refresh, retire).	2026-10-08 23:55:55 -07:00
Daniel Salazar	80736de0f1	test(events): wait for the recorded failure, not any future lease score (#4127) ... The in-flight claim's lease (2x the invoke timeout) is also a future score, so polling heldForMs > 0 could pass before the failed attempt's hold was written: the backoff test read 600 instead of 2000, and the 404/hang/429/suspended-owner tests ended while their attempt was still in flight, letting it land in the next test. Wait on the subscription's failure counter instead, which only moves after the hold or gap marker is written.	2026-10-08 11:39:06 -07:00
Filip Kujundžić	b6150d63f3	Merge pull request #4090 from HeyPuter/fix/ai-stream-cancellation ... fix(ai): propagate cancellation and stream termination failures	2026-10-08 13:33:50 +02:00
Daniel Salazar	199d72cbdf	fix(events): refuse scoped access tokens on durable events routes (PUT-2048) (#4105) ... Durable subscribe, list, fetch and unsubscribe now apply the same scoped token check as the handler, worker and kv-handle routes: `isAccessTokenActor(actor) && !isAccountContext(actor)`. An access token an app issued (effectiveApp = the app) is treated the same as one the account issued: subscribe answers 403 `events_durable_requires_account`, list and fetch return an empty page, unsubscribe answers 404. `rowInActorScope` returns false for these tokens, which covers durable unsubscribe and ack. The session verbs that also use it already refuse access tokens at the socket handshake. Full-access tokens, app tokens and sessions are unchanged.	2026-10-07 22:35:09 -07:00
Daniel Salazar	e477bca3eb	test(events): wait for the failure hold, not any future lease score ... The in-flight claim's lease (2x the invoke timeout) is also a future score, so polling for "held" could read it before the failed attempt's backoff hold was written. Wait for deferAfterFailure to settle instead.	2026-10-06 22:32:02 -07:00
Juan Fernando CastroandDaniel Salazar	a46eb15d5e	Let the account's own API token reach the routes a privileged app needs (#4032) ... * feat: let the account's own API token reach the routes it should A privileged app is launched with the GUI session token today. Running one on a full-access personal access token instead costs much less when it leaks, but several routes those apps depend on refuse every access token, so the swap would break them. Admitted, each a read or a write the token's own HTTP reach already covers: - the realtime socket handshake, which refused access tokens outright; FS live updates and notifications stop without it - GET /get-dev-profile and POST /profile, both puter.js surface - GET /auth/list-permissions — a read of what was granted, not a grant - GET /share/shared-by-me/apps and GET /share/audit, over grants /share/shared-by-me already lists for the same credential - POST /rao, which puter.js calls on every setAuthToken Gated on `isAccountContext`, not the `full_access` claim alone, so a token with an app anywhere in its chain is still refused and a socket for one never joins the user room. Left session-only: mint, session, 2FA, grant, team and billing, plus two the list named. /share/blocks is a personal safety control whose only caller is the desktop's Blocked Senders window. /app-feedback is reachable only from our own GUI pages, and being unsubmittable programmatically on a user's behalf is the property it was built with. The new HTTP suite drives a real server with a minted token and fails on all six admissions without the change; the block routes' existing metadata check already pins their refusal. * test: sort the two token shapes at the events handshake * fix: carry the account-context invariant into the HTTP gate `requireUserActorGate` admitted on the raw `full_access` claim while the socket and /rao use `isAccountContext`, so one shape — a token carrying the claim with an app in its chain, or an actor built without `makeActor` — would have been refused in two places and admitted in the third. Unreachable today, since the mint and the auth-time read both drop the claim when `app_uid` is present, but it is the invariant this change states and it should hold at every enforcement point. The payout address goes back behind a plain session: `/get-dev-profile` is open to the account's own token now, and the rest of what it answers is a developer's own name and incentive-program status. Also drops a cross-reference in EventsService to the socket posture this change relaxed, and rewords the block-routes comment, which read as if access tokens were admitted there rather than refused. * fix: drop a revoked token's sockets, and read the dev-profile columns Two gaps the review found, both reachable only because this branch lets the account's own token hold a socket and call /get-dev-profile. `#revokeAccessTokenTail` soft-revoked the session row and told no one, so a revoked token kept its connection — and the account's `outer.gui.*` fan with it — until the five-minute reauth sweep. Sockets now join a per-token room and a new `auth.access-token.revoked` event drops exactly that room, so the account's other tabs stay up. Session revoke keeps its account-wide eviction. `/get-dev-profile` read `first_name`, `last_name`, `paypal` and the two incentive flags off the user row, which carries none of them: the columns have always been `dev_`-prefixed, so the endpoint answered nulls to everyone and `puter.apps.getDeveloperProfile()` has never returned anything. Reads the real columns, keeping the unprefixed name as a fallback for a deployment carrying both. The payout address stays behind a plain session, so opening the route to the account's token hands over a name and incentive status and nothing else. Both fail without the fix: the socket test waits five seconds on a connection that should already be gone, and the profile test reads back a seeded row. * fix: close /open_item to tokens, and stop the node runner leaking sockets `/open_item` carried `allowFullAccessToken`, but it grants an app a permission on the user's behalf and mints the app token to go with it — a grant and a mint, which stay session-only. Pre-existing, and missed by the first sweep here, which only asked whether routes lacking the flag should gain it and never re-read the ones that already had it. Refused at the top of the handler too, so a credential that gets past the gate cannot leave an ACL row behind. The node runner loads each SDK into a vm context and never closed it, so every test leaked a socket, and every socket holds a per-(user, origin) connection slot. Admitting the account's own token to the handshake pushed that past the cap, and the whole events block failed with `events_connection_failed` while the browser and workerd runners — which do not leak — passed. Closing the FS socket and the events channel after each test gives the slots back. * fix: announce only the revokes that can have a socket to drop Every access-token revoke broadcast an eviction cluster-wide, including the read-URL tokens `revokeReadUrl` retires constantly. A scoped token is refused at the handshake, so none of those broadcasts could ever reach a connection. The announcement now goes out only for a token the handshake would have admitted: full access, no app in the chain. `revokeOwnAccessToken` refuses full-access tokens outright, so that whole path is silent; a revoke by raw uuid says nothing about the token and still announces, which costs a no-op broadcast rather than leaving a revoked socket up. Fails without it: the test revokes a scoped token and a full-access one through the same entry point and counts the announcements. * test: pin the mint route against the account's own token `/auth/create-access-token` carries only `requireAuth`, so it never showed up in the sweep over `requireUserActor` routes — the refusal lives in `AuthService.createAccessToken`, which turns away any access-token actor outright. That holds today, and a leaked full-access token minting itself a sibling that survives revoking the original is exactly what the token swap is meant to rule out, so it is worth a test rather than a reading. * fix: let the recommended-apps test follow the list it tests Two commits on main rewrote `RECOMMENDED_APP_NAMES` without touching the test, which seeds `editor` and `camera` and asserts the result contains them. Neither is on the list any more, so the resolved set came back empty and `test (base)` has failed on every PR since. The test now seeds the first two names off the list itself and asserts on those, so the next edit to it cannot strand the test: prepending two unknown apps keeps all four cases green. * feat: let an extension waive a plan gate for one route or driver method A plan gate about to refuse a caller now emits `subscription.gate.<surface>` through `emitAndWait` first: `route.<method>.<path>` for a route, `driver.<iface>.<method>` for a driver method, `share.anyone` for link shares. The payload carries the actor, the request and the requirement, and a listener that sets `allow` waives the plan for that call. Callers the plan already covers never reach the hook, and a listener that throws leaves the refusal in place. * feat: mint godmode apps their own full-access token `/auth/get-user-app-token` answers a desktop launch of a godmode app (the `godmode` column) with a full-access token instead of an app token. Origin lookups, the sign-in popups for pages outside the desktop, keep getting ordinary app tokens, and only a web session may mint one. The token carries the app in `godmode_app_uid`, read into `accessToken.godmodeApp` for attribution only: `effectiveApp` stays null, so it keeps the account's reach and passes the same gates as a personal access token, including the vendor-compatible AI routes. It is refused once the app is gone or no longer godmode. Its row is an `access_token` session parented to the web session that asked, with `app_uid` set, so signing out, revoking that session or revoking all sessions takes it along, and the sessions list shows it under the app. Each mint re-signs onto the same row with a 12h `exp` and moves the row's expiry out, so the desktop keeps an open app alive while a copy taken earlier still expires on schedule. A full-access token may now mint scoped tokens (read URLs), parented to its own row; it still may not mint full access. Revoking a full-access token cascades to the tokens it minted. An access token whose session row has a parent now authenticates only while that parent is live, so a read URL a godmode app minted lapses with the app's token, and through it with the desktop session. * feat: launch godmode apps on their own token and renew it The desktop asks for a godmode app's token at launch and blocks the launch if it can't get one, as it does for other apps; a godmode app never runs on the desktop's session token. While the window is open the desktop re-mints the token once less than 6h of its 12h remain, checking on a timer and when the tab becomes visible, and answers the app's `reauth_required` with a fresh one, posted to the app's origin. puter.js keeps a godmode token in sessionStorage like a session token, never in localStorage. In app mode a 401 on a godmode token, background requests included, asks the desktop for a new token and replays, and a `puter.token` message is only taken from the embedding frame. The sessions list shows a godmode token row with its app's title and icon. * fix: keep a godmode token the same across renewals Apps hand their launch token to code that never sees a replacement (the terminal's Drive mount, a CLI's API key), so a token re-signed every renewal broke those copies once the first one expired. The JWT now carries no `exp` and a renewal moves only its row's expiry, so every copy keeps working while the desktop keeps the app alive. What still bounds it: - only the desktop's web session can extend it; using the token never does, and signing out or revoking that session revokes it; - it lapses 12h after the desktop last asked, and a lapsed row is never revived: the next launch gets a new token; - however often it is renewed, a token is replaced after 7 days, and the old one is revoked on the spot along with the tokens it minted and its sockets. The desktop no longer re-posts a renewal that kept the same token, so the app's sockets aren't rebuilt every few hours; an app that reports its token stopped working is always answered. * fix: reserve the feedback and no-reply mailbox names again Signup checks names against `util/reservedUsernames`, but `fbl`, `noreply` and `no-reply` were added to a second copy of the list in AuthController that nothing reads any more, so they became claimable. Move them into the live list and drop the copy. --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-10-05 01:10:29 -07:00
Daniel Salazar	007d5391a1	fix(events): carry handler depth onto tokens a handler mints (#4049) ... An events handler could mint a fresh token for its own app through /sign or /auth/create-access-token, and writes made with it started a new chain. An app token minted from a handler token now carries its handler_depth and expires with it; an access token carries the depth at the lifetime asked for.	2026-10-04 00:25:54 -07:00
Daniel Salazar	a242105540	fix(events): keep app-less scoped tokens off the account's durable rows and mailbox (#4042) ... A personal access token scoped below the account (e.g. the one in an account-minted getReadURL() URL) has effectiveApp === null, which the durable list/unsubscribe/ack scope check and notif fetch read as account context. Gate those on isAccountContext, as workers/handlers/kv-handles already do, and refuse such tokens on durable subscribe.	2026-10-02 22:37:07 -07:00
Daniel Salazar	91c8b2385b	fix: run a broadcast persistent handler in the worker or the clients, not both (PUT-1889) (#4016)	2026-10-02 00:08:02 -07:00
Daniel Salazar	ed7b8f828a	fix: kv path errors and ttl hardening, events subscription end and race fixes (#4004)	2026-10-01 09:50:34 -07:00
Juan Castro	d1b11c793c	Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a ... One conflict, in workerInvoker.integration.test.ts: #3997 fixed the same gap-marker flake this branch did, with a different barrier. Took theirs — waiting on the `ev:qf` counter, which `#handlerFailed` writes strictly after `discard`, so the marker is in and the lease cleared by the time it reads 1. Their ordering also asserts the counter before flipping `answer` back to 200, which closes the redelivery-clears-failures hole this branch's version had removed the line to avoid. Converted the two `vi.waitFor` calls #3997 added to `waitUntil`, the helper this branch introduced for the rest of the file. Neither is wrong today, since the new block fakes no timers, but a `jump()` added near them later would hit exactly the drained-backoff trap the helper exists for.	2026-09-30 17:49:36 -04:00
Daniel Salazar	4448103ef0	fix: misc events and kv issues found during recipes (#3997) ... * fix: misc events and kv issues found during recipes * test: cover handler depth on narrowed actors and created notifications * test: fix stale legacy batch directory upload test and seat user-agent flake * test: wait for the refusal to be recorded before claiming its gap marker	2026-09-30 14:40:47 -07:00
Juan Castro	79e496e79d	test: wait on the gap marker rather than a depth that never moves ... The refused-delivery test waited for `depth(subId)` to reach 1 before claiming the marker, but `discard` removes the entry and appends the marker in its place, so the depth is 1 before, during and after. The wait returned at once, and the claim came back `inflight` — the refused delivery still held the lease it was invoked on — leaving the assertion reading `undefined`. It only passed on the slack in `invoked`'s poll interval, which a loaded runner takes away. Waiting on the claim itself is the barrier the depth was standing in for. Dropping `answer = 200` with it: `beforeEach` already resets it, and a redelivery settling before the last assertion would clear the failure count it reads.	2026-09-30 16:36:57 -04:00
Juan Castro	f8d8003fea	test: stop the delivery-backoff test reading holds against a moving clock ... `holds one it could not answer, for longer each time` freezes `Date` so a hold written as `now + backoff` reads back as exactly `backoff`. But every wait in the file went through `vi.waitFor`, which advances the faked clock once per poll — measured at 50ms a poll. Each poll between the hold being written and read therefore ate into the value: CI read a 2s hold as 525ms and failed, on main one run and on this branch the next. Polling now goes through `waitUntil`, which retries on `performance.now` and real `setTimeout` — neither faked — so the clock only moves where `jump` moves it. The holds then read exactly 2s/4s/8s/16s, so the assertion is an equality rather than a one-second window that a slow enough runner could always slide out of.	2026-09-30 12:25:40 -04:00
Daniel Salazar	f80d4ae136	fix: events docs (#3968) ... * fix: events docs * fix: docs	2026-09-28 17:24:37 -07:00
Daniel Salazar	132cae9f6a	Bug bash fixes (PUT-1876, 1877, 1880, 1881, 1948, 1949, 1951) and KV event fan-out by plan (#3943) ... * fix(kv): keep disableSharing entries out of other apps' events (PUT-1876) Private kv keys and values are no longer delivered to cross-app subscriptions, including forwarded and durable deliveries. * fix(fs): refuse usernames whose home path still holds another account's rows (PUT-1877) Username claims (signup, save_account, change-username, OIDC, team provisioning) now also check for leftover rows under /<name>/, and the root listing only returns the actor's own home. * fix(puter-js): reconnect the events channel after a server-side disconnect (PUT-1881) A server hang-up is retried with a bounded backoff; persistent handlers survive it and get an optional onError when the connection can't be restored. * fix(backend): let the graceful shutdown finish before telemetry exits (PUT-1948) The telemetry preload no longer exits on signals; the entry point drains, tears down top-down, flushes telemetry last, then exits. DB pools stay open until teardown. * fix(fs): reserve quota for signed uploads in progress (PUT-1880) startWrite/startBatchWrite hold each upload's declared size in a per-owner cache reservation until it completes, aborts or expires, so a burst of starts can't all read the same committed usage. Caps in-flight uploads at 10,000 per account. * fix(fs): refuse to complete a signed upload whose object never arrived (PUT-1951) Completion now 400s when the object store reports the key missing (transient errors stay lenient), abort no longer deletes an object a live entry still uses, and a ghost entry is only removed after a second miss at its recorded location. * fix(metering): move usage to v2 keys with a small totals item, per-model detail shards and a 3-month TTL (PUT-1949) The billed item holds only totals; per-model detail lives in 100 hash-sharded items read in one batch and cached for 60s, and more than 5,000 distinct usage types in a month fold into other. Global and per-app aggregates store totals only, appTotals comes from a prefix listing, a refused path only skips the key that refused it, exact reads near the allowance are throttled per key, and every metering record expires after 3 months. September usage restarts at deploy; the monthly-charge claim stays on the v1 key through September so recurring charges don't fire twice. * test(events): let the worker backoff test tolerate a loaded run * feat(events): scale KV event fan-out with the key owner's plan A KV change delivers to up to 512 matching subscriptions (2,048 filter evaluations) for a paid key owner and 128 (512) otherwise, inline values are dropped once more than 128 match, and paid accounts may hold up to 512 share handles per app. The owner's plan is only looked up when a change has more candidates than any cap. * test(metering): seed the 150-app listing test directly so it fits under coverage	2026-09-24 16:00:52 -07:00
Daniel Salazar	4c3c684cb4	fix: refuse app-issued scoped tokens on events handler and worker routes (#3935) ... * fix: new cache costs for claude * fix: refuse app-issued scoped tokens on events handler and worker routes An access token an app mints (e.g. a getReadURL() token) resolves effectiveApp to the issuing app, so #handlerApp and listEventsWorkers treated it as the app: it could publish, remove and list handlers, and list or destroy the app's worker, regardless of its permission manifest. Refuse any scoped token there, as the kv-handle routes already do. * test: freeze the clock before the first backoff read in workerInvoker The first hold was read on real time, so a slow run measured the 2s backoff as 1s.	2026-09-23 13:58:49 -07:00
Daniel Salazar	b3fda186e7	fix: let apps read opted-in teams, scope worker listing, fix revokeReadURL (#3933) ... - GET /teams and /teams/:uid/members admit apps for teams whose owner opened the directory; apps get active members' usernames only - events.workers.list() from an app returns only its own worker - hide account-session-only SDK methods from the public types - unpublish the updateProfile docs page (redirects to getProfile) - revokeReadURL() works through a new /auth/revoke-own-access-token route	2026-09-23 12:06:56 -07:00
404oopsandClaude Fable 5.1	027e9a71f3	fix(ai-image): refresh model catalogs, unify txt2img requests, expand docs (#3889) ... * fix(ai-image): sync image catalogs, unify request shape, document models Model sync against every vendor listing and a live generation sweep: Gemini stable ids replace the retired preview spellings (2.5 Flash Image delisted ahead of its 2026-10-02 shutdown), OpenAI gpt-image-1/-1-mini/-1.5 are delisted but routable until their shutdown dates with gpt-image-2 as the default, xAI gains grok-imagine-image-2.0 with its quality tiers, BytePlus gains the 3K/4K tiers and per-model pixel bounds, Cloudflare gains SDXL Lightning/Base and SD 1.5 Inpainting, Replicate gains a schema-driven catalog of 88 additional models with version-pinned community predictions, and every Together image route is excluded for the third-party data-sharing requirement. Request normalization: the driver collapses ratio/width/height/aspect_ratio into one imageSize with aspect-versus-pixel intent, validates prompt, quality and resolution once, resolves provider hints (short names or full driver ids) and aliases with exact ids winning over resellers, and hands each provider an immutable copy of the caller's args. Providers share prompt validation, aspect snapping, and content-sniffed data URIs so responses carry the right MIME type. Replicate predictions are created once, cancelled on abort or deadline, and bounded in input fan-out. SDK: txt2img copies caller options, rejects blank prompts with prompt_required in every call form, and documents that normalize has no effect because the image return shape is already uniform. Docs: txt2img options per provider, provider defaults and discovery, availability notes, a new image model catalog and pricing page, and the image-generation limits in the quotas page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(ai-image): sniff SVG outputs correctly, reject hex/exponent dimensions imageDataUri only decoded the first 24 base64 chars (18 bytes) before sniffImageMime, which cannot see an <svg> root behind an XML prolog - SVG outputs from recraft-v*-svg models were being labeled image/png. Decode enough of the payload to cover the 8 KB SVG sniff window. dimension() accepted string number literals ('0x10', '1e3') as if they were decimal dimensions; restrict to plain decimal notation. * docs(ai): link to the model directory instead of a static catalog The image model catalog and pricing page duplicated the always up to date model directory at developer.puter.com/ai/models. Link to the directory from txt2img-related docs and keep the Together data-sharing exclusion note self-contained. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-21 15:00:02 -07:00
Juan Castro	4e821c128c	Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging ... # Conflicts: # src/backend/services/share/ShareService.ts # src/backend/stores/share/ShareStore.js	2026-09-21 10:28:33 -04:00
Daniel Salazar	9292771554	fix: hardening (#3904)	2026-09-18 11:22:39 -07:00
Daniel Salazar	0be3bc55c2	feat(metering): AI cost multiplier hook for AI drivers (#3898) ... Emits ai.cost.multiplier.<driver>.<provider>:<model> before recording AI usage, so what a model costs to charge is policy an extension owns rather than a number in core. Nothing listening records the provider cost. MeteringService.withAiCostMultiplier(driver) returns a view of the service whose recording paths scale costOverride by the hook's answer; every AI driver hands that view to its providers, so all of them are covered without touching provider code.	2026-09-17 15:18:38 -07:00
Daniel Salazar	51ce868d1e	fix: shared kv handles also allow value opt in (#3895)	2026-09-17 10:34:35 -07:00
Daniel Salazar	bb97660ad7	feat: kv events value opt in (#3894)	2026-09-17 00:17:12 -07:00
Daniel Salazar	194ca7c789	fix: list a link share only while the owner's plan covers it (#3891) ... * fix: list a link share only while the owner's plan covers it * chore: types + jsdoc cleanup	2026-09-16 14:42:52 -07:00
Daniel Salazar	ed7e0b9cc4	fix: sharing, events and worker-credential authorization hardening (#3855) ... - PUT-1800: gate `createWorkerSessionToken` on actor type, so an app or an access token can no longer mint an app-less, root-shaped worker session; `WorkerDriver` binds on `effectiveApp` instead of `app`. - PUT-1799: add `isAccountContext` and read it where "no app" was being read as "the account" — handler publish, events-worker listing, kv handle mint/revoke/list. A scoped API token is no longer an account session. - PUT-1802: re-authorize a durable row before its backlog drains, settling it permanently when the grant is gone. Covers an ancestor-level unshare, which the revoke settle deliberately leaves to the delivery re-check. - PUT-1803: mask the owner's absolute path out of deliveries and subscription anchors on a foreign node, the way every FS surface already does. - PUT-1804: let a revoke reach rows already suspended for a resumable reason, re-stamping them so a resume cannot hand over the held backlog. - PUT-1805: apply the subscribe path's audience gate to `/events/fetch` before the query, so a cursor can no longer count and name invisible notifications. - PUT-1807: refuse `mode: 'manage'` from any actor holding an app — inside its own AppData the ACL short-circuit would otherwise supply the reach. - PUT-1808: take the sending peer from the verified signature header rather than the request body. - PUT-1810: re-base a kv share-handle row's stored match filter on the handle, so the owner's absolute key prefix stays hidden. - PUT-1814: escape LIKE wildcards and anchor the issuer-prefix queries on a segment; anchor `manage:` stripping; reject a backslash in a share prefix; assert a resolved actor in `subscribeDurable`. - PUT-1815: bound the char/varchar columns behind `event_subscriptions` and `kv_share_handles` at the store layer.	2026-09-11 21:18:49 -07:00
Daniel Salazar	e5f322f68c	fix: misc event hardening + perm fixes (#3831)	2026-09-08 19:16:21 -07:00
Daniel Salazar	86050f131b	fix: hardening events for shared kv (#3827)	2026-09-08 09:04:50 -07:00