Pipeline
- CORS preflights send Access-Control-Max-Age: browsers stop re-preflighting authed calls every 5 s.
- `req.ua` is parsed on first read instead of on every request.
- JSON `rawBody` reuses the parsed buffer instead of copying it.
- `emitAndWait` returns early when nothing listens: no span or closure per unlistened driver call.
- SQLBatcher sends a lone query on the next tick while fewer than 8 flushes are in flight, and
waits `maxTimeInQueue` as before otherwise. Lone reads skip the 10/30 ms wait; batching under
load is unchanged.
Auth and permissions
- Token-to-actor auth starts the user, session and app lookups together and checks them in the
same order: fewer sequential Redis round trips per authed request.
- get-user-app-token and check-app reuse the resolved app row. getOrCreateApp skips the re-select
after its own insert and doesn't wait on the cache write for replica rows.
- getOrCreateApp reads the race winner from the primary when its insert loses, so it no longer
hands out a session that was never written.
- whoami runs the email check and the OIDC/taskbar/team lookups in parallel; key order unchanged.
- OIDC providers are memoized once every discovery succeeds, with one discovery fetch in flight
per URL, and the Apple key is parsed once.
- gui-token and sync-cookie use `req.actor.user` instead of re-reading the user.
- listSessions batches app lookups. App icon probes run together, and the original-icon probe
skips the lineage fallback.
- Default-granted permissions are answered from config before the scan cache, and exploded
options are deduped: one less Redis read per /drivers/call.
- ACL resolves ancestors lazily (owners and full-access tokens skip them) and scans the node
first, then the ancestors concurrently on a miss.
Filesystem
- Missing-path lineage lookups run 10 at a time: batch-upload path resolution drops from about
1.3 s to about 40 ms per 100 new files.
- MySQL single-file inserts return the row built from the written values, falling back to the
primary read: one less primary round trip per new file.
- Batch thumbnails are stored with one UPDATE per chunk (200 rows or 4M chars), with a per-row
fallback.
- storage.quota.bonus listeners run concurrently next to the usage SUM, and the payload carries
`userUuid`: quota checks and /df no longer serialize the listener reads.
- readdir cache backfill is one pipeline instead of 3xN SETEX.
- Profile reads skip the lineage fallback; the own profile is read alongside the user lookup.
- Legacy /stat runs is_empty (new `hasChildrenByParentUid`), owner and thumbnail concurrently.
- Downloads and the video proxy pipe through `pipeToResponse`: aborts release the storage stream
at once with no extra 'close' listeners, and the video proxy no longer crashes on a mid-stream
upstream error.
- mkdir looks up target and parent together.
- Upload completion reuses the session from the access check (status is still re-read before
completing), and multipart signing reads it once.
- Moving a shared file to Trash skips two lookups that always returned the entry itself.
Drivers and events
- Subscription resolvers run concurrently and are folded in registration order.
- SubdomainDriver and WorkerDriver overlap independent lookups.
- The events "anyone subscribed?" check is one Redis pipeline when the region is warm.
- Handler publish overlaps the count with the size check.
- Broadcast flushes peers concurrently over a keep-alive agent, so a slow peer no longer delays
cross-region invalidations to the others.
Sites, share and AI
- Hosted sites resolve the owner first, then the app and root together. Site config loads only
on 404s. The hosted-app query selects only the columns it uses and moves into AppStore.
- shareFlags asks about the plan only when an item is shared by link only.
- Share listings run the count, teams and plan reads alongside their page query.
- Claude attachment cleanup no longer blocks the response.
- New util `awaitLater` for promises started early and awaited later.
- relay sign-in popups take the opener origin from the referrer only
- session revokes read the primary, batch per user, and tombstone caches in every region
- wisp relay tokens bound to the minting session; verify rechecks the account
- hosted origins keep app identity with the site owner; deleted app uids recorded in deleted_apps
- subdomain cache tombstoned on delete, update, reclaim and app/account cascades
- email lookups require an exact address match after the SQL lookup
- workers deploy only from files the deployer owns
- share_app_data:false enforced at ACL time, plus copy/move/rename/share/recursive listing
- kv remove returns no value without read; kv:expire op grant usable
- fs errors name the entry instead of the owner's path
- shared-by-me drops rows a delegate no longer has authority over
- private apps can't move index_url onto another user's origin
- GUI: default apps saved and launched by uid; save picker accepts only copy/move; permission dialog waits for focus; Publish as Worker hidden on others' files
* PUT-2012 fix broadcast service drops
* Revert pre-commit lint churn on files merged from main
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix test bug
* Remove other test
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(workers): skip hot-reload lookups for files no worker is bound to
The fs.write.file / fs.remove.node / fs.move.node listeners listed every
workers.puter.* row of the file's owner on every event, for every user.
The event's FSEntry already carries the worker rows bound to it, so a file
with none returns before any I/O; a bound file reads just its own rows,
batched, from the primary.
* perf(apps): read the uid alias only after a direct lookup misses
Every AppDriver uid lookup fired an uncached system-KV GetItem for the
merge alias in parallel with the direct read, even when the direct read
hit. The alias is now read only on a miss.
* refactor(apps): one app view with the launch gates, built by AppService
AppDriver's serializer and its private-app / hosted-backing gates move to
AppService.views, built on a pure toAppView that is the one place
index_url is withheld. AppDriver, the subdomains driver's embedded apps and
the GUI shell view all use it. Visibility of protected apps
(filterReadable) moves with it.
Subdomain embeds (associated_app, app_owner) used to return index_url with
no gate; they now withhold it from a viewer who may not open a private app
or whose hosted backing is gone, and carry privateAccess, godmode and
feedback_enabled like a direct app read.
* refactor(apps): one toAppSummary; batch the recommended and taskbar lookups
SuggestedApps, RecommendedApps, /get-launch-apps and the taskbar each built
their own launch summary; they now share toAppSummary (the extra
feedback_enabled / external flags appear on the lists that lacked them).
getRecommendedApps awaited getByName once per recommended app (29) and
getTaskbarItems did the same per taskbar entry on every whoami; both now
make one batched call per key through the new AppStore.getByNames, which
keeps getByName's case-insensitive and old-name matching. The taskbar's
raw UPDATE of the user row goes through UserStore.update, and
/get-launch-apps checks hosted backings in one batch and reaches the
stores without casts.
* refactor(apps): app routes read through AppService; /delete-site calls the subdomains driver
AppController used the global driversContainers through a getter whose
boot-order comment no longer held; it uses the injected drivers and
AppService. GET /apps/:name looked each name up, read it again through the
driver and resolved private access a second time, emitting
app.privateAccess.resolveLaunch twice per private app; it now batches the
names and resolves once. POST /query/app did a lookup, a driver read and a
raw app_filetype_association SELECT per selector (up to 200, serially); it
now makes one batched lookup per key, one grant check and one filetype read.
The driver's read shape is a pure toAppReadView both use.
HostingController's /delete-site re-implemented the subdomains driver's
delete (protected check, delete, subdomain.delete event); it keeps its
owner-scoped 404 and calls the driver. Hosting and Desktop controllers drop
constructors that discarded the drivers argument, and trivial getters.
* refactor(drivers): one per-method policy validator and resolver; drop @Driver
Four policy blocks (rateLimit, concurrent, requireSubscription,
requireReputation) share the { default, methods } shape but each had its
own validator and resolver. They become validatePerMethod(value, label,
policy) and resolvePerMethod(cfg, method); error messages are unchanged.
The @Driver class decorator and the prototype keys it wrote had no users
outside its own tests; drivers declare their fields, and resolveDriverMeta
reads only those. DriverController takes the driver name from the meta it
already resolved instead of probing the prototype.
resolveDriverMethodRateLimit/Concurrent stay as deprecated aliases for the
email extension.
* refactor(drivers): shared actor guards and crud-q concurrency policy
The apps, subdomains, workers and notifications drivers each carried a
private #requireActor, two carried the same #requireVerified, and apps and
subdomains the same owner/app_owner write predicate. These become
requireContextActor, requireContextUserActor, assertActorEmailVerified and
actorOwnsRow in core/actor.ts; each driver keeps its own error for the
cases that differed (system actors still read apps; notifications still
answer 403 for an actor without a user). The identical crud-q concurrency
block is one CRUD_CONCURRENT constant.
The metering, installedApps and appTelemetry extensions re-checked an actor
their routes already require; those checks and the tests for them go.
* refactor(extensions): store methods instead of raw SQL in OSS extensions
installedApps and appTelemetry queried user_to_app_permissions directly
and metering read apps by name with its own SELECT. PermissionStore gains
listAppsGrantedByUser, listAppAuthenticatedUsers (with the email-share
check folded in) and countAppUsers; metering uses AppStore.getByName.
* refactor(server): one layer builder and one lifecycle hook runner
PuterServer built each of its five layers with its own copy of the same
loop (built-ins, then extension additions, into the layer and its export
container), and ran each lifecycle hook with five near-identical loops per
hook. Both are now one helper each: #instantiateLayer and
#runHook(order, hook). Construction order, the args each layer receives,
and hook order (start and prepare bottom-up, shutdown top-down) are
unchanged.
* fix(workers): tear down a deleted source's worker after its FK nulls root_dir_id
* fix(events): bill reached deliveries only, bound dispatch reads, seal cursors
- A broadcast socket copy is billed only when a connection for its holder
exists on this node, anywhere in this region (connection count) or, for a
durable row, in a region presence names. Without peers it bills as before.
- Compiled match filters are held in a bounded LRU with a TTL, and dropped
when another process reports a subscription ended.
- GET /events/fetch reads the continuation from `cursor`, falling back to
`after`.
- Cursors from /events/fetch, /events/subscriptions and /events/kv-handles
are encrypted with sealCursor/openCursor; plain cursors still read.
- A durable subscribe counts again after its insert and removes its own row
when over the cap; the generation bump for that removal is published.
- Subscribe checks access before reading the anchor's owner, and an anchor
gone in between answers with the subject the caller sent.
- Dispatch reads at most the rows per token routing can use, scanning large
token hashes; a removal still settles every row on the removed anchor.
- puter.events.unsubscribe() stops routing after the server ends the
subscription or reports it gone; fetch() accepts `cursor`.
* fix(events): forward kv values only to regions with rows that ask
- A region counts its session rows asking for kv values per token and
announces the token's value variant (`v#<token>`) over the existing
`watch` item on the first such row, withdrawing it with the last. Add,
remove, reap, reanchor and refresh all keep it, reading `includeValue` off
the row as it is dropped.
- A kv write reads the value variants alongside the watch index and attaches
the value only to forwards bound for a region that announced one.
* fix(pagination): seal id-keyed cursors on every list endpoint
Apps, subdomains, workers, team (members, directory, teams, audit, own
audit), shares (inbound, outbound), the user-to-user permission audit and
readdir/descendant listings now hand out cursors sealed with sealCursor and
read them with openCursor, so a cursor no longer carries a row id. Plain
cursors issued before still read.
* test(events): pass socket ids to noteConnect in the billing tests
- PUT-1800: gate `createWorkerSessionToken` on actor type, so an app or an
access token can no longer mint an app-less, root-shaped worker session;
`WorkerDriver` binds on `effectiveApp` instead of `app`.
- PUT-1799: add `isAccountContext` and read it where "no app" was being read
as "the account" — handler publish, events-worker listing, kv handle
mint/revoke/list. A scoped API token is no longer an account session.
- PUT-1802: re-authorize a durable row before its backlog drains, settling it
permanently when the grant is gone. Covers an ancestor-level unshare, which
the revoke settle deliberately leaves to the delivery re-check.
- PUT-1803: mask the owner's absolute path out of deliveries and subscription
anchors on a foreign node, the way every FS surface already does.
- PUT-1804: let a revoke reach rows already suspended for a resumable reason,
re-stamping them so a resume cannot hand over the held backlog.
- PUT-1805: apply the subscribe path's audience gate to `/events/fetch` before
the query, so a cursor can no longer count and name invisible notifications.
- PUT-1807: refuse `mode: 'manage'` from any actor holding an app — inside its
own AppData the ACL short-circuit would otherwise supply the reach.
- PUT-1808: take the sending peer from the verified signature header rather
than the request body.
- PUT-1810: re-base a kv share-handle row's stored match filter on the handle,
so the owner's absolute key prefix stays hidden.
- PUT-1814: escape LIKE wildcards and anchor the issuer-prefix queries on a
segment; anchor `manage:` stripping; reject a backslash in a share prefix;
assert a resolved actor in `subscribeDurable`.
- PUT-1815: bound the char/varchar columns behind `event_subscriptions` and
`kv_share_handles` at the store layer.
* feat: bake published handlers into a generated events worker
* feat: deploy and address the per-app events worker behind a flag
* test: single delivery end to end through a real local worker
* feat: events workers run their own runtime, in their own namespace
An events worker was being deployed as an ordinary worker: default dispatch
namespace, a `subdomains` row, the router preamble, and an app-scoped worker
token baked in. The public dispatcher resolves any script in that namespace
straight off the hostname, so the worker answered at `<name>.puter.work`, and
the only thing in front of it was an unguessable name plus a check that a
`puter-auth` header was present — which the router never validates. Anyone who
learned the hostname could run an app's handlers with a body of their choosing,
in an isolate holding the owner's token as `me`.
Instead:
- Handlers run on their own runtime (`src/worker/src/events-runtime.js`), which
provides no `router` and no `me`, owns the single invoke route, and hands a
handler only `{ event, ctx, user, fetch, ack }`. `user` is built from the
invocation's delivery token, so a handler acts as the subscriber whose
delivery it is and nothing wider. The preamble build emits one bundle per
runtime; the shared half of the template is now included by both.
- The deploy target carries the runtime to prepend, the source to deploy, and
whether to mint a worker token at all, so an events worker deploys into the
`events` dispatch namespace from generated source with no token binding, no
`subdomains` row, and no claim on the owner's worker quota or worker list.
- An invocation carries a key derived from the deployment secret and the script
name, bound as a secret and checked in constant time inside the isolate,
which reads it once and drops it before handler code runs.
- Scripts are named after the handler set they contain, so publishing writes
rows and deploys nothing: a set is deployed the first time a delivery needs
it, and a changed set is a new script rather than an overwrite of a running
one. Publish responses keep the shape they had before the runtime existed.
- Invocations reach a worker only through the events dispatcher, which has no
zone route and requires the internal secret; the backend's own deploy path is
the rehydrate route the dispatcher calls on a namespace miss. Locally there is
no dispatcher, so the controller hands the service an in-process transport
that deploys on miss itself.
The SDK stops allowlisting `puter` as a handler global — a handler that reaches
for an ambient SDK is now refused at publish time, naming `user` instead, rather
than passing the scan and failing on its first delivery.
Requires `events.workerNamespace`, `events.dispatcherUrl` and
`events.internalSecret`; without them nothing is addressable and background
deliveries stay retriable, as they did with the runtime off.
* fix: a handler's delivery token gets through the read routes
An events handler acts as the subscriber through the access token its
invocation carried, but every FS read route refused scoped access tokens
outright, so `user.fs.stat(event.path)` — the design's own example — answered
403 inside the worker. The read-side routes now admit them; the ACL each
handler already runs intersects the token's grant with its issuer's, which is
the check that keeps a token to what it was minted for. The end-to-end suite
asserts the stat from inside the isolate.
* fix: shorthand-method handlers publish as functions
`{ ingest({ event }) { … } }` stringifies without the `function` keyword, so
its source is not an expression and the events worker baked it as a broken
stub — every delivery a retriable 500 until the subscription suspended, with
nothing at publish time to say why. The SDK now gives a shorthand method the
keyword before hashing and sending; getters, setters and computed names are
left for the server-side check to refuse.
* feat: an app's events worker is listable and destroyable
An app with published handlers has an events worker, and hosted deployments
bill it monthly per app, so its owner needs to see it and be able to take it
down. The core announces the lifecycle on the bus — `events.worker.create`
when an app's first handler is published, `events.worker.destroy` when its last
one goes — with the owner as the actor, so pricing can plug in from outside.
`GET /events/workers` lists the caller's workers (paginated, with the script
each set deploys as) and `POST /events/workers/destroy` removes every handler
of an app under the same owner scoping as the handler routes, suspending the
subscriptions bound to them. `puter.events.workers.list/destroy` in the SDK,
a docs page, and a 5 MB cap on an app's combined handler source
(`events_worker_too_large`) so a set that publishes can always deploy.
* fix: harden the events worker runtime for production
- A 4xx is terminal only when it carries the handled marker the runtime (and
the dispatcher) stamp on every answer that came from a script; an unmarked
4xx — an edge 404 for a wrong dispatcher hostname, a WAF page — stays
retriable and is logged, once per script per minute, with the runtime's
reason header.
- Script names are scoped to this backend's exposed API origin, so two
backends sharing a namespace never resolve one script with the wrong
endpoint binding or key. Shape unchanged.
- Each handler is validated in the exact context it is emitted into and the
whole generated file is compiled once; a source that would break the script
marks every handler broken instead of deploying a SyntaxError.
- Locally, events scripts live under their own registry key: the public local
worker host cannot reach them and an ordinary worker cannot take their name.
- A suspended or deleted app owner stops invocations; deploys are throttled
per app per hour; in-flight deploys are keyed by app and script; the
upstream deploy call times out; the generated source is size-capped with a
margin over the publish cap; boot fails when the runtime is on but its
preamble is not built. Byte-length secret compare, appUid shape check,
dispatcher URL prefix preserved, wider connection pool.
* feat: background workers are listed in the sessions manager
A user paying for an app's events worker needs somewhere to see it and take it
down. The sessions manager gets a section listing the apps that run event
handlers in the background, with a Destroy action that removes their published
handlers.
- declare rate + concurrency limits on every route and driver that lacked one
- add acquireConcurrent for websocket connections and the DAV mount
- bucket AI models by identity key only; keep resold duplicates of any vendor
- skip recently-failed provider routes; cap the fallback chain at 3 attempts
- let full-access access tokens bind a worker to an app their own user owns
- cache resolved subscriptions so tiered limits don't add a round trip
* feat(perms): add cross-app app-data permission vocabulary
* feat(perms): sweep app grants by permission prefix
* feat(perms): resolve and withdraw cross-app data grants
* feat(kv): support an authorized namespace override and per-key privacy
* feat(kv): gate cross-app KV access behind app-data grants
* feat(fs): allow cross-app AppData access and require a scope to delete
* feat(auth): accept permission lists and gate app-data grants
* feat(perms): add requestAppData to the puter.js SDK
* feat(gui): carry permission lists through the IPC and popup transports
* feat(gui): describe cross-app data requests in the consent dialog
* docs: document requestAppData and per-entry KV privacy
* perf(perms): sweep cross-app grants only for origin-bootstrapped apps
* fix(gui): stop double-encoding cross-app consent text
* fix(perms): close three gaps in cross-app grant enforcement
* fix(kv): meter and batch the per-entry privacy probe
* fix(perms): resolve app identifiers and scopes more strictly in the SDK
* test(perms): cover the cross-app consent flow end to end
* fix: small missing token resolution for app
also adds the same exclusion for the batchPut api, small change
* fix: make resolved actor optional
---------
Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>
* typeify subdomains wip
* initial (untested) logic for LocalWorkerService
* Make it work, add lifecycle expiry since workers are process heavy in current implementation
* fix type errors
---------
Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>
* feat: worker sessions get their own kind + per-(user, app, worker_name) row
Schema
------
- mysql_mig_11.sql + sqlite 0054: add idx_sessions_user_worker_active,
a partial unique index over (user_id, app_uid, meta.worker_name) for
kind='worker' rows. Active worker sessions are deduped by that triple
so each named worker gets its own session row and they don't fight
the existing idx_sessions_user_app_active (which still constrains
kind='app' only). app_uid is allowed NULL for user-scoped workers
with no app binding.
SessionStore
------------
- getOrCreateWorker(userId, { appUid, workerName, ... }): mirrors the
getOrCreateApp pattern — cache lookup, partial-unique re-SELECT on
insert-ignore, all keyed on the worker triple. expires_at lands at
WORKER_WINDOW_SECONDS (~99y) so the worker doesn't have to re-mint
on any cadence.
- #cacheKeyWorker + #allCacheKeysForRow worker branch so revoke /
update invalidates the worker cache view alongside the by-uuid one.
AuthService
-----------
- createWorkerSessionToken(user, workerName, meta?) now takes the
workerName explicitly and routes through getOrCreateWorker. Emits
the same { session, token, gui_token } shape but both JWTs carry
{ worker: true, worker_name }.
- createWorkerAppToken(actor, appUid, workerName) likewise — JWT
carries the worker_name claim so a verifier can tell two workers
under the same app apart without a DB round-trip.
- Both methods 400 on empty workerName.
WorkerDriver
------------
- Five auth-mint call sites swapped over: app-bound deploy (3x:
appId branch, actor.app fallback, hot-reload redeploy), user-bound
fallback (2x: cold deploy, hot-reload). All pass `workerName` so
the worker's session row is naturally idempotent across redeploys.
GUI manage-sessions
-------------------
- sessionTitle adds a kind='worker' branch ("name (app)" for
app-scoped workers, just "name" for user-scoped), pulling worker_name
from the meta-spread that listSessions already surfaces.
- en.js adds ui_session_kind_worker.
* fix(workers): MySQL JSON_EXTRACT quoting + revoke cache invalidation +
SQLite NULL-distinct in worker index
Three real bugs in the worker session plumbing from the prior commit,
plus a misleading comment. Schema design kept (worker_name lives in
`meta` rather than a dedicated column) per offline review:
1. `#selectWorkerRow` compared `JSON_EXTRACT(meta, '$.worker_name')`
directly to a bind parameter. MySQL's `JSON_EXTRACT` returns a
JSON-typed value with embedded quotes (`"name"`, not `name`), so
the comparison never matched. After the first INSERT, every
follow-up getOrCreateWorker call missed the existing row in the
SELECT, hit INSERT-IGNORE, then missed again in the re-SELECT —
the caller would receive whatever the INSERT-IGNORE returned (a
no-op row in conflict cases). Wrap with `JSON_UNQUOTE` on MySQL
via `db.case`; SQLite's `json_extract` already returns the
unwrapped scalar so it keeps the literal form.
2. mig_11's generated `worker_unique_key` had the same JSON-quoting
bug. Mirror the fix: `IFNULL(JSON_UNQUOTE(JSON_EXTRACT(...)), '')`
so the concatenated unique key is a plain string that lines up
with what `#selectWorkerRow` now binds against.
3. SQLite UNIQUE indexes treat NULL columns as distinct (per the SQL
standard), so two user-scoped workers (app_uid NULL) with the same
worker_name would both insert. Wrap the index expression with
`IFNULL(app_uid, '')` so they correctly conflict — matches the
MySQL side's `IFNULL` in the generated column.
4. `removeByUuid` / `revokeCascade` SELECTed only the identity
columns (no `meta`), so `#allCacheKeysForRow`'s worker branch
couldn't read `meta.worker_name` and the composite
`sessions:v2:worker:<user>:<app>:<name>` cache key survived
revocation. Up to CACHE_TTL_SECONDS (15min) afterwards,
getOrCreateWorker would short-circuit to the cached (revoked) row.
Add `meta` to both SELECTs; the existing meta-parsing logic in
`#allCacheKeysForRow` handles the rest.
Juan Fernando Castro