Commit Graph
205 Commits
Author SHA1 Message Date
Daniel Salazar da65b7f569 feat: the invoking backend deploys an events worker the dispatcher cannot find (#3807)
The dispatcher's rehydrate callback reaches whichever backend answers the
API's public hostname. A backend with the runtime flag on that is not behind
that hostname, or the only one in the fleet with it on, could never get its
scripts deployed that way — the callback answered "disabled". The invoking
backend already knows the app and script, so on a dispatcher miss it deploys
the set itself and retries once, telling the dispatcher to skip its callback
and negative cache. The callback stays the path for evicted scripts.
2026-09-05 14:40:44 -07:00
Daniel Salazar bc9cb2d7e7 fix: withdrawing background consent revokes the app's events session (#3777)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
* fix: withdrawing background consent revokes the app's events session

A background handler runs as a worker session for the subscriber and app.
Revoking `events:background` or uninstalling the app suspended the
subscriptions but left that session valid, so a token a handler had copied
out kept working until the user found the row in the sessions list. The
revocation settle now revokes the session too; the next consented delivery
mints a fresh one.

* fix: cleanup docs
2026-09-04 21:10:27 -07:00
Daniel Salazar b784b51cf3 fix: harden the events stack for flag-on (#3752)
* fix: harden events dispatch, single delivery and KV share handles

Dispatch: a filtered subscription used the anchor path stored at subscribe
time, so renaming or moving the anchor folder silently ended its deliveries;
dispatch now resolves the anchor's live path from the event's own ancestor
chain. A move out of a watched folder now reaches that folder's subscribers,
with `from` only for rows that watched the source side. Gap markers are
authorized like deliveries and coalesced per subscription and subject instead
of fanning per lost event. Session subscriptions: the per-socket cap decides
on the write, not before it; an orphaned watched-set token heals on refresh;
durable rows keep their watch window when a session subscribe touches the
same keys. `self` is false when the acting user is unknown.

Single delivery: a subscription in backoff or suspended with a backlog pinned
the sweeper's head and starved everyone behind it — the sweep now defers it.
Only a settled handler run bills a delivery. A socket-only account row no
longer wedges after two attempts nobody received. The lease is twice the
handler timeout; remote candidates have their own attempt counter; the region
depth reconcile runs once a minute region-wide with a bounded scan.

KV share handles: a grantee no longer sees the owner's namespace and absolute
prefix on the subscribe answer or listing, nor in the delivery token; revoking
a wider handle retires the handles it covers; minting the same handle twice
returns the existing one, after the delegation check; a row whose event
cannot be re-based onto its handle is dropped rather than delivered raw.

* fix: presence survives replication, long sessions and region churn

One presence item per (user, app) with per-region map fields lost a region
whenever two regions joined inside the replication window, and nothing ever
put it back. Presence is now one item per (user, app, region): each region
writes only its own, a leave or repair retires it conditionally on its own
write stamp, and a read is a prefix query. Items carry a 48 h ttl refreshed by
a claim-gated write off the existing socket renew path, at most once per
12 h, so a tab that stays connected keeps its region in the row. A region
that answered "no socket" or completed a leave releases a shared pin, so a
reconnect on another node rejoins and a flapping client cannot force a
replicated write per cycle. Cached rows expire after a minute; unaddressable
region names are filtered and pruned; relayed acks settle under a bounded
concurrency; the forward queue is bounded in bytes as well as items.

* feat: indexes for the event_subscriptions hot queries

Handler publish, remove and listing, and the hourly expiry and suspension
sweeps, all scanned `event_subscriptions`. Adds (app_uid, handler_name),
(expires_at) and (suspended_at, id), guarded on every engine. Existing
migrations: the postgres widens are now guarded so a boot does not take an
exclusive lock for a no-op, the kv_share_handles grantee FK gets an index,
the sqlite notification rebuild is transactional and idempotent.

* fix: notification writes go through the registry

The driver's `create` bypassed the type registry, producing uncatalogued
rows with no size bound; it now requires a registered type, caps the payload,
and answers 400 rather than 500 for a bad one. `mark_acknowledged` emits the
ack other tabs listen for, and only when a row was actually changed.

* fix: the handler scanner, unsubscribe, and the in-tab handler environment

The free-variable scanner skipped arrows inside a declaration's initializer,
so `const ids = event.items.map(x => x.id)` was refused, and treated a name
after a comma in a nested initializer as bound, so a real free variable slipped
through to fail on first delivery. `unsubscribe()` now drops the durable
routing entry so the events socket can close. A broadcast handler running in
the tab gets `user` and `fetch` like the worker gives it. `single` without
a handler name is refused before the round trip.

* docs: events limits, error codes and the background-workers section

Retention is deployment-configured rather than a fixed 14 days, and the
template no longer ships it armed. Documents `events_terminal`, the two
per-event gap reasons, the subject length and listing caps, the `from` field
on moves, and the handle-relative anchor. The sessions manager hides the
background-workers section when the server has none to show.

* feat: a background handler acts as the app does for its user

A handler's `user` was a five-minute access token scoped to the subscription's
`list` grant, which could stat the changed file but not read it, and could
not reach the app's KV or AppData — so an app told that a file was written
could do nothing with it. It now runs with the same authority the app has for
that user in a tab: an app-under-user worker session, one row per (user, app)
named `events:handlers`, visible and revocable in the sessions list. The
`events:background` consent is what authorizes running it unattended, and is
re-checked before every mint.

The wider token exposed two things: puter.js opens a filesystem socket the
moment it has a token, which would have parked the isolate in the app's own
delivery room and steered deliveries at it; the events client now opts out of
sockets (and the per-open bookkeeping) before construction, and is memoized
per token in the isolate. And four filesystem operations assumed a socket
exists; they no longer do.
2026-09-04 17:32:57 -07:00
Daniel Salazar f30baa2a1c feat: the per-app events worker runtime (#3697)
* feat: bake published handlers into a generated events worker

* feat: deploy and address the per-app events worker behind a flag

* test: single delivery end to end through a real local worker

* feat: events workers run their own runtime, in their own namespace

An events worker was being deployed as an ordinary worker: default dispatch
namespace, a `subdomains` row, the router preamble, and an app-scoped worker
token baked in. The public dispatcher resolves any script in that namespace
straight off the hostname, so the worker answered at `<name>.puter.work`, and
the only thing in front of it was an unguessable name plus a check that a
`puter-auth` header was present — which the router never validates. Anyone who
learned the hostname could run an app's handlers with a body of their choosing,
in an isolate holding the owner's token as `me`.

Instead:

- Handlers run on their own runtime (`src/worker/src/events-runtime.js`), which
  provides no `router` and no `me`, owns the single invoke route, and hands a
  handler only `{ event, ctx, user, fetch, ack }`. `user` is built from the
  invocation's delivery token, so a handler acts as the subscriber whose
  delivery it is and nothing wider. The preamble build emits one bundle per
  runtime; the shared half of the template is now included by both.
- The deploy target carries the runtime to prepend, the source to deploy, and
  whether to mint a worker token at all, so an events worker deploys into the
  `events` dispatch namespace from generated source with no token binding, no
  `subdomains` row, and no claim on the owner's worker quota or worker list.
- An invocation carries a key derived from the deployment secret and the script
  name, bound as a secret and checked in constant time inside the isolate,
  which reads it once and drops it before handler code runs.
- Scripts are named after the handler set they contain, so publishing writes
  rows and deploys nothing: a set is deployed the first time a delivery needs
  it, and a changed set is a new script rather than an overwrite of a running
  one. Publish responses keep the shape they had before the runtime existed.
- Invocations reach a worker only through the events dispatcher, which has no
  zone route and requires the internal secret; the backend's own deploy path is
  the rehydrate route the dispatcher calls on a namespace miss. Locally there is
  no dispatcher, so the controller hands the service an in-process transport
  that deploys on miss itself.

The SDK stops allowlisting `puter` as a handler global — a handler that reaches
for an ambient SDK is now refused at publish time, naming `user` instead, rather
than passing the scan and failing on its first delivery.

Requires `events.workerNamespace`, `events.dispatcherUrl` and
`events.internalSecret`; without them nothing is addressable and background
deliveries stay retriable, as they did with the runtime off.

* fix: a handler's delivery token gets through the read routes

An events handler acts as the subscriber through the access token its
invocation carried, but every FS read route refused scoped access tokens
outright, so `user.fs.stat(event.path)` — the design's own example — answered
403 inside the worker. The read-side routes now admit them; the ACL each
handler already runs intersects the token's grant with its issuer's, which is
the check that keeps a token to what it was minted for. The end-to-end suite
asserts the stat from inside the isolate.

* fix: shorthand-method handlers publish as functions

`{ ingest({ event }) { … } }` stringifies without the `function` keyword, so
its source is not an expression and the events worker baked it as a broken
stub — every delivery a retriable 500 until the subscription suspended, with
nothing at publish time to say why. The SDK now gives a shorthand method the
keyword before hashing and sending; getters, setters and computed names are
left for the server-side check to refuse.

* feat: an app's events worker is listable and destroyable

An app with published handlers has an events worker, and hosted deployments
bill it monthly per app, so its owner needs to see it and be able to take it
down. The core announces the lifecycle on the bus — `events.worker.create`
when an app's first handler is published, `events.worker.destroy` when its last
one goes — with the owner as the actor, so pricing can plug in from outside.
`GET /events/workers` lists the caller's workers (paginated, with the script
each set deploys as) and `POST /events/workers/destroy` removes every handler
of an app under the same owner scoping as the handler routes, suspending the
subscriptions bound to them. `puter.events.workers.list/destroy` in the SDK,
a docs page, and a 5 MB cap on an app's combined handler source
(`events_worker_too_large`) so a set that publishes can always deploy.

* fix: harden the events worker runtime for production

- A 4xx is terminal only when it carries the handled marker the runtime (and
  the dispatcher) stamp on every answer that came from a script; an unmarked
  4xx — an edge 404 for a wrong dispatcher hostname, a WAF page — stays
  retriable and is logged, once per script per minute, with the runtime's
  reason header.
- Script names are scoped to this backend's exposed API origin, so two
  backends sharing a namespace never resolve one script with the wrong
  endpoint binding or key. Shape unchanged.
- Each handler is validated in the exact context it is emitted into and the
  whole generated file is compiled once; a source that would break the script
  marks every handler broken instead of deploying a SyntaxError.
- Locally, events scripts live under their own registry key: the public local
  worker host cannot reach them and an ordinary worker cannot take their name.
- A suspended or deleted app owner stops invocations; deploys are throttled
  per app per hour; in-flight deploys are keyed by app and script; the
  upstream deploy call times out; the generated source is size-capped with a
  margin over the publish cap; boot fails when the runtime is on but its
  preamble is not built. Byte-length secret compare, appUid shape check,
  dispatcher URL prefix preserved, wider connection pool.

* feat: background workers are listed in the sessions manager

A user paying for an app's events worker needs somewhere to see it and take it
down. The sessions manager gets a section listing the apps that run event
handlers in the background, with a Destroy action that removes their published
handlers.
2026-09-04 17:32:57 -07:00
Daniel Salazar b451d05d10 feat: app-minted kv share handles (PUT-1688) (#3692) 2026-09-03 19:43:27 -07:00
Daniel Salazar f139588f89 feat: revocation settles kv handles (PUT-1687) (#3691) 2026-09-03 19:43:27 -07:00
Daniel Salazar ac5446f877 feat: cross-user KV share grants and handles (PUT-1686) (#3690) 2026-09-03 19:43:27 -07:00
Daniel Salazar adde6c3a29 feat: notification GUI rides the events surface (PUT-1685) (#3689)
Widen the notif: match filter and fetch scope for a session's own
generic developer/app-user subscribe: today it pins ref to the
session's own uuid, so a row naming an app (handler-suspension
notices, app-bound worker deploys) never matches live and never
replays on reconnect, even though the audience predicate already
grants the holder every such row it owns. The predicate is the
authority and already reruns per row/page after the match, so
widening the filter to it (account is unaffected — it never names an
app) adds no exposure.
2026-09-03 15:39:22 -07:00
Daniel Salazar fe5f81e52e feat: scoped app access to notifications (PUT-1684) (#3688)
An actor holding an app reads the `app-user` rows naming that app, plus its
`developer` rows when the holder owns it. `account` rows reach no app, and a
slice an actor may not see comes back empty rather than refused. The audience
predicate becomes the enforced read path in the same change that lifts the
blanket app-actor 403, layered behind an audience/app_uid SQL scope; two-segment
`notif:` subjects expand server-side from the actor's own app, so an app can
never name another app's uid.

No feature flag: `audience` defaults to 'account', so every pre-registry row is
default-denied to app actors and the backfill can only narrow.
2026-09-03 15:39:22 -07:00
Daniel Salazar 796683a133 feat: missed-event fetch and notification fold-in (PUT-1681) (#3687) 2026-09-03 15:39:22 -07:00
Daniel Salazar eb8f497e9f feat: presence and cross-region event forwarding (PUT-1679) (#3686)
* feat: presence and cross-region event forwarding (PUT-1679)

* fix: fan cache bumps to sibling nodes and stop the forward shed cascading (PUT-1679)

`outer.events.generationBumped` and `outer.events.presenceBumped` rode
`outer.*`, which the broadcast service only webhooks to peer regions;
only `outer.pubsub.*` also fans over Redis to a region's other nodes.
Both caches are per-process maps with no expiry, so a bump landing on
one node left its siblings stale until that user's next transition.
Renamed onto `outer.pubsub.events.*`; the listeners already accept the
`from_outside` copy the Redis re-emit carries.

`PeerForwardQueue.push` called `onOverflow` synchronously and the
handler pushed markers straight back, each of which re-tripped the
bound and shed the next item: one item over a 5000 bound recursed ~2200
deep, threw a RangeError, and turned ~2200 queued deliveries into gap
markers. It also re-summed `bytes` over the whole queue per drop. The
handler now returns its markers and the queue appends them past the
bound check, sheds deliveries before markers, keeps one pending marker
per (peer, subscription), and subtracts bytes per dropped item.
2026-09-03 15:39:21 -07:00
Daniel Salazar 32a838d2e2 fix: one ** per match pattern and one * per segment (#3736)
`compileMatch` turned every `*` into an unbounded `[^/]*`, so a subject
like `fs:~/x/*a*a*a*a*a*a*a*a*a*a*z` tested against a 24-char filename
with no `z` — both named by the same subscriber — cost the regex engine
C(34,10) splits before failing: 1.0 s per event at ten stars, 3.7 s at
eleven, on the fs-write dispatch path, inside the 256-character cap.

With one `*` per segment the delimiters pin each star and a wrong split
dies in one step; the single `**` is the only choice point left, so the
worst allowed shape is O(depth × length): 0.4 ms at depth 60 of 200-char
segments. Every documented pattern (`*.png`, `**/build.log`, `**/*.png`,
`dir/**`, `report-?.csv`) stays valid; `*a*`, `a**b` and `**/x/**` are
refused with `invalid_subject_pattern`. The rule is stated where the
syntax is introduced, on the limits page, and in onLocal's error table.
2026-09-03 15:08:49 -07:00
Daniel Salazar 799fc4ac3c feat: send app icons as a subdomain URL plus an API fallback (#3734)
App payloads carried only the /app-icon endpoint URL, which 302s to the
icons hosting subdomain. Networks that mangle that redirect render no icon
at all, and every icon load pays a round trip for the hop.

Ship the direct subdomain URL as `iconCdnUrl` alongside it (taskbar items,
installedApps, recent/recommended launch apps, suggested apps), and have the
GUI load that first with the endpoint URL as a one-shot retry - desktop
taskbar, start menu, dashboard app grid and recents. Only rows whose `icon`
column is already an http(s) URL get one: a data: column means the resize
pipeline has not written anything to the subdomain yet.

Also folds the four copies of the generated-size list into one exported
APP_ICON_SIZES.
2026-09-03 13:32:55 -07:00
Juan Castro 9bc2cd741c feat: add TeamController, the teams_enabled flag, and the isolation suite
Covers PUT-1708, PUT-1709 and PUT-1743.

Twelve routes, every one setting requireUserActor -- that option is what
installs requireAuthGate, requireVerifiedAccount and requireNonAccessTokenGate,
because server.ts derives `needsAuth` from the route options. Reads need it as
much as writes: without an auth option a route gets no suspension check and
admits access tokens, so a just-disabled member could still read the roster and
a scoped third-party token could read the audit log.

Authority is checked before anything observable. Validating the body first made
POST /members answer 400 before 403, and resolving :username first turned the
member routes into a global username-existence oracle.

Provisioning applies the same username and email rules as signup rather than
its own -- USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and
validator.isEmail, now exported from AuthController. Without them a workspace
could mint accounts signup would refuse, claim unregistered reserved names, and
mail arbitrary unvalidated addresses.

Handle problems are 400 or 409 rather than a bare Error, which the server turns
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable drops sessions through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key; without that a
disabled member kept authenticating from cache for the session TTL, which is
exactly the "takes effect on the next request, not after a cache TTL" property
disable is supposed to have. Revoking also preserves last_ip/last_user_agent,
which the member-facing audit view reads.

Audit writes live in TeamService at the point of each action rather than in the
route, so a caller reaching the service directly cannot skip them, and the SQL
lives in TeamStore. Audit reads map internal user ids to usernames, and remain
readable by the owner after the workspace is soft-deleted -- otherwise the
delete_team entry was written and immediately unreachable.

teams_enabled gates route registration through an optional isEnabled() the
server honours, so with it off the paths do not exist rather than existing and
refusing. It does not gate DDL.

TeamIsolation.http.test.ts asserts the negative the feature rests on: the
workspace manages accounts and cannot read them, including through a
full-access token and after the member is disabled. It asserts outcomes rather
than the absence of an implicator.
2026-09-03 15:57:23 -04:00
Juan Castro 673f7fe75f feat: provision org accounts from a workspace
Covers PUT-1705. The master account supplies { username, email }; the account
is created with no password, gets the default filesystem tree, joins with
org_owned = 1, and receives a one-shot activation link.

Activation reuses password recovery rather than new token machinery: the same
pass_recovery_token, the same one-hour purpose-scoped JWT, the same
/action/set-new-password link. No team_activation table, no new token type,
and no unauthenticated endpoint on the team surface. Activation state needs no
column either -- an unactivated account is one with no password.

Applies the same username and email rules as signup rather than its own:
USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and validator.isEmail,
now exported from AuthController. Without them a workspace could mint accounts
signup would refuse -- the username becomes the /username home-directory
segment -- claim unregistered reserved names, and send activation mail to
arbitrary unvalidated addresses at the route's daily limit.

Usernames come from Puter's global pool, so a taken one is refused with free
alternatives rather than silently modified: a suffixed name would appear in
every share dialog that person ever sees, and they never agreed to it. The
check runs before any write, so a rejected provision leaves no orphaned user
row -- asserted by a test on the workspace's member count.

The new account carries requires_email_confirmation, since the address came
from the administrator rather than its holder.

Adds a team_account_activation email template stating what the workspace can
and cannot do -- including that it can reset the password, which the design
requires be said rather than only claiming files are private.

free_storage stamping and the billing event are phase 3.
2026-09-03 15:57:23 -04:00
Juan Castro da2b42d7fe feat: add TeamService workspace lifecycle and disable/re-enable
Covers PUT-1704 and PUT-1707: creating a workspace, admitting the master
account, and the whole of offboarding.

`createWorkspace` admits the creator with org_owned = 0, which is what makes
the master pay for itself and stay an invalid target of every member route.
`checkOwnerInvariant` asserts the rule no dialect can express -- the owner is
a member with org_owned = 0 and the only such member -- and a test breaks it
deliberately, since the schema cannot refuse a second one.

Three authority checks: 404 to a stranger so the endpoint is not an existence
oracle, 403 to a member who is not the master, and the master refused as a
target of member routes.

Handle problems surface as 400 (unusable) or 409 (taken), including the
unique-index race. `TeamStore` throws a bare Error, which the server would turn
into a 500 and a deduped critical alarm -- an uppercase handle should not page
on-call.

Disable writes `user.suspended` as well as suspended_at and suspended_reason.
PUT-1707 named only the latter two, but those are siblings added by 0061 and
0063 -- `userProtected` rejects on `if (user.suspended)` and reads neither.
Setting only the timestamp and reason would have recorded a disable that never
took effect, and disable is the whole of offboarding here.

Sessions are dropped through SessionStore.removeByUuid rather than a raw
DELETE. The store invalidates every composite cache key with its double-delete
pattern; without that a disabled member keeps authenticating from cache for the
session TTL, which is exactly the "next request, not after a cache TTL"
property disable is supposed to have. Revoking also preserves last_ip and
last_user_agent, which the member-facing audit view reads.

Files are untouched and re-enable restores the account.

Adds team_not_found, not_the_master_account and not_an_org_account to the
HttpError legacy codes, which the controller also needs.

Billing events, invalidateActorSubscription, audit rows and the GUI push are
deliberately not here -- they belong to phase 3 and PUT-1708.
2026-09-03 15:57:23 -04:00
Daniel Salazar 72011fa6ed feat: events metering and quotas (PUT-1683) (#3685)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
* feat: events metering and quotas (PUT-1683)

* fix: drop the standing subscription charge and price single deliveries at 100 µ¢

An idle durable row costs nothing worth billing; the plan quotas bound how
many an account holds. Removing the daily line also removes the global
day-claim, the whole-table scan and the timer it rode on.

* test: stop asserting on documentation pages

The limits test read rate-limits-and-quotas.md and grepped it for numbers,
so every rewording of the page failed the backend suite. Docs are kept in
step by the PR and checked in review; AGENTS.md now says so.
2026-09-03 01:39:07 -07:00
Daniel Salazar 41b837b398 feat: background delivery seam, retries, and consent (PUT-1682) (#3684) 2026-09-03 01:39:06 -07:00
Daniel Salazar 04c00385b9 feat: event handlers, context, and suspension machine (PUT-1680) (#3683) 2026-09-03 01:39:06 -07:00
Daniel Salazar 9626ab9c71 feat: KV change events and KV subjects (PUT-1678) (#3682) 2026-09-03 01:39:06 -07:00
Daniel Salazar d8ebfc8023 feat: delivery re-check cache, revocation and anchor settle (PUT-1677) (#3681)
* feat: delivery re-check cache, revocation and anchor settle (PUT-1677)

* fix: authorize re-anchors, settle each row once, purge revoked backlog (PUT-1677)

- A path-form row whose anchor is deleted only climbs to an ancestor its
  holder may still watch under the mode it subscribed with; otherwise it ends
  with `anchor_deleted`. It used to land on any surviving ancestor (a guest's
  row on the owner's home), where the re-check denied every delivery but the
  row still held an anchor slot and a filter evaluation there.
- After a climb the new anchor is re-verified and the climb repeated if a
  recursive delete took that level too, instead of leaving the row on a dead uid.
- suspend() is one conditional write per row and reports which rows it was the
  one to suspend; concurrent settles of the same grant (an unshare revokes
  several strings) no longer each purge, forget and notify the same rows.
- One "subscriptions ended" notification per holder and app, carrying the count
  and subjects, instead of one per row.
- A revoke that removed nothing no longer announces; the sweeper purges (not
  defers) the backlog of a permission_revoked row; the reap purges pending
  entries with the row.
- The delivery auth cache indexes entries by subscription so forget() is not a
  scan of the whole cache.
2026-09-02 16:00:14 -07:00
Daniel Salazar 41d4850ab4 feat: pending event deliveries and delivery-class invariants (PUT-1676) (#3680)
* feat: pending event deliveries and delivery-class invariants (PUT-1676)

* fix: make pending delivery claims and drains atomic, keep the region under its ceiling (PUT-1676)

- claim() and the drain-time reindex run as Lua over the subscription's own
  {subId}-tagged keys. Two claimers can no longer both lease the head, and a
  drain that finds the queue empty deletes it in the same step it checks, so a
  concurrent enqueue is never wiped between the two.
- An append writes the entry and its queue position in one MULTI (same slot),
  with the index seeded before it and corrected after, so an entry is never
  visible without its position and never left out of the sweeper's index.
- Pipelines no longer mix slots (index/counter vs. per-subscription keys), so
  the store works on a multi-shard cluster, not only a single-shard one.
- Region shedding counts the marker it leaves behind; it used to stop one over
  the ceiling and convert a real event into a marker on every enqueue after.
- A claimed or suspended subscription moves to the back of the sweeper's index,
  so a delivery nobody settles cannot hold the head against every other backlog.
- `single` rows must carry a `worker` target: with sockets exhausted and no
  handler, an unacknowledged delivery would sit at the head forever.
- A gap marker for a row with no socket target is dropped rather than counted
  as a delivery of nothing.
- Backlog keys carry a 7-day TTL, refreshed by every claim, as a backstop for
  keys a purge/enqueue race left unindexed.
2026-09-02 16:00:14 -07:00
Daniel Salazar 0dfbceb047 feat: durable event subscriptions store, cache, and routes (PUT-1673) (#3679)
* feat: durable event subscriptions store, cache, and routes (PUT-1673)

* fix: durable subscription hardening (PUT-1673)

- Expired rows stop delivering at dispatch time and no longer count toward
  the per-account cap, instead of waiting for the sweep.
- The expiry sweep runs hourly with a jittered first pass shortly after boot;
  a 24 h interval never fired on a fleet that redeploys more often than that.
- Only a durable generation bump marks peer regions cold. A session
  subscribe/unsubscribe in one region used to force a primary read in every
  other region on its next dispatch.
- `subject`/`anchor_path` widen to varchar(4096) to match `fsentries.path`,
  and subjects longer than that are refused with `invalid_subject` rather than
  failing the insert on MySQL/Postgres.
- The dispatch and durable integration suites wait for the specific delivery
  they expect and assert only within their own folder; the old any-delivery
  `settle()` let a late event from a previous test satisfy or pollute the
  next one under CI load.
2026-09-02 16:00:13 -07:00
Daniel Salazar 66dc0a6396 feat: subscribe ACL and app scoping for event subscriptions (PUT-1672) (#3677) 2026-09-02 13:08:23 -07:00
Neal Shah 262f1dc5c5 allow concept of "home regions" (#3699)
* allow concept of "home regions"

* remove extraneous config value not applicable to repo
2026-09-02 14:11:55 -04:00
Daniel Salazar 7ec674d33b feat: app sockets join their own events room (PUT-1671) (#3676) 2026-09-02 09:01:22 -07:00
Daniel Salazar b07d2e109f feat: session event subscriptions and dispatch hot path (PUT-1666) (#3675) 2026-09-02 09:01:22 -07:00
Daniel Salazar 92e1b40361 feat: events subject registry, parser, anchors, filters (PUT-1660) (#3674) 2026-09-02 09:01:21 -07:00
Daniel Salazar 82076ea767 feat: notification retention sweep (PUT-1668) (#3672) 2026-09-01 22:04:32 -07:00
Daniel Salazar 5f37817a5c feat: notification type registry and audience rules (PUT-1665) (#3671) 2026-09-01 22:04:32 -07:00
Daniel Salazar 1736d23ee7 Ds/put 1674 (#3715)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
* feat: app-scoped share listing and revoke surface (PUT-1670)

* fix: address review on app-scoped share surface

- Scope the uid-addressed revoke to the named row: only that row's
  issuer's grant is withdrawn, and only that one invite cancelled —
  an app or owner addressing one row no longer takes another issuer's
  grant on the same (item, recipient) pair with it.
- Delete a pending row directly on uid-addressed revoke, so an invite
  whose address registered but never claimed can still be withdrawn.
- Read the legacy `issuerAppUid` data key in the SQL app filter and
  grouping, alongside the unified `issuedByApp`.
- Refuse malformed `appUid` input (duplicated param, empty string)
  instead of silently listing everything, and refuse app-listing
  cursors that decode but name no appUid.
- Derive the acting app from `effectiveApp` alone, per the actor
  contract; drop the second derivation site.
- Pin the attribution semantics with tests: one row records one
  issuance, so re-sharing the same pair re-attributes it to whoever
  issued last, in both directions.
- Soften the uniform-404 docblocks to what the gates actually answer.

* feat: readable grant audit trail (PUT-1674)

* fix: cover the apps summary's no-app-group first page (PUT-1670)

listOutboundApps sorts the no-app group first via an empty-string
sentinel. Add a regression test pinning that a first page (no cursor)
actually returns it, and that the cursor it hands back resumes past it
into the app-keyed groups rather than skipping or repeating.
2026-09-01 14:07:15 -07:00
Daniel Salazar 66a975f659 feat: global outbound share listing (PUT-1664) (#3694)
* feat: global outbound share listing (PUT-1664)

* fix: address review on outbound share listing

- Check share-row liveness per (holder, entry, issuer) so a grant
  withdrawn outside unshare doesn't stay listed while another issuer
  still reaches the same holder; batch the permission reads across the
  whole page instead of per holder.
- Retire a revoked issuer's unclaimed invites in the revoke cascade,
  and hide invites whose issuer lost their authority at read time.
- Unify the pending/active app-attribution key on `issuedByApp` and
  dual-read the legacy `issuerAppUid` spelling.
- Add the missing share issuer index (sqlite, postgres) and correct
  the listOutbound plan comment.
- Refuse cursors that decode but name no id instead of silently
  restarting from page one.
- Consolidate the five hand-built ResolvedShare literals and the two
  listing endpoints' parse/shape code.
- Ship the SDK surface: puter.fs.listSharedByMe() with docs, types,
  suite coverage, and the rate-limit page entry.
2026-09-01 13:52:37 -07:00
Nariman Jelveh fdd86ceb03 Stop server-rendered landings flashing for signed-in users (#3707)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
The shell renders its anonymous markup — the marketing homepage, an
`/app/<name>` landing — off the session cookie alone, and that cookie is
set with no maxAge, so a browser drops it on quit while the GUI's
localStorage token lives on. A returning user is served the anonymous
page and the GUI only tears it down once `whoami` answers, a network
round-trip after first paint. That teardown is the flash.

Gate it before the paint instead. The shell now emits, as the first thing
in <head>, a rule hiding `.hide-if-logged-in` under an <html> class that
an inline script adds iff `auth_token_v2` is in localStorage. The rule is
already in the cascade when the markup is parsed, so a browser holding a
token never paints it at all.

`initgui` settles the guess the token represents: `whoami` confirming the
session removes the nodes outright (replacing the old `#appLanding`
removal), and no session — none stored, or one `whoami` rejected — drops
the class so the markup comes back. The gate carries its own 12s failsafe
so a bundle that never boots can't strand a blank page.

SEO is unaffected: the HTML is byte-identical for every client, nothing
branches on user-agent, and a crawler has no stored token so it never
adds the class. Unreadable storage fails open the same way.

Anonymous markup opts in with `class="hide-if-logged-in"`, which
`home.html` already carried.
2026-09-01 09:19:00 -07:00
Juan Fernando Castro cdfd38bd4b fix: stop an embedded fs: in a path from escalating to full FS access (#3666)
The fs-path-to-uid permission rewriter split permission strings on the raw
\`fs:\` substring instead of parsing on component boundaries. A path can
itself contain \`fs:\` — a home dir named \`…fs\`, or \`fs\` in the mode
position — and the raw split mistook that for the mode delimiter.

Crafting \`fs:/<victim>fs:junk:read\` made the split do two things at once:
it dropped the \`junk:read\` mode, and it consumed the trailing \`fs\`, turning
the harmless-looking nonexistent path shown in the consent dialog into the
victim's real home path. The rewriter then stored a bare \`fs:<home_uuid>\`,
which subsumes every mode via parent-permission matching — full
read+write+delete from a request the user saw as \`junk:read\`.

Parse with PermissionUtil.split() in both matches() and rewrite(), matching
every other permission parser, and locate the \`fs\` component by position
(after an optional \`manage\` prefix). The path component is now exactly what
sits between \`fs\` and the next unescaped colon, and all trailing components
are preserved as the mode — so an embedded \`fs:\` is data, never a delimiter.
The crafted input now addresses the nonexistent \`/<victim>fs\` and 404s.

Regression tests pin both halves: the exact PoC must 404, and an \`fs:\` in
the mode position must survive the rewrite instead of collapsing to a bare
permission. Both fail on the old rewriter and pass on the fix.
2026-08-29 01:41:40 -07:00
Daniel Salazar 9fab4742c9 fix: complete the malformed-input guards, including the two paths still live (#3663)
Review of the previous change found three of its claims unmet.

The image-generation crash it reported fixed is still reachable. The assert was
scattered across three helpers, and Gemini and OpenAI call `isHttpUrl` directly
on `input_images` without going through any of them — so two of seven providers
still 500 on a non-string. `isHttpUrl` now refuses a non-string itself, and the
shape is settled once in `ImageGenerationDriver.generate`, where the driver call
arrives, rather than per helper. That also covers `input_images` that isn't an
array, which produced a different crash per provider.

The sixth case in the ticket, previously unlocated, is
`Messages.js` reading `tool_call.function.name` with no guard — reachable with
`{"messages":[{"role":"assistant","tool_calls":[{"id":"x"}]}]}`. Guarded, along
with the same shape in `make_claude_tools`: a TypeError there carries no status,
so the retry loop reads it as a provider failure and marks the route unhealthy
for every caller.

`#hardExpiryFromExpiresIn` returning null for a bad type moved the failure past
the session INSERT, leaving an orphaned non-expiring row and still answering
500. Reverted; the controller guard is the fix, now covering fractions,
negatives and unparseable durations rather than only wrong types.

Also: the batch write handlers check that the body is an array but not what is
in it, so a null element 500s the same way; `#requireObjectBody` accepted an
array despite its name; `handleCreateAccessToken` destructured a body that may
be absent; and two AGPL notices had been rewrapped with a Markdown link.
2026-08-28 11:14:30 -07:00
Daniel Salazar f3a46a9be4 fix: stop sockets outliving the session that authenticated them (#3658)
A socket was checked once at handshake and never again. Nothing in the backend
disconnected one, so logout-everywhere, password reset, session revoke and
suspension all left every connection streaming legacy FS entries, upload paths
and notification bodies — up to 400 per account — on a credential that had
already been revoked.

Three gaps, three fixes:

- The handshake skipped the suspension and pending-verification checks every
  authenticated HTTP route gets. `decideSocketAuth` now applies both.
- `revokeCascade` reports which rows it revoked, AuthService announces that as
  `auth.sessions.revoked`, and SocketService drops the account's room. The
  whole room goes, not just the revoked session: narrowing it would need
  `fetchSockets`, which the adapter builds on `serverCount()` — and that calls
  node-redis's `send_command`, which ioredis does not implement. A connection
  whose session survived reconnects on its own and re-authenticates.
- A bulk suspension writes `user.suspended` without touching `sessions`, so no
  revoke fires. A five-minute sweep re-verifies each live socket's token and
  drops the ones that no longer authenticate. De-duplicated by token, since a
  browser's tabs share one.
2026-08-28 11:12:27 -07:00
Daniel Salazar e460e9034c fix: answer 400 instead of 500 on four malformed-input paths (#3662)
Each of these read a field off caller input that wasn't the shape the code
assumed, threw a TypeError, and was served as a 500 with a critical page.

- POST /fs/write and /startWrite: a request whose body never parsed left
  `req.body` undefined, and the first read of `fileMetadata` threw.
- POST /drivers/call, image generation: `input_image` / `input_images` entries
  are documented as strings but nothing checked, so a number or an object
  reached `.startsWith`. Type confusion on caller input, so reachable on
  demand rather than by accident.
- POST /auth/create-access-token: `expiresIn` went to the expiry parser
  unvalidated, where anything but a string or a number has no `.trim`.
- POST /login/wait: destructuring `session` out of an absent body threw.
  Same class as the POST /login body ticket; that one covers /login itself.
2026-08-28 10:22:51 -07:00
Daniel Salazar 88b851a423 fix: derive the signup client IP from req.ip instead of x-forwarded-for (#3661)
Both signup paths read the x-forwarded-for header directly for the
puter.signup.validate event, the puter.signup.success event and the
signup_ip_forwarded column. Behind an appending proxy a client can prefix that
header with anything it likes, which gives every per-IP abuse signal a fresh
bucket per request.

req.ip is the value `trust proxy` resolves, so it is the one the server can
stand behind; server.ts already uses it for the ip.validate gate for exactly
this reason. All three now share one derivation, so a per-IP counter is written
and read under the same key. The raw chain is still recorded, but only as
audit_metadata.ip_fwd, where nothing keys on it.

Rows written before this hold whatever the proxy appended, so per-IP velocity
over the trailing window is undercounted until they age out.
2026-08-28 10:19:24 -07:00
Nariman Jelveh 7332993ff3 Show share notifications in the dashboard (#3653)
* Show share notifications in the dashboard

The dashboard opens its own socket but never listened for notif.* events,
so a file shared with a user on the dashboard was announced to nobody.

Add a notification center to the dashboard: a bell in the sidebar with the
unread count (a dot when collapsed, and on the mobile hamburger), a panel
anchored to it — a bottom sheet on phones — listing what the server still
holds as unacknowledged, and toasts for what arrives live. Clicking a share
lands on Files › Shared with the item selected; dismissing, and "Mark all as
read", acknowledge on the server so other tabs clear too. A toast timing out
is not a dismissal: the entry stays unread in the panel.

Arrivals fold in by uid (a regrouped share rewrites its row and toast in
place), the panel refreshes on open, reconnect and tab focus so shares the
backend folds in silently past the sender's budget still surface, and a
burst on connect is capped at three toasts plus a summary.

Backend: notif.unreads carries created_at so delivered-on-connect items can
be dated, and listings break same-second ties by id so order is stable.

The desktop's mark-ack calls move to a shared helper; UINotification gains an
optional auto-hide timeout that pauses on hover/focus.

* fix: look up notification glyphs and icons by own key only

A notification whose `source` or `icon` was a prototype key such as
`constructor` made the glyph lookup return a function, and the toast
icon builder then threw inside the socket handler — which dropped the
whole burst of toasts it was part of. The list rendered the same entry
as "[object Object]".

* fix: show dashboard toasts above open app windows

App windows opened from the dashboard are stay-on-top and stack in the
99999999+ band, so the toast container at 10000000 sat underneath them:
any notification arriving while an app was open — the usual state of a
dashboard session — was drawn behind the app and never seen.

* fix: reveal the dashboard when a toast is clicked over an open app

Clicking a share toast switches the dashboard to Shared, and the "N new
notifications" toast opens the panel — both inside the dashboard window.
With an app maximized over it, all of that happened out of sight and the
click appeared to do nothing. Minimize the covering app windows first,
the way the minimize controls do, landing the URL on the dashboard's own
route, and act once the history pop has settled so the tab switch is not
traversed over.

* fix: keep keyboard focus in the notifications panel across re-renders

Every change to the list rebuilds its rows, which dropped focus to the
document body: dismissing an entry with Enter threw the keyboard out of
the dialog (the row it had just been moved to was replaced), and so did
any arrival while an entry was focused. Put focus back on the same entry
after a render, or on the one that took a dismissed entry's place, and
let Shift+Tab from the panel itself wrap to the last control instead of
leaving the dialog.

* fix: keep the notifications panel inside short viewports

The anchored panel is bottom-aligned with the bell and grows upward, but
its height was capped only by the viewport (560px or 100vh - 24px), not
by the room above the bell — on a short window the header ran off the
top of the page. Cap it to that room, so the list scrolls instead.

* fix: make notifications with nowhere to go inert in the panel

An entry with no target (a worker deploy result) showed a default cursor
yet, when clicked, was dismissed and took the panel with it — nothing
opened, the entry was gone, and the list had closed. Render such entries
as text with the ✕ as their only action; only share entries remain
buttons. The ✕ now names its notification for screen readers, since on
an inert entry it is the only focusable control.

* fix: stop a listing in flight from resurrecting dismissed notifications

The list is refreshed on open, on reconnect, and when the tab comes back
into view; a dismissal landing while that listing is in flight was undone
when it resolved, since the server's snapshot predates the ack. When the
listing also resolved after the server's own ack event, nothing removed
the entry again and it stayed until the next refresh. Leave out of the
reconciliation whatever was acknowledged — here or in another tab —
since the listing was requested.
2026-08-27 12:49:20 -07:00
Juan Fernando Castro 5a6a1020a1 Merge pull request #3644 from HeyPuter/juancastro/put-1585-share-flag-in-readdir-shared-users-in-stat
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
🔧 PUT-1585: Share flag in readdir shared users in stat
2026-08-26 18:51:30 -04:00
Juan Castro 5739f4e59f fix: stop pushing file metadata to revoked recipients (PUT-1611)
The realtime fan-out resolved its audience straight from the `share`
index, which has no live-grant check. `/auth/revoke-user-user` deletes
the permission and leaves the index row, so a revoked recipient's socket
kept receiving name, size, masked path and mtime for every write and
move under the folder, with no expiry.

The service already solves this elsewhere — `#reachingHolders` returns
exactly the holder/entry pairs whose grant still stands, and
`listSharedWithMe` was moved onto it for the same reason. The realtime
path never got the same treatment; it does now.

Free on the unshared path: with no share rows reaching the entry there
are no holders to check, so the write path every user takes is unchanged.
Pinned by a test that counts permission reads.
2026-08-26 16:35:38 -04:00
Juan Castro cae51ce67a fix: trashing a top-level share is a removal, not a move
A grant on the entry itself is keyed on uuid, so it follows the entry
into the owner's Trash. Both ends of the move then resolved, and the
recipient was told the shared item had moved — to the GUID name Trash
gave it. Their own copy got renamed to a GUID and stayed on screen.

`shared-with-me` has always omitted trashed entries, so the listing and
the event disagreed; only the event was wrong. Trashing now reports
item.removed at the path the recipient knew, which is also what the
desktop's data-path selector needs to find the row. Restoring out of
Trash reports item.added.

A move that leaves the recipient's masked address unchanged now stays
quiet — a share masks its own root, so the owner shuffling it around
their tree is invisible to the recipient and the event carried nothing.
2026-08-26 16:25:02 -04:00
Juan Castro 8e622189ca fix: tell every recipient when a shared file disappears
Both ways a file can vanish were silent for anyone holding the folder
above it, so a third party's window kept showing a file that was gone
and 404'd on click.

A delete built its audience from the permission rows it removed, and a
file inside a shared folder has no grant of its own — only the folder
does — so the audience was empty. A move resolved its audience from the
entry's new path, and the GUI's Delete is a move to the owner's Trash,
where no recipient has a share.

Resolve the audience from where the entry was rather than from the
grants that went with it:

- Deletes also fan out to holders reaching the entry through an
  ancestor, coalesced by parent so a subtree stays a couple of queries.
  A holder covered by both passes is told once.
- Moves resolve both ends. Reaching both is item.moved, only the
  destination item.added, only the origin item.removed.

Recipients are named by the path they knew, masked through their own
share rather than the owner's tree.
2026-08-26 14:54:06 -04:00
Juan Castro 4b3d1112d1 Say when a share changed nothing
Sharing a file with someone who already had it answered "Shared with
X", the same as a first share, so the dialog claimed to have done
something it had not.

The service already knew — it computes isNew to decide whether to
notify the recipient — but the flag stopped at the controller. It now
travels on share results only; a listing describes standing access and
says nothing about it.

Which of the three things happened is settled client-side, since the
mode each recipient holds is already on screen: created access, raised
or lowered it, or changed nothing. That keeps the previous mode off the
wire, and an older backend that omits isNew still reads as a share,
which is what these dialogs said before.
2026-08-26 11:57:05 -04:00
Juan Castro 70edef561f Stop offering an access level a delegate cannot grant
A recipient given "can edit & share" could not pass that level on: the
dialog offered it, the server refused it, and the refusal was a bare
403 Forbidden that reads as a bug. Handing out manage needs authority
over manage, which only the owner has — the refusal is right, the dead
end and the silence were not.

The dropdown now withholds it from anyone who does not own the item; a
row already set to it keeps it, so opening the dialog cannot downgrade
the owner's own grant, and a mixed selection follows its strictest item.

The server says why, and only to someone who can already share the item
— a stranger still gets the ACL's own safe error, which does not admit
the node exists.

Verified against a running server: a delegate grants read and write as
before, and manage now answers cannot_delegate_manage with a sentence
naming the owner as the one who can.
2026-08-26 11:48:54 -04:00
Juan Castro 1514ab3cb5 Ignore fs events replayed by replication
Raised in review: could an event from another node re-trigger the
fan-out? Not today — broadcast carries outer.* and pubsub.* only, so
fs.* never crosses a node boundary, and the emitted outer.gui.* is
consumed on the peer by SocketService while ShareService listens to
fs.* alone, so nothing re-enters.

That safety is a property of what broadcast happens to replicate, which
is not this service's to rely on. The handlers now skip anything tagged
from_outside: the node that did the write has already told the audience,
and a second fan-out would only duplicate it.
2026-08-26 09:22:49 -04:00
Juan Castro ee2b5adcf9 Tell share recipients about creates and renames
A recipient's client keeps its cache fresh from fs events pushed over
their socket, and ShareService fans those out to holders — but only for
write, move and delete. A new entry emits fs.create.<flavor>, not
fs.write.file, and an in-place rename emits fs.rename; neither had a
listener, so a recipient watching a shared folder never learned that a
file appeared in it or was renamed. Part of why: those keys and
outer.gui.item.renamed were missing from the typed event map, so a
listener for them did not compile.

Delivering the event is only half of it. Paths were masked against the
entry itself, so item.added named a parent no cached listing was keyed
on, and the payload carried no dirpath, which is how the desktop finds
the container to render into — the event would have arrived and changed
nothing. Paths are now masked at the share the holder reached the entry
through, which is the address their own reads returned, and from_path on
a move and old_path on a rename travel the same way (dropped when the
move started outside the share, self-masked when the share is on the
entry itself, where the root already carries the new path).

Creates fire per entry, so an upload would have cost one share lookup
per file; they are coalesced by parent folder the way subtree deletes
already are. Measured on a 25-file burst into one folder: 25 lookups
before, 1 after. A holder with a share on both a folder and something
inside it is told once, by the nearer of the two.
2026-08-25 19:38:38 -04:00
Juan Castro 1600dc38a5 Retire the share index row with the grant it records
revoke-user-user withdraws a grant without touching the share index, so
the row outlived the access — invisible until now, because listSharesOf
filters against live grants, but the new flag reads the index and would
report a file as shared to nobody, permanently.

Drop the row where the grant goes. The alternative, filtering liveness on
the read side, is the per-entry work the flag exists to avoid.
2026-08-25 18:14:52 -04:00
Juan Castro 5115fdd48c Answer which entries are shared in one indexed query
Directory listings need a shared/not-shared flag per entry. Ask the share
index once per batch, chunked, rather than a query per entry, and answer
only for entries the caller owns so listing a foreign directory never
queries at all. Direct shares only — inherited access stays with
listSharesOf, which already reports it.
2026-08-25 17:44:25 -04:00
Daniel Salazar 717c098503 fix: metering buffer improvements for manual editing (#3633)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
2026-08-24 18:36:43 -07:00