mirror of
https://github.com/HeyPuter/puter.git
synced 2026-09-25 06:36:00 +00:00
eb28012897bd6b9895c1c47c52f2bc9e3228c19e
125
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Juan Castro
|
0c6045ed90 |
feat: emit team billing events for the payment integration
The charge lives outside this repo, the way the marketplace extension cancels Stripe subscriptions off `user.delete`. This is the trigger, and it is the whole of what OSS owes billing. team.account.created a seat exists and can be used team.account.disabled it stopped, and still holds its bytes team.account.enabled it resumed team.account.deleted it is gone team.deleted the team is gone; its accounts are not Each carries the team uid, the affected account, and the owner's `stripe_customer_id`. That column ships in the mysql and postgres schemas but not sqlite, so the read is guarded and degrades to null, as `cascadeDelete` already does. Deleting a team emits one `team.account.disabled` per seat plus the team event, rather than one bulk event: the accounts persist, suspended, holding their files and their usernames. Deleting a team is not a way to stop paying for the accounts in it. `team.account.deleted` is captured before the row is deleted. `jct_user_group.user_id` is ON DELETE CASCADE, so by the time a listener on `user.delete` runs, nothing can say which team paid for the account. `getOrgSeat` deliberately admits soft-deleted teams: their accounts still exist, so the charge is still running. Closes PUT-1712. |
||
|
Juan Fernando Castro
|
00afea1285 |
feat: a seat is created, never adopted (#3720)
`addMember` refuses to turn an account that already has a password into a team seat. No service path did this — `provisionAccount` always creates — but the store permitted it, and the design rules out existing accounts joining a team. Provisioning passes the guard because it admits the account before setting its temporary password. There is no bypass parameter. Both HTTP suites now provision a real seat and authenticate as it, using the same token-minting the harness uses for `POST /login`. That surfaced something worth knowing: an unactivated seat cannot call the API at all. Provisioning leaves `requires_email_confirmation` set and `requireVerified` rejects it, so the suites activate the seat first — which is the state a member is actually in when making requests. `listMembers` and `getMembership` also return `u.uuid`, which the billing events need in order to name the account without a second lookup. |
||
|
Daniel Salazar
|
86050f131b | fix: hardening events for shared kv (#3827) | ||
|
Daniel Salazar
|
927317bc4e | fix: events hardening (#3814) | ||
|
Daniel Salazar
|
a441e7f748 |
fix: harden events (#3813)
* fix(events): rename carries from, one retry on deploy-timeout, forward-path counters - FSService.rename passes the pre-rename path so a folder subscription sees op move with from, like a real move does - deploy-timeout from the dispatcher gets one retry with the deployed header and no second upload (ALREADY_DEPLOYED_MISS_REASONS) - OTel counters events.forward.sent/received and events.single.attempt - cross-app KV subscribe error hints at the three-segment parse - docs: lease is 60 s, kv prefix example is fully qualified, move covers rename * feat(events): forward session subscriptions across regions, fast bumps, worker session cleanup - session (onLocal) subscriptions now receive writes committed in other regions: a transition-maintained remote-watch index (ev:sc / ev:rw), watch/event forward items, replay through dispatchForwarded against session rows only; events.forwardSession=false is the kill switch - subscription and presence generation bumps also ride the addressed forward channel (kind bump) so a peer sees a new durable row within a queue window; the webhook fan stays as backstop - workers.destroy revokes every holder's events:handlers session; app deletion reaps the app's rows, backlog and handlers and revokes the sessions; an hourly sweep revokes sessions whose app is gone - docs: cross-region latency, per-region caps footnote, session lifecycle |
||
|
Daniel Salazar
|
bc9cb2d7e7 |
fix: withdrawing background consent revokes the app's events session (#3777)
* fix: withdrawing background consent revokes the app's events session A background handler runs as a worker session for the subscriber and app. Revoking `events:background` or uninstalling the app suspended the subscriptions but left that session valid, so a token a handler had copied out kept working until the user found the row in the sessions list. The revocation settle now revokes the session too; the next consented delivery mints a fresh one. * fix: cleanup docs |
||
|
Daniel Salazar
|
b784b51cf3 |
fix: harden the events stack for flag-on (#3752)
* fix: harden events dispatch, single delivery and KV share handles Dispatch: a filtered subscription used the anchor path stored at subscribe time, so renaming or moving the anchor folder silently ended its deliveries; dispatch now resolves the anchor's live path from the event's own ancestor chain. A move out of a watched folder now reaches that folder's subscribers, with `from` only for rows that watched the source side. Gap markers are authorized like deliveries and coalesced per subscription and subject instead of fanning per lost event. Session subscriptions: the per-socket cap decides on the write, not before it; an orphaned watched-set token heals on refresh; durable rows keep their watch window when a session subscribe touches the same keys. `self` is false when the acting user is unknown. Single delivery: a subscription in backoff or suspended with a backlog pinned the sweeper's head and starved everyone behind it — the sweep now defers it. Only a settled handler run bills a delivery. A socket-only account row no longer wedges after two attempts nobody received. The lease is twice the handler timeout; remote candidates have their own attempt counter; the region depth reconcile runs once a minute region-wide with a bounded scan. KV share handles: a grantee no longer sees the owner's namespace and absolute prefix on the subscribe answer or listing, nor in the delivery token; revoking a wider handle retires the handles it covers; minting the same handle twice returns the existing one, after the delegation check; a row whose event cannot be re-based onto its handle is dropped rather than delivered raw. * fix: presence survives replication, long sessions and region churn One presence item per (user, app) with per-region map fields lost a region whenever two regions joined inside the replication window, and nothing ever put it back. Presence is now one item per (user, app, region): each region writes only its own, a leave or repair retires it conditionally on its own write stamp, and a read is a prefix query. Items carry a 48 h ttl refreshed by a claim-gated write off the existing socket renew path, at most once per 12 h, so a tab that stays connected keeps its region in the row. A region that answered "no socket" or completed a leave releases a shared pin, so a reconnect on another node rejoins and a flapping client cannot force a replicated write per cycle. Cached rows expire after a minute; unaddressable region names are filtered and pruned; relayed acks settle under a bounded concurrency; the forward queue is bounded in bytes as well as items. * feat: indexes for the event_subscriptions hot queries Handler publish, remove and listing, and the hourly expiry and suspension sweeps, all scanned `event_subscriptions`. Adds (app_uid, handler_name), (expires_at) and (suspended_at, id), guarded on every engine. Existing migrations: the postgres widens are now guarded so a boot does not take an exclusive lock for a no-op, the kv_share_handles grantee FK gets an index, the sqlite notification rebuild is transactional and idempotent. * fix: notification writes go through the registry The driver's `create` bypassed the type registry, producing uncatalogued rows with no size bound; it now requires a registered type, caps the payload, and answers 400 rather than 500 for a bad one. `mark_acknowledged` emits the ack other tabs listen for, and only when a row was actually changed. * fix: the handler scanner, unsubscribe, and the in-tab handler environment The free-variable scanner skipped arrows inside a declaration's initializer, so `const ids = event.items.map(x => x.id)` was refused, and treated a name after a comma in a nested initializer as bound, so a real free variable slipped through to fail on first delivery. `unsubscribe()` now drops the durable routing entry so the events socket can close. A broadcast handler running in the tab gets `user` and `fetch` like the worker gives it. `single` without a handler name is refused before the round trip. * docs: events limits, error codes and the background-workers section Retention is deployment-configured rather than a fixed 14 days, and the template no longer ships it armed. Documents `events_terminal`, the two per-event gap reasons, the subject length and listing caps, the `from` field on moves, and the handle-relative anchor. The sessions manager hides the background-workers section when the server has none to show. * feat: a background handler acts as the app does for its user A handler's `user` was a five-minute access token scoped to the subscription's `list` grant, which could stat the changed file but not read it, and could not reach the app's KV or AppData — so an app told that a file was written could do nothing with it. It now runs with the same authority the app has for that user in a tab: an app-under-user worker session, one row per (user, app) named `events:handlers`, visible and revocable in the sessions list. The `events:background` consent is what authorizes running it unattended, and is re-checked before every mint. The wider token exposed two things: puter.js opens a filesystem socket the moment it has a token, which would have parked the isolate in the app's own delivery room and steered deliveries at it; the events client now opts out of sockets (and the per-open bookkeeping) before construction, and is memoized per token in the isolate. And four filesystem operations assumed a socket exists; they no longer do. |
||
|
Daniel Salazar
|
f30baa2a1c |
feat: the per-app events worker runtime (#3697)
* feat: bake published handlers into a generated events worker
* feat: deploy and address the per-app events worker behind a flag
* test: single delivery end to end through a real local worker
* feat: events workers run their own runtime, in their own namespace
An events worker was being deployed as an ordinary worker: default dispatch
namespace, a `subdomains` row, the router preamble, and an app-scoped worker
token baked in. The public dispatcher resolves any script in that namespace
straight off the hostname, so the worker answered at `<name>.puter.work`, and
the only thing in front of it was an unguessable name plus a check that a
`puter-auth` header was present — which the router never validates. Anyone who
learned the hostname could run an app's handlers with a body of their choosing,
in an isolate holding the owner's token as `me`.
Instead:
- Handlers run on their own runtime (`src/worker/src/events-runtime.js`), which
provides no `router` and no `me`, owns the single invoke route, and hands a
handler only `{ event, ctx, user, fetch, ack }`. `user` is built from the
invocation's delivery token, so a handler acts as the subscriber whose
delivery it is and nothing wider. The preamble build emits one bundle per
runtime; the shared half of the template is now included by both.
- The deploy target carries the runtime to prepend, the source to deploy, and
whether to mint a worker token at all, so an events worker deploys into the
`events` dispatch namespace from generated source with no token binding, no
`subdomains` row, and no claim on the owner's worker quota or worker list.
- An invocation carries a key derived from the deployment secret and the script
name, bound as a secret and checked in constant time inside the isolate,
which reads it once and drops it before handler code runs.
- Scripts are named after the handler set they contain, so publishing writes
rows and deploys nothing: a set is deployed the first time a delivery needs
it, and a changed set is a new script rather than an overwrite of a running
one. Publish responses keep the shape they had before the runtime existed.
- Invocations reach a worker only through the events dispatcher, which has no
zone route and requires the internal secret; the backend's own deploy path is
the rehydrate route the dispatcher calls on a namespace miss. Locally there is
no dispatcher, so the controller hands the service an in-process transport
that deploys on miss itself.
The SDK stops allowlisting `puter` as a handler global — a handler that reaches
for an ambient SDK is now refused at publish time, naming `user` instead, rather
than passing the scan and failing on its first delivery.
Requires `events.workerNamespace`, `events.dispatcherUrl` and
`events.internalSecret`; without them nothing is addressable and background
deliveries stay retriable, as they did with the runtime off.
* fix: a handler's delivery token gets through the read routes
An events handler acts as the subscriber through the access token its
invocation carried, but every FS read route refused scoped access tokens
outright, so `user.fs.stat(event.path)` — the design's own example — answered
403 inside the worker. The read-side routes now admit them; the ACL each
handler already runs intersects the token's grant with its issuer's, which is
the check that keeps a token to what it was minted for. The end-to-end suite
asserts the stat from inside the isolate.
* fix: shorthand-method handlers publish as functions
`{ ingest({ event }) { … } }` stringifies without the `function` keyword, so
its source is not an expression and the events worker baked it as a broken
stub — every delivery a retriable 500 until the subscription suspended, with
nothing at publish time to say why. The SDK now gives a shorthand method the
keyword before hashing and sending; getters, setters and computed names are
left for the server-side check to refuse.
* feat: an app's events worker is listable and destroyable
An app with published handlers has an events worker, and hosted deployments
bill it monthly per app, so its owner needs to see it and be able to take it
down. The core announces the lifecycle on the bus — `events.worker.create`
when an app's first handler is published, `events.worker.destroy` when its last
one goes — with the owner as the actor, so pricing can plug in from outside.
`GET /events/workers` lists the caller's workers (paginated, with the script
each set deploys as) and `POST /events/workers/destroy` removes every handler
of an app under the same owner scoping as the handler routes, suspending the
subscriptions bound to them. `puter.events.workers.list/destroy` in the SDK,
a docs page, and a 5 MB cap on an app's combined handler source
(`events_worker_too_large`) so a set that publishes can always deploy.
* fix: harden the events worker runtime for production
- A 4xx is terminal only when it carries the handled marker the runtime (and
the dispatcher) stamp on every answer that came from a script; an unmarked
4xx — an edge 404 for a wrong dispatcher hostname, a WAF page — stays
retriable and is logged, once per script per minute, with the runtime's
reason header.
- Script names are scoped to this backend's exposed API origin, so two
backends sharing a namespace never resolve one script with the wrong
endpoint binding or key. Shape unchanged.
- Each handler is validated in the exact context it is emitted into and the
whole generated file is compiled once; a source that would break the script
marks every handler broken instead of deploying a SyntaxError.
- Locally, events scripts live under their own registry key: the public local
worker host cannot reach them and an ordinary worker cannot take their name.
- A suspended or deleted app owner stops invocations; deploys are throttled
per app per hour; in-flight deploys are keyed by app and script; the
upstream deploy call times out; the generated source is size-capped with a
margin over the publish cap; boot fails when the runtime is on but its
preamble is not built. Byte-length secret compare, appUid shape check,
dispatcher URL prefix preserved, wider connection pool.
* feat: background workers are listed in the sessions manager
A user paying for an app's events worker needs somewhere to see it and take it
down. The sessions manager gets a section listing the apps that run event
handlers in the background, with a Destroy action that removes their published
handlers.
|
||
|
Daniel Salazar
|
f139588f89 | feat: revocation settles kv handles (PUT-1687) (#3691) | ||
|
Daniel Salazar
|
ac5446f877 | feat: cross-user KV share grants and handles (PUT-1686) (#3690) | ||
|
Daniel Salazar
|
adde6c3a29 |
feat: notification GUI rides the events surface (PUT-1685) (#3689)
Widen the notif: match filter and fetch scope for a session's own generic developer/app-user subscribe: today it pins ref to the session's own uuid, so a row naming an app (handler-suspension notices, app-bound worker deploys) never matches live and never replays on reconnect, even though the audience predicate already grants the holder every such row it owns. The predicate is the authority and already reruns per row/page after the match, so widening the filter to it (account is unaffected — it never names an app) adds no exposure. |
||
|
Daniel Salazar
|
fe5f81e52e |
feat: scoped app access to notifications (PUT-1684) (#3688)
An actor holding an app reads the `app-user` rows naming that app, plus its `developer` rows when the holder owns it. `account` rows reach no app, and a slice an actor may not see comes back empty rather than refused. The audience predicate becomes the enforced read path in the same change that lifts the blanket app-actor 403, layered behind an audience/app_uid SQL scope; two-segment `notif:` subjects expand server-side from the actor's own app, so an app can never name another app's uid. No feature flag: `audience` defaults to 'account', so every pre-registry row is default-denied to app actors and the backfill can only narrow. |
||
|
Daniel Salazar
|
796683a133 | feat: missed-event fetch and notification fold-in (PUT-1681) (#3687) | ||
|
Daniel Salazar
|
eb8f497e9f |
feat: presence and cross-region event forwarding (PUT-1679) (#3686)
* feat: presence and cross-region event forwarding (PUT-1679) * fix: fan cache bumps to sibling nodes and stop the forward shed cascading (PUT-1679) `outer.events.generationBumped` and `outer.events.presenceBumped` rode `outer.*`, which the broadcast service only webhooks to peer regions; only `outer.pubsub.*` also fans over Redis to a region's other nodes. Both caches are per-process maps with no expiry, so a bump landing on one node left its siblings stale until that user's next transition. Renamed onto `outer.pubsub.events.*`; the listeners already accept the `from_outside` copy the Redis re-emit carries. `PeerForwardQueue.push` called `onOverflow` synchronously and the handler pushed markers straight back, each of which re-tripped the bound and shed the next item: one item over a 5000 bound recursed ~2200 deep, threw a RangeError, and turned ~2200 queued deliveries into gap markers. It also re-summed `bytes` over the whole queue per drop. The handler now returns its markers and the queue appends them past the bound check, sheds deliveries before markers, keeps one pending marker per (peer, subscription), and subtracts bytes per dropped item. |
||
|
Juan Castro
|
9bc2cd741c |
feat: add TeamController, the teams_enabled flag, and the isolation suite
Covers PUT-1708, PUT-1709 and PUT-1743. Twelve routes, every one setting requireUserActor -- that option is what installs requireAuthGate, requireVerifiedAccount and requireNonAccessTokenGate, because server.ts derives `needsAuth` from the route options. Reads need it as much as writes: without an auth option a route gets no suspension check and admits access tokens, so a just-disabled member could still read the roster and a scoped third-party token could read the audit log. Authority is checked before anything observable. Validating the body first made POST /members answer 400 before 403, and resolving :username first turned the member routes into a global username-existence oracle. Provisioning applies the same username and email rules as signup rather than its own -- USERNAME_REGEX, USERNAME_MAX_LENGTH, RESERVED_USERNAMES and validator.isEmail, now exported from AuthController. Without them a workspace could mint accounts signup would refuse, claim unregistered reserved names, and mail arbitrary unvalidated addresses. Handle problems are 400 or 409 rather than a bare Error, which the server turns into a 500 and a deduped critical alarm -- an uppercase handle should not page on-call. Disable drops sessions through SessionStore.removeByUuid rather than a raw DELETE. The store invalidates every composite cache key; without that a disabled member kept authenticating from cache for the session TTL, which is exactly the "takes effect on the next request, not after a cache TTL" property disable is supposed to have. Revoking also preserves last_ip/last_user_agent, which the member-facing audit view reads. Audit writes live in TeamService at the point of each action rather than in the route, so a caller reaching the service directly cannot skip them, and the SQL lives in TeamStore. Audit reads map internal user ids to usernames, and remain readable by the owner after the workspace is soft-deleted -- otherwise the delete_team entry was written and immediately unreachable. teams_enabled gates route registration through an optional isEnabled() the server honours, so with it off the paths do not exist rather than existing and refusing. It does not gate DDL. TeamIsolation.http.test.ts asserts the negative the feature rests on: the workspace manages accounts and cannot read them, including through a full-access token and after the member is disabled. It asserts outcomes rather than the absence of an implicator. |
||
|
Juan Castro
|
8b4e259421 |
feat: add team membership reads and writes to TeamStore
Membership management for workspaces: addMember, removeMember, getMembership, isMember, listMembers and listTeamsForUser. The permission scan is untouched -- readUserGroupPerms already joins jct_user_group and resolves group grants; this is the management side. Resolves the ticket's "do not leave two writers" by splitting domains and enforcing the split in SQL rather than by convention. Every existing caller of GroupStore targets a seeded system group -- ADMIN_GROUP_UID, default_user_group, default_temp_group -- never a team, so the two stores were already disjoint in practice. GroupStore.addUsers/removeUsers now carry `AND kind IS NULL`, making a team uid a no-op there, which costs no extra query because it folds into the existing subquery and matches how addUsers already treats an unknown username. TeamStore's writes select group_id from a kind-filtered subquery, so neither store can reach the other's rows. org_owned is written here but never accepted from a request; TeamService sets it at provisioning and workspace creation only. listMembers is keyset-paginated on id per doc/pagination.md, using the shared cursor and limit helpers and fetching one row past the limit to decide whether a cursor is warranted. Passes 1/0 for org_owned rather than db.booleanValue, which yields a real boolean on postgres and is rejected by the smallint column there -- sqlite accepted it silently. |
||
|
Juan Castro
|
37cb11df8c |
feat: add TeamStore for workspace CRUD
`GroupStore` has only addUsers/removeUsers; nothing creates, reads back or lists a group at runtime. `TeamStore` is that missing half, scoped to rows with `kind = 'team'`. A workspace is addressed by `uid`, which `group` has carried as NOT NULL UNIQUE since 0015. `handle` is a mutable display label with no addressing role, so a rename invalidates nothing and a stale reference can never resolve to a different workspace. Soft delete releases the handle and keeps `name`. Nothing points at a handle, so the name returns to the pool instead of being reserved forever by a global unique index that cannot exclude dead rows -- mysql has no partial indexes, so that exclusion was never available. Handles validate to ^[a-z0-9]+(-[a-z0-9]+)*$, 3-64 chars, against a reserved list. The charset is deliberately narrower than the column so the engines' collations cannot disagree: mysql's utf8mb4_unicode_ci also folds accents and eszett, which sqlite's NOCASE and postgres's lower() do not. Every read filters `kind = 'team' AND deleted_at IS NULL`, which is what makes the seeded admin/system groups unreachable rather than merely absent. Handle lookups compare lower(handle) on postgres, where the index is on that expression rather than the column. |
||
|
Juan Castro
|
dc7b456afa |
fix: run the membership dedup once instead of on every boot
Review follow-ups on the dedup migration. mysql and postgres track no per-file applied state and re-execute every migration on each boot, so the unguarded DELETE self-joined the whole table at every process start, forever. Both now sit behind the same index-existence check that guards the ALTER, which also stops a rolling deploy deleting on one instance while another adds the index. The dedup test was a false green. `targetVersion: 67` never applies 0071 -- the loop breaks on `threshold + 1 >= targetVersion` but stamps the target anyway -- so the fixture asserted the current schema version on a database missing a migration. It now replays the real 0072 file against a fully migrated database, and a second test pins the off-by-one so nobody builds a fixture on it again. |
||
|
Juan Castro
|
c1c1588939 |
fix: deduplicate jct_user_group and make membership writes unique
\`jct_user_group\` had no unique constraint on (user_id, group_id) and \`GroupStore.addUsers\` had no conflict clause, so re-adding a member inserted a second row. \`readUserGroupPerms\` joins the junction table on group_id alone, so each duplicate returned another copy of every group permission the user holds. Deduplicate keeping the lowest id, add the unique pair index, and make \`addUsers\` ignore conflicts via the existing \`insertIgnoreInto\` helpers -- without that last part the index turns a re-add into a raised error, which five call sites would log as a failed signup step. mysql cannot delete from a table it reads in a subquery (error 1093), so it uses a self-join with the same lowest-id-wins semantics. |
||
|
Daniel Salazar
|
72011fa6ed |
feat: events metering and quotas (PUT-1683) (#3685)
* feat: events metering and quotas (PUT-1683) * fix: drop the standing subscription charge and price single deliveries at 100 µ¢ An idle durable row costs nothing worth billing; the plan quotas bound how many an account holds. Removing the daily line also removes the global day-claim, the whole-table scan and the timer it rode on. * test: stop asserting on documentation pages The limits test read rate-limits-and-quotas.md and grepped it for numbers, so every rewording of the page failed the backend suite. Docs are kept in step by the PR and checked in review; AGENTS.md now says so. |
||
|
Daniel Salazar
|
41b837b398 | feat: background delivery seam, retries, and consent (PUT-1682) (#3684) | ||
|
Daniel Salazar
|
04c00385b9 | feat: event handlers, context, and suspension machine (PUT-1680) (#3683) | ||
|
Daniel Salazar
|
9626ab9c71 | feat: KV change events and KV subjects (PUT-1678) (#3682) | ||
|
Daniel Salazar
|
d8ebfc8023 |
feat: delivery re-check cache, revocation and anchor settle (PUT-1677) (#3681)
* feat: delivery re-check cache, revocation and anchor settle (PUT-1677) * fix: authorize re-anchors, settle each row once, purge revoked backlog (PUT-1677) - A path-form row whose anchor is deleted only climbs to an ancestor its holder may still watch under the mode it subscribed with; otherwise it ends with `anchor_deleted`. It used to land on any surviving ancestor (a guest's row on the owner's home), where the re-check denied every delivery but the row still held an anchor slot and a filter evaluation there. - After a climb the new anchor is re-verified and the climb repeated if a recursive delete took that level too, instead of leaving the row on a dead uid. - suspend() is one conditional write per row and reports which rows it was the one to suspend; concurrent settles of the same grant (an unshare revokes several strings) no longer each purge, forget and notify the same rows. - One "subscriptions ended" notification per holder and app, carrying the count and subjects, instead of one per row. - A revoke that removed nothing no longer announces; the sweeper purges (not defers) the backlog of a permission_revoked row; the reap purges pending entries with the row. - The delivery auth cache indexes entries by subscription so forget() is not a scan of the whole cache. |
||
|
Daniel Salazar
|
41d4850ab4 |
feat: pending event deliveries and delivery-class invariants (PUT-1676) (#3680)
* feat: pending event deliveries and delivery-class invariants (PUT-1676)
* fix: make pending delivery claims and drains atomic, keep the region under its ceiling (PUT-1676)
- claim() and the drain-time reindex run as Lua over the subscription's own
{subId}-tagged keys. Two claimers can no longer both lease the head, and a
drain that finds the queue empty deletes it in the same step it checks, so a
concurrent enqueue is never wiped between the two.
- An append writes the entry and its queue position in one MULTI (same slot),
with the index seeded before it and corrected after, so an entry is never
visible without its position and never left out of the sweeper's index.
- Pipelines no longer mix slots (index/counter vs. per-subscription keys), so
the store works on a multi-shard cluster, not only a single-shard one.
- Region shedding counts the marker it leaves behind; it used to stop one over
the ceiling and convert a real event into a marker on every enqueue after.
- A claimed or suspended subscription moves to the back of the sweeper's index,
so a delivery nobody settles cannot hold the head against every other backlog.
- `single` rows must carry a `worker` target: with sockets exhausted and no
handler, an unacknowledged delivery would sit at the head forever.
- A gap marker for a row with no socket target is dropped rather than counted
as a delivery of nothing.
- Backlog keys carry a 7-day TTL, refreshed by every claim, as a backstop for
keys a purge/enqueue race left unindexed.
|
||
|
Daniel Salazar
|
0dfbceb047 |
feat: durable event subscriptions store, cache, and routes (PUT-1673) (#3679)
* feat: durable event subscriptions store, cache, and routes (PUT-1673) * fix: durable subscription hardening (PUT-1673) - Expired rows stop delivering at dispatch time and no longer count toward the per-account cap, instead of waiting for the sweep. - The expiry sweep runs hourly with a jittered first pass shortly after boot; a 24 h interval never fired on a fleet that redeploys more often than that. - Only a durable generation bump marks peer regions cold. A session subscribe/unsubscribe in one region used to force a primary read in every other region on its next dispatch. - `subject`/`anchor_path` widen to varchar(4096) to match `fsentries.path`, and subjects longer than that are refused with `invalid_subject` rather than failing the insert on MySQL/Postgres. - The dispatch and durable integration suites wait for the specific delivery they expect and assert only within their own folder; the old any-delivery `settle()` let a late event from a previous test satisfy or pollute the next one under CI load. |
||
|
Daniel Salazar
|
66dc0a6396 | feat: subscribe ACL and app scoping for event subscriptions (PUT-1672) (#3677) | ||
|
Neal Shah
|
262f1dc5c5 |
allow concept of "home regions" (#3699)
* allow concept of "home regions" * remove extraneous config value not applicable to repo |
||
|
Daniel Salazar
|
b07d2e109f | feat: session event subscriptions and dispatch hot path (PUT-1666) (#3675) | ||
|
Daniel Salazar
|
d0ee19a2c2 | chore: remove dead notification surfaces (PUT-1669) (#3673) | ||
|
Daniel Salazar
|
82076ea767 | feat: notification retention sweep (PUT-1668) (#3672) | ||
|
Daniel Salazar
|
5f37817a5c | feat: notification type registry and audience rules (PUT-1665) (#3671) | ||
|
Daniel Salazar
|
148b930182 | feat: notification scope columns and backfill (PUT-1659) (#3670) | ||
|
Daniel Salazar
|
1736d23ee7 |
Ds/put 1674 (#3715)
* feat: app-scoped share listing and revoke surface (PUT-1670) * fix: address review on app-scoped share surface - Scope the uid-addressed revoke to the named row: only that row's issuer's grant is withdrawn, and only that one invite cancelled — an app or owner addressing one row no longer takes another issuer's grant on the same (item, recipient) pair with it. - Delete a pending row directly on uid-addressed revoke, so an invite whose address registered but never claimed can still be withdrawn. - Read the legacy `issuerAppUid` data key in the SQL app filter and grouping, alongside the unified `issuedByApp`. - Refuse malformed `appUid` input (duplicated param, empty string) instead of silently listing everything, and refuse app-listing cursors that decode but name no appUid. - Derive the acting app from `effectiveApp` alone, per the actor contract; drop the second derivation site. - Pin the attribution semantics with tests: one row records one issuance, so re-sharing the same pair re-attributes it to whoever issued last, in both directions. - Soften the uniform-404 docblocks to what the gates actually answer. * feat: readable grant audit trail (PUT-1674) * fix: cover the apps summary's no-app-group first page (PUT-1670) listOutboundApps sorts the no-app group first via an empty-string sentinel. Add a regression test pinning that a first page (no cursor) actually returns it, and that the cursor it hands back resumes past it into the app-keyed groups rather than skipping or repeating. |
||
|
Daniel Salazar
|
66a975f659 |
feat: global outbound share listing (PUT-1664) (#3694)
* feat: global outbound share listing (PUT-1664) * fix: address review on outbound share listing - Check share-row liveness per (holder, entry, issuer) so a grant withdrawn outside unshare doesn't stay listed while another issuer still reaches the same holder; batch the permission reads across the whole page instead of per holder. - Retire a revoked issuer's unclaimed invites in the revoke cascade, and hide invites whose issuer lost their authority at read time. - Unify the pending/active app-attribution key on `issuedByApp` and dual-read the legacy `issuerAppUid` spelling. - Add the missing share issuer index (sqlite, postgres) and correct the listOutbound plan comment. - Refuse cursors that decode but name no id instead of silently restarting from page one. - Consolidate the five hand-built ResolvedShare literals and the two listing endpoints' parse/shape code. - Ship the SDK surface: puter.fs.listSharedByMe() with docs, types, suite coverage, and the rate-limit page entry. |
||
|
Daniel Salazar
|
f3a46a9be4 |
fix: stop sockets outliving the session that authenticated them (#3658)
A socket was checked once at handshake and never again. Nothing in the backend disconnected one, so logout-everywhere, password reset, session revoke and suspension all left every connection streaming legacy FS entries, upload paths and notification bodies — up to 400 per account — on a credential that had already been revoked. Three gaps, three fixes: - The handshake skipped the suspension and pending-verification checks every authenticated HTTP route gets. `decideSocketAuth` now applies both. - `revokeCascade` reports which rows it revoked, AuthService announces that as `auth.sessions.revoked`, and SocketService drops the account's room. The whole room goes, not just the revoked session: narrowing it would need `fetchSockets`, which the adapter builds on `serverCount()` — and that calls node-redis's `send_command`, which ioredis does not implement. A connection whose session survived reconnects on its own and re-authenticates. - A bulk suspension writes `user.suspended` without touching `sessions`, so no revoke fires. A five-minute sweep re-verifies each live socket's token and drops the ones that no longer authenticate. De-duplicated by token, since a browser's tabs share one. |
||
|
Nariman Jelveh
|
7332993ff3 |
Show share notifications in the dashboard (#3653)
* Show share notifications in the dashboard The dashboard opens its own socket but never listened for notif.* events, so a file shared with a user on the dashboard was announced to nobody. Add a notification center to the dashboard: a bell in the sidebar with the unread count (a dot when collapsed, and on the mobile hamburger), a panel anchored to it — a bottom sheet on phones — listing what the server still holds as unacknowledged, and toasts for what arrives live. Clicking a share lands on Files › Shared with the item selected; dismissing, and "Mark all as read", acknowledge on the server so other tabs clear too. A toast timing out is not a dismissal: the entry stays unread in the panel. Arrivals fold in by uid (a regrouped share rewrites its row and toast in place), the panel refreshes on open, reconnect and tab focus so shares the backend folds in silently past the sender's budget still surface, and a burst on connect is capped at three toasts plus a summary. Backend: notif.unreads carries created_at so delivered-on-connect items can be dated, and listings break same-second ties by id so order is stable. The desktop's mark-ack calls move to a shared helper; UINotification gains an optional auto-hide timeout that pauses on hover/focus. * fix: look up notification glyphs and icons by own key only A notification whose `source` or `icon` was a prototype key such as `constructor` made the glyph lookup return a function, and the toast icon builder then threw inside the socket handler — which dropped the whole burst of toasts it was part of. The list rendered the same entry as "[object Object]". * fix: show dashboard toasts above open app windows App windows opened from the dashboard are stay-on-top and stack in the 99999999+ band, so the toast container at 10000000 sat underneath them: any notification arriving while an app was open — the usual state of a dashboard session — was drawn behind the app and never seen. * fix: reveal the dashboard when a toast is clicked over an open app Clicking a share toast switches the dashboard to Shared, and the "N new notifications" toast opens the panel — both inside the dashboard window. With an app maximized over it, all of that happened out of sight and the click appeared to do nothing. Minimize the covering app windows first, the way the minimize controls do, landing the URL on the dashboard's own route, and act once the history pop has settled so the tab switch is not traversed over. * fix: keep keyboard focus in the notifications panel across re-renders Every change to the list rebuilds its rows, which dropped focus to the document body: dismissing an entry with Enter threw the keyboard out of the dialog (the row it had just been moved to was replaced), and so did any arrival while an entry was focused. Put focus back on the same entry after a render, or on the one that took a dismissed entry's place, and let Shift+Tab from the panel itself wrap to the last control instead of leaving the dialog. * fix: keep the notifications panel inside short viewports The anchored panel is bottom-aligned with the bell and grows upward, but its height was capped only by the viewport (560px or 100vh - 24px), not by the room above the bell — on a short window the header ran off the top of the page. Cap it to that room, so the list scrolls instead. * fix: make notifications with nowhere to go inert in the panel An entry with no target (a worker deploy result) showed a default cursor yet, when clicked, was dismissed and took the panel with it — nothing opened, the entry was gone, and the list had closed. Render such entries as text with the ✕ as their only action; only share entries remain buttons. The ✕ now names its notification for screen readers, since on an inert entry it is the only focusable control. * fix: stop a listing in flight from resurrecting dismissed notifications The list is refreshed on open, on reconnect, and when the tab comes back into view; a dismissal landing while that listing is in flight was undone when it resolved, since the server's snapshot predates the ack. When the listing also resolved after the server's own ack event, nothing removed the entry again and it stayed until the next refresh. Leave out of the reconciliation whatever was acknowledged — here or in another tab — since the listing was requested. |
||
|
Juan Castro
|
5115fdd48c |
Answer which entries are shared in one indexed query
Directory listings need a shared/not-shared flag per entry. Ask the share index once per batch, chunked, rather than a query per entry, and answer only for entries the caller owns so listing a foreign directory never queries at all. Direct shares only — inherited access stays with listSharesOf, which already reports it. |
||
|
Daniel Salazar
|
717c098503 | fix: metering buffer improvements for manual editing (#3633) | ||
|
Daniel Salazar
|
5918e3f4e2 | fix: kv max number value (#3629) | ||
|
|
f0cd251626 |
🛠️ PUT-1521: Cleanup puter js permissions api + backend routes (#3607)
* refactor(puter-js): collapse puter.perms request* to resource + access
Fifteen request methods differed only by a folder name or an access level, so
every new resource meant another method. Replace them with requestFolder,
requestApps, requestSubdomains and requestAppRootDir, each taking the access
level as an argument.
The old names stay as @deprecated aliases: puter.js ships unpinned from
js.puter.com/v2, so removing them would break live apps. They remain in the
generated declarations because stripInternal has no effect on declarations
emitted from JavaScript, and hand-omitting them would break TypeScript callers
the runtime still serves.
Also drops the user-to-user and user-to-group grant wrappers (groups.js and the
grantUser/grantGroup half of grants.js) plus the req_ shim, none of which were
documented or called. The app, origin and dev-app grants stay: the dashboard
uses puter.perms.revokeApp() to clear grants on app uninstall.
* docs(perms): document the collapsed puter.perms surface
Replace the twelve one-method-per-task pages with requestFolder, requestApps
and requestSubdomains, and rewrite the Perms overview around the seven public
methods. The deprecated aliases keep working but are no longer documented.
Boy Scout: drops the long-dead commented-out grantUser/revokeOrigin sidebar
block for pages that were never published.
* refactor(perms): drop unused user-to-user and user-to-group permission routes
Filesystem access is shared through /share, which records the grant so the
owner can see and revoke it. The older direct-grant paths were left behind with
no caller anywhere - not the GUI, not a doc, not an app: grant-user-user
(already a 501 stub), revoke-user-user, grant/revoke-user-group, and the five
/group/* CRUD routes.
Removing them orphans PermissionService.grant/revokeUserGroupPermission and its
group-members cache bump, the three PermissionStore group writers, and six
GroupStore methods, so those go too.
What stays, and why:
- grant/revokeUserUserPermission - ACLService and ShareService power fs.share
through them.
- The group permission read path (#scanUserGroup, readUserGroupPerms) - a
migration seeds the admin group unrestricted driver access, so it is
load-bearing.
- GroupStore getByUid/addUsers/removeUsers - signup, save_account, OIDC and the
self-hosted default user assign group membership.
No schema change: user_to_user_permissions, user_to_group_permissions and their
audit tables are untouched. Group rows now come only from migrations, so tests
that need one seed it with SQL the way a migration does.
* refactor(perms): reduce GroupStore to membership writes
With the /group/* routes gone, `getByUid` had no production caller left — the
routes were the only thing that read a group back. Removing it takes the row
decoder and the GroupRow type with it, since they exist only to shape its
result.
What remains is `addUsers`/`removeUsers`: signup, save_account, OIDC and the
self-hosted admin bootstrap all assign group membership. Permissions attached to
a group are read through PermissionStore, which joins the junction table itself
and never needed the store.
Tests that wanted a group id now select it, which is all `getByUid` was doing
for them.
* chore(perms): drop the three by-hand groups no code reads
freeai, experimental and dangerous exist in prod but in no migration — they
were added by hand when hardcoded permissions were keyed by group name. That
map is now a flat per-user floor (`default_user_permissions`), so a group
nothing looks up grants nothing.
Guarded rather than unconditional, because both tables the delete can reach
cascade: dropping a group that still carries permissions or members would
silently revoke them from every member. Only a group with neither goes. One that
survives has dependents and needs a deliberate decision — query
user_to_group_permissions by group_id to see what it holds.
system, admin, user and temp are untouched: config names two of them and code
names the others.
Matches on `extra.name`, not `metadata.name` — `metadata` carries the display
title and colour, and `critical: true` is set on all of these including freeai,
so it does not discriminate.
* feat(puter-js): collapse puter.perms onto request(resource, details) + check()
One method per task meant a new method, doc page and sidebar entry for every
resource. `request` now takes the resource and a payload whose accepted fields
depend on it, and `check` answers the same question without prompting.
request('folder', { name: 'Documents', access: 'write' }) -> path
request('apps', { access: 'read' }) -> boolean
request('email') -> address
check('folder', { name: 'Documents', access: 'write' }) -> boolean
Returns stay per-resource: a folder gives its path, email the address, the rest
a boolean, and anything denied is falsy so one `if` covers both.
An array asks for several at once. Everything already held is settled first, so
the prompt covers only what is missing and does not appear when the whole set is
held - the user answers once for the lot. `check` answers per entry, in order,
so a caller can tell which parts are missing rather than only that some are.
Each resource declares four things in one registry entry: how to ask for it
alone, whether it is held, the strings a batch pools into a prompt, and the
value once held. The strings themselves are defined once in
lib/permissionStrings.js, so a request and its check cannot name them
differently. `check` is built on /auth/check-permissions, already live and
already used by UI.js, and it throws rather than answering false when the check
cannot run: a caller that cannot tell "denied" from "never ran" would prompt
someone who had already granted it.
Backward compatibility: all 22 older methods stay callable and typed, marked
@deprecated with the call that replaces them. A lone string still routes to the
raw-permission path - no resource name contains a `:` and every permission
string does, so the two forms cannot collide. The grant/revoke app methods are
untouched; the consent dialog and the dashboard's uninstall path use them.
Also drops three copies of the access-level assertion onto one shared
validator, and gives `appRootDir` a non-prompting server probe, since
`app-root-dir:` only resolves while a grant is being written and a permission
check on it always answers false.
* docs(perms): document request() and check() as the perms surface
Five per-method pages became one `request()` page carrying the resource table,
the batch form and the raw-string escape hatch, plus a `check()` page. The
overview is rewritten around the two methods.
requestAppData's page is re-homed as /Perms/appData rather than deleted - its
scope table, private-entry guidance and lifetime notes are not signature
documentation and have nowhere else to live. Inbound links from KV/set.md and
Objects/app.md follow it.
Playground examples move to the new call form. They are not wired into
examples.js, but an example demonstrating a deprecated method is worse than one
nobody loads.
* fix(perms): keep /auth/revoke-user-user as a deprecated route
Dropping this route with the rest of the unused user-to-user plumbing went too
far. The grant side is retired and stays retired - puter.fs.share() is the only
way in - but access those grants left behind has to remain withdrawable, and a
caller reaching the endpoint over HTTP directly had no replacement. Revoking can
only ever narrow what someone can reach, so keeping it carries no risk.
revokeUserUserPermission never left the permission service; it is load-bearing
for puter.fs.share(). This only re-wires the handler to it, with the gates it
always had.
Nothing in this repo calls the route, which makes it exactly what a later
cleanup reads as dead, so a test pins the registration and its gate alongside
the restored 400 and grant/revoke round-trip cases.
The 501 stub at grant-user-user and the never-called /group/* routes stay
deleted, as does puter.perms.revokeUser - puter.fs.unshare() replaces it and
falls back to live grants when no share row exists.
* fix(perms): let a write grant satisfy a read check on apps and subdomains
`apps-of-user:<uuid>:write` covers managing the user's apps, which includes
reading them, but nothing said so to the permission system. Prefix implication
only widens the other way — an `apps-of-user:<uuid>` grant covers both modes —
so a scan for `:read` missed a `:write` grant, and `puter.perms.check('apps')`
reported an app holding write as holding nothing. A batched request would then
prompt again for access already granted.
Adds the read-from-write exploder for both namespaces, mirroring
`fs-access-levels`. The widening runs one way only, and does not cross into
another user's namespace; both are covered by tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(perms): answer the app-root-dir check without provisioning it
`/auth/request-app-root-dir` conflates two questions: may the caller claim its
root directory, and where is it. The second provisions `AppData/<uid>` on first
ask, so a caller that only wanted the first — `puter.perms.check('appRootDir')`
— created a directory by asking about it.
Adds `check: true`, which runs the same actor guard and stops at the answer.
A caller that may not claim it still gets the 403, so the flag can't widen
anything. Existing callers are unaffected: without it the route behaves exactly
as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(perms): put request() and check() on one path
`request` dispatched to the old per-task methods while `check` asked the
permission tables, so the two answered different questions about the same
access. Concretely, before this: `request('folder', { access: 'write' })`,
`'apps'`, `'subdomains'`, `'appData'` and `'permission'` prompted every time,
whether or not the access was held — which the docs said they wouldn't;
`check('folder')` reported false for a folder the app could read through an ACL
grant that no `fs:` string names, so a batch prompted for it needlessly; a
batch entry for `'appRootDir'` skipped the post-grant retry the single call
does, resolving `undefined` after a grant that had in fact succeeded; and an
N-entry batch made N permission reads plus 2N `whoami` calls.
Both now run the same pipeline — resolve the permission strings, read what is
held once, prompt for the remainder, resolve each entry — with per-resource
hooks for the parts only that resource can answer. So a batch costs one
permission read and one `whoami`, a check reports exactly what a request would
skip the prompt for, and `'folder'` uses the same stat-or-permission reading in
both.
Also:
- A resource is looked up as an own property, so `request('constructor')` is
the permission string it always was rather than a TypeError.
- A permission read that fails no longer decides anything: `request` falls
through to the prompt it would have raised anyway, `check` throws. Before,
`check('appRootDir')` folded a failed check into "not granted", which is what
the documentation says must not happen.
- Drops `requestFolder`, `requestApps`, `requestSubdomains` and
`requestAppRootDir`. They were added in this branch and immediately deprecated
— never shipped, and `request()` no longer needs to route through them. The 22
methods that did ship keep their exact behaviour, prompting without consulting
what is held, which the suite now asserts alongside the new behaviour.
- Documents `'appRootDir'`, which was a supported resource in every overload and
in `PermsResource` but named in none of the docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(perms): act on review of the three preceding commits
- `src/puter-js/test/perms.test.js` still called `requestApps`,
`requestFolder`, `requestSubdomains` and `requestAppRootDir`, which the
previous commit removed. Four cases in the interactive browser harness threw.
Pointed at `request(...)` instead.
- `request('appRootDir', …)` made two round trips where the shipped method
makes one: a read-only probe, then the call that names the directory. A
request is going to claim it either way, so the claim is now the check, and
the entry it returns carries through to the result. `check` keeps the
read-only mode, which is the reason that mode exists. Matters because the
route sits on the FS_SIGN bucket, shared with signed-URL minting.
- `requestPermission` is one of the shipped methods, and the previous commit's
message was wrong to say all 22 keep their exact behaviour: it forwards to
`request`, so it now settles a permission the caller already holds instead of
prompting for it. The value can differ, not just the prompt count — a user who
would have clicked Deny on a re-prompt used to get `false`. It is the more
honest answer (the app does hold the access, and denying a re-prompt never
took it away), but it is a change, and the suite assertion had been switched
to an unheld permission, which hid it. Asserted both ways instead, in the unit
tests and the API suite.
- An entry that names no permission no longer rides a grant given for the other
entries in the same call. Unreachable today — every resource either names one
or reports itself held — but nothing pinned it.
- Reverted three type-union reformats in `LegacyFSController.ts` that a
formatter had folded into the app-root-dir commit. That file was not
prettier-clean to begin with; reformatting it is somebody else's change.
- Docs and types: `Perms.md`'s `appRootDir` row now matches `request.md`'s,
`check.md` says that a `true` is per entry and a batch still prompts if any
one entry is missing, and `types.js` no longer names `requestFolder` /
`requestAppData` in prose that ships in the generated declarations.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Daniel Salazar
|
a552ce0b87 |
fix: better metrics for cache hit rates (#3609)
* fix: better metrics for cache hit rates * chore: type + docs |
||
|
|
b795b219a8 |
✨ PUT-1497: Share file link sharing and notifications (#3595)
* refactor(share): move share notifications into their own service
* feat(share): invite an address with no account, and email it
* feat(share): surface pending invites in the SDK and share dialog
* fix(share): unreachable revoke confirmation, and double-encoded labels
* feat(share): budget share announcements, group them, and let people block senders
Sharing had one defence against noise: a 15-minute quiet window per (sender,
recipient) pair, which dropped the second share rather than folding it in.
Twenty senders each under their own window could still bury someone, and there
was no way to make one of them stop.
Announcements are now budgeted on two axes through the existing sliding-window
limiter: 1 per 15 minutes and 20 per day from one sender, and 10 per hour /
50 per day to one recipient from anyone. Over budget the share still succeeds
and the recipient's notification is still brought up to date — only the
interruption is dropped. Invite email to an address with no account is budgeted
the same way, keyed on a hash of the canonical address.
Notifications now fold across senders: a new share rewrites the notification
the recipient hasn't dismissed, so "alice and bob shared 5 items with you"
replaces a stack of five. The record is written even when suppressed, so the
count is right whenever they next look.
Blocking is a new `user_block` table with enforcement in ShareService: a blocked
sender's share is refused with `recipient_not_accepting_shares`, spends no
quota, and writes no row, and their unclaimed invite is dropped when the address
is confirmed. Existing access is untouched — that is what revoke is for.
Managed from a Blocked people card in the dashboard's Security tab.
Also publishes the sharing limits, including the ones already on this branch
that were never documented.
* fix(share): name the item in share email, instead of 'an item'
* fix(share): make the invite lifecycle canonical, authorized, and race-safe
* refactor(email): drop EmailClient.isConfigured; callers read config.email
* feat(share): batch share email into a per-recipient digest, durably
* docs(share): document the share error codes; steady the disk migration tests
* fix(share): log why a digest wasn't sent, and recover orphaned ones
* feat(share): email recipients about shares by default, with a way to decline
Share email was off unless a deployment opted in, which meant an account
holder was told about a share in the app only. It is now on unless
`share_email_notifications` is set to false.
The reason it defaulted off was that nobody could decline. So this also
honors `user.unsubscribed` — the account-wide opt-out the /unsubscribe page
already writes and app feedback already respects, which share email ignored —
and the digest carries that link. Sharing and the in-app notification are
unaffected by it; only the mail stops.
The link is composed in the template around an interpolated uuid rather than
passed pre-built: Handlebars escapes interpolated values, so a whole URL came
out as `user_uuid=…`, which browsers decode but link scanners and older
mail clients need not.
* fix(share): count every shared file in the digest, not just the first
* feat(share): let a recipient refuse shares from everyone
Blocking answered "not from this person" but had no answer to "not from
anyone", so the only way to stop a stream of unwanted shares was to name
each sender after they had already reached you.
Stored as a key in the user row's existing `metadata` blob rather than a
column: the share path already holds the recipient's row by the time it
asks, so reading it costs nothing, and a one-bit preference doesn't earn
a migration per dialect. `updateMetadata` merges and refreshes the cached
row, so the switch bites on the very next share.
Refusing everyone reports the same code as refusing one person — which of
the two it is is the recipient's business, not the sender's. Enforced at
both moments the per-sender block is: when the share is issued, and when
a pending invite is claimed. The per-sender list is untouched while the
blanket switch is on, so turning it off restores what it hid.
`GET /share/blocks` now carries `all`; `POST`/`DELETE` take `{ all: true }`
beside the existing `{ username }`. Managed from the same Blocked people
card in the dashboard's Security tab.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(share): keep the digest sweep off a window that still has a timer
The sweep treated an entry as orphaned the moment its window closed, which
is also the moment the node that armed it fires. Claiming an entry is only
exclusive among flushers that can see each other's deletes, so the two
could each claim a share of the same digest and both send. It now waits
out a grace period first, which costs a genuinely stranded digest that
much delay and nothing else.
Both digest listings were capped at 200 with no word when they hit it — a
truncated flush sends a digest that undercounts and reads as complete.
The cap is named and logged.
Also: `#emailHolder` still described share email as off by default, which
it stopped being; the config doc said the batch window defaults to 60s
when it is 90; and the two tests that need several calls inside one window
were racing a 50ms window across four sequential round trips, so they
failed under full-suite load rather than on the behaviour they cover.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(share): stop double-encoding the recipient in two dialog messages
`i18n()` encodes what it returns, replacements included, so encoding the
recipient first showed the entities to anyone whose address or username
contains one. Same pattern already fixed two lines above.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(notification): widen the mysql shown/acknowledged columns
Both arrived from the v1 schema as `tinyint(1)`, where they were flags.
The backend rework changed the writes to a unix second; sqlite (`INTEGER`)
and postgres (`bigint`) took it and mysql did not, so on mysql every
`markShown` and `markAcknowledged` has failed with
ER_WARN_DATA_OUT_OF_RANGE and left the column NULL. Dismissing a
notification never stuck — the unacknowledged count never moved and one
already delivered came back on every reconnect.
No backfill: every reader tests `IS NULL` / `IS NOT NULL` only, so a
legacy `1` keeps meaning "yes" once widened. Guarded on the current type,
because changing a column type copies the table and this directory
replays on every boot.
Not reachable from the test suite — it runs against sqlite and postgres,
both of which already have the right type. Verified by hand against mysql:
`/notif/mark-read` and `/notif/mark-ack` now persist, and a dismissed
share notification is no longer the one a later share folds into.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Daniel Salazar
|
ec3c33fea6 | fix: misc hardening fixes (#3600) | ||
|
Daniel Salazar
|
bd06e88185 | feat: bring back referrals PUT-1463 (#3599) | ||
|
|
2c852bf6b3 |
✨ PUT-1412 File sharing backend api (#3553)
* refactor(permissions): drop hardcoded group permission map for a flat default
* test(drivers): assert credential-gate intent instead of a 403 proxy
* fix(permissions): report whether a revoke removed anything and persist the linked grant row before the flat view
* feat(permissions): replicate permission invalidations across regions
* feat(share): extend the share table into an index of active shares
* feat(share): query and maintain active shares in ShareStore
* feat(users): add a batched lookup by email
* fix(cache): apply cache updates broadcast from peer regions
* fix(permissions): scope a revoke to the issuer that granted it
* feat(share): add ShareService with a per-day share limit
A share is two writes that belong together: the permission grant, which
authorizes access, and a share row, which makes it listable and ties it to an
fsentry so it dies with the file. Nothing else grants fs:* to a user.
Authorization reuses canManagePermission — an owner satisfies it through the
is-owner implicator, a delegate through an explicit manage:fs:<uid> grant. An
owner may clear any issuer's share of their node; anyone else only the ones
they issued, or their own access. Self-revoke skips the manage gate but still
requires `see`, so it cannot be used to probe for files.
The per-day limit counts shares created rather than live rows, so revoking and
re-sharing cannot recycle a slot, and changing an existing share's mode is not
new reach and does not spend budget. Tunable via share_daily_limit.
* feat(share): expose sharing over HTTP
POST /share, POST /share/revoke, GET /share/shared-with-me, GET /share/shares.
The controller was registered but entirely commented out.
Recipients × items fan out concurrently — every pair is a distinct
(holder, entry) key, so none of them contend — bounded by
runWithConcurrencyLimitSettled, which returns results index-aligned with the
input for the per-pair outcome list. Responses carry usernames only, never
internal ids, and the 404-not-403 rule is preserved so a failed call cannot
confirm a file the caller could not otherwise see. Notifications are fired off
the response path; a share must not fail over its own notification.
Per-request caps on recipients and items bound one call's fan-out; the daily
limit bounds the total.
* feat(share): keep recipients consistent when a shared item changes
* fix(fs): stop listing issuer homes at the filesystem root
* fix(acl): serialize concurrent mode changes on one node and pin app containment on shared paths
* fix(fs): expire signed URLs over entries the signer doesn't own
signFile defaults to a ~317k-year TTL and verifySignature checks only uid,
expires and signature — never the ACL. A recipient who ever signed a shared
file therefore held a permanent, revocation-proof URL to its bytes: revoking
the share did nothing to it.
signEntry now takes the acting user and drops to NON_OWNER_SIGNATURE_TTL_SECONDS
(1 hour) when the signer is not the entry's owner. Owners keep the permanent
default, so no existing client changes behavior.
The signature-authenticated directory listing bounds its children
unconditionally: that route has no session actor, and a signature proves
possession rather than ownership, so a recipient holding a short-lived
directory signature could otherwise mint permanent URLs for every child.
A bounded window is not revocation — the durable fix is a per-entry signature
epoch folded into the HMAC and bumped on any permission change.
* refactor(permissions): drop the unused permission-issuer lookup
listUserPermissionIssuers and its store method listUserPermissionIssuerIds
existed to synthesize the filesystem root from the home directories of everyone
who had granted the caller a permission. That listing is gone — it advertised
folders readdir then refused to open — and the share index answers "who shared
with me" directly, so nothing wants them back.
One removed test only asserted that the call returned an array; the other
covered readLinkedUserUserPerms round-tripping and is kept, rewritten without
the issuer lookup.
* fix(share): return the created share, not just an acknowledgement
* feat(puter.js): add file sharing to puter.fs
share(), unshare(), listShared() and getShares() on puter.fs, following the
existing FS operation shape: positional and options-object forms through
defineOperation, JSDoc overloads as the published signature, relative paths
resolved against the app's root directory.
A bare recipient string is read as an email when it contains @ and as a
username otherwise. Sharing an item with someone who already has it replaces
their access rather than stacking a second grant, so raising read to write is
one more call.
Adds a sharing suite to the API runner, which passes unchanged on node,
browser and workerd. Documents all four methods with runnable examples, and
corrects the FS overview callout that told readers one user cannot read
another's files — true before this, not after.
* feat(gui): add a Shared folder for items others shared with you
A sidebar entry listing everything other users have shared with you, backed by
puter.fs.listShared().
The path is the sentinel `puter://shared` rather than /<user>/Shared: this is a
query, not a directory, and a path-shaped value could collide with a folder
someone actually creates. refresh_item_container and update_window_path both
branch on it to skip the stat there is no fsentry for, and the listing swaps
readdir for listShared.
Entries render at their real paths under their owners' directories — the item
container already preferred an explicit fsentry.path over joining onto the
container, so nothing else had to change. Each carries who shared it and at
what level, which the context menu reads next.
* feat(gui): share items from the context menu
A sharing dialog shaped like its neighbours — options object, HTML-string
template, jQuery wiring, delegating to UIWindow() — with a recipient field, a
read/edit/share dropdown, and the current access list with revoke buttons.
Reached from a new "Share…" context menu entry, which is hidden on items shared
*with* you: re-sharing needs manage, so the dialog would only surface an error.
Those items get "Remove from Shared" in place of Delete. Delete moves an item
to *your* trash, which for someone else's file means moving their data out of
their tree — FSService refuses it, and the user saw a bare 403. Removing your
own access is what the action was reaching for, so that is what it now does.
* fix(share): withdraw what a removed recipient re-shared
* feat(share): report access inherited from a parent folder
* fix(gui): load the puter.js bundle the server configured
* refactor(gui): extract the action icon set into a helper
* feat(gui): surface Shared in the file browser
* feat(gui): manage access from the share dialog
* test(share): cover access inherited from a parent folder
* fix(share): keep downstream access from surviving a delegate who leaves
* fix(gui): name the real owner in the share dialog
* fix(gui): page through every shared item instead of the first 50
* feat(share): return item metadata with a share
* fix(share): invalidate a holder's cache when the entry is deleted
* fix(gui): treat items inside a shared folder as someone else's
* feat(permissions): let manage inherit down the filesystem tree
Access already reached descendants through the ancestor chain while authority did not, so someone trusted to manage a shared folder could re-share the folder but nothing inside it, and could not see who had access to a file within it.
A manage-inherits-from-ancestor implicator resolves it in the permission layer, beside is-owner, so every caller agrees rather than just ShareService. It consults only the immediate parent — resolving that re-enters one level up, making a chain of depth d cost d checks rather than d².
That makes two cascade gaps reachable, both fixed here. A revoke now walks the subtree, since a grant on a descendant can rest on authority held at the folder. And it stops at a delegate whose authority survives another issuer, because what they granted was never theirs to lose.
Also pins that manage is not transitive: granting it needs manage:manage:fs:<uid>, which only the owner holds, so delegation is one level deep by construction.
* fix(gui): offer sharing inside a folder you manage
The menus encoded "manage does not inherit" and would now hide an action that works. The Shared listing records each root's mode; the menus resolve a child's by longest matching ancestor, loading on demand so a deep link or restored window works too.
* fix(share): make the daily share limit hold under concurrency
* test(share): cover concurrency, measure cost, and name cases for what they verify
* fix(gui): import the ownership helpers the item menu calls
The single-item context menu handler calls is_owned_by_me and
shared_mode_for, but the imports were only ever added to
generate_file_context_menu.js — so every right-click on an item threw a
ReferenceError before the menu could build, and the non-owner Delete
gating never ran.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): authorize before resolving the recipient
share() looked up the recipient in parallel with the entry, before the
manage check — and the two failures carried different error codes. Any
verified user with a real entry uid could probe arbitrary emails and
usernames for account existence, at no quota cost. Resolve the entry,
authorize, and only then resolve the recipient: an unauthorized caller
now sees the identical safe 404 whether or not the recipient exists.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(permissions): broadcast permission row-cache invalidations to peer regions
Every publishCacheKeys call for the u2u, u2a, and access-token row
caches omitted broadcast, so a revoke only cleared the mutating
region's Redis. A peer region applied the replicated generation bump,
re-scanned, read the deleted row from its own still-warm 5-minute row
cache, and re-warmed the flat view from it — revoked access outlived
the revoke by the row-cache TTL instead of the intended 60-second
bound. CacheReplicationService already consumes these events; the
emits were just never sent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(fs): refuse to rename an entry owned by another user
remove and move both refuse to act on an entry the caller does not
own, even when the ACL allows the write — rename had no such guard, so
a write-mode share recipient could rename the owner's file, or the
shared folder itself, rewriting the owner's whole subtree's paths.
rename now takes the acting user and applies the same policy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(permissions): decide a flat delete from the primary, not a lagging replica
revokeUserUserPermission deletes the SQL grant, then only drops the
flat KV entry once no issuer still grants the permission. That
remaining-check read through the row cache the delete had just
invalidated, straight to a replica — under any lag the deleted row
reappeared, the flat delete was skipped, and the stale rows were
re-cached for another five minutes. Grant-path flat entries carry no
TTL, so the holder kept working access with zero SQL rows behind it,
invisible to every listing. The check now reads the primary and
re-warms the cache with what it actually saw.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(permissions): keep a failed remote flat-invalidation from crashing the process
The outer.permission.flatInvalidated applier was fire-and-forget with
no catch, and it awaits a KV delete — one transient KV error while
applying a peer region's revoke became an unhandled rejection, which
is process-fatal under default Node. Its sibling appliers were already
guarded; this one now logs and moves on, leaving the entry to the next
invalidation or its TTL, same as a lost event.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): revoke every requested item and recipient, not just the first
revokeShare destructured only the first recipient and first item while
the parsers accept arrays up to the request caps — unshare({items:
[a, b, c]}) returned success having revoked only a, leaving access the
caller believes is gone. Revoke now fans out over every (recipient,
item) pair exactly like POST /share, reports per-pair outcomes, and
sums the revoked count; the response stays backward compatible.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): only a confirmed email designates a recipient
Recipient resolution by email accepted unconfirmed accounts, so
pre-registering someone else's address (unconfirmed) was enough to
receive shares meant for them once no confirmed account held it.
An email now only resolves to an account that has confirmed it;
username shares are unaffected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): accept tilde-rooted paths like the FS routes do
The SDK resolves relative paths to ~/..., but the share routes never
expanded the tilde — a ~-prefixed string was read as a uid and every
relative-path call 404'd. Item parsing now treats ~ as path-shaped and
expands it to the actor's home with the same helper the legacy FS
routes use, on share, revoke, and the shares listing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): walk a directory revoke by parent linkage, not path prefix
listByFsentrySubtree matched descendants with fsentry_id = ? OR path
LIKE ?, which has two problems: fsentries.path is lazily backfilled
and NULL on old rows, so those descendants' shares silently survived a
directory revoke, and the OR'd predicates forced a scan of every
active share. A recursive CTE over parent_id — the same shape the
lineage resolver already uses — covers every descendant and runs on
idx_parentId_name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(gui): give each item its own share dialog
single_instance keyed the dialog on the app id alone, so opening
Share… on a second file focused the first file's dialog — typing a
recipient there granted access to the wrong file, with only the title
hinting at it. The dialog is now instanced per path: same item
refocuses, different item opens fresh. Also stops pre-encoding the
title, which UIWindow encodes again.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(permissions): let manage answer a write check
A manage grant let its holder re-share a folder but not work in it: the
ACL mode family stops at write, and the fs exploder had no rule for the
narrowest mode, so `manage:fs:<uid>` never satisfied `fs:<uid>:write`.
Fold manage into the candidate list for every non-manage mode, in both
the access-token branch and the scan branch, and give `write` an (empty)
exploder rule so the manage arm is emitted for it too.
* fix(fs): authorize restructuring by write on the parent
* fix(fs): let a share recipient work inside a shared folder
rename, remove and move refused outright when the entry belonged to
someone else, so a recipient with write could neither delete nor rename
anything inside a folder shared with them. The GUI compounded it by
hiding Delete for any item it did not own.
Authorize the three by ACL write on the entry's parent. For an owner
that is the same answer; for a recipient it grants the inside of a
shared folder and withholds the folder itself, whose parent is the
owner's private tree.
Deleting sends the item to its owner's trash rather than the deleter's,
so it leaves the recipient's view without leaving the owner's account
and without changing hands. A move may not otherwise carry someone
else's entry out of their tree.
* fix(fs): give a new entry to the owner of the folder it lands in
A file a share recipient added to a shared folder was recorded as
theirs while living in the owner's tree, so a subtree could hold rows
belonging to several people — and the storage it consumed was checked
against the writer while being counted against the owner.
Take the owner from the parent row at every insert, charge the
allowance to that owner, and hand a moved entry over to the tree it
moves into. An entry now always belongs to whoever owns the directory
holding it.
* feat(fs): address shared entries as ~/share/<uid>
A recipient could read the owner's whole path off any shared entry —
where they keep the file and what sits beside it, neither of which the
share is about.
Give shares their own namespace. `~/share/<entry-uid>/rel/path` resolves
to the real path on the way in, and outgoing paths are rewritten to it
on the way out. Entries the actor owns pass through untouched, so no
existing client contract moves.
* revert(fs): mask only the directory bar, not the addressing
|
||
|
Daniel Salazar
|
a64a444a17 | feat: batch delete for flush + metering fixes (#3593) | ||
|
Daniel Salazar
|
fb7968a1c7 | fix: metering hardening; handle burst of unfinished ai requests (#3585) | ||
|
Daniel Salazar
|
f15d835eeb | fix: restrict openai and anthropic compatible endpoints to be subscription (#3583) | ||
|
Daniel SalazarandClaude Fable 5
|
d641f6f7a1 |
tests: count only this test's dynamo reads in the block-window assertion (#3576)
The spy sits on the test server's shared dynamo client, so a raw call count also picks up background work and the async tail of earlier tests in the file — which made the block-window test flakily report a third read. Each test runs in its own random namespace, so filtering the spy's calls to that namespace removes the cross-talk without loosening the assertion. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |