Commit Graph
257 Commits
Author SHA1 Message Date
Daniel Salazar 179ad990d6 feat(events): simplify onPersistent delivery (#4395)
* feat(events): simplify onPersistent delivery

Replace delivery/targets with a boolean single. Without single, every
event runs the app's published handler in its events worker and is also
sent to every connected client; the server no longer routes between the
two. single keeps leasing each event to one consumer that acks.

delivery: 'single' | 'broadcast' is still accepted and targets is
ignored. Stored rows are read by their delivery column alone.

* docs(events): compare onLocal, onPersistent and single

* feat(events): rename single to queue

onPersistent({ queue: true }) is the work-queue mode: each event goes to
one consumer at a time and is offered again until acknowledged.
delivery: 'single' is still accepted as an alias, and subscription
objects report queue instead of single.

The docs explain the three modes (onLocal, default onPersistent, queue)
and each events recipe says which one it uses and why.

* docs(events): retries are the only repeat for a queued event

* feat(events): queue deliveries fall through to the worker on decline

A queued delivery gets one socket attempt under a 15 s lease instead of
two 60 s ones. A client with no handler for it, or whose handler fails,
sends events.nack and the worker takes it without waiting for the lease.
Declines are relayed to the region holding the lease like acks are, and
a decline for a lease that is no longer the client's is ignored.

* feat(events): drop thrown errors, retry only temporary ones

A queued handler that throws now drops the event and counts one failure
toward the pause. Network errors, timeouts, rate limits, 5xx responses
and errors thrown with retry: true are retried with the existing
backoff. One classifier is shared by puter.js and the events worker
runtime, which inlines it at build time. A dropped event is billed once,
like a completed one. ack() remains as an optional early release.

* docs(events): queued events reach every tab with a handler
2026-10-28 12:58:51 -07:00
Daniel Salazar e1a36215f6 fix(sharing): preserve team grants and scope link invalidation (#4371) 2026-10-26 22:06:16 -07:00
Daniel Salazar 09da68424e fix(fs): preserve stored path spelling and app ACL identity (#4368) 2026-10-26 22:04:36 -07:00
Daniel Salazar b7a0276337 fix(fs): preserve paging and file read-copy contracts (#4367) 2026-10-26 22:03:09 -07:00
Daniel Salazar bbc056cc49 fix(hosting): track trashed roots and cached backing misses (#4363) 2026-10-26 21:58:05 -07:00
Daniel Salazar e2f6c2ed05 fix(cache): widen tombstone window to 5s (#4381) 2026-10-26 21:42:23 -07:00
Daniel Salazar 745b3a4318 perf(auth): coalesce session activity writes across processes (#4360) 2026-10-26 21:37:54 -07:00
Daniel Salazar 2dfd93687f perf(database): defer primary fallback for user lookups (#4359) 2026-10-26 21:37:06 -07:00
Daniel Salazar 91554a137e fix(apps): keep app provenance and permission caches consistent (#4357) 2026-10-26 21:28:40 -07:00
Daniel Salazar 52da7b571c perf(permission): read a large holder's user-to-user grants per permission (#4349)
A user-to-user grant check loaded the holder's whole row set, from one
Redis blob or one SELECT, and filtered it in JS. Every KV share handle a
user receives adds a row, so the cost of each check grew with incoming
shares.

Holders with up to 200 grants keep the whole-set cache. Past that the
cache holds a marker and each check reads only the permissions it names,
off a new (holder_user_id, permission) index. Matches stay exact in JS,
since MySQL compares `permission` case-insensitively. The batched
multi-holder read shares the same chunked query.

Migrations: mysql_mig_51, postgres_mig_40, sqlite 0097.
2026-10-26 20:51:46 -07:00
Daniel Salazar f850155411 KV share handles: hardening, higher limits, batch share, uninstall data deletion, account-deletion cleanup (#4344)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
* fix(kv): sign share handle ids and drop retired handles from every node's cache

New handle ids carry a short HMAC (kvh-<uuid>.<mac>); an id whose mac
does not verify is refused before any lookup. Unsigned ids keep working
but no longer fall back to the primary on a replica miss.

Retiring or changing a handle now broadcasts outer.pubsub.kv.share.forget
so every node drops its cached row. Event subscribes and delivery
rechecks refuse a handle whose owner is suspended, as the driver does.

* fix(kv): an app its user removed stops reaching shares

An app acting for a user exercises a share grant only while the user
still has the app authorized: at least one user-to-app permission row,
which every app token path leaves and revoke-user-app '*' removes. The
share itself stays; authorizing the app again restores access. The
answer rides the permission cache, keyed by the app-under-user
generation the revoke bumps, so live subscriptions go quiet too.

* feat(kv): list keys through a watch-level share handle

list with as: 'keys' through a handle needs only watch, which already
learns every key written in the region. Values and entries still need
read.

* feat(kv): filter share listings by app and by state

GET /kv/shares and GET /kv/shares/with-me take an optional appUid: an
app uid, or null for the app-less store. An account session may name
any app; an app is refused any namespace but its own. GET /kv/shares
also takes state=live|revoked|all, all by default.

* feat(puter.js): appUid and state options on the KV share listings

listShared() and listSharedByMe() pass appUid (null for the app-less
store), and listSharedByMe() passes state. Docs cover the new options
and that a watch share can list keys without their values.

* feat(kv): indexes and store counts for share-handle quotas and retention

Adds covering indexes for the owner's live count, the owner-holder pair,
app-filtered listings and retired rows, plus store methods to count a
pair's live handles and delete rows retired before a cutoff. The live
handle cache holds up to 50,000 entries; the miss cache stays at 10,000.

* feat(kv): higher share-handle caps, a per-holder cap, and sharing with several recipients

Live share handles per account go to 25,000 paid / 5,000 free, and per
app to 10,000 / 2,000. One owner may hold out 256 live handles to one
holder on every plan, which bounds the rows a revoke or unshare walks.
Re-sharing an existing region takes no slot and is never refused.

POST /kv/shares takes recipients (1 to 50) as an alternative to
recipient: one region, level and label for all. Caps and a new
per-recipient grant budget (300/min, 2,000/hour) are checked for the
whole batch up front; failures about one recipient are reported on its
item. The single-recipient request and response are unchanged and run
through the same code path.

* feat(kv): delete share handles 30 days after they stop

A daily job deletes retired share-handle rows older than 30 days, in
batches of 500. One node runs it per day; it stops on shutdown.

* feat(puter.js): share a KV prefix with several users in one call

puter.kv.share() takes an array of recipients, positionally or as
recipients in the options form, and resolves to one item per recipient:
the share, or an error with its message and code. The single-recipient
forms are unchanged.

* feat(kv): delete an app's data on uninstall, and close account-deletion gaps

Uninstall:
- POST /auth/revoke-user-app takes an optional `deleteData` boolean, only
  with `permission: '*'` (400 otherwise). AuthService.uninstallApp revokes
  every grant, deletes the app's worker sessions, then awaits
  `app.uninstalled` { userId, userUuid, appUid, deleteData }.
- With deleteData, KvShareService retires the handles the user shares out
  of that app's namespace (as the platform, in batches) and then empties
  the namespace in the background, 5,000 items per step, stopping on
  shutdown. Handles shared with the user are not touched.
- SystemKVStore.purgeNamespace(namespace, { maxItems }) deletes up to
  maxItems items of one `v1:<uuid>:<appUid>` namespace, reading keys only,
  with no mutation events or usage, and reports `done` once it is empty.
  It refuses the system namespace and anything else. flush shares its
  page-delete path.
- puter.perms.revokeApp(appUid, '*', { deleteData: true }); the desktop's
  uninstall dialog has an unchecked "Also delete this app's data" option.

Account deletion:
- EventsService settles each of a deleted account's apps on `user.delete`
  (they cascade without `app.changed`) and drops the owner's subscription
  keys.
- The flat permission entries the account held are purged in pages.

Suspension:
- `user.suspended` / `user.unsuspended` events; the share-handle store
  drops cached rows owned by that user here and on other nodes.

* fix(permission): await a peer's flat-permission deletes so emitAndWait sees them applied

* fix(kv): strongly consistent purge pages, and an uninstall purge stops once the app is authorized again

A stale empty page could end a purge as done while items remained. The
background purge after an uninstall now checks before each step that the
user hasn't authorized the app again, so data written after that is kept.

* fix(kv): flush keeps eventually consistent page reads; only a purge reads strongly consistent

* fix(user): emit user.suspended and user.unsuspended when the user store changes an account's suspension
2026-10-26 09:43:55 -07:00
Daniel Salazar 589a98b12c feat(kv): share KV prefixes with other users at watch/read/write, and bound what apps can be granted (#4331)
* feat(kv): share key-value prefixes with other users, and bound what apps can be granted

- KV share handles move out of Events into KvShareService, served at /kv/shares
  (the /events/kv-handles routes stay as an alias).
- Shares have watch, read and write levels, encoded in the permission root
  (kv-share, kv-share-read, kv-share-write) with write implying read and watch.
- Every puter.kv method takes optConfig.handle; keys are relative to the shared
  prefix, cursors are sealed, and private entries stay hidden.
- New puter.kv.share, unshare, listShared and listSharedByMe; optional label;
  kv.share.received notice in the app's mailbox only; recipient blocks apply.
- puter.kv.set gains onlyIfAbsent.
- fs and kv events gain viaWorker, and kv events report self from the writer.
- /auth/grant-user-app only accepts canonical permissions from a fixed set of
  families; the consent dialog, puter.perms and docs follow.
- Migrations: kv_share_handles gains revoked_by, label, and grantee and app
  indexes (mysql_mig_49, postgres_mig_38, sqlite 0095).

* fix(kv): never name a private key in a cross-app or handle cursor, and bound flat-permission scans

- Listings of another namespace (cross-app or through a handle) resume after
  the last entry they returned, and their cursors are sealed, bound to the
  namespace or handle, and padded to one length. Old plain cross-app cursors
  are still read when they name the target namespace; other namespaces get 400.
- PermissionStore scans flat permissions a few roots at a time and skips
  roots nested under another root in the same call.
2026-10-25 08:16:19 -07:00
Daniel Salazar 8b8dd19eae fix(teams): refuse to boot when teams_enabled has nothing registering Teams (#4300)
* fix(teams): refuse to boot when teams_enabled has nothing registering Teams

Core's seat checks (team 2FA, seat password reset, email and username
changes, self-delete, the signup placeholder claim) read the team store and
skip when it is absent. With `teams_enabled` on and no extension registering
the store and service, the server now fails at boot instead of serving
without those checks.

- whoami no longer string-matches the extension proxy's "missing property"
  error: with the flag on, the store is always there.
- Tests that turned the flag on without Teams drop it; the puter.js harness
  sets it only under the `teams` capability.
- Drop a stale services/index.ts comment.

* refactor(gui): drop the Teams console styles, which ship with the console now
2026-10-22 15:11:36 -07:00
Juan Fernando Castro 1627650abc refactor: make teams an optional feature (#4248)
release-please / release-please (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
* refactor: let core run without a teams registration

Teams becomes optional: `services.team` and `stores.team` may be absent, and
core stops assuming either is there.

Seventeen call sites across share, signup, account deletion and the homepage
now degrade instead of throwing: a team recipient that cannot be resolved is
"no such recipient" rather than a fault, a group share folds in nothing, and
account deletion stops asking what a seat was charged for. This generalises
the pattern `gates.ts` and `SocketService` already used.

Teams' own files take a private handle to their store and service, so the
feature's code stays free of optionality that only core needs.

No behaviour changes where Teams is registered.

* refactor: put a contract between core and teams

`core/teams.ts` holds the shapes core reads and the calls it makes, and the
registries declare the pair as provided rather than built in. The service,
store and controller are registered by whatever runs the feature.

Typing them properly turned up eight AuthController call sites the previous
commit could not see: the controller reads its stores from the built-in
registry, so an absent `team` fell to an index signature and typed as
`unknown` instead of erroring. They now go through the same optional path as
the rest.

The feature's own tests move with it. The cases in core suites that build a
team to exercise a core rule still need relocating, so those suites are red
here.

* test: stub core's seat rules, rehome the rest with the feature

A core suite can no longer build a team, so the rules that refuse a seat get
`testFixtures/seatStub.ts` instead: it answers the contract, which is all
those rules read, and puts the registry back when the test ends.

Seventeen cases move that way. Four could not: group shares and a seat's 2FA
enrolment over HTTP need the real feature and the `/teams` route, so they go
with it, as does the whole of `GroupGrants.test.ts`.

Two whoami cases asserted a seat's team through `extension.import`, which
resolves the registry rather than the stores object. They now assert the
absent case, and the behaviour they covered is asserted against
`buildUserDetails`, which takes its stores as an argument.

Suite: 12252 passed, 38 skipped.

* refactor: take the teams console out of the built-in tabs

`TabTeams`, `teamsConsole` and `teamActionIcons` are no longer built in. The
dashboard already invites a tab through its `dashboard-will-open` event, so
whatever provides Teams contributes the console the same way.

Its 113 strings go with it and register into the shipped English dictionary
at load, so `i18n('teams_*')` resolves as it did. Only `en.js` ever carried
them, so no translation is lost.

Three modules stay, because things other than the console use them:
`teamBadge` (the dashboard shell), `orgSeat` (Home, Account, Security) and
`shareTeams` (the share picker). They already render nothing when the
account has no team. `teams_account_of` stays with `teamBadge` for the same
reason, and the `share_*` and `plan_team_*` strings stay with their callers.

The GUI builds clean with no console at all, and builds it in when the tab
is contributed.

* docs: seats past the owner's current limit get no allowance
2026-10-22 15:32:39 -04:00
Daniel Salazar 5b0205f2bf fix(hosting): apps publish with write again (#4291)
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Reverts the hosting part of #4290. manage stays a user-to-user grant;
an app with write on its user's directory can publish and keep serving it.
2026-10-20 17:36:36 -07:00
Daniel Salazar 3bce3b056e misc: hardening (#4290) 2026-10-20 17:12:13 -07:00
Daniel Salazar 41e4b582d3 fix(subdomains): emit subdomain.delete for sites deleted with an app (#4273)
Sites and workers tied to an app go with it by FK cascade, on app delete and
on deleting the app owner's account. Only caches were dropped for them, so
the subdomain.delete listeners (CDN purge, worker script teardown, worker
sessions, site cleanup) never ran.
2026-10-20 12:31:40 -07:00
Juan Fernando CastroandDaniel Salazar b54b76715a feat(fs): finish a home rename that stopped partway (#4255)
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
* feat(fs): finish a home rename that stopped partway

FSEntryStore.rebaseUserSubtree runs renameUserHome's owner-scoped
cascade on its own (same subtree matching, same rebased-cache
invalidation) and also drops the home row's cached copies at either
path. countUserSubtree counts the owner's rows under a prefix with that
same matching, from the primary. renameUserHome now uses the shared
cascade.

* fix(telemetry): resolve the tracer per span so a test's provider is the one used

withSpan held a ProxyTracer built at module load, which caches the first
provider it delegates to. Test files sharing a worker then sent spans to a
provider an earlier file had already shut down.

* fix(fs): drop cached copies of only the rows a home rebase moves

The owner-scoped rewrite read its invalidation set from everything under the new prefix, which for a finishing rebase is the whole live home; it now reads the rows it is about to move.

---------

Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>
2026-10-20 00:59:40 -07:00
e83e27f306 feat(ai): edit a source video with txt2vid (Gemini Omni, FLUX 3) (#4272)
* feat(ai): edit a source video with txt2vid on Gemini Omni

txt2vid takes an `inputVideo` option (sent as `input_video`): a URL, data
URI or Puter file path to an MP4, QuickTime or WebM video of up to 32 MiB.
Gemini Omni receives it inline next to the prompt, which describes the
edit or extension. Google enforces the 10-second limit.

Models declare `videoInput` in the catalog, and the driver rejects a
source video for any other model instead of letting the provider drop it.
The credit hold adds 16,000 input tokens for a source video, since
countTokens reports far fewer tokens than the reply bills.

loadFileInput takes an optional `timeoutMs` for web URLs, and cancels
the body of a failed response.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ai): edit a source video with FLUX 3 on Together AI

FLUX 3 takes `input_video` as Together's `media.source_video`. A web URL
goes through for Together to fetch; a data URI or Puter file is read with
the caller's access, capped at 32 MiB, sniffed as a video and sent inline.
Editing is estimated and billed at the 1080p rate of $0.54/s, since FLUX 3
jobs report no cost. Image inputs can't be combined with a source video.

Seedance 2.0 ignores `source_video`, treats `reference_videos` as a loose
reference rather than an edit, and Seedance 2.5 fails on reference videos,
so neither declares `videoInput`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(ai): name FLUX 3 as a video editor in the AI overview

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ai): match Hailuo 02 to Together's price and document estimate billing

Together quotes $0.56 for a Hailuo 02 clip; the catalog had $0.49.

Together's video jobs no longer report what they cost, so every Together
clip is billed at the catalog's highest published rate. The docs said the
charge was the amount Together reports; they now describe the estimate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: enforce video edit durations and propagate source fetch cancellation

* fix(ai): pass media inputs as Puter files and stream source videos

Source videos were read whole into the API process, base64-encoded and
sent twice to Gemini (countTokens and the interaction), which held about
128 MiB per in-flight edit for up to ten minutes.

- Video inputs are a URL, a Puter file path or a data URI of at most
  1 MiB. Together gets a Puter file as a 15-minute presigned S3 URL and
  fetches it itself; Gemini streams it into the request body with an
  exact Content-Length, so it's never held as one string.
- Gemini counts only the prompt and image for a source video, and both
  providers read the source only after the credit hold.
- txt2img and txt2vid image inputs accept Puter file paths, read within
  the caller's access right before the provider call. Existing URL,
  data URI and base64 inputs are unchanged.
- puter.js writes a Blob, File or data URI over 1 MiB to a temporary
  file under .tmp, sends its path, and deletes it once the call settles.
- FLUX 3 edits bill at the 720p editing rate, matching their 1280x704
  output.

* fix(ai): stage inputs after website sign-in, keep images working without a write

- puter.js chose the temporary file path before signing a website visitor
  in, so it fell outside the app's data directory and the upload was
  refused. It now signs in first, then picks the path.
- An image that can't be written (no access, storage full) is sent inline
  as before instead of failing the call. Videos still fail, since a
  large inline video isn't accepted.
- Telling raw JPEG base64 from a path decodes only the first 24
  characters instead of the whole input.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: 404oops <oops404@util.404oops.com>
Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>
2026-10-19 23:33:03 -07:00
Daniel Salazar 18b96d529c fix: misc scan fixes (#4270) 2026-10-19 12:32:13 -07:00
Daniel Salazar 9ff224e22f perf(backend): cut sequential round trips on auth, fs, driver, event and site paths (#4259)
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Pipeline
- CORS preflights send Access-Control-Max-Age: browsers stop re-preflighting authed calls every 5 s.
- `req.ua` is parsed on first read instead of on every request.
- JSON `rawBody` reuses the parsed buffer instead of copying it.
- `emitAndWait` returns early when nothing listens: no span or closure per unlistened driver call.
- SQLBatcher sends a lone query on the next tick while fewer than 8 flushes are in flight, and
  waits `maxTimeInQueue` as before otherwise. Lone reads skip the 10/30 ms wait; batching under
  load is unchanged.

Auth and permissions
- Token-to-actor auth starts the user, session and app lookups together and checks them in the
  same order: fewer sequential Redis round trips per authed request.
- get-user-app-token and check-app reuse the resolved app row. getOrCreateApp skips the re-select
  after its own insert and doesn't wait on the cache write for replica rows.
- getOrCreateApp reads the race winner from the primary when its insert loses, so it no longer
  hands out a session that was never written.
- whoami runs the email check and the OIDC/taskbar/team lookups in parallel; key order unchanged.
- OIDC providers are memoized once every discovery succeeds, with one discovery fetch in flight
  per URL, and the Apple key is parsed once.
- gui-token and sync-cookie use `req.actor.user` instead of re-reading the user.
- listSessions batches app lookups. App icon probes run together, and the original-icon probe
  skips the lineage fallback.
- Default-granted permissions are answered from config before the scan cache, and exploded
  options are deduped: one less Redis read per /drivers/call.
- ACL resolves ancestors lazily (owners and full-access tokens skip them) and scans the node
  first, then the ancestors concurrently on a miss.

Filesystem
- Missing-path lineage lookups run 10 at a time: batch-upload path resolution drops from about
  1.3 s to about 40 ms per 100 new files.
- MySQL single-file inserts return the row built from the written values, falling back to the
  primary read: one less primary round trip per new file.
- Batch thumbnails are stored with one UPDATE per chunk (200 rows or 4M chars), with a per-row
  fallback.
- storage.quota.bonus listeners run concurrently next to the usage SUM, and the payload carries
  `userUuid`: quota checks and /df no longer serialize the listener reads.
- readdir cache backfill is one pipeline instead of 3xN SETEX.
- Profile reads skip the lineage fallback; the own profile is read alongside the user lookup.
- Legacy /stat runs is_empty (new `hasChildrenByParentUid`), owner and thumbnail concurrently.
- Downloads and the video proxy pipe through `pipeToResponse`: aborts release the storage stream
  at once with no extra 'close' listeners, and the video proxy no longer crashes on a mid-stream
  upstream error.
- mkdir looks up target and parent together.
- Upload completion reuses the session from the access check (status is still re-read before
  completing), and multipart signing reads it once.
- Moving a shared file to Trash skips two lookups that always returned the entry itself.

Drivers and events
- Subscription resolvers run concurrently and are folded in registration order.
- SubdomainDriver and WorkerDriver overlap independent lookups.
- The events "anyone subscribed?" check is one Redis pipeline when the region is warm.
- Handler publish overlaps the count with the size check.
- Broadcast flushes peers concurrently over a keep-alive agent, so a slow peer no longer delays
  cross-region invalidations to the others.

Sites, share and AI
- Hosted sites resolve the owner first, then the app and root together. Site config loads only
  on 404s. The hosted-app query selects only the columns it uses and moves into AppStore.
- shareFlags asks about the plan only when an item is shared by link only.
- Share listings run the count, teams and plan reads alongside their page query.
- Claude attachment cleanup no longer blocks the response.
- New util `awaitLater` for promises started early and awaited later.
2026-10-17 12:03:35 -07:00
Daniel Salazar 28f162ee92 fix(metering): fence settles, idempotent monthly claim, split addon consumption (PUT-2001) (#4236)
* fix(metering): pin metering writes to one replica, fence settles, idempotent monthly claim (PUT-2001)

store-kv-v1 is a last-writer-wins global table, and every region settled
its metering buffer into its local replica, so concurrent increments to the
same per-actor item from two regions could drop one side. Writes and
consistent reads of metering:* and policy:actor:* items now go to the
dynamo.home replica.

Buffered settles record each write on the claim with a fence token before
sending it, and the KV store refuses a token the item already holds, so a
re-driven claim applies its write once. Claims are serialized per counter
within a cache so one writer's tokens land in order.

The monthly charge claim is a set-if-absent of a random token, and a failed
claim write keeps its token for the retry, so a write that landed anyway
still charges exactly once.

* fix(metering): drop home-replica pinning; keep addon consumption off the purchase record (PUT-2001)

Pinning every metering write and exact read to one replica put a
cross-region round trip on credit checks near the allowance and on every
settle, and made one region's outage stall metering everywhere. The
per-item clash it guarded against is rare and undercharges at most a
flush window.

The one clash that loses money is a credit purchase and an overage settle
landing on the addons item in different regions at once. Consumption now
goes to its own item and getActorAddons sums it with what the purchase
record already holds.

* perf(metering): read addon consumption only for accounts with purchased credit (PUT-2001)

* perf(metering): settle counters through a sliding pool instead of waves (PUT-2001)

* feat(metering): consistent-read option on getActorAddons (PUT-2001)

Admin credit checks read the purchase record directly to get a consistent
read, which misses consumption now that it lives on its own item. They can
call getActorAddons with consistentRead instead. Also formats the lines this
PR added.
2026-10-16 23:49:23 -07:00
Daniel Salazar 2b254b8892 fix: misc bugs (#4251)
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
2026-10-15 23:49:17 -07:00
Juan Fernando Castro e32bfa839d fix: count one app open per account per app per hour (#4152)
* fix: count one app open per account per app per day

`app_opens` is read two ways at once: `open_count` is a row count and
`user_count` is distinct accounts. A repeated open moves the first
without the first moving the second, and the incentive report prices
opens.

An open is now marked as it is recorded and skipped for the rest of the
day, before both the row and the event ClickHouse ingests, so the two
stores agree. The marker is set only when absent, so the first open of
the day wins a race; an unreachable marker counts the open, since
counting twice beats not counting.

* fix: count one app open per account per app per day, at the count

Dropped the row-suppression approach. `app_opens` is not only a counter:
`getRecentAppOpens` orders the desktop's recent list by `MAX(_id)`, and
`app.opened` is the socket event other tabs reorder on. Suppressing the
repeat row froze both at the first open of the day.

Every open is recorded as before. `open_count` counts each account once
per day instead, which is the figure `user_count` is read against.

`countDistinctUserDays` folds the pair into one number because
`COUNT(DISTINCT a, b)` is MySQL-only, and `dayBucket` covers MySQL's
decimal division. ClickHouse has uniqExact over the tuple.

Note the figure's meaning changes at deploy: a period spanning it mixes
rows with account-days, so a comparison across that point is not one.

* fix: count one app open per account per app per hour

Review feedback: a day is too long a window for the repeat to collapse
into. The period is now an argument rather than a baked-in 86400, held in
one constant on AppStore beside the other store policy numbers, so the SQL
and ClickHouse sites cannot drift and a future change is one line.

The fold `COUNT(DISTINCT user * N + bucket)` is only injective while the
bucket index stays under N. Hourly buckets are 24x the day index and pass
the old 1000000 in 2084, so N moves to 10000000.

Buckets are clock hours, not a rolling window, so two opens a minute apart
either side of the hour still count twice — tested, and the same property
the day version had at UTC midnight.

The detailed-stats fixture gave its repeat open a 60s offset, which under
hourly buckets would have counted twice whenever the suite ran in the first
minute of an hour. It now shares a timestamp with the open it repeats.
2026-10-14 11:54:17 -04:00
Daniel Salazar ff815f730d fix: harden token delivery, session revokes, app identity and fs scoping (#4229)
release-please / release-please (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
- relay sign-in popups take the opener origin from the referrer only
- session revokes read the primary, batch per user, and tombstone caches in every region
- wisp relay tokens bound to the minting session; verify rechecks the account
- hosted origins keep app identity with the site owner; deleted app uids recorded in deleted_apps
- subdomain cache tombstoned on delete, update, reclaim and app/account cascades
- email lookups require an exact address match after the SQL lookup
- workers deploy only from files the deployer owns
- share_app_data:false enforced at ACL time, plus copy/move/rename/share/recursive listing
- kv remove returns no value without read; kv:expire op grant usable
- fs errors name the entry instead of the owner's path
- shared-by-me drops rows a delegate no longer has authority over
- private apps can't move index_url onto another user's origin
- GUI: default apps saved and launched by uid; save picker accepts only copy/move; permission dialog waits for focus; Publish as Worker hidden on others' files
2026-10-14 06:56:44 -07:00
Daniel Salazar f6a1e54642 fix(fs): keep S3 read stream errors from crashing the process (#4227)
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
getObjectStream now attaches an 'error' listener to every body it returns.
A body that fails before its consumer attaches (socket timeout, reset, or a
checksum mismatch on a truncated read) used to emit 'error' with no listener
and exit the process, failing every in-flight request on the task. It now
logs a warning; consumers that attach later still receive the error.
2026-10-13 17:42:28 -07:00
Neal Shah 62270ff9aa Delete worker sessions when workers are deleted, don't treat sessions from deleted workers as still valid. (#4163) 2026-10-13 12:43:45 -04:00
Daniel Salazar b52fc01712 perf: cut godmode app launch latency (cached token renewal, launch reuse) (#4221)
* perf(auth): renew godmode app tokens from cache instead of the primary

Every godmode launch read its token row from the primary and rewrote its
expiry, a few cross-region round trips per launch far from the primary.

- A cache pointer names the row per (user, web session, app). The row is
  read through the uuid cache token auth uses and re-checked (owner,
  session, app, revoked, expired, max age); anything else falls back to the
  primary read. Revoking the row drops the pointer.
- The expiry write is skipped when it would move by under an hour. A
  renewal that hits a revoked or lapsed row mints a new one instead.
- The response carries the row's actual expires_at.

* perf(gui): reuse a just-minted godmode token for repeat launches

A launch of a godmode app reuses the token this page minted for it within
the last ten minutes under the same desktop session, instead of asking
again. The cache is dropped when an app reports its token dead, after a
session revoke from the session manager or an uninstall, and across tabs
via a BroadcastChannel that carries token row ids, never tokens. A mint in
flight during a drop is not remembered.
2026-10-13 08:53:55 -07:00
404oops f9cde23937 Merge branch 'filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so' into filipkujundzic/put-1907-ai-streaming-buffering-and-error-shape-cleanup
# Conflicts:
#	src/backend/drivers/ai-chat/utils/inlineImages.test.ts
#	src/backend/drivers/ai-chat/utils/inlineImages.ts
#	src/backend/drivers/util/fileInput.ts
2026-10-12 12:39:35 +02:00
Daniel Salazar d8f9aff3dd fix: follow-ups to #4161–#4203 (rate limits, broadcast, AI routing and billing, fs subtree walks) (#4205)
* fix(rate-limit): count the newest KV rate rows first

The KV backend's rate check, peek and concurrent acquire list the key's
rows with a limit + 1 cap that the store applies before dropping expired
rows, so unswept rows from past windows (oldest keys) could fill the fetch
and the live count read as zero. They now list newest first.

* fix(broadcast): split webhook flushes under a byte budget, per-peer backlogs

Flushes go out in bodies of at most `broadcast.outboundMaxBodyBytes` (default
4 MiB) instead of one POST, and each peer keeps its own backlog of what it
missed, capped by count and four bodies' worth of bytes, oldest dropped first,
so one failing peer can no longer grow every peer's body past their 50 MB limit.

* fix(ai-chat): classify dropped upstream connections as provider failures

A connection reset or refusal (SDK APIConnectionError, Mistral's
ConnectionError, an undici reset on the cause) carries no status, so a chain
of them fell through to 500 internal_error. Attempts now record
connectionFailed; a chain of only resets and 5xx answers 400
upstream_provider_unavailable, and a reset counts as an upstream signal in a
mixed chain.

* fix(ai-chat): keep SDK retries on Anthropic count_tokens

The Claude client is built with SDK retries off so the driver can fall back,
but count_tokens has no fallback route. It now passes sdkRetryOptions, like
the OpenAI moderation call, so a transient 429 or reset is retried.

* fix(ai-chat): mark a route unhealthy for 60 s after a transient failure

With SDK retries off, one 5xx, 429, timeout or dropped connection kept a
route marked for 10 minutes, sending that model's traffic to resellers. Those
failures now mark the route for 60 s; bad credentials, an exhausted account
and other route-level failures keep 10 minutes.

* fix(metering): don't memoise spent addons inside the force-exact window

The spent-addons memo could cache a pre-grant record read from a lagging
replica right after a credits-changed announcement, so overage skipped the
new credit and requireCredits kept answering 402 for up to 15 s. A spent
record is now only cached once the actor's force-exact window has ended.

* fix(share): send the digest entries a flush claimed when another take fails

Digest entries are taken concurrently, and a single rejected take threw
away every entry the other takes had already removed from KV. The flush
now logs the failed takes and sends what it claimed; an entry whose take
failed stays queued for a later flush.

* fix(fs): announce an overwriting move's replaced entry only when it took its name

/move and v2 /fs/move looked up the entry at the destination name before
moving and always announced its removal. A move into someone else's Trash
renames instead of replacing, so the owner's desktop hid a live entry and
legacy /move reported it as `overwritten`. Both routes now pass the target
on only when the moved entry kept the name it was looked up by.

* fix(ai): name the vendor in upstream error fields from media providers

upstreamFetch now takes the provider id plus a display label, and
UpstreamResponseError carries the id. translateProviderError copies a
string `provider` off the error into `fields`, and the BytePlus video and
Cloudflare image errors set one, so upstream_auth_failed and
upstream_credits_exhausted alarms say which vendor failed again.

* fix(teams): delete a provisioned seat whose home folders could not be created

provisionUser logs a failed default-folder creation and still returns the
account, so provisionAccount went on to add the membership, issue a password,
mail the seat and emit team.account.created for an account with no home. It
now cascade-deletes the new user and returns 500 internal_error before any of
that.

* fix(ai-chat): load model catalogs without holding up server start

onServerStart awaited every gateway catalog (up to its 10s fetch timeout)
while HTTP was already listening, so drivers started after it had no
providers yet and chat requests found an empty model map. The first build
now runs in the background; complete(), models() and token counting wait
on it instead of failing.

* fix(ai-chat): serve direct models while gateway catalogs load at boot

The first model-map build now publishes a fresh map as each provider's
catalog lands, so static catalogs are routable at once. complete() and
token counting wait for the full first build only when the model, or the
provider asked for, isn't in the map yet. Refreshes still swap once.

* fix(ai-chat): split gateway-billed cost across input and output

A completion billed at the cost OpenRouter, Infron or Neuralwatt reports
was quoted to `ai.prompt.cost-calculated` as all input and no output. The
ledger still records one `billedUsage` line; the reported per-key costs now
spread that bill over the usage keys at the catalog's relative rates, in
whole µ¢ that sum to it. Neuralwatt energy units take no share.

* fix(ai-image): hold Replicate credits for the whole prediction

The output and input-surcharge holds took the default 10-minute TTL, the
same as the prediction window alone, so a slow prediction could outlive
them once creation and input measurement were added. They now last
15 minutes, like the video job holds.

* feat(ai-chat): report what a completion was charged as usage.cost

Chat usage (the non-stream result and the stream's final usage line) now
carries `cost`: the µ¢ the ledger records for the request after the cost
factor, from the request's recorded charges and the gate's factor lookup.
`usd_cents` is unchanged. A hooked recording now copies the usage object
first, so keys the driver adds later never land in the ledger.

* fix(ai): charge streamed speech once its first byte arrives

Per-character TTS and the voice changer recorded their charge when the
upstream's response headers arrived, so a body that failed before any audio
was billed in full. A shared relay now charges on the first byte, or when
the caller hangs up, and charges nothing for an upstream that fails or ends
before sending audio.

* fix(ai): release credit holds only after the request's usage lands

A hold was released as soon as its operation returned, while the usage it
stood for was still being written (behind the cost-factor lookup), so a
concurrent request briefly saw neither. AI metering now tracks the
request's usage writes and its holds (chat included) release once those
settle, without delaying the response; a streamed charge chains its own.
A failed write still releases the hold, and the hold's TTL backstops it.

* test(ai-chat): wait for a stream's credit hold to come back after its usage lands

* perf(fs): page subtree delete and copy by parent link, not by depth over the path range

Each page of a recursive delete or copy sorted the whole remaining subtree by a
computed depth, so a walk cost grew with the square of its size. It now pages
the children of up to 100 sibling directories at a time by parent_uid, depth
first, keyset on (parent_uid, id), so a page reads only its own rows; rows with
no stored path are reached too. SQLite gets the parent_uid index the others have.

* fix(fs): find children by parent_uid in the empty check and supplementary-name lookup

Rows the old backend wrote carry parent_uid and no parent_id, so a non-recursive
delete of a directory holding only such rows passed the empty check and orphaned
them, and on MySQL the supplementary-character child lookup missed them, leaving
their subtrees out of listings, counts and renames. Both now key on parent_uid;
the empty check still probes parent_id too, which the delete cascade follows.

* docs(fs): drop the single-POST note from the entry broadcast cap
2026-10-12 01:10:54 -07:00
Daniel Salazar b88aa55b24 fix: drop moved descendants from the entry cache; read cursor=null as the first page (#4204)
Renaming or moving a directory rewrote its descendants' paths but left their
cached copies (by id, uuid and old path) live for up to 60 s. A lookup of an
old path kept resolving, and a write beneath one landed at a path its parent
no longer had. updatePathPrefixForUser and renameUserHome now drop those keys,
here and, within the broadcast budget, in peer regions.

puter.js sent `cursor=null` for `listShared`/`listSharedByMe({ cursor: null })`,
which decoded to null and failed every keyset listing with a 500. decodeCursor
reads it as no cursor, and the SDK no longer sends it.
2026-10-12 00:19:11 -07:00
Daniel Salazar 21131ad3d6 fix(kv): spread upload sessions across system KV partitions (#4162)
Upload sessions shared the system namespace's single partition key. They
now go to one of 16 system partitions hashed from the session id, via a
new system-actor-only `systemPartition` KV option; partitions are never
cached and emit no mutation events. Reads fall back to the unpartitioned
key for sessions written before this (max session lifetime is one hour).
2026-10-11 22:14:11 -07:00
Daniel Salazar 38c986927c fix(fs): page subtree walks over supplementary-character names on MySQL (#4203)
listDescendantsInOrder, which drives paged recursive delete and copy, still
used a bare `path LIKE` range, so on MySQL a child whose name starts with a
supplementary character (and its subtree) was skipped: delete left it to the
parent_id cascade and copy dropped it. It now ORs the subtree patterns like the
other subtree queries, paging each pattern group with the same seek and
merging the pages in order.
2026-10-11 22:11:39 -07:00
Daniel Salazar 590c87c00a refactor(stores): one row cache and keyset page helper, batched cache round trips, dead store methods removed (#4191)
* refactor(stores): one read-through RowCache for UserStore and AppStore

UserStore and AppStore each carried the same multi-key row cache: key
builder, cache read, tombstone-guarded backfill, broadcast refresh,
invalidate and tombstone-on-delete. They now share RowCache on PuterStore,
which also single-flights concurrent misses for one key and batches
getByIds/getByUids: one pipelined tombstone check for the loaded rows and
one pipelined backfill instead of per-row awaits. Cache keys and TTLs are
unchanged.

* perf(apps): invalidate cached app lists by generation, not by tracker scan

Every app write SMEMBERS'd a global set of every cached list key, parsed
each key back into its filters and deleted the matches. List pages are now
keyed by their owner's generation token (or a shared one for unscoped
lists), and a write replaces the tokens it affects: O(1) per write, and a
page loaded before a write can no longer be cached after it.

* refactor(stores): one keyset page helper for store listings

Thirteen store listings each cut a LIMIT n+1 read to a page and sealed the
last row's position, and each opened an id cursor its own way. They now use
keysetPage() and openIdCursor() from util/pagination. Cursor payloads and
sealing are unchanged; a cursor without a usable id still fails with 400 in
ShareStore and still restarts the listing elsewhere (strict option).

* refactor(stores): delete store methods nothing calls

AppStore.existsByIndexUrl, OIDCStore.unlinkByUserId,
TeamStore.listByOwner/listActiveSeatUuids, SubdomainStore.getByDomain/
listByDomain, FSEntryStore.ensureDirectoriesForUser,
ShareStore.listByIssuer/listByRecipientEmail/deleteByRecipientEmail/
getDailyShareCount/listByHolderAndFsentries and
PermissionStore.invalidateScanCache/deleteUserUserPermsByPermissionPrefix
had no production callers. ShareStore.listByFsentries was a copy of
listReaching; its callers use the latter.

SubdomainStore SMEMBERS'd a per-user prefix-list tracker on every write and
broadcast its deletion, but nothing ever wrote that set or cached a prefix
listing. The reads are gone.

* refactor(stores): shared escapeLike and bumpGeneration helpers

escapeLike replaces the LIKE escaping copied into PermissionStore (twice),
ShareStore and FSEntryStore, and now also guards SubdomainStore's prefix
filters, which passed caller prefixes into LIKE unescaped.

bumpGeneration sends INCR and EXPIRE in one pipeline instead of two awaited
commands in PermissionStore, EventSubscriptionStore and PresenceStore.

* refactor(stores): read affected rows off WriteResult

Every database client returns affectedRows and anyRowsAffected, so the
`affectedRows ?? changes ?? 0` fallbacks in ShareStore, NotificationStore,
UserStore and UserBlockStore read those directly.

* perf(stores): batch flat-permission KV deletes and durable cache drops

PermissionStore deleted each flat KV entry with its own kv.del; one
kv.batchDel now covers the set. DurableSubscriptionStore's suspend and reap
dropped each row from the cache separately; EventSubscriptionStore gains
dropDurables, which drops them per owner in one pass, and dropDurable calls
it. suspend keeps one conditional UPDATE per row, which is how concurrent
settles learn which rows each of them suspended.

* refactor(fs): fold FSEntryStore's duplicate getters, sort maps and cache keys

- getEntryByUuid and getEntryById share one by-column getter.
- listChildren and listChildrenPage read one sort table; the offset listing
  now coalesces NULL modified/size like the cursor listing, so both order
  NULLs the same way on every engine.
- Entry cache keys come from one builder instead of hand-built strings.
- getEntriesByIds, getEntriesByPaths and the per-user path reader fetch
  their cache hits in one pipelined round trip instead of a GET per key,
  through readCacheValues, which RowCache.readMany now uses too.

* fix(apps): cache the row a lost createFromOrigin race reads from the primary

Keeps AppStore's private #writeCache as a RowCache write, so code that
still calls it (#4166's read-back) loads in either merge order.
2026-10-11 22:07:53 -07:00
Daniel Salazar 45f6e9d8ee refactor(drivers-wiring): AppService views and launch gates, batched app lookups, shared driver policy and guards (#4186)
* perf(workers): skip hot-reload lookups for files no worker is bound to

The fs.write.file / fs.remove.node / fs.move.node listeners listed every
workers.puter.* row of the file's owner on every event, for every user.
The event's FSEntry already carries the worker rows bound to it, so a file
with none returns before any I/O; a bound file reads just its own rows,
batched, from the primary.

* perf(apps): read the uid alias only after a direct lookup misses

Every AppDriver uid lookup fired an uncached system-KV GetItem for the
merge alias in parallel with the direct read, even when the direct read
hit. The alias is now read only on a miss.

* refactor(apps): one app view with the launch gates, built by AppService

AppDriver's serializer and its private-app / hosted-backing gates move to
AppService.views, built on a pure toAppView that is the one place
index_url is withheld. AppDriver, the subdomains driver's embedded apps and
the GUI shell view all use it. Visibility of protected apps
(filterReadable) moves with it.

Subdomain embeds (associated_app, app_owner) used to return index_url with
no gate; they now withhold it from a viewer who may not open a private app
or whose hosted backing is gone, and carry privateAccess, godmode and
feedback_enabled like a direct app read.

* refactor(apps): one toAppSummary; batch the recommended and taskbar lookups

SuggestedApps, RecommendedApps, /get-launch-apps and the taskbar each built
their own launch summary; they now share toAppSummary (the extra
feedback_enabled / external flags appear on the lists that lacked them).

getRecommendedApps awaited getByName once per recommended app (29) and
getTaskbarItems did the same per taskbar entry on every whoami; both now
make one batched call per key through the new AppStore.getByNames, which
keeps getByName's case-insensitive and old-name matching. The taskbar's
raw UPDATE of the user row goes through UserStore.update, and
/get-launch-apps checks hosted backings in one batch and reaches the
stores without casts.

* refactor(apps): app routes read through AppService; /delete-site calls the subdomains driver

AppController used the global driversContainers through a getter whose
boot-order comment no longer held; it uses the injected drivers and
AppService. GET /apps/:name looked each name up, read it again through the
driver and resolved private access a second time, emitting
app.privateAccess.resolveLaunch twice per private app; it now batches the
names and resolves once. POST /query/app did a lookup, a driver read and a
raw app_filetype_association SELECT per selector (up to 200, serially); it
now makes one batched lookup per key, one grant check and one filetype read.

The driver's read shape is a pure toAppReadView both use.
HostingController's /delete-site re-implemented the subdomains driver's
delete (protected check, delete, subdomain.delete event); it keeps its
owner-scoped 404 and calls the driver. Hosting and Desktop controllers drop
constructors that discarded the drivers argument, and trivial getters.

* refactor(drivers): one per-method policy validator and resolver; drop @Driver

Four policy blocks (rateLimit, concurrent, requireSubscription,
requireReputation) share the { default, methods } shape but each had its
own validator and resolver. They become validatePerMethod(value, label,
policy) and resolvePerMethod(cfg, method); error messages are unchanged.

The @Driver class decorator and the prototype keys it wrote had no users
outside its own tests; drivers declare their fields, and resolveDriverMeta
reads only those. DriverController takes the driver name from the meta it
already resolved instead of probing the prototype.
resolveDriverMethodRateLimit/Concurrent stay as deprecated aliases for the
email extension.

* refactor(drivers): shared actor guards and crud-q concurrency policy

The apps, subdomains, workers and notifications drivers each carried a
private #requireActor, two carried the same #requireVerified, and apps and
subdomains the same owner/app_owner write predicate. These become
requireContextActor, requireContextUserActor, assertActorEmailVerified and
actorOwnsRow in core/actor.ts; each driver keeps its own error for the
cases that differed (system actors still read apps; notifications still
answer 403 for an actor without a user). The identical crud-q concurrency
block is one CRUD_CONCURRENT constant.

The metering, installedApps and appTelemetry extensions re-checked an actor
their routes already require; those checks and the tests for them go.

* refactor(extensions): store methods instead of raw SQL in OSS extensions

installedApps and appTelemetry queried user_to_app_permissions directly
and metering read apps by name with its own SELECT. PermissionStore gains
listAppsGrantedByUser, listAppAuthenticatedUsers (with the email-share
check folded in) and countAppUsers; metering uses AppStore.getByName.

* refactor(server): one layer builder and one lifecycle hook runner

PuterServer built each of its five layers with its own copy of the same
loop (built-ins, then extension additions, into the layer and its export
container), and ran each lifecycle hook with five near-identical loops per
hook. Both are now one helper each: #instantiateLayer and
#runHook(order, hook). Construction order, the args each layer receives,
and hook order (start and prepare bottom-up, shutdown top-down) are
unchanged.

* fix(workers): tear down a deleted source's worker after its FK nulls root_dir_id
2026-10-11 22:04:23 -07:00
Daniel Salazar 576e893a5b refactor(services-core): read-only socket reauth, one token-grant read per check, shared grant/revoke and hosted-asset paths (#4187)
* perf(socket): re-check live sockets without touching their sessions

The 5-minute sweep reused the request-path authenticate(), which slid
every connected session's expiry and wrote user activity each tick, and
could fill SessionStore's touch-throttle map. authenticate() now takes
touch: false for re-checks; the sweep uses it, checks credentials
through runWithConcurrencyLimitSettled, and a call while a sweep runs
joins it instead of starting another.

* perf(acl): read a scoped token's grants once per check

ACLService.check did a Redis GET + JSON.parse of the token's grant list
for every ancestor x mode, and the permission scan did the same per
option. Both now read the list once (PermissionStore.listAccessTokenPerms,
which replaces the per-permission hasAccessTokenPerm) and test it in
memory.

* refactor(metering): incrementUsage goes through batchIncrementUsages

incrementUsage was a copy of the batch path that let an infinite amount
through and skipped NaN, where the batch path counts either as one unit.
It now calls batchIncrementUsages with a one-entry batch, so both apply
the batch clamping (documented on batchIncrementUsages). The overuse
alarm loses the single-path fields nothing sets any more.

* perf(metering): memoize an actor's spent addons on the increment path

Every increment and credit refresh read the actor's addons record from
the KV store. A record with no purchased credit left can only change
through a grant, and every grant announces credits-changed, so it is now
kept per actor beside the credit memo and dropped with it. Records with
credit left are still read fresh, since increments on other nodes draw
them down without announcing it. A read that straddles a drop isn't
kept.

* refactor(services): route raw table writes through their stores

- access_token_permissions: AuthService inserted grants one row at a
  time through a cast to the store's db client, read the authorizer, and
  deleted the manifest with raw SQL, then asked PermissionStore to drop
  its cache. PermissionStore now owns insertAccessTokenPerms (chunked
  multi-row insert), deleteAccessTokenPerms and
  getAccessTokenAuthorizerId, and invalidates inside the writes.
- TeamService's 2FA reset clears the OTP columns through UserStore.update.
- AppIconService's data-URL icon swap is AppStore.replaceDataUrlIcon.

* fix(user): correct cascadeDelete's FK note; seat cleanup is best effort

The header said most user_id foreign keys are ON DELETE SET NULL; the
migrations cascade nearly all of them and keep SET NULL for audit and
analytics history. forgetSeat ran unguarded after the row was already
gone, so a cache failure there failed the whole delete; it is now
logged and stepped over like the other teardown steps.

* refactor(acl): one grant/revoke path per holder kind

ACLService's user and group stat/set pairs were copies differing only in
which PermissionService methods they called. They now share #stat and
#set over a holder adapter ({ lockKey, query, grant, revoke }); lock keys
and errors are unchanged.

PermissionService's eleven grant/revoke methods repeated the same guard
blocks. Each keeps its own order of checks, now through shared helpers
(#requireUserId, #refuseAppActor, #assertCanManage, #requireUser,
#requireApp, #assertFitsColumn, #audit), so every refusal keeps its
status, legacyCode and wording. Audit writes stay fire-and-forget.

* refactor(auth): one hosted-asset token API keyed by kind

The private-asset and public hosted-actor cookies had separate create/
verify methods, four cookie-name getters and two identical options
getters, differing only in the kind claim and what the host field was
called. They are now createHostedAssetToken(kind, claims),
verifyHostedAssetToken(kind, token, expected),
getHostedAssetCookieOptions(), and the exported HOSTED_ASSET_COOKIES
names. Token claims, cookie names and every error message are unchanged;
puterSite and privateAppGate call the new methods.

* refactor(auth): one revoked/expired check for every actor builder

The session, app-under-user and access-token builders (and the access
token's parent-session check) each spelled out the same revoked/expired
tests. They now share #endedSession, which returns the same reauth
results; only a session token passes its auth_id. Adds coverage for an
access token whose parent session expires, is revoked, or is gone.
2026-10-11 21:55:20 -07:00
Daniel Salazar ecef154630 refactor(services): page fs subtree walks, gate share fan-out, one fs ACL helper (#4190)
* refactor(acl): read ancestor chains from FSEntryStore and share one fs ACL helper

getAncestorChain (split a path, batch-read the entries) moves to
FSEntryStore, with getAncestors for callers that want the rows. Services
read it from the store instead of calling FSService sideways;
FSService.getAncestorChain stays for drivers and extensions.

ACLService gains fsDescriptor (memoized chain), assertFsAccess,
assertFsCreate and refusal, plus the pure aclRefusal mapping. The eight
hand-rolled descriptor and safe-error copies in FSService, ShareService,
events/authorization, FSController, legacyFsHelpers, LegacyFSController,
AuthController and WebDAVController use them. Each route keeps its
status, legacyCode and message: fs routes answer access_denied on a 403,
the legacy routes still hide refusals from apps as 404, share and events
keep forbidden and their own 404 text.

* perf(fs): page recursive deletes and batch their event dispatch

A recursive delete read every descendant into memory, and an empty-check
for a non-recursive one did the same. It now walks the subtree a page at a
time in descending path order (children before the directory holding
them, so the parent_id cascade never takes a row before it is handled),
deleting objects and rows and publishing each page as it goes. A
non-recursive delete probes with one LIMIT 1 query.

Removals publish once per entry, after the row is gone (the same change
as the open fix for the double emit), with one surviving-ancestor chain
read at most once for the whole subtree. Events-API dispatch for a batch
runs as one background task at bounded concurrency instead of a
fire-and-forget promise per entry.

FSEntryStore gains hasDescendants and listDescendantsInOrder (keyset
paged by path, then id).

* perf(fs): copy directories a level at a time

A directory copy walked every descendant in memory and copied them one
by one: an insert per directory and, per file, an S3 CopyObject, a
batchCreateEntries of one and its read-back. It now pages the source in
ascending path order (parents before children) and takes each page a
depth at a time: rows for directories, links and empty files at bounded
concurrency, then the depth's S3 copies at bounded concurrency and one
batchCreateEntries for all of them. A single-file copy goes through the
same path.

A failed S3 copy still fails the call (a ghost file still answers 404),
but only after the objects that did copy have their rows, so nothing is
left in the bucket unreferenced. Copied entries publish per depth in one
dispatch.

* refactor(share): build masked share paths in one place

sharePathMask exports maskedRoot(owner, uuid, name) and
maskUnder(root, realPath). ShareService's maskedPathVia goes, its
maskedSelfPath and shareDeepLink's maskedSharePath build on them, and the
masker, maskUnderAnchor and learnShareRoots use maskedRoot. Output is
byte-for-byte the same; tests pin it.

* fix(share): send old_path on a recipient's item.moved

The desktop's item.moved handler reads old_path; ShareService sent the
former masked path only as from_path. Both are sent now.

* perf(share): skip the audience lookup for owners with nothing shared

Every fs write, create, rename, move and delete looked for share
recipients: an ancestor walk plus two share queries, for an owner who in
nearly every case has shared nothing. #sharesReaching now reads the
chain through FSEntryStore.getAncestors (no third copy of the walk) and
asks a per-owner "has anything shared" flag before the share queries.

The flag is the per-user cache events already keeps for subscriptions
(SubscriptionCache): one ShareStore.ownerHasShares read per owner per
TTL, coalesced across a burst. Creating a share (user, team, claimed
invite) or moving a subtree to a new owner bumps it locally and over
outer.pubsub.share.ownerShared; the TTL covers a process that misses
the bump. A failed lookup reads as shared, so the full lookup decides.

* perf(share): batch the per-row reads behind share listings

Share listings looked things up one row at a time:

- teams by id: TeamStore.getByIdsIncludingDeleted (the single-id read
  calls it) for listSharedByMe, listSharesOf and #revokeGroupSharesBy
- team shares per node and ancestor: ShareStore.listGroupOnFsentries,
  one query for the chain, in the same node-then-ancestor order
- #liveGroupGrants: PermissionStore.queryIssuerGroupPermsByPrefixes
  reads every issuer/group pair under every prefix in one query
- #revokeGroupSharesBy: the five canManagePermission checks per row run
  together instead of one after another
- ShareNotificationService #unblocked: users via getByIds and blocks via
  UserBlockStore.blockersAmong, two queries instead of two per member
- digest flush: entries are taken at bounded concurrency rather than one
  at a time

* perf(events): ask background consent once per holder and app on a revoke settle

Withdrawing events:background re-checked consent row by row, resolving the holder's actor and asking the permission service for every one of their background rows. Rows of one holder and app stand on one consent, so it is asked once and shared.

* refactor(events): drop the registry's unused push fields

notify, defaultDelivery, PushMessage, NotifyProjection and pushProjection were read only by registry.test. Every subject was unpushable and broadcast by default; the subscription's delivery class stays in DeliveryClass, which the stores use.

* refactor(share): retire trashed shares through onEntryOwnerChanged

onEntryTrashed was a copy of onEntryOwnerChanged: the same subtree share sweep, grant retire and row delete.

* fix(fs): page subtree walks by depth, not path

MySQL sorts a PAD SPACE string on its first max_sort_length (1024) bytes,
so under utf8mb4_unicode_ci paths past about 512 characters tie and fall
back to id. A file moved under a newer directory then sorted after it: the
delete walk reached the directory first and the parent_id cascade took the
file without its object delete or event, and the keyset skipped or
repeated rows. Pages now order by segment count, then id, which no
collation can reorder.

* fix(share): cache an owner as shared when the bump arrives

A bump only dropped the cached answer, so the next lookup could read a
replica that hadn't applied the new share yet and cache "nothing shared"
for another TTL. The bump now records "shared" outright.

* fix(fs): probe a directory for children by parent_id, from the primary

* fix(fs): keep the copied-size check in the batched directory copy

A copied file records the size CopyObject actually duplicated; growth past
the source row is charged against the allowance, and a copy that no longer
fits is deleted and refused, as a single-file copy already does.
2026-10-11 21:54:13 -07:00
Daniel Salazar b5eefaa9cc fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135) (#4198)
* fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135)

MySQL ends the idx_fsentries_path range for `path LIKE 'p/%'` at 'p/' +
U+FFFF, and utf8mb4_unicode_ci sorts supplementary characters above it, so
a child whose name starts with one (most emoji) was skipped with its whole
subtree by listing, delete, copy, rename/move rewrites, counts and sizes.
On MySQL, FSEntryStore now looks up those children by parent_id and ORs a
LIKE pattern for each onto the subtree match, recursing into directories.
Every extra pattern starts with the prefix, so only the scanned ranges
widen; rewrites past 500 patterns run in several statements.

The prefix rewrite also passed a UTF-16 length to SUBSTR, which counts
characters, so a rename under a path holding an emoji dropped characters
from every descendant path on all engines. It now counts code points.

* test(fs): seed the supplementary-name subtree with createNonFileEntry
2026-10-11 21:21:18 -07:00
Daniel Salazar 6a938bd8e8 fix(db): detect duplicate keys with isUniqueViolation everywhere (#4175)
FSService, OIDCStore, UserStore and TeamService each carried their own
duplicate-key check. FSService only matched mysql and the bare
SQLITE_CONSTRAINT code, so the mkdir/move race recovery never ran for a
SQLite or Postgres unique violation. OIDCStore matched any message
containing "UNIQUE" or the bare SQLITE_CONSTRAINT code, UserStore any
SQLITE_CONSTRAINT_* code, and TeamService any message mentioning unique or
duplicate. All four now use util/dbError's isUniqueViolation; UserStore
still checks the index name on top.
2026-10-11 18:24:13 -07:00
Daniel Salazar 847a4fc031 fix(fs): route thumbnail writes through FSEntryStore (#4174)
* fix(fs): route thumbnail writes through FSEntryStore

Add FSEntryStore.updateEntryThumbnailByUuid, which sets a thumbnail with
no owner check (callers authorize) and refreshes the entry cache, with an
optional expected-value guard.

The thumbnails extension's inline data: URL migration, its copy
duplication, and the legacy /update-fsentry-thumbnail endpoint now write
through it instead of raw UPDATEs, so cached entries stop serving the old
thumbnail. The migration validates the image like the upload path, awaits
the write, and only repoints a row that still holds the data URL; a read
that loses that race deletes its upload.

* fix(fs): report a thumbnail update by whether the row changed

updateEntryThumbnailByUuid returned the refreshed entry, which is null
when the cache misses and the replica hasn't seen the row yet. The
read-time migration took that null for a lost race and deleted the object
the row had just been pointed at. It now returns whether the UPDATE
changed a row; the cache refresh still runs after it.
2026-10-11 18:23:58 -07:00
Daniel Salazar 7ad963bacf fix(cache): broadcast fs entry and team invalidations to peer regions (#4169)
* fix(cache): broadcast fs entry and team invalidations to peer regions

FSEntryStore's invalidations called publishCacheKeys without broadcast,
so a delete, move, rename or overwrite only cleared the writer's
cluster; other regions served the old row until the 60 s TTL. They now
broadcast. Batch paths (deleteEntries per chunk, the batch overwrite)
send one event for the whole batch. Read-path backfills stay local.

TeamStore's busts were a local redis.del, so a removed member kept
passing getMembership in other regions until the TTL. They now go
through publishCacheKeys with broadcast, and softDelete busts every key
in one call.

Remove FSEntryCacheInvalidationEventHandler. It re-derived the same
invalidations from outer.gui.item.* with a replica SELECT per holder,
and its fs.remove.node branch never ran (FSEntry has no get()). The
store's own broadcast covers what it did.

* fix(fs): cap how many entry invalidations a node broadcasts per window

FSEntryStore now broadcasts at most 10,000 entries' cache invalidations
per broadcast flush window (outbound_flush_ms, 2 s by default). Past
that, keys are dropped locally only and peers fall back to the 60 s
entry-cache TTL; the store logs once per window when this happens.

Each flush goes to a peer as one POST that is re-sent whole on failure,
so a large recursive delete could otherwise build a batch the peer
rejects on every retry. The budget is per store instance, not per call,
so a delete issued page by page is bounded too, and single-entry writes
count against the same budget.
2026-10-11 17:53:26 -07:00
Daniel Salazar b67acdbf6f fix(stores): return written rows without reading them from a lagging replica (#4166)
* fix(stores): read rows back from the primary after writing them

ShareStore (create, upsertPending, upsertActive, upsertActiveGroup,
applyPending, upsertAnyone), AppStore (create, createFromOrigin, update,
claimOwnership), NotificationStore.create, SubdomainStore.update and
NotificationDriver.create returned the row they had just written by
reading it back through `db.read`, which goes to the replica. Under
replica lag that came back null or stale. Those read-backs now go to the
primary, through a `primary` option on the getters. AppStore caches the
primary copy on create, and claimOwnership no longer reads through
getById, which cached the replica's pre-claim row on a miss.

* fix(stores): skip the notify read-back; read OIDC link conflicts from the primary

NotificationStore gains `insert`, which writes and returns the uid;
`create` is now `insert` plus the primary read-back. NotificationService
.notify ignores the row, so it uses `insert` and no longer pays a primary
round trip per recipient. NotificationDriver.create still reads the row
back from the primary.

OIDCStore.link reads the conflicting row from the primary after a unique
violation, so a replica that has not seen another account's link can no
longer make the call report success.

* perf(stores): return written rows instead of reading them back

ShareStore.create, NotificationStore.create, AppStore.create and
AppStore.createFromOrigin build the row they return from the values they
inserted and the insert id, instead of reading it back from the primary.
The DB-defaulted creation timestamp is now written explicitly from unix
seconds (`timestampFromUnixSql`), so the returned value matches what a
read returns (`timestampValue`); postgres INSERTs add RETURNING id. The
AppStore rows fill every column the INSERT leaves to the schema and are
cached as before.

ShareStore.upsertPending's re-invite branch selects the full pending row
in its dedupe read and returns it with the new mode and data; it falls
back to the primary only when the UPDATE matched nothing.

The remaining primary read-backs stay and say why: the share upserts and
the new-invite insert (an upsert's surviving uid/created_at and the
entry-owner subquery are only known to the primary), applyPending and
claimOwnership (only an id or uid in hand), AppStore.update and
SubdomainStore.update (the base row is a replica copy that would be
cached), NotificationDriver.create (notify returns only the uid) and the
OIDC link conflict.
2026-10-11 17:51:38 -07:00
Daniel Salazar 491729e060 fix: backend test flakes and a socket shutdown hang (#4170)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
- SocketService's prepare-shutdown hook awaited io.close(), which also
  closes the http server and waits for every HTTP connection to end. A
  connection mid-request when close() runs stays open for the keep-alive
  timeout, so shutdown stalled before PuterServer severs connections. The
  hook no longer waits on the http close.
- Test servers disable Ollama unless a test configures it. Its boot probe
  logged after files had ended, which failed shared-worker shards with
  EnvironmentTeardownError.
- vitest ignores that teardown error for a console call still in flight.
- setupPuterTestEnv boots again on a fresh port after EADDRINUSE, and a
  failed setupTestServer start shuts the server down.
- UploadReservationStore expiry tests pin the clock instead of racing a
  50 ms TTL.
2026-10-10 19:34:32 -04:00
Daniel Salazar d4589caefe fix: session store revokes (#4168)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
2026-10-10 07:55:43 -07:00
Neal Shah 5fb025c109 Misc subdomain and worker fixes (#4157)
* PUT 1902

* LocalWorker fetch policy fixes for 127.0.0.1
2026-10-09 19:59:52 -04:00
Juan Fernando Castro 50bcdc9c6c fix: let a revocation be seen by the reads and the nodes that outlive it (#4151)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
* fix: let a revocation be seen by the reads and the nodes that outlive it

Four places where a revoke lands on the primary and the next reader can
still be told the old answer.

The u2a cache key was dropped but not re-filled, so the next miss read a
replica that may still carry the row and cached it for the TTL -- and a
revoke's own settling pass is usually that miss. It re-fills from the
primary, as the u2u side already does.

A revoked session dropped its keys the same way. It now leaves the
revoked row in the cache, read from the primary, so there is nothing for
a lagging replica to re-introduce.

A link share is its own authority rather than an index over a
permission, so a replica still carrying a withdrawn row answers with it.
For a short window after one is withdrawn, those reads go to the
primary.

Socket eviction rode a local event, so it only reached the node that did
the revoking. It is announced on the fan-out channel too, which is where
the node actually holding the connection hears it.

* refactor: carry the revocation on outer.* instead of outer.pubsub.*

Eviction runs through `io.in(room).disconnectSockets()`, which goes via the
socket.io Redis adapter a cluster shares, so one node handling the event
disconnects that user on every node in it. Fanning out to siblings had each
of them repeat the same cluster-wide disconnect.

The adapter's Redis is per-cluster, so peer clusters still need the hop --
`outer.*` reaches them and skips the sibling fan-out. The local
`auth.sessions.revoked` goes with it: `outer.*` fires on the emitting node
too, and nothing else subscribed.
2026-10-09 16:36:26 -04:00
Juan Fernando Castro ff35fbd66e Share attribution and cascade: finish PUT-1856 and PUT-1857 (#4119)
Replayed onto main after #4118 squash-merged. Same tree as the reviewed
head (8d8bd871f); only the redundant merges from the old base are gone.
2026-10-09 16:23:18 -04:00
Juan Fernando Castro 1ff5d93244 fix: a signed overwrite lands on the live key only once its completion is accepted (#4136)
* fix: a signed overwrite lands on the live key only once its completion is accepted

A presigned overwrite URL was signed against the live object key, so the
browser's PUT replaced the owner's bytes directly. That PUT never passes
through Puter, which leaves the access check on `completeWrite` with
nothing left to protect: by the time it runs, what /read serves has
already changed.

The upload now goes to a key of its own. `completeWrite` is already the
authorized step, so that is where the staged object is copied onto the
live key, server-side, and the staged one dropped. The entry keeps its
uuid, so everything written against it still holds. Aborting deletes the
staged object, which the old shared key made unsafe to do.

`copyObjectOfAnySize` carries the copy past the single-request ceiling
by copying part by part, so a large file is no different.

Two behaviours follow from the key no longer being shared: completing an
overwrite that uploaded nothing is refused rather than reported as an
overwrite of unchanged bytes, and abort always drops the session's
object. Only the two presigned paths changed; `prepareBatchWrites`
uploads server-side within one authorized request and still writes in
place.

* fix: the review's findings on the staged overwrite

The batch signed path still signed against the live key, so the route
the SDK's signed upload actually takes was unchanged. It stages now,
which also makes the abort simplification safe: the guard that had
protected a shared key was removed while that path still shared one, so
aborting a batch overwrite would have deleted the owner's object.

The completion is allowed against a path but lands on the uuid captured
at start, and a move between the two puts the bytes somewhere the caller
was never authorized for. The promote refuses unless the entry is still
where the check looked.

The batch promote failing left every session pending with its lease
held; it now fails them as the rest of the function does.

The pending GUI tile and the thumbnail key were keyed on the staged
uuid, which no entry ever carries, so neither correlated with what the
completion emitted. Responses carry `entryUid` for that.

* fix: do not treat a pre-staging session's key as this upload's own

A session signed by the previous release points straight at the live key,
since `objectKey` was `existingEntry?.uuid ?? uuidv4()`. Completing one is
already handled -- the promote returns early when the two keys match -- but
abort deleted `objectKey` unconditionally, which during a rolling deploy
would take the owner's file rather than the upload's litter.

Abort now deletes only a key the upload owns. The multipart branch was
always safe: discarding parts never touches an object already in place.
2026-10-09 16:22:36 -04:00
Juan Fernando Castro 86cc3490f7 fix: settle which plan an org seat resolves to, and confirm a seat's address (#4139)
* fix: settle which default plan an org seat resolves to, and what counts as paid

Default subscription resolvers ran in registration order, which is boot
order: whichever layer registered first answered first. A seat is
provisioned by a team and its address is never confirmed, so the seat
resolver now states a priority and wins over anything a deployment
infers from an address.

The seat cap asked whether the owner's plan was outside the free set,
which a default plan can be without anyone having bought it. Metering
now reports whether an explicit subscription answered, and the cap rests
on that.

* fix: the review's findings on plan resolution

`fromDefault` was read off which resolver answered, not off which policy
applied. An explicit id naming a policy nobody registered falls through
to the default one, and that read as bought. It follows the applied
policy now.

A deployment with unlimited metering has no explicit resolver, so every
owner on it became unpaid and its teams dropped from the paid seat cap
to the free one. Unlimited answers for what a paid plan would, so it
counts as one.

Also: the new constants had been placed between `FREE_SUBSCRIPTION_IDS`
and its docblock, the register docstring still described first-wins, and
prettier had reflowed an untouched licence header.

* feat: a seat confirms the address its team typed in

An address on a seat is the team's claim about someone else's inbox, and
nothing asked the inbox. It is now demanded and sent a code, so the
address counts for nothing until the seat hands it back. Resolvers that
read a domain are the reason this matters.

No address is still fine and demands nothing: the team is the trust
anchor for the account, not for a mailbox it does not hold.

`/confirm-email` allows an unconfirmed actor, so a seat can sign in with
its temporary credential and clear the demand; nothing locks it out.

* fix: clear the confirmation demand with the address it names

`clearUnconfirmedEmail` releases a seat's address so the inbox's owner can
sign up with it. The demand to confirm that address has to go at the same
write: the gate reads `requires_email_confirmation && !email_confirmed`, and
once the address is gone there is nothing left to send a code to.

* fix: keep a seat's address correctable, and only ask what can be answered

Review of the confirmation demand turned up two ways a seat could be left
holding one it had no way to meet:

- With no mail transport configured, the code is never sent and a resend
  cannot reach the seat either, so the demand stood with nothing able to
  lift it. The seat is now gated only when a code can actually travel.
- An address the owner mistyped gated the seat on an inbox nobody could
  read, and the gate itself closes `/user-protected/change-email`, the
  route the seat would have fixed it by. `updateSeatEmail` lets the owner
  who typed the address retarget it, and refuses once it is confirmed,
  when it belongs to whoever proved it.

Also resolves the owner's plan sequentially, so the second read takes the
cache the first wrote and the pair cannot come from two resolutions, and
restores the licence header a jsdoc reflow had rewrapped.

* fix: let the confirm-email window open before the profile has loaded

Booted from a stored token, the window renders before `window.user` is
populated, and reading `.email` off null threw inside the window itself --
so the one screen that could have asked for the code never drew, leaving a
blank page. Reproducible without a team: sign up, then load the GUI again
before confirming.

The address is now encoded rather than interpolated raw, and both forms of
the line are i18n keys; it was hardcoded English before.
2026-10-09 16:17:51 -04:00
Juan Fernando Castro 7de8360a6e Sharing cleanup: bounded outbound listings, and what see does not reach (#4118)
* fix: keep a file name to one line in a subject, and the inbound total to the account

**A name reaches the digest subject as the user typed it.** Subjects compile
with `noEscape` on purpose -- a header is not HTML, and escaping would show
entities to the reader -- and the comment there says free-form values collapse
whitespace upstream. A file name does not: `#assertUsableName` refuses only an
empty name, a slash, `.` and `..`, so a name may carry newlines. It is
collapsed to one line and bounded before it goes in.

**`includeTotal` answered an app with the account's figure.** The page is
filtered by what the caller can reach -- an app sees only what its user handed
it -- while `countByHolder` counts every inbound share. The docs already call
`total` an approximation for the filtering; they did not cover an app reading
the size of everything else. It is answered only for a caller acting as the
account now, and the page documents that.

The team-share fixture built its actor as `{ user }` with no `effectiveApp`,
which reads as an app rather than a session. A session acts as the account, so
it says so.

* fix: stop `see` reaching inside a folder, and bound mode changes

**A `see`-only folder recipient was pushed every child's name, size and type.**
The fan-out picked holders by reach -- any live grant on a reaching root -- and
`see` is a grant. But `see` says a node is there; `list` is what says what is
inside it, and the payload carries exactly what `list` would: name, path,
`is_dir`, type, size, modified. Descendants now go only to a holder whose mode
reaches inside. A change to the shared node itself still goes to everyone who
holds it, which is what `see` covers.

**Mode changes skipped the daily quota entirely.** Not spending share budget is
right -- changing a mode is not new reach, and the existing test says so -- but
unbounded it lets one account write a pair as fast as it can ask. They have
their own budget now, ten times the share limit by default and configurable,
so the exemption stays an exemption from *that* budget rather than from any.

* fix: let a half-finished downgrade be finished, and stop Trash answering

**A mode downgrade that failed between its two halves could not be repaired.**
`setUserUser` grants the new mode and then revokes the superseded ones. If the
revokes did not all land, the higher grant stood -- and the retry returned
early, because the mode it was asked for was now present. "Nothing to do" now
means this mode is the only one standing, so asking again is what finishes it.
The same two halves, and the same early return, were in the group variant.

**Trashing someone else's item answered for what is in their Trash.** Moving
into the owner's Trash is deliberately authorized without write on it -- that
is what lets a recipient delete from a shared folder -- but a name collision
there came back as `An entry already exists at /owner/Trash/<name>`. The mover
picks the name, so that is a question they can ask about a folder they cannot
list. It dedupes instead, which is also what the GUI wants: trashing should
not fail over a name.

* perf: record the entry's owner on the share row

The outbound listings answer "shares on things I own that someone else issued"
by joining `share` to `fsentries` and filtering on the owner there. Neither side
of that join bounds the work: driven from `fsentries` it walks every file the
user owns, driven from `share` it walks every share until it has a page. For a
user who owns a lot and delegates nothing, that is the whole table for an empty
result. `#outboundAppsSql` is worse than the ticket says -- it has no LIMIT at
all, so every call aggregates the join across both tables.

The owner now rides on the row, indexed with `id`, so all three read it
directly. An entry cannot change owner -- a move out of the owner's tree is
refused -- so it is written once, with the row.

Set in SQL from `fsentry_id` rather than threaded through callers: four insert
sites, each of which already has the id, and none of which can now forget.
The risk in denormalising is a write path that does: a row with no owner
recorded is a share that silently stops appearing in its owner's listing. The
new test covers exactly that -- it fails when one insert is made to write NULL.

Verified against a real boot: a pre-existing share row, on a database that
genuinely predates the column, comes out with the owner its entry has.

* fix: the review's findings on this branch

**The Trash collision still had both halves open.** `overwrite` was checked
before the trash case, so a caller setting it -- it is wire-supplied -- could
still delete whatever it collided with in the owner's Trash, and the 409
survived for anyone who did not. The trash case is decided first now.

Dedupe alone did not close it either: a name that changes only when there is a
collision answers the same question the refusal did, and the mover sees that
name even though the path is masked. Trashing someone else's item renames it
every time, so the result carries no signal. `original_name` already records
the real name, which is what a restore reads. Two tests pinned the old name;
they say what it is now.

**The owner column had no fallback.** Reads went only through it, so a row
written by anything that predates it -- a pod mid-roll -- would never appear in
its owner's listing again. The three reads now also cover `IS NULL` rows
through the old join, indexed on the null and empty once none remain.

**The grant became conditional on a replica read.** `if (!already)` skipped it
on a stale row while the revokes below stayed unconditional, which could leave
the holder with nothing where the old early return left their access alone. It
grants unconditionally again; only the early return changed.

**The re-issue budget covered one path of four.** Team, link and invite
re-issues reserved nothing, so three ways round it were left open. They share
one `#reserveRemodeQuota` now.

Two docblocks had been pushed off the functions they describe, and two comments
still described the join that is gone.

* chore: one-line the comments added here

* test: measure the prefix walk by what it keys on, not by the clock

`checks many long paths without re-reading every prefix of each` gave
the walk 1500ms of wall clock. It measured nothing useful: the shape it
guards against — keying each step on the accumulated prefix text rather
than on the parent id — costs 29ms against the current 8ms on that
input, so both ends sit far under the bound. Under coverage the same
test ran 1511ms and failed, which is the only thing the number could
ever report on.

`PathPrefixIds` now counts the characters it keys on, and the walk is
asserted on that: proportional to the input, and twice the depth costs
twice as much rather than four times. Keying on the accumulated text
fails both. The original case keeps its real assertion, that a request
this size is refused.

* fix: keep the recorded entry owner current, and the trash guard on the retry

A move into someone else's tree re-owns the whole subtree, which the
migration comment claimed could not happen. The shares on it are
re-pointed at the new owner, or they drop out of the listing the column
keys and stay in the old owner's.

The unique-violation retry re-entered the collision resolver without the
trash flag, which is the one call site the guard was not threaded
through.
2026-10-09 16:14:23 -04:00