Durable subscribe, list, fetch and unsubscribe now apply the same scoped
token check as the handler, worker and kv-handle routes:
`isAccessTokenActor(actor) && !isAccountContext(actor)`. An access token
an app issued (effectiveApp = the app) is treated the same as one the
account issued: subscribe answers 403 `events_durable_requires_account`,
list and fetch return an empty page, unsubscribe answers 404.
`rowInActorScope` returns false for these tokens, which covers durable
unsubscribe and ack. The session verbs that also use it already refuse
access tokens at the socket handshake.
Full-access tokens, app tokens and sessions are unchanged.
The duplicate-username check in the signup placeholder claim, change-username
and save-account runs before the user-row write, so a concurrent write can
take the name in between and the unique index rejects ours. Each path now
re-reads the holder from the primary on a unique violation and returns the
400 its pre-check returns for a taken name, instead of a 500. In all three,
the home-directory rename, rename budget charge and username-changed event
run after the user-row write, so nothing has happened yet when it fails.
The boot-key batch now starts only when puter.env is 'gui'. In every other
environment a get() of one of those names reads that key on its own, like
any other key, instead of fetching all 14 and serving them for 4 s.
- signup: a username taken by a concurrent signup between the duplicate
check and the insert now returns the pre-check's 400 instead of
surfacing the unique-key error as a 500. The holder is read from the
primary.
- fs move: when the UPDATE trips the (parent_id, name) unique key after
the collision probe missed the occupant, re-read it from the primary and
resolve it the same way the probe would have (overwrite, dedupe, or the
409 item_with_same_name_exists), then retry the move once.
- server health: a check that hits the per-check timeout reports how long
the event loop was blocked during it, like the database-liveness latency
failure already does.
- flush follows LastEvaluatedKey until the namespace is empty and rejects
when a batch delete fails; cached reads for every key a page returned are
still invalidated and the flushed event still fires.
- decr validates the amount map before negating it, so it accepts exactly
what incr accepts.
- del checks the key like every other store method (oversized key is a 400).
- list offset emulation treats a missing continuation key as end of data
instead of restarting from the first key.
- Array get maps results through a Map instead of a linear find per key.
- The driver gates includeTotal on credit like value listings.
- New driver caps in drivers/kv/limits.ts: 1,000 keys per array get, 1,000
items per batchPut (both rejected past the cap), list limit lowered to
1,000. Published in rate-limits-and-quotas.md and the set/list pages.
- puter.kv.list() no longer throws synchronously for invalid options; it
returns a rejected promise, or for stream: true an iterator whose first
next() rejects. Error codes unchanged.
- Writes through puter.kv (set, del, incr, decr, add, remove, update,
expire, expireAt, flush) stop the boot-key batch from serving the keys
they touch, so the next get() reads the store. Untouched boot keys are
still served from the batch.
- Client-side key/value size checks measure UTF-8 bytes, and values are
measured as their JSON encoding, matching the store. Same codes and
messages.
- stat: only path-addressed results are cached; uid-addressed stats no
longer share one cache key.
- readdir: overloads declare the page result for recursive + limit/offset
and for the positional page/stream forms.
- move: only a not-found destination lookup means "new path"; other
lookup failures reject. A uid destination skips the lookup.
- getAbsolutePathForApp always resolves paths. read/copy/move/delete use
getAbsolutePathOrUidForApp, which keeps sending UID-shaped strings as uids.
- Docs for read, copy, move, delete and readdir.
Two findings from review, both of which contradict what the last commit
claimed.
**The backfill could merge keys, not only split them.** I asserted it "can only
split rows apart, so it cannot collide". Not true, and reproducible: a row
predating the `clean_email` column has a NULL there, so its index key is
`lower(email)` -- filling it in folds `+tag`, and `john+news@me.com` lands on a
key `john@me.com` already owns. A legacy row holding an un-lowercased
`clean_email` collides the same way once lowercased. Either throws, and the two
engines that re-run every file on every boot would then refuse to start until
someone operated on the database by hand.
Both updates now fire only where the stored value is exactly what the old rule
produced and differs from the correct one. That makes "splits only" true: the
sole change is dots coming back. NULLs and legacy values are left alone.
The same predicate makes them write-idempotent. Postgres has no no-op-update
suppression, so without it every boot rewrote a tuple per Apple row forever.
**The pending-invite residue misdelivers rather than just going missing.**
`claimPendingShares` looks the row up by the canonical address, so whoever
confirms `jsmith@icloud.com` claims the invites addressed to
`j.smith@icloud.com` -- someone else's file, handed over on confirmation. I had
written this off in the PR as unfindable invites that "expire on their own";
`share` has no expiry column and the lookup has no time predicate, so they are
claimable indefinitely.
Fixed here rather than later: the typed address is kept as `invitedAddress`
whenever it differed from the canonical, which is exactly the collapsed rows,
so the invites recompute from it under the same guard.
Verified by seeding a database at the old version with the two rows that used
to abort the migration, and booting: it reaches version 87, repairs the
collapsed row, and leaves the NULL and legacy rows untouched. Replaying the
statements changes zero rows.
The sqlite migration list is hand-maintained, so the file alone did nothing —
a fresh boot still reported 87 files and stopped at version 86. MySQL and
Postgres read their directories, so only this one needed the entry.
Caught by booting a local instance rather than by the suite: the client tests
pin `CURRENT_SCHEMA_VERSION`, which fails once the entry exists but says
nothing while the file is simply unregistered.
Verified against a seeded database at the old version. `John.Smith@iCloud.com`
went from `johnsmith@icloud.com` to `john.smith@icloud.com`, `a.b+tag@me.com`
from `ab@me.com` to `a.b@me.com` — subaddressing still folded — and a gmail row
was left alone.
`cleanEmail()` applied gmail's `dots_dont_matter` rule to icloud.com, me.com
and mac.com. Apple allocates the exact string: `j.smith@icloud.com` and
`jsmith@icloud.com` are two mailboxes with two owners. Collapsing them made one
canonical address stand for both.
What that canonical address feeds:
- `findEmailOwner` falls back to it, so an OIDC sign-in whose verified claim is
the dotted address resolves to the account holding the undotted one, and
`linkProviderToUser` has nothing left to object to -- the provider verified a
real mailbox and the account's address is confirmed.
- The unique index on COALESCE(clean_email, lower(email)) reads it, so the
rightful owner of a dotted Apple address is refused signup because the
undotted account already "owns" it.
- Pending share invites are stored and claimed on it.
Subaddressing is untouched: Apple does fold `+tag`, so that rule stays. Gmail
keeps both rules -- there the fold is correct.
A migration recomputes `clean_email` for those domains on all three engines,
since rows written before this still hold the collapsed value and the index
reads the stored one. Verified the SQL against the function rather than by
inspection: same five inputs through `cleanEmail()` and through sqlite give
byte-identical output. It can only split rows apart, never merge them, so it
cannot collide with the unique index.
Not changed: `findUserByEmail` still matches on the canonical address. The
cross-match it documents is correct where the provider really does fold --
`foo.bar+tag@gmail.com` is the same inbox as `foobar@gmail.com`. Requiring an
exact match would break that for no benefit here; the fix is for the data to
say what the provider does.
The in-flight claim's lease (2x the invoke timeout) is also a future score,
so polling for "held" could read it before the failed attempt's backoff
hold was written. Wait for deferAfterFailure to settle instead.
- Chat driver runs each attempt through one helper and checks the abort
signal only on entry and in the attempt catch blocks, where it keeps a
cancelled request from marking the route unhealthy.
- A stream ended after cancellation still emits ai.prompt.cost-calculated;
only ai.prompt.complete is skipped.
- pipeNdjsonStream owns its line buffer and onEnd is optional; drop the
errored guards that can no longer fire.
- Drop the Claude post-loop abort check (finalMessage already throws).
- Skip provider error logs when the caller cancelled.
- Simplify the usage_limited cap condition.
Local testing against a running instance found the service-side fix was not
enough. Both answers were 403, but they did not read the same:
exists -> permission_denied: fs:/other/rd:read
missing -> Cannot create outside your home directory (forbidden)
The second comes from `#planFsPathCreate` in the controller, which probes the
path itself and only calls the create guard once it knows the path is absent.
Its comment said so outright -- checked "only once the path is confirmed
missing, so an existing shared path outside the caller's home is not a new
rejection" -- which is the oracle stated as an intention.
A foreign path now refuses word for word whichever it is. Responses are
byte-identical apart from the caller's own input.
`rewritePermissionForActor` collapses the create guard's `forbidden` the same
way it already collapsed the 404, for the paths that reach it through the
service rather than the controller.
The existing test pinned `forbidden` here; its subject is that nothing is
created in someone else's tree, which still holds. Alongside it is the
invariant itself: refuse a present and an absent foreign path, mask the path,
compare the whole error. Without the controller check that fails.
**The path oracle was not closed.** The rewrite only remapped the *error*, so
success-vs-403 still answered whether a foreign path exists. Narrower than
reported: every other grant path already gates on `canManagePermission`, and
only `grantUserAppPermission` has no authority check -- deliberately, since
granting your own app something you don't hold is inert. It now refuses a
foreign path the actor cannot see, with the same error a missing one gives.
My first test for this passed without the fix: `makeUserActor` creates no
fsentries, so both paths 404'd and nothing distinguished them. It inserts a
real entry now, and the control fails with "expected null to match 403" --
null being the grant that succeeded, which is the oracle.
**`#validateEmail` still failed open** on the main signup / change-email /
confirm path while its OIDC twin was hardened. Same shape as OIDC now: a gate
that did not run is not a pass.
**The 16-permission cap broke `puter.perms.check()`** for longer lists. The
SDK's own docblock promised "one round trip however many are asked about",
which the cap made untrue, so it batches instead. The reported second half --
the UI grant poller burning its timeout -- is not reachable:
`MAX_REQUESTED_PERMISSIONS` already caps that path at the same 16.
**`listener_failed` was set by wildcard listeners.** An unrelated observer on
`email.*` throwing would refuse every signup. Only an exact-key listener
implements the event, so only that one speaks for the gate.
**`whoami()` labelled everything `Unauthorized`.** A 502 with an HTML body is
not a 401; the server's `code` now rides along and the fallback names the
status.
Switching `getSubtreeSize` to `entry.userId` fixed a recipient reading 0 for a
shared folder, but `/stat` is gated on `see` and `return_size` is caller-
supplied. `see` is the weakest level -- the folder is visible, what is under it
is not -- so the sum now covered entries the caller had no way to enumerate.
It is answered only for a caller who passes a `list` check, which is where the
contents become enumerable anyway. Everyone below that gets the directory row's
own size, which is what they got before this branch.
`#assertAccess` grew a non-throwing sibling for this; the descriptor both need
moves into `#aclDescriptor`.
Test grants `see`, asserts 0, grants `list`, asserts the real size. Without the
gate the `see` case reads the file's bytes.
Measured first, against a real server, counting DB round trips: a 40-seat
delete was 284 queries, 7.1 per seat, scaling linearly. Four of those seven
answered for the whole page and had no reason to be asked per account.
120x SELECT * FROM `user` WHERE `id` = ?
41x INSERT INTO `audit_team_membership`
40x SELECT COALESCE(SUM(size), 0) FROM fsentries
40x UPDATE `user` SET `suspended` = ?
40x SELECT `uuid` FROM `sessions` WHERE `user_id` = ?
The page now reads who is already suspended, what each holds, and the rows the
notices need in one query each, and writes the audit as one statement. What is
genuinely per account -- the suspension, its billing event, the mail -- stays
per account under the same concurrency bound.
Same 40-seat delete is now 170 queries, 4.3 per seat.
Three new store methods rather than raw SQL at the call site:
`getSuspendedByIds` reads the primary, deliberately not `getByIds`, which is
cache-first. A stale `suspended` is exactly what re-charges a seat on a retry,
which is the thing this path was just fixed to stop doing.
`getHeldBytesForUsers` is `getHeldBytes` with a GROUP BY.
`appendAuditMany` is `appendAudit` as one multi-row insert, still written
before any suspension, so a failed append cannot leave an unlogged one.
What is left is concentrated in `#suspend`: the UPDATE, the forced re-read the
user store does afterwards to refresh its cache, and the session drop. Those
are shared store paths and batching them means owning their cache invalidation,
so they are not in scope here.
Reynaldi Chernando