- signup: a username taken by a concurrent signup between the duplicate
check and the insert now returns the pre-check's 400 instead of
surfacing the unique-key error as a 500. The holder is read from the
primary.
- fs move: when the UPDATE trips the (parent_id, name) unique key after
the collision probe missed the occupant, re-read it from the primary and
resolve it the same way the probe would have (overwrite, dedupe, or the
409 item_with_same_name_exists), then retry the move once.
- server health: a check that hits the per-check timeout reports how long
the event loop was blocked during it, like the database-liveness latency
failure already does.
- flush follows LastEvaluatedKey until the namespace is empty and rejects
when a batch delete fails; cached reads for every key a page returned are
still invalidated and the flushed event still fires.
- decr validates the amount map before negating it, so it accepts exactly
what incr accepts.
- del checks the key like every other store method (oversized key is a 400).
- list offset emulation treats a missing continuation key as end of data
instead of restarting from the first key.
- Array get maps results through a Map instead of a linear find per key.
- The driver gates includeTotal on credit like value listings.
- New driver caps in drivers/kv/limits.ts: 1,000 keys per array get, 1,000
items per batchPut (both rejected past the cap), list limit lowered to
1,000. Published in rate-limits-and-quotas.md and the set/list pages.
- puter.kv.list() no longer throws synchronously for invalid options; it
returns a rejected promise, or for stream: true an iterator whose first
next() rejects. Error codes unchanged.
- Writes through puter.kv (set, del, incr, decr, add, remove, update,
expire, expireAt, flush) stop the boot-key batch from serving the keys
they touch, so the next get() reads the store. Untouched boot keys are
still served from the batch.
- Client-side key/value size checks measure UTF-8 bytes, and values are
measured as their JSON encoding, matching the store. Same codes and
messages.
- stat: only path-addressed results are cached; uid-addressed stats no
longer share one cache key.
- readdir: overloads declare the page result for recursive + limit/offset
and for the positional page/stream forms.
- move: only a not-found destination lookup means "new path"; other
lookup failures reject. A uid destination skips the lookup.
- getAbsolutePathForApp always resolves paths. read/copy/move/delete use
getAbsolutePathOrUidForApp, which keeps sending UID-shaped strings as uids.
- Docs for read, copy, move, delete and readdir.
Two findings from review, both of which contradict what the last commit
claimed.
**The backfill could merge keys, not only split them.** I asserted it "can only
split rows apart, so it cannot collide". Not true, and reproducible: a row
predating the `clean_email` column has a NULL there, so its index key is
`lower(email)` -- filling it in folds `+tag`, and `john+news@me.com` lands on a
key `john@me.com` already owns. A legacy row holding an un-lowercased
`clean_email` collides the same way once lowercased. Either throws, and the two
engines that re-run every file on every boot would then refuse to start until
someone operated on the database by hand.
Both updates now fire only where the stored value is exactly what the old rule
produced and differs from the correct one. That makes "splits only" true: the
sole change is dots coming back. NULLs and legacy values are left alone.
The same predicate makes them write-idempotent. Postgres has no no-op-update
suppression, so without it every boot rewrote a tuple per Apple row forever.
**The pending-invite residue misdelivers rather than just going missing.**
`claimPendingShares` looks the row up by the canonical address, so whoever
confirms `jsmith@icloud.com` claims the invites addressed to
`j.smith@icloud.com` -- someone else's file, handed over on confirmation. I had
written this off in the PR as unfindable invites that "expire on their own";
`share` has no expiry column and the lookup has no time predicate, so they are
claimable indefinitely.
Fixed here rather than later: the typed address is kept as `invitedAddress`
whenever it differed from the canonical, which is exactly the collapsed rows,
so the invites recompute from it under the same guard.
Verified by seeding a database at the old version with the two rows that used
to abort the migration, and booting: it reaches version 87, repairs the
collapsed row, and leaves the NULL and legacy rows untouched. Replaying the
statements changes zero rows.
The sqlite migration list is hand-maintained, so the file alone did nothing —
a fresh boot still reported 87 files and stopped at version 86. MySQL and
Postgres read their directories, so only this one needed the entry.
Caught by booting a local instance rather than by the suite: the client tests
pin `CURRENT_SCHEMA_VERSION`, which fails once the entry exists but says
nothing while the file is simply unregistered.
Verified against a seeded database at the old version. `John.Smith@iCloud.com`
went from `johnsmith@icloud.com` to `john.smith@icloud.com`, `a.b+tag@me.com`
from `ab@me.com` to `a.b@me.com` — subaddressing still folded — and a gmail row
was left alone.
`cleanEmail()` applied gmail's `dots_dont_matter` rule to icloud.com, me.com
and mac.com. Apple allocates the exact string: `j.smith@icloud.com` and
`jsmith@icloud.com` are two mailboxes with two owners. Collapsing them made one
canonical address stand for both.
What that canonical address feeds:
- `findEmailOwner` falls back to it, so an OIDC sign-in whose verified claim is
the dotted address resolves to the account holding the undotted one, and
`linkProviderToUser` has nothing left to object to -- the provider verified a
real mailbox and the account's address is confirmed.
- The unique index on COALESCE(clean_email, lower(email)) reads it, so the
rightful owner of a dotted Apple address is refused signup because the
undotted account already "owns" it.
- Pending share invites are stored and claimed on it.
Subaddressing is untouched: Apple does fold `+tag`, so that rule stays. Gmail
keeps both rules -- there the fold is correct.
A migration recomputes `clean_email` for those domains on all three engines,
since rows written before this still hold the collapsed value and the index
reads the stored one. Verified the SQL against the function rather than by
inspection: same five inputs through `cleanEmail()` and through sqlite give
byte-identical output. It can only split rows apart, never merge them, so it
cannot collide with the unique index.
Not changed: `findUserByEmail` still matches on the canonical address. The
cross-match it documents is correct where the provider really does fold --
`foo.bar+tag@gmail.com` is the same inbox as `foobar@gmail.com`. Requiring an
exact match would break that for no benefit here; the fix is for the data to
say what the provider does.
Local testing against a running instance found the service-side fix was not
enough. Both answers were 403, but they did not read the same:
exists -> permission_denied: fs:/other/rd:read
missing -> Cannot create outside your home directory (forbidden)
The second comes from `#planFsPathCreate` in the controller, which probes the
path itself and only calls the create guard once it knows the path is absent.
Its comment said so outright -- checked "only once the path is confirmed
missing, so an existing shared path outside the caller's home is not a new
rejection" -- which is the oracle stated as an intention.
A foreign path now refuses word for word whichever it is. Responses are
byte-identical apart from the caller's own input.
`rewritePermissionForActor` collapses the create guard's `forbidden` the same
way it already collapsed the 404, for the paths that reach it through the
service rather than the controller.
The existing test pinned `forbidden` here; its subject is that nothing is
created in someone else's tree, which still holds. Alongside it is the
invariant itself: refuse a present and an absent foreign path, mask the path,
compare the whole error. Without the controller check that fails.
**The path oracle was not closed.** The rewrite only remapped the *error*, so
success-vs-403 still answered whether a foreign path exists. Narrower than
reported: every other grant path already gates on `canManagePermission`, and
only `grantUserAppPermission` has no authority check -- deliberately, since
granting your own app something you don't hold is inert. It now refuses a
foreign path the actor cannot see, with the same error a missing one gives.
My first test for this passed without the fix: `makeUserActor` creates no
fsentries, so both paths 404'd and nothing distinguished them. It inserts a
real entry now, and the control fails with "expected null to match 403" --
null being the grant that succeeded, which is the oracle.
**`#validateEmail` still failed open** on the main signup / change-email /
confirm path while its OIDC twin was hardened. Same shape as OIDC now: a gate
that did not run is not a pass.
**The 16-permission cap broke `puter.perms.check()`** for longer lists. The
SDK's own docblock promised "one round trip however many are asked about",
which the cap made untrue, so it batches instead. The reported second half --
the UI grant poller burning its timeout -- is not reachable:
`MAX_REQUESTED_PERMISSIONS` already caps that path at the same 16.
**`listener_failed` was set by wildcard listeners.** An unrelated observer on
`email.*` throwing would refuse every signup. Only an exact-key listener
implements the event, so only that one speaks for the gate.
**`whoami()` labelled everything `Unauthorized`.** A 502 with an HTML body is
not a 401; the server's `code` now rides along and the fallback names the
status.
Switching `getSubtreeSize` to `entry.userId` fixed a recipient reading 0 for a
shared folder, but `/stat` is gated on `see` and `return_size` is caller-
supplied. `see` is the weakest level -- the folder is visible, what is under it
is not -- so the sum now covered entries the caller had no way to enumerate.
It is answered only for a caller who passes a `list` check, which is where the
contents become enumerable anyway. Everyone below that gets the directory row's
own size, which is what they got before this branch.
`#assertAccess` grew a non-throwing sibling for this; the descriptor both need
moves into `#aclDescriptor`.
Test grants `see`, asserts 0, grants `list`, asserts the real size. Without the
gate the `see` case reads the file's bytes.
Measured first, against a real server, counting DB round trips: a 40-seat
delete was 284 queries, 7.1 per seat, scaling linearly. Four of those seven
answered for the whole page and had no reason to be asked per account.
120x SELECT * FROM `user` WHERE `id` = ?
41x INSERT INTO `audit_team_membership`
40x SELECT COALESCE(SUM(size), 0) FROM fsentries
40x UPDATE `user` SET `suspended` = ?
40x SELECT `uuid` FROM `sessions` WHERE `user_id` = ?
The page now reads who is already suspended, what each holds, and the rows the
notices need in one query each, and writes the audit as one statement. What is
genuinely per account -- the suspension, its billing event, the mail -- stays
per account under the same concurrency bound.
Same 40-seat delete is now 170 queries, 4.3 per seat.
Three new store methods rather than raw SQL at the call site:
`getSuspendedByIds` reads the primary, deliberately not `getByIds`, which is
cache-first. A stale `suspended` is exactly what re-charges a seat on a retry,
which is the thing this path was just fixed to stop doing.
`getHeldBytesForUsers` is `getHeldBytes` with a GROUP BY.
`appendAuditMany` is `appendAudit` as one multi-row insert, still written
before any suspension, so a failed append cannot leave an unlogged one.
What is left is concentrated in `#suspend`: the UPDATE, the forced re-read the
user store does afterwards to refresh its cache, and the session drop. Those
are shared store paths and batching them means owning their cache invalidation,
so they are not in scope here.
Four, all real.
The gate threw on a non-string. `validator.isEmail` asserts its argument, and
two call sites hand it an unvalidated body value -- `send-pass-recovery-email`
is public, so `{"email": 1}` was a 500 rather than a 400. The check belongs in
the gate, not in each caller, which is the whole premise of having one.
OIDC signup never shape-checked the claim at all. `claims.email` went from a
truthiness check straight into `user.create`, so a provider we do not run could
store exactly the string this branch exists to keep out. `email_verified`
attests the mailbox, not the characters.
The change-email confirm handler promoted the staged column without re-reading
it through anything. A row staged before this deploys still lands on `email`;
one check at promotion closes that window.
The invite path kept its own regex, and it was never the same rule. It admitted
`a..b@x.com`, `.a@x.com`, `a@-x.com`, `a;b@x.com` and `a,b@x.com` -- the last of
which nodemailer reads as two recipients -- each one a permanent pending row
that spent daily quota, which is the failure the comment above the constant
said it existed to prevent. It uses the gate now, and the second regex is gone.
A regression test each, all four confirmed to fail without their fix.
`validator.isEmail` accepts a quoted local part, so an address whose local part
is an arbitrary string passes it. Five call sites relied on that check alone
before writing the address to `user.email`, and the invite path's own shape
regex was equally permissive.
`isStorableEmail` is now the single gate: the library check, plus a refusal of
`<`, `>` and `"`, which are legal in an address only inside the quoted form.
Nothing deliverable uses that form, so the rule costs no real address --
apostrophes and ampersands in a local part still pass, and the suite covers
them.
One function rather than a check each caller has to remember, because the next
place that stores an address will not remember.
Six findings, all real.
`total` described rows the page withheld. The count ran over every live team the
user belongs to, while the app-actor filter ran afterwards on the items only, so
`GET /teams?includeTotal=true` from an app returned `items: []` with a non-zero
`total` -- the exact thing the filter exists to prevent, and newly reachable
because `includeTotal` was inert before this branch. The directory-open
restriction moves into the query, so items, cursor and total all describe the
same set.
The GUI told owners the opposite of what the server now does: "we email the
username and temporary password to it." An owner who believed it would dismiss
the one copy of the password and strand the seat. Rewritten.
`runWithConcurrencyLimit` rejects on the first error but cannot cancel the
workers beside it, so a failed seat let up to seven others finish suspending and
emailing "your team has been closed" for a team that was still live. The page
now settles before the rejection is rethrown, which leaves a state the retry can
resume from instead of an arbitrary one. My earlier claim that failure
semantics were unchanged was wrong.
`#disableSeatForDelete` returned `true` on both paths, so the boolean read as if
it distinguished them. It returns nothing; the counter uses the settled results.
`listTeamsForUser` had picked up a second stacked docblock -- the same defect
this branch removes from `#cached` one file over.
`reissueCredential` sent the provisioning mail, which after the credential
change was a byte-identical duplicate saying nothing about the password it had
just invalidated, while the GUI promises the member hears about it. It has its
own template now.
Regression tests for the count leak and the reissue notice; both confirmed to
fail without their fix.
Both docblocks on the directory route said it carries nothing a colleague
cannot already read through `/members`. It carries `uuid`, and `/members`
withholds `uuid` from everyone but the owner, with its own comment pointing
apps at the directory to get one.
The disclosure itself is the opt-in working as designed: `directory_enabled` is
the owner's consent to expose a roster, every member reaches the route the same
way whether they call it from a session or through an app, and the SDK and the
teams recipes already document the directory as where a stable id comes from.
So the comments are what was wrong, not the route.
Removing `uuid` from the directory would be the other way to make them agree,
but it breaks `listDirectory()` for every caller and contradicts the published
recipes; that wants a decision, not a cleanup commit.
`#resolveRecipient` already refuses to treat an unconfirmed address as an
identity, and says why: "resolving it would hand the share to whoever registered
the address first." `#invite` then looked the same address up again and honoured
whatever that row said about blocking -- `blocksAllShares`, or a block on this
particular sender.
So registering an address you do not own, and never confirming it, was enough to
refuse every invite anyone sends to that mailbox. Nothing about that row is
evidence the mailbox is yours, which is the premise resolution already works
from. A team seat reaches this cheaply: those accounts are provisioned with
`requires_email_confirmation: false` and an address the owner typed.
The block is now honoured only for a row that confirmed the address. The alias
reasoning behind the lookup is unchanged -- a confirmed holder still cannot be
reached through a variant they did not type.
Not addressed here: the same unconfirmed seat still holds the address against
the real owner's signup, because `idx_user_owned_email` keys on having a
password rather than on confirmation. Narrowing that is a schema change and
wants its own ticket.
Provisioning a seat lets the owner supply any `email` that passes a format
check. Puter then mailed a working temporary password to it. Nothing confirms
the owner controls that address -- provisioned accounts are created with
`requires_email_confirmation: false`, because the team is the trust anchor for
the account, not for the mailbox. `resendActivation` reissues the credential and
mails it again, up to the 20/day `team:reset-password` budget, so the same
address can be hit repeatedly.
The credential now stays out of the mail on both paths, which is what
`resetMemberPassword` in the same file already does. Nothing is lost: all three
routes return `temporary_password` to the owner, and the SDK has always
documented that as the delivery -- "The returned password is shown once and is
not retrievable afterwards -- deliver it out of band." The mail still goes out,
naming the account and saying the team will supply the password.
The two tests that pinned the old shape are inverted rather than deleted; one of
them carried the rationale "without it this is the only copy", which the return
value contradicts.
`deleteTeam` tore seats down one at a time, each one awaiting a mail send, so a
full 40-seat team was 40 round trips end to end inside a single request. The
per-seat work is independent, so it now runs under the same bounded-concurrency
helper the rest of the backend uses for fan-out sends.
The loop also had no equivalent of the guard `disableMember` carries. That one
checks `suspended` before emitting, with the reason written down: a second
`team.account.disabled` opens a second byte charge that only one `enabled` will
ever close. Deleting a team emitted unconditionally, so any partial failure
followed by a retry double-charged every seat the first attempt had already
disabled. The seat teardown moves into one method both paths can state the same
way, and a retry now resumes instead of re-charging.
Failure semantics are unchanged: a seat that throws still aborts the delete and
leaves the team live, which is what makes the retry meaningful.
Touching this file reformats a few untouched spans -- it carried ten
pre-existing `prettier/prettier` errors that the repo's own pre-commit `eslint
--fix` resolves on any commit that stages it.
The cache docblock claimed `isMember` and `getByUid` were left uncached "for
that reason". Both go through it -- `getByUid` directly, `isMember` through
`getMembership` -- so the comment told a reader the opposite of the code, on the
two reads that decide authorization.
What makes it safe is the busting, not an exemption: every mutation here drops
the keys it touches, and the TTL is the backstop for the one path that does not
come through this store, the ON DELETE CASCADE on the membership row. No stale
window is constructible today; the comment now says why.
Also drops a one-line docblock stranded above it, describing a method that is no
longer in the file.
`listTeamsForUser` returned every row and the controller never read `req.query`,
so `limit`, `cursor` and `includeTotal` were accepted and ignored on the one
team route that did not paginate. The docs promise a capped page there, and
every sibling listing already honours it.
Keyset-paginated on `id` per doc/pagination.md, with the envelope the other
routes return. The app-actor directory filter still runs over the page, so an
app sees the same teams it saw before, just a page at a time.
dependabot[bot]