185 Commits

Include renames

Author	SHA1	Message	Date
Juan Fernando Castro	e32bfa839d	fix: count one app open per account per app per hour (#4152) ... * fix: count one app open per account per app per day `app_opens` is read two ways at once: `open_count` is a row count and `user_count` is distinct accounts. A repeated open moves the first without the first moving the second, and the incentive report prices opens. An open is now marked as it is recorded and skipped for the rest of the day, before both the row and the event ClickHouse ingests, so the two stores agree. The marker is set only when absent, so the first open of the day wins a race; an unreachable marker counts the open, since counting twice beats not counting. * fix: count one app open per account per app per day, at the count Dropped the row-suppression approach. `app_opens` is not only a counter: `getRecentAppOpens` orders the desktop's recent list by `MAX(_id)`, and `app.opened` is the socket event other tabs reorder on. Suppressing the repeat row froze both at the first open of the day. Every open is recorded as before. `open_count` counts each account once per day instead, which is the figure `user_count` is read against. `countDistinctUserDays` folds the pair into one number because `COUNT(DISTINCT a, b)` is MySQL-only, and `dayBucket` covers MySQL's decimal division. ClickHouse has uniqExact over the tuple. Note the figure's meaning changes at deploy: a period spanning it mixes rows with account-days, so a comparison across that point is not one. * fix: count one app open per account per app per hour Review feedback: a day is too long a window for the repeat to collapse into. The period is now an argument rather than a baked-in 86400, held in one constant on AppStore beside the other store policy numbers, so the SQL and ClickHouse sites cannot drift and a future change is one line. The fold `COUNT(DISTINCT user * N + bucket)` is only injective while the bucket index stays under N. Hourly buckets are 24x the day index and pass the old 1000000 in 2084, so N moves to 10000000. Buckets are clock hours, not a rolling window, so two opens a minute apart either side of the hour still count twice — tested, and the same property the day version had at UTC midnight. The detailed-stats fixture gave its repeat open a 60s offset, which under hourly buckets would have counted twice whenever the suite ran in the first minute of an hour. It now shares a timestamp with the open it repeats.	2026-10-14 11:54:17 -04:00
Daniel Salazar	ff815f730d	fix: harden token delivery, session revokes, app identity and fs scoping (#4229) ... - relay sign-in popups take the opener origin from the referrer only - session revokes read the primary, batch per user, and tombstone caches in every region - wisp relay tokens bound to the minting session; verify rechecks the account - hosted origins keep app identity with the site owner; deleted app uids recorded in deleted_apps - subdomain cache tombstoned on delete, update, reclaim and app/account cascades - email lookups require an exact address match after the SQL lookup - workers deploy only from files the deployer owns - share_app_data:false enforced at ACL time, plus copy/move/rename/share/recursive listing - kv remove returns no value without read; kv:expire op grant usable - fs errors name the entry instead of the owner's path - shared-by-me drops rows a delegate no longer has authority over - private apps can't move index_url onto another user's origin - GUI: default apps saved and launched by uid; save picker accepts only copy/move; permission dialog waits for focus; Publish as Worker hidden on others' files	2026-10-14 06:56:44 -07:00
404oops	f9cde23937	Merge branch 'filipkujundzic/put-1954-elevenlabs-tts-passes-voice-names-through-as-voice-ids-so' into filipkujundzic/put-1907-ai-streaming-buffering-and-error-shape-cleanup ... # Conflicts: # src/backend/drivers/ai-chat/utils/inlineImages.test.ts # src/backend/drivers/ai-chat/utils/inlineImages.ts # src/backend/drivers/util/fileInput.ts	2026-10-12 12:39:35 +02:00
Daniel Salazar	d8f9aff3dd	fix: follow-ups to #4161–#4203 (rate limits, broadcast, AI routing and billing, fs subtree walks) (#4205) ... * fix(rate-limit): count the newest KV rate rows first The KV backend's rate check, peek and concurrent acquire list the key's rows with a limit + 1 cap that the store applies before dropping expired rows, so unswept rows from past windows (oldest keys) could fill the fetch and the live count read as zero. They now list newest first. * fix(broadcast): split webhook flushes under a byte budget, per-peer backlogs Flushes go out in bodies of at most `broadcast.outboundMaxBodyBytes` (default 4 MiB) instead of one POST, and each peer keeps its own backlog of what it missed, capped by count and four bodies' worth of bytes, oldest dropped first, so one failing peer can no longer grow every peer's body past their 50 MB limit. * fix(ai-chat): classify dropped upstream connections as provider failures A connection reset or refusal (SDK APIConnectionError, Mistral's ConnectionError, an undici reset on the cause) carries no status, so a chain of them fell through to 500 internal_error. Attempts now record connectionFailed; a chain of only resets and 5xx answers 400 upstream_provider_unavailable, and a reset counts as an upstream signal in a mixed chain. * fix(ai-chat): keep SDK retries on Anthropic count_tokens The Claude client is built with SDK retries off so the driver can fall back, but count_tokens has no fallback route. It now passes sdkRetryOptions, like the OpenAI moderation call, so a transient 429 or reset is retried. * fix(ai-chat): mark a route unhealthy for 60 s after a transient failure With SDK retries off, one 5xx, 429, timeout or dropped connection kept a route marked for 10 minutes, sending that model's traffic to resellers. Those failures now mark the route for 60 s; bad credentials, an exhausted account and other route-level failures keep 10 minutes. * fix(metering): don't memoise spent addons inside the force-exact window The spent-addons memo could cache a pre-grant record read from a lagging replica right after a credits-changed announcement, so overage skipped the new credit and requireCredits kept answering 402 for up to 15 s. A spent record is now only cached once the actor's force-exact window has ended. * fix(share): send the digest entries a flush claimed when another take fails Digest entries are taken concurrently, and a single rejected take threw away every entry the other takes had already removed from KV. The flush now logs the failed takes and sends what it claimed; an entry whose take failed stays queued for a later flush. * fix(fs): announce an overwriting move's replaced entry only when it took its name /move and v2 /fs/move looked up the entry at the destination name before moving and always announced its removal. A move into someone else's Trash renames instead of replacing, so the owner's desktop hid a live entry and legacy /move reported it as `overwritten`. Both routes now pass the target on only when the moved entry kept the name it was looked up by. * fix(ai): name the vendor in upstream error fields from media providers upstreamFetch now takes the provider id plus a display label, and UpstreamResponseError carries the id. translateProviderError copies a string `provider` off the error into `fields`, and the BytePlus video and Cloudflare image errors set one, so upstream_auth_failed and upstream_credits_exhausted alarms say which vendor failed again. * fix(teams): delete a provisioned seat whose home folders could not be created provisionUser logs a failed default-folder creation and still returns the account, so provisionAccount went on to add the membership, issue a password, mail the seat and emit team.account.created for an account with no home. It now cascade-deletes the new user and returns 500 internal_error before any of that. * fix(ai-chat): load model catalogs without holding up server start onServerStart awaited every gateway catalog (up to its 10s fetch timeout) while HTTP was already listening, so drivers started after it had no providers yet and chat requests found an empty model map. The first build now runs in the background; complete(), models() and token counting wait on it instead of failing. * fix(ai-chat): serve direct models while gateway catalogs load at boot The first model-map build now publishes a fresh map as each provider's catalog lands, so static catalogs are routable at once. complete() and token counting wait for the full first build only when the model, or the provider asked for, isn't in the map yet. Refreshes still swap once. * fix(ai-chat): split gateway-billed cost across input and output A completion billed at the cost OpenRouter, Infron or Neuralwatt reports was quoted to `ai.prompt.cost-calculated` as all input and no output. The ledger still records one `billedUsage` line; the reported per-key costs now spread that bill over the usage keys at the catalog's relative rates, in whole µ¢ that sum to it. Neuralwatt energy units take no share. * fix(ai-image): hold Replicate credits for the whole prediction The output and input-surcharge holds took the default 10-minute TTL, the same as the prediction window alone, so a slow prediction could outlive them once creation and input measurement were added. They now last 15 minutes, like the video job holds. * feat(ai-chat): report what a completion was charged as usage.cost Chat usage (the non-stream result and the stream's final usage line) now carries `cost`: the µ¢ the ledger records for the request after the cost factor, from the request's recorded charges and the gate's factor lookup. `usd_cents` is unchanged. A hooked recording now copies the usage object first, so keys the driver adds later never land in the ledger. * fix(ai): charge streamed speech once its first byte arrives Per-character TTS and the voice changer recorded their charge when the upstream's response headers arrived, so a body that failed before any audio was billed in full. A shared relay now charges on the first byte, or when the caller hangs up, and charges nothing for an upstream that fails or ends before sending audio. * fix(ai): release credit holds only after the request's usage lands A hold was released as soon as its operation returned, while the usage it stood for was still being written (behind the cost-factor lookup), so a concurrent request briefly saw neither. AI metering now tracks the request's usage writes and its holds (chat included) release once those settle, without delaying the response; a streamed charge chains its own. A failed write still releases the hold, and the hold's TTL backstops it. * test(ai-chat): wait for a stream's credit hold to come back after its usage lands * perf(fs): page subtree delete and copy by parent link, not by depth over the path range Each page of a recursive delete or copy sorted the whole remaining subtree by a computed depth, so a walk cost grew with the square of its size. It now pages the children of up to 100 sibling directories at a time by parent_uid, depth first, keyset on (parent_uid, id), so a page reads only its own rows; rows with no stored path are reached too. SQLite gets the parent_uid index the others have. * fix(fs): find children by parent_uid in the empty check and supplementary-name lookup Rows the old backend wrote carry parent_uid and no parent_id, so a non-recursive delete of a directory holding only such rows passed the empty check and orphaned them, and on MySQL the supplementary-character child lookup missed them, leaving their subtrees out of listings, counts and renames. Both now key on parent_uid; the empty check still probes parent_id too, which the delete cascade follows. * docs(fs): drop the single-POST note from the entry broadcast cap	2026-10-12 01:10:54 -07:00
Daniel Salazar	ecef154630	refactor(services): page fs subtree walks, gate share fan-out, one fs ACL helper (#4190) ... * refactor(acl): read ancestor chains from FSEntryStore and share one fs ACL helper getAncestorChain (split a path, batch-read the entries) moves to FSEntryStore, with getAncestors for callers that want the rows. Services read it from the store instead of calling FSService sideways; FSService.getAncestorChain stays for drivers and extensions. ACLService gains fsDescriptor (memoized chain), assertFsAccess, assertFsCreate and refusal, plus the pure aclRefusal mapping. The eight hand-rolled descriptor and safe-error copies in FSService, ShareService, events/authorization, FSController, legacyFsHelpers, LegacyFSController, AuthController and WebDAVController use them. Each route keeps its status, legacyCode and message: fs routes answer access_denied on a 403, the legacy routes still hide refusals from apps as 404, share and events keep forbidden and their own 404 text. * perf(fs): page recursive deletes and batch their event dispatch A recursive delete read every descendant into memory, and an empty-check for a non-recursive one did the same. It now walks the subtree a page at a time in descending path order (children before the directory holding them, so the parent_id cascade never takes a row before it is handled), deleting objects and rows and publishing each page as it goes. A non-recursive delete probes with one LIMIT 1 query. Removals publish once per entry, after the row is gone (the same change as the open fix for the double emit), with one surviving-ancestor chain read at most once for the whole subtree. Events-API dispatch for a batch runs as one background task at bounded concurrency instead of a fire-and-forget promise per entry. FSEntryStore gains hasDescendants and listDescendantsInOrder (keyset paged by path, then id). * perf(fs): copy directories a level at a time A directory copy walked every descendant in memory and copied them one by one: an insert per directory and, per file, an S3 CopyObject, a batchCreateEntries of one and its read-back. It now pages the source in ascending path order (parents before children) and takes each page a depth at a time: rows for directories, links and empty files at bounded concurrency, then the depth's S3 copies at bounded concurrency and one batchCreateEntries for all of them. A single-file copy goes through the same path. A failed S3 copy still fails the call (a ghost file still answers 404), but only after the objects that did copy have their rows, so nothing is left in the bucket unreferenced. Copied entries publish per depth in one dispatch. * refactor(share): build masked share paths in one place sharePathMask exports maskedRoot(owner, uuid, name) and maskUnder(root, realPath). ShareService's maskedPathVia goes, its maskedSelfPath and shareDeepLink's maskedSharePath build on them, and the masker, maskUnderAnchor and learnShareRoots use maskedRoot. Output is byte-for-byte the same; tests pin it. * fix(share): send old_path on a recipient's item.moved The desktop's item.moved handler reads old_path; ShareService sent the former masked path only as from_path. Both are sent now. * perf(share): skip the audience lookup for owners with nothing shared Every fs write, create, rename, move and delete looked for share recipients: an ancestor walk plus two share queries, for an owner who in nearly every case has shared nothing. #sharesReaching now reads the chain through FSEntryStore.getAncestors (no third copy of the walk) and asks a per-owner "has anything shared" flag before the share queries. The flag is the per-user cache events already keeps for subscriptions (SubscriptionCache): one ShareStore.ownerHasShares read per owner per TTL, coalesced across a burst. Creating a share (user, team, claimed invite) or moving a subtree to a new owner bumps it locally and over outer.pubsub.share.ownerShared; the TTL covers a process that misses the bump. A failed lookup reads as shared, so the full lookup decides. * perf(share): batch the per-row reads behind share listings Share listings looked things up one row at a time: - teams by id: TeamStore.getByIdsIncludingDeleted (the single-id read calls it) for listSharedByMe, listSharesOf and #revokeGroupSharesBy - team shares per node and ancestor: ShareStore.listGroupOnFsentries, one query for the chain, in the same node-then-ancestor order - #liveGroupGrants: PermissionStore.queryIssuerGroupPermsByPrefixes reads every issuer/group pair under every prefix in one query - #revokeGroupSharesBy: the five canManagePermission checks per row run together instead of one after another - ShareNotificationService #unblocked: users via getByIds and blocks via UserBlockStore.blockersAmong, two queries instead of two per member - digest flush: entries are taken at bounded concurrency rather than one at a time * perf(events): ask background consent once per holder and app on a revoke settle Withdrawing events:background re-checked consent row by row, resolving the holder's actor and asking the permission service for every one of their background rows. Rows of one holder and app stand on one consent, so it is asked once and shared. * refactor(events): drop the registry's unused push fields notify, defaultDelivery, PushMessage, NotifyProjection and pushProjection were read only by registry.test. Every subject was unpushable and broadcast by default; the subscription's delivery class stays in DeliveryClass, which the stores use. * refactor(share): retire trashed shares through onEntryOwnerChanged onEntryTrashed was a copy of onEntryOwnerChanged: the same subtree share sweep, grant retire and row delete. * fix(fs): page subtree walks by depth, not path MySQL sorts a PAD SPACE string on its first max_sort_length (1024) bytes, so under utf8mb4_unicode_ci paths past about 512 characters tie and fall back to id. A file moved under a newer directory then sorted after it: the delete walk reached the directory first and the parent_id cascade took the file without its object delete or event, and the keyset skipped or repeated rows. Pages now order by segment count, then id, which no collation can reorder. * fix(share): cache an owner as shared when the bump arrives A bump only dropped the cached answer, so the next lookup could read a replica that hadn't applied the new share yet and cache "nothing shared" for another TTL. The bump now records "shared" outright. * fix(fs): probe a directory for children by parent_id, from the primary * fix(fs): keep the copied-size check in the batched directory copy A copied file records the size CopyObject actually duplicated; growth past the source row is charged against the allowance, and a copy that no longer fits is deleted and refused, as a single-file copy already does.	2026-10-11 21:54:13 -07:00
Daniel Salazar	c815dc82a1	refactor(core-util): shared bounded map, cheaper route lifecycle, DDB/DB/rate-limit/gate dedup (#4185) ... * refactor(util): add BoundedTtlMap for core caches and log throttles One size-capped, LRU-evicting map with an optional TTL and a once-per-window shouldEmit helper. Replaces the hand-rolled maps in the DynamoDB throttle and repair loggers (the repair one was unbounded), the Slack repeat throttle, the auth probe's reauth log throttle, and the card-verification status memo. * perf(http): skip route lifecycle work when nothing listens The per-route lifecycle middleware built a payload, opened an event.emitAndWait span and hooked finish/close on every request, listeners or not. Each phase now checks hasListeners first. EventClient's three copies of the key-prefix walk become one memoized #matchKeys. * perf(ddb): write batch chunks concurrently and retry the TTL sweep batchPut/batchDel sent their 25-item chunks one after another; they now run up to four at a time through runWithConcurrencyLimit, and a failed chunk stops the ones not yet started. batchGet and the batch write share one #sendWithUnprocessedRetry instead of two copies of the backoff loop, and the TTL sweep goes through #batchWrite, so the unprocessed deletes it used to drop are retried. The AWS and local clients are built by one #bindClient. * refactor(db): share migrations and tryHardRead in the base client MySQL and Postgres each carried a copy of the migration-path runner and of the replica-racing tryHardRead; both now live on AbstractDatabaseClient (applyMigrationPaths, and tryHardRead gated on hasReadReplica), and SQLite drops its own tryHardRead override. Removes MySQL reinitPrimary, reinitReplica, isRetriableError and readWithRetry and the base requireRead, which had no callers outside tests. SQLite prepared statements and the Postgres SQL rewrite are memoized per query text in a bounded map. * refactor(rate-limit): route every entry point through one charge() The nine rate/peek/acquire entry points each repeated key and limit resolution, the backend call and the fail-open catch. They now share charge(), routeKey() and guardSlot(); log lines are unchanged. Driver concurrency slots release at most once like the other slot handles, and checkRateLimit takes { failOpen: false } for limits that must hold when the backend is down. * refactor(util): add secretsEqual for constant-time secret comparison One HMAC-then-timingSafeEqual helper, safe for inputs of any length, so callers stop hand-rolling the length guard. File signature verification uses it. * refactor(fs): infer signed-file MIME types with mime-types mimeFromName looks extensions up in mime-types instead of a 35-entry inline table, keeping the two types the table reported differently (ico, wav). The purpose-labelled signing key is derived once per secret instead of on every signature, where signFile derived it twice. * refactor(http): build route gates from one gate() helper Each gate repeated a try/catch -> next(err) wrapper or a .then(next, next) chain around an assertion. gate(check) does that once: a throw or rejection goes to next(err), and a synchronous check still calls next synchronously. The credits, subscription and reputation gate files held nothing but such a wrapper; their factories move into gates.ts and the files go. * perf(hosting): build the puter-site hosting config once createPuterSiteMiddleware rebuilt the normalized hosting-domain config on every hosted-site request; it is derived from static config, like the domain lists the middleware already builds at construction. * fix(db): tryHardRead's replica leg doesn't fail over to the primary MySQL's read() retries on the primary after a load-shed or transient replica error, so routing tryHardRead through it read the primary twice on that path. The replica leg now reads the replica alone. * fix(ddb): settle in-flight batch chunks before rethrowing A failed chunk stopped unstarted chunks but rejected while others were still retrying, so writes could land after the caller had handled the error and invalidated its cache.	2026-10-11 21:53:33 -07:00
Daniel Salazar	906d6b7886	fix(email): one canonicalization and blocklist for every address path (#4176) ... Share invites, share notifications and app feedback checked addresses through EmailClient.clean/validate, a second copy of the rules in util/email.ts that stripped dots on iCloud, stripped `+tags` on every domain, and matched the blocklist case-sensitively. They now use util/email.ts: isBlockedEmail for the blocklist, abuseKey for the per-address notification budget, and cleanEmail for the invite digest key. EmailClient keeps transport and templates only. isBlockedEmail also lowercases the configured suffixes, so a mixed-case blocklist entry still matches.	2026-10-11 18:24:28 -07:00
Daniel Salazar	cf9df8719a	fix(alarm): forget idle alarms and cap how many the client holds (#4171) ... AlarmClient kept every alarm id it had seen for the life of the process, and ids that embed a user, key or invoice grew it without bound. The registry is now ordered by last occurrence: alarms idle for 24 hours are dropped on the next create, and past 1000 the least recently raised goes first. Clearing or evicting an alarm only removes its short-id alias when the alias still points at it.	2026-10-11 17:54:26 -07:00
Daniel Salazar	b67acdbf6f	fix(stores): return written rows without reading them from a lagging replica (#4166) ... * fix(stores): read rows back from the primary after writing them ShareStore (create, upsertPending, upsertActive, upsertActiveGroup, applyPending, upsertAnyone), AppStore (create, createFromOrigin, update, claimOwnership), NotificationStore.create, SubdomainStore.update and NotificationDriver.create returned the row they had just written by reading it back through `db.read`, which goes to the replica. Under replica lag that came back null or stale. Those read-backs now go to the primary, through a `primary` option on the getters. AppStore caches the primary copy on create, and claimOwnership no longer reads through getById, which cached the replica's pre-claim row on a miss. * fix(stores): skip the notify read-back; read OIDC link conflicts from the primary NotificationStore gains `insert`, which writes and returns the uid; `create` is now `insert` plus the primary read-back. NotificationService .notify ignores the row, so it uses `insert` and no longer pays a primary round trip per recipient. NotificationDriver.create still reads the row back from the primary. OIDCStore.link reads the conflicting row from the primary after a unique violation, so a replica that has not seen another account's link can no longer make the call report success. * perf(stores): return written rows instead of reading them back ShareStore.create, NotificationStore.create, AppStore.create and AppStore.createFromOrigin build the row they return from the values they inserted and the insert id, instead of reading it back from the primary. The DB-defaulted creation timestamp is now written explicitly from unix seconds (`timestampFromUnixSql`), so the returned value matches what a read returns (`timestampValue`); postgres INSERTs add RETURNING id. The AppStore rows fill every column the INSERT leaves to the schema and are cached as before. ShareStore.upsertPending's re-invite branch selects the full pending row in its dedupe read and returns it with the new mode and data; it falls back to the primary only when the UPDATE matched nothing. The remaining primary read-backs stay and say why: the share upserts and the new-invite insert (an upsert's surviving uid/created_at and the entry-owner subquery are only known to the primary), applyPending and claimOwnership (only an id or uid in hand), AppStore.update and SubdomainStore.update (the base row is a replica copy that would be cached), NotificationDriver.create (notify returns only the uid) and the OIDC link conflict.	2026-10-11 17:51:38 -07:00
Juan Fernando Castro	50bcdc9c6c	fix: let a revocation be seen by the reads and the nodes that outlive it (#4151) ... * fix: let a revocation be seen by the reads and the nodes that outlive it Four places where a revoke lands on the primary and the next reader can still be told the old answer. The u2a cache key was dropped but not re-filled, so the next miss read a replica that may still carry the row and cached it for the TTL -- and a revoke's own settling pass is usually that miss. It re-fills from the primary, as the u2u side already does. A revoked session dropped its keys the same way. It now leaves the revoked row in the cache, read from the primary, so there is nothing for a lagging replica to re-introduce. A link share is its own authority rather than an index over a permission, so a replica still carrying a withdrawn row answers with it. For a short window after one is withdrawn, those reads go to the primary. Socket eviction rode a local event, so it only reached the node that did the revoking. It is announced on the fan-out channel too, which is where the node actually holding the connection hears it. * refactor: carry the revocation on outer.* instead of outer.pubsub.* Eviction runs through `io.in(room).disconnectSockets()`, which goes via the socket.io Redis adapter a cluster shares, so one node handling the event disconnects that user on every node in it. Fanning out to siblings had each of them repeat the same cluster-wide disconnect. The adapter's Redis is per-cluster, so peer clusters still need the hop -- `outer.*` reaches them and skips the sibling fan-out. The local `auth.sessions.revoked` goes with it: `outer.*` fires on the emitting node too, and nothing else subscribed.	2026-10-09 16:36:26 -04:00
Juan Fernando Castro	7de8360a6e	Sharing cleanup: bounded outbound listings, and what see does not reach (#4118) ... * fix: keep a file name to one line in a subject, and the inbound total to the account **A name reaches the digest subject as the user typed it.** Subjects compile with `noEscape` on purpose -- a header is not HTML, and escaping would show entities to the reader -- and the comment there says free-form values collapse whitespace upstream. A file name does not: `#assertUsableName` refuses only an empty name, a slash, `.` and `..`, so a name may carry newlines. It is collapsed to one line and bounded before it goes in. **`includeTotal` answered an app with the account's figure.** The page is filtered by what the caller can reach -- an app sees only what its user handed it -- while `countByHolder` counts every inbound share. The docs already call `total` an approximation for the filtering; they did not cover an app reading the size of everything else. It is answered only for a caller acting as the account now, and the page documents that. The team-share fixture built its actor as `{ user }` with no `effectiveApp`, which reads as an app rather than a session. A session acts as the account, so it says so. * fix: stop `see` reaching inside a folder, and bound mode changes **A `see`-only folder recipient was pushed every child's name, size and type.** The fan-out picked holders by reach -- any live grant on a reaching root -- and `see` is a grant. But `see` says a node is there; `list` is what says what is inside it, and the payload carries exactly what `list` would: name, path, `is_dir`, type, size, modified. Descendants now go only to a holder whose mode reaches inside. A change to the shared node itself still goes to everyone who holds it, which is what `see` covers. **Mode changes skipped the daily quota entirely.** Not spending share budget is right -- changing a mode is not new reach, and the existing test says so -- but unbounded it lets one account write a pair as fast as it can ask. They have their own budget now, ten times the share limit by default and configurable, so the exemption stays an exemption from *that* budget rather than from any. * fix: let a half-finished downgrade be finished, and stop Trash answering **A mode downgrade that failed between its two halves could not be repaired.** `setUserUser` grants the new mode and then revokes the superseded ones. If the revokes did not all land, the higher grant stood -- and the retry returned early, because the mode it was asked for was now present. "Nothing to do" now means this mode is the only one standing, so asking again is what finishes it. The same two halves, and the same early return, were in the group variant. **Trashing someone else's item answered for what is in their Trash.** Moving into the owner's Trash is deliberately authorized without write on it -- that is what lets a recipient delete from a shared folder -- but a name collision there came back as `An entry already exists at /owner/Trash/<name>`. The mover picks the name, so that is a question they can ask about a folder they cannot list. It dedupes instead, which is also what the GUI wants: trashing should not fail over a name. * perf: record the entry's owner on the share row The outbound listings answer "shares on things I own that someone else issued" by joining `share` to `fsentries` and filtering on the owner there. Neither side of that join bounds the work: driven from `fsentries` it walks every file the user owns, driven from `share` it walks every share until it has a page. For a user who owns a lot and delegates nothing, that is the whole table for an empty result. `#outboundAppsSql` is worse than the ticket says -- it has no LIMIT at all, so every call aggregates the join across both tables. The owner now rides on the row, indexed with `id`, so all three read it directly. An entry cannot change owner -- a move out of the owner's tree is refused -- so it is written once, with the row. Set in SQL from `fsentry_id` rather than threaded through callers: four insert sites, each of which already has the id, and none of which can now forget. The risk in denormalising is a write path that does: a row with no owner recorded is a share that silently stops appearing in its owner's listing. The new test covers exactly that -- it fails when one insert is made to write NULL. Verified against a real boot: a pre-existing share row, on a database that genuinely predates the column, comes out with the owner its entry has. * fix: the review's findings on this branch **The Trash collision still had both halves open.** `overwrite` was checked before the trash case, so a caller setting it -- it is wire-supplied -- could still delete whatever it collided with in the owner's Trash, and the 409 survived for anyone who did not. The trash case is decided first now. Dedupe alone did not close it either: a name that changes only when there is a collision answers the same question the refusal did, and the mover sees that name even though the path is masked. Trashing someone else's item renames it every time, so the result carries no signal. `original_name` already records the real name, which is what a restore reads. Two tests pinned the old name; they say what it is now. **The owner column had no fallback.** Reads went only through it, so a row written by anything that predates it -- a pod mid-roll -- would never appear in its owner's listing again. The three reads now also cover `IS NULL` rows through the old join, indexed on the null and empty once none remain. **The grant became conditional on a replica read.** `if (!already)` skipped it on a stale row while the revokes below stayed unconditional, which could leave the holder with nothing where the old early return left their access alone. It grants unconditionally again; only the early return changed. **The re-issue budget covered one path of four.** Team, link and invite re-issues reserved nothing, so three ways round it were left open. They share one `#reserveRemodeQuota` now. Two docblocks had been pushed off the functions they describe, and two comments still described the join that is gone. * chore: one-line the comments added here * test: measure the prefix walk by what it keys on, not by the clock `checks many long paths without re-reading every prefix of each` gave the walk 1500ms of wall clock. It measured nothing useful: the shape it guards against — keying each step on the accumulated prefix text rather than on the parent id — costs 29ms against the current 8ms on that input, so both ends sit far under the bound. Under coverage the same test ran 1511ms and failed, which is the only thing the number could ever report on. `PathPrefixIds` now counts the characters it keys on, and the walk is asserted on that: proportional to the input, and twice the depth costs twice as much rather than four times. Keying on the accumulated text fails both. The original case keeps its real assertion, that a request this size is refused. * fix: keep the recorded entry owner current, and the trash guard on the retry A move into someone else's tree re-owns the whole subtree, which the migration comment claimed could not happen. The shares on it are re-pointed at the new owner, or they drop out of the listing the column keys and stay in the old owner's. The unique-violation retry re-entered the collision resolver without the trash flag, which is the one call site the guard was not threaded through.	2026-10-09 16:14:23 -04:00
Juan Fernando Castro	cf574e4b34	Merge pull request #4110 from HeyPuter/juancastro/icloud-dot-significance ... Treat dots as significant on Apple's mail domains	2026-10-07 17:11:59 -04:00
Daniel Salazar	2e275a8f6b	fix: godmode installation (#4116) ... * fix: godmode installation * fix: godmode classifier * fix: vitest	2026-10-07 13:16:10 -07:00
Juan Castro	0555ea6479	Merge branch 'main' into juancastro/icloud-dot-significance	2026-10-07 11:01:12 -04:00
Juan Castro	4b5f509066	fix: only rewrite rows the old rule actually wrote, and fix the invites too ... Two findings from review, both of which contradict what the last commit claimed. **The backfill could merge keys, not only split them.** I asserted it "can only split rows apart, so it cannot collide". Not true, and reproducible: a row predating the `clean_email` column has a NULL there, so its index key is `lower(email)` -- filling it in folds `+tag`, and `john+news@me.com` lands on a key `john@me.com` already owns. A legacy row holding an un-lowercased `clean_email` collides the same way once lowercased. Either throws, and the two engines that re-run every file on every boot would then refuse to start until someone operated on the database by hand. Both updates now fire only where the stored value is exactly what the old rule produced and differs from the correct one. That makes "splits only" true: the sole change is dots coming back. NULLs and legacy values are left alone. The same predicate makes them write-idempotent. Postgres has no no-op-update suppression, so without it every boot rewrote a tuple per Apple row forever. **The pending-invite residue misdelivers rather than just going missing.** `claimPendingShares` looks the row up by the canonical address, so whoever confirms `jsmith@icloud.com` claims the invites addressed to `j.smith@icloud.com` -- someone else's file, handed over on confirmation. I had written this off in the PR as unfindable invites that "expire on their own"; `share` has no expiry column and the lookup has no time predicate, so they are claimable indefinitely. Fixed here rather than later: the typed address is kept as `invitedAddress` whenever it differed from the canonical, which is exactly the collapsed rows, so the invites recompute from it under the same guard. Verified by seeding a database at the old version with the two rows that used to abort the migration, and booting: it reaches version 87, repairs the collapsed row, and leaves the NULL and legacy rows untouched. Replaying the statements changes zero rows.	2026-10-07 10:23:30 -04:00
Juan Castro	ece8ad5f1d	fix: register the Apple-domain backfill with the sqlite runner ... The sqlite migration list is hand-maintained, so the file alone did nothing — a fresh boot still reported 87 files and stopped at version 86. MySQL and Postgres read their directories, so only this one needed the entry. Caught by booting a local instance rather than by the suite: the client tests pin `CURRENT_SCHEMA_VERSION`, which fails once the entry exists but says nothing while the file is simply unregistered. Verified against a seeded database at the old version. `John.Smith@iCloud.com` went from `johnsmith@icloud.com` to `john.smith@icloud.com`, `a.b+tag@me.com` from `ab@me.com` to `a.b@me.com` — subaddressing still folded — and a gmail row was left alone.	2026-10-07 09:45:26 -04:00
Juan Castro	cf817b0f0a	fix: treat dots as significant on Apple's mail domains ... `cleanEmail()` applied gmail's `dots_dont_matter` rule to icloud.com, me.com and mac.com. Apple allocates the exact string: `j.smith@icloud.com` and `jsmith@icloud.com` are two mailboxes with two owners. Collapsing them made one canonical address stand for both. What that canonical address feeds: - `findEmailOwner` falls back to it, so an OIDC sign-in whose verified claim is the dotted address resolves to the account holding the undotted one, and `linkProviderToUser` has nothing left to object to -- the provider verified a real mailbox and the account's address is confirmed. - The unique index on COALESCE(clean_email, lower(email)) reads it, so the rightful owner of a dotted Apple address is refused signup because the undotted account already "owns" it. - Pending share invites are stored and claimed on it. Subaddressing is untouched: Apple does fold `+tag`, so that rule stays. Gmail keeps both rules -- there the fold is correct. A migration recomputes `clean_email` for those domains on all three engines, since rows written before this still hold the collapsed value and the index reads the stored one. Verified the SQL against the function rather than by inspection: same five inputs through `cleanEmail()` and through sqlite give byte-identical output. It can only split rows apart, never merge them, so it cannot collide with the unique index. Not changed: `findUserByEmail` still matches on the canonical address. The cross-match it documents is correct where the provider really does fold -- `foo.bar+tag@gmail.com` is the same inbox as `foobar@gmail.com`. Requiring an exact match would break that for no benefit here; the fix is for the data to say what the provider does.	2026-10-07 09:17:03 -04:00
Juan Castro	6efb96d560	Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup	2026-10-07 09:00:40 -04:00
Daniel Salazar	11e48c68a9	fix: verification for sms (#4101)	2026-10-06 22:03:07 -07:00
Daniel Salazar	464f758a97	fix: verification for sms (#4100)	2026-10-06 22:00:14 -07:00
Daniel Salazar	d2cdde5147	fix: verification for sms (#4098)	2026-10-06 19:36:18 -07:00
Juan Castro	ad1be70d06	Merge branch 'juancastro/flat-permission-view-scope' into juancastro/permissions-access-control-cleanup	2026-10-06 19:44:37 -04:00
Juan Castro	b8fcbffe3f	fix: the five remaining review findings ... **The path oracle was not closed.** The rewrite only remapped the *error*, so success-vs-403 still answered whether a foreign path exists. Narrower than reported: every other grant path already gates on `canManagePermission`, and only `grantUserAppPermission` has no authority check -- deliberately, since granting your own app something you don't hold is inert. It now refuses a foreign path the actor cannot see, with the same error a missing one gives. My first test for this passed without the fix: `makeUserActor` creates no fsentries, so both paths 404'd and nothing distinguished them. It inserts a real entry now, and the control fails with "expected null to match 403" -- null being the grant that succeeded, which is the oracle. **`#validateEmail` still failed open** on the main signup / change-email / confirm path while its OIDC twin was hardened. Same shape as OIDC now: a gate that did not run is not a pass. **The 16-permission cap broke `puter.perms.check()`** for longer lists. The SDK's own docblock promised "one round trip however many are asked about", which the cap made untrue, so it batches instead. The reported second half -- the UI grant poller burning its timeout -- is not reachable: `MAX_REQUESTED_PERMISSIONS` already caps that path at the same 16. **`listener_failed` was set by wildcard listeners.** An unrelated observer on `email.*` throwing would refuse every signup. Only an exact-key listener implements the event, so only that one speaks for the gate. **`whoami()` labelled everything `Unauthorized`.** A 502 with an HTML body is not a 401; the server's `code` now rides along and the fallback names the status.	2026-10-06 19:23:40 -04:00
Juan Castro	25f132b28d	fix: address code review on the teams cleanup ... Six findings, all real. `total` described rows the page withheld. The count ran over every live team the user belongs to, while the app-actor filter ran afterwards on the items only, so `GET /teams?includeTotal=true` from an app returned `items: []` with a non-zero `total` -- the exact thing the filter exists to prevent, and newly reachable because `includeTotal` was inert before this branch. The directory-open restriction moves into the query, so items, cursor and total all describe the same set. The GUI told owners the opposite of what the server now does: "we email the username and temporary password to it." An owner who believed it would dismiss the one copy of the password and strand the seat. Rewritten. `runWithConcurrencyLimit` rejects on the first error but cannot cancel the workers beside it, so a failed seat let up to seven others finish suspending and emailing "your team has been closed" for a team that was still live. The page now settles before the rejection is rethrown, which leaves a state the retry can resume from instead of an arbitrary one. My earlier claim that failure semantics were unchanged was wrong. `#disableSeatForDelete` returned `true` on both paths, so the boolean read as if it distinguished them. It returns nothing; the counter uses the settled results. `listTeamsForUser` had picked up a second stacked docblock -- the same defect this branch removes from `#cached` one file over. `reissueCredential` sent the provisioning mail, which after the credential change was a byte-identical duplicate saying nothing about the password it had just invalidated, while the GUI promises the member hears about it. It has its own template now. Regression tests for the count leak and the reissue notice; both confirmed to fail without their fix.	2026-10-06 16:22:48 -04:00
Juan Castro	6826071156	fix: stop mailing a plaintext team credential to an unconfirmed address ... Provisioning a seat lets the owner supply any `email` that passes a format check. Puter then mailed a working temporary password to it. Nothing confirms the owner controls that address -- provisioned accounts are created with `requires_email_confirmation: false`, because the team is the trust anchor for the account, not for the mailbox. `resendActivation` reissues the credential and mails it again, up to the 20/day `team:reset-password` budget, so the same address can be hit repeatedly. The credential now stays out of the mail on both paths, which is what `resetMemberPassword` in the same file already does. Nothing is lost: all three routes return `temporary_password` to the owner, and the SDK has always documented that as the delivery -- "The returned password is shown once and is not retrievable afterwards -- deliver it out of band." The mail still goes out, naming the account and saying the team will supply the password. The two tests that pinned the old shape are inverted rather than deleted; one of them carried the rationale "without it this is the only copy", which the return value contradicts.	2026-10-06 15:56:33 -04:00
Juan Castro	3389d3f8a1	fix: treat an abuse hook that could not run as a refusal ... The signup paths read `allow` off a payload they initialise to `true`, so a harness listener that throws leaves the verdict untouched and the signup goes through. The outer try/catch the report points at is not where it happens: `emitAndWait` already swallows each listener's error and carries on, so that catch almost never fires and the throw never reaches it. `emitAndWait` now records the failure on the metadata it was handed, and both signup paths — OIDC and the ordinary one — refuse when it is set. Other emitters are unaffected: the flag is only read where somebody is asking for a verdict. The report named OIDC alone; the ordinary signup had the same hole, and it is the path that matters more. Test boots a server, registers a listener that throws, and posts a signup: 200 before, 403 now.	2026-10-06 11:13:18 -04:00
Daniel Salazar	ba081e2332	feat(ddb): log the table and key of throttled DynamoDB requests (#4078) ... A client middleware inside the SDK retry loop logs every throttled attempt (including ones that succeed on retry) with its operation, table, key, client region and DynamoDB's throttling reason, plus batch items returned unprocessed. Each target logs at most once a minute with a count of the throttles folded into it. Puts log only the key attributes, using key schemas registered by the KV store at startup.	2026-10-05 02:01:38 -07:00
Juan Fernando CastroandDaniel Salazar	a46eb15d5e	Let the account's own API token reach the routes a privileged app needs (#4032) ... * feat: let the account's own API token reach the routes it should A privileged app is launched with the GUI session token today. Running one on a full-access personal access token instead costs much less when it leaks, but several routes those apps depend on refuse every access token, so the swap would break them. Admitted, each a read or a write the token's own HTTP reach already covers: - the realtime socket handshake, which refused access tokens outright; FS live updates and notifications stop without it - GET /get-dev-profile and POST /profile, both puter.js surface - GET /auth/list-permissions — a read of what was granted, not a grant - GET /share/shared-by-me/apps and GET /share/audit, over grants /share/shared-by-me already lists for the same credential - POST /rao, which puter.js calls on every setAuthToken Gated on `isAccountContext`, not the `full_access` claim alone, so a token with an app anywhere in its chain is still refused and a socket for one never joins the user room. Left session-only: mint, session, 2FA, grant, team and billing, plus two the list named. /share/blocks is a personal safety control whose only caller is the desktop's Blocked Senders window. /app-feedback is reachable only from our own GUI pages, and being unsubmittable programmatically on a user's behalf is the property it was built with. The new HTTP suite drives a real server with a minted token and fails on all six admissions without the change; the block routes' existing metadata check already pins their refusal. * test: sort the two token shapes at the events handshake * fix: carry the account-context invariant into the HTTP gate `requireUserActorGate` admitted on the raw `full_access` claim while the socket and /rao use `isAccountContext`, so one shape — a token carrying the claim with an app in its chain, or an actor built without `makeActor` — would have been refused in two places and admitted in the third. Unreachable today, since the mint and the auth-time read both drop the claim when `app_uid` is present, but it is the invariant this change states and it should hold at every enforcement point. The payout address goes back behind a plain session: `/get-dev-profile` is open to the account's own token now, and the rest of what it answers is a developer's own name and incentive-program status. Also drops a cross-reference in EventsService to the socket posture this change relaxed, and rewords the block-routes comment, which read as if access tokens were admitted there rather than refused. * fix: drop a revoked token's sockets, and read the dev-profile columns Two gaps the review found, both reachable only because this branch lets the account's own token hold a socket and call /get-dev-profile. `#revokeAccessTokenTail` soft-revoked the session row and told no one, so a revoked token kept its connection — and the account's `outer.gui.*` fan with it — until the five-minute reauth sweep. Sockets now join a per-token room and a new `auth.access-token.revoked` event drops exactly that room, so the account's other tabs stay up. Session revoke keeps its account-wide eviction. `/get-dev-profile` read `first_name`, `last_name`, `paypal` and the two incentive flags off the user row, which carries none of them: the columns have always been `dev_`-prefixed, so the endpoint answered nulls to everyone and `puter.apps.getDeveloperProfile()` has never returned anything. Reads the real columns, keeping the unprefixed name as a fallback for a deployment carrying both. The payout address stays behind a plain session, so opening the route to the account's token hands over a name and incentive status and nothing else. Both fail without the fix: the socket test waits five seconds on a connection that should already be gone, and the profile test reads back a seeded row. * fix: close /open_item to tokens, and stop the node runner leaking sockets `/open_item` carried `allowFullAccessToken`, but it grants an app a permission on the user's behalf and mints the app token to go with it — a grant and a mint, which stay session-only. Pre-existing, and missed by the first sweep here, which only asked whether routes lacking the flag should gain it and never re-read the ones that already had it. Refused at the top of the handler too, so a credential that gets past the gate cannot leave an ACL row behind. The node runner loads each SDK into a vm context and never closed it, so every test leaked a socket, and every socket holds a per-(user, origin) connection slot. Admitting the account's own token to the handshake pushed that past the cap, and the whole events block failed with `events_connection_failed` while the browser and workerd runners — which do not leak — passed. Closing the FS socket and the events channel after each test gives the slots back. * fix: announce only the revokes that can have a socket to drop Every access-token revoke broadcast an eviction cluster-wide, including the read-URL tokens `revokeReadUrl` retires constantly. A scoped token is refused at the handshake, so none of those broadcasts could ever reach a connection. The announcement now goes out only for a token the handshake would have admitted: full access, no app in the chain. `revokeOwnAccessToken` refuses full-access tokens outright, so that whole path is silent; a revoke by raw uuid says nothing about the token and still announces, which costs a no-op broadcast rather than leaving a revoked socket up. Fails without it: the test revokes a scoped token and a full-access one through the same entry point and counts the announcements. * test: pin the mint route against the account's own token `/auth/create-access-token` carries only `requireAuth`, so it never showed up in the sweep over `requireUserActor` routes — the refusal lives in `AuthService.createAccessToken`, which turns away any access-token actor outright. That holds today, and a leaked full-access token minting itself a sibling that survives revoking the original is exactly what the token swap is meant to rule out, so it is worth a test rather than a reading. * fix: let the recommended-apps test follow the list it tests Two commits on main rewrote `RECOMMENDED_APP_NAMES` without touching the test, which seeds `editor` and `camera` and asserts the result contains them. Neither is on the list any more, so the resolved set came back empty and `test (base)` has failed on every PR since. The test now seeds the first two names off the list itself and asserts on those, so the next edit to it cannot strand the test: prepending two unknown apps keeps all four cases green. * feat: let an extension waive a plan gate for one route or driver method A plan gate about to refuse a caller now emits `subscription.gate.<surface>` through `emitAndWait` first: `route.<method>.<path>` for a route, `driver.<iface>.<method>` for a driver method, `share.anyone` for link shares. The payload carries the actor, the request and the requirement, and a listener that sets `allow` waives the plan for that call. Callers the plan already covers never reach the hook, and a listener that throws leaves the refusal in place. * feat: mint godmode apps their own full-access token `/auth/get-user-app-token` answers a desktop launch of a godmode app (the `godmode` column) with a full-access token instead of an app token. Origin lookups, the sign-in popups for pages outside the desktop, keep getting ordinary app tokens, and only a web session may mint one. The token carries the app in `godmode_app_uid`, read into `accessToken.godmodeApp` for attribution only: `effectiveApp` stays null, so it keeps the account's reach and passes the same gates as a personal access token, including the vendor-compatible AI routes. It is refused once the app is gone or no longer godmode. Its row is an `access_token` session parented to the web session that asked, with `app_uid` set, so signing out, revoking that session or revoking all sessions takes it along, and the sessions list shows it under the app. Each mint re-signs onto the same row with a 12h `exp` and moves the row's expiry out, so the desktop keeps an open app alive while a copy taken earlier still expires on schedule. A full-access token may now mint scoped tokens (read URLs), parented to its own row; it still may not mint full access. Revoking a full-access token cascades to the tokens it minted. An access token whose session row has a parent now authenticates only while that parent is live, so a read URL a godmode app minted lapses with the app's token, and through it with the desktop session. * feat: launch godmode apps on their own token and renew it The desktop asks for a godmode app's token at launch and blocks the launch if it can't get one, as it does for other apps; a godmode app never runs on the desktop's session token. While the window is open the desktop re-mints the token once less than 6h of its 12h remain, checking on a timer and when the tab becomes visible, and answers the app's `reauth_required` with a fresh one, posted to the app's origin. puter.js keeps a godmode token in sessionStorage like a session token, never in localStorage. In app mode a 401 on a godmode token, background requests included, asks the desktop for a new token and replays, and a `puter.token` message is only taken from the embedding frame. The sessions list shows a godmode token row with its app's title and icon. * fix: keep a godmode token the same across renewals Apps hand their launch token to code that never sees a replacement (the terminal's Drive mount, a CLI's API key), so a token re-signed every renewal broke those copies once the first one expired. The JWT now carries no `exp` and a renewal moves only its row's expiry, so every copy keeps working while the desktop keeps the app alive. What still bounds it: - only the desktop's web session can extend it; using the token never does, and signing out or revoking that session revokes it; - it lapses 12h after the desktop last asked, and a lapsed row is never revived: the next launch gets a new token; - however often it is renewed, a token is replaced after 7 days, and the old one is revoked on the spot along with the tokens it minted and its sockets. The desktop no longer re-posts a renewal that kept the same token, so the app's sockets aren't rebuilt every few hours; an app that reports its token stopped working is always answered. * fix: reserve the feedback and no-reply mailbox names again Signup checks names against `util/reservedUsernames`, but `fbl`, `noreply` and `no-reply` were added to a second copy of the list in AuthController that nothing reads any more, so they became claimable. Move them into the live list and drop the copy. --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-10-05 01:10:29 -07:00
Daniel Salazar	794469845d	fix(thumbnails): bind thumbnail keys to their entry and size-bound uploads (PUT-1999) (#4054)	2026-10-04 00:46:35 -07:00
Daniel Salazar	2cfc1461e4	fix: scope whoami billing details and monthly usage to the calling app (PUT-2015) (#4050) ... * fix: scope whoami billing details and monthly usage to the calling app (PUT-2015) * fix(metering): refuse an app another app's detailed usage (PUT-2015)	2026-10-03 16:38:31 -07:00
Neal Shah	04406373f4	event based FBL implementation (#3969)	2026-10-03 05:54:04 -04:00
Juan Castro	a67cdac380	fix: bound the inputs on the grant and token routes ... Three permission routes sized their work by the request body rather than by the route, so one call could cost far more than its rate-limit slot implies. - `extra` and `meta` were only type-checked. They ride every row a grant writes (up to 16) and are re-serialised into the per-(user, app) cache on each miss, so they are now capped at 4 KiB. - `/auth/create-access-token` took a `permissions` array of any length: one permission check and one sequential INSERT each. It now uses the same 16-per-request cap the grant routes already had, and runs every entry through the validator those routes use. - Withdrawing an app's cross-app data grants looks them up by permission text, and no index on `user_to_app_permissions` led with `permission`. Indexed per engine, mirroring what mysql_mig_22 / postgres_mig_11 / sqlite 0067 did for `user_to_user_permissions`. `grant-dev-app` validated none of its input — not even the type of `extra` — so it now runs the same validator as its user-app sibling. Two paths could also carry a permission wider than the `varchar(255)` column it lands in. `grantUserAppPermission` already rejected that after rewriting; `grantDevAppPermission` now does the same, and `createAccessToken` checks it before the session row a failing INSERT would otherwise orphan. Caps are published in rate-limits-and-quotas.md. Every in-tree caller sends one permission and a small `extra`, so none of them change behaviour.	2026-10-01 16:51:39 -04:00
Daniel Salazar	ed7b8f828a	fix: kv path errors and ttl hardening, events subscription end and race fixes (#4004)	2026-10-01 09:50:34 -07:00
Juan Castro	d1b11c793c	Merge branch 'main' into juancastro/put-1953-get-user-app-token-and-login-500-on-fk-failures-when-a ... One conflict, in workerInvoker.integration.test.ts: #3997 fixed the same gap-marker flake this branch did, with a different barrier. Took theirs — waiting on the `ev:qf` counter, which `#handlerFailed` writes strictly after `discard`, so the marker is in and the lease cleared by the time it reads 1. Their ordering also asserts the counter before flipping `answer` back to 200, which closes the redelivery-clears-failures hole this branch's version had removed the line to avoid. Converted the two `vi.waitFor` calls #3997 added to `waitUntil`, the helper this branch introduced for the rest of the file. Neither is wrong today, since the new block fakes no timers, but a `jump()` added near them later would hit exactly the drained-backoff trap the helper exists for.	2026-09-30 17:49:36 -04:00
Daniel Salazar	4448103ef0	fix: misc events and kv issues found during recipes (#3997) ... * fix: misc events and kv issues found during recipes * test: cover handler depth on narrowed actors and created notifications * test: fix stale legacy batch directory upload test and seat user-agent flake * test: wait for the refusal to be recorded before claiming its gap marker	2026-09-30 14:40:47 -07:00
Juan Castro	480d9d032b	Merge remote-tracking branch 'origin/main' into juancastro/put-1988-team-force-2fa ... # Conflicts: # src/backend/clients/database/SqliteDatabaseClient.test.ts # src/backend/clients/database/SqliteDatabaseClient.ts # src/backend/clients/database/migrations/mysql/mysql_mig_42.sql # src/backend/clients/database/migrations/postgres/postgres_mig_31.sql	2026-09-28 10:24:39 -04:00
Daniel Salazar	dcb61bc4fd	fix: PUT-1824 + PUT-1884 + PUT-1952 + PUT-1827 (#3951)	2026-09-25 17:47:03 -07:00
Nariman JelvehandDaniel Salazar	3d3678a555	feat: let the Contact Us form carry screenshots and recordings (#3563) ... * feat: let the Contact Us form carry screenshots and recordings The form doubles as our bug-report channel, and the bugs worth reporting are often the ones that need a picture: a visual glitch, or something that takes five steps to reach. Up to 5 images or videos may now ride along with the message, delivered as attachments on the support email. Nothing the client says about a file is believed. The endpoint takes bare base64 and re-derives all three of the things that matter from the decoded bytes: - Type, sniffed from magic numbers and matched against an allow-list of PNG/JPEG/GIF/WebP and MP4/QuickTime/WebM. A declared MIME is never read, so it cannot smuggle anything past the list. SVG is excluded on purpose (it carries script, and support tooling renders what it is sent), as are the non-video brands that share MP4's `ftyp` box -- HEIC, M4A, JPEG 2000. - The file name, reduced to a display label with the extension taken from the sniffed type. PNG bytes named `payload.html` arrive as `payload.png`; a name can never carry the CR/LF that would break out of a Content-Disposition header, nor the bidi overrides that make `report<RLO>gnp.exe` render as `report.exe.mp4`. - Size: 10 MB per file, 15 MB per submission, counted on decoded bytes. Encoded length is capped before decoding, so an oversized payload costs a length check rather than a 10 MB allocation. The total stays well under the 25 MB most providers enforce, since the outgoing mail base64s these again. Base64 is decoded strictly, reusing the round-tripping decoder that already guards app icons -- `Buffer.from(s, 'base64')` silently drops characters it does not recognise, and the round-trip is what rejects bytes smuggled after the payload. That decoder and the image sniffer move from appIcon.ts to a new mediaSniff.ts, which gains the video counterpart. One request is now worth megabytes of parsing and outbound mail, so the existing per-user rate limit gains a per-IP backstop (which the per-user counter cannot see through freshly minted accounts), a concurrency cap, and a Content-Length gate that refuses an impossible body before anything decodes it. Payloads are not stored. They ride the email; the new `feedback.attachments` column records names, types and sizes only, so an abusive submission stays attributable once the mail has been dealt with. Also fixes the form posting an empty message when Send was pressed with nothing typed, and surfaces submit failures instead of leaving the button disabled with no explanation. * fix: stream Contact Us attachments as multipart instead of base64 JSON Base64 in a JSON body went through the global JSON parser before the route ran: raw buffer, rawBody copy, decoded string, parsed strings, then decoded Buffers plus a re-encode for the strict check. A max-size submission peaked around 110-135 MB of heap for 15 MB of files, and the route's Content-Length check ran after all of it. Attachments now arrive as multipart/form-data, which the global parser skips. Auth, rate limit, concurrency and the Content-Length 413 all run before the body is read, and busboy enforces the count, per-file and total caps while streaming, so only the decoded file bytes are held (~16 MB peak). On a broken limit the reader stops and the response closes the connection. JSON stays supported for message-only posts; a JSON `attachments` field is rejected. * fix: deliver Contact Us 413s instead of resetting the upload Closing the socket on a mid-stream limit sent the 413 and then reset the connection with the client still uploading, so the client could see a reset instead of the error. After a limit trips, the rest of the body is now read and discarded, bounded by the body budget; past the budget the request is destroyed. Adds an HTTP-level test through the full middleware stack (FormData upload, unauthenticated refusal, 413 on declared length before the body is sent, 413 on a chunked upload over the per-file cap), trims comments to the AGENTS.md length rule, and drops a box-drawing test divider. --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>	2026-09-25 14:19:35 -07:00
Juan Castro	cab70254b1	feat: let a team owner reset a seat's second factor (PUT-1988) ... Forcing 2FA makes lockout materially likelier, and nothing could clear a factor except the account holding it — `resetMemberPassword` leaves 2FA alone on purpose, and admin only displays it. A seat that lost both its authenticator and its recovery codes had no way back. Its own route and its own audit action rather than folded into the password reset, so that method keeps the property its comment claims: a password reset alone is not takeover. The two together are, and for an account the team created and pays for that is the team's to do — but it has to be chosen and recorded, not arrived at sideways. Bounded to `org_owned = 1`: a joined member's own login is not a team's to reach into. The member is told, and signed out everywhere.	2026-09-25 11:15:23 -04:00
Juan Castro	db1a95260a	feat: let a team require 2FA of the accounts it provisioned (PUT-1988) ... `group.require_2fa`, toggled by the owner through `PUT /teams/:uid` and recorded in the audit log like `directory_enabled` — it changes what every seat must hold to sign in, so it is not something a team should be able to alter silently. Enforced by deriving, not by stamping a flag on the user: a stamp left behind by a member leaving, or by the rule being turned off, would lock someone out of their own account with no team left to clear it. The seat row already joins `group` and is already read per request, so it carries the flag and the gate costs nothing extra. Toggling busts those rows, or turning the rule off would leave members locked out until the TTL lapsed. The gate rides the same `allowUnconfirmed` opt-out as the other pending- requirement gates, so the routes a gated member must still reach stay reachable — `configure-2fa` among them, which would otherwise be walled off behind the very requirement it clears. Scoped to `org_owned = 1`: the owner is not a seat of their own team, and a joined member is an ordinary self-paying account whose personal login is not a team's to dictate. The owner is held to the rule at the point it matters — they cannot turn it on without 2FA of their own.	2026-09-25 10:35:24 -04:00
Daniel Salazar	132cae9f6a	Bug bash fixes (PUT-1876, 1877, 1880, 1881, 1948, 1949, 1951) and KV event fan-out by plan (#3943) ... * fix(kv): keep disableSharing entries out of other apps' events (PUT-1876) Private kv keys and values are no longer delivered to cross-app subscriptions, including forwarded and durable deliveries. * fix(fs): refuse usernames whose home path still holds another account's rows (PUT-1877) Username claims (signup, save_account, change-username, OIDC, team provisioning) now also check for leftover rows under /<name>/, and the root listing only returns the actor's own home. * fix(puter-js): reconnect the events channel after a server-side disconnect (PUT-1881) A server hang-up is retried with a bounded backoff; persistent handlers survive it and get an optional onError when the connection can't be restored. * fix(backend): let the graceful shutdown finish before telemetry exits (PUT-1948) The telemetry preload no longer exits on signals; the entry point drains, tears down top-down, flushes telemetry last, then exits. DB pools stay open until teardown. * fix(fs): reserve quota for signed uploads in progress (PUT-1880) startWrite/startBatchWrite hold each upload's declared size in a per-owner cache reservation until it completes, aborts or expires, so a burst of starts can't all read the same committed usage. Caps in-flight uploads at 10,000 per account. * fix(fs): refuse to complete a signed upload whose object never arrived (PUT-1951) Completion now 400s when the object store reports the key missing (transient errors stay lenient), abort no longer deletes an object a live entry still uses, and a ghost entry is only removed after a second miss at its recorded location. * fix(metering): move usage to v2 keys with a small totals item, per-model detail shards and a 3-month TTL (PUT-1949) The billed item holds only totals; per-model detail lives in 100 hash-sharded items read in one batch and cached for 60s, and more than 5,000 distinct usage types in a month fold into other. Global and per-app aggregates store totals only, appTotals comes from a prefix listing, a refused path only skips the key that refused it, exact reads near the allowance are throttled per key, and every metering record expires after 3 months. September usage restarts at deploy; the monthly-charge claim stays on the v1 key through September so recurring charges don't fire twice. * test(events): let the worker backoff test tolerate a loaded run * feat(events): scale KV event fan-out with the key owner's plan A KV change delivers to up to 512 matching subscriptions (2,048 filter evaluations) for a paid key owner and 128 (512) otherwise, inline values are dropped once more than 128 match, and paid accounts may hold up to 512 share handles per app. The owner's plan is only looked up when a change has more candidates than any cap. * test(metering): seed the 150-app listing test directly so it fits under coverage	2026-09-24 16:00:52 -07:00
Juan Castro	8ecaa77a0e	Merge remote-tracking branch 'origin/main' into juancastro/put-1883-alternate-host-bootstrap-app-row-shadows-a-private-app-and ... # Conflicts: # src/backend/controllers/auth/AuthController.test.ts	2026-09-24 10:04:00 -04:00
Neal Shah	07a11240ea	PUT-1868 (#3940)	2026-09-24 00:30:47 -04:00
Daniel Salazar	58ed2f485f	feat(auth): signup bonus codes ... Signup links can carry a bonusCode. Core checks its shape and previews it through `puter.signup-bonus.check`, refuses a dead code before the abuse hook records the attempt, then hands a live one to `puter.signup-bonus.validate` after `puter.signup.validate`, which may raise the phone/card gates; the accepted code rides on `puter.signup.success`, and OIDC carries it in the signed state. `user.card-verified` is now awaited like `user.phone-verified`. The signup form shows the offer and sends the code.	2026-09-23 14:47:34 -07:00
Daniel Salazar	1261976088	feat: user profiles in a system-owned store, public only for paid accounts (#3919) ... - ProfileService keeps each profile as /system/profiles/<uuid>.profile, admin-owned and served by the protected puter-profiles subdomain - GET/POST /profile; puter.auth.getProfile/updateProfile, with getProfilePicture reading through them - another user's profile is served only while its owner is on a paid plan (profileGate.enabled kill switch); the hosted file is gated through the new site.access.check hook the hosting middleware asks before streaming any file - SubdomainStore.create takes isProtected - the GUI reads and writes the profile through the SDK - docs: getProfile, updateProfile, UserProfile, limits PUT-1859 PUT-1860 PUT-1861 PUT-1862	2026-09-21 23:28:56 -07:00
Juan Castro	4e821c128c	Merge remote-tracking branch 'origin/main' into juancastro/put-1813-team-share-blocklist-and-unshare-paging ... # Conflicts: # src/backend/services/share/ShareService.ts # src/backend/stores/share/ShareStore.js	2026-09-21 10:28:33 -04:00
Juan Fernando Castro	76e0d2ec93	Merge pull request #3871 from HeyPuter/juancastro/put-1798-sharing-email-notification-changes ... feat: name the issuing app in share emails, and cut the digest window to 5s	2026-09-21 09:58:09 -04:00
Daniel Salazar	0be3bc55c2	feat(metering): AI cost multiplier hook for AI drivers (#3898) ... Emits ai.cost.multiplier.<driver>.<provider>:<model> before recording AI usage, so what a model costs to charge is policy an extension owns rather than a number in core. Nothing listening records the provider cost. MeteringService.withAiCostMultiplier(driver) returns a view of the service whose recording paths scale costOverride by the hook's answer; every AI driver hands that view to its providers, so all of them are covered without touching provider code.	2026-09-17 15:18:38 -07:00
Daniel Salazar	51ce868d1e	fix: shared kv handles also allow value opt in (#3895)	2026-09-17 10:34:35 -07:00
Daniel Salazar	bb97660ad7	feat: kv events value opt in (#3894)	2026-09-17 00:17:12 -07:00