mirror of
https://github.com/HeyPuter/puter.git
synced 2026-10-13 07:10:05 +00:00
576e893a5ba7aefb6d5a718c0b6be83fd4e97e96
7016
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Daniel Salazar
|
576e893a5b |
refactor(services-core): read-only socket reauth, one token-grant read per check, shared grant/revoke and hosted-asset paths (#4187)
* perf(socket): re-check live sockets without touching their sessions
The 5-minute sweep reused the request-path authenticate(), which slid
every connected session's expiry and wrote user activity each tick, and
could fill SessionStore's touch-throttle map. authenticate() now takes
touch: false for re-checks; the sweep uses it, checks credentials
through runWithConcurrencyLimitSettled, and a call while a sweep runs
joins it instead of starting another.
* perf(acl): read a scoped token's grants once per check
ACLService.check did a Redis GET + JSON.parse of the token's grant list
for every ancestor x mode, and the permission scan did the same per
option. Both now read the list once (PermissionStore.listAccessTokenPerms,
which replaces the per-permission hasAccessTokenPerm) and test it in
memory.
* refactor(metering): incrementUsage goes through batchIncrementUsages
incrementUsage was a copy of the batch path that let an infinite amount
through and skipped NaN, where the batch path counts either as one unit.
It now calls batchIncrementUsages with a one-entry batch, so both apply
the batch clamping (documented on batchIncrementUsages). The overuse
alarm loses the single-path fields nothing sets any more.
* perf(metering): memoize an actor's spent addons on the increment path
Every increment and credit refresh read the actor's addons record from
the KV store. A record with no purchased credit left can only change
through a grant, and every grant announces credits-changed, so it is now
kept per actor beside the credit memo and dropped with it. Records with
credit left are still read fresh, since increments on other nodes draw
them down without announcing it. A read that straddles a drop isn't
kept.
* refactor(services): route raw table writes through their stores
- access_token_permissions: AuthService inserted grants one row at a
time through a cast to the store's db client, read the authorizer, and
deleted the manifest with raw SQL, then asked PermissionStore to drop
its cache. PermissionStore now owns insertAccessTokenPerms (chunked
multi-row insert), deleteAccessTokenPerms and
getAccessTokenAuthorizerId, and invalidates inside the writes.
- TeamService's 2FA reset clears the OTP columns through UserStore.update.
- AppIconService's data-URL icon swap is AppStore.replaceDataUrlIcon.
* fix(user): correct cascadeDelete's FK note; seat cleanup is best effort
The header said most user_id foreign keys are ON DELETE SET NULL; the
migrations cascade nearly all of them and keep SET NULL for audit and
analytics history. forgetSeat ran unguarded after the row was already
gone, so a cache failure there failed the whole delete; it is now
logged and stepped over like the other teardown steps.
* refactor(acl): one grant/revoke path per holder kind
ACLService's user and group stat/set pairs were copies differing only in
which PermissionService methods they called. They now share #stat and
#set over a holder adapter ({ lockKey, query, grant, revoke }); lock keys
and errors are unchanged.
PermissionService's eleven grant/revoke methods repeated the same guard
blocks. Each keeps its own order of checks, now through shared helpers
(#requireUserId, #refuseAppActor, #assertCanManage, #requireUser,
#requireApp, #assertFitsColumn, #audit), so every refusal keeps its
status, legacyCode and wording. Audit writes stay fire-and-forget.
* refactor(auth): one hosted-asset token API keyed by kind
The private-asset and public hosted-actor cookies had separate create/
verify methods, four cookie-name getters and two identical options
getters, differing only in the kind claim and what the host field was
called. They are now createHostedAssetToken(kind, claims),
verifyHostedAssetToken(kind, token, expected),
getHostedAssetCookieOptions(), and the exported HOSTED_ASSET_COOKIES
names. Token claims, cookie names and every error message are unchanged;
puterSite and privateAppGate call the new methods.
* refactor(auth): one revoked/expired check for every actor builder
The session, app-under-user and access-token builders (and the access
token's parent-session check) each spelled out the same revoked/expired
tests. They now share #endedSession, which returns the same reauth
results; only a session token passes its auth_id. Adds coverage for an
access token whose parent session expires, is revoked, or is gone.
|
||
|
Daniel Salazar
|
ecef154630 |
refactor(services): page fs subtree walks, gate share fan-out, one fs ACL helper (#4190)
* refactor(acl): read ancestor chains from FSEntryStore and share one fs ACL helper getAncestorChain (split a path, batch-read the entries) moves to FSEntryStore, with getAncestors for callers that want the rows. Services read it from the store instead of calling FSService sideways; FSService.getAncestorChain stays for drivers and extensions. ACLService gains fsDescriptor (memoized chain), assertFsAccess, assertFsCreate and refusal, plus the pure aclRefusal mapping. The eight hand-rolled descriptor and safe-error copies in FSService, ShareService, events/authorization, FSController, legacyFsHelpers, LegacyFSController, AuthController and WebDAVController use them. Each route keeps its status, legacyCode and message: fs routes answer access_denied on a 403, the legacy routes still hide refusals from apps as 404, share and events keep forbidden and their own 404 text. * perf(fs): page recursive deletes and batch their event dispatch A recursive delete read every descendant into memory, and an empty-check for a non-recursive one did the same. It now walks the subtree a page at a time in descending path order (children before the directory holding them, so the parent_id cascade never takes a row before it is handled), deleting objects and rows and publishing each page as it goes. A non-recursive delete probes with one LIMIT 1 query. Removals publish once per entry, after the row is gone (the same change as the open fix for the double emit), with one surviving-ancestor chain read at most once for the whole subtree. Events-API dispatch for a batch runs as one background task at bounded concurrency instead of a fire-and-forget promise per entry. FSEntryStore gains hasDescendants and listDescendantsInOrder (keyset paged by path, then id). * perf(fs): copy directories a level at a time A directory copy walked every descendant in memory and copied them one by one: an insert per directory and, per file, an S3 CopyObject, a batchCreateEntries of one and its read-back. It now pages the source in ascending path order (parents before children) and takes each page a depth at a time: rows for directories, links and empty files at bounded concurrency, then the depth's S3 copies at bounded concurrency and one batchCreateEntries for all of them. A single-file copy goes through the same path. A failed S3 copy still fails the call (a ghost file still answers 404), but only after the objects that did copy have their rows, so nothing is left in the bucket unreferenced. Copied entries publish per depth in one dispatch. * refactor(share): build masked share paths in one place sharePathMask exports maskedRoot(owner, uuid, name) and maskUnder(root, realPath). ShareService's maskedPathVia goes, its maskedSelfPath and shareDeepLink's maskedSharePath build on them, and the masker, maskUnderAnchor and learnShareRoots use maskedRoot. Output is byte-for-byte the same; tests pin it. * fix(share): send old_path on a recipient's item.moved The desktop's item.moved handler reads old_path; ShareService sent the former masked path only as from_path. Both are sent now. * perf(share): skip the audience lookup for owners with nothing shared Every fs write, create, rename, move and delete looked for share recipients: an ancestor walk plus two share queries, for an owner who in nearly every case has shared nothing. #sharesReaching now reads the chain through FSEntryStore.getAncestors (no third copy of the walk) and asks a per-owner "has anything shared" flag before the share queries. The flag is the per-user cache events already keeps for subscriptions (SubscriptionCache): one ShareStore.ownerHasShares read per owner per TTL, coalesced across a burst. Creating a share (user, team, claimed invite) or moving a subtree to a new owner bumps it locally and over outer.pubsub.share.ownerShared; the TTL covers a process that misses the bump. A failed lookup reads as shared, so the full lookup decides. * perf(share): batch the per-row reads behind share listings Share listings looked things up one row at a time: - teams by id: TeamStore.getByIdsIncludingDeleted (the single-id read calls it) for listSharedByMe, listSharesOf and #revokeGroupSharesBy - team shares per node and ancestor: ShareStore.listGroupOnFsentries, one query for the chain, in the same node-then-ancestor order - #liveGroupGrants: PermissionStore.queryIssuerGroupPermsByPrefixes reads every issuer/group pair under every prefix in one query - #revokeGroupSharesBy: the five canManagePermission checks per row run together instead of one after another - ShareNotificationService #unblocked: users via getByIds and blocks via UserBlockStore.blockersAmong, two queries instead of two per member - digest flush: entries are taken at bounded concurrency rather than one at a time * perf(events): ask background consent once per holder and app on a revoke settle Withdrawing events:background re-checked consent row by row, resolving the holder's actor and asking the permission service for every one of their background rows. Rows of one holder and app stand on one consent, so it is asked once and shared. * refactor(events): drop the registry's unused push fields notify, defaultDelivery, PushMessage, NotifyProjection and pushProjection were read only by registry.test. Every subject was unpushable and broadcast by default; the subscription's delivery class stays in DeliveryClass, which the stores use. * refactor(share): retire trashed shares through onEntryOwnerChanged onEntryTrashed was a copy of onEntryOwnerChanged: the same subtree share sweep, grant retire and row delete. * fix(fs): page subtree walks by depth, not path MySQL sorts a PAD SPACE string on its first max_sort_length (1024) bytes, so under utf8mb4_unicode_ci paths past about 512 characters tie and fall back to id. A file moved under a newer directory then sorted after it: the delete walk reached the directory first and the parent_id cascade took the file without its object delete or event, and the keyset skipped or repeated rows. Pages now order by segment count, then id, which no collation can reorder. * fix(share): cache an owner as shared when the bump arrives A bump only dropped the cached answer, so the next lookup could read a replica that hadn't applied the new share yet and cache "nothing shared" for another TTL. The bump now records "shared" outright. * fix(fs): probe a directory for children by parent_id, from the primary * fix(fs): keep the copied-size check in the batched directory copy A copied file records the size CopyObject actually duplicated; growth past the source row is charged against the allowance, and a copy that no longer fits is deleted and refused, as a single-file copy already does. |
||
|
Daniel Salazar
|
c815dc82a1 |
refactor(core-util): shared bounded map, cheaper route lifecycle, DDB/DB/rate-limit/gate dedup (#4185)
* refactor(util): add BoundedTtlMap for core caches and log throttles
One size-capped, LRU-evicting map with an optional TTL and a once-per-window
shouldEmit helper. Replaces the hand-rolled maps in the DynamoDB throttle and
repair loggers (the repair one was unbounded), the Slack repeat throttle, the
auth probe's reauth log throttle, and the card-verification status memo.
* perf(http): skip route lifecycle work when nothing listens
The per-route lifecycle middleware built a payload, opened an
event.emitAndWait span and hooked finish/close on every request, listeners
or not. Each phase now checks hasListeners first. EventClient's three copies
of the key-prefix walk become one memoized #matchKeys.
* perf(ddb): write batch chunks concurrently and retry the TTL sweep
batchPut/batchDel sent their 25-item chunks one after another; they now run
up to four at a time through runWithConcurrencyLimit, and a failed chunk
stops the ones not yet started. batchGet and the batch write share one
#sendWithUnprocessedRetry instead of two copies of the backoff loop, and the
TTL sweep goes through #batchWrite, so the unprocessed deletes it used to
drop are retried. The AWS and local clients are built by one #bindClient.
* refactor(db): share migrations and tryHardRead in the base client
MySQL and Postgres each carried a copy of the migration-path runner and of
the replica-racing tryHardRead; both now live on AbstractDatabaseClient
(applyMigrationPaths, and tryHardRead gated on hasReadReplica), and SQLite
drops its own tryHardRead override. Removes MySQL reinitPrimary,
reinitReplica, isRetriableError and readWithRetry and the base requireRead,
which had no callers outside tests. SQLite prepared statements and the
Postgres SQL rewrite are memoized per query text in a bounded map.
* refactor(rate-limit): route every entry point through one charge()
The nine rate/peek/acquire entry points each repeated key and limit
resolution, the backend call and the fail-open catch. They now share
charge(), routeKey() and guardSlot(); log lines are unchanged. Driver
concurrency slots release at most once like the other slot handles, and
checkRateLimit takes { failOpen: false } for limits that must hold when the
backend is down.
* refactor(util): add secretsEqual for constant-time secret comparison
One HMAC-then-timingSafeEqual helper, safe for inputs of any length, so
callers stop hand-rolling the length guard. File signature verification
uses it.
* refactor(fs): infer signed-file MIME types with mime-types
mimeFromName looks extensions up in mime-types instead of a 35-entry inline
table, keeping the two types the table reported differently (ico, wav). The
purpose-labelled signing key is derived once per secret instead of on every
signature, where signFile derived it twice.
* refactor(http): build route gates from one gate() helper
Each gate repeated a try/catch -> next(err) wrapper or a .then(next, next)
chain around an assertion. gate(check) does that once: a throw or rejection
goes to next(err), and a synchronous check still calls next synchronously.
The credits, subscription and reputation gate files held nothing but such a
wrapper; their factories move into gates.ts and the files go.
* perf(hosting): build the puter-site hosting config once
createPuterSiteMiddleware rebuilt the normalized hosting-domain config on
every hosted-site request; it is derived from static config, like the domain
lists the middleware already builds at construction.
* fix(db): tryHardRead's replica leg doesn't fail over to the primary
MySQL's read() retries on the primary after a load-shed or transient
replica error, so routing tryHardRead through it read the primary twice
on that path. The replica leg now reads the replica alone.
* fix(ddb): settle in-flight batch chunks before rethrowing
A failed chunk stopped unstarted chunks but rejected while others were
still retrying, so writes could land after the caller had handled the
error and invalidated its cache.
|
||
|
Daniel Salazar
|
b5eefaa9cc |
fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135) (#4198)
* fix(fs): include supplementary-character names in subtree path queries on MySQL (PUT-2135) MySQL ends the idx_fsentries_path range for `path LIKE 'p/%'` at 'p/' + U+FFFF, and utf8mb4_unicode_ci sorts supplementary characters above it, so a child whose name starts with one (most emoji) was skipped with its whole subtree by listing, delete, copy, rename/move rewrites, counts and sizes. On MySQL, FSEntryStore now looks up those children by parent_id and ORs a LIKE pattern for each onto the subtree match, recursing into directories. Every extra pattern starts with the prefix, so only the scanned ranges widen; rewrites past 500 patterns run in several statements. The prefix rewrite also passed a UTF-16 length to SUBSTR, which counts characters, so a rename under a path holding an emoji dropped characters from every descendant path on all engines. It now counts code points. * test(fs): seed the supplementary-name subtree with createNonFileEntry |
||
|
Daniel Salazar
|
af9ebb0ef1 | fix(ai): map BytePlus video and Cloudflare image upstream errors at the driver boundary (#4184) | ||
|
Daniel Salazar
|
2131b980ce |
fix(redis): release locks with an atomic compare-and-delete (#4183)
* fix(redis): release locks with an atomic compare-and-delete ACLService, TeamService and the share digest flush each hand-rolled a SET NX lock whose release was a GET-then-DEL (or a blind DEL), so a holder whose TTL lapsed could delete the next holder's lock. They now share withRedisLock, which claims with a random token and releases via a Lua compare-and-delete. * chore(team): place the redis lock import with the other runtime imports |
||
|
Daniel Salazar
|
96909b0800 |
fix(fs): send the same outer.gui.item.* payload from every fs controller (#4179)
* fix(fs): send the same outer.gui.item.* payload from every fs controller The v2 /fs/move and /fs/delete routes spread the raw camelCase FSEntry into item.moved / item.removed and named the old path `from_path`, while the desktop reads the legacy snake_case shape and `old_path`. One helper, emitGuiItemEvent, now builds the payload for LegacyFSController, FSController and WebDAVController. v2 and WebDAV copy/move with overwrite now also emit item.removed for the entry they replace, as the legacy routes do, and v2 /fs/move forwards new_metadata to the service like the legacy /move. * fix(fs): announce a replaced entry before the one replacing it The desktop hides rows by path on item.removed, so a removal sent after the overwriting copy's added (or move's moved) hid the new row too. The shared emitter now takes the replaced entry and sends its removal first, awaited, on the v2, legacy and WebDAV copy/move paths. |
||
|
Daniel Salazar
|
906d6b7886 |
fix(email): one canonicalization and blocklist for every address path (#4176)
Share invites, share notifications and app feedback checked addresses through EmailClient.clean/validate, a second copy of the rules in util/email.ts that stripped dots on iCloud, stripped `+tags` on every domain, and matched the blocklist case-sensitively. They now use util/email.ts: isBlockedEmail for the blocklist, abuseKey for the per-address notification budget, and cleanEmail for the invite digest key. EmailClient keeps transport and templates only. isBlockedEmail also lowercases the configured suffixes, so a mixed-case blocklist entry still matches. |
||
|
Daniel Salazar
|
6a938bd8e8 |
fix(db): detect duplicate keys with isUniqueViolation everywhere (#4175)
FSService, OIDCStore, UserStore and TeamService each carried their own duplicate-key check. FSService only matched mysql and the bare SQLITE_CONSTRAINT code, so the mkdir/move race recovery never ran for a SQLite or Postgres unique violation. OIDCStore matched any message containing "UNIQUE" or the bare SQLITE_CONSTRAINT code, UserStore any SQLITE_CONSTRAINT_* code, and TeamService any message mentioning unique or duplicate. All four now use util/dbError's isUniqueViolation; UserStore still checks the index name on top. |
||
|
Daniel Salazar
|
847a4fc031 |
fix(fs): route thumbnail writes through FSEntryStore (#4174)
* fix(fs): route thumbnail writes through FSEntryStore Add FSEntryStore.updateEntryThumbnailByUuid, which sets a thumbnail with no owner check (callers authorize) and refreshes the entry cache, with an optional expected-value guard. The thumbnails extension's inline data: URL migration, its copy duplication, and the legacy /update-fsentry-thumbnail endpoint now write through it instead of raw UPDATEs, so cached entries stop serving the old thumbnail. The migration validates the image like the upload path, awaits the write, and only repoints a row that still holds the data URL; a read that loses that race deletes its upload. * fix(fs): report a thumbnail update by whether the row changed updateEntryThumbnailByUuid returned the refreshed entry, which is null when the cache misses and the replica hasn't seen the row yet. The read-time migration took that null for a lost race and deleted the object the row had just been pointed at. It now returns whether the UPDATE changed a row; the cache refresh still runs after it. |
||
|
Daniel Salazar
|
3296526e8e |
fix(ai-chat): derive Azure model pricing from the OpenAI and xAI catalogs (#4173)
Azure entries that front an OpenAI or xAI model now come from a `mirror` helper that copies every non-identity field from the source catalog entry. The hand-copied entries had drifted: none carried `long_context_pricing`, so long prompts on Azure-served gpt-5.4 and Grok 4.3/4.20 were billed at 1x; grok-4.3 had lost `pdf` input and the Grok 4.20 deployments still listed the old 2M context. Models no longer in the direct catalogs stay inline. |
||
|
Daniel Salazar
|
c553a380ae |
fix(ai-chat): forward max_tokens to xAI; moderate Azure through OpenAI (#4172)
xAI sent a fixed max_tokens of 1000, ignoring both the caller's value and the credit gate's cap. It now forwards max_tokens like the other OpenAI-compatible providers and omits it when unset. Azure chat's checkModeration threw "Method not implemented", so `moderation: true` on an Azure route metered the completion, then failed it as a route error: Azure was marked unhealthy and the fallback billed a second completion. Azure's v1 API has no /moderations endpoint, so both Azure providers now delegate moderation to the OpenAI chat provider (Azure chat via its Responses sibling, as it does for web_search). A moderation call that throws is now a 400 `moderation_unavailable` that the driver treats like `moderation_flagged`: final, no fallback, and the route stays healthy. |
||
|
Daniel Salazar
|
cf9df8719a |
fix(alarm): forget idle alarms and cap how many the client holds (#4171)
AlarmClient kept every alarm id it had seen for the life of the process, and ids that embed a user, key or invoice grew it without bound. The registry is now ordered by last occurrence: alarms idle for 24 hours are dropped on the next create, and past 1000 the least recently raised goes first. Clearing or evicting an alarm only removes its short-id alias when the alias still points at it. |
||
|
Daniel Salazar
|
7ad963bacf |
fix(cache): broadcast fs entry and team invalidations to peer regions (#4169)
* fix(cache): broadcast fs entry and team invalidations to peer regions FSEntryStore's invalidations called publishCacheKeys without broadcast, so a delete, move, rename or overwrite only cleared the writer's cluster; other regions served the old row until the 60 s TTL. They now broadcast. Batch paths (deleteEntries per chunk, the batch overwrite) send one event for the whole batch. Read-path backfills stay local. TeamStore's busts were a local redis.del, so a removed member kept passing getMembership in other regions until the TTL. They now go through publishCacheKeys with broadcast, and softDelete busts every key in one call. Remove FSEntryCacheInvalidationEventHandler. It re-derived the same invalidations from outer.gui.item.* with a replica SELECT per holder, and its fs.remove.node branch never ran (FSEntry has no get()). The store's own broadcast covers what it did. * fix(fs): cap how many entry invalidations a node broadcasts per window FSEntryStore now broadcasts at most 10,000 entries' cache invalidations per broadcast flush window (outbound_flush_ms, 2 s by default). Past that, keys are dropped locally only and peers fall back to the 60 s entry-cache TTL; the store logs once per window when this happens. Each flush goes to a peer as one POST that is re-sent whole on failure, so a large recursive delete could otherwise build a batch the peer rejects on every retry. The budget is per store instance, not per call, so a delete issued page by page is bounded too, and single-entry writes count against the same budget. |
||
|
Daniel Salazar
|
0066d49f74 |
fix(fs): emit fs.remove.node once per entry on recursive delete (#4167)
#removeDescendantsStorage emitted fs.remove.node for every file descendant while grouping S3 keys, and remove() emitted it again for every descendant after deleting the rows, so file listeners ran twice. Drop the storage-pass emit; remove() now emits once per entry after its row is deleted. |
||
|
Daniel Salazar
|
b67acdbf6f |
fix(stores): return written rows without reading them from a lagging replica (#4166)
* fix(stores): read rows back from the primary after writing them ShareStore (create, upsertPending, upsertActive, upsertActiveGroup, applyPending, upsertAnyone), AppStore (create, createFromOrigin, update, claimOwnership), NotificationStore.create, SubdomainStore.update and NotificationDriver.create returned the row they had just written by reading it back through `db.read`, which goes to the replica. Under replica lag that came back null or stale. Those read-backs now go to the primary, through a `primary` option on the getters. AppStore caches the primary copy on create, and claimOwnership no longer reads through getById, which cached the replica's pre-claim row on a miss. * fix(stores): skip the notify read-back; read OIDC link conflicts from the primary NotificationStore gains `insert`, which writes and returns the uid; `create` is now `insert` plus the primary read-back. NotificationService .notify ignores the row, so it uses `insert` and no longer pays a primary round trip per recipient. NotificationDriver.create still reads the row back from the primary. OIDCStore.link reads the conflicting row from the primary after a unique violation, so a replica that has not seen another account's link can no longer make the call report success. * perf(stores): return written rows instead of reading them back ShareStore.create, NotificationStore.create, AppStore.create and AppStore.createFromOrigin build the row they return from the values they inserted and the insert id, instead of reading it back from the primary. The DB-defaulted creation timestamp is now written explicitly from unix seconds (`timestampFromUnixSql`), so the returned value matches what a read returns (`timestampValue`); postgres INSERTs add RETURNING id. The AppStore rows fill every column the INSERT leaves to the schema and are cached as before. ShareStore.upsertPending's re-invite branch selects the full pending row in its dedupe read and returns it with the new mode and data; it falls back to the primary only when the UPDATE matched nothing. The remaining primary read-backs stay and say why: the share upserts and the new-invite insert (an upsert's surviving uid/created_at and the entry-owner subquery are only known to the primary), applyPending and claimOwnership (only an id or uid in hand), AppStore.update and SubdomainStore.update (the base row is a replica copy that would be cached), NotificationDriver.create (notify returns only the uid) and the OIDC link conflict. |
||
|
Daniel Salazar
|
f88cadb4b9 |
fix: a copy records the size of the bytes it duplicated (#4197)
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
A copied file's row took its size from the source row the request had resolved, which can predate an overwrite whose bytes CopyObject then copied. The copy now reads the new object's size and records that. When it is larger than the source row said, the allowance is checked again against the real size; a copy that no longer fits is refused with 413 and its object deleted. If the size check fails, the copy falls back to the source row's size, as completing a signed upload already does. |
||
|
Daniel Salazar
|
aef0a32d72 |
fix(ai): hold credits for image and video generation (#4164)
Image and video providers checked the balance with hasEnoughCredits / getRemainingUsage and took no hold, so concurrent requests each saw the full balance until the first one recorded usage. They now reserve the estimate through the cost-factored reserveAiCredits and release it in a finally block once usage is recorded, like chat, speech and OCR. The video duration clamp now prices seconds at the model's cost factor and holds the clamped clip's cost for the job (poll window plus create time). Replicate holds the output-only estimate before fetching inputs and the input surcharge on top once it is measured. |
||
|
Daniel Salazar
|
491729e060 |
fix: backend test flakes and a socket shutdown hang (#4170)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
- SocketService's prepare-shutdown hook awaited io.close(), which also closes the http server and waits for every HTTP connection to end. A connection mid-request when close() runs stays open for the keep-alive timeout, so shutdown stalled before PuterServer severs connections. The hook no longer waits on the http close. - Test servers disable Ollama unless a test configures it. Its boot probe logged after files had ended, which failed shared-worker shards with EnvironmentTeardownError. - vitest ignores that teardown error for a console call still in flight. - setupPuterTestEnv boots again on a fresh port after EADDRINUSE, and a failed setupTestServer start shuts the server down. - UploadReservationStore expiry tests pin the clock instead of racing a 50 ms TTL. |
||
|
Daniel Salazar
|
d4589caefe |
fix: session store revokes (#4168)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
|
||
|
Daniel Salazar
|
4fca4809b9 |
fix(health): discount total event-loop stall time from db liveness latency (#4160)
Notify HeyPuter / notify (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
The stall watcher kept only the longest gap, so a saturated loop with many sub-second stalls still failed database-liveness. It now sums blocked time across the probe (ignoring timer jitter) and reports total and longest. |
||
|
Daniel Salazar
|
cc8b9858a2 |
feat: let extensions claim a subdomain (PUT-2100) (#4155)
`extension.claimSubdomain(name, { authorize, checkRoute? })` lets an
extension serve `<name>.<domain>` without a hard-coded reserved name.
- A claimed host is dispatched where the unknown-subdomain 404 used to be,
before CORS, the OPTIONS responder, body parsing, auth and every route
(including `subdomain: '*'` ones).
- `authorize` returning anything but true, or throwing a non-HttpError,
yields exactly the unknown-subdomain 404. A thrown HttpError goes to the
error handler.
- Routes whose `subdomain` is the claimed name (decorated controllers and
extension routes alike) are materialized onto the claim's own router, so
they only run after `authorize` and never on the main app. Their options
are limited to ones that don't rely on the main pipeline's actor; an
optional `checkRoute` lets the claim refuse routes at boot. Case-variant
labels and claimed names mixed into subdomain arrays fail the boot.
- The unknown-subdomain check now compares the port-stripped hostname and
requires a dot boundary, so a host with an explicit port behaves like one
without it. Hosts served by the local worker proxy still pass through.
|
||
|
Daniel Salazar
|
6060de8fbd |
fix(alerts): overspend alarm in credits at warning; no-choices completions stop paging (#4154)
- The per-account overspend alarm now reads in credits and raises at warning so it reaches the pager instead of only chat. - A non-stream completion with no choices (an upstream answering 200 with an error body) threw a TypeError that classified as internal_error and paged critical. It now throws bad_response, which falls back like any other empty response and surfaces as a 400. |
||
|
Daniel Salazar
|
0e0cedb02b |
fix(auth): no email password recovery for built-in admin accounts (#4153)
- /send-pass-recovery-email for `admin`/`system` (by username or email) returns the usual generic message, writes no token, sends no mail, and raises a critical alarm with the requester's IP, user agent and, when signed in, their user. - /verify-pass-recovery-token and /set-pass-using-token reject recovery tokens for those accounts as no longer valid. |
||
|
Neal Shah
|
43094a84a0 | PUT-1901 (#4159) | ||
|
Neal Shah
|
5fb025c109 |
Misc subdomain and worker fixes (#4157)
* PUT 1902 * LocalWorker fetch policy fixes for 127.0.0.1 |
||
|
Juan Fernando Castro
|
50bcdc9c6c |
fix: let a revocation be seen by the reads and the nodes that outlive it (#4151)
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
* fix: let a revocation be seen by the reads and the nodes that outlive it Four places where a revoke lands on the primary and the next reader can still be told the old answer. The u2a cache key was dropped but not re-filled, so the next miss read a replica that may still carry the row and cached it for the TTL -- and a revoke's own settling pass is usually that miss. It re-fills from the primary, as the u2u side already does. A revoked session dropped its keys the same way. It now leaves the revoked row in the cache, read from the primary, so there is nothing for a lagging replica to re-introduce. A link share is its own authority rather than an index over a permission, so a replica still carrying a withdrawn row answers with it. For a short window after one is withdrawn, those reads go to the primary. Socket eviction rode a local event, so it only reached the node that did the revoking. It is announced on the fan-out channel too, which is where the node actually holding the connection hears it. * refactor: carry the revocation on outer.* instead of outer.pubsub.* Eviction runs through `io.in(room).disconnectSockets()`, which goes via the socket.io Redis adapter a cluster shares, so one node handling the event disconnects that user on every node in it. Fanning out to siblings had each of them repeat the same cluster-wide disconnect. The adapter's Redis is per-cluster, so peer clusters still need the hop -- `outer.*` reaches them and skips the sibling fan-out. The local `auth.sessions.revoked` goes with it: `outer.*` fires on the emitting node too, and nothing else subscribed. |
||
|
Juan Fernando Castro
|
ff35fbd66e |
Share attribution and cascade: finish PUT-1856 and PUT-1857 (#4119)
Replayed onto main after #4118 squash-merged. Same tree as the reviewed head (8d8bd871f); only the redundant merges from the old base are gone. |
||
|
Juan Fernando Castro
|
1ff5d93244 |
fix: a signed overwrite lands on the live key only once its completion is accepted (#4136)
* fix: a signed overwrite lands on the live key only once its completion is accepted A presigned overwrite URL was signed against the live object key, so the browser's PUT replaced the owner's bytes directly. That PUT never passes through Puter, which leaves the access check on `completeWrite` with nothing left to protect: by the time it runs, what /read serves has already changed. The upload now goes to a key of its own. `completeWrite` is already the authorized step, so that is where the staged object is copied onto the live key, server-side, and the staged one dropped. The entry keeps its uuid, so everything written against it still holds. Aborting deletes the staged object, which the old shared key made unsafe to do. `copyObjectOfAnySize` carries the copy past the single-request ceiling by copying part by part, so a large file is no different. Two behaviours follow from the key no longer being shared: completing an overwrite that uploaded nothing is refused rather than reported as an overwrite of unchanged bytes, and abort always drops the session's object. Only the two presigned paths changed; `prepareBatchWrites` uploads server-side within one authorized request and still writes in place. * fix: the review's findings on the staged overwrite The batch signed path still signed against the live key, so the route the SDK's signed upload actually takes was unchanged. It stages now, which also makes the abort simplification safe: the guard that had protected a shared key was removed while that path still shared one, so aborting a batch overwrite would have deleted the owner's object. The completion is allowed against a path but lands on the uuid captured at start, and a move between the two puts the bytes somewhere the caller was never authorized for. The promote refuses unless the entry is still where the check looked. The batch promote failing left every session pending with its lease held; it now fails them as the rest of the function does. The pending GUI tile and the thumbnail key were keyed on the staged uuid, which no entry ever carries, so neither correlated with what the completion emitted. Responses carry `entryUid` for that. * fix: do not treat a pre-staging session's key as this upload's own A session signed by the previous release points straight at the live key, since `objectKey` was `existingEntry?.uuid ?? uuidv4()`. Completing one is already handled -- the promote returns early when the two keys match -- but abort deleted `objectKey` unconditionally, which during a rolling deploy would take the owner's file rather than the upload's litter. Abort now deletes only a key the upload owns. The multipart branch was always safe: discarding parts never touches an object already in place. |
||
|
Juan Fernando Castro
|
86cc3490f7 |
fix: settle which plan an org seat resolves to, and confirm a seat's address (#4139)
* fix: settle which default plan an org seat resolves to, and what counts as paid Default subscription resolvers ran in registration order, which is boot order: whichever layer registered first answered first. A seat is provisioned by a team and its address is never confirmed, so the seat resolver now states a priority and wins over anything a deployment infers from an address. The seat cap asked whether the owner's plan was outside the free set, which a default plan can be without anyone having bought it. Metering now reports whether an explicit subscription answered, and the cap rests on that. * fix: the review's findings on plan resolution `fromDefault` was read off which resolver answered, not off which policy applied. An explicit id naming a policy nobody registered falls through to the default one, and that read as bought. It follows the applied policy now. A deployment with unlimited metering has no explicit resolver, so every owner on it became unpaid and its teams dropped from the paid seat cap to the free one. Unlimited answers for what a paid plan would, so it counts as one. Also: the new constants had been placed between `FREE_SUBSCRIPTION_IDS` and its docblock, the register docstring still described first-wins, and prettier had reflowed an untouched licence header. * feat: a seat confirms the address its team typed in An address on a seat is the team's claim about someone else's inbox, and nothing asked the inbox. It is now demanded and sent a code, so the address counts for nothing until the seat hands it back. Resolvers that read a domain are the reason this matters. No address is still fine and demands nothing: the team is the trust anchor for the account, not for a mailbox it does not hold. `/confirm-email` allows an unconfirmed actor, so a seat can sign in with its temporary credential and clear the demand; nothing locks it out. * fix: clear the confirmation demand with the address it names `clearUnconfirmedEmail` releases a seat's address so the inbox's owner can sign up with it. The demand to confirm that address has to go at the same write: the gate reads `requires_email_confirmation && !email_confirmed`, and once the address is gone there is nothing left to send a code to. * fix: keep a seat's address correctable, and only ask what can be answered Review of the confirmation demand turned up two ways a seat could be left holding one it had no way to meet: - With no mail transport configured, the code is never sent and a resend cannot reach the seat either, so the demand stood with nothing able to lift it. The seat is now gated only when a code can actually travel. - An address the owner mistyped gated the seat on an inbox nobody could read, and the gate itself closes `/user-protected/change-email`, the route the seat would have fixed it by. `updateSeatEmail` lets the owner who typed the address retarget it, and refuses once it is confirmed, when it belongs to whoever proved it. Also resolves the owner's plan sequentially, so the second read takes the cache the first wrote and the pair cannot come from two resolutions, and restores the licence header a jsdoc reflow had rewrapped. * fix: let the confirm-email window open before the profile has loaded Booted from a stored token, the window renders before `window.user` is populated, and reading `.email` off null threw inside the window itself -- so the one screen that could have asked for the code never drew, leaving a blank page. Reproducible without a team: sign up, then load the GUI again before confirming. The address is now encoded rather than interpolated raw, and both forms of the line are i18n keys; it was hardcoded English before. |
||
|
Juan Fernando Castro
|
7de8360a6e |
Sharing cleanup: bounded outbound listings, and what see does not reach (#4118)
* fix: keep a file name to one line in a subject, and the inbound total to the account
**A name reaches the digest subject as the user typed it.** Subjects compile
with `noEscape` on purpose -- a header is not HTML, and escaping would show
entities to the reader -- and the comment there says free-form values collapse
whitespace upstream. A file name does not: `#assertUsableName` refuses only an
empty name, a slash, `.` and `..`, so a name may carry newlines. It is
collapsed to one line and bounded before it goes in.
**`includeTotal` answered an app with the account's figure.** The page is
filtered by what the caller can reach -- an app sees only what its user handed
it -- while `countByHolder` counts every inbound share. The docs already call
`total` an approximation for the filtering; they did not cover an app reading
the size of everything else. It is answered only for a caller acting as the
account now, and the page documents that.
The team-share fixture built its actor as `{ user }` with no `effectiveApp`,
which reads as an app rather than a session. A session acts as the account, so
it says so.
* fix: stop `see` reaching inside a folder, and bound mode changes
**A `see`-only folder recipient was pushed every child's name, size and type.**
The fan-out picked holders by reach -- any live grant on a reaching root -- and
`see` is a grant. But `see` says a node is there; `list` is what says what is
inside it, and the payload carries exactly what `list` would: name, path,
`is_dir`, type, size, modified. Descendants now go only to a holder whose mode
reaches inside. A change to the shared node itself still goes to everyone who
holds it, which is what `see` covers.
**Mode changes skipped the daily quota entirely.** Not spending share budget is
right -- changing a mode is not new reach, and the existing test says so -- but
unbounded it lets one account write a pair as fast as it can ask. They have
their own budget now, ten times the share limit by default and configurable,
so the exemption stays an exemption from *that* budget rather than from any.
* fix: let a half-finished downgrade be finished, and stop Trash answering
**A mode downgrade that failed between its two halves could not be repaired.**
`setUserUser` grants the new mode and then revokes the superseded ones. If the
revokes did not all land, the higher grant stood -- and the retry returned
early, because the mode it was asked for was now present. "Nothing to do" now
means this mode is the only one standing, so asking again is what finishes it.
The same two halves, and the same early return, were in the group variant.
**Trashing someone else's item answered for what is in their Trash.** Moving
into the owner's Trash is deliberately authorized without write on it -- that
is what lets a recipient delete from a shared folder -- but a name collision
there came back as `An entry already exists at /owner/Trash/<name>`. The mover
picks the name, so that is a question they can ask about a folder they cannot
list. It dedupes instead, which is also what the GUI wants: trashing should
not fail over a name.
* perf: record the entry's owner on the share row
The outbound listings answer "shares on things I own that someone else issued"
by joining `share` to `fsentries` and filtering on the owner there. Neither side
of that join bounds the work: driven from `fsentries` it walks every file the
user owns, driven from `share` it walks every share until it has a page. For a
user who owns a lot and delegates nothing, that is the whole table for an empty
result. `#outboundAppsSql` is worse than the ticket says -- it has no LIMIT at
all, so every call aggregates the join across both tables.
The owner now rides on the row, indexed with `id`, so all three read it
directly. An entry cannot change owner -- a move out of the owner's tree is
refused -- so it is written once, with the row.
Set in SQL from `fsentry_id` rather than threaded through callers: four insert
sites, each of which already has the id, and none of which can now forget.
The risk in denormalising is a write path that does: a row with no owner
recorded is a share that silently stops appearing in its owner's listing. The
new test covers exactly that -- it fails when one insert is made to write NULL.
Verified against a real boot: a pre-existing share row, on a database that
genuinely predates the column, comes out with the owner its entry has.
* fix: the review's findings on this branch
**The Trash collision still had both halves open.** `overwrite` was checked
before the trash case, so a caller setting it -- it is wire-supplied -- could
still delete whatever it collided with in the owner's Trash, and the 409
survived for anyone who did not. The trash case is decided first now.
Dedupe alone did not close it either: a name that changes only when there is a
collision answers the same question the refusal did, and the mover sees that
name even though the path is masked. Trashing someone else's item renames it
every time, so the result carries no signal. `original_name` already records
the real name, which is what a restore reads. Two tests pinned the old name;
they say what it is now.
**The owner column had no fallback.** Reads went only through it, so a row
written by anything that predates it -- a pod mid-roll -- would never appear in
its owner's listing again. The three reads now also cover `IS NULL` rows
through the old join, indexed on the null and empty once none remain.
**The grant became conditional on a replica read.** `if (!already)` skipped it
on a stale row while the revokes below stayed unconditional, which could leave
the holder with nothing where the old early return left their access alone. It
grants unconditionally again; only the early return changed.
**The re-issue budget covered one path of four.** Team, link and invite
re-issues reserved nothing, so three ways round it were left open. They share
one `#reserveRemodeQuota` now.
Two docblocks had been pushed off the functions they describe, and two comments
still described the join that is gone.
* chore: one-line the comments added here
* test: measure the prefix walk by what it keys on, not by the clock
`checks many long paths without re-reading every prefix of each` gave
the walk 1500ms of wall clock. It measured nothing useful: the shape it
guards against — keying each step on the accumulated prefix text rather
than on the parent id — costs 29ms against the current 8ms on that
input, so both ends sit far under the bound. Under coverage the same
test ran 1511ms and failed, which is the only thing the number could
ever report on.
`PathPrefixIds` now counts the characters it keys on, and the walk is
asserted on that: proportional to the input, and twice the depth costs
twice as much rather than four times. Keying on the accumulated text
fails both. The original case keeps its real assertion, that a request
this size is refused.
* fix: keep the recorded entry owner current, and the trash guard on the retry
A move into someone else's tree re-owns the whole subtree, which the
migration comment claimed could not happen. The shares on it are
re-pointed at the new owner, or they drop out of the listing the column
keys and stay in the old owner's.
The unique-violation retry re-entered the collision resolver without the
trash flag, which is the one call site the guard was not threaded
through.
|
||
|
Daniel Salazar
|
be78281d9d |
ci: shard backend tests and share workers across server-booting files (#4149)
Backend Tests / changes (push) Canceled after 0s
Backend Tests / typecheck (push) Canceled after 0s
Backend Tests / test shard 1/4 (push) Canceled after 0s
Backend Tests / test shard 2/4 (push) Canceled after 0s
Backend Tests / test shard 3/4 (push) Canceled after 0s
Backend Tests / test shard 4/4 (push) Canceled after 0s
Backend Tests / test (main) (push) Canceled after 0s
Backend Tests / coverage (push) Canceled after 0s
Maintain Release Merge PR / update-release-pr (push) Canceled after 0s
release-please / release-please (push) Canceled after 0s
Notify HeyPuter / notify (push) Canceled after 0s
The suite was CPU-bound on one runner, and about half of that CPU went to re-importing the backend for every test file. - Run the suite as 4 shards and merge their blob reports for coverage. `test (pr)` stays as the required check and fails if any shard fails. - Drop the per-PR base leg; pushes to main publish the coverage PRs compare against. - Files that boot a server without module mocks or extensions run in a `shared-worker` vitest project (`isolate: false`), so the backend is imported once per worker. testSharedWorkerSetup.ts resets the redis mock, configContainer and extensionStore between files. |
||
|
Daniel Salazar
|
8a6929847d |
Auth limits, reborn-app tokens, seat addresses, drop-area encode (#4148)
* auth: per-account 2FA code limit, single-use otp-login tokens, recovery mail limits - /login/otp and /login/recovery-code share a per-account bucket charged on wrong codes; an otp-login token completes one sign-in. - /send-pass-recovery-email adds an IP bucket, a per-account cap and captcha, and reuses the outstanding recovery token instead of rotating it. - Reauth tokens carry the rejection reason; /signup re-attaches a temp account only from an expired session. * auth: app-issued access tokens older than their app stop authenticating An access token whose row or parent session predates the app now holding its uid is refused, the same check app and worker sessions already get. * teams: signup can take an address a seat never confirmed Password signup, save-account and a provider-verified OIDC sign-up take an address a team seat holds unconfirmed; the seat's copy is cleared at the write. A seat address that was confirmed still blocks. * dev-center: encode dropped directory child names |
||
|
Daniel Salazar
|
360f6b1079 |
fix(events): bill reached deliveries, bound dispatch reads, seal cursors (PUT-1900) (#4146)
* fix(events): bill reached deliveries only, bound dispatch reads, seal cursors - A broadcast socket copy is billed only when a connection for its holder exists on this node, anywhere in this region (connection count) or, for a durable row, in a region presence names. Without peers it bills as before. - Compiled match filters are held in a bounded LRU with a TTL, and dropped when another process reports a subscription ended. - GET /events/fetch reads the continuation from `cursor`, falling back to `after`. - Cursors from /events/fetch, /events/subscriptions and /events/kv-handles are encrypted with sealCursor/openCursor; plain cursors still read. - A durable subscribe counts again after its insert and removes its own row when over the cap; the generation bump for that removal is published. - Subscribe checks access before reading the anchor's owner, and an anchor gone in between answers with the subject the caller sent. - Dispatch reads at most the rows per token routing can use, scanning large token hashes; a removal still settles every row on the removed anchor. - puter.events.unsubscribe() stops routing after the server ends the subscription or reports it gone; fetch() accepts `cursor`. * fix(events): forward kv values only to regions with rows that ask - A region counts its session rows asking for kv values per token and announces the token's value variant (`v#<token>`) over the existing `watch` item on the first such row, withdrawing it with the last. Add, remove, reap, reanchor and refresh all keep it, reading `includeValue` off the row as it is dropped. - A kv write reads the value variants alongside the watch index and attaches the value only to forwards bound for a region that announced one. * fix(pagination): seal id-keyed cursors on every list endpoint Apps, subdomains, workers, team (members, directory, teams, audit, own audit), shares (inbound, outbound), the user-to-user permission audit and readdir/descendant listings now hand out cursors sealed with sealCursor and read them with openCursor, so a cursor no longer carries a row id. Plain cursors issued before still read. * test(events): pass socket ids to noteConnect in the billing tests |
||
|
Daniel Salazar
|
8ecfeec8c3 |
fix(puter-js): request idle timeout, callback/listener leaks, guarded message handlers (PUT-1908) (#4143)
* fix(puter-js): free one-shot callbacks and listeners, guard message handlers - setAuthToken starts the GUI cache refresh loop once instead of per call - IPC reply callbacks are dropped after their reply; dehydrated callbacks stay - AppConnection removes its window listener when the target app closes - reply branches for app data and instance count drop their callback - UI, Debug, tool and GUI-token message listeners no longer reject; an unknown or failing tool answers toolResponse with an error - parseResponse handles a blob response with no content-type - retry waits remove their abort listener when the wait ends - EmailConfirmationDialog escapes its message - EventListener uses a real Map and snapshots listeners per emit - Context get/set/runWithContext route every known key the same way * fix(puter-js): time out requests that make no progress - buildXhr starts an idle clock on send, reset by state changes and body progress (upload progress only on requests already carrying a bearer) - read-safe requests get 60 s while progress is observable, everything else 15 min; timeout: 0 on fetchUrl/driverCall/sendWithRetry turns it off - a timed-out read replays once; writes and AI calls never replay - timeouts reject with code request_timeout on fetchUrl (TypeError), driverCall, streams and the legacy XHR path - document the limits in rate-limits-and-quotas and the chat stream note |
||
|
Daniel Salazar
|
ae234ffcc2 |
fix: only sign-in and picker popups hand the opener a token (#4147)
* fix: only sign-in and picker popups hand the opener a token Popup actions are an allowlist now: sign-in popups (no action, sign-in, login, signup) deliver at boot, pickers deliver with the pick, and request-permission keeps its exchange. Any other action runs no exchange and posts no token. The consent check in postAuthActions applies to every action that delivers at boot instead of only no-action and sign-in. * fix: a dismissed account picker hands the opener no token |
||
|
Daniel Salazar
|
2ae2e093e7 | keep archive entries inside the extract folder, encode template labels (#4145) | ||
|
Daniel Salazar
|
dcfb1263de |
fix(events): single-writer presence items, live-socket counts and region-wide leave grace (PUT-1759) (#4142)
* fix(events): keep each region's presence item written only by that region - A region that receives a forward for a pair it holds no socket for retires its own presence item through the pin-ordered leave path, gated by a region-shared claim. The sender no longer retires another region's item, and the forward reply no longer carries noSocket. - Presence reads skip items for regions outside the peer list instead of retiring them. - removeConnection decrements and deletes the count in one script. - A failed join releases the join pin, and any connect that finds the pin free writes the join. - A single delivery treats a socket on any node of the region as local, using the region-wide connection count, and picks the next remote candidate by the regions already tried instead of by index. - Correct the connection-count TTL comment. * fix(events): count presence by live socket and honor the leave window region-wide - A region's presence for a pair is a sorted set of socket ids scored by last renewal under a new key, replacing the integer count. A socket counts while renewed within the concurrency-slot window; add, renew and remove each prune lapsed members and report the live count in one script, so a node that dies stops holding the pair. - The last socket going marks the pair as leaving region-wide for the grace window, in the same script; a connect clears it. A forward that lands on another node inside the window does not retire the item. - Add the events.presence.write counter, labelled by op (join, leave, refresh, retire). |
||
|
velzie
|
50e2af34bf |
feat(puter-js): Add custom streams and perfect negotiation to Peer (#4058)
* add perfect renegotiation
* more advanced peer logic
* add video tracks to peer api
* several fixes
* small fixes, revert
|
||
|
Daniel Salazar
|
7adc03cb75 | fix: team seats caps (#4141) | ||
|
dependabot[bot]
|
807368f7b8 |
chore(deps): bump handlebars from 4.7.9 to 4.7.10 (#4137)
Bumps [handlebars](https://github.com/handlebars-lang/handlebars.js) from 4.7.9 to 4.7.10. - [Release notes](https://github.com/handlebars-lang/handlebars.js/releases) - [Changelog](https://github.com/handlebars-lang/handlebars.js/blob/v4.7.10/release-notes.md) - [Commits](https://github.com/handlebars-lang/handlebars.js/compare/v4.7.9...v4.7.10) --- updated-dependencies: - dependency-name: handlebars dependency-version: 4.7.10 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Daniel Salazar
|
4f3a524ace | fix: credit usage gui (#4140) | ||
|
Krish KumarandDaniel Salazar
|
95b6f86fcc |
fix(events): resolve relative fs: paths like fs operations do (#4085)
* fix(events): resolve relative fs: paths like fs operations do * docs(events): use bare relative paths in examples, test both relative forms * docs(events): clarify that relative fs: subjects come back resolved --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com> |
||
|
Juan Fernando Castro
|
bb829885ef |
fix: a picker popup hands its opener a token once the user has picked (#4133)
* fix: a picker popup hands its opener a token once the user has picked
A picker opens because the site asked, not because the user agreed to
anything, so the hand-off now waits for the answer the popup was opened
to get. The exchange still runs at boot — the app row and host_app_uid
are what the dialog is opened against — but the token travels with
fileOpenPicked, directoryPicked or fileSaved. Cancelling sends nothing.
deliversTokenAtBoot is the one gate every boot-time hand-off reads, so
the plain exchange, the cross-origin-isolated /login/set path, temp-user
creation and manual signup all follow the same rule. A failed exchange
still reports and closes, having nothing to defer.
* fix: the review's findings on the picker hand-off
Double-clicking a file answers the opener from openItem.js, not from the
UIWindow button, so that pick returned the item and no token. The
deliverer is a global now and every path that answers an opener calls
it: both UIWindow handlers, openItem, and the save flow.
A failed exchange posted `{success: false, token: null}`, and the SDK's
handler sets that as readily as a real token, so a picker that failed
wiped whatever the site already held. The failure now goes only where a
boot-time token would have.
|
||
|
Juan Fernando Castro
|
c55437245b |
fix: settle the url file-access gate from a grant held on a folder above (#4132)
* fix: settle the url file-access gate from a grant held on a folder above The gate asks `/auth/check-permissions` with `app_uid`, which resolved through the permission scan. `fs:` access inherits down a tree and only ACL walks that, so a grant on a folder never answered for a file inside it and the gate asked again, writing another per-file row on each Allow. The `app_uid` branch now routes `fs:<uid>:<mode>` through `ACLService.check` as the app-under-user actor, which is the same authority the operation itself will face. Everything else keeps the scan, including `puter.perms.holds`, which sends no `app_uid`. * fix: the review's findings on the consent gate A permission that is not a string reached `startsWith` and faulted the whole request, where it used to answer false inside the catch; both parsers now refuse a non-string, which also covers the other callers that read a request's list. The path spelling of a permission took the scan while the uid spelling took ACL, so one file could answer two ways. Both spellings resolve to their entry and take the same route. The checks run together rather than one after another: the caller gives up after five seconds and prompts, and a deep path is several reads. |
||
|
Juan Fernando Castro
|
d1d55a6bc3 |
SDK auth plumbing: cache scope, dedupe keys and the token listener (#4112)
* fix: pin the token sender, scope the grant flag, and stop two stragglers
Five of the SDK auth-plumbing items.
**The global `puter.token` handler pinned origin but not source.** Its three
siblings all pin both, and each says why: the GUI origin is shared by every
frame on that domain, so origin alone identifies a domain, not a sender.
`signIn` pins its popup and binds `msg_id`, the consent dialog pins its popup,
and the reauth wait pins the embedder. This one pinned the embedder only when
`env === 'app'`. Since `authenticateWithPuter` now hands off to `signIn`, which
adopts the token itself, the embedder is the only sender this handler is left
serving, so it pins in every environment.
**`is_grant_user_app_permission` was a request-wide flag held across an await.**
It tells the `app-root-dir` rewriter that a row is being written, and while it
is set that rewriter resolves to a real `fs:` permission instead of
`PERMISSION_FOR_NOTHING_IN_PARTICULAR`. Anything evaluating an `app-root-dir:`
permission beside the grant -- the grant route runs one under `Promise.all`
with fs work -- read the same flag. It now lives in a scope derived for that
call, so what the write needs to say about itself is not also said to its
neighbours. `runInDerivedContext` is new: `runWithContext` starts an empty
store, which would drop the `actor` the rewriter reads.
**`fs.stat` and `fs.readdir` deduped on parameters alone**, so two identities
in one realm could share a result. They now key on origin and token as
`os/user.js` does, which documents the reason.
**`signIn`'s relay poll never stopped.** Closing the popup settles the promise;
the loop kept asking `/login/wait` every second for the life of the page.
**`triggerReauth`'s five-minute timeout was never cleared**, so a successful
reauth left a timer to fire into a settled promise.
The concurrency test parks the rewrite and reads the flag while it is held --
my first attempt read before the flag was even set and passed either way.
* fix: scope the fs cache per identity, and retarget two paths that never ran
The remaining SDK auth-plumbing items.
**`puter._cache` outlived the identity that filled it.** `_clearAuthToken`
dropped the token and the stored session but left the cache and `whoami` as
they were, and its keys were raw paths -- `item:/alice/Documents` and a
`~`-relative key alike, identical whoever asked. A GUI user switch without a
reload could be served the previous user's stat or readdir. Keys now carry the
origin and token they were read as, built in one place so the writers, the
readers, the invalidations and the warm-up probes cannot drift apart, and
signing out drops the cache and `whoami` with the token.
**The driver permission prompt could not fire.** Both paths waited for a 200
carrying `{success:false, error:{code:'permission_denied'}}`. The driver route
answers `success: true` or throws, so a refusal arrives as a 403 -- the shape
the retry now reads. Its two tests synthesised the old envelope, so they passed
against a response the API does not send; they use the real one now, and the
prompt goes from never called to called once.
This is a behaviour change: a driver call refused for permissions will now
prompt, which is what the path was written to do. Happy to delete it instead if
the prompt is not wanted.
The half-written copy of the same idea in `utils.js` is gone -- it prompted for
one hard-coded interface and its retry was a `// todo`.
**The verification-gate single-flight was keyed on nothing**, so a phone-gated
request in flight beside an email-gated one took the email answer, replayed,
failed again and surfaced raw without ever asking. Keyed per gate: same gate
still shares one dialog, a different gate gets its own.
* test: ask the cache the same way the code does
The API suites built `item:<path>` keys by hand, so they stopped finding
anything once the real keys carried the identity they were read as. They go
through `fsCacheKey` now, which is the point: a test that spells the key itself
passes until the moment the spelling changes, and then fails for a reason that
has nothing to do with the behaviour it is guarding.
These run against a live server, so a plain `vitest run` does not reach them --
they were green locally and red in CI.
* fix: the review's findings on this branch
Eight, and three of them were regressions this branch introduced.
**Pinning the token sender to `globalThis.parent` in every environment broke
the case it was meant to protect.** On a top-level page `parent` is the page
itself, and the token arrives from a popup, so the handler stopped accepting
anything. The GUI posts `puter.token` to `window.opener` for the file pickers
too, not only for sign-in, and this handler is their only consumer -- a site
calling `showOpenFilePicker()` would have signed the user in, dropped the token
and run the following read with none. It now pins the embedder when framed and
a window this SDK opened when not; `UI.js` already tracked its picker popups
for the same reason, so that set is shared rather than duplicated.
**The GUI still spelled a cache key by hand.** `updateSubdomainsForItems` built
`item:<path>`, which no longer matches what the SDK writes, so its `get` could
not hit and its `set` wrote an orphan nothing reads and nothing evicts -- a
published website's badge quietly stopped appearing on a cached render. It goes
through `fsCacheKey` now, and that is the last hand-spelled key in the tree.
**Retargeting the driver permission prompt to 403 was wrong and is reverted to
the deletion the ticket asked for.** The route refuses with `forbidden`, not
`permission_denied`, so the retarget still missed it; meanwhile a driver that
raises `permission_denied` for its own reasons would have opened a prompt that
had nothing to do with the refusal. And the prompt asks for
`driver:<iface>:<method>` while the route checks `service:<driver>:ii:<iface>`,
so granting it changes nothing. The path cannot work as written.
The rest were pre-existing or latent:
`cacheWhoami_` wrote its answer without checking the token it was issued under
was still current, so a reauth landing mid-flight could repopulate `whoami`
with the previous identity. `_dropIdentityCaches` also left `whoamiCache_`,
which other call sites await.
`setAuthToken` now drops the identity caches when the token actually changes:
keys carry the token and the store is persisted with no TTL, so a rotation that
did not go through `_clearAuthToken` orphaned a whole key space with nothing to
reclaim it.
The verification-gate entry was registered after the body could already have
deleted it -- a synchronous throw stranded a resolved "not verified" in the map
for the life of the page. It registers first and clears on a microtask.
`signIn`'s poll is bounded as well as settle-checked: on a cross-origin
isolated page opened from a gesture, nothing watches the popup, so closing it
never settled the promise and the loop ran on exactly as before.
`runInDerivedContext` says in its docblock that callee writes do not survive;
only the one flag needs the scope, and a future caller should not expect to
hand a value back through it.
* fix: the review's findings on the SDK auth plumbing
**The relay bound stopped asking without saying so.** The deadline left the
`signIn` promise pending and its listener attached. On an isolated page opened
from a gesture the relay is the only thing that can settle it -- nothing watches
the popup there -- so a user who took longer than the deadline was left with a
promise that never resolved, and `authenticateWithPuter` queues every later
implicit-auth call behind it. It rejects now.
**Pinning the token source left `signIn`'s own popup untracked.** Only the
picker popups registered, so on a top-level page the global handler bailed on a
sign-in it had opened itself: `puter.onAuth` never fired, and
`puterAuthState.authGranted` was never set. `openAuthPopup` is the one place
auth popups are opened, so they register there.
**A closed popup is ours too.** Rejecting one that had already closed loses a
message from a popup that posts and then closes itself, which is why `UI.js`
tolerates the same case. Kept, with the set bounded instead.
**The gate key carried the factors**, so one route naming them and another not
opened two dialogs for the same gate. It keys on the gate, as `ctx.done` does.
**`fsCacheKey` left `~` unresolved**, so a writer and an invalidation spelling
the same place differently missed each other -- the drift the one key builder
was meant to end.
The dead `opts.permission` went with the path that used to read it.
Four workerd-runner failures are left: `ai > chat stream adapter supports
cancellation`, `kv > a GUI boot key written by another client reads fresh` and
two cache-warmer ones. All four fail the same way on `origin/main`, and the node
and browser runners pass.
* chore: one-line the comments added here
* chore: reflow the comments the removed retry cause left ragged
|
||
|
Michael Lip
|
4ea93e6a73 |
fix(i18n): add GUI keys that rendered as raw key names (#4074)
The 2FA spinner showed "verifying", the files selection bar showed a "done" button, and two login and signup errors showed their key names, because i18n() returns the key itself when en.js has no entry for it. The login error now uses the existing login_email_username_required key, which already has translations. A test checks that every literal i18n() key in the GUI exists in en.js. |
||
|
Daniel Salazar
|
80736de0f1 |
test(events): wait for the recorded failure, not any future lease score (#4127)
The in-flight claim's lease (2x the invoke timeout) is also a future score, so polling heldForMs > 0 could pass before the failed attempt's hold was written: the backoff test read 600 instead of 2000, and the 404/hang/429/suspended-owner tests ended while their attempt was still in flight, letting it land in the next test. Wait on the subscription's failure counter instead, which only moves after the hold or gap marker is written. |
||
|
dependabot[bot]
|
085ab1774f |
chore(deps): bump shell-quote from 1.10.0 to 1.12.0 in /src/puter-js (#4120)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.10.0 to 1.12.0. - [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md) - [Commits](https://github.com/ljharb/shell-quote/compare/v1.10.0...v1.12.0) --- updated-dependencies: - dependency-name: shell-quote dependency-version: 1.12.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
505fd13d97 |
chore(deps): bump shell-quote from 1.9.0 to 1.12.0 (#4121)
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.9.0 to 1.12.0. - [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md) - [Commits](https://github.com/ljharb/shell-quote/compare/v1.9.0...v1.12.0) --- updated-dependencies: - dependency-name: shell-quote dependency-version: 1.12.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |