mirror of
https://github.com/HeyPuter/puter.git
synced 2026-09-27 15:46:30 +00:00
5065bcbf0a6f55ae4dfc06ce33c1cb0400a80524
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Daniel Salazar
|
ec3c33fea6 | fix: misc hardening fixes (#3600) | ||
|
|
2c852bf6b3 |
✨ PUT-1412 File sharing backend api (#3553)
* refactor(permissions): drop hardcoded group permission map for a flat default
* test(drivers): assert credential-gate intent instead of a 403 proxy
* fix(permissions): report whether a revoke removed anything and persist the linked grant row before the flat view
* feat(permissions): replicate permission invalidations across regions
* feat(share): extend the share table into an index of active shares
* feat(share): query and maintain active shares in ShareStore
* feat(users): add a batched lookup by email
* fix(cache): apply cache updates broadcast from peer regions
* fix(permissions): scope a revoke to the issuer that granted it
* feat(share): add ShareService with a per-day share limit
A share is two writes that belong together: the permission grant, which
authorizes access, and a share row, which makes it listable and ties it to an
fsentry so it dies with the file. Nothing else grants fs:* to a user.
Authorization reuses canManagePermission — an owner satisfies it through the
is-owner implicator, a delegate through an explicit manage:fs:<uid> grant. An
owner may clear any issuer's share of their node; anyone else only the ones
they issued, or their own access. Self-revoke skips the manage gate but still
requires `see`, so it cannot be used to probe for files.
The per-day limit counts shares created rather than live rows, so revoking and
re-sharing cannot recycle a slot, and changing an existing share's mode is not
new reach and does not spend budget. Tunable via share_daily_limit.
* feat(share): expose sharing over HTTP
POST /share, POST /share/revoke, GET /share/shared-with-me, GET /share/shares.
The controller was registered but entirely commented out.
Recipients × items fan out concurrently — every pair is a distinct
(holder, entry) key, so none of them contend — bounded by
runWithConcurrencyLimitSettled, which returns results index-aligned with the
input for the per-pair outcome list. Responses carry usernames only, never
internal ids, and the 404-not-403 rule is preserved so a failed call cannot
confirm a file the caller could not otherwise see. Notifications are fired off
the response path; a share must not fail over its own notification.
Per-request caps on recipients and items bound one call's fan-out; the daily
limit bounds the total.
* feat(share): keep recipients consistent when a shared item changes
* fix(fs): stop listing issuer homes at the filesystem root
* fix(acl): serialize concurrent mode changes on one node and pin app containment on shared paths
* fix(fs): expire signed URLs over entries the signer doesn't own
signFile defaults to a ~317k-year TTL and verifySignature checks only uid,
expires and signature — never the ACL. A recipient who ever signed a shared
file therefore held a permanent, revocation-proof URL to its bytes: revoking
the share did nothing to it.
signEntry now takes the acting user and drops to NON_OWNER_SIGNATURE_TTL_SECONDS
(1 hour) when the signer is not the entry's owner. Owners keep the permanent
default, so no existing client changes behavior.
The signature-authenticated directory listing bounds its children
unconditionally: that route has no session actor, and a signature proves
possession rather than ownership, so a recipient holding a short-lived
directory signature could otherwise mint permanent URLs for every child.
A bounded window is not revocation — the durable fix is a per-entry signature
epoch folded into the HMAC and bumped on any permission change.
* refactor(permissions): drop the unused permission-issuer lookup
listUserPermissionIssuers and its store method listUserPermissionIssuerIds
existed to synthesize the filesystem root from the home directories of everyone
who had granted the caller a permission. That listing is gone — it advertised
folders readdir then refused to open — and the share index answers "who shared
with me" directly, so nothing wants them back.
One removed test only asserted that the call returned an array; the other
covered readLinkedUserUserPerms round-tripping and is kept, rewritten without
the issuer lookup.
* fix(share): return the created share, not just an acknowledgement
* feat(puter.js): add file sharing to puter.fs
share(), unshare(), listShared() and getShares() on puter.fs, following the
existing FS operation shape: positional and options-object forms through
defineOperation, JSDoc overloads as the published signature, relative paths
resolved against the app's root directory.
A bare recipient string is read as an email when it contains @ and as a
username otherwise. Sharing an item with someone who already has it replaces
their access rather than stacking a second grant, so raising read to write is
one more call.
Adds a sharing suite to the API runner, which passes unchanged on node,
browser and workerd. Documents all four methods with runnable examples, and
corrects the FS overview callout that told readers one user cannot read
another's files — true before this, not after.
* feat(gui): add a Shared folder for items others shared with you
A sidebar entry listing everything other users have shared with you, backed by
puter.fs.listShared().
The path is the sentinel `puter://shared` rather than /<user>/Shared: this is a
query, not a directory, and a path-shaped value could collide with a folder
someone actually creates. refresh_item_container and update_window_path both
branch on it to skip the stat there is no fsentry for, and the listing swaps
readdir for listShared.
Entries render at their real paths under their owners' directories — the item
container already preferred an explicit fsentry.path over joining onto the
container, so nothing else had to change. Each carries who shared it and at
what level, which the context menu reads next.
* feat(gui): share items from the context menu
A sharing dialog shaped like its neighbours — options object, HTML-string
template, jQuery wiring, delegating to UIWindow() — with a recipient field, a
read/edit/share dropdown, and the current access list with revoke buttons.
Reached from a new "Share…" context menu entry, which is hidden on items shared
*with* you: re-sharing needs manage, so the dialog would only surface an error.
Those items get "Remove from Shared" in place of Delete. Delete moves an item
to *your* trash, which for someone else's file means moving their data out of
their tree — FSService refuses it, and the user saw a bare 403. Removing your
own access is what the action was reaching for, so that is what it now does.
* fix(share): withdraw what a removed recipient re-shared
* feat(share): report access inherited from a parent folder
* fix(gui): load the puter.js bundle the server configured
* refactor(gui): extract the action icon set into a helper
* feat(gui): surface Shared in the file browser
* feat(gui): manage access from the share dialog
* test(share): cover access inherited from a parent folder
* fix(share): keep downstream access from surviving a delegate who leaves
* fix(gui): name the real owner in the share dialog
* fix(gui): page through every shared item instead of the first 50
* feat(share): return item metadata with a share
* fix(share): invalidate a holder's cache when the entry is deleted
* fix(gui): treat items inside a shared folder as someone else's
* feat(permissions): let manage inherit down the filesystem tree
Access already reached descendants through the ancestor chain while authority did not, so someone trusted to manage a shared folder could re-share the folder but nothing inside it, and could not see who had access to a file within it.
A manage-inherits-from-ancestor implicator resolves it in the permission layer, beside is-owner, so every caller agrees rather than just ShareService. It consults only the immediate parent — resolving that re-enters one level up, making a chain of depth d cost d checks rather than d².
That makes two cascade gaps reachable, both fixed here. A revoke now walks the subtree, since a grant on a descendant can rest on authority held at the folder. And it stops at a delegate whose authority survives another issuer, because what they granted was never theirs to lose.
Also pins that manage is not transitive: granting it needs manage:manage:fs:<uid>, which only the owner holds, so delegation is one level deep by construction.
* fix(gui): offer sharing inside a folder you manage
The menus encoded "manage does not inherit" and would now hide an action that works. The Shared listing records each root's mode; the menus resolve a child's by longest matching ancestor, loading on demand so a deep link or restored window works too.
* fix(share): make the daily share limit hold under concurrency
* test(share): cover concurrency, measure cost, and name cases for what they verify
* fix(gui): import the ownership helpers the item menu calls
The single-item context menu handler calls is_owned_by_me and
shared_mode_for, but the imports were only ever added to
generate_file_context_menu.js — so every right-click on an item threw a
ReferenceError before the menu could build, and the non-owner Delete
gating never ran.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): authorize before resolving the recipient
share() looked up the recipient in parallel with the entry, before the
manage check — and the two failures carried different error codes. Any
verified user with a real entry uid could probe arbitrary emails and
usernames for account existence, at no quota cost. Resolve the entry,
authorize, and only then resolve the recipient: an unauthorized caller
now sees the identical safe 404 whether or not the recipient exists.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(permissions): broadcast permission row-cache invalidations to peer regions
Every publishCacheKeys call for the u2u, u2a, and access-token row
caches omitted broadcast, so a revoke only cleared the mutating
region's Redis. A peer region applied the replicated generation bump,
re-scanned, read the deleted row from its own still-warm 5-minute row
cache, and re-warmed the flat view from it — revoked access outlived
the revoke by the row-cache TTL instead of the intended 60-second
bound. CacheReplicationService already consumes these events; the
emits were just never sent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(fs): refuse to rename an entry owned by another user
remove and move both refuse to act on an entry the caller does not
own, even when the ACL allows the write — rename had no such guard, so
a write-mode share recipient could rename the owner's file, or the
shared folder itself, rewriting the owner's whole subtree's paths.
rename now takes the acting user and applies the same policy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(permissions): decide a flat delete from the primary, not a lagging replica
revokeUserUserPermission deletes the SQL grant, then only drops the
flat KV entry once no issuer still grants the permission. That
remaining-check read through the row cache the delete had just
invalidated, straight to a replica — under any lag the deleted row
reappeared, the flat delete was skipped, and the stale rows were
re-cached for another five minutes. Grant-path flat entries carry no
TTL, so the holder kept working access with zero SQL rows behind it,
invisible to every listing. The check now reads the primary and
re-warms the cache with what it actually saw.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(permissions): keep a failed remote flat-invalidation from crashing the process
The outer.permission.flatInvalidated applier was fire-and-forget with
no catch, and it awaits a KV delete — one transient KV error while
applying a peer region's revoke became an unhandled rejection, which
is process-fatal under default Node. Its sibling appliers were already
guarded; this one now logs and moves on, leaving the entry to the next
invalidation or its TTL, same as a lost event.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): revoke every requested item and recipient, not just the first
revokeShare destructured only the first recipient and first item while
the parsers accept arrays up to the request caps — unshare({items:
[a, b, c]}) returned success having revoked only a, leaving access the
caller believes is gone. Revoke now fans out over every (recipient,
item) pair exactly like POST /share, reports per-pair outcomes, and
sums the revoked count; the response stays backward compatible.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): only a confirmed email designates a recipient
Recipient resolution by email accepted unconfirmed accounts, so
pre-registering someone else's address (unconfirmed) was enough to
receive shares meant for them once no confirmed account held it.
An email now only resolves to an account that has confirmed it;
username shares are unaffected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): accept tilde-rooted paths like the FS routes do
The SDK resolves relative paths to ~/..., but the share routes never
expanded the tilde — a ~-prefixed string was read as a uid and every
relative-path call 404'd. Item parsing now treats ~ as path-shaped and
expands it to the actor's home with the same helper the legacy FS
routes use, on share, revoke, and the shares listing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(share): walk a directory revoke by parent linkage, not path prefix
listByFsentrySubtree matched descendants with fsentry_id = ? OR path
LIKE ?, which has two problems: fsentries.path is lazily backfilled
and NULL on old rows, so those descendants' shares silently survived a
directory revoke, and the OR'd predicates forced a scan of every
active share. A recursive CTE over parent_id — the same shape the
lineage resolver already uses — covers every descendant and runs on
idx_parentId_name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(gui): give each item its own share dialog
single_instance keyed the dialog on the app id alone, so opening
Share… on a second file focused the first file's dialog — typing a
recipient there granted access to the wrong file, with only the title
hinting at it. The dialog is now instanced per path: same item
refocuses, different item opens fresh. Also stops pre-encoding the
title, which UIWindow encodes again.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(permissions): let manage answer a write check
A manage grant let its holder re-share a folder but not work in it: the
ACL mode family stops at write, and the fs exploder had no rule for the
narrowest mode, so `manage:fs:<uid>` never satisfied `fs:<uid>:write`.
Fold manage into the candidate list for every non-manage mode, in both
the access-token branch and the scan branch, and give `write` an (empty)
exploder rule so the manage arm is emitted for it too.
* fix(fs): authorize restructuring by write on the parent
* fix(fs): let a share recipient work inside a shared folder
rename, remove and move refused outright when the entry belonged to
someone else, so a recipient with write could neither delete nor rename
anything inside a folder shared with them. The GUI compounded it by
hiding Delete for any item it did not own.
Authorize the three by ACL write on the entry's parent. For an owner
that is the same answer; for a recipient it grants the inside of a
shared folder and withholds the folder itself, whose parent is the
owner's private tree.
Deleting sends the item to its owner's trash rather than the deleter's,
so it leaves the recipient's view without leaving the owner's account
and without changing hands. A move may not otherwise carry someone
else's entry out of their tree.
* fix(fs): give a new entry to the owner of the folder it lands in
A file a share recipient added to a shared folder was recorded as
theirs while living in the owner's tree, so a subtree could hold rows
belonging to several people — and the storage it consumed was checked
against the writer while being counted against the owner.
Take the owner from the parent row at every insert, charge the
allowance to that owner, and hand a moved entry over to the tree it
moves into. An entry now always belongs to whoever owns the directory
holding it.
* feat(fs): address shared entries as ~/share/<uid>
A recipient could read the owner's whole path off any shared entry —
where they keep the file and what sits beside it, neither of which the
share is about.
Give shares their own namespace. `~/share/<entry-uid>/rel/path` resolves
to the real path on the way in, and outgoing paths are rewritten to it
on the way out. Entries the actor owns pass through untouched, so no
existing client contract moves.
* revert(fs): mask only the directory bar, not the addressing
|
||
|
Daniel Salazar
|
20b3b88e39 |
metering: big fixes to metering + jsdoc types (#3547)
Changes are: - global egress metering - remove file egress cost - introduce file op cost for the per request cost s3 has - enforce fs read/download etc to through 402 when out of usage; allow for subdomains - enforce kv metering when out of usage through 402; allow for workers - jsdoc as source of truth for puter.js types - kv driver caching for get and batchget operations with decreased costs |
||
|
Nariman Jelveh
|
62be61642d |
fix: phantom name conflict when pasting onto a just-renamed path (#3495)
Renaming (or moving) an entry left the OLD path's cache key serving the pre-update entry for the full 60s TTL: updateEntry invalidated only keys derived from the updated row, whose path is already the new one. Pasting an item under the freed name then hit the stale cache in the collision check and reported a conflict for a file that no longer exists — and accepting the Replace it offered deleted the renamed file, since the stale entry carries its uuid. Read the pre-update entry when the patch changes the path and invalidate its keys alongside the new ones. The move flow was shielded by the outer.gui.item.moved cache-invalidation handler at the controller layer; rename had no such band-aid, and fixing the store covers every caller regardless of which events fire. |
||
|
Daniel Salazar
|
116d6e6663 | tests: big test push for better coverage (#3490) | ||
|
Daniel Salazar
|
e85cd9d53d | feat: readdir with depth (#3446) | ||
|
Daniel Salazar
|
3a8b6394de | feat: standardized api pagination (#3412) | ||
|
Daniel Salazar
|
17a9485fb2 | Add spans to drivers and high frequency paths (#3376) | ||
|
Daniel Salazar
|
34a7595a6d | fix: bad empty file error handling (#3293) | ||
|
Daniel Salazar
|
240a733285 |
sec: misc fable hardening (#3244)
* sec: misc fable hardening * more fixes * more fixes * fix: cors issue |
||
|
Daniel Salazar
|
6d2f277ce2 | fix: some error handling (#3190) | ||
|
Ron HernausandDaniel Salazar
|
d2fee51844 |
Support PostgreSQL database backend (#3167)
* feat(database): add postgres database client Adds the PostgreSQL database client, native bootstrap migration, SQL preparation helpers, and database config/factory wiring.\n\nRefs #3165. * feat(database): make backend queries postgres-aware Updates runtime SQL call sites for database-specific booleans, identifiers, insert-ignore, upserts, JSON extraction, intervals, and Postgres insert ids.\n\nRefs #3165. * test(database): cover postgres client behavior Adds unit coverage for SQL preparation, factory selection, write-result mapping, and transaction rollback/commit ordering, plus an env-gated PostgreSQL integration flow.\n\nRefs #3165. * docs(self-hosting): document postgres database setup Adds PostgreSQL configuration examples and migration path guidance for self-hosted deployments.\n\nRefs #3165. * fix: harden postgres oidc tests * fix(postgres): normalize query results and SQL prep * fix(user): preserve normalized cache booleans * test(postgres): run integration coverage with pgmock * tests: add way to run all tests with postgres though slow Also adding note that postgres is not in active use so might not work out the box --------- Co-authored-by: Daniel Salazar <daniel.salazar@puter.com> |
||
|
Daniel Salazar
|
b188942436 |
feat (PUT-1016 & PUT-1020) (#3164)
* feat (PUT-1016 & PUT-1020) temp account preservation on forced relogin hosted asset cookies to v2 token too * fix: remove llm dashes and ugly comments * update agents |
||
|
Daniel Salazar
|
101e0bc930 | fix: search leaking metadata (#3150) | ||
|
Daniel Salazar
|
b04e8b046a |
fix: fixing misc issues around alerting and bugs (#3141)
- decrease ai related alerts and make errors not 500 - decreased bad fs controller checks to be 404s instead of 500s - force bucket regions for now - validate webdav perms for locking/unlocking |
||
|
ProgrammerIn-wonderland
|
93edec0d01 | assert normalized (#3119) | ||
|
Daniel Salazar
|
9bcd77c85b | chore: add legacy codes back to all errors (#3022) | ||
|
Daniel Salazar
|
f9ac017faf | fix: fsEntry and tscofnig (#2923) | ||
|
Daniel SalazarandCopilot
|
50679e6a01 |
selfhosting: p2 fixed default configs and guide + some other errors (#2903)
* fix: ollama support * fix: more self host cleanup * fix: bad auth * fix: more session cookie stuff * allow emailclient Co-authored-by: Copilot <copilot@github.com> * fix: broken migration * fix: full-stack comment * fix: hardcoded perms * new docker build steps --------- Co-authored-by: Copilot <copilot@github.com> |
||
|
Daniel Salazar
|
1796fdb37b |
fix: misc fs issues (#2883)
* fix: deps * fix: FS issues |
||
|
Nariman Jelveh
|
267f464232 | Add AGPL license headers to source files (#2877) | ||
|
|
d4d78ac7db |
rework: change backend and backend extensions to use simpler code structure and patterns (#2815)
* fix: dynamodb health checks and client recreation (#2789) * wip: no nanoServices groundwork * feat: data clients in new shape * wip: auth and perms in new system * more wip * middlewaters mainly done * wip: fsv2 in new layout * old fs v2 migration * driver system * driver and old fs fixes * ai drivers wip * stream support * metering in ai chat driver * wip: new auth * rate limit and auth routes * captcha and anti csrf * fix: types * auth store * app logic * wip most other dricvers * fs * mostly kill all legacy stuff * fs finish * fix: redis usage * ai controller * driver cleanup * socket io in v2 * broadcast and crudq stuff * subdomains * notifcations and shares * fix bad syntaxes * auth wip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * extensions * extension setup * more routes * sql migrations and default services * home router * tier 7 * everything else * everything else * remaining missing bits * server health * logs * cleanup * deps * cleanup 2 * more cleanup 2 * boot * fix launch * config fix * move file * fix: tsconfig things * fix: extension loading * launching * fix: drivers * fix: others * fix: icons * fix: file uploads * fs fixes * fix: fs api * fix: dev-center * config * add back telemetry * lint stuff * husky hooks * fix: fs oss * fix: config migration * config migration * migrate scripts + replicate * runner * fix: merge defafult config * fix: default region * fix: api domain * fix paths in readfile * fix fs entry default s3 * NS: Remove Referral && Entri Service * dep cleanups * fix: static assets * fix: kv and perms * fix: driver registrations * fix: home mapping * fix: rao * adding back 500 alarm * fix: build paths * fix: fs and kv shapes * fix: kv shape * more kv coercing and ai chat matching format as prior * fix: private app gates * private app caches * fix: whole bunch of legacy shape issues * update template jsonc * fix caching partial oidc and fs signed paths * more oidc fixes * fix: wip * fix: private apps * admin route fixes * fix: last few things hopefully * claude uploads * fix security for app only routes * fix kv system namespace * stuff * fix: app and kv and suggested apps * fix:open item * fix: FS operations * fix: default app icons * add back token-read and WSL support * metering fixes * fix: fsEntry * perm scanners and implicators * proper download endpoint * fix: download * fix anti csrft on v2 * fix file extensions, app icons * fold in v1 fixes from origin/main into v2 equivalents Re-applies the v1 fixes that landed on origin/main into their v2 counterparts since the v1 files were deleted on DS/wip during the v2 migration. v1 commits referenced below. - SQLBatcher: flush immediately when queue hits maxBatchSize instead of racing the timer (v1 12f48238). - RedisClient: drop maxRetriesPerRequest from 2 to 1 to shrink failure window (v1 |