159 Commits

Author SHA1 Message Date
ProgrammerIn-wonderland ce18b914b4 backport devwatcher (#2861) 2026-04-30 16:23:34 -07:00
Daniel Salazar d4d78ac7db rework: change backend and backend extensions to use simpler code structure and patterns (#2815)
* fix:  dynamodb health checks and client recreation (#2789)

* wip: no nanoServices groundwork

* feat: data clients in new shape

* wip: auth and perms in new system

* more wip

* middlewaters mainly done

* wip: fsv2 in new layout

* old fs v2 migration

* driver system

* driver and old fs fixes

* ai drivers wip

* stream support

* metering in ai chat driver

* wip: new auth

* rate limit and auth routes

* captcha and anti csrf

* fix: types

* auth store

* app logic

* wip most other dricvers

* fs

* mostly kill all legacy stuff

* fs finish

* fix: redis usage

* ai controller

* driver cleanup

* socket io in v2

* broadcast and crudq stuff

* subdomains

* notifcations and shares

* fix bad syntaxes

* auth wip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* extensions

* extension setup

* more routes

* sql migrations and default services

* home router

* tier 7

* everything else

* everything else

* remaining missing bits

* server health

* logs

* cleanup

* deps

* cleanup 2

* more cleanup 2

* boot

* fix launch

* config fix

* move file

* fix: tsconfig things

* fix: extension loading

* launching

* fix: drivers

* fix: others

* fix: icons

* fix: file uploads

* fs fixes

* fix: fs api

* fix: dev-center

* config

* add back telemetry

* lint stuff

* husky hooks

* fix: fs oss

* fix: config migration

* config migration

* migrate scripts + replicate

* runner

* fix: merge defafult config

* fix: default region

* fix: api domain

* fix paths in readfile

* fix fs entry default s3

* NS: Remove Referral && Entri Service

* dep cleanups

* fix: static assets

* fix: kv and perms

* fix: driver registrations

* fix: home mapping

* fix: rao

* adding back 500 alarm

* fix: build paths

* fix: fs and kv shapes

* fix: kv shape

* more kv coercing and ai chat matching format as prior

* fix:  private app gates

* private app caches

* fix: whole bunch of legacy shape issues

* update template jsonc

* fix caching partial oidc and fs signed paths

* more oidc fixes

* fix: wip

* fix: private apps

* admin route fixes

* fix: last few things hopefully

* claude uploads

* fix security for app only routes

* fix kv system namespace

* stuff

* fix: app and kv and suggested apps

* fix:open item

* fix: FS operations

* fix: default app icons

* add back token-read and WSL support

* metering fixes

* fix: fsEntry

* perm scanners and implicators

* proper download endpoint

* fix: download

* fix anti csrft on v2

* fix file extensions, app icons

* fold in v1 fixes from origin/main into v2 equivalents

Re-applies the v1 fixes that landed on origin/main into their v2
counterparts since the v1 files were deleted on DS/wip during the v2
migration. v1 commits referenced below.

- SQLBatcher: flush immediately when queue hits maxBatchSize instead
  of racing the timer (v1 12f48238).
- RedisClient: drop maxRetriesPerRequest from 2 to 1 to shrink failure
  window (v1 b6776ab4).
- ChatCompletionDriver: default minimumCredits to 1 when unset/zero so
  zero-cost precheck doesn't auto-pass (v1 36bd6073).
- OpenAiImageProvider: add gpt-image-2 support — open-ended size rules,
  token-based cost estimator, arbitrary-size normalizer, isGpt prefix
  broadened to gpt-image- (v1 f14f1bf4). models.ts auto-merged via
  rename detection.
- AppStore: bump row cache TTL from 5m to 24h (v1 6b3196ed).

Not ported: v1 app-object Redis cache (bdfa12b5/b886dde3) — v2's
#toClient recomputes filetype_associations/created_from_origin per
read; adding a second cache layer is a larger change for a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* remoe anti-csrf from auth routes that had not used them

* more icon fixes

* fix worker functionality

* fix: app and subdomain es

Co-authored-by: Copilot <copilot@github.com>

* fix PUT-761

* fix: PUT-748

* fix: rename fsService

* Add security back to WorkerDriver

* Migrate worker from fsEntry to fs. Fix cache issue

* remove ability to create symlinks

* strict webdav acl

* require auth for wisp

* chore: service renames

* Add metering back to puter peer api

* fix: PUT-760 PUT-749

* fix: PUT-746

* fix: peer cost

Co-authored-by: Copilot <copilot@github.com>

* fix: 771

* change order of peer controller

* fix: create appdata folder for app on get auth token

* fix: align delete site and list sites

* delete: putility

* fix subdomains

* Add support for tilde in subdomains, fix subdomain update

* cleanup PeerController.ts and fix billing oversight (#2844)

* fix: PUT-786

* fix: bugs

* fix: issues with multiple subdomain queries, or permission checks

* fix: harden response shapes to not contain uneeded fields

* fix: move state to redis

* fix: missing kv methods + better sec

Co-authored-by: Copilot <copilot@github.com>

* fix: subdomainStore limit

* fix: missing path resolution

Co-authored-by: Copilot <copilot@github.com>

* fs fixes

* fix: undef error

* fix fs + cleanup

* fix: npm audit fixes

* heal path entries where missing

Co-authored-by: Copilot <copilot@github.com>

* fix: caching

Co-authored-by: Copilot <copilot@github.com>

* fix: cache inconsistencies

Co-authored-by: Copilot <copilot@github.com>

* fix: app driver metadata

Co-authored-by: Copilot <copilot@github.com>

* remove extraneous comma

* fix: associated app icons

* fix: bad tool call

* Add validation to WorkerDriver#getFilePaths

* misc fs and auth issues

Co-authored-by: Copilot <copilot@github.com>

* fix: oidc errors

Co-authored-by: Copilot <copilot@github.com>

* fix: PUT-797

* fix: legacy appdata_app

Co-authored-by: Copilot <copilot@github.com>

* fix: add alert logs

Co-authored-by: Copilot <copilot@github.com>

* fix: error handling

* Disable sharecontroller

* fix: remove private user identifier for ai

* fix: private app fixes

* Add backback signup_server

* fix: completionId size

Co-authored-by: Copilot <copilot@github.com>

* fix: revalidate path for oidc

* fix: revalidate path for oidc

* fix: email validation

Co-authored-by: Copilot <copilot@github.com>

* fix: user create query

* fix: middleware extensions

Co-authored-by: Copilot <copilot@github.com>

* use x-forwarded-for for req ip forwarded

* fix: missing last_activity ts

* feat: add cache broadcast to subdomains

* fix: update config typing

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ProgrammerIn-wonderland <3838shah@gmail.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Nariman Jelveh <nj@puter.com>
Co-authored-by: velzie <velzie@velzie.rip>
2026-04-30 12:13:43 -07:00
Daniel Salazar 7f9873edd2 fix: cdn cache invalidation (#2826) 2026-04-20 11:59:35 -07:00
ProgrammerIn-wonderland 649db33763 Ns/simplify 1 (#2764)
* Rid devconsole

* lots of removals

* more commandservice
2026-04-06 14:05:48 -04:00
Daniel Salazar 5319940156 fix: app id shortcut batch (#2768)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
Notify HeyPuter / notify (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-04-04 17:21:58 -04:00
Daniel Salazar 95cbbc5de6 feat: enable singed uploads on oss and fix broken migration (#2765)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
Notify HeyPuter / notify (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
* fix: proper shutdown

* feat: enable singed uploads on oss

* fix: s3 in oss

* fix: thumbnails in oss
2026-04-03 07:29:22 -07:00
Daniel Salazar 99d96edd9c feat: s3 fs in oss (#2761)
* feat: s3 fs in oss

* feat: new endpoints in OSS

* fix: name of fs extension

* perf: signed uploads

* fix: await socket events to align fs events

* fix: default bucket names

* fix: backend tests

* fix: deps

* fix: order
2026-04-02 11:20:59 -07:00
Daniel Salazar ef243c9854 feat: signed upload urls (#2753)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
Notify HeyPuter / notify (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
* feat: signed upload urls

* fix: make sessions write to dynamo

* fix: add logs for failures
2026-03-30 16:27:29 -07:00
jelveh 813e78843a Remove debug log; update captcha modal styles 2026-03-28 11:04:10 -07:00
Daniel Salazar b3656fdaa1 fix: handle closing server nicer on shutdown signal (#2741)
* fix: handle closing server nicer on shutdown signal

* fix: bad check
2026-03-27 17:24:33 -07:00
Daniel Salazar b857fc8743 fix: return fallback icons if no icon present (#2731)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
Notify HeyPuter / notify (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-03-25 16:50:34 -07:00
Daniel Salazar ec766eecc2 fix: syntax issue with sql (#2730) 2026-03-25 15:05:43 -07:00
Daniel Salazar 187b14dacd fix: remove last opened (#2729) 2026-03-25 14:43:06 -07:00
Daniel Salazar 08fa2d7a41 fix: installed apps api (#2725) 2026-03-25 09:41:53 -07:00
Daniel Salazar 25b9911651 fix: extension type import (#2723)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
Notify HeyPuter / notify (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-03-24 23:32:12 -07:00
Daniel Salazar 9511323cdf fix: init installed apps controller (#2722) 2026-03-24 23:17:46 -07:00
Daniel Salazar e75ccb0a41 feat: root level kv accesses, and installed app listing + server health check fix (#2719)
* feat: root level kv accesses, and installed app listing

* fix: revert server health check
2026-03-24 19:18:42 -07:00
Eric Dubé 7ff1ecfcd7 dev(fs): fetch subdomains using join in readdir (#2647)
* dev(fs): fetch subdomains using join in readdir

* fix(fs): handle readdir of root directory
2026-03-11 20:06:13 -04:00
Eric Dubé 2a027ed410 fix(puterfs): await queue items for fsentry action (#2616)
* style(puterfs): update adherence to linter rules

* fix(puterfs): await queue items for fsentry actions

This will ensure some operations always function as expected in a
sequence, although it may incur some performance costs.
2026-03-06 20:04:18 -05:00
Daniel Salazar a861c3e27a chore: typeFixes (#2615)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-03-06 00:04:39 -08:00
Daniel Salazar 911c163fc8 feat: private app config to use app urls + app routing (#2587)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
* feat: private app config to use app urls

* fix: launch app

* fix: cookie origin
2026-03-03 18:34:33 -08:00
Daniel Salazar adce8c64db feat: add private app access extension event contract (#2556)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
* feat: add private app access extension event contract

Define app.private-access.check in extension API typings with mutable allow/redirect decision fields for entitlement handlers.

* refactor: camelCase private access event contract

Rename private access extension event and payload fields to camelCase for consistency with repo conventions.
2026-02-26 15:04:23 -08:00
Daniel Salazar 2c1b21e197 feat: type extension cache update events (#2548)
* feat: type extension cache update events

Expose outer.cacheUpdate in extension API typings and consolidate extension service typing via ServicesMap.

* fix: batch broadcast events

* fix: bad import

* fix: import socket io

* fix: bad undefined call

* fix: simplify await for broadcast processing
2026-02-25 17:05:05 -08:00
Daniel Salazar 4c863cc5bc fix: make invalidations more robust (#2529) 2026-02-24 15:45:46 -08:00
KernelDeimos 0112f097db style(oidc): if instead of return with ternary expression 2026-02-19 17:18:48 -05:00
KernelDeimos 4374281070 dev: add re-authentication flow for protect actions
When users make sensitive changes to their account they are asked to
re-enter their password. This prevents a hijacked session from causing
futher damage.

Users created with the new OIDC flow do not necessarily have a password
set on their account, and they need to also be able to make these
changes. While removal of the password entry requirement for these users
would solve this problem, it would also make their accounts more
vulnerable. To solve this problem while maintaining the same security
standard for OIDC users, we need them to confirm via either 2FA or
re-authentication via OIDC. Since users aren't required to have 2FA, the
re-authentication via OIDC approach is also the minimum viable solution.

This commit adds OIDC re-authentication support for all endpoints under
UserProtectedEndpointsService, and makes updates to the
UIWindowChangeUsername dialog for manual testing.

Currently this implementation fails at the final submission to change
the username because of a separate issue with the correct authentication
token not being set; this is related to the separation of GUI tokens vs
http-only tokens.
2026-02-19 16:10:21 -05:00
Neal Shah 562671e498 add extra permission check for granted apps (#2503) 2026-02-17 01:29:54 -05:00
KernelDeimos d9167744ef patch(puterfs): limit directory depth to 35
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
This is a temporary measure while we fix a production issue.
2026-02-13 20:53:56 -05:00
Neal Shah a23f272fd2 add sandboxing by default in god mode created workers (#2481)
* add sandboxing by default in god mode workers

* closes #2481

* Update UIWindowPublishWorker.js

* Add sandbox option to worker publish UI

Add a collapsible 'Advanced' section with a 'Sandboxed' checkbox (checked by default) to UIWindowPublishWorker. On publish the code reads the checkbox state and builds createOptions ({ sandbox: true } or { sandbox: false }) and passes it to puter.workers.create as an argument so workers can be created in sandboxed or non-sandboxed mode. Small UI styling and markup for the details/summary block included.

---------

Co-authored-by: jelveh <nj@puter.com>
2026-02-12 16:06:40 -08:00
Daniel Salazar afeac494ac feat: remove file cache service (#2464) 2026-02-11 12:57:10 -08:00
Neal Shah 8e3d285671 merge users (#2441) 2026-02-07 18:44:31 -08:00
KernelDeimos 5433dde6d7 dev(extensions): [+] dev-socket
This extensions brings back the dev-socket functionality, which is
really important when testing things like broadcast, alarms, events, etc
; it saves a lot of time if you can invoke a command directly to the
backend.

This is an optional extension that will not be included in production
deployments. This is for development purposes only.
2026-02-03 19:39:07 -05:00
Daniel Salazar e938d5183a fix: limit open router expensive models for now (#2407)
* fix: limit open router expensive models for now

* fix: import extension
2026-02-03 14:43:15 -08:00
Daniel Salazar 0234e34b46 Reapply: reverted redis migration changes (#2403) 2026-02-03 11:25:28 -08:00
Daniel Salazar baceb05b48 Revert "feat: replace serializible caches with redis instead of kvjs 🚀 (#2381)"
This reverts commit 7a47047c0d.
2026-02-03 12:43:00 -05:00
Daniel Salazar 7a47047c0d feat: replace serializible caches with redis instead of kvjs 🚀 (#2381)
* wip: redis move

* fix: redis in extensions

* fix: bad isEMpty assignment

* fix: bad redis client config

* wip

* fix: redis keys cache

* fix: redis batch delete

* fix: change bulk cache times to allow for more instaces

* fix: broken tests
2026-02-03 02:18:31 -08:00
Daniel Salazar afbb76f95f fix: ai metering (#2393)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
* fix: expose getUserService in extension typings

* fix: ai metering
2026-02-01 18:14:14 -08:00
Daniel Salazar 114fbff2cd fix: expose getUserService in extension typings (#2385)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-01-31 13:19:28 -08:00
Daniel Salazar 7a9302f479 [PUT-487] fix: set max-min on app-user-count queries 🐛 (#2384)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-01-31 00:52:53 -08:00
Daniel Salazar e1b52a9828 cleanup: remove informationService in favour of existing helper methods and logic (#2374)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-01-29 17:25:28 -08:00
Lui Duarte bbe6f9dc27 Feat: Add system info (Client + Server metrics) (#2311)
* Add ststem info to user options extensions - Add UIWindowSystemInfo, add ui sections for client and server, add basic getClientinfo function

* Fix typo

* Replace accidentally deleted es.js file

* Refactor client information to be consistant with project standard

* Complete Client information in ststem information window

* Remove console logs

* Add basic api functionality for getting server system information

* Structure return data from system server information endpoint | Add copyright to UIWindowSystemInfo

* Add function to format server system api data | Add loading element to server container while waiting for data | Complete System Information

* fix: disallow non admin for backend + move to extensions

---------

Co-authored-by: Daniel Salazar <daniel.salazar@puter.com>
2026-01-27 10:29:02 -08:00
Daniel Salazar 929fc6956e [PUT-456] fix: delete apps from marketplace when deleting anywhere else (#2310)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-01-20 23:42:24 -08:00
Daniel Salazar 4e6d9c9f33 perf: improve tel + decrease logs (#2309)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
* perfmon: lower healthcheck status cache, add kv health signal

* perf: improve tel + decrease logs

* logs
2026-01-20 20:21:08 -08:00
Daniel Salazar 35797536ba feat: usage endpoints, allow appName (#2303) 2026-01-19 15:15:06 -08:00
Daniel Salazar cdb422659c feat: more extension controller decorators (#2272)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
2026-01-13 00:32:03 -08:00
Daniel Salazar b1e7bc5fca feat: method for updating users metering directly (#2252)
* chore: cleanup ts extensions controller

* feat: method for updating users metering directly
2026-01-07 12:53:00 -08:00
Daniel Salazar 2ef9aa3fb1 fix: add catch all for minimum usage amounts (#2248)
Docker Image CI / build-and-push-image (push) Has been cancelled
Maintain Release Merge PR / update-release-pr (push) Has been cancelled
release-please / release-please (push) Has been cancelled
test / test-backend (24.x) (push) Has been cancelled
test / API tests (node env, api-test) (24.x) (push) Has been cancelled
test / puterjs (node env, vitest) (24.x) (push) Has been cancelled
* ts: extension type updates

* fix: add catch all for minimum usage amounts
2026-01-06 23:09:29 -08:00
Daniel Salazar 93afbe4dd9 fix: ts extensions (#2247) 2026-01-06 18:03:38 -08:00
KernelDeimos a5bbc7e95b fix: issues creating shortcuts
There were two different things here:
- invalid requests have an ambiguous error message
- migration of PuterFSProvider to an extension caused an issue
  for shortcut creation in some flows

Closes #2200
2026-01-06 19:09:48 -05:00
jelveh 987effc5ec Revert "Remove debug logs and improve query coalescing timeout (#2241)"
This reverts commit 71bee556f6.
2026-01-03 19:34:30 -08:00