lee/puter
1
0
Fork 0
mirror of https://github.com/HeyPuter/puter.git synced 2026-05-27 20:01:35 +00:00
Code Issues Actions 9 Packages Projects Releases Wiki Activity

doc: code architecture doc + contributing guide for backend (#2953)

Browse Source 
* architecture doc

* agents and contributor md guides

* fix: userStore
This commit is contained in:
Daniel Salazar
2026-05-08 13:51:41 -07:00
committed by GitHub
parent 46a56e474b
commit 960bbb6965
6 changed files with 304 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.MD
+93
View File
@@ -0,0 +1,93 @@
# AGENTS.md
Guidance for AI coding agents working in this repository. Read this before making changes.
## Documentation Index
Use these as the source of truth before exploring further:
- [README.md](README.md) — project overview and quickstart.
- [doc/architecture.md](doc/architecture.md) — backend layered stack (controllers → drivers → services → stores → clients), `PuterServer` wiring, `Context` (ALS), and extensions.
- [doc/self-hosting.md](doc/self-hosting.md) — running Puter outside hosted infra.
- [CONTRIBUTING.md](CONTRIBUTING.md) — testing, security, AI-assisted code, PR conventions, Boy Scout Rule.
- [SECURITY.md](SECURITY.md) — how to report vulnerabilities (do not file them publicly).
- [BUG-BOUNTY.md](BUG-BOUNTY.md) — bounty program scope.
- [TRADEMARK.md](TRADEMARK.md) — trademark usage.
---
## Backend
The backend is organized as a layered stack inspired by Controller–Service–Repository with dependency injection. Every layer only depends on the layers beneath it, and `PuterServer` ([src/backend/server.ts](src/backend/server.ts)) wires the whole thing together.
### Layers (top → bottom)
| Layer | Lives in | Responsibility |
| --------------- | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Controllers** | [src/backend/controllers/](src/backend/controllers/) | Route handlers. Parse + validate input, apply per-route gates (auth, subdomain, rate limit, body parsers via `RouteOptions`), call services, format responses. |
| **Drivers** | [src/backend/drivers/](src/backend/drivers/) | Optional. RPC-style handlers exposed over `/drivers/*`. Thin shells that validate RPC inputs and call into services/stores. |
| **Services** | [src/backend/services/](src/backend/services/) | Business logic. Assume the caller is already authenticated/authorized — services do not run auth gates themselves. |
| **Stores** | [src/backend/stores/](src/backend/stores/) | Persistence. Wraps clients with the domain shape services consume (rows, entities, KV namespaces). |
| **Clients** | [src/backend/clients/](src/backend/clients/) | Adapters for external/internal services (sql, redis, s3, dynamo, email, event bus). Knows protocols, not domain concepts. |
| **Config** | `config.*.json` → `IConfig` | Flat, typed config object every layer receives at construction. |
Each layer receives the layers beneath it through its constructor. Dependencies are explicit and traceable from `PuterServer`.
### Cross-layer rules
- **Don't reach across layers.** Controllers do not poke clients directly; services do not register routes. If you want to, the abstraction is wrong — fix the abstraction.
- **Don't call sideways within a layer for code reuse.** If two services need the same logic, lift it into a util/helper. Services should not depend on other services for shared code.
- **Prefer explicit arguments over `Context` (ALS).** Reach for [Context](src/backend/core/context.ts) only when the value is genuinely request-scoped and would otherwise thread through many layers. Today it's used sparingly — mostly for `actor` and `req`.
### Extensions
Extensions live in [extensions/](extensions/) and parallel the layered stack. They are for **non-crucial parts of the system** — things Puter still works without if removed. If a feature is load-bearing for clients, it belongs in core, not in an extension (see [whoami](extensions/whoami.ts) as the cautionary example).
The `extension` global ([src/backend/extensions.ts](src/backend/extensions.ts)) exposes:
- `extension.registerClient/Store/Service/Driver/Controller(name, Class)` for first-class additions.
- `extension.on(event, handler)` and `extension.get/post/put/delete/patch/use(path, opts?, handler)` for the lightweight common case. `opts` is the same `RouteOptions` controllers use.
- `extension.import('client' | 'store' | 'service' | 'controller' | 'driver')` returns a lazy proxy to instantiated objects. `extension.config` exposes live config.
### Language & file conventions
- **Modules:** We transpile and build as needed — write ES modules, not CommonJS.
- **TypeScript preferred for new files.** Existing JS is fine; convert opportunistically when you're already touching a file.
- **Reuse types before inventing them.** Search for an existing type first; extend it if close. Only define a new type when nothing fits.
- **Make new types findable.** Co-locate them with the layer/module that owns them, export from the obvious entry point, and use a descriptive `PascalCase` name. Don't hide types in random files where future readers won't grep them.
- **Naming:** `camelCase` for variables/functions, `PascalCase` for classes and for files containing a class (`AuthService.ts`, `KVStoreDriver.ts`).
### Comments
Keep comments light. Prefer self-documenting code — clear names, small functions, obvious flow. Add a comment **only** when:
- The _why_ is non-obvious (a hidden constraint, a workaround, a subtle invariant).
- A non-trivial usage detail would otherwise trip the next reader.
Don't restate what the code already says. Don't write comments that reference the current task or PR — those rot.
### Tests
When adding new behavior (function, endpoint, driver method, branch of logic), add a test for it.
- **Mock data, not methods.** Stub the inputs (fixtures, fake rows, sample payloads) rather than mocking the function under test or the layer beneath it. Over-mocking produces tests that pass while production breaks. If you must mock, mock at a real boundary (a client/external service), not within the same layer you're testing.
- **Hit a real database / real client where reasonable.** Integration shapes catch the things unit tests with mocks miss.
- **Regression tests for bug fixes.** A test that fails before your fix and passes after is the cheapest insurance against the bug coming back.
- If something is genuinely hard to test (UI animation, third-party glue), skip it but say so in the PR.
### Security & privacy
Before opening a PR, scan the diff for:
- Logs, error messages, or responses leaking internal paths, secrets, tokens, env vars, or other users' data.
- Debug routes, test credentials, commented-out auth checks.
- Endpoints returning more than the caller actually needs.
When in doubt, return less. Auth-, permission-, or data-export-related changes deserve an explicit callout in the PR description.
### Working rules of thumb
- **Run it, don't just compile it.** "It type-checks" is not "it works." Exercise the code path end-to-end at least once.
- **Read the neighbors before writing.** Match the shape of similar things already in the tree. If you genuinely think the existing pattern is wrong, raise it — don't quietly diverge.
- **Boy Scout Rule, proportional to the change.** Fix the obvious typo, dead import, or missing typehint in files you're already touching. Don't ride a refactor along with a bug fix — that just makes review harder.
- **Understand what you commit.** AI assistance is fine; shipping code you couldn't defend in review is not.
CONTRIBUTING.md
+94
View File
@@ -0,0 +1,94 @@
# Contributing to Puter Backend
Thanks for taking the time to contribute. Puter moves fast and we want to keep it that way without sacrificing quality, so this guide leans on shared judgment more than strict gates. None of these rules are harshly enforced — but every PR that follows them makes the next one easier.
If you're not sure about something, open the PR anyway and ask. It's almost always better than not contributing.
New to the backend? Start with [doc/architecture.md](doc/architecture.md) — it covers the layered stack (controllers → services → stores → clients), how extensions plug in, and the core conventions.
---
## 1. Test it. Run it.
Before you open a PR, the change should actually work — not just compile.
- Run the affected code path end-to-end at least once. If it's a UI change, click through it. If it's an API change, hit the endpoint.
- If something is hard to test, that's worth a comment in the PR — it usually points at a design issue we can fix together.
"It builds" and "it passes type checks" are not the same as "it works."
### 1.1 Add tests for new things, where applicable
If you're introducing new behavior — a new function, endpoint, component, or branch of logic — add a test for it where it's reasonable to do so. Untested new code is the easiest place for regressions to hide later.
- New behavior with a clear input/output? Write a test.
- New bug fix? A regression test stops it coming back.
- Genuinely hard to cover (UI animation, third-party integration, infra glue)? Skip it, but say so in the PR so a reviewer can sanity-check the call.
We're not chasing a coverage number. We're trying to make sure the things we care about don't break silently.
## 2. Follow the patterns that are already there
Puter has established structures for the backend, the frontend, drivers, extensions, and more. When you add something new, look at how similar things are done elsewhere and match that shape. Consistency is worth more than personal preference here, because it lowers the cost of reading code for everyone else.
[**doc/architecture.md**](doc/architecture.md) is the source of truth for the backend: layer responsibilities (controllers, drivers, services, stores, clients), how `PuterServer` wires them, when to use `Context`, what belongs in an extension vs. core, and the naming/dedup/cross-layer rules. When you're adding something new, that doc tells you which layer it belongs in and what shape it should take. When in doubt, read the neighbors before you write.
If you genuinely think the existing pattern is wrong, that's a great conversation to have — raise it in the PR or an issue, don't quietly diverge.
## 3. Don't expose system or user information
We take security and privacy seriously, and most leaks happen by accident — a stray `console.log`, a debug endpoint left in, an error message that includes a file path or a user ID.
Before opening a PR, scan your diff for:
- Logs, error messages, or responses that include internal paths, secrets, tokens, env vars, or user data that the caller shouldn't see.
- Debug routes, test credentials, or commented-out auth checks.
- New endpoints that return more than the caller actually needs.
When in doubt, return less. If your change touches anything auth-, permission-, or data-export-related, call that out in the PR description so reviewers know to look closely.
For security issues you don't want to discuss in public, see [SECURITY.md](SECURITY.md).
## 4. AI-assisted code is welcome — understood code is required
Use whatever tools help you ship. Copilot, Claude, Cursor — all fine. We use them too.
The line is simple: **don't commit code you couldn't have written, debugged, or defended yourself.**
That means:
- You've read the diff, not just accepted it.
- You understand what each function does and why it's there.
- You ran it (see rule 1) and you can explain in review what you'd check if it broke.
If a reviewer asks "why does this work?" and the honest answer is "the model said so," that's the bar we're trying to stay above. AI is a great accelerator for things you understand, and a quiet way to ship bugs for things you don't.
## Opening a PR
- Keep PRs focused on one thing where you can. Two small PRs review faster than one mixed one.
- Write a description that says **what** changed and **why**. The diff already shows the how.
- If your change is user-visible, mention how you tested it.
- Drafts are welcome — open early if you want feedback before the change is finished.
## 5. Boy Scout Rule — leave it 1% better
Small improvements compound. Big rewrites rarely happen. So when you're already in a file:
- Fix the obvious typo, the missing typehint, the dead import.
- Add the test that should have existed.
- Tidy the bit you had to read three times to understand.
You don't need to clean up the whole module — just leave the area you touched a little better than you found it.
```
1% better every day 1.01^365 = 37.38
1% worse every day 0.99^365 = 0.03
```
The opposite is also true: a small mess left behind, every day, eventually buries us. Don't be the 0.99.
Keep cleanup proportional to the change. A bug fix doesn't need a refactor riding along — that just makes review harder. Use judgment.
---
That's it. Welcome aboard, and thanks for making Puter better.
doc/architecture.md
+104
View File
@@ -0,0 +1,104 @@
# Backend Architecture
Loosely inspired by the Controller–Service–Repository pattern with dependency injection. The backend is organized as a stack of layers where each layer only depends on the layers beneath it, and `PuterServer` ([src/backend/server.ts](../src/backend/server.ts)) instantiates each layer in order and hands the instances down to the next.
## Layers
```mermaid
block-beta
columns 1
REQ["HTTP request"]
CTRL["Controllers — route handlers, gates, I/O shaping"]
DRV["Drivers (optional) — RPC handlers on /drivers/*"]
SVC["Services — business logic, no auth"]
STR["Stores — persistence / domain shapes"]
CLI["Clients — sql, redis, s3, dynamo, email, …"]
CFG["Config — IConfig"]
REQ --> CTRL
CTRL --> DRV
DRV --> SVC
SVC --> STR
STR --> CLI
CLI --> CFG
style REQ fill:#0ea5e9,stroke:#0369a1,color:#fff
style CTRL fill:#1d4ed8,stroke:#1e3a8a,color:#fff
style DRV fill:#2563eb,stroke:#1e40af,color:#fff
style SVC fill:#4f46e5,stroke:#3730a3,color:#fff
style STR fill:#7c3aed,stroke:#5b21b6,color:#fff
style CLI fill:#9333ea,stroke:#6b21a8,color:#fff
style CFG fill:#334155,stroke:#1e293b,color:#fff
```
Each layer only depends on the layers beneath it, and every dependency is injected through the constructor by `PuterServer`. Extensions sit alongside this stack and can register into any layer — see [Extensions](#extensions) below.
| Layer | Lives in | Responsibility |
| --- | --- | --- |
| **Controllers** | [src/backend/controllers/](../src/backend/controllers/) | Route handlers. Parse + validate input, apply per-route gates (auth, subdomain, rate limit, body parsers — see `RouteOptions`), call into services, format responses. |
| **Drivers** | [src/backend/drivers/](../src/backend/drivers/) | Optional. RPC-style handlers exposed over the `/drivers/*` surface (`puter-kvstore`, `puter-chat-completion`, …). A driver is a thin shell that validates RPC inputs and calls into services/stores; controllers can hold a typed reference to drivers when they need the same logic over HTTP. |
| **Services** | [src/backend/services/](../src/backend/services/) | Business logic. Assume the caller is already authenticated/authorized — services do not run auth gates themselves. |
| **Stores** | [src/backend/stores/](../src/backend/stores/) | Persistence and storage logic. Wraps clients with the domain shape services consume (rows, entities, KV namespaces). |
| **Clients** | [src/backend/clients/](../src/backend/clients/) | Adapters for external/internal services (sql, redis, s3, dynamodb, email, event bus, …). Knows protocols, not domain concepts. |
| **Config** | `config.*.json` → `IConfig` | The flat, typed config object every layer receives at construction. |
Each layer receives the layers beneath it through its constructor, so dependencies are explicit and traceable from `PuterServer`. A controller does not reach into a client directly; if it needs one, the right move is usually a service.
## Entry point: `PuterServer`
`PuterServer` is the bootstrap. It:
1. Loads any configured extension directories (`config.extensions`) so extensions can register before instantiation begins.
2. Instantiates each layer in order — clients → stores → services → drivers → controllers — merging in anything extensions have registered for that layer.
3. Wires global middleware, mounts controller routes through `PuterRouter` (which translates `RouteOptions` into the gate/parser middleware chain), and mounts extension routes through the same materializer.
4. Fires `onServerStart` hooks across every layer once HTTP is listening, and `onServerPrepareShutdown` / `onServerShutdown` on the way down.
## Context (ALS)
We use [`Context`](../src/backend/core/context.ts) — backed by `AsyncLocalStorage` — to carry per-request state without threading it through every function signature. It is used **sparingly**, mostly for `actor` and `req`. The request-context middleware opens a scope per request after the auth probe runs; anything inside a request handler can call `Context.get('actor')` / `Context.get('req')` instead of plumbing it as an argument.
Prefer explicit arguments. Reach for `Context` only when the value is truly request-scoped and would otherwise need to thread through many layers.
## Extensions
Extensions live alongside core ([packages/puter/extensions/](../extensions/)) and parallel the layered stack. They are meant for **non-crucial parts of the system** — things Puter still works without if removed.
- **Good extensions**: [thumbnails](../extensions/thumbnails.ts), [serverInfo](../extensions/serverInfo.ts), [devWatcher](../extensions/devWatcher.ts) — opt-in features cleanly bolted on.
- **Should probably be core**: [metering](../extensions/metering.ts), [appTelemetry](../extensions/appTelemetry.ts) — clients now expect these to be present, so the "extension" framing is misleading.
- **Shouldn't have been an extension**: [whoami](../extensions/whoami.ts) — it's load-bearing for every authenticated client. Keep this one in mind as a cautionary example when deciding whether something belongs in an extension.
### Extension API
The `extension` global ([src/backend/extensions.ts](../src/backend/extensions.ts)) exposes:
- **Layer registration** for first-class additions:
- `extension.registerClient(name, ClientClass)`
- `extension.registerStore(name, StoreClass)`
- `extension.registerService(name, ServiceClass)`
- `extension.registerDriver(name, DriverClass)`
- `extension.registerController(name, ControllerClass)`
- **Lightweight wrappers** for the common case where a full class isn't worth it:
- `extension.on(event, handler)` — subscribe to event-bus events.
- `extension.get(path, opts?, handler)` / `.post` / `.put` / `.delete` / `.patch` / `.head` / `.options` / `.all` / `.use` — register routes. The `opts` shape is the same `RouteOptions` controllers use, so `subdomain`, `requireAuth`, `adminOnly`, body parsers, etc. all work identically.
- **Cross-layer access**: `extension.import('client' | 'store' | 'service' | 'controller' | 'driver')` returns a lazy proxy to instantiated objects, and `extension.config` exposes the live config.
```ts
import { extension } from '@heyputer/backend/src/extensions';
const services = extension.import('service');
extension.get('/healthcheck/deep', { subdomain: 'api', adminOnly: true }, async (_req, res) => {
res.json({ ok: await services.health.runDeepCheck() });
});
extension.on('user.signup', (_key, data) => {
console.log('new user', data.user.username);
});
```
## Conventions
- **TypeScript preferred** in new code where feasible. Existing JS is fine; convert opportunistically when you're already touching a file.
- **`camelCase`** for variable/function names; **`PascalCase`** for classes and for files that contain a class (`AuthService.ts`, `KVStoreDriver.ts`).
- **Deduplicate**. If two services need the same logic, lift it into a util/helper rather than calling sideways across the same layer — services should not depend on other services for code reuse.
- **Don't reach across layers.** Controllers do not poke clients directly; services do not register routes. If you find yourself wanting to, that's usually a signal the abstraction is wrong.
package-lock.json
Generated
+11
View File
@@ -7227,6 +7227,16 @@
"integrity": "sha512-u5vzlcR14ge/4pMTTMDQr3MF0wEe38B2F9o84uC4F43vN5DGTy63npRrB6jQhyt+C0lGv4ZfiRcRkqJoZuPnmg==",
"license": "MIT"
},
"node_modules/@types/bcrypt": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/@types/bcrypt/-/bcrypt-6.0.0.tgz",
"integrity": "sha512-/oJGukuH3D2+D+3H4JWLaAsJ/ji86dhRidzZ/Od7H/i8g+aCmvkeCc6Ni/f9uxGLSQVCRZkX2/lqEFG2BvWtlQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/better-sqlite3": {
"version": "7.6.13",
"resolved": "https://registry.npmjs.org/@types/better-sqlite3/-/better-sqlite3-7.6.13.tgz",
@@ -18383,6 +18393,7 @@
"validator": "^13.15.35"
},
"devDependencies": {
"@types/bcrypt": "^6.0.0",
"@types/busboy": "^1.5.4",
"@types/node": "^24.0.0",
"chai": "^4.3.7",
src/backend/package.json
+1
View File
@@ -73,6 +73,7 @@
"validator": "^13.15.35"
},
"devDependencies": {
"@types/bcrypt": "^6.0.0",
"@types/busboy": "^1.5.4",
"@types/node": "^24.0.0",
"chai": "^4.3.7",
src/backend/stores/user/UserStore.ts
+1
View File
@@ -43,6 +43,7 @@ export interface UserRow {
requires_email_confirmation?: boolean;
/** Metadata JSON blob; decoded on read when the DB returns it as a string. */
metadata?: Record<string, unknown>;
password?: string;
[k: string]: unknown;
}