mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-15 10:20:40 +00:00
* perf(workers): skip hot-reload lookups for files no worker is bound to
The fs.write.file / fs.remove.node / fs.move.node listeners listed every
workers.puter.* row of the file's owner on every event, for every user.
The event's FSEntry already carries the worker rows bound to it, so a file
with none returns before any I/O; a bound file reads just its own rows,
batched, from the primary.
* perf(apps): read the uid alias only after a direct lookup misses
Every AppDriver uid lookup fired an uncached system-KV GetItem for the
merge alias in parallel with the direct read, even when the direct read
hit. The alias is now read only on a miss.
* refactor(apps): one app view with the launch gates, built by AppService
AppDriver's serializer and its private-app / hosted-backing gates move to
AppService.views, built on a pure toAppView that is the one place
index_url is withheld. AppDriver, the subdomains driver's embedded apps and
the GUI shell view all use it. Visibility of protected apps
(filterReadable) moves with it.
Subdomain embeds (associated_app, app_owner) used to return index_url with
no gate; they now withhold it from a viewer who may not open a private app
or whose hosted backing is gone, and carry privateAccess, godmode and
feedback_enabled like a direct app read.
* refactor(apps): one toAppSummary; batch the recommended and taskbar lookups
SuggestedApps, RecommendedApps, /get-launch-apps and the taskbar each built
their own launch summary; they now share toAppSummary (the extra
feedback_enabled / external flags appear on the lists that lacked them).
getRecommendedApps awaited getByName once per recommended app (29) and
getTaskbarItems did the same per taskbar entry on every whoami; both now
make one batched call per key through the new AppStore.getByNames, which
keeps getByName's case-insensitive and old-name matching. The taskbar's
raw UPDATE of the user row goes through UserStore.update, and
/get-launch-apps checks hosted backings in one batch and reaches the
stores without casts.
* refactor(apps): app routes read through AppService; /delete-site calls the subdomains driver
AppController used the global driversContainers through a getter whose
boot-order comment no longer held; it uses the injected drivers and
AppService. GET /apps/:name looked each name up, read it again through the
driver and resolved private access a second time, emitting
app.privateAccess.resolveLaunch twice per private app; it now batches the
names and resolves once. POST /query/app did a lookup, a driver read and a
raw app_filetype_association SELECT per selector (up to 200, serially); it
now makes one batched lookup per key, one grant check and one filetype read.
The driver's read shape is a pure toAppReadView both use.
HostingController's /delete-site re-implemented the subdomains driver's
delete (protected check, delete, subdomain.delete event); it keeps its
owner-scoped 404 and calls the driver. Hosting and Desktop controllers drop
constructors that discarded the drivers argument, and trivial getters.
* refactor(drivers): one per-method policy validator and resolver; drop @Driver
Four policy blocks (rateLimit, concurrent, requireSubscription,
requireReputation) share the { default, methods } shape but each had its
own validator and resolver. They become validatePerMethod(value, label,
policy) and resolvePerMethod(cfg, method); error messages are unchanged.
The @Driver class decorator and the prototype keys it wrote had no users
outside its own tests; drivers declare their fields, and resolveDriverMeta
reads only those. DriverController takes the driver name from the meta it
already resolved instead of probing the prototype.
resolveDriverMethodRateLimit/Concurrent stay as deprecated aliases for the
email extension.
* refactor(drivers): shared actor guards and crud-q concurrency policy
The apps, subdomains, workers and notifications drivers each carried a
private #requireActor, two carried the same #requireVerified, and apps and
subdomains the same owner/app_owner write predicate. These become
requireContextActor, requireContextUserActor, assertActorEmailVerified and
actorOwnsRow in core/actor.ts; each driver keeps its own error for the
cases that differed (system actors still read apps; notifications still
answer 403 for an actor without a user). The identical crud-q concurrency
block is one CRUD_CONCURRENT constant.
The metering, installedApps and appTelemetry extensions re-checked an actor
their routes already require; those checks and the tests for them go.
* refactor(extensions): store methods instead of raw SQL in OSS extensions
installedApps and appTelemetry queried user_to_app_permissions directly
and metering read apps by name with its own SELECT. PermissionStore gains
listAppsGrantedByUser, listAppAuthenticatedUsers (with the email-share
check folded in) and countAppUsers; metering uses AppStore.getByName.
* refactor(server): one layer builder and one lifecycle hook runner
PuterServer built each of its five layers with its own copy of the same
loop (built-ins, then extension additions, into the layer and its export
container), and ran each lifecycle hook with five near-identical loops per
hook. Both are now one helper each: #instantiateLayer and
#runHook(order, hook). Construction order, the args each layer receives,
and hook order (start and prepare bottom-up, shutdown top-down) are
unchanged.
* fix(workers): tear down a deleted source's worker after its FK nulls root_dir_id
357 lines
13 KiB
TypeScript
357 lines
13 KiB
TypeScript
/*
|
|
* Copyright (C) 2024-present Puter Technologies Inc.
|
|
*
|
|
* This file is part of Puter.
|
|
*
|
|
* Puter is free software: you can redistribute it and/or modify
|
|
* it under the terms of the GNU Affero General Public License as published
|
|
* by the Free Software Foundation, either version 3 of the License, or
|
|
* (at your option) any later version.
|
|
*
|
|
* This program is distributed in the hope that it will be useful,
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
* GNU Affero General Public License for more details.
|
|
*
|
|
* You should have received a copy of the GNU Affero General Public License
|
|
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
*/
|
|
|
|
import type { Readable } from 'node:stream';
|
|
import {
|
|
validateReputationRequirement,
|
|
type ReputationRequirement,
|
|
} from '../core/reputation.js';
|
|
import {
|
|
validateSubscriptionRequirement,
|
|
type SubscriptionRequirement,
|
|
} from '../services/metering/enforcement.js';
|
|
import type { WithLifecycle } from '../types';
|
|
|
|
// -- Stream result convention ----------------------------------------
|
|
//
|
|
// Driver methods that return a stream instead of JSON wrap the readable
|
|
// in this shape. The `/drivers/call` handler detects it and pipes to the
|
|
// HTTP response instead of calling `res.json()`.
|
|
|
|
export interface DriverStreamResult {
|
|
/** Discriminant — must be `'stream'`. */
|
|
dataType: 'stream';
|
|
/** MIME type sent as Content-Type (e.g. `'application/x-ndjson'`). */
|
|
content_type: string;
|
|
/** When true, sets `Transfer-Encoding: chunked`. */
|
|
chunked?: boolean;
|
|
/** The readable stream to pipe to the response. */
|
|
stream: Readable;
|
|
}
|
|
|
|
export function isDriverStreamResult(v: unknown): v is DriverStreamResult {
|
|
return (
|
|
!!v &&
|
|
typeof v === 'object' &&
|
|
(v as Record<string, unknown>).dataType === 'stream' &&
|
|
'stream' in v
|
|
);
|
|
}
|
|
|
|
// -- Per-method driver policies ----------------------------------------
|
|
//
|
|
// Every policy a driver declares has the same shape: a `default` entry and
|
|
// per-method overrides. `DriverController` resolves the entry for the called
|
|
// method, so methods can differ in limits, plan and reputation floor.
|
|
|
|
/** A driver policy: `default` for any method not listed in `methods`. */
|
|
export interface PerMethodConfig<T> {
|
|
default?: T;
|
|
/** Keys are driver method names. */
|
|
methods?: Record<string, T>;
|
|
}
|
|
|
|
export const RATE_LIMIT_BACKEND_NAMES = ['memory', 'redis', 'kv'] as const;
|
|
export type RateLimitBackend = (typeof RATE_LIMIT_BACKEND_NAMES)[number];
|
|
|
|
export interface DriverRateLimitSpec {
|
|
/** Maximum hits per window. */
|
|
limit: number;
|
|
/** Window length, in milliseconds. */
|
|
window: number;
|
|
/** Per-`SubscriptionPolicy.id` overrides for `limit`. */
|
|
bySubscription?: Record<string, number>;
|
|
/** Defaults to `config.rate_limit.backend`. */
|
|
backend?: RateLimitBackend;
|
|
}
|
|
|
|
export interface DriverConcurrentSpec {
|
|
/** Maximum simultaneous in-flight requests. */
|
|
limit: number;
|
|
/** Per-`SubscriptionPolicy.id` overrides for `limit`. */
|
|
bySubscription?: Record<string, number>;
|
|
/** `memory` is per-process; use `redis` on multi-node deployments. */
|
|
backend?: RateLimitBackend;
|
|
}
|
|
|
|
export type DriverRateLimitConfig = PerMethodConfig<DriverRateLimitSpec>;
|
|
export type DriverConcurrentConfig = PerMethodConfig<DriverConcurrentSpec>;
|
|
/**
|
|
* Per-method counterpart of `RouteOptions.requireSubscription`: `/drivers/call`
|
|
* is one shared route, so a route option would apply to every driver at once.
|
|
* `undefined` for a method means it is open to every plan.
|
|
*/
|
|
export type DriverRequireSubscriptionConfig =
|
|
PerMethodConfig<SubscriptionRequirement>;
|
|
/**
|
|
* Per-method counterpart of `RouteOptions.requireReputation`. A tier is only a
|
|
* name; the score it takes is deployment config.
|
|
*/
|
|
export type DriverRequireReputationConfig =
|
|
PerMethodConfig<ReputationRequirement>;
|
|
|
|
function validateLimitSpec(
|
|
value: unknown,
|
|
label: string,
|
|
{ windowed }: { windowed: boolean },
|
|
): void {
|
|
if (typeof value !== 'object' || value === null || Array.isArray(value)) {
|
|
throw new Error(`${label}: expected an object`);
|
|
}
|
|
const spec = value as Record<string, unknown>;
|
|
if (
|
|
typeof spec.limit !== 'number' ||
|
|
!Number.isFinite(spec.limit) ||
|
|
spec.limit <= 0
|
|
) {
|
|
throw new Error(`${label}.limit: expected a positive number`);
|
|
}
|
|
if (
|
|
windowed &&
|
|
(typeof spec.window !== 'number' ||
|
|
!Number.isFinite(spec.window) ||
|
|
spec.window <= 0)
|
|
) {
|
|
throw new Error(`${label}.window: expected a positive number (ms)`);
|
|
}
|
|
if (spec.backend !== undefined) {
|
|
if (
|
|
typeof spec.backend !== 'string' ||
|
|
!RATE_LIMIT_BACKEND_NAMES.includes(spec.backend as RateLimitBackend)
|
|
) {
|
|
throw new Error(
|
|
`${label}.backend: expected one of ${RATE_LIMIT_BACKEND_NAMES.join(', ')}`,
|
|
);
|
|
}
|
|
}
|
|
if (spec.bySubscription !== undefined) {
|
|
validateBySubscription(spec.bySubscription, label);
|
|
}
|
|
}
|
|
|
|
function validateBySubscription(value: unknown, label: string): void {
|
|
if (typeof value !== 'object' || value === null || Array.isArray(value)) {
|
|
throw new Error(`${label}.bySubscription: expected an object`);
|
|
}
|
|
for (const [id, n] of Object.entries(value as Record<string, unknown>)) {
|
|
if (typeof n !== 'number' || !Number.isFinite(n) || n <= 0) {
|
|
throw new Error(
|
|
`${label}.bySubscription.${id}: expected a positive number`,
|
|
);
|
|
}
|
|
}
|
|
}
|
|
|
|
/** The entry validator for each policy a driver may declare. */
|
|
const POLICY_ENTRY_VALIDATORS = {
|
|
rateLimit: (value: unknown, label: string) =>
|
|
validateLimitSpec(value, label, { windowed: true }),
|
|
concurrent: (value: unknown, label: string) =>
|
|
validateLimitSpec(value, label, { windowed: false }),
|
|
requireSubscription: validateSubscriptionRequirement,
|
|
requireReputation: validateReputationRequirement,
|
|
};
|
|
|
|
export type DriverPolicyName = keyof typeof POLICY_ENTRY_VALIDATORS;
|
|
|
|
interface DriverPolicyConfigs {
|
|
rateLimit: DriverRateLimitConfig;
|
|
concurrent: DriverConcurrentConfig;
|
|
requireSubscription: DriverRequireSubscriptionConfig;
|
|
requireReputation: DriverRequireReputationConfig;
|
|
}
|
|
|
|
/**
|
|
* Validate a driver's `policy` block. Throws with a labelled path at
|
|
* registration, so a malformed entry surfaces at boot rather than on the first
|
|
* call.
|
|
*/
|
|
export function validatePerMethod<K extends DriverPolicyName>(
|
|
value: unknown,
|
|
label: string,
|
|
policy: K,
|
|
): DriverPolicyConfigs[K] {
|
|
if (value == null) return {};
|
|
if (typeof value !== 'object' || Array.isArray(value)) {
|
|
throw new Error(`${label}: ${policy} must be an object`);
|
|
}
|
|
const validateEntry = POLICY_ENTRY_VALIDATORS[policy];
|
|
const cfg = value as Record<string, unknown>;
|
|
if (cfg.default !== undefined) {
|
|
validateEntry(cfg.default, `${label}.${policy}.default`);
|
|
}
|
|
if (cfg.methods !== undefined) {
|
|
if (
|
|
typeof cfg.methods !== 'object' ||
|
|
cfg.methods === null ||
|
|
Array.isArray(cfg.methods)
|
|
) {
|
|
throw new Error(`${label}.${policy}.methods must be an object`);
|
|
}
|
|
for (const [name, entry] of Object.entries(cfg.methods)) {
|
|
validateEntry(entry, `${label}.${policy}.methods.${name}`);
|
|
}
|
|
}
|
|
return cfg as DriverPolicyConfigs[K];
|
|
}
|
|
|
|
/**
|
|
* The entry that applies to `method`: its own, else `default`, else `undefined`
|
|
* (the caller decides what an undeclared policy means).
|
|
*/
|
|
export function resolvePerMethod<T>(
|
|
cfg: PerMethodConfig<T> | undefined,
|
|
method: string,
|
|
): T | undefined {
|
|
if (!cfg) return undefined;
|
|
return cfg.methods?.[method] ?? cfg.default;
|
|
}
|
|
|
|
/** @deprecated Use `resolvePerMethod`; kept for extensions still on it. */
|
|
export const resolveDriverMethodRateLimit =
|
|
resolvePerMethod<DriverRateLimitSpec>;
|
|
/** @deprecated Use `resolvePerMethod`; kept for extensions still on it. */
|
|
export const resolveDriverMethodConcurrent =
|
|
resolvePerMethod<DriverConcurrentSpec>;
|
|
|
|
/**
|
|
* Resolved metadata for a registered driver, read from its instance properties.
|
|
* The gates live here rather than on a route because every driver shares the
|
|
* `/drivers/call` dispatch route.
|
|
*/
|
|
export interface DriverMeta {
|
|
/** E.g. 'puter-chat-completion'. */
|
|
interfaceName: string;
|
|
/** Unique within the interface. */
|
|
driverName: string;
|
|
isDefault: boolean;
|
|
/**
|
|
* Other names resolving to this driver, so legacy calls naming a provider
|
|
* (`aws-polly`) still find the unified driver. The requested alias reaches
|
|
* the method via `Context.get('driverName')`.
|
|
*/
|
|
aliases: string[];
|
|
/** Falls back to the global driver default when absent. */
|
|
rateLimit?: DriverRateLimitConfig;
|
|
/** No concurrency cap when absent. */
|
|
concurrent?: DriverConcurrentConfig;
|
|
/** Reject bare account-session tokens. */
|
|
noUserSession?: boolean;
|
|
/**
|
|
* Per-method plan requirement; a method covered by neither `default` nor
|
|
* `methods` is open.
|
|
*/
|
|
requireSubscription?: DriverRequireSubscriptionConfig;
|
|
/** Per-method reputation tier; same coverage rule as `requireSubscription`. */
|
|
requireReputation?: DriverRequireReputationConfig;
|
|
}
|
|
|
|
/**
|
|
* Extract a driver's metadata, validating its policy blocks so a malformed one
|
|
* fails loud at registration. Returns `null` if the driver doesn't declare an
|
|
* interface and name.
|
|
*/
|
|
export function resolveDriverMeta(
|
|
driver: WithLifecycle & Record<string, unknown>,
|
|
): DriverMeta | null {
|
|
const interfaceName = driver.driverInterface as string | undefined;
|
|
const driverName = driver.driverName as string | undefined;
|
|
if (!interfaceName || !driverName) return null;
|
|
|
|
const label = `driver '${driverName}'`;
|
|
const policy = <K extends DriverPolicyName>(name: K) =>
|
|
driver[name] === undefined
|
|
? undefined
|
|
: validatePerMethod(driver[name], label, name);
|
|
|
|
return {
|
|
interfaceName,
|
|
driverName,
|
|
isDefault: (driver.isDefault as boolean | undefined) ?? false,
|
|
aliases: (driver.driverAliases as string[] | undefined) ?? [],
|
|
rateLimit: policy('rateLimit'),
|
|
concurrent: policy('concurrent'),
|
|
noUserSession: (driver.noUserSession as boolean | undefined) ?? false,
|
|
requireSubscription: policy('requireSubscription'),
|
|
requireReputation: policy('requireReputation'),
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Framework/lifecycle method names that must never be reachable via
|
|
* `/drivers/call`. These live on `PuterDriver` (see `drivers/types.ts`) and are
|
|
* the machinery the dispatch surface must exclude. For class-based drivers a
|
|
* concrete `override` of one still carries the same name and is caught here;
|
|
* for plain-object drivers (registered by extensions — see `server.ts`, `typeof
|
|
* DriverClass === 'object'`) there is no base prototype to distinguish them, so
|
|
* this denylist is the _only_ thing keeping a hook off the RPC surface. Any
|
|
* lifecycle hook added to `PuterDriver` must be added here too — the per-driver
|
|
* guard test (`callableMethods.test.ts`) fails loudly if a base method starts
|
|
* leaking into every driver's surface.
|
|
*/
|
|
export const RESERVED_DRIVER_METHODS: ReadonlySet<string> = new Set([
|
|
'onServerStart',
|
|
'onServerPrepareShutdown',
|
|
'onServerShutdown',
|
|
'getReportedCosts',
|
|
]);
|
|
|
|
/**
|
|
* Compute the set of method names a driver exposes over `/drivers/call`.
|
|
*
|
|
* The RPC surface is defined structurally rather than by a hand-maintained
|
|
* per-method allow-list. Walking from the instance up to (but not including)
|
|
* `Object.prototype`, a name is callable iff it resolves to a function and is
|
|
* neither `constructor` nor a `RESERVED_DRIVER_METHODS` entry. This covers both
|
|
* driver shapes the server accepts (`server.ts`): class instances (RPC methods
|
|
* on the concrete prototype, config on the instance) and plain objects
|
|
* (everything own, used verbatim by extensions). It excludes all
|
|
* `Object.prototype` members (`toString`, `valueOf`, `__proto__`, …), the
|
|
* `constructor`, and the lifecycle hooks.
|
|
*
|
|
* `#`-private helpers need no handling: they are not real property keys, so
|
|
* `getOwnPropertyNames` never lists them and `driver['#x']` is `undefined`.
|
|
* Only _plain_ public methods can appear here.
|
|
*
|
|
* Getters are excluded (we read the descriptor's `.value`, never access the
|
|
* property), so evaluating this set never runs driver code. Intended to be
|
|
* called once per driver at registration and cached — not on the hot path.
|
|
*/
|
|
export function resolveCallableMethods(driver: object): Set<string> {
|
|
const callable = new Set<string>();
|
|
const seen = new Set<string>();
|
|
for (
|
|
let o: object | null = driver;
|
|
o && o !== Object.prototype;
|
|
o = Object.getPrototypeOf(o) as object | null
|
|
) {
|
|
for (const name of Object.getOwnPropertyNames(o)) {
|
|
// First (lowest) definition wins — a subclass override shadows
|
|
// the base, and we decide against the resolved descriptor.
|
|
if (seen.has(name)) continue;
|
|
seen.add(name);
|
|
if (name === 'constructor') continue;
|
|
if (RESERVED_DRIVER_METHODS.has(name)) continue;
|
|
const desc = Object.getOwnPropertyDescriptor(o, name);
|
|
if (desc && typeof desc.value === 'function') callable.add(name);
|
|
}
|
|
}
|
|
return callable;
|
|
}
|