mirror of
https://github.com/HeyPuter/puter.git
synced 2026-10-04 02:48:26 +00:00
* fix(ai): send stable, non-sequential user identifiers to AI providers A precedence bug in the AI providers' identifier expression made every request send `user: ":undefined"` (the ternary bound the app-uid suffix to the whole `actor.user.id + actor.app?.uid` sum instead of just the suffix), or read `actor.user.id` on a missing user. The same expression also shipped the sequential internal user id, letting AI vendors correlate a single account across apps and sessions. All eight OpenAI-, Azure-, xAI-, Meta- and ZAI-style providers now build the identifier through one shared helper, `aiUserIdentifier()`: - `puter-<user-uuid>[-<app-token>]`: the random user UUID is always preserved in full; `maxLength` constrains only the app-bearing form - app attribution reads `effectiveApp`, so access-token requests name the issuing app instead of looking like direct user traffic - the app token is truncated to fit the budget, and omitted entirely when the remaining budget is below 8 chars, where a truncation could collide with another app's uid - nothing is sent for the system actor - Meta and ZAI keep a caller-supplied `safety_identifier` / `user_id` override, applied before the helper result `user` is deprecated by OpenAI; the SDK types direct callers to `safety_identifier` (abuse detection) and `prompt_cache_key` (cache-hit bucketing). The four OpenAI/Azure chat providers and MetaProvider now send `prompt_cache_key` as well, defaulting it to the same per-user identifier unless the caller supplies one; Azure's Grok branch drops both fields, matching its rejection of unknown args. The cap comment cites only verified limits: OpenAI's 64 for `safety_identifier` (from the SDK types) and Z.AI's 6-128 for `user_id` (from Z.AI's docs); Meta and xAI document none, so none is claimed. The xAI image `#edit` path now carries the identifier like generation, and takes a named-options param so `user` cannot be transposed with the adjacent same-typed `aspectRatio`. Tests share a four-actor matrix (`user` / `user+app` / `access token` / `system`) with `assertActorMatrixIdentifiers()` across the six OpenAI-style suites; the helper has exact-string and boundary coverage (size caps, zero-budget and sub-base cases, no dangling separator, UUID never truncated, collision guard); the Azure Grok assertions run under a real user actor so they cannot pass vacuously. 212 provider-suite tests pass; typecheck and ESLint are clean. * fix(ai): lock the vendor identifier down and keep vitest out of the test util Meta and Z.AI no longer let `custom` override the abuse identifier; it is Puter's attribution, not the caller's. The shared test util exposes pure field pickers instead of importing vitest into a file the production tsconfig compiles. The helper's length-cap comment now matches vendor docs (Meta does cap `safety_identifier` at 64), the redundant budget branch and the unused export are gone, and the per-user `prompt_cache_key` trade-off is stated once in the helper instead of five times in providers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: 404oops <me@404oops.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
139 lines
4.8 KiB
TypeScript
139 lines
4.8 KiB
TypeScript
/*
|
||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||
*
|
||
* This file is part of Puter.
|
||
*
|
||
* Puter is free software: you can redistribute it and/or modify it under the
|
||
* terms of the GNU Affero General Public License as published by the Free
|
||
* Software Foundation, either version 3 of the License, or (at your option) any
|
||
* later version.
|
||
*
|
||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
|
||
* details.
|
||
*
|
||
* You should have received a copy of the GNU Affero General Public License
|
||
* along with this program. If not, see
|
||
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
|
||
*/
|
||
|
||
/**
|
||
* Shared utilities for AI provider integration tests.
|
||
*
|
||
* Each test reads its credentials from `PUTER_TEST_AI_*` env vars (loaded by
|
||
* the vitest config's `PUTER_` prefix) and skips itself when the var is missing
|
||
* — tests run only on developer machines and in CI environments that supply the
|
||
* right secrets.
|
||
*
|
||
* Filename intentionally omits `.test.` so vitest does not treat this helper as
|
||
* a test file.
|
||
*/
|
||
|
||
import type { Actor } from '../core/actor.js';
|
||
import { SYSTEM_ACTOR, makeActor } from '../core/actor.js';
|
||
import { runWithContext } from '../core/context.js';
|
||
import type { MeteringService } from '../services/metering/MeteringService.js';
|
||
|
||
/**
|
||
* Returns the env var value, or `undefined` if missing/empty. Used as the gate
|
||
* for `describe.skipIf` blocks.
|
||
*/
|
||
export const optionalEnv = (name: string): string | undefined => {
|
||
const v = process.env[name];
|
||
return v && v.length > 0 ? v : undefined;
|
||
};
|
||
|
||
/**
|
||
* Returns true when the env var is unset, signaling the test block should be
|
||
* skipped. Pair with `describe.skipIf(skipUnlessEnv(...))`.
|
||
*/
|
||
export const skipUnlessEnv = (name: string): boolean => !optionalEnv(name);
|
||
|
||
/**
|
||
* Per-test timeout for provider integration tests. The default 5s vitest
|
||
* timeout is way too short for real API calls — image generation in particular
|
||
* routinely takes 15–30s. Pass this as the third argument to `it(...)`.
|
||
*/
|
||
export const INTEGRATION_TEST_TIMEOUT_MS = 90_000;
|
||
|
||
/**
|
||
* Returns a no-op MeteringService stub. Real metering would write to DynamoDB /
|
||
* Redis, which integration tests for AI providers don't care about — we just
|
||
* need the provider's metering calls to not throw and to short-circuit credit
|
||
* checks.
|
||
*/
|
||
export const makeMeteringStub = (): MeteringService =>
|
||
({
|
||
utilRecordUsageObject: () => Promise.resolve([] as never),
|
||
incrementUsage: () => Promise.resolve({} as never),
|
||
batchIncrementUsages: () => Promise.resolve([] as never),
|
||
hasEnoughCredits: () => Promise.resolve(true),
|
||
getRemainingUsage: () => Promise.resolve(Number.MAX_SAFE_INTEGER),
|
||
getReportedCosts: () => [],
|
||
}) as unknown as MeteringService;
|
||
|
||
/**
|
||
* Run `fn` inside a request-scoped context with `SYSTEM_ACTOR` set, which is
|
||
* what providers expect (`Context.get('actor')`). The system actor bypasses
|
||
* metering / quota gates by design.
|
||
*/
|
||
export const withTestActor = <T>(
|
||
fn: () => T | Promise<T>,
|
||
actor: Actor = SYSTEM_ACTOR,
|
||
): Promise<T> =>
|
||
Promise.resolve(
|
||
runWithContext({ actor, requestId: 'integration-test' }, fn),
|
||
);
|
||
|
||
/**
|
||
* The four actor shapes provider tests exercise: a direct user session, the
|
||
* user's own app, an app-issued access token (attributed through
|
||
* `effectiveApp`), and the system actor. Shared so identifiers are tested
|
||
* identically across providers instead of copied per suite.
|
||
*/
|
||
export const makeActorMatrix = (): Actor[] => {
|
||
const user = { id: 42, uuid: 'u42', username: 'alice' };
|
||
const app = { uid: 'app-abc' };
|
||
return [
|
||
makeActor({ user }),
|
||
makeActor({ user, app }),
|
||
makeActor({
|
||
user,
|
||
accessToken: { uid: 'tok-1', issuer: makeActor({ user, app }) },
|
||
}),
|
||
SYSTEM_ACTOR,
|
||
];
|
||
};
|
||
|
||
/** Identifiers `makeActorMatrix()` should produce, in matrix order. */
|
||
export const ACTOR_MATRIX_IDENTIFIERS: (string | undefined)[] = [
|
||
'puter-u42',
|
||
'puter-u42-app-abc',
|
||
'puter-u42-app-abc',
|
||
undefined,
|
||
];
|
||
|
||
/**
|
||
* Picks `fields` off the first argument of each recorded mock call, keyed by
|
||
* field, so a suite can compare what a provider sent against
|
||
* `expectedIdentifierFields(fields)` with one `toEqual`.
|
||
*/
|
||
export const sentIdentifierFields = (
|
||
calls: unknown[][],
|
||
fields: string[],
|
||
): Record<string, unknown[]> =>
|
||
Object.fromEntries(
|
||
fields.map((field) => [
|
||
field,
|
||
calls.map((call) => (call[0] as Record<string, unknown>)[field]),
|
||
]),
|
||
);
|
||
|
||
export const expectedIdentifierFields = (
|
||
fields: string[],
|
||
): Record<string, unknown[]> =>
|
||
Object.fromEntries(
|
||
fields.map((field) => [field, ACTOR_MATRIX_IDENTIFIERS]),
|
||
);
|