Files
puter/src/backend/drivers/integrationTestUtil.ts
T
922d203e18 fix(ai): send stable, non-sequential user identifiers to AI providers (#3856)
* fix(ai): send stable, non-sequential user identifiers to AI providers

A precedence bug in the AI providers' identifier expression made every
request send `user: ":undefined"` (the ternary bound the app-uid suffix to
the whole `actor.user.id + actor.app?.uid` sum instead of just the suffix),
or read `actor.user.id` on a missing user. The same expression also shipped
the sequential internal user id, letting AI vendors correlate a single
account across apps and sessions.

All eight OpenAI-, Azure-, xAI-, Meta- and ZAI-style providers now build
the identifier through one shared helper, `aiUserIdentifier()`:

- `puter-<user-uuid>[-<app-token>]`: the random user UUID is always
  preserved in full; `maxLength` constrains only the app-bearing form
- app attribution reads `effectiveApp`, so access-token requests name the
  issuing app instead of looking like direct user traffic
- the app token is truncated to fit the budget, and omitted entirely when
  the remaining budget is below 8 chars, where a truncation could collide
  with another app's uid
- nothing is sent for the system actor
- Meta and ZAI keep a caller-supplied `safety_identifier` / `user_id`
  override, applied before the helper result

`user` is deprecated by OpenAI; the SDK types direct callers to
`safety_identifier` (abuse detection) and `prompt_cache_key` (cache-hit
bucketing). The four OpenAI/Azure chat providers and MetaProvider now send
`prompt_cache_key` as well, defaulting it to the same per-user identifier
unless the caller supplies one; Azure's Grok branch drops both fields,
matching its rejection of unknown args. The cap comment cites only verified
limits: OpenAI's 64 for `safety_identifier` (from the SDK types) and Z.AI's
6-128 for `user_id` (from Z.AI's docs); Meta and xAI document none, so none
is claimed.

The xAI image `#edit` path now carries the identifier like generation, and
takes a named-options param so `user` cannot be transposed with the
adjacent same-typed `aspectRatio`.

Tests share a four-actor matrix (`user` / `user+app` / `access token` /
`system`) with `assertActorMatrixIdentifiers()` across the six
OpenAI-style suites; the helper has exact-string and boundary coverage
(size caps, zero-budget and sub-base cases, no dangling separator, UUID
never truncated, collision guard); the Azure Grok assertions run under a
real user actor so they cannot pass vacuously. 212 provider-suite tests
pass; typecheck and ESLint are clean.

* fix(ai): lock the vendor identifier down and keep vitest out of the test util

Meta and Z.AI no longer let `custom` override the abuse identifier; it is
Puter's attribution, not the caller's. The shared test util exposes pure
field pickers instead of importing vitest into a file the production
tsconfig compiles. The helper's length-cap comment now matches vendor docs
(Meta does cap `safety_identifier` at 64), the redundant budget branch and
the unused export are gone, and the per-user `prompt_cache_key` trade-off is
stated once in the helper instead of five times in providers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: 404oops <me@404oops.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 16:21:47 -07:00

139 lines
4.8 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify it under the
* terms of the GNU Affero General Public License as published by the Free
* Software Foundation, either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
* details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see
* [https://www.gnu.org/licenses/](https://www.gnu.org/licenses/).
*/
/**
* Shared utilities for AI provider integration tests.
*
* Each test reads its credentials from `PUTER_TEST_AI_*` env vars (loaded by
* the vitest config's `PUTER_` prefix) and skips itself when the var is missing
* — tests run only on developer machines and in CI environments that supply the
* right secrets.
*
* Filename intentionally omits `.test.` so vitest does not treat this helper as
* a test file.
*/
import type { Actor } from '../core/actor.js';
import { SYSTEM_ACTOR, makeActor } from '../core/actor.js';
import { runWithContext } from '../core/context.js';
import type { MeteringService } from '../services/metering/MeteringService.js';
/**
* Returns the env var value, or `undefined` if missing/empty. Used as the gate
* for `describe.skipIf` blocks.
*/
export const optionalEnv = (name: string): string | undefined => {
const v = process.env[name];
return v && v.length > 0 ? v : undefined;
};
/**
* Returns true when the env var is unset, signaling the test block should be
* skipped. Pair with `describe.skipIf(skipUnlessEnv(...))`.
*/
export const skipUnlessEnv = (name: string): boolean => !optionalEnv(name);
/**
* Per-test timeout for provider integration tests. The default 5s vitest
* timeout is way too short for real API calls — image generation in particular
* routinely takes 15–30s. Pass this as the third argument to `it(...)`.
*/
export const INTEGRATION_TEST_TIMEOUT_MS = 90_000;
/**
* Returns a no-op MeteringService stub. Real metering would write to DynamoDB /
* Redis, which integration tests for AI providers don't care about — we just
* need the provider's metering calls to not throw and to short-circuit credit
* checks.
*/
export const makeMeteringStub = (): MeteringService =>
({
utilRecordUsageObject: () => Promise.resolve([] as never),
incrementUsage: () => Promise.resolve({} as never),
batchIncrementUsages: () => Promise.resolve([] as never),
hasEnoughCredits: () => Promise.resolve(true),
getRemainingUsage: () => Promise.resolve(Number.MAX_SAFE_INTEGER),
getReportedCosts: () => [],
}) as unknown as MeteringService;
/**
* Run `fn` inside a request-scoped context with `SYSTEM_ACTOR` set, which is
* what providers expect (`Context.get('actor')`). The system actor bypasses
* metering / quota gates by design.
*/
export const withTestActor = <T>(
fn: () => T | Promise<T>,
actor: Actor = SYSTEM_ACTOR,
): Promise<T> =>
Promise.resolve(
runWithContext({ actor, requestId: 'integration-test' }, fn),
);
/**
* The four actor shapes provider tests exercise: a direct user session, the
* user's own app, an app-issued access token (attributed through
* `effectiveApp`), and the system actor. Shared so identifiers are tested
* identically across providers instead of copied per suite.
*/
export const makeActorMatrix = (): Actor[] => {
const user = { id: 42, uuid: 'u42', username: 'alice' };
const app = { uid: 'app-abc' };
return [
makeActor({ user }),
makeActor({ user, app }),
makeActor({
user,
accessToken: { uid: 'tok-1', issuer: makeActor({ user, app }) },
}),
SYSTEM_ACTOR,
];
};
/** Identifiers `makeActorMatrix()` should produce, in matrix order. */
export const ACTOR_MATRIX_IDENTIFIERS: (string | undefined)[] = [
'puter-u42',
'puter-u42-app-abc',
'puter-u42-app-abc',
undefined,
];
/**
* Picks `fields` off the first argument of each recorded mock call, keyed by
* field, so a suite can compare what a provider sent against
* `expectedIdentifierFields(fields)` with one `toEqual`.
*/
export const sentIdentifierFields = (
calls: unknown[][],
fields: string[],
): Record<string, unknown[]> =>
Object.fromEntries(
fields.map((field) => [
field,
calls.map((call) => (call[0] as Record<string, unknown>)[field]),
]),
);
export const expectedIdentifierFields = (
fields: string[],
): Record<string, unknown[]> =>
Object.fromEntries(
fields.map((field) => [field, ACTOR_MATRIX_IDENTIFIERS]),
);