mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-08 06:54:18 +00:00
Daniel Salazar
f3134b2530
* refactor(ai): route TTS, STT and voice-changer REST calls through upstreamFetch upstreamFetch bounds a provider call with AbortSignal.timeout and throws an UpstreamResponseError carrying the upstream status and code, which the /drivers/call boundary maps like any SDK error. Replaces five hand-written status translators (ElevenLabs TTS, xAI TTS, Speechify, xAI STT, voice changer); caller-facing messages are now sanitized by the shared mapper. * perf(ai): bound input image fetches and share one data-URI parser fetchImageBytes now streams under a byte cap (30 MB by default) with a 30-second timeout, via a readBoundedBody helper shared with loadFileInput's web branch and Replicate's input measuring. assertInputImagesShape caps input_images at 16. Chat image inlining uses fetchImageBytes instead of buffering with no timeout. fileInput, imageInput and mediaParts parse data URIs through one parseDataUri. * refactor(ai-tts): one per-character metering path for TTS providers meterPerCharacter holds text length x price, runs the synthesis and meters the same amount once it succeeds; characterCostReport builds the per-character cost lines. Both sit on a new withAiCreditHold (reserve, run, release) that Gemini TTS uses directly. SAMPLE_AUDIO_URL is defined once. The TTSProvider base class is gone: its methods were stubs every provider overrode, and only Polly read its untyped providerConfig. * refactor(ai): one ProviderRegistry and key reader for the AI media drivers ProviderRegistry holds a driver's provider catalog (ids, default, aliases) and its configured instances: alias resolution, the not-found 400, default fallback and parallel 'all' listing. TTS, speech-to-text, OCR and the voice changer route through it. readProviderKey reads apiKey, secret_key, api_key and key in that order for every AI media driver, so providers.xai.secret_key now enables xAI TTS as it already did xAI image and speech-to-text. ElevenLabs caches its voice list for 10 minutes, as Polly does. * perf(ai): stream TTS and voice-changer audio instead of buffering it OpenAI, xAI and ElevenLabs TTS and the voice changer hand the upstream body straight to the caller through upstreamBodyStream; their costs are known before the call, so nothing waits on the download. A body that fails mid-stream ends the response early instead of raising an unhandled stream error. xAI speech-to-text keeps one copy of the audio (the upload Blob) instead of up to three. The voice changer and xAI STT hold credits through withAiCreditHold. * perf(ai-ocr): read Puter files only when the provider needs the bytes resolveFileInput resolves and access-checks a Puter file and defers the read to load(); loadFileInput is that plus the read. OCR resolves first: Textract reads the object straight from S3 and downloads it only if Textract can't reach the bucket, and only Mistral loads the bytes. A file over the size cap is refused from its recorded size before any read. countPdfPages inflates object streams on the libuv pool instead of the request thread. * perf(ai): build image and video model catalogs once and share output saving ModelCatalog holds what the public model routes serve (models, sorted names, cost lines), computed once after boot instead of on every listing. The video driver now awaits its model map before serving. resolveOutputPath and saveGeneratedMediaToFS replace the copies of puter_output_path handling in both drivers, using expandTildePath and normalizeAbsolutePath. The video driver normalizes a copy of the caller's args instead of mutating them, and its never-taken stream branch is gone. * fix(drivers): abort the response when a streamed driver result fails mid-body upstreamBodyStream now errors when the provider body fails instead of ending cleanly, and DriverController pipes stream results through pipeStreamResult (stream.pipeline): a failed source destroys the response so the client sees an aborted transfer, the failure is logged once, and nothing is left to raise an unhandled stream error. A client hanging up stops the source. * fix(drivers): drop a streamed driver result when the caller already hung up pipeline() throws synchronously on a closed destination, which surfaced as a 500 alarm on every early disconnect and left the upstream body open. * fix(ai): bound streamed upstream calls by time to first byte, not download time upstreamFetch takes a `stream` option: the timeout covers the request until the response (or a non-2xx error body) is read, then is cleared so a body piped to the client drains at the client's pace. ElevenLabs TTS, xAI TTS and the voice changer pass it; non-streamed calls keep the full-response bound. A stalled upstream body is still cut by undici's body idle timeout. * fix(drivers): type pipeStreamResult's response by the members it uses
64 lines
2.1 KiB
TypeScript
64 lines
2.1 KiB
TypeScript
/*
|
|
* Copyright (C) 2024-present Puter Technologies Inc.
|
|
*
|
|
* This file is part of Puter.
|
|
*
|
|
* Puter is free software: you can redistribute it and/or modify
|
|
* it under the terms of the GNU Affero General Public License as published
|
|
* by the Free Software Foundation, either version 3 of the License, or
|
|
* (at your option) any later version.
|
|
*
|
|
* This program is distributed in the hope that it will be useful,
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
* GNU Affero General Public License for more details.
|
|
*
|
|
* You should have received a copy of the GNU Affero General Public License
|
|
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
*/
|
|
|
|
import { HttpError } from '../../core/http/HttpError.js';
|
|
|
|
const DATA_URI_PATTERN = /^data:([^;,]*)((?:;[^;,]*)*),(.*)$/s;
|
|
|
|
export interface DataUri {
|
|
/** Lowercased; `defaultMimeType` when the URI names none. */
|
|
mimeType: string;
|
|
base64: boolean;
|
|
data: string;
|
|
}
|
|
|
|
/**
|
|
* Split a `data:` URI into its MIME type, base64 flag and payload. Returns null
|
|
* for anything that isn't a data URI. RFC 2397 defaults the type to text/plain;
|
|
* callers that know better pass their own.
|
|
*/
|
|
export function parseDataUri(
|
|
url: string,
|
|
defaultMimeType = 'text/plain',
|
|
): DataUri | null {
|
|
const match = DATA_URI_PATTERN.exec(url);
|
|
if (!match) return null;
|
|
const params = (match[2] ?? '')
|
|
.split(';')
|
|
.filter(Boolean)
|
|
.map((p) => p.trim().toLowerCase());
|
|
return {
|
|
mimeType: (match[1] || defaultMimeType).toLowerCase(),
|
|
base64: params.includes('base64'),
|
|
data: match[3] ?? '',
|
|
};
|
|
}
|
|
|
|
/** The bytes a data URI carries: base64-decoded, or percent-decoded text. */
|
|
export function dataUriBytes(uri: DataUri): Buffer {
|
|
if (uri.base64) return Buffer.from(uri.data, 'base64');
|
|
try {
|
|
return Buffer.from(decodeURIComponent(uri.data));
|
|
} catch {
|
|
throw new HttpError(400, 'Invalid data URL', {
|
|
legacyCode: 'bad_request',
|
|
});
|
|
}
|
|
}
|