mirror of
https://github.com/HeyPuter/puter.git
synced 2026-10-13 07:10:05 +00:00
Daniel Salazar
9030b45459
* perf(ai-chat): cache gateway catalogs with one remote-catalog helper
cachedRemoteCatalog serves a fetched catalog for 15 minutes, shares one
in-flight fetch between callers, abandons a fetch after 10 s, remembers a
failure for 30 s and keeps the last good value through a failed refresh.
Together, OpenRouter, Infron, Neuralwatt (models and quota) and Ollama use it
in place of their kv-singleton caches. The cached value is the converted
catalog, so a completion no longer re-coerces the whole gateway catalog
(OpenRouter did it twice per request). Ollama now refreshes instead of caching
its first non-empty list forever, and Together's catalog failure no longer
rejects models().
* refactor(ai-chat): build every chat SDK client with retries off
sdkClientOptions() gives each OpenAI, Groq, Together and Anthropic client
maxRetries: 0 and an explicit timeout. The SDK default of two retries, each up
to the full timeout, stacked under the driver's three-route fallback chain; a
failed attempt now goes straight back to the driver, which marks the route
and moves to the next provider for the model.
Timeouts keep what each provider already had: ten minutes, Groq's SDK default
of one minute, and Claude's 10 m 0.6 s. Non-streaming Mistral calls, which had
no timeout once a cancellation signal was passed, are bounded at ten minutes.
The Azure providers are left for after the open Azure fixes land.
* refactor(ai-chat): meter every OpenAI-family completion through one pricing path
meterChatUsage records a completion's usage at buildCostsOverride's rates, or
as one billedUsage line when the gateway reports what it billed, and returns
the per-key costs it recorded. Providers hand those to the driver through the
usageCosts channel, now carried by the Chat Completions handlers too, so the
usd_cents and cost-calculated figures equal the ledger. The driver's own
#computeCost is replaced by pricing.ts usageCost, which shares the rate
resolution with buildCostsOverride.
Billing changes, all toward what the upstream charges:
- DeepSeek, Moonshot, Alibaba, Groq, MiniMax, Z.AI, Hoonify, BytePlus and the
Neuralwatt catalog fallback took cached reads out of prompt_tokens nowhere,
so a cached token was billed at input + cache rate. splitCachedPrompt bills
it once, at the cache rate; usage.prompt_tokens on those routes now excludes
cached reads, as it already did for OpenAI, Gemini and Meta.
- Catalog entries that listed cached_tokens: 0 (38 Alibaba, 5 Groq) relied on
that double count to bill cached reads at the input rate. The zero is
dropped, so they keep exactly that price through the input-rate fallback.
Neuralwatt models with no published cache price likewise.
- An unpriced key falls back to the input/output rate instead of NaN (0), e.g.
OpenRouter cached reads on its no-cost fallback path.
- A gateway-reported cost of 0 records 0 µ¢ rather than 1 µ¢.
Claude, Azure and xAI keep their metering; xAI's cached-token handling waits
on the open Grok decision.
* refactor(ai-chat): one OpenAICompatProvider for the OpenAI-compatible vendors
DeepSeek, Moonshot, Alibaba, Groq, MiniMax, Z.AI, Hoonify, BytePlus, Together,
Gemini, Meta, Ollama and the complete() paths of OpenRouter, Infron and
Neuralwatt were copies of one Chat Completions call. Each is now an
OpenAICompatOptions entry (client, catalog, metering prefix, wire-id prefix,
max-token param and default, passthrough fields, openAICompatParams scope,
message cleanup, image inlining, stream usage, context-length retry) plus the
hooks its API needs: prepareMessages (DeepSeek, Groq, Gemini), vendorParams
(MiniMax, Hoonify, Z.AI, BytePlus, Meta, OpenRouter, Infron, Neuralwatt),
meteredUsage (Gemini and the gateways) and reportedUsage (Neuralwatt). xAI is
left until the open xAI fix lands.
Deliberate changes:
- Tools go through make_openai_tools for every vendor (six did, ten sent them
raw): Anthropic-only tool extras and server tools no longer reach these
upstreams, and an empty tool list is omitted.
- Model lookup is one indexed, case-insensitive pass per catalog for every
vendor (Together and Neuralwatt already ignored case).
- Per-provider console logging of upstream errors is gone; the driver logs
every failed attempt.
- Every vendor honours a configured apiBaseUrl.
tool_choice forwarding stays as each vendor had it (MiniMax, Hoonify, Z.AI
auto-only, BytePlus, Meta auto-only, OpenRouter); widening it needs each
vendor's support confirmed.
* perf(ai-chat): resolve the model once in the driver and keep its map fresh
The driver hands each provider its own catalog entry for the route it chose,
so providers stop re-resolving the id per request; the OpenAI-compatible
providers only look an id up (indexed, case-insensitively) for a direct
caller, and an unknown id is a 400 instead of silently becoming the default
model. Gateways with mixed-case ids (Infron, Neuralwatt) used to miss the
driver's lowercased id and fall through to that default.
The model map is now built from every provider's catalog in parallel, with a
failing catalog left out rather than aborting the build, swapped in whole,
awaited in onServerStart, and rebuilt every 15 minutes so gateway catalog
changes reach routing without a restart.
* refactor(ai-chat): drop provider surface and OpenAIUtil code nothing uses
- list() leaves IChatProvider and every provider that isn't Azure or xAI
(both wait on the open fix for those files); only tests called it.
- OpenAI Chat, OpenAI Responses and Meta no longer inline puter_path parts:
the driver does it before every attempt, so theirs found nothing to do. The
stores and FS service they took for it go too.
- providers/ChatProvider.ts had no subclasses.
- OpenAIUtil: one hoistToolUseBlocks for the Chat and Responses message
passes (Responses keeps canonical_id), and the dead create_usage_calculator,
unused Responses deviations and constant mode/toolblock are gone.
checkModeration stays on IChatProvider: the open Azure moderation fix calls it
through that interface.
* refactor(ai-chat): register keyed providers from one table
Sixteen near-identical blocks in #registerProviders, with three constructor
argument orders, become a { name, configKey, Class } table read by one loop;
every entry takes (config, metering) and gets apiBaseUrl when configured.
Claude, Azure, OpenAI (sibling wiring), Ollama (enabled unless turned off) and
the fake provider stay explicit. Gemini's constructor takes config first like
the rest.
* fix(ai-chat): send Ollama the model name it listed
Ollama catalog ids are ollama:ollama/<name>, built from /api/tags. Only the
ollama: part was stripped, so the local server was asked for ollama/<name>, a
model it doesn't have. Strip the whole ollama:ollama/ namespace.
* fix(ai-chat): keep SDK retries on the OpenAI moderation call
The completion clients run with SDK retries off so the driver can fall back.
A moderation check has no fallback, so it passes the SDK's default two
retries per request.
The Open-Source Internet Computer!
« LIVE DEMO »
Puter.com
·
App Store
·
AI Builder
·
Developers
·
X
Puter
Puter is an advanced, open-source, self-hostable internet computer designed to be feature-rich, fast, and highly extensible.
For Users
Puter's goal is to provide you with every app and feature you need to work, create, and play under one roof. From a simple Notepad and Voice Recorder to Spreadsheet and Camera, Puter wants to be the all-in-one solution for your digital life.
For Developers
Puter provides everything you need to build and publish web apps and games. From AI to Cloud Storage and Database to Serverless Workers, Puter has you covered. Puter also helps you get users! Once you build your app, you can publish it on our App Store to reach and monetize users.
Getting Started
💻 Local Development
git clone https://github.com/HeyPuter/puter
cd puter
npm install
npm start
→ This should launch Puter at http://puter.localhost:4100
To run this checkout with Docker, follow Building from source. Create a local docker-compose.override.yml to select the local build; keeping these settings out of docker-compose.yml avoids conflicts when pulling updates and keeps local configuration out of pull requests.
🚀 Self-Hosting
Linux/macOS
curl -fsSL https://puter.com/selfhost | sh
Windows
irm https://puter.com/selfhost?os=windows | iex
→ For more details, see Self-Hosting Puter.
☁️ Puter.com
Puter is available as a hosted service at puter.com.
Support
Connect with the maintainers and community through these channels:
- Bug report or feature request? Please open an issue.
- X (Twitter): x.com/HeyPuter
- Security issues or abuse reports? security@puter.com
- Email maintainers at hi@puter.com
We are always happy to help you with any questions you may have. Don't hesitate to ask!
License
This repository, including all its contents, sub-projects, modules, and components, is licensed under AGPL-3.0 unless explicitly stated otherwise. Third-party libraries included in this repository may be subject to their own licenses.
Translations
- Arabic / العربية
- Armenian / Հայերեն
- Bengali / বাংলা
- Chinese / 中文
- Danish / Dansk
- English
- Farsi / فارسی
- Finnish / Suomi
- French / Français
- German / Deutsch
- Hebrew/ עברית
- Hindi / हिंदी
- Hungarian / Magyar
- Indonesian / Bahasa Indonesia
- Italian / Italiano
- Japanese / 日本語
- Korean / 한국어
- Malay / Bahasa Malaysia
- Malayalam / മലയാളം
- Dutch / Nederlands
- Polish / Polski
- Odia / ଓଡ଼ିଆ
- Portuguese / Português
- Punjabi / ਪੰਜਾਬੀ
- Romanian / Română
- Russian / Русский
- Spanish / Español
- Swedish / Svenska
- Tamil / தமிழ்
- Telugu / తెలుగు
- Thai / ไทย
- Turkish / Türkçe
- Ukrainian / Українська
- Urdu / اردو
- Vietnamese / Tiếng Việt