feat: paginated fetching for all (#3431)

* feat: paginated fetching for all

* fix: metering top up gui reporting

src/docs/src/Apps/list.md

@@ -31,6 +31,8 @@ An object containing the following properties:
 
 - `includeTotal` (optional): If `true`, the paginated result includes a `total` count of the user's apps.
 
+- `stream` (optional): If `true`, the method returns an async iterator of page objects instead of a promise, for use with `for await ... of`. Combine with `limit` to control the page size, or `cursor` to resume from a previous page. Cannot be combined with `offset`. With `includeTotal`, only the first page carries `total`.
+
 ## Return value
 
 A `Promise` that will resolve to an array of all [`App`](/Objects/app/) objects belonging to the user that this app has access to.
@@ -41,7 +43,17 @@ When the request includes `cursor` (even `null`), `offset`, or `includeTotal`, t
 - `cursor` (String) (optional): Present while more pages exist; pass it to the next call.
 - `total` (Number) (optional): Total app count, present when `includeTotal` was set.
 
-Requests without pagination params keep returning the full list as a plain array, so existing code is unaffected.
+Requests without pagination params keep returning the full list as a plain array, so existing code is unaffected — under the hood the SDK now fetches it page by page.
+
+With `stream: true`, the method returns an async iterator of page objects instead:
+
+```js
+for await (const page of puter.apps.list({ stream: true })) {
+    for (const app of page.items) {
+        console.log(app.name);
+    }
+}
+```
 
 ## Examples
 

src/docs/src/FS/readdir.md

@@ -33,6 +33,7 @@ An object with the following properties:
 - `sortOrder` (String) (optional) - `asc` or `desc`. Default is `asc`.
 - `cursor` (String | null) (optional) - Opts into paginated results. Pass `null` for the first page, then the `cursor` from each page to fetch the next one. The cursor pins the sort, so later pages must not request a different `sortBy`/`sortOrder`.
 - `includeTotal` (Boolean) (optional) - If `true`, the paginated result includes a `total` count of all entries in the directory.
+- `stream` (Boolean) (optional) - If `true`, the method returns an async iterator of page objects instead of a promise, for use with `for await ... of`. Combine with `limit` to control the page size, or `cursor` to resume from a previous page. Cannot be combined with `offset`. With `includeTotal`, only the first page carries `total`.
 
 ## Return value
 
@@ -44,7 +45,17 @@ When the request includes `cursor` (even `null`) or `includeTotal`, the promise
 - `cursor` (String) (optional): Present while more pages exist; pass it to the next call.
 - `total` (Number) (optional): Total entry count, present when `includeTotal` was set.
 
-Requests without pagination params keep returning the full listing as a plain array, so existing code is unaffected.
+Requests without pagination params keep returning the full listing as a plain array, so existing code is unaffected — under the hood the SDK now fetches it page by page.
+
+With `stream: true`, the method returns an async iterator of page objects instead:
+
+```js
+for await (const page of puter.fs.readdir({ path: './large-dir', stream: true })) {
+    for (const item of page.items) {
+        console.log(item.name);
+    }
+}
+```
 
 ## Examples
 

src/docs/src/Hosting/list.md

@@ -22,6 +22,7 @@ An object with the following optional properties:
 - `offset` (Number): Skips the given number of subdomains. Prefer `cursor` for paging through large lists.
 - `cursor` (String | null): Opts into paginated results. Pass `null` for the first page, then the `cursor` from each page to fetch the next one.
 - `includeTotal` (Boolean): If `true`, the paginated result includes a `total` count.
+- `stream` (Boolean): If `true`, the method returns an async iterator of page objects instead of a promise, for use with `for await ... of`. Combine with `limit` to control the page size, or `cursor` to resume from a previous page. Cannot be combined with `offset`. With `includeTotal`, only the first page carries `total`.
 
 ## Return value
 A `Promise` that will resolve to an array of all [`Subdomain`](/Objects/subdomain/) objects belonging to the user that this app has access to.
@@ -32,7 +33,17 @@ When the request includes `cursor` (even `null`) or `includeTotal`, the promise
 - `cursor` (String) (optional): Present while more pages exist; pass it to the next call.
 - `total` (Number) (optional): Present when `includeTotal` was set.
 
-Requests without pagination params keep returning the full list as a plain array, so existing code is unaffected.
+Requests without pagination params keep returning the full list as a plain array, so existing code is unaffected — under the hood the SDK now fetches it page by page.
+
+With `stream: true`, the method returns an async iterator of page objects instead:
+
+```js
+for await (const page of puter.hosting.list({ stream: true })) {
+    for (const site of page.items) {
+        console.log(site.subdomain);
+    }
+}
+```
 
 Worker-backed subdomains are never included in the results — pages and `total` only count sites. Use [`puter.workers.list()`](/Workers/list/) to list workers.
 

src/docs/src/KV/list.md

@@ -37,8 +37,9 @@ An object with the following optional properties:
 - `limit` (Number): Maximum number of items to return in a single call.
 - `cursor` (String): A pagination cursor from a previous call. Pass the `cursor` value returned by the previous page to fetch the next one.
 - `offset` (Number): Skips the given number of items before the page starts. Not recommended — requests get slower and more expensive the larger the offset; prefer `cursor`. Maximum `5000`, and cannot be combined with `cursor`.
-- `includeTotal` (Boolean): If `true`, the result includes a `total` count of every item matching the query (across all pages). Computing the total costs more the more items you have, so request it on the first page only rather than on every page.
+- `includeTotal` (Boolean): If `true`, the result includes a `total` count of every item matching the query (across all pages). The count is metered and its cost grows with the size of your store — request it once (on the first page) and avoid it in hot paths. If you only need to know whether more pages exist, check for `cursor` instead of counting.
 - `fetchUntilFull` (Boolean): A page can come back with fewer than `limit` items even when more exist (for example when expired keys are excluded). If `true`, the page is filled up to `limit` items when possible. Requires `limit`.
+- `stream` (Boolean): If `true`, the method returns an async iterator of [`KVListPage`](/Objects/kvlistpage) objects instead of a promise, for use with `for await ... of`. Combine with `limit` to control the page size, or `cursor` to resume from a previous page. Cannot be combined with `offset`. With `includeTotal`, only the first page carries `total`.
 
 ## Return value
 
@@ -52,6 +53,18 @@ If the user has no keys, the array will be empty.
 
 When paginating, iterate until the result has no `cursor` — a page may hold fewer than `limit` items while more pages still exist.
 
+Full (non-paginated) listings keep resolving to a plain array, so existing code is unaffected — under the hood the SDK now fetches them page by page. They still read the entire store, though: every page is metered, so on large stores a bare `list()` gets slow and costly (the SDK logs a one-time console warning when a full listing spans multiple pages). Prefer `stream: true` or explicit `limit`/`cursor` pages, and narrow the scan with a `pattern`.
+
+With `stream: true`, the method returns an async iterator of [`KVListPage`](/Objects/kvlistpage) objects instead:
+
+```js
+for await (const page of puter.kv.list({ pattern: 'log:*', stream: true })) {
+    for (const key of page.items) {
+        console.log(key);
+    }
+}
+```
+
 ## Examples
 
 <strong class="example-title">Retrieve all keys in the user's key-value store for the current app</strong>

src/docs/src/Objects/kvlistpage.md

@@ -22,4 +22,4 @@ A page may hold fewer than `limit` items while `cursor` is still present — alw
 
 #### `total` (Number) (optional)
 
-The total number of items matching the query across all pages. Present only when the request set `includeTotal: true`.
+The total number of items matching the query across all pages. Present only when the request set `includeTotal: true`. Computing it is metered and its cost grows with the store — request it once (on the first page) and avoid it in hot paths. If you only need to know whether more pages exist, check for `cursor` instead.

src/docs/src/Workers/list.md

@@ -23,6 +23,7 @@ An object with the following optional properties:
 - `offset` (Number): Skips the given number of workers. Prefer `cursor` for paging through large lists.
 - `cursor` (String | null): Opts into paginated results. Pass `null` for the first page, then the `cursor` from each page to fetch the next one.
 - `includeTotal` (Boolean): If `true`, the paginated result includes a `total` count.
+- `stream` (Boolean): If `true`, the method returns an async iterator of page objects instead of a promise, for use with `for await ... of`. Combine with `limit` to control the page size, or `cursor` to resume from a previous page. Cannot be combined with `offset`. With `includeTotal`, only the first page carries `total`.
 
 ## Return Value
 
@@ -34,7 +35,17 @@ When the request includes any pagination option, the promise instead resolves to
 - `cursor` (String) (optional): Present while more pages exist; pass it to the next call.
 - `total` (Number) (optional): Present when `includeTotal` was set.
 
-Requests without pagination params keep returning the full list as a plain array, so existing code is unaffected.
+Requests without pagination params keep returning the full list as a plain array, so existing code is unaffected — under the hood the SDK now fetches it page by page.
+
+With `stream: true`, the method returns an async iterator of page objects instead:
+
+```js
+for await (const page of puter.workers.list({ stream: true })) {
+    for (const worker of page.items) {
+        console.log(worker.name);
+    }
+}
+```
 
 ## Examples