Open Document Picture-in-Picture windows on behalf of apps

Browsers only allow documentPictureInPicture.requestWindow() from a top-level document, and an app lives in an iframe, so an app calling it gets NotAllowedError ("only allowed from a top-level browsing context"). The `document-picture-in-picture` token in the iframe's `allow` list does nothing — it is not a policy feature the browser knows. Video PiP (video.requestPictureInPicture) already works inside apps. The GUI is the top-level document, so a new PictureInPictureService opens the window for the app and fills it with an iframe of a page the app names, which must come from the app's own origin (checked against the message's origin, now carried on the IPC caller context). One window per app instance; it closes with the app's window, and the app hears about a close it didn't ask for. The window's opener is the GUI, so the page inside it can reach its app's frame through parent.opener.frames and share objects directly — a MediaStream included, which postMessage cannot carry (tracks are not transferable between windows in Chromium). puter.js gains puter.ui.requestPictureInPicture({ url, width, height, onClose }) and puter.ui.exitPictureInPicture(), with docs.
2026-08-27 16:37:18 +00:00 · 2026-08-26 09:10:02 -07:00
parent e430ade7bd
commit 40667bc811
9 changed files with 413 additions and 0 deletions
@@ -50,6 +50,8 @@ The UI API provides a comprehensive set of tools for creating rich user interfac
 ### Additional UI Elements
 - **[`puter.ui.contextMenu()`](/UI/contextMenu/)** - Show a context menu at the cursor
 - **[`puter.ui.hideSpinner()`](/UI/hideSpinner/)** - Hide spinner
+- **[`puter.ui.requestPictureInPicture()`](/UI/requestPictureInPicture/)** - Float a page of the app in a picture-in-picture window
+- **[`puter.ui.exitPictureInPicture()`](/UI/exitPictureInPicture/)** - Close the app's picture-in-picture window
 - **[`puter.ui.showColorPicker()`](/UI/showColorPicker/)** - Show color picker
 - **[`puter.ui.showFontPicker()`](/UI/showFontPicker/)** - Show font picker
 - **[`puter.ui.showSpinner()`](/UI/showSpinner/)** - Show spinner
@@ -0,0 +1,32 @@
+---
+title: puter.ui.exitPictureInPicture()
+description: Closes the picture-in-picture window your app opened.
+platforms: [apps]
+---
+
+Closes the picture-in-picture window opened with [`puter.ui.requestPictureInPicture()`](/UI/requestPictureInPicture/), if one is up. Its `onClose` callback does not run for this — you asked for the close.
+
+## Syntax
+```js
+puter.ui.exitPictureInPicture()
+```
+
+## Return value
+A `Promise` that resolves to `true` if there was a window to close, `false` otherwise.
+
+## Examples
+
+```html
+<html>
+<body>
+    <script src="https://js.puter.com/v2/"></script>
+    <button id="exit">Exit picture-in-picture</button>
+    <script>
+        document.getElementById('exit').onclick = async () => {
+            const wasOpen = await puter.ui.exitPictureInPicture();
+            console.log(wasOpen ? 'closed' : 'nothing was open');
+        };
+    </script>
+</body>
+</html>
+```
@@ -0,0 +1,95 @@
+---
+title: puter.ui.requestPictureInPicture()
+description: Floats a page of your app in an always-on-top picture-in-picture window.
+platforms: [apps]
+---
+
+Floats a page of your app in a picture-in-picture window: a small always-on-top window that stays in view while the user works in other windows or tabs.
+
+Browsers only let a top-level page open a Document Picture-in-Picture window, and an app runs inside an iframe — so calling `documentPictureInPicture.requestWindow()` yourself fails with `NotAllowedError`. Puter opens the window on your app's behalf and loads the page you name in it.
+
+The page must come from your app's own origin. Inside it, your app's main frame is one of `window.parent.opener.frames`: probe them in a `try`/`catch` (frames from other origins throw), and the two pages can share objects directly — a `MediaStream`, which `postMessage` cannot carry, included. `BroadcastChannel` works between them as well.
+
+Call it from a user gesture such as a click; browsers refuse otherwise. One window per app: asking again replaces the one that is up.
+
+## Syntax
+```js
+puter.ui.requestPictureInPicture(options)
+```
+
+## Parameters
+
+#### `options.url` (String) (required)
+The page to show in the window. Resolved against your app's own page, and must be on the same origin.
+
+#### `options.width` (Number) (optional)
+Window width in CSS pixels. The browser may clamp it.
+
+#### `options.height` (Number) (optional)
+Window height in CSS pixels. The browser may clamp it.
+
+#### `options.onClose` (Function) (optional)
+Runs when the window goes away other than through [`puter.ui.exitPictureInPicture()`](/UI/exitPictureInPicture/) — the user closing it, typically.
+
+## Return value
+A `Promise` that resolves once the window is up. It rejects with an error named the way the DOM would name it:
+
+- `NotSupportedError` — the browser has no Document Picture-in-Picture, or the code isn't running as an app on the Puter desktop.
+- `NotAllowedError` — not called from a user gesture.
+- `SecurityError` — `url` is not on your app's origin.
+- `TypeError` — `url` is not a URL.
+
+## Examples
+
+<strong class="example-title">Float a page from a button</strong>
+
+```html
+<html>
+<body>
+    <script src="https://js.puter.com/v2/"></script>
+    <button id="pip">Picture-in-picture</button>
+    <script>
+        document.getElementById('pip').onclick = async () => {
+            try {
+                await puter.ui.requestPictureInPicture({
+                    url: '/pip.html',
+                    width: 400,
+                    height: 300,
+                    onClose: () => console.log('the user closed it'),
+                });
+            } catch (err) {
+                console.error(err.name, err.message);
+            }
+        };
+    </script>
+</body>
+</html>
+```
+
+<strong class="example-title">Reach the main frame from the floating page</strong>
+
+```html
+<!-- pip.html -->
+<html>
+<body>
+    <video id="v" autoplay muted playsinline></video>
+    <script>
+        // The desktop opened this window, so its opener is the desktop, and
+        // your app's main frame is one of the desktop's frames — the only
+        // one this page is allowed to read.
+        const opener = window.parent.opener;
+        for (let i = 0; i < opener.frames.length; i++) {
+            try {
+                const main = opener.frames[i];
+                if (main.myAppStream) {
+                    document.getElementById('v').srcObject = main.myAppStream;
+                    break;
+                }
+            } catch (e) {
+                // a frame from another origin
+            }
+        }
+    </script>
+</body>
+</html>
+```
@@ -896,6 +896,22 @@ let sidebar = [
                source: '/UI/setWindowY.md',
                path: '/UI/setWindowY',
            },
+            {
+                title: '<code>requestPictureInPicture()</code>',
+                page_title: '<code>puter.ui.requestPictureInPicture()</code>',
+                title_tag: 'puter.ui.requestPictureInPicture()',
+                icon: '/assets/img/function.svg',
+                source: '/UI/requestPictureInPicture.md',
+                path: '/UI/requestPictureInPicture',
+            },
+            {
+                title: '<code>exitPictureInPicture()</code>',
+                page_title: '<code>puter.ui.exitPictureInPicture()</code>',
+                title_tag: 'puter.ui.exitPictureInPicture()',
+                icon: '/assets/img/function.svg',
+                source: '/UI/exitPictureInPicture.md',
+                path: '/UI/exitPictureInPicture',
+            },
            {
                title: '<code>showColorPicker()</code>',
                page_title: '<code>puter.ui.showColorPicker()</code>',
@@ -142,6 +142,9 @@ const ipc_listener = async (event, handled) => {
        const ipc_context = {
            caller: {
                process: process,
+                // The frame's origin as it is now (the message's), for
+                // handlers that must know who they are acting for.
+                origin: event.origin,
                app: {
                    appInstanceID: event.data.appInstanceID,
                    iframe,
@@ -3768,6 +3768,10 @@ $.fn.close = async function (options) {
            // notify other apps that we're closing
            window.report_app_closed(window_uuid, options.status_code ?? 0);

+            // A picture-in-picture window the app opened is the app's; it
+            // goes with it.
+            globalThis.services?.get?.('pip')?.close_for_app?.(window_uuid);
+
            // remove backdrop
            $(this).closest('.window-backdrop').remove();

@@ -62,6 +62,7 @@ import { AntiCSRFService } from './services/AntiCSRFService.js';
 import { BroadcastService } from './services/BroadcastService.js';
 import { DebugService } from './services/DebugService.js';
 import { ExecService } from './services/ExecService.js';
+import { PictureInPictureService } from './services/PictureInPictureService.js';
 import { IPCService } from './services/IPCService.js';
 import { LaunchOnInitService } from './services/LaunchOnInitService.js';
 import { LocaleService } from './services/LocaleService.js';
@@ -829,6 +830,7 @@ const launch_services = async function (options) {
    // === Builtin Services ===
    register('ipc', new IPCService());
    register('exec', new ExecService());
+    register('pip', new PictureInPictureService());
    register('debug', new DebugService());
    register('broadcast', new BroadcastService());
    register('theme', new ThemeService());
@@ -0,0 +1,171 @@
+/*
+ * Copyright (C) 2024-present Puter Technologies Inc.
+ *
+ * This file is part of Puter.
+ *
+ * Puter is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as published
+ * by the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+import { Service } from '../definitions.js';
+
+/**
+ * Opens Document Picture-in-Picture windows on behalf of apps.
+ *
+ * `documentPictureInPicture.requestWindow()` is only allowed from a
+ * top-level document, and an app lives in an iframe — so an app cannot
+ * float anything itself, whatever its iframe's `allow` list says (the
+ * `document-picture-in-picture` token there is not a policy feature the
+ * browser knows). The GUI is the top-level document, so it opens the
+ * window and fills it with an iframe of a page the app names. That page
+ * must come from the app's own origin: an app may float its own content
+ * over the user's screen, nobody else's.
+ *
+ * The PiP window's `opener` is the GUI, so a page in it can reach its
+ * app's frame as one of `parent.opener.frames` (same-origin access) and
+ * share objects with it directly — a MediaStream, which postMessage
+ * cannot carry, included. Apps that only need to talk can use
+ * BroadcastChannel; both frames are the same origin.
+ *
+ * One window per app instance; it goes away with the app's window.
+ */
+export class PictureInPictureService extends Service {
+    static description = `
+        Opens Document Picture-in-Picture windows for apps.
+    `;
+
+    async _init ({ services }) {
+        /** @type {Map<string, {pipWindow: Window}>} appInstanceID -> open window */
+        this.windows_ = new Map();
+
+        const svc_ipc = services.get('ipc');
+        svc_ipc.register_ipc_handler('requestPictureInPicture', {
+            handler: this.requestPictureInPicture.bind(this),
+        });
+        svc_ipc.register_ipc_handler('exitPictureInPicture', {
+            handler: this.exitPictureInPicture.bind(this),
+        });
+    }
+
+    get supported () {
+        return typeof globalThis.documentPictureInPicture?.requestWindow === 'function';
+    }
+
+    /**
+     * IPC: float `url` (a page of the caller's own origin) in a
+     * picture-in-picture window. Must run off a user gesture in the app —
+     * activation propagates from the app's frame to this window, and
+     * requestWindow() insists on it.
+     *
+     * @param {{url?: string, width?: number, height?: number}} params
+     * @returns {Promise<{ok: true} | {ok: false, error: {name: string, message: string}}>}
+     */
+    async requestPictureInPicture ({ url, width, height } = {}, { ipc_context } = {}) {
+        const caller = ipc_context?.caller;
+        const instance_id = caller?.app?.appInstanceID;
+        const iframe = caller?.app?.iframe;
+        if ( ! instance_id || ! iframe ) {
+            return fail('InvalidStateError', 'There is no app window to float from.');
+        }
+        if ( ! this.supported ) {
+            return fail('NotSupportedError', 'This browser has no Document Picture-in-Picture.');
+        }
+
+        let target;
+        try {
+            target = new URL(String(url));
+        } catch {
+            return fail('TypeError', '`url` must be an absolute http(s) URL.');
+        }
+        if ( target.protocol !== 'https:' && target.protocol !== 'http:' ) {
+            return fail('TypeError', '`url` must be an absolute http(s) URL.');
+        }
+        // `origin` is the message's — what the frame actually is right now,
+        // not what it was launched as. srcdoc apps have the opaque 'null'.
+        const app_origin = caller.origin;
+        if ( ! app_origin || app_origin === 'null' || target.origin !== app_origin ) {
+            return fail('SecurityError', 'The page must come from the app’s own origin.');
+        }
+
+        // One per app: asking again replaces what is up.
+        this.close_for_app(instance_id);
+
+        const size = {};
+        for ( const [key, value] of [['width', width], ['height', height]] ) {
+            const n = Number(value);
+            if ( Number.isFinite(n) && n > 0 ) size[key] = Math.round(n);
+        }
+
+        let pipWindow;
+        try {
+            pipWindow = await globalThis.documentPictureInPicture.requestWindow(size);
+        } catch ( e ) {
+            return fail(e?.name || 'NotAllowedError', e?.message || 'Could not open a picture-in-picture window.');
+        }
+
+        const doc = pipWindow.document;
+        doc.documentElement.style.height = '100%';
+        doc.body.style.cssText = 'margin:0;height:100%;overflow:hidden;';
+        const pip_iframe = doc.createElement('iframe');
+        pip_iframe.src = target.href;
+        pip_iframe.setAttribute('allow', 'autoplay; encrypted-media');
+        // The same box the app's own frame runs in.
+        pip_iframe.setAttribute('sandbox', 'allow-forms allow-modals allow-pointer-lock allow-popups allow-popups-to-escape-sandbox allow-same-origin allow-scripts allow-downloads');
+        pip_iframe.style.cssText = 'display:block;border:0;width:100%;height:100%;';
+        doc.body.append(pip_iframe);
+
+        const entry = { pipWindow };
+        this.windows_.set(instance_id, entry);
+        // pagehide is the window going away for any reason. Our own close()
+        // takes the entry out first, so only a close we didn't ask for —
+        // the user's, typically — reaches the app.
+        pipWindow.addEventListener('pagehide', () => {
+            if ( this.windows_.get(instance_id) !== entry ) return;
+            this.windows_.delete(instance_id);
+            iframe.contentWindow?.postMessage({ msg: 'pictureInPictureClosed' }, '*');
+        });
+
+        return { ok: true };
+    }
+
+    /**
+     * IPC: close the caller's picture-in-picture window, if it has one.
+     *
+     * @returns {Promise<{ok: true, wasOpen: boolean}>}
+     */
+    async exitPictureInPicture (_params, { ipc_context } = {}) {
+        const instance_id = ipc_context?.caller?.app?.appInstanceID;
+        return { ok: true, wasOpen: this.close_for_app(instance_id) };
+    }
+
+    /**
+     * Closes the window an app instance has up, if any. Called when the
+     * app's window closes, too — the floating window is the app's.
+     *
+     * @param {string} instance_id
+     * @returns {boolean} whether there was one
+     */
+    close_for_app (instance_id) {
+        const entry = this.windows_.get(instance_id);
+        if ( ! entry ) return false;
+        this.windows_.delete(instance_id);
+        try {
+            entry.pipWindow.close();
+        } catch {
+            // already gone
+        }
+        return true;
+    }
+}
+
+const fail = (name, message) => ({ ok: false, error: { name, message } });
@@ -265,6 +265,17 @@ const FILE_OPEN_CANCELLED = Symbol('FILE_OPEN_CANCELLED');

 // A consent prompt covers a handful of scopes at most, and the popup carries
 // them in its URL.
+/**
+ * An error shaped like the DOM's, so `err.name` reads the way it would from
+ * the native picture-in-picture APIs.
+ */
+const pipError = (name, message) => {
+    if ( typeof DOMException === 'function' ) return new DOMException(message, name);
+    const err = new Error(message);
+    err.name = name;
+    return err;
+};
+
 const MAX_REQUESTED_PERMISSIONS = 16;

 /**
@@ -467,6 +478,9 @@ export class UIModule extends EventListener {

    #onLaunchedWithItems;

+    // Runs when the window from requestPictureInPicture() goes away on its own.
+    #onPictureInPictureClosed = null;
+
    // List of events that can be listened to.
    #eventNames;

@@ -659,6 +673,14 @@ export class UIModule extends EventListener {
                    lastDraggedOverElement = null;
                }
            }
+            // pictureInPictureClosed: the window requestPictureInPicture()
+            // opened went away without exitPictureInPicture() — the user
+            // closed it, most likely.
+            else if ( e.data.msg === 'pictureInPictureClosed' ) {
+                const onClose = this.#onPictureInPictureClosed;
+                this.#onPictureInPictureClosed = null;
+                onClose?.();
+            }
            // windowWillClose
            else if ( e.data.msg === 'windowWillClose' ) {
                // If the user has not overridden onWindowClose() then send a message back to the host environment
@@ -1309,6 +1331,72 @@ export class UIModule extends EventListener {
        });
    };

+    /**
+     * Floats a page of this app in a picture-in-picture window: a small
+     * always-on-top window that stays in view while the user works in other
+     * windows or tabs.
+     *
+     * Browsers only let a top-level page open a Document Picture-in-Picture
+     * window, and an app runs in an iframe, so the desktop opens it on the
+     * app's behalf and loads `url` in it. The page must come from this
+     * app's own origin. Inside it, the app's main frame is one of
+     * `window.parent.opener.frames` — probe them in a try/catch, since the
+     * others belong to other origins and throw — so the two can share
+     * objects directly, a MediaStream (which postMessage cannot carry)
+     * included. BroadcastChannel works between them too.
+     *
+     * Call it from a user gesture; browsers refuse otherwise. One window
+     * per app: asking again replaces the one that is up.
+     *
+     * @param {{url: string, width?: number, height?: number, onClose?: () => void}} options
+     *   `url` is resolved against the app's own page. `width`/`height` size
+     *   the window in CSS pixels (the browser may clamp them). `onClose`
+     *   runs when the window goes away other than through
+     *   {@link exitPictureInPicture} — the user closing it, typically.
+     * @returns {Promise<void>} resolves once the window is up. Rejects with
+     *   an error named as the DOM would name it: `NotSupportedError` (no
+     *   Document PiP in this browser, or not running as a desktop app),
+     *   `NotAllowedError` (no user gesture), `SecurityError` (`url` is not
+     *   this app's origin), `TypeError` (`url` is not a URL).
+     */
+    async requestPictureInPicture ({ url, width, height, onClose } = {}) {
+        if ( this.env !== 'app' ) {
+            throw pipError('NotSupportedError', 'requestPictureInPicture() is only available to apps running on the Puter desktop.');
+        }
+        let href;
+        try {
+            href = new URL(String(url), globalThis.location?.href).href;
+        } catch {
+            throw pipError('TypeError', '`url` must be a URL.');
+        }
+        const result = await this.#ipc_stub({
+            method: 'requestPictureInPicture',
+            parameters: { url: href, width, height },
+        });
+        if ( ! result?.ok ) {
+            throw pipError(result?.error?.name ?? 'NotAllowedError',
+                result?.error?.message ?? 'Could not open a picture-in-picture window.');
+        }
+        this.#onPictureInPictureClosed = typeof onClose === 'function' ? onClose : null;
+    }
+
+    /**
+     * Closes the picture-in-picture window opened with
+     * {@link requestPictureInPicture}, if one is up. Its `onClose` does not
+     * run for this — you asked.
+     *
+     * @returns {Promise<boolean>} whether there was a window to close
+     */
+    async exitPictureInPicture () {
+        if ( this.env !== 'app' ) return false;
+        this.#onPictureInPictureClosed = null;
+        const result = await this.#ipc_stub({
+            method: 'exitPictureInPicture',
+            parameters: {},
+        });
+        return result?.wasOpen === true;
+    }
+
    /**
     * Asks the desktop to show its upgrade flow.
     *