diff --git a/src/docs/src/UI.md b/src/docs/src/UI.md index d5d61edb1..027840363 100644 --- a/src/docs/src/UI.md +++ b/src/docs/src/UI.md @@ -50,6 +50,8 @@ The UI API provides a comprehensive set of tools for creating rich user interfac ### Additional UI Elements - **[`puter.ui.contextMenu()`](/UI/contextMenu/)** - Show a context menu at the cursor - **[`puter.ui.hideSpinner()`](/UI/hideSpinner/)** - Hide spinner +- **[`puter.ui.requestPictureInPicture()`](/UI/requestPictureInPicture/)** - Float a page of the app in a picture-in-picture window +- **[`puter.ui.exitPictureInPicture()`](/UI/exitPictureInPicture/)** - Close the app's picture-in-picture window - **[`puter.ui.showColorPicker()`](/UI/showColorPicker/)** - Show color picker - **[`puter.ui.showFontPicker()`](/UI/showFontPicker/)** - Show font picker - **[`puter.ui.showSpinner()`](/UI/showSpinner/)** - Show spinner diff --git a/src/docs/src/UI/exitPictureInPicture.md b/src/docs/src/UI/exitPictureInPicture.md new file mode 100644 index 000000000..7b2826d72 --- /dev/null +++ b/src/docs/src/UI/exitPictureInPicture.md @@ -0,0 +1,32 @@ +--- +title: puter.ui.exitPictureInPicture() +description: Closes the picture-in-picture window your app opened. +platforms: [apps] +--- + +Closes the picture-in-picture window opened with [`puter.ui.requestPictureInPicture()`](/UI/requestPictureInPicture/), if one is up. Its `onClose` callback does not run for this — you asked for the close. + +## Syntax +```js +puter.ui.exitPictureInPicture() +``` + +## Return value +A `Promise` that resolves to `true` if there was a window to close, `false` otherwise. + +## Examples + +```html + + + + + + + +``` diff --git a/src/docs/src/UI/requestPictureInPicture.md b/src/docs/src/UI/requestPictureInPicture.md new file mode 100644 index 000000000..5e66d724c --- /dev/null +++ b/src/docs/src/UI/requestPictureInPicture.md @@ -0,0 +1,95 @@ +--- +title: puter.ui.requestPictureInPicture() +description: Floats a page of your app in an always-on-top picture-in-picture window. +platforms: [apps] +--- + +Floats a page of your app in a picture-in-picture window: a small always-on-top window that stays in view while the user works in other windows or tabs. + +Browsers only let a top-level page open a Document Picture-in-Picture window, and an app runs inside an iframe — so calling `documentPictureInPicture.requestWindow()` yourself fails with `NotAllowedError`. Puter opens the window on your app's behalf and loads the page you name in it. + +The page must come from your app's own origin. Inside it, your app's main frame is one of `window.parent.opener.frames`: probe them in a `try`/`catch` (frames from other origins throw), and the two pages can share objects directly — a `MediaStream`, which `postMessage` cannot carry, included. `BroadcastChannel` works between them as well. + +Call it from a user gesture such as a click; browsers refuse otherwise. One window per app: asking again replaces the one that is up. + +## Syntax +```js +puter.ui.requestPictureInPicture(options) +``` + +## Parameters + +#### `options.url` (String) (required) +The page to show in the window. Resolved against your app's own page, and must be on the same origin. + +#### `options.width` (Number) (optional) +Window width in CSS pixels. The browser may clamp it. + +#### `options.height` (Number) (optional) +Window height in CSS pixels. The browser may clamp it. + +#### `options.onClose` (Function) (optional) +Runs when the window goes away other than through [`puter.ui.exitPictureInPicture()`](/UI/exitPictureInPicture/) — the user closing it, typically. + +## Return value +A `Promise` that resolves once the window is up. It rejects with an error named the way the DOM would name it: + +- `NotSupportedError` — the browser has no Document Picture-in-Picture, or the code isn't running as an app on the Puter desktop. +- `NotAllowedError` — not called from a user gesture. +- `SecurityError` — `url` is not on your app's origin. +- `TypeError` — `url` is not a URL. + +## Examples + +Float a page from a button + +```html + + + + + + + +``` + +Reach the main frame from the floating page + +```html + + + + + + + +``` diff --git a/src/docs/src/sidebar.js b/src/docs/src/sidebar.js index 5ae307a12..2f1051dc5 100755 --- a/src/docs/src/sidebar.js +++ b/src/docs/src/sidebar.js @@ -896,6 +896,22 @@ let sidebar = [ source: '/UI/setWindowY.md', path: '/UI/setWindowY', }, + { + title: 'requestPictureInPicture()', + page_title: 'puter.ui.requestPictureInPicture()', + title_tag: 'puter.ui.requestPictureInPicture()', + icon: '/assets/img/function.svg', + source: '/UI/requestPictureInPicture.md', + path: '/UI/requestPictureInPicture', + }, + { + title: 'exitPictureInPicture()', + page_title: 'puter.ui.exitPictureInPicture()', + title_tag: 'puter.ui.exitPictureInPicture()', + icon: '/assets/img/function.svg', + source: '/UI/exitPictureInPicture.md', + path: '/UI/exitPictureInPicture', + }, { title: 'showColorPicker()', page_title: 'puter.ui.showColorPicker()', diff --git a/src/gui/src/IPC.js b/src/gui/src/IPC.js index d4522a2bc..9a1a16f66 100644 --- a/src/gui/src/IPC.js +++ b/src/gui/src/IPC.js @@ -142,6 +142,9 @@ const ipc_listener = async (event, handled) => { const ipc_context = { caller: { process: process, + // The frame's origin as it is now (the message's), for + // handlers that must know who they are acting for. + origin: event.origin, app: { appInstanceID: event.data.appInstanceID, iframe, diff --git a/src/gui/src/UI/UIWindow.js b/src/gui/src/UI/UIWindow.js index e0a466c22..19df4d922 100644 --- a/src/gui/src/UI/UIWindow.js +++ b/src/gui/src/UI/UIWindow.js @@ -3768,6 +3768,10 @@ $.fn.close = async function (options) { // notify other apps that we're closing window.report_app_closed(window_uuid, options.status_code ?? 0); + // A picture-in-picture window the app opened is the app's; it + // goes with it. + globalThis.services?.get?.('pip')?.close_for_app?.(window_uuid); + // remove backdrop $(this).closest('.window-backdrop').remove(); diff --git a/src/gui/src/initgui.js b/src/gui/src/initgui.js index 691effce4..f6d8b3027 100644 --- a/src/gui/src/initgui.js +++ b/src/gui/src/initgui.js @@ -62,6 +62,7 @@ import { AntiCSRFService } from './services/AntiCSRFService.js'; import { BroadcastService } from './services/BroadcastService.js'; import { DebugService } from './services/DebugService.js'; import { ExecService } from './services/ExecService.js'; +import { PictureInPictureService } from './services/PictureInPictureService.js'; import { IPCService } from './services/IPCService.js'; import { LaunchOnInitService } from './services/LaunchOnInitService.js'; import { LocaleService } from './services/LocaleService.js'; @@ -829,6 +830,7 @@ const launch_services = async function (options) { // === Builtin Services === register('ipc', new IPCService()); register('exec', new ExecService()); + register('pip', new PictureInPictureService()); register('debug', new DebugService()); register('broadcast', new BroadcastService()); register('theme', new ThemeService()); diff --git a/src/gui/src/services/PictureInPictureService.js b/src/gui/src/services/PictureInPictureService.js new file mode 100644 index 000000000..34af78725 --- /dev/null +++ b/src/gui/src/services/PictureInPictureService.js @@ -0,0 +1,171 @@ +/* + * Copyright (C) 2024-present Puter Technologies Inc. + * + * This file is part of Puter. + * + * Puter is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +import { Service } from '../definitions.js'; + +/** + * Opens Document Picture-in-Picture windows on behalf of apps. + * + * `documentPictureInPicture.requestWindow()` is only allowed from a + * top-level document, and an app lives in an iframe — so an app cannot + * float anything itself, whatever its iframe's `allow` list says (the + * `document-picture-in-picture` token there is not a policy feature the + * browser knows). The GUI is the top-level document, so it opens the + * window and fills it with an iframe of a page the app names. That page + * must come from the app's own origin: an app may float its own content + * over the user's screen, nobody else's. + * + * The PiP window's `opener` is the GUI, so a page in it can reach its + * app's frame as one of `parent.opener.frames` (same-origin access) and + * share objects with it directly — a MediaStream, which postMessage + * cannot carry, included. Apps that only need to talk can use + * BroadcastChannel; both frames are the same origin. + * + * One window per app instance; it goes away with the app's window. + */ +export class PictureInPictureService extends Service { + static description = ` + Opens Document Picture-in-Picture windows for apps. + `; + + async _init ({ services }) { + /** @type {Map} appInstanceID -> open window */ + this.windows_ = new Map(); + + const svc_ipc = services.get('ipc'); + svc_ipc.register_ipc_handler('requestPictureInPicture', { + handler: this.requestPictureInPicture.bind(this), + }); + svc_ipc.register_ipc_handler('exitPictureInPicture', { + handler: this.exitPictureInPicture.bind(this), + }); + } + + get supported () { + return typeof globalThis.documentPictureInPicture?.requestWindow === 'function'; + } + + /** + * IPC: float `url` (a page of the caller's own origin) in a + * picture-in-picture window. Must run off a user gesture in the app — + * activation propagates from the app's frame to this window, and + * requestWindow() insists on it. + * + * @param {{url?: string, width?: number, height?: number}} params + * @returns {Promise<{ok: true} | {ok: false, error: {name: string, message: string}}>} + */ + async requestPictureInPicture ({ url, width, height } = {}, { ipc_context } = {}) { + const caller = ipc_context?.caller; + const instance_id = caller?.app?.appInstanceID; + const iframe = caller?.app?.iframe; + if ( ! instance_id || ! iframe ) { + return fail('InvalidStateError', 'There is no app window to float from.'); + } + if ( ! this.supported ) { + return fail('NotSupportedError', 'This browser has no Document Picture-in-Picture.'); + } + + let target; + try { + target = new URL(String(url)); + } catch { + return fail('TypeError', '`url` must be an absolute http(s) URL.'); + } + if ( target.protocol !== 'https:' && target.protocol !== 'http:' ) { + return fail('TypeError', '`url` must be an absolute http(s) URL.'); + } + // `origin` is the message's — what the frame actually is right now, + // not what it was launched as. srcdoc apps have the opaque 'null'. + const app_origin = caller.origin; + if ( ! app_origin || app_origin === 'null' || target.origin !== app_origin ) { + return fail('SecurityError', 'The page must come from the app’s own origin.'); + } + + // One per app: asking again replaces what is up. + this.close_for_app(instance_id); + + const size = {}; + for ( const [key, value] of [['width', width], ['height', height]] ) { + const n = Number(value); + if ( Number.isFinite(n) && n > 0 ) size[key] = Math.round(n); + } + + let pipWindow; + try { + pipWindow = await globalThis.documentPictureInPicture.requestWindow(size); + } catch ( e ) { + return fail(e?.name || 'NotAllowedError', e?.message || 'Could not open a picture-in-picture window.'); + } + + const doc = pipWindow.document; + doc.documentElement.style.height = '100%'; + doc.body.style.cssText = 'margin:0;height:100%;overflow:hidden;'; + const pip_iframe = doc.createElement('iframe'); + pip_iframe.src = target.href; + pip_iframe.setAttribute('allow', 'autoplay; encrypted-media'); + // The same box the app's own frame runs in. + pip_iframe.setAttribute('sandbox', 'allow-forms allow-modals allow-pointer-lock allow-popups allow-popups-to-escape-sandbox allow-same-origin allow-scripts allow-downloads'); + pip_iframe.style.cssText = 'display:block;border:0;width:100%;height:100%;'; + doc.body.append(pip_iframe); + + const entry = { pipWindow }; + this.windows_.set(instance_id, entry); + // pagehide is the window going away for any reason. Our own close() + // takes the entry out first, so only a close we didn't ask for — + // the user's, typically — reaches the app. + pipWindow.addEventListener('pagehide', () => { + if ( this.windows_.get(instance_id) !== entry ) return; + this.windows_.delete(instance_id); + iframe.contentWindow?.postMessage({ msg: 'pictureInPictureClosed' }, '*'); + }); + + return { ok: true }; + } + + /** + * IPC: close the caller's picture-in-picture window, if it has one. + * + * @returns {Promise<{ok: true, wasOpen: boolean}>} + */ + async exitPictureInPicture (_params, { ipc_context } = {}) { + const instance_id = ipc_context?.caller?.app?.appInstanceID; + return { ok: true, wasOpen: this.close_for_app(instance_id) }; + } + + /** + * Closes the window an app instance has up, if any. Called when the + * app's window closes, too — the floating window is the app's. + * + * @param {string} instance_id + * @returns {boolean} whether there was one + */ + close_for_app (instance_id) { + const entry = this.windows_.get(instance_id); + if ( ! entry ) return false; + this.windows_.delete(instance_id); + try { + entry.pipWindow.close(); + } catch { + // already gone + } + return true; + } +} + +const fail = (name, message) => ({ ok: false, error: { name, message } }); diff --git a/src/puter-js/src/modules/UI.js b/src/puter-js/src/modules/UI.js index a9b13d9c5..fd194672c 100644 --- a/src/puter-js/src/modules/UI.js +++ b/src/puter-js/src/modules/UI.js @@ -265,6 +265,17 @@ const FILE_OPEN_CANCELLED = Symbol('FILE_OPEN_CANCELLED'); // A consent prompt covers a handful of scopes at most, and the popup carries // them in its URL. +/** + * An error shaped like the DOM's, so `err.name` reads the way it would from + * the native picture-in-picture APIs. + */ +const pipError = (name, message) => { + if ( typeof DOMException === 'function' ) return new DOMException(message, name); + const err = new Error(message); + err.name = name; + return err; +}; + const MAX_REQUESTED_PERMISSIONS = 16; /** @@ -467,6 +478,9 @@ export class UIModule extends EventListener { #onLaunchedWithItems; + // Runs when the window from requestPictureInPicture() goes away on its own. + #onPictureInPictureClosed = null; + // List of events that can be listened to. #eventNames; @@ -659,6 +673,14 @@ export class UIModule extends EventListener { lastDraggedOverElement = null; } } + // pictureInPictureClosed: the window requestPictureInPicture() + // opened went away without exitPictureInPicture() — the user + // closed it, most likely. + else if ( e.data.msg === 'pictureInPictureClosed' ) { + const onClose = this.#onPictureInPictureClosed; + this.#onPictureInPictureClosed = null; + onClose?.(); + } // windowWillClose else if ( e.data.msg === 'windowWillClose' ) { // If the user has not overridden onWindowClose() then send a message back to the host environment @@ -1309,6 +1331,72 @@ export class UIModule extends EventListener { }); }; + /** + * Floats a page of this app in a picture-in-picture window: a small + * always-on-top window that stays in view while the user works in other + * windows or tabs. + * + * Browsers only let a top-level page open a Document Picture-in-Picture + * window, and an app runs in an iframe, so the desktop opens it on the + * app's behalf and loads `url` in it. The page must come from this + * app's own origin. Inside it, the app's main frame is one of + * `window.parent.opener.frames` — probe them in a try/catch, since the + * others belong to other origins and throw — so the two can share + * objects directly, a MediaStream (which postMessage cannot carry) + * included. BroadcastChannel works between them too. + * + * Call it from a user gesture; browsers refuse otherwise. One window + * per app: asking again replaces the one that is up. + * + * @param {{url: string, width?: number, height?: number, onClose?: () => void}} options + * `url` is resolved against the app's own page. `width`/`height` size + * the window in CSS pixels (the browser may clamp them). `onClose` + * runs when the window goes away other than through + * {@link exitPictureInPicture} — the user closing it, typically. + * @returns {Promise} resolves once the window is up. Rejects with + * an error named as the DOM would name it: `NotSupportedError` (no + * Document PiP in this browser, or not running as a desktop app), + * `NotAllowedError` (no user gesture), `SecurityError` (`url` is not + * this app's origin), `TypeError` (`url` is not a URL). + */ + async requestPictureInPicture ({ url, width, height, onClose } = {}) { + if ( this.env !== 'app' ) { + throw pipError('NotSupportedError', 'requestPictureInPicture() is only available to apps running on the Puter desktop.'); + } + let href; + try { + href = new URL(String(url), globalThis.location?.href).href; + } catch { + throw pipError('TypeError', '`url` must be a URL.'); + } + const result = await this.#ipc_stub({ + method: 'requestPictureInPicture', + parameters: { url: href, width, height }, + }); + if ( ! result?.ok ) { + throw pipError(result?.error?.name ?? 'NotAllowedError', + result?.error?.message ?? 'Could not open a picture-in-picture window.'); + } + this.#onPictureInPictureClosed = typeof onClose === 'function' ? onClose : null; + } + + /** + * Closes the picture-in-picture window opened with + * {@link requestPictureInPicture}, if one is up. Its `onClose` does not + * run for this — you asked. + * + * @returns {Promise} whether there was a window to close + */ + async exitPictureInPicture () { + if ( this.env !== 'app' ) return false; + this.#onPictureInPictureClosed = null; + const result = await this.#ipc_stub({ + method: 'exitPictureInPicture', + parameters: {}, + }); + return result?.wasOpen === true; + } + /** * Asks the desktop to show its upgrade flow. *