diff --git a/.github/instructions/general.instructions.md b/.github/instructions/general.instructions.md new file mode 100644 index 00000000..79eba3df --- /dev/null +++ b/.github/instructions/general.instructions.md @@ -0,0 +1 @@ +Review this PR. Do not leave line comments. Provide exactly one summary comment including: 1. A letter grade (A-F), 2. Major shortcomings blocking merge 3. Nice-to-have shortcomings 4. Merge readiness (Yes/No). \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5ca47169..d386d2ff 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,10 +4,13 @@ Thank you for considering contributing to this project! By contributing, you hel Areas where help is especially valuable: -- Native capture pipeline for Linux +- Native screen recording for Linux - **Webcam overlay bubble** -- **UI/UX design improvements** -- Export speed improvements +- **UI/UX design improvements (very helpful)** +- German localisation +- Code optimisation/refactoring +- Auto-zoom suggestion logic improvements +- Regular updating of Chinese Readme ## How to Contribute diff --git a/README.md b/README.md index b40509de..af3b6d94 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,7 @@ Language: EN | [简中](README.zh-CN.md) ### Create polished, pro-grade screen recordings. [Recordly](https://www.recordly.dev) is an **open-source screen recorder and editor** for creating **polished walkthroughs, demos, tutorials, and product videos**. Contribution encouraged. -**FAQ**: What are the changes between this and **Openscreen**? A: Recordly adds a full cursor animation/rendering pipeline, native screen capture for Mac and Windows, zoom animations faithful to Screen Studio, cursor loops, smoother panning behaviour, and more major tweaks. -> This fork exists because the original maintainer does not wish implementing the architectural changes that make some of these features possible i.e. different recording pipeline. +**FAQ**: What are the changes between this and the **upstream project?** A: Recordly adds a full cursor animation/rendering pipeline, native macOS and Windows screen recording system, zoom animations faithful to Screen Studio, cursor loops, audio tracks, and more major tweaks.

Recordly demo video @@ -33,11 +32,11 @@ Recordly lets you record your screen and automatically transform it into a polis Recordly runs on: -- **macOS** -- **Windows** -- **Linux** +- **macOS** 12.3+ +- **Windows** 10 Build 19041+ +- **Linux** (modern distros) -Linux currently use Electron's capture path, which means the OS cursor cannot always be hidden during recording. +On Windows, builds older than 19041 fall back to Electron capture and the cursor cannot be hidden. On Linux, cursor hiding is not supported (contribute). @@ -69,12 +68,12 @@ Linux currently use Electron's capture path, which means the OS cursor cannot al - Click bounce animation - macOS-style cursor assets -### Cursor Loops +### Infinite Loops

Recordly demo video

-- Cursor returns to original position in a freeze-frame at end of video/GIF (off by default) +- Toggle to make cursor return to original position at end of video/GIF for clean loops ### Editing Tools @@ -106,11 +105,11 @@ Linux currently use Electron's capture path, which means the OS cursor cannot al # Screenshots

- Recordly editor screenshot + Recordly editor screenshot

- Recordly recording interface screenshot + Recordly recording interface screenshot

--- @@ -159,6 +158,19 @@ xattr -rd com.apple.quarantine /Applications/Recordly.app --- +# System Requirements + +| Platform | Minimum version | Notes | +|---|---|---| +| **macOS** | macOS 12.3 (Monterey) | Required for ScreenCaptureKit. Recording and cursor hiding will not work on older versions. | +| **Windows** | Windows 10 20H1 (Build 19041, May 2020) | Required for Windows Graphics Capture (`IsCursorCaptureEnabled`). Older builds fall back to Electron browser capture — cursor will be visible in recordings. | +| **Linux** | Any modern distro | Recording works via Electron capture. Cursor is always visible in recordings. System audio requires PipeWire (Ubuntu 22.04+, Fedora 34+). | + +> [!IMPORTANT] +> On Windows, if your build is older than 19041, recording will still work but **the cursor cannot be hidden** from the captured video. + +--- + # Usage ## Record @@ -204,11 +216,13 @@ Adjust: # Limitations -### Linux Cursor Capture +### Cursor Capture -Electron’s desktop capture API does not allow hiding the system cursor during recording. +**macOS**: Cursor is excluded from the recording at the ScreenCaptureKit level — always clean. -If you enable the animated cursor layer, recordings may contain **two cursors**. +**Windows**: Cursor is excluded via Windows Graphics Capture (`IsCursorCaptureEnabled(false)`) — requires **Windows 10 Build 19041+**. On older builds the app falls back to Electron’s browser capture and the real cursor will be visible in the recording. + +**Linux**: Electron’s desktop capture API does not support cursor hiding. The real OS cursor will always be visible in recordings. If you also enable the animated cursor overlay in the editor, you may see **two cursors** in the output. Improving cross-platform cursor capture is an area where contributions are welcome. @@ -219,7 +233,8 @@ Improving cross-platform cursor capture is an area where contributions are welco System audio capture depends on platform support. **Windows** -- Works out of the box +- Works out of the box via native WASAPI +- Requires Windows 10 Build 19041+ **Linux** - Requires PipeWire (Ubuntu 22.04+, Fedora 34+) @@ -311,7 +326,7 @@ Recordly is licensed under the **MIT License**. ## Acknowledgements -Built on top of the excellent [OpenScreen](https://github.com/siddharthvaddem/openscreen) project, you should go check it out! +Originally built on top of the excellent [OpenScreen](https://github.com/siddharthvaddem/openscreen) project. Created by [@webadderall](https://x.com/webadderall) diff --git a/README.zh-CN.md b/README.zh-CN.md index 0d2dfbb4..3387b380 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -2,8 +2,6 @@ 语言: [English](README.md) | 简体中文 -[Product Hunt 页面 - 欢迎支持！](https://www.producthunt.com/products/recordly-make-stunning-product-demos?launch=recordly-2) -

Recordly logo

diff --git a/electron/electron-env.d.ts b/electron/electron-env.d.ts index f859957d..371ea789 100644 --- a/electron/electron-env.d.ts +++ b/electron/electron-env.d.ts @@ -83,6 +83,12 @@ interface Window { muxWgcRecording: () => Promise<{ success: boolean; path?: string; message?: string; error?: string }> /** Hide the OS cursor before browser capture starts. */ hideOsCursor: () => Promise<{ success: boolean }> + /** Countdown timer before recording */ + getCountdownDelay: () => Promise<{ success: boolean; delay: number }> + setCountdownDelay: (delay: number) => Promise<{ success: boolean; error?: string }> + startCountdown: (seconds: number) => Promise<{ success: boolean; cancelled?: boolean }> + cancelCountdown: () => Promise<{ success: boolean }> + onCountdownTick: (callback: (seconds: number) => void) => () => void } } diff --git a/electron/ipc/handlers.ts b/electron/ipc/handlers.ts index 3c3430dc..00f0da6e 100644 --- a/electron/ipc/handlers.ts +++ b/electron/ipc/handlers.ts @@ -9,6 +9,7 @@ import path from 'node:path' import { fileURLToPath, pathToFileURL } from 'node:url' import { promisify } from 'node:util' import { RECORDINGS_DIR } from '../main' +import { createCountdownWindow, getCountdownWindow, closeCountdownWindow } from '../windows' const execFileAsync = promisify(execFile) const nodeRequire = createRequire(import.meta.url) @@ -17,6 +18,7 @@ const PROJECT_FILE_EXTENSION = 'recordly' const LEGACY_PROJECT_FILE_EXTENSIONS = ['openscreen'] const SHORTCUTS_FILE = path.join(app.getPath('userData'), 'shortcuts.json') const RECORDINGS_SETTINGS_FILE = path.join(app.getPath('userData'), 'recordings-settings.json') +const COUNTDOWN_SETTINGS_FILE = path.join(app.getPath('userData'), 'countdown-settings.json') const AUTO_RECORDING_PREFIX = 'recording-' const AUTO_RECORDING_RETENTION_COUNT = 20 const AUTO_RECORDING_MAX_AGE_MS = 14 * 24 * 60 * 60 * 1000 @@ -77,6 +79,9 @@ let customRecordingsDir: string | null = null let recordingsDirLoaded = false let cachedSystemCursorAssets: Record | null = null let cachedSystemCursorAssetsSourceMtimeMs: number | null = null +let countdownTimer: ReturnType | null = null +let countdownCancelled = false +let countdownInProgress = false type SystemCursorAsset = { dataUrl: string @@ -1219,6 +1224,7 @@ let isCursorCaptureActive = false let interactionCaptureCleanup: (() => void) | null = null let hasLoggedInteractionHookFailure = false let lastLeftClick: { timeMs: number; cx: number; cy: number } | null = null +let linuxCursorScreenPoint: { x: number; y: number; updatedAt: number } | null = null let selectedWindowBounds: WindowBounds | null = null let windowBoundsCaptureInterval: NodeJS.Timeout | null = null @@ -1313,21 +1319,26 @@ async function resolveMacWindowBounds(source: SelectedSource): Promise { + if (process.platform !== 'linux' || !isCursorCaptureActive) { + return + } + + const point = getHookCursorScreenPoint(event) + if (!point) { + return + } + + linuxCursorScreenPoint = { x: point.x, y: point.y, updatedAt: Date.now() } + } + hook.on('mousedown', onMouseDown) hook.on('mouseup', onMouseUp) + hook.on('mousemove', onMouseMove) hook.start() @@ -1517,9 +1561,11 @@ async function startInteractionCapture() { if (typeof hook.off === 'function') { hook.off('mousedown', onMouseDown) hook.off('mouseup', onMouseUp) + hook.off('mousemove', onMouseMove) } else if (typeof hook.removeListener === 'function') { hook.removeListener('mousedown', onMouseDown) hook.removeListener('mouseup', onMouseUp) + hook.removeListener('mousemove', onMouseMove) } } catch { // ignore listener cleanup errors @@ -2224,6 +2270,7 @@ export function registerIpcHandlers( activeCursorSamples = [] pendingCursorSamples = [] cursorCaptureStartTimeMs = Date.now() + linuxCursorScreenPoint = null lastLeftClick = null sampleCursorPoint() cursorCaptureInterval = setInterval(sampleCursorPoint, CURSOR_SAMPLE_INTERVAL_MS) @@ -2234,6 +2281,7 @@ export function registerIpcHandlers( stopInteractionCapture() stopWindowBoundsCapture() stopNativeCursorMonitor() + linuxCursorScreenPoint = null snapshotCursorTelemetryForPersistence() activeCursorSamples = [] } @@ -2485,6 +2533,35 @@ export function registerIpcHandlers( } }); + ipcMain.handle('open-audio-file-picker', async () => { + try { + const result = await dialog.showOpenDialog({ + title: 'Select Audio File', + filters: [ + { name: 'Audio Files', extensions: ['mp3', 'wav', 'aac', 'm4a', 'flac', 'ogg'] }, + { name: 'All Files', extensions: ['*'] } + ], + properties: ['openFile'] + }); + + if (result.canceled || result.filePaths.length === 0) { + return { success: false, canceled: true }; + } + + return { + success: true, + path: result.filePaths[0] + }; + } catch (error) { + console.error('Failed to open audio file picker:', error); + return { + success: false, + message: 'Failed to open audio file picker', + error: String(error) + }; + } + }); + ipcMain.handle('reveal-in-folder', async (_, filePath: string) => { try { // shell.showItemInFolder doesn't return a value, it throws on error @@ -2714,9 +2791,11 @@ export function registerIpcHandlers( // The IPC promise resolves only after the cursor hide attempt completes. // --------------------------------------------------------------------------- ipcMain.handle('hide-cursor', () => { - // No-op: macOS uses native ScreenCaptureKit (cursor excluded at capture - // level), and Win/Linux use Electron desktopCapturer where cursor hiding - // is not reliably supported. + // No-op: macOS excludes the cursor at the ScreenCaptureKit capture level. + // Windows excludes the cursor via IsCursorCaptureEnabled(false) in wgc_session.cpp. + // Linux uses Electron desktopCapturer which does not support cursor hiding; + // if WGC is unavailable on Windows the call also falls back to browser capture + // where cursor hiding is unsupported — those users may see the real cursor. return { success: true } }) @@ -2738,5 +2817,92 @@ export function registerIpcHandlers( return { success: false, error: String(error) }; } }); + + // --------------------------------------------------------------------------- + // Countdown timer before recording + // --------------------------------------------------------------------------- + ipcMain.handle('get-countdown-delay', async () => { + try { + const content = await fs.readFile(COUNTDOWN_SETTINGS_FILE, 'utf-8') + const parsed = JSON.parse(content) as { delay?: number } + return { success: true, delay: parsed.delay ?? 3 } + } catch { + return { success: true, delay: 3 } + } + }) + + ipcMain.handle('set-countdown-delay', async (_, delay: number) => { + try { + await fs.writeFile(COUNTDOWN_SETTINGS_FILE, JSON.stringify({ delay }, null, 2), 'utf-8') + return { success: true } + } catch (error) { + console.error('Failed to save countdown delay:', error) + return { success: false, error: String(error) } + } + }) + + ipcMain.handle('start-countdown', async (_, seconds: number) => { + if (countdownInProgress) { + return { success: false, error: 'Countdown already in progress' } + } + + countdownInProgress = true + countdownCancelled = false + + const countdownWin = createCountdownWindow() + + await new Promise((resolve) => { + countdownWin.webContents.once('did-finish-load', () => { + resolve() + }) + }) + + return new Promise<{ success: boolean; cancelled?: boolean }>((resolve) => { + let remaining = seconds + + countdownWin.webContents.send('countdown-tick', remaining) + + countdownTimer = setInterval(() => { + if (countdownCancelled) { + if (countdownTimer) { + clearInterval(countdownTimer) + countdownTimer = null + } + closeCountdownWindow() + countdownInProgress = false + resolve({ success: false, cancelled: true }) + return + } + + remaining-- + + if (remaining <= 0) { + if (countdownTimer) { + clearInterval(countdownTimer) + countdownTimer = null + } + closeCountdownWindow() + countdownInProgress = false + resolve({ success: true }) + } else { + const win = getCountdownWindow() + if (win && !win.isDestroyed()) { + win.webContents.send('countdown-tick', remaining) + } + } + }, 1000) + }) + }) + + ipcMain.handle('cancel-countdown', () => { + countdownCancelled = true + countdownInProgress = false + if (countdownTimer) { + clearInterval(countdownTimer) + countdownTimer = null + } + closeCountdownWindow() + return { success: true } + }) } diff --git a/electron/main.ts b/electron/main.ts index adc010ea..163a6507 100644 --- a/electron/main.ts +++ b/electron/main.ts @@ -84,10 +84,25 @@ function focusOrCreateMainWindow() { } if (mainWindow && !mainWindow.isDestroyed()) { - if (mainWindow.isMinimized()) mainWindow.restore() + // On Linux/Wayland, focus() often doesn't take effect (compositor ignores it). Apps like Telegram + // work because they receive an XDG activation token via StatusNotifierItem.ProvideXdgActivationToken; + // Electron's tray doesn't handle that yet. Workaround: destroy and recreate the HUD so the new + // window gets focus (creation path works). Only for HUD, not editor. + if ( + process.platform === 'linux' && + !mainWindow.isFocused() && + !isEditorWindow(mainWindow) + ) { + const win = mainWindow + mainWindow = null + win.once('closed', () => createWindow()) + win.destroy() + return + } mainWindow.show() - mainWindow.focus() + if (mainWindow.isMinimized()) mainWindow.restore() mainWindow.moveTop() + mainWindow.focus() } } @@ -204,6 +219,7 @@ function setupApplicationMenu() { function createTray() { tray = new Tray(defaultTrayIcon); + tray.on('click', () => focusOrCreateMainWindow()) } function getPublicAssetPath(filename: string) { diff --git a/electron/native/bin/darwin-arm64/openscreen-native-cursor-monitor b/electron/native/bin/darwin-arm64/openscreen-native-cursor-monitor index 9b3e16e1..77d85ce8 100755 Binary files a/electron/native/bin/darwin-arm64/openscreen-native-cursor-monitor and b/electron/native/bin/darwin-arm64/openscreen-native-cursor-monitor differ diff --git a/electron/native/bin/darwin-arm64/openscreen-screencapturekit-helper b/electron/native/bin/darwin-arm64/openscreen-screencapturekit-helper index 2f5bda23..447588fd 100755 Binary files a/electron/native/bin/darwin-arm64/openscreen-screencapturekit-helper and b/electron/native/bin/darwin-arm64/openscreen-screencapturekit-helper differ diff --git a/electron/native/bin/darwin-arm64/openscreen-system-cursors b/electron/native/bin/darwin-arm64/openscreen-system-cursors index 131caccd..926b24e0 100755 Binary files a/electron/native/bin/darwin-arm64/openscreen-system-cursors and b/electron/native/bin/darwin-arm64/openscreen-system-cursors differ diff --git a/electron/native/bin/darwin-arm64/openscreen-window-list b/electron/native/bin/darwin-arm64/openscreen-window-list index 34ec60b7..977c0283 100755 Binary files a/electron/native/bin/darwin-arm64/openscreen-window-list and b/electron/native/bin/darwin-arm64/openscreen-window-list differ diff --git a/electron/preload.ts b/electron/preload.ts index ebc4fed1..bc282d8b 100644 --- a/electron/preload.ts +++ b/electron/preload.ts @@ -113,6 +113,9 @@ contextBridge.exposeInMainWorld('electronAPI', { openVideoFilePicker: () => { return ipcRenderer.invoke('open-video-file-picker') }, + openAudioFilePicker: () => { + return ipcRenderer.invoke('open-audio-file-picker') + }, setCurrentVideoPath: (path: string) => { return ipcRenderer.invoke('set-current-video-path', path) }, @@ -182,5 +185,15 @@ contextBridge.exposeInMainWorld('electronAPI', { muxWgcRecording: () => ipcRenderer.invoke('mux-wgc-recording'), // Cursor visibility control for cursor-free browser capture fallback hideOsCursor: () => ipcRenderer.invoke('hide-cursor'), + // Countdown timer before recording + getCountdownDelay: () => ipcRenderer.invoke('get-countdown-delay'), + setCountdownDelay: (delay: number) => ipcRenderer.invoke('set-countdown-delay', delay), + startCountdown: (seconds: number) => ipcRenderer.invoke('start-countdown', seconds), + cancelCountdown: () => ipcRenderer.invoke('cancel-countdown'), + onCountdownTick: (callback: (seconds: number) => void) => { + const listener = (_event: Electron.IpcRendererEvent, seconds: number) => callback(seconds) + ipcRenderer.on('countdown-tick', listener) + return () => ipcRenderer.removeListener('countdown-tick', listener) + }, }) diff --git a/electron/windows.ts b/electron/windows.ts index d2f7466d..30674666 100644 --- a/electron/windows.ts +++ b/electron/windows.ts @@ -12,6 +12,7 @@ const RENDERER_DIST = path.join(APP_ROOT, 'dist') const WINDOW_ICON_PATH = path.join(process.env.VITE_PUBLIC || RENDERER_DIST, 'app-icons', 'recordly-512.png') let hudOverlayWindow: BrowserWindow | null = null; +let countdownWindow: BrowserWindow | null = null; function getScreen() { return nodeRequire('electron').screen as typeof import('electron').screen @@ -28,8 +29,8 @@ export function createHudOverlayWindow(): BrowserWindow { const { workArea } = primaryDisplay; - const windowWidth = 600; - const windowHeight = 155; + const windowWidth = 660; + const windowHeight = 170; const x = Math.floor(workArea.x + (workArea.width - windowWidth) / 2); const y = Math.floor(workArea.y + workArea.height - windowHeight - 5); @@ -37,10 +38,10 @@ export function createHudOverlayWindow(): BrowserWindow { const win = new BrowserWindow({ width: windowWidth, height: windowHeight, - minWidth: 600, - maxWidth: 600, - minHeight: 155, - maxHeight: 155, + minWidth: 660, + maxWidth: 660, + minHeight: 170, + maxHeight: 170, x: x, y: y, frame: false, @@ -49,6 +50,7 @@ export function createHudOverlayWindow(): BrowserWindow { alwaysOnTop: true, skipTaskbar: true, hasShadow: false, + show: false, webPreferences: { preload: path.join(__dirname, 'preload.mjs'), nodeIntegration: false, @@ -60,6 +62,11 @@ export function createHudOverlayWindow(): BrowserWindow { win.webContents.on('did-finish-load', () => { win?.webContents.send('main-process-message', (new Date).toLocaleString()) + setTimeout(() => { + if (!win.isDestroyed()) { + win.show() + } + }, 100) }) hudOverlayWindow = win; @@ -102,6 +109,7 @@ export function createEditorWindow(): BrowserWindow { alwaysOnTop: false, skipTaskbar: false, title: 'Recordly', + show: false, backgroundColor: '#000000', webPreferences: { preload: path.join(__dirname, 'preload.mjs'), @@ -112,8 +120,10 @@ export function createEditorWindow(): BrowserWindow { }, }) - // Maximize the window by default - win.maximize(); + win.once('ready-to-show', () => { + win.show() + win.maximize() + }) win.webContents.on('did-finish-load', () => { win?.webContents.send('main-process-message', (new Date).toLocaleString()) @@ -144,6 +154,7 @@ export function createSourceSelectorWindow(): BrowserWindow { resizable: false, alwaysOnTop: true, transparent: true, + show: false, ...(process.platform !== 'darwin' && { icon: WINDOW_ICON_PATH, }), @@ -155,14 +166,82 @@ export function createSourceSelectorWindow(): BrowserWindow { }, }) + win.webContents.on('did-finish-load', () => { + setTimeout(() => { + if (!win.isDestroyed()) { + win.show() + } + }, 100) + }) + if (VITE_DEV_SERVER_URL) { win.loadURL(VITE_DEV_SERVER_URL + '?windowType=source-selector') } else { - win.loadFile(path.join(RENDERER_DIST, 'index.html'), { - query: { windowType: 'source-selector' } + win.loadFile(path.join(RENDERER_DIST, 'index.html'), { + query: { windowType: 'source-selector' } }) } return win } +export function createCountdownWindow(): BrowserWindow { + const primaryDisplay = getScreen().getPrimaryDisplay(); + const { width, height } = primaryDisplay.workAreaSize; + + const windowSize = 200; + const x = Math.floor((width - windowSize) / 2); + const y = Math.floor((height - windowSize) / 2); + + const win = new BrowserWindow({ + width: windowSize, + height: windowSize, + x: x, + y: y, + frame: false, + transparent: true, + resizable: false, + alwaysOnTop: true, + skipTaskbar: true, + hasShadow: false, + focusable: true, + webPreferences: { + preload: path.join(__dirname, 'preload.mjs'), + nodeIntegration: false, + contextIsolation: true, + }, + }) + + countdownWindow = win; + + // Show on all workspaces/spaces so it follows the user + win.setVisibleOnAllWorkspaces(true, { visibleOnFullScreen: true }); + + win.on('closed', () => { + if (countdownWindow === win) { + countdownWindow = null; + } + }); + + if (VITE_DEV_SERVER_URL) { + win.loadURL(VITE_DEV_SERVER_URL + '?windowType=countdown') + } else { + win.loadFile(path.join(RENDERER_DIST, 'index.html'), { + query: { windowType: 'countdown' } + }) + } + + return win +} + +export function getCountdownWindow(): BrowserWindow | null { + return countdownWindow; +} + +export function closeCountdownWindow(): void { + if (countdownWindow && !countdownWindow.isDestroyed()) { + countdownWindow.close(); + countdownWindow = null; + } +} + diff --git a/package-lock.json b/package-lock.json index 09d8925f..74cf3d25 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,13 @@ { "name": "recordly", - "version": "1.1.3", + "version": "1.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "recordly", - "version": "1.1.3", + "version": "1.0.0", + "hasInstallScript": true, "dependencies": { "@fix-webm-duration/fix": "^1.0.1", "@pixi/filter-drop-shadow": "^5.2.0", @@ -14118,4 +14119,3 @@ } } } - diff --git a/scripts/build-native-helpers.mjs b/scripts/build-native-helpers.mjs index fc703007..032ae2f3 100644 --- a/scripts/build-native-helpers.mjs +++ b/scripts/build-native-helpers.mjs @@ -44,7 +44,12 @@ for (const helper of helpers) { const sourcePath = path.join(nativeRoot, helper.source); const outputPath = path.join(outputDir, helper.output); - const result = spawnSync('swiftc', ['-O', sourcePath, '-o', outputPath], { + const result = spawnSync('swiftc', [ + '-O', + '-target', process.arch === 'arm64' ? 'arm64-apple-macos14.0' : 'x86_64-apple-macos14.0', + sourcePath, + '-o', outputPath + ], { encoding: 'utf8', timeout: 120000, }); diff --git a/src/App.tsx b/src/App.tsx index 2a471f1f..2418ce53 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -1,4 +1,5 @@ import { useEffect, useState } from "react"; +import { CountdownOverlay } from "./components/countdown/CountdownOverlay"; import { LaunchWindow } from "./components/launch/LaunchWindow"; import { SourceSelector } from "./components/launch/SourceSelector"; import VideoEditor from "./components/video-editor/VideoEditor"; @@ -16,7 +17,7 @@ export default function App() { const type = params.get('windowType') || ''; setWindowType(type); - if (type === 'hud-overlay' || type === 'source-selector') { + if (type === 'hud-overlay' || type === 'source-selector' || type === 'countdown') { document.body.style.background = 'transparent'; document.documentElement.style.background = 'transparent'; document.getElementById('root')?.style.setProperty('background', 'transparent'); @@ -39,6 +40,8 @@ export default function App() { return ; case 'source-selector': return ; + case 'countdown': + return ; case 'editor': return ( diff --git a/src/components/countdown/CountdownOverlay.tsx b/src/components/countdown/CountdownOverlay.tsx new file mode 100644 index 00000000..50c68e29 --- /dev/null +++ b/src/components/countdown/CountdownOverlay.tsx @@ -0,0 +1,64 @@ +import { useCallback, useEffect, useState } from "react"; + +export function CountdownOverlay() { + const [countdown, setCountdown] = useState(null); + + useEffect(() => { + const cleanup = window.electronAPI.onCountdownTick((seconds: number) => { + setCountdown(seconds); + }); + + return cleanup; + }, []); + + const handleCancel = useCallback(() => { + window.electronAPI.cancelCountdown(); + }, []); + + const handleKeyDown = useCallback( + (event: KeyboardEvent) => { + if (event.key === "Escape") { + handleCancel(); + } + }, + [handleCancel], + ); + + useEffect(() => { + window.addEventListener("keydown", handleKeyDown); + return () => window.removeEventListener("keydown", handleKeyDown); + }, [handleKeyDown]); + + if (countdown === null) { + return null; + } + + return ( +
e.key === "Escape" && handleCancel()} + > +
+ + {countdown} + +
+
+ ); +} diff --git a/src/components/launch/LaunchWindow.tsx b/src/components/launch/LaunchWindow.tsx index 20d121b0..06c48067 100644 --- a/src/components/launch/LaunchWindow.tsx +++ b/src/components/launch/LaunchWindow.tsx @@ -4,7 +4,7 @@ import { FaRegStopCircle } from "react-icons/fa"; import { FaFolderOpen } from "react-icons/fa6"; import { FiMinus, FiX } from "react-icons/fi"; import { MdMic, MdMicOff, MdMonitor, MdVideoFile, MdVolumeOff, MdVolumeUp } from "react-icons/md"; -import { Languages } from "lucide-react"; +import { Languages, Timer } from "lucide-react"; import { RxDragHandleDots2 } from "react-icons/rx"; import { useAudioLevelMeter } from "../../hooks/useAudioLevelMeter"; import { useMicrophoneDevices } from "../../hooks/useMicrophoneDevices"; @@ -31,6 +31,7 @@ export function LaunchWindow() { const LOCALE_LABELS: Record = { en: "EN", es: "ES", "zh-CN": "中文" }; const { recording, + countdownActive, toggleRecording, microphoneEnabled, setMicrophoneEnabled, @@ -38,6 +39,8 @@ export function LaunchWindow() { setMicrophoneDeviceId, systemAudioEnabled, setSystemAudioEnabled, + countdownDelay, + setCountdownDelay, } = useScreenRecorder(); const [recordingStart, setRecordingStart] = useState(null); const [elapsed, setElapsed] = useState(0); @@ -245,11 +248,43 @@ export function LaunchWindow() {
+ + + + + + {[0, 3, 5, 10].map((delay) => ( + setCountdownDelay(delay)} + className={`text-xs cursor-pointer ${ + countdownDelay === delay ? "text-white font-medium" : "text-white/60" + }`} + > + {delay === 0 ? t('recording.noDelay') : `${delay}s`} + + ))} + + + +
)} diff --git a/src/components/video-editor/VideoEditor.tsx b/src/components/video-editor/VideoEditor.tsx index f410b42b..34bec02c 100644 --- a/src/components/video-editor/VideoEditor.tsx +++ b/src/components/video-editor/VideoEditor.tsx @@ -5,84 +5,72 @@ import { Panel, PanelGroup, PanelResizeHandle } from "react-resizable-panels"; import { toast } from "sonner"; import { Toaster } from "@/components/ui/sonner"; import { useI18n } from "@/contexts/I18nContext"; +import { useShortcuts } from "@/contexts/ShortcutsContext"; import { SUPPORTED_LOCALES } from "@/i18n/config"; import type { AppLocale } from "@/i18n/config"; -import { useShortcuts } from "@/contexts/ShortcutsContext"; -import { getAssetPath } from "@/lib/assetPath"; -import { - calculateOutputDimensions, - type ExportFormat, - type ExportProgress, - type ExportQuality, - type ExportSettings, - GIF_SIZE_PRESETS, - GifExporter, - type GifFrameRate, - type GifSizePreset, - VideoExporter, -} from "@/lib/exporter"; -import { matchesShortcut } from "@/lib/shortcuts"; -import { DEFAULT_WALLPAPER_RELATIVE_PATH, WALLPAPER_PATHS } from "@/lib/wallpapers"; -import { type AspectRatio, getAspectRatioValue } from "@/utils/aspectRatioUtils"; -import { ExportDialog } from "./ExportDialog"; -import PlaybackControls from "./PlaybackControls"; -import { - createProjectData, - deriveNextId, - fromFileUrl, - normalizeProjectEditor, - toFileUrl, - validateProjectData, -} from "./projectPersistence"; -import { SettingsPanel } from "./SettingsPanel"; -import TimelineEditor from "./timeline/TimelineEditor"; -import { - detectInteractionCandidates, - normalizeCursorTelemetry, -} from "./timeline/zoomSuggestionUtils"; -import { - type AnnotationRegion, - type CropRegion, - type CursorTelemetryPoint, - clampFocusToDepth, - DEFAULT_ANNOTATION_POSITION, - DEFAULT_ANNOTATION_SIZE, - DEFAULT_ANNOTATION_STYLE, - DEFAULT_CROP_REGION, - DEFAULT_CURSOR_CLICK_BOUNCE, - DEFAULT_CURSOR_MOTION_BLUR, - DEFAULT_CURSOR_SIZE, - DEFAULT_CURSOR_SMOOTHING, - DEFAULT_FIGURE_DATA, - DEFAULT_PLAYBACK_SPEED, - DEFAULT_ZOOM_DEPTH, - DEFAULT_ZOOM_MOTION_BLUR, - type FigureData, - type PlaybackSpeed, - type SpeedRegion, - type TrimRegion, - type ZoomDepth, - type ZoomFocus, - type ZoomRegion, -} from "./types"; + import VideoPlayback, { VideoPlaybackRef } from "./VideoPlayback"; +import PlaybackControls from "./PlaybackControls"; +import TimelineEditor from "./timeline/TimelineEditor"; +import { SettingsPanel } from "./SettingsPanel"; +import { ExportDialog } from "./ExportDialog"; +import { DEFAULT_WALLPAPER_RELATIVE_PATH, WALLPAPER_PATHS } from "@/lib/wallpapers"; import { - buildLoopedCursorTelemetry, - getDisplayedTimelineWindowMs, -} from "./videoPlayback/cursorLoopTelemetry"; + createProjectData, + deriveNextId, + fromFileUrl, + normalizeProjectEditor, + toFileUrl, + validateProjectData, +} from "./projectPersistence"; + +import { + DEFAULT_CURSOR_CLICK_BOUNCE, + DEFAULT_CURSOR_MOTION_BLUR, + DEFAULT_CURSOR_SIZE, + DEFAULT_CURSOR_SMOOTHING, + DEFAULT_ZOOM_DEPTH, + DEFAULT_ZOOM_MOTION_BLUR, + clampFocusToDepth, + DEFAULT_CROP_REGION, + DEFAULT_ANNOTATION_POSITION, + DEFAULT_ANNOTATION_SIZE, + DEFAULT_ANNOTATION_STYLE, + DEFAULT_FIGURE_DATA, + DEFAULT_PLAYBACK_SPEED, + type ZoomDepth, + type ZoomFocus, + type ZoomRegion, + type CursorTelemetryPoint, + type TrimRegion, + type AnnotationRegion, + type CropRegion, + type FigureData, + type SpeedRegion, + type AudioRegion, + type PlaybackSpeed, +} from "./types"; +import { VideoExporter, GifExporter, type ExportProgress, type ExportQuality, type ExportSettings, type ExportFormat, type GifFrameRate, type GifSizePreset, GIF_SIZE_PRESETS, calculateOutputDimensions } from "@/lib/exporter"; +import { type AspectRatio, getAspectRatioValue } from "@/utils/aspectRatioUtils"; +import { getAssetPath } from "@/lib/assetPath"; +import { matchesShortcut } from "@/lib/shortcuts"; +import { detectInteractionCandidates, normalizeCursorTelemetry } from "./timeline/zoomSuggestionUtils"; +import { buildLoopedCursorTelemetry, getDisplayedTimelineWindowMs } from "./videoPlayback/cursorLoopTelemetry"; import { findDominantRegion } from "./videoPlayback/zoomRegionUtils"; const LOOP_CURSOR_END_WINDOW_MS = 670; type EditorHistorySnapshot = { - zoomRegions: ZoomRegion[]; - trimRegions: TrimRegion[]; - speedRegions: SpeedRegion[]; - annotationRegions: AnnotationRegion[]; - selectedZoomId: string | null; - selectedTrimId: string | null; - selectedSpeedId: string | null; - selectedAnnotationId: string | null; + zoomRegions: ZoomRegion[]; + trimRegions: TrimRegion[]; + speedRegions: SpeedRegion[]; + annotationRegions: AnnotationRegion[]; + audioRegions: AudioRegion[]; + selectedZoomId: string | null; + selectedTrimId: string | null; + selectedSpeedId: string | null; + selectedAnnotationId: string | null; + selectedAudioId: string | null; }; type PendingExportSave = { @@ -142,6 +130,8 @@ export default function VideoEditor() { const [selectedSpeedId, setSelectedSpeedId] = useState(null); const [annotationRegions, setAnnotationRegions] = useState([]); const [selectedAnnotationId, setSelectedAnnotationId] = useState(null); + const [audioRegions, setAudioRegions] = useState([]); + const [selectedAudioId, setSelectedAudioId] = useState(null); const [isExporting, setIsExporting] = useState(false); const [exportProgress, setExportProgress] = useState(null); const [exportError, setExportError] = useState(null); @@ -160,6 +150,7 @@ export default function VideoEditor() { const nextZoomIdRef = useRef(1); const nextTrimIdRef = useRef(1); const nextSpeedIdRef = useRef(1); + const nextAudioIdRef = useRef(1); const { shortcuts, isMac } = useShortcuts(); const nextAnnotationIdRef = useRef(1); @@ -178,10 +169,12 @@ export default function VideoEditor() { trimRegions: JSON.parse(JSON.stringify(snapshot.trimRegions)), speedRegions: JSON.parse(JSON.stringify(snapshot.speedRegions)), annotationRegions: JSON.parse(JSON.stringify(snapshot.annotationRegions)), + audioRegions: JSON.parse(JSON.stringify(snapshot.audioRegions)), selectedZoomId: snapshot.selectedZoomId, selectedTrimId: snapshot.selectedTrimId, selectedSpeedId: snapshot.selectedSpeedId, selectedAnnotationId: snapshot.selectedAnnotationId, + selectedAudioId: snapshot.selectedAudioId, }; }, []); @@ -191,20 +184,24 @@ export default function VideoEditor() { trimRegions, speedRegions, annotationRegions, + audioRegions, selectedZoomId, selectedTrimId, selectedSpeedId, selectedAnnotationId, + selectedAudioId, }; }, [ zoomRegions, trimRegions, speedRegions, annotationRegions, + audioRegions, selectedZoomId, selectedTrimId, selectedSpeedId, selectedAnnotationId, + selectedAudioId, ]); const applyHistorySnapshot = useCallback((snapshot: EditorHistorySnapshot) => { @@ -214,15 +211,18 @@ export default function VideoEditor() { setTrimRegions(cloned.trimRegions); setSpeedRegions(cloned.speedRegions); setAnnotationRegions(cloned.annotationRegions); + setAudioRegions(cloned.audioRegions); setSelectedZoomId(cloned.selectedZoomId); setSelectedTrimId(cloned.selectedTrimId); setSelectedSpeedId(cloned.selectedSpeedId); setSelectedAnnotationId(cloned.selectedAnnotationId); + setSelectedAudioId(cloned.selectedAudioId); nextZoomIdRef.current = deriveNextId("zoom", cloned.zoomRegions.map((region) => region.id)); nextTrimIdRef.current = deriveNextId("trim", cloned.trimRegions.map((region) => region.id)); nextSpeedIdRef.current = deriveNextId("speed", cloned.speedRegions.map((region) => region.id)); nextAnnotationIdRef.current = deriveNextId("annotation", cloned.annotationRegions.map((region) => region.id)); + nextAudioIdRef.current = deriveNextId("audio", cloned.audioRegions.map((region) => region.id)); nextAnnotationZIndexRef.current = cloned.annotationRegions.reduce((max, region) => Math.max(max, region.zIndex), 0) + 1; }, [cloneSnapshot]); @@ -292,6 +292,7 @@ export default function VideoEditor() { setTrimRegions(normalizedEditor.trimRegions); setSpeedRegions(normalizedEditor.speedRegions); setAnnotationRegions(normalizedEditor.annotationRegions); + setAudioRegions(normalizedEditor.audioRegions); setAspectRatio(normalizedEditor.aspectRatio); setExportQuality(normalizedEditor.exportQuality); setExportFormat(normalizedEditor.exportFormat); @@ -303,10 +304,12 @@ export default function VideoEditor() { setSelectedTrimId(null); setSelectedSpeedId(null); setSelectedAnnotationId(null); + setSelectedAudioId(null); nextZoomIdRef.current = deriveNextId("zoom", normalizedEditor.zoomRegions.map((region) => region.id)); nextTrimIdRef.current = deriveNextId("trim", normalizedEditor.trimRegions.map((region) => region.id)); nextSpeedIdRef.current = deriveNextId("speed", normalizedEditor.speedRegions.map((region) => region.id)); + nextAudioIdRef.current = deriveNextId("audio", normalizedEditor.audioRegions.map((region) => region.id)); nextAnnotationIdRef.current = deriveNextId( "annotation", normalizedEditor.annotationRegions.map((region) => region.id), @@ -343,6 +346,7 @@ export default function VideoEditor() { trimRegions, speedRegions, annotationRegions, + audioRegions, aspectRatio, exportQuality, exportFormat, @@ -371,6 +375,7 @@ export default function VideoEditor() { zoomRegions, trimRegions, speedRegions, + audioRegions, annotationRegions, aspectRatio, exportQuality, @@ -480,6 +485,7 @@ export default function VideoEditor() { trimRegions, speedRegions, annotationRegions, + audioRegions, aspectRatio, exportQuality, exportFormat, @@ -814,7 +820,10 @@ export default function VideoEditor() { const handleSelectZoom = useCallback((id: string | null) => { setSelectedZoomId(id); - if (id) setSelectedTrimId(null); + if (id) { + setSelectedTrimId(null); + setSelectedAudioId(null); + } }, []); const handleSelectTrim = useCallback((id: string | null) => { @@ -822,6 +831,7 @@ export default function VideoEditor() { if (id) { setSelectedZoomId(null); setSelectedAnnotationId(null); + setSelectedAudioId(null); } }, []); @@ -830,6 +840,7 @@ export default function VideoEditor() { if (id) { setSelectedZoomId(null); setSelectedTrimId(null); + setSelectedAudioId(null); } }, []); @@ -952,6 +963,7 @@ export default function VideoEditor() { setSelectedZoomId(null); setSelectedTrimId(null); setSelectedAnnotationId(null); + setSelectedAudioId(null); } }, []); @@ -991,6 +1003,54 @@ export default function VideoEditor() { } }, [selectedSpeedId]); + const handleSelectAudio = useCallback((id: string | null) => { + setSelectedAudioId(id); + if (id) { + setSelectedZoomId(null); + setSelectedTrimId(null); + setSelectedAnnotationId(null); + setSelectedSpeedId(null); + } + }, []); + + const handleAudioAdded = useCallback((span: Span, audioPath: string) => { + const id = `audio-${nextAudioIdRef.current++}`; + const newRegion: AudioRegion = { + id, + startMs: Math.round(span.start), + endMs: Math.round(span.end), + audioPath, + volume: 1, + }; + setAudioRegions((prev) => [...prev, newRegion]); + setSelectedAudioId(id); + setSelectedZoomId(null); + setSelectedTrimId(null); + setSelectedAnnotationId(null); + setSelectedSpeedId(null); + }, []); + + const handleAudioSpanChange = useCallback((id: string, span: Span) => { + setAudioRegions((prev) => + prev.map((region) => + region.id === id + ? { + ...region, + startMs: Math.round(span.start), + endMs: Math.round(span.end), + } + : region, + ), + ); + }, []); + + const handleAudioDelete = useCallback((id: string) => { + setAudioRegions((prev) => prev.filter((region) => region.id !== id)); + if (selectedAudioId === id) { + setSelectedAudioId(null); + } + }, [selectedAudioId]); + const handleSpeedChange = useCallback((speed: PlaybackSpeed) => { if (!selectedSpeedId) return; setSpeedRegions((prev) => @@ -1210,6 +1270,78 @@ export default function VideoEditor() { } }, [selectedSpeedId, speedRegions]); + useEffect(() => { + if (selectedAudioId && !audioRegions.some((region) => region.id === selectedAudioId)) { + setSelectedAudioId(null); + } + }, [selectedAudioId, audioRegions]); + + // Audio playback sync: manage Audio elements that play in sync with video + const audioElementsRef = useRef>(new Map()); + + useEffect(() => { + const existing = audioElementsRef.current; + const currentIds = new Set(audioRegions.map(r => r.id)); + + // Remove old audio elements + for (const [id, audio] of existing) { + if (!currentIds.has(id)) { + audio.pause(); + audio.src = ''; + existing.delete(id); + } + } + + // Create/update audio elements + for (const region of audioRegions) { + let audio = existing.get(region.id); + if (!audio) { + audio = new Audio(); + audio.preload = 'auto'; + existing.set(region.id, audio); + } + const expectedSrc = toFileUrl(region.audioPath); + if (audio.src !== expectedSrc) { + audio.src = expectedSrc; + } + audio.volume = Math.max(0, Math.min(1, region.volume)); + } + + return () => { + for (const audio of existing.values()) { + audio.pause(); + audio.src = ''; + } + existing.clear(); + }; + }, [audioRegions]); + + // Sync audio playback with video currentTime and isPlaying state + useEffect(() => { + for (const region of audioRegions) { + const audio = audioElementsRef.current.get(region.id); + if (!audio) continue; + + const currentTimeMs = currentTime * 1000; + const isInRegion = currentTimeMs >= region.startMs && currentTimeMs < region.endMs; + + if (isPlaying && isInRegion) { + const audioOffset = (currentTimeMs - region.startMs) / 1000; + // Only seek if significantly out of sync (> 200ms) + if (Math.abs(audio.currentTime - audioOffset) > 0.2) { + audio.currentTime = audioOffset; + } + if (audio.paused) { + audio.play().catch(() => {}); + } + } else { + if (!audio.paused) { + audio.pause(); + } + } + } + }, [isPlaying, currentTime, audioRegions]); + const showExportSuccessToast = useCallback((filePath: string) => { toast.success(`Exported successfully to ${filePath}`, { action: { @@ -1393,12 +1525,21 @@ export default function VideoEditor() { bitrate = 80_000_000; } } else { - // Use quality-based target resolution - const targetHeight = quality === 'medium' ? 720 : 1080; + // Use source-relative quality scaling. + // "source" is handled above; this branch maps the remaining tiers. + const qualityScale = + quality === 'medium' ? 0.6 : quality === 'good' ? 0.75 : 0.9; + const maxWidth = Math.max(2, Math.floor((sourceWidth * qualityScale) / 2) * 2); + const maxHeight = Math.max(2, Math.floor((sourceHeight * qualityScale) / 2) * 2); + const maxAspect = maxWidth / maxHeight; - // Calculate dimensions maintaining aspect ratio - exportHeight = Math.floor(targetHeight / 2) * 2; - exportWidth = Math.floor((exportHeight * aspectRatioValue) / 2) * 2; + if (aspectRatioValue >= maxAspect) { + exportWidth = maxWidth; + exportHeight = Math.max(2, Math.floor((exportWidth / aspectRatioValue) / 2) * 2); + } else { + exportHeight = maxHeight; + exportWidth = Math.max(2, Math.floor((exportHeight * aspectRatioValue) / 2) * 2); + } // Adjust bitrate for lower resolutions const totalPixels = exportWidth * exportHeight; @@ -1437,6 +1578,7 @@ export default function VideoEditor() { cursorSmoothing, cursorMotionBlur, cursorClickBounce, + audioRegions, previewWidth, previewHeight, onProgress: (progress: ExportProgress) => { @@ -1746,6 +1888,12 @@ export default function VideoEditor() { onSpeedDelete={handleSpeedDelete} selectedSpeedId={selectedSpeedId} onSelectSpeed={handleSelectSpeed} + audioRegions={audioRegions} + onAudioAdded={handleAudioAdded} + onAudioSpanChange={handleAudioSpanChange} + onAudioDelete={handleAudioDelete} + selectedAudioId={selectedAudioId} + onSelectAudio={handleSelectAudio} annotationRegions={annotationRegions} onAnnotationAdded={handleAnnotationAdded} onAnnotationSpanChange={handleAnnotationSpanChange} diff --git a/src/components/video-editor/audio.test.ts b/src/components/video-editor/audio.test.ts new file mode 100644 index 00000000..c4578df8 --- /dev/null +++ b/src/components/video-editor/audio.test.ts @@ -0,0 +1,208 @@ +import { describe, it, expect } from "vitest"; +import * as fc from "fast-check"; +import { toFileUrl, fromFileUrl, normalizeProjectEditor } from "./projectPersistence"; + +describe("Audio path handling", () => { + describe("toFileUrl produces valid file:// URLs for audio paths", () => { + it("should handle Unix absolute paths", () => { + expect(toFileUrl("/Users/music/song.mp3")).toBe("file:///Users/music/song.mp3"); + }); + + it("should handle Windows drive paths", () => { + expect(toFileUrl("C:/Users/music/song.mp3")).toBe("file:///C:/Users/music/song.mp3"); + }); + + it("should handle backslash Windows paths", () => { + const result = toFileUrl("C:\\Users\\music\\song.mp3"); + expect(result).toMatch(/^file:\/\//); + expect(result).toContain("C:"); + expect(result).toContain("song.mp3"); + }); + + it("should encode spaces in path segments", () => { + const result = toFileUrl("/Users/my music/my song.mp3"); + expect(result).toContain("my%20music"); + expect(result).toContain("my%20song.mp3"); + }); + + it("should encode special characters like spaces", () => { + const result = toFileUrl("/Users/music/song file.mp3"); + expect(result).toContain("song%20file.mp3"); + // Result should be a valid file:// URL + expect(result).toMatch(/^file:\/\//); + }); + + it("should roundtrip through fromFileUrl for simple paths", () => { + const paths = [ + "/Users/music/song.mp3", + "/tmp/audio.wav", + "/home/user/my-file.aac", + "/data/recordings/track_01.flac", + ]; + for (const originalPath of paths) { + const fileUrl = toFileUrl(originalPath); + const recovered = fromFileUrl(fileUrl); + expect(recovered).toBe(originalPath); + } + }); + + it("should roundtrip paths with spaces through fromFileUrl", () => { + const originalPath = "/Users/my user/my music/song file.mp3"; + const fileUrl = toFileUrl(originalPath); + const recovered = fromFileUrl(fileUrl); + expect(recovered).toBe(originalPath); + }); + }); +}); + +describe("Audio region normalization", () => { + describe("volume is clamped to [0, 1]", () => { + it("should clamp volume > 1 down to 1", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 0, endMs: 1000, audioPath: "/test.mp3", volume: 5 }, + ], + } as any); + expect(result.audioRegions[0].volume).toBe(1); + }); + + it("should clamp negative volume to 0", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 0, endMs: 1000, audioPath: "/test.mp3", volume: -0.5 }, + ], + } as any); + expect(result.audioRegions[0].volume).toBe(0); + }); + + it("should preserve valid volume values in [0, 1]", () => { + fc.assert( + fc.property( + fc.double({ min: 0, max: 1, noNaN: true }), + (volume) => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 0, endMs: 1000, audioPath: "/test.mp3", volume }, + ], + } as any); + expect(result.audioRegions[0].volume).toBeCloseTo(volume, 10); + }, + ), + ); + }); + + it("should default to 1 when volume is NaN", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 0, endMs: 1000, audioPath: "/test.mp3", volume: NaN }, + ], + } as any); + expect(result.audioRegions[0].volume).toBe(1); + }); + + it("should default to 1 when volume is undefined", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 0, endMs: 1000, audioPath: "/test.mp3" }, + ], + } as any); + expect(result.audioRegions[0].volume).toBe(1); + }); + }); + + describe("startMs and endMs boundaries", () => { + it("should clamp negative startMs to 0", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: -500, endMs: 1000, audioPath: "/test.mp3", volume: 1 }, + ], + } as any); + expect(result.audioRegions[0].startMs).toBe(0); + }); + + it("should ensure endMs > startMs when endMs < startMs", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 1000, endMs: 500, audioPath: "/test.mp3", volume: 1 }, + ], + } as any); + expect(result.audioRegions[0].endMs).toBeGreaterThan(result.audioRegions[0].startMs); + }); + + it("should handle equal startMs and endMs by ensuring minimum gap", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 1000, endMs: 1000, audioPath: "/test.mp3", volume: 1 }, + ], + } as any); + expect(result.audioRegions[0].endMs).toBeGreaterThan(result.audioRegions[0].startMs); + }); + + it("should preserve valid startMs/endMs for arbitrary non-negative values", () => { + fc.assert( + fc.property( + fc.nat({ max: 100000 }), + fc.integer({ min: 1, max: 100000 }), + (startMs, duration) => { + const endMs = startMs + duration; + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs, endMs, audioPath: "/test.mp3", volume: 0.5 }, + ], + } as any); + expect(result.audioRegions[0].startMs).toBe(startMs); + expect(result.audioRegions[0].endMs).toBe(endMs); + }, + ), + ); + }); + }); + + describe("audioPath normalization", () => { + it("should preserve a valid string path", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 0, endMs: 1000, audioPath: "/Users/music/song.mp3", volume: 1 }, + ], + } as any); + expect(result.audioRegions[0].audioPath).toBe("/Users/music/song.mp3"); + }); + + it("should default to empty string for missing audioPath", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { id: "audio-1", startMs: 0, endMs: 1000, volume: 1 }, + ], + } as any); + expect(result.audioRegions[0].audioPath).toBe(""); + }); + + it("should filter out regions without a valid id", () => { + const result = normalizeProjectEditor({ + audioRegions: [ + { startMs: 0, endMs: 1000, audioPath: "/test.mp3", volume: 1 }, + { id: "audio-1", startMs: 0, endMs: 1000, audioPath: "/test.mp3", volume: 1 }, + ], + } as any); + expect(result.audioRegions).toHaveLength(1); + expect(result.audioRegions[0].id).toBe("audio-1"); + }); + }); + + describe("empty or missing audioRegions", () => { + it("should return empty array when audioRegions is undefined", () => { + const result = normalizeProjectEditor({} as any); + expect(result.audioRegions).toEqual([]); + }); + + it("should return empty array when audioRegions is not an array", () => { + const result = normalizeProjectEditor({ audioRegions: "invalid" } as any); + expect(result.audioRegions).toEqual([]); + }); + + it("should return empty array when audioRegions is null", () => { + const result = normalizeProjectEditor({ audioRegions: null } as any); + expect(result.audioRegions).toEqual([]); + }); + }); +}); diff --git a/src/components/video-editor/projectPersistence.ts b/src/components/video-editor/projectPersistence.ts index be25b8fd..af4e1aff 100644 --- a/src/components/video-editor/projectPersistence.ts +++ b/src/components/video-editor/projectPersistence.ts @@ -18,6 +18,7 @@ import { type CropRegion, type SpeedRegion, type TrimRegion, + type AudioRegion, type ZoomRegion, } from "./types"; @@ -42,6 +43,7 @@ export interface ProjectEditorState { trimRegions: TrimRegion[]; speedRegions: SpeedRegion[]; annotationRegions: AnnotationRegion[]; + audioRegions: AudioRegion[]; aspectRatio: AspectRatio; exportQuality: ExportQuality; exportFormat: ExportFormat; @@ -289,6 +291,25 @@ export function normalizeProjectEditor(editor: Partial): Pro }) : []; + const normalizedAudioRegions: AudioRegion[] = Array.isArray((editor as Partial).audioRegions) + ? ((editor as Partial).audioRegions as AudioRegion[]) + .filter((region): region is AudioRegion => Boolean(region && typeof region.id === "string")) + .map((region) => { + const rawStart = isFiniteNumber(region.startMs) ? Math.round(region.startMs) : 0; + const rawEnd = isFiniteNumber(region.endMs) ? Math.round(region.endMs) : rawStart + 1000; + const startMs = Math.max(0, Math.min(rawStart, rawEnd)); + const endMs = Math.max(startMs + 1, rawEnd); + + return { + id: region.id, + startMs, + endMs, + audioPath: typeof region.audioPath === "string" ? region.audioPath : "", + volume: isFiniteNumber(region.volume) ? clamp(region.volume, 0, 1) : 1, + }; + }) + : []; + const rawCropX = isFiniteNumber(editor.cropRegion?.x) ? editor.cropRegion.x : DEFAULT_CROP_REGION.x; const rawCropY = isFiniteNumber(editor.cropRegion?.y) ? editor.cropRegion.y : DEFAULT_CROP_REGION.y; const rawCropWidth = isFiniteNumber(editor.cropRegion?.width) ? editor.cropRegion.width : DEFAULT_CROP_REGION.width; @@ -331,12 +352,19 @@ export function normalizeProjectEditor(editor: Partial): Pro trimRegions: normalizedTrimRegions, speedRegions: normalizedSpeedRegions, annotationRegions: normalizedAnnotationRegions, + audioRegions: normalizedAudioRegions, aspectRatio: typeof editor.aspectRatio === "string" && (validAspectRatios.has(editor.aspectRatio as AspectRatio) || isCustomAspectRatio(editor.aspectRatio)) ? (editor.aspectRatio as AspectRatio) : "16:9", - exportQuality: editor.exportQuality === "medium" || editor.exportQuality === "source" ? editor.exportQuality : "good", + exportQuality: + editor.exportQuality === "medium" || + editor.exportQuality === "good" || + editor.exportQuality === "high" || + editor.exportQuality === "source" + ? editor.exportQuality + : "good", exportFormat: editor.exportFormat === "gif" ? "gif" : "mp4", gifFrameRate: editor.gifFrameRate === 15 || diff --git a/src/components/video-editor/timeline/Item.tsx b/src/components/video-editor/timeline/Item.tsx index 2870fc92..50670f4c 100644 --- a/src/components/video-editor/timeline/Item.tsx +++ b/src/components/video-editor/timeline/Item.tsx @@ -1,21 +1,20 @@ import type { Span } from "dnd-timeline"; import { useItem } from "dnd-timeline"; -import { Gauge, MessageSquare, Scissors, ZoomIn } from "lucide-react"; +import { Gauge, MessageSquare, Music, Scissors, ZoomIn } from "lucide-react"; import { useMemo } from "react"; -import { useScopedT } from "@/contexts/I18nContext"; import { cn } from "@/lib/utils"; import glassStyles from "./ItemGlass.module.css"; interface ItemProps { - id: string; - span: Span; - rowId: string; - children: React.ReactNode; - isSelected?: boolean; - onSelect?: () => void; - zoomDepth?: number; - speedValue?: number; - variant?: "zoom" | "trim" | "annotation" | "speed"; + id: string; + span: Span; + rowId: string; + children: React.ReactNode; + isSelected?: boolean; + onSelect?: () => void; + zoomDepth?: number; + speedValue?: number; + variant?: 'zoom' | 'trim' | 'annotation' | 'speed' | 'audio'; } // Map zoom depth to multiplier labels @@ -49,26 +48,36 @@ export default function Item({ variant = "zoom", children, }: ItemProps) { - const t = useScopedT("timeline"); const { setNodeRef, attributes, listeners, itemStyle, itemContentStyle } = useItem({ id, span, data: { rowId }, }); - const isZoom = variant === "zoom"; - const isTrim = variant === "trim"; - const isSpeed = variant === "speed"; + const isZoom = variant === 'zoom'; + const isTrim = variant === 'trim'; + const isSpeed = variant === 'speed'; + const isAudio = variant === 'audio'; - const glassClass = isZoom - ? glassStyles.glassGreen - : isTrim - ? glassStyles.glassRed - : isSpeed - ? glassStyles.glassAmber - : glassStyles.glassYellow; + const glassClass = isZoom + ? glassStyles.glassGreen + : isTrim + ? glassStyles.glassRed + : isSpeed + ? glassStyles.glassAmber + : isAudio + ? glassStyles.glassPurple + : glassStyles.glassYellow; - const endCapColor = isZoom ? "#2563EB" : isTrim ? "#ef4444" : isSpeed ? "#d97706" : "#B4A046"; + const endCapColor = isZoom + ? '#2563EB' + : isTrim + ? '#ef4444' + : isSpeed + ? '#d97706' + : isAudio + ? '#a855f7' + : '#B4A046'; const timeLabel = useMemo( () => `${formatMs(span.start)} – ${formatMs(span.end)}`, @@ -78,93 +87,88 @@ export default function Item({ const MIN_ITEM_PX = 6; const safeItemStyle = { ...itemStyle, minWidth: MIN_ITEM_PX }; - return ( -
onSelect?.()} - className="group" - > -
-
{ - event.stopPropagation(); - onSelect?.(); - }} - > -
-
- {/* Content */} -
-
- {isZoom ? ( - <> - - - {ZOOM_LABELS[zoomDepth] || `${zoomDepth}×`} - - - ) : isTrim ? ( - <> - - - {t("trim.label", undefined, { index: "" }).trim()} - - - ) : isSpeed ? ( - <> - - - {speedValue !== undefined ? `${speedValue}×` : t("speed.label")} - - - ) : ( - <> - - - {children} - - - )} -
- - {timeLabel} - -
-
-
-
- ); + return ( +
onSelect?.()} + className="group" + > +
+
{ + event.stopPropagation(); + onSelect?.(); + }} + > +
+
+ {/* Content */} +
+
+ {isZoom ? ( + <> + + + {ZOOM_LABELS[zoomDepth] || `${zoomDepth}×`} + + + ) : isTrim ? ( + <> + + + Trim + + + ) : isSpeed ? ( + <> + + + {speedValue !== undefined ? `${speedValue}×` : 'Speed'} + + + ) : isAudio ? ( + <> + + + {children} + + + ) : ( + <> + + + {children} + + + )} +
+ + {timeLabel} + +
+
+
+
+ ); } diff --git a/src/components/video-editor/timeline/ItemGlass.module.css b/src/components/video-editor/timeline/ItemGlass.module.css index 23754429..9c96ea22 100644 --- a/src/components/video-editor/timeline/ItemGlass.module.css +++ b/src/components/video-editor/timeline/ItemGlass.module.css @@ -102,6 +102,32 @@ z-index: 10; } +.glassPurple { + position: relative; + border-radius: 8px; + -corner-smoothing: antialiased; + background: rgba(168, 85, 247, 0.15); + border: 1px solid rgba(168, 85, 247, 0.3); + box-shadow: 0 2px 12px 0 rgba(168, 85, 247, 0.1) inset; + margin: 2px 0; + backdrop-filter: blur(4px); + -webkit-backdrop-filter: blur(4px); + transition: all 0.2s cubic-bezier(0.4, 0, 0.2, 1); +} + +.glassPurple:hover { + background: rgba(168, 85, 247, 0.25); + border-color: rgba(168, 85, 247, 0.5); + box-shadow: 0 4px 20px 0 rgba(168, 85, 247, 0.2) inset; +} + +.glassPurple.selected { + background: rgba(168, 85, 247, 0.35); + border-color: #a855f7; + box-shadow: 0 0 0 1px #a855f7, 0 4px 20px 0 rgba(168, 85, 247, 0.3) inset; + z-index: 10; +} + .zoomEndCap { position: absolute; top: 0; @@ -120,7 +146,9 @@ .glassYellow:hover .zoomEndCap, .glassYellow.selected .zoomEndCap, .glassAmber:hover .zoomEndCap, -.glassAmber.selected .zoomEndCap { +.glassAmber.selected .zoomEndCap, +.glassPurple:hover .zoomEndCap, +.glassPurple.selected .zoomEndCap { opacity: 1; } diff --git a/src/components/video-editor/timeline/TimelineEditor.tsx b/src/components/video-editor/timeline/TimelineEditor.tsx index 13ce0719..27d02dce 100644 --- a/src/components/video-editor/timeline/TimelineEditor.tsx +++ b/src/components/video-editor/timeline/TimelineEditor.tsx @@ -1,18 +1,5 @@ -import type { Range, Span } from "dnd-timeline"; -import { useTimelineContext } from "dnd-timeline"; -import { - Check, - ChevronDown, - Gauge, - MessageSquare, - Plus, - Scissors, - WandSparkles, - ZoomIn, -} from "lucide-react"; import { useCallback, useEffect, useMemo, useRef, useState, type KeyboardEvent as ReactKeyboardEvent, type WheelEvent } from "react"; -import { toast } from "sonner"; -import { v4 as uuidv4 } from "uuid"; +import { useTimelineContext } from "dnd-timeline"; import { Button } from "@/components/ui/button"; import { DropdownMenu, @@ -20,66 +7,71 @@ import { DropdownMenuItem, DropdownMenuTrigger, } from "@/components/ui/dropdown-menu"; +import { Plus, Scissors, ZoomIn, MessageSquare, ChevronDown, Check, Gauge, WandSparkles, Music } from "lucide-react"; +import { toast } from "sonner"; +import { cn } from "@/lib/utils"; +import { v4 as uuidv4 } from 'uuid'; import { useShortcuts } from "@/contexts/ShortcutsContext"; import { matchesShortcut } from "@/lib/shortcuts"; -import { cn } from "@/lib/utils"; import { ASPECT_RATIOS, type AspectRatio, getAspectRatioLabel, isCustomAspectRatio } from "@/utils/aspectRatioUtils"; import { formatShortcut } from "@/utils/platformUtils"; import { TutorialHelp } from "../TutorialHelp"; -import type { - AnnotationRegion, - CursorTelemetryPoint, - SpeedRegion, - TrimRegion, - ZoomFocus, - ZoomRegion, -} from "../types"; +import TimelineWrapper from "./TimelineWrapper"; +import Row from "./Row"; import Item from "./Item"; import KeyframeMarkers from "./KeyframeMarkers"; -import Row from "./Row"; -import TimelineWrapper from "./TimelineWrapper"; +import type { Range, Span } from "dnd-timeline"; +import type { ZoomRegion, TrimRegion, AnnotationRegion, SpeedRegion, AudioRegion, CursorTelemetryPoint, ZoomFocus } from "../types"; +import { toFileUrl } from "../projectPersistence"; import { detectInteractionCandidates, normalizeCursorTelemetry } from "./zoomSuggestionUtils"; const ZOOM_ROW_ID = "row-zoom"; const TRIM_ROW_ID = "row-trim"; const ANNOTATION_ROW_ID = "row-annotation"; const SPEED_ROW_ID = "row-speed"; +const AUDIO_ROW_ID = "row-audio"; const FALLBACK_RANGE_MS = 1000; const TARGET_MARKER_COUNT = 12; const SUGGESTION_SPACING_MS = 1800; interface TimelineEditorProps { - videoDuration: number; - currentTime: number; - onSeek?: (time: number) => void; - cursorTelemetry?: CursorTelemetryPoint[]; - zoomRegions: ZoomRegion[]; - onZoomAdded: (span: Span) => void; - onZoomSuggested?: (span: Span, focus: ZoomFocus) => void; - onZoomSpanChange: (id: string, span: Span) => void; - onZoomDelete: (id: string) => void; - selectedZoomId: string | null; - onSelectZoom: (id: string | null) => void; - trimRegions?: TrimRegion[]; - onTrimAdded?: (span: Span) => void; - onTrimSpanChange?: (id: string, span: Span) => void; - onTrimDelete?: (id: string) => void; - selectedTrimId?: string | null; - onSelectTrim?: (id: string | null) => void; - annotationRegions?: AnnotationRegion[]; - onAnnotationAdded?: (span: Span) => void; - onAnnotationSpanChange?: (id: string, span: Span) => void; - onAnnotationDelete?: (id: string) => void; - selectedAnnotationId?: string | null; - onSelectAnnotation?: (id: string | null) => void; - speedRegions?: SpeedRegion[]; - onSpeedAdded?: (span: Span) => void; - onSpeedSpanChange?: (id: string, span: Span) => void; - onSpeedDelete?: (id: string) => void; - selectedSpeedId?: string | null; - onSelectSpeed?: (id: string | null) => void; - aspectRatio: AspectRatio; - onAspectRatioChange: (aspectRatio: AspectRatio) => void; + videoDuration: number; + currentTime: number; + onSeek?: (time: number) => void; + cursorTelemetry?: CursorTelemetryPoint[]; + zoomRegions: ZoomRegion[]; + onZoomAdded: (span: Span) => void; + onZoomSuggested?: (span: Span, focus: ZoomFocus) => void; + onZoomSpanChange: (id: string, span: Span) => void; + onZoomDelete: (id: string) => void; + selectedZoomId: string | null; + onSelectZoom: (id: string | null) => void; + trimRegions?: TrimRegion[]; + onTrimAdded?: (span: Span) => void; + onTrimSpanChange?: (id: string, span: Span) => void; + onTrimDelete?: (id: string) => void; + selectedTrimId?: string | null; + onSelectTrim?: (id: string | null) => void; + annotationRegions?: AnnotationRegion[]; + onAnnotationAdded?: (span: Span) => void; + onAnnotationSpanChange?: (id: string, span: Span) => void; + onAnnotationDelete?: (id: string) => void; + selectedAnnotationId?: string | null; + onSelectAnnotation?: (id: string | null) => void; + speedRegions?: SpeedRegion[]; + onSpeedAdded?: (span: Span) => void; + onSpeedSpanChange?: (id: string, span: Span) => void; + onSpeedDelete?: (id: string) => void; + selectedSpeedId?: string | null; + onSelectSpeed?: (id: string | null) => void; + audioRegions?: AudioRegion[]; + onAudioAdded?: (span: Span, audioPath: string) => void; + onAudioSpanChange?: (id: string, span: Span) => void; + onAudioDelete?: (id: string) => void; + selectedAudioId?: string | null; + onSelectAudio?: (id: string | null) => void; + aspectRatio: AspectRatio; + onAspectRatioChange: (aspectRatio: AspectRatio) => void; } interface TimelineScaleConfig { @@ -89,13 +81,13 @@ interface TimelineScaleConfig { } interface TimelineRenderItem { - id: string; - rowId: string; - span: Span; - label: string; - zoomDepth?: number; - speedValue?: number; - variant: "zoom" | "trim" | "annotation" | "speed"; + id: string; + rowId: string; + span: Span; + label: string; + zoomDepth?: number; + speedValue?: number; + variant: 'zoom' | 'trim' | 'annotation' | 'speed' | 'audio'; } const SCALE_CANDIDATES = [ @@ -443,33 +435,37 @@ function TimelineAxis({ } function Timeline({ - items, - videoDurationMs, - currentTimeMs, - onSeek, - onSelectZoom, - onSelectTrim, - onSelectAnnotation, - onSelectSpeed, - selectedZoomId, - selectedTrimId, - selectedAnnotationId, - selectedSpeedId, - keyframes = [], + items, + videoDurationMs, + currentTimeMs, + onSeek, + onSelectZoom, + onSelectTrim, + onSelectAnnotation, + onSelectSpeed, + onSelectAudio, + selectedZoomId, + selectedTrimId, + selectedAnnotationId, + selectedSpeedId, + selectedAudioId, + keyframes = [], }: { - items: TimelineRenderItem[]; - videoDurationMs: number; - currentTimeMs: number; - onSeek?: (time: number) => void; - onSelectZoom?: (id: string | null) => void; - onSelectTrim?: (id: string | null) => void; - onSelectAnnotation?: (id: string | null) => void; - onSelectSpeed?: (id: string | null) => void; - selectedZoomId: string | null; - selectedTrimId?: string | null; - selectedAnnotationId?: string | null; - selectedSpeedId?: string | null; - keyframes?: { id: string; time: number }[]; + items: TimelineRenderItem[]; + videoDurationMs: number; + currentTimeMs: number; + onSeek?: (time: number) => void; + onSelectZoom?: (id: string | null) => void; + onSelectTrim?: (id: string | null) => void; + onSelectAnnotation?: (id: string | null) => void; + onSelectSpeed?: (id: string | null) => void; + onSelectAudio?: (id: string | null) => void; + selectedZoomId: string | null; + selectedTrimId?: string | null; + selectedAnnotationId?: string | null; + selectedSpeedId?: string | null; + selectedAudioId?: string | null; + keyframes?: { id: string; time: number }[]; }) { const { setTimelineRef, style, sidebarWidth, range, pixelsToValue } = useTimelineContext(); const localTimelineRef = useRef(null); @@ -486,12 +482,13 @@ function Timeline({ (e: React.MouseEvent) => { if (!onSeek || videoDurationMs <= 0) return; - // Only clear selection if clicking on empty space (not on items) - // This is handled by event propagation - items stop propagation - onSelectZoom?.(null); - onSelectTrim?.(null); - onSelectAnnotation?.(null); - onSelectSpeed?.(null); + // Only clear selection if clicking on empty space (not on items) + // This is handled by event propagation - items stop propagation + onSelectZoom?.(null); + onSelectTrim?.(null); + onSelectAnnotation?.(null); + onSelectSpeed?.(null); + onSelectAudio?.(null); const rect = e.currentTarget.getBoundingClientRect(); const clickX = e.clientX - rect.left - sidebarWidth; @@ -502,25 +499,14 @@ function Timeline({ const absoluteMs = Math.max(0, Math.min(range.start + relativeMs, videoDurationMs)); const timeInSeconds = absoluteMs / 1000; - onSeek(timeInSeconds); - }, - [ - onSeek, - onSelectZoom, - onSelectTrim, - onSelectAnnotation, - onSelectSpeed, - videoDurationMs, - sidebarWidth, - range.start, - pixelsToValue, - ], - ); + onSeek(timeInSeconds); + }, [onSeek, onSelectZoom, onSelectTrim, onSelectAnnotation, onSelectSpeed, onSelectAudio, videoDurationMs, sidebarWidth, range.start, pixelsToValue]); - const zoomItems = items.filter((item) => item.rowId === ZOOM_ROW_ID); - const trimItems = items.filter((item) => item.rowId === TRIM_ROW_ID); - const annotationItems = items.filter((item) => item.rowId === ANNOTATION_ROW_ID); - const speedItems = items.filter((item) => item.rowId === SPEED_ROW_ID); + const zoomItems = items.filter(item => item.rowId === ZOOM_ROW_ID); + const trimItems = items.filter(item => item.rowId === TRIM_ROW_ID); + const annotationItems = items.filter(item => item.rowId === ANNOTATION_ROW_ID); + const speedItems = items.filter(item => item.rowId === SPEED_ROW_ID); + const audioItems = items.filter(item => item.rowId === AUDIO_ROW_ID); return (
- - {speedItems.map((item) => ( - onSelectSpeed?.(item.id)} - variant="speed" - speedValue={item.speedValue} - > - {item.label} - - ))} - -
- ); + + {speedItems.map((item) => ( + onSelectSpeed?.(item.id)} + variant="speed" + speedValue={item.speedValue} + > + {item.label} + + ))} + + + + {audioItems.map((item) => ( + onSelectAudio?.(item.id)} + variant="audio" + > + {item.label} + + ))} + +
+ ); } export default function TimelineEditor({ - videoDuration, - currentTime, - onSeek, - cursorTelemetry = [], - zoomRegions, - onZoomAdded, - onZoomSuggested, - onZoomSpanChange, - onZoomDelete, - selectedZoomId, - onSelectZoom, - trimRegions = [], - onTrimAdded, - onTrimSpanChange, - onTrimDelete, - selectedTrimId, - onSelectTrim, - annotationRegions = [], - onAnnotationAdded, - onAnnotationSpanChange, - onAnnotationDelete, - selectedAnnotationId, - onSelectAnnotation, - speedRegions = [], - onSpeedAdded, - onSpeedSpanChange, - onSpeedDelete, - selectedSpeedId, - onSelectSpeed, - aspectRatio, - onAspectRatioChange, + videoDuration, + currentTime, + onSeek, + cursorTelemetry = [], + zoomRegions, + onZoomAdded, + onZoomSuggested, + onZoomSpanChange, + onZoomDelete, + selectedZoomId, + onSelectZoom, + trimRegions = [], + onTrimAdded, + onTrimSpanChange, + onTrimDelete, + selectedTrimId, + onSelectTrim, + annotationRegions = [], + onAnnotationAdded, + onAnnotationSpanChange, + onAnnotationDelete, + selectedAnnotationId, + onSelectAnnotation, + speedRegions = [], + onSpeedAdded, + onSpeedSpanChange, + onSpeedDelete, + selectedSpeedId, + onSelectSpeed, + audioRegions = [], + onAudioAdded, + onAudioSpanChange, + onAudioDelete, + selectedAudioId, + onSelectAudio, + aspectRatio, + onAspectRatioChange, }: TimelineEditorProps) { const totalMs = useMemo(() => Math.max(0, Math.round(videoDuration * 1000)), [videoDuration]); const currentTimeMs = useMemo(() => Math.round(currentTime * 1000), [currentTime]); @@ -749,6 +757,12 @@ export default function TimelineEditor({ onSelectSpeed(null); }, [selectedSpeedId, onSpeedDelete, onSelectSpeed]); + const deleteSelectedAudio = useCallback(() => { + if (!selectedAudioId || !onAudioDelete || !onSelectAudio) return; + onAudioDelete(selectedAudioId); + onSelectAudio(null); + }, [selectedAudioId, onAudioDelete, onSelectAudio]); + useEffect(() => { setRange(createInitialRange(totalMs)); }, [totalMs]); @@ -759,9 +773,11 @@ export default function TimelineEditor({ const zoomRegionsRef = useRef(zoomRegions); const trimRegionsRef = useRef(trimRegions); const speedRegionsRef = useRef(speedRegions); + const audioRegionsRef = useRef(audioRegions); zoomRegionsRef.current = zoomRegions; trimRegionsRef.current = trimRegions; speedRegionsRef.current = speedRegions; + audioRegionsRef.current = audioRegions; useEffect(() => { if (totalMs === 0 || safeMinDurationMs <= 0) { @@ -803,8 +819,20 @@ export default function TimelineEditor({ onSpeedSpanChange?.(region.id, { start: normalizedStart, end: normalizedEnd }); } }); + + audioRegionsRef.current.forEach((region) => { + const clampedStart = Math.max(0, Math.min(region.startMs, totalMs)); + const minEnd = clampedStart + safeMinDurationMs; + const clampedEnd = Math.min(totalMs, Math.max(minEnd, region.endMs)); + const normalizedStart = Math.max(0, Math.min(clampedStart, totalMs - safeMinDurationMs)); + const normalizedEnd = Math.max(minEnd, Math.min(clampedEnd, totalMs)); + + if (normalizedStart !== region.startMs || normalizedEnd !== region.endMs) { + onAudioSpanChange?.(region.id, { start: normalizedStart, end: normalizedEnd }); + } + }); // Only re-run when the timeline scale changes, not on every region edit - }, [totalMs, safeMinDurationMs, onZoomSpanChange, onTrimSpanChange, onSpeedSpanChange]); + }, [totalMs, safeMinDurationMs, onZoomSpanChange, onTrimSpanChange, onSpeedSpanChange, onAudioSpanChange]); const hasOverlap = useCallback((newSpan: Span, excludeId?: string): boolean => { // Determine which row the item belongs to @@ -812,13 +840,14 @@ export default function TimelineEditor({ const isTrimItem = trimRegions.some(r => r.id === excludeId); const isAnnotationItem = annotationRegions.some(r => r.id === excludeId); const isSpeedItem = speedRegions.some(r => r.id === excludeId); + const isAudioItem = audioRegions.some(r => r.id === excludeId); if (isAnnotationItem) { return false; } // Helper to check overlap against a specific set of regions - const checkOverlap = (regions: (ZoomRegion | TrimRegion | SpeedRegion)[]) => { + const checkOverlap = (regions: (ZoomRegion | TrimRegion | SpeedRegion | AudioRegion)[]) => { return regions.some((region) => { if (region.id === excludeId) return false; // True overlap: regions actually intersect (not just adjacent) @@ -838,8 +867,12 @@ export default function TimelineEditor({ return checkOverlap(speedRegions); } + if (isAudioItem) { + return checkOverlap(audioRegions); + } + return false; - }, [zoomRegions, trimRegions, annotationRegions, speedRegions]); + }, [zoomRegions, trimRegions, annotationRegions, speedRegions, audioRegions]); // Keep newly added timeline regions at the original short default instead of // scaling them with the full recording length. @@ -1023,6 +1056,52 @@ export default function TimelineEditor({ onSpeedAdded({ start: startPos, end: startPos + actualDuration }); }, [videoDuration, totalMs, currentTimeMs, speedRegions, onSpeedAdded, defaultRegionDurationMs]); + const handleAddAudio = useCallback(async () => { + if (!videoDuration || videoDuration === 0 || totalMs === 0 || !onAudioAdded) { + return; + } + + const result = await (window as any).electronAPI.openAudioFilePicker(); + if (!result?.success || !result.path) { + return; + } + + // Load the audio file to get its full duration + const audioDurationMs = await new Promise((resolve) => { + const audio = new Audio(toFileUrl(result.path)); + audio.addEventListener('loadedmetadata', () => { + resolve(Math.round(audio.duration * 1000)); + }); + audio.addEventListener('error', () => { + resolve(0); + }); + }); + + if (audioDurationMs <= 0) { + toast.error("Could not read audio file", { + description: "The selected file may be corrupted or in an unsupported format.", + }); + return; + } + + const startPos = Math.max(0, Math.min(currentTimeMs, totalMs)); + const sorted = [...audioRegions].sort((a, b) => a.startMs - b.startMs); + const nextRegion = sorted.find(region => region.startMs > startPos); + const gapToNext = nextRegion ? nextRegion.startMs - startPos : totalMs - startPos; + + const isOverlapping = sorted.some(region => startPos >= region.startMs && startPos < region.endMs); + if (isOverlapping || gapToNext <= 0) { + toast.error("Cannot place audio here", { + description: "Audio region already exists at this location or not enough space available.", + }); + return; + } + + // Use full audio duration, but clamp to available gap and video length + const actualDuration = Math.min(audioDurationMs, gapToNext, totalMs - startPos); + onAudioAdded({ start: startPos, end: startPos + actualDuration }, result.path); + }, [videoDuration, totalMs, currentTimeMs, audioRegions, onAudioAdded]); + const handleAddAnnotation = useCallback(() => { if (!videoDuration || videoDuration === 0 || totalMs === 0 || !onAnnotationAdded) { return; @@ -1096,12 +1175,14 @@ export default function TimelineEditor({ deleteSelectedAnnotation(); } else if (selectedSpeedId) { deleteSelectedSpeed(); + } else if (selectedAudioId) { + deleteSelectedAudio(); } } }; window.addEventListener('keydown', handleKeyDown); return () => window.removeEventListener('keydown', handleKeyDown); - }, [addKeyframe, handleAddZoom, handleAddTrim, handleAddAnnotation, handleAddSpeed, deleteSelectedKeyframe, deleteSelectedZoom, deleteSelectedTrim, deleteSelectedAnnotation, deleteSelectedSpeed, selectedKeyframeId, selectedZoomId, selectedTrimId, selectedAnnotationId, selectedSpeedId, annotationRegions, currentTime, onSelectAnnotation, keyShortcuts, isMac]); + }, [addKeyframe, handleAddZoom, handleAddTrim, handleAddAnnotation, handleAddSpeed, deleteSelectedKeyframe, deleteSelectedZoom, deleteSelectedTrim, deleteSelectedAnnotation, deleteSelectedSpeed, deleteSelectedAudio, selectedKeyframeId, selectedZoomId, selectedTrimId, selectedAnnotationId, selectedSpeedId, selectedAudioId, annotationRegions, currentTime, onSelectAnnotation, keyShortcuts, isMac]); const clampedRange = useMemo(() => { if (totalMs === 0) { @@ -1163,16 +1244,28 @@ export default function TimelineEditor({ variant: 'speed', })); - return [...zooms, ...trims, ...annotations, ...speeds]; - }, [zoomRegions, trimRegions, annotationRegions, speedRegions]); + const audios: TimelineRenderItem[] = audioRegions.map((region) => { + const fileName = region.audioPath.split(/[\\/]/).pop()?.replace(/\.[^.]+$/, '') || 'Audio'; + return { + id: region.id, + rowId: AUDIO_ROW_ID, + span: { start: region.startMs, end: region.endMs }, + label: fileName, + variant: 'audio', + }; + }); + + return [...zooms, ...trims, ...annotations, ...speeds, ...audios]; + }, [zoomRegions, trimRegions, annotationRegions, speedRegions, audioRegions]); // Flat list of all non-annotation region spans for neighbour-clamping during drag/resize const allRegionSpans = useMemo(() => { const zooms = zoomRegions.map((r) => ({ id: r.id, start: r.startMs, end: r.endMs })); const trims = trimRegions.map((r) => ({ id: r.id, start: r.startMs, end: r.endMs })); const speeds = speedRegions.map((r) => ({ id: r.id, start: r.startMs, end: r.endMs })); - return [...zooms, ...trims, ...speeds]; - }, [zoomRegions, trimRegions, speedRegions]); + const audios = audioRegions.map((r) => ({ id: r.id, start: r.startMs, end: r.endMs })); + return [...zooms, ...trims, ...speeds, ...audios]; + }, [zoomRegions, trimRegions, speedRegions, audioRegions]); const handleItemSpanChange = useCallback((id: string, span: Span) => { // Check if it's a zoom, trim, speed, or annotation item @@ -1184,8 +1277,10 @@ export default function TimelineEditor({ onSpeedSpanChange?.(id, span); } else if (annotationRegions.some(r => r.id === id)) { onAnnotationSpanChange?.(id, span); + } else if (audioRegions.some(r => r.id === id)) { + onAudioSpanChange?.(id, span); } - }, [zoomRegions, trimRegions, speedRegions, annotationRegions, onZoomSpanChange, onTrimSpanChange, onSpeedSpanChange, onAnnotationSpanChange]); + }, [zoomRegions, trimRegions, speedRegions, annotationRegions, audioRegions, onZoomSpanChange, onTrimSpanChange, onSpeedSpanChange, onAnnotationSpanChange, onAudioSpanChange]); const panTimelineRange = useCallback((deltaMs: number) => { if (!Number.isFinite(deltaMs) || deltaMs === 0 || totalMs <= 0) { @@ -1297,6 +1392,15 @@ export default function TimelineEditor({ > +
@@ -1407,10 +1511,12 @@ export default function TimelineEditor({ onSelectTrim={onSelectTrim} onSelectAnnotation={onSelectAnnotation} onSelectSpeed={onSelectSpeed} + onSelectAudio={onSelectAudio} selectedZoomId={selectedZoomId} selectedTrimId={selectedTrimId} selectedAnnotationId={selectedAnnotationId} selectedSpeedId={selectedSpeedId} + selectedAudioId={selectedAudioId} keyframes={keyframes} /> diff --git a/src/components/video-editor/types.ts b/src/components/video-editor/types.ts index fa782ce0..46f66f79 100644 --- a/src/components/video-editor/types.ts +++ b/src/components/video-editor/types.ts @@ -137,6 +137,14 @@ export const DEFAULT_CROP_REGION: CropRegion = { height: 1, }; +export interface AudioRegion { + id: string; + startMs: number; + endMs: number; + audioPath: string; + volume: number; +} + export type PlaybackSpeed = 0.25 | 0.5 | 0.75 | 1.25 | 1.5 | 1.75 | 2; export interface SpeedRegion { diff --git a/src/components/video-editor/videoPlayback/cursorRenderer.ts b/src/components/video-editor/videoPlayback/cursorRenderer.ts index 974a0844..3b8f29d6 100644 --- a/src/components/video-editor/videoPlayback/cursorRenderer.ts +++ b/src/components/video-editor/videoPlayback/cursorRenderer.ts @@ -167,6 +167,7 @@ function getAvailableCursorKeys(): CursorAssetKey[] { export async function preloadCursorAssets() { if (!cursorAssetsPromise) { cursorAssetsPromise = (async () => { + const isLinux = typeof navigator !== 'undefined' && /linux/i.test(navigator.platform); let systemCursors: Record = {}; try { @@ -182,7 +183,9 @@ export async function preloadCursorAssets() { SUPPORTED_CURSOR_KEYS.map(async (key) => { const systemAsset = systemCursors[key]; const uploadedAsset = uploadedCursorAssets[key]; - const assetUrl = uploadedAsset?.url ?? systemAsset?.dataUrl; + const assetUrl = isLinux + ? uploadedAsset?.url + : uploadedAsset?.url ?? systemAsset?.dataUrl; if (!assetUrl) { console.warn(`[CursorRenderer] No cursor image for: ${key}`); diff --git a/src/contexts/I18nContext.tsx b/src/contexts/I18nContext.tsx index 77670801..dda55641 100644 --- a/src/contexts/I18nContext.tsx +++ b/src/contexts/I18nContext.tsx @@ -123,7 +123,9 @@ function getInitialLocale(): AppLocale { return storedLocale } - return normalizeLocale(window.navigator.language) + // Product default must be English on first launch unless user explicitly + // selected another locale and we persisted it in localStorage. + return DEFAULT_LOCALE } function getMessageValue(source: unknown, key: string): string | undefined { diff --git a/src/hooks/useScreenRecorder.ts b/src/hooks/useScreenRecorder.ts index 53ae6720..4765bd40 100644 --- a/src/hooks/useScreenRecorder.ts +++ b/src/hooks/useScreenRecorder.ts @@ -28,6 +28,7 @@ const MIC_GAIN_BOOST = 1.4; type UseScreenRecorderReturn = { recording: boolean; + countdownActive: boolean; toggleRecording: () => void; preparePermissions: (options?: { startup?: boolean }) => Promise; isMacOS: boolean; @@ -37,15 +38,19 @@ type UseScreenRecorderReturn = { setMicrophoneDeviceId: (deviceId: string | undefined) => void; systemAudioEnabled: boolean; setSystemAudioEnabled: (enabled: boolean) => void; + countdownDelay: number; + setCountdownDelay: (delay: number) => void; }; export function useScreenRecorder(): UseScreenRecorderReturn { const [recording, setRecording] = useState(false); const [starting, setStarting] = useState(false); + const [countdownActive, setCountdownActive] = useState(false); const [isMacOS, setIsMacOS] = useState(false); const [microphoneEnabled, setMicrophoneEnabled] = useState(false); const [microphoneDeviceId, setMicrophoneDeviceId] = useState(undefined); const [systemAudioEnabled, setSystemAudioEnabled] = useState(false); + const [countdownDelay, setCountdownDelayState] = useState(3); const mediaRecorder = useRef(null); const stream = useRef(null); const screenStream = useRef(null); @@ -57,6 +62,7 @@ export function useScreenRecorder(): UseScreenRecorderReturn { const wgcRecording = useRef(false); const startInFlight = useRef(false); const hasPromptedForReselect = useRef(false); + const countdownDelayLoaded = useRef(false); const preparePermissions = useCallback(async (options: { startup?: boolean } = {}) => { const platform = await window.electronAPI.getPlatform(); @@ -194,6 +200,23 @@ export function useScreenRecorder(): UseScreenRecorderReturn { })(); }, []); + useEffect(() => { + if (countdownDelayLoaded.current) return; + countdownDelayLoaded.current = true; + + void (async () => { + const result = await window.electronAPI.getCountdownDelay(); + if (result.success && typeof result.delay === "number") { + setCountdownDelayState(result.delay); + } + })(); + }, []); + + const setCountdownDelay = useCallback((delay: number) => { + setCountdownDelayState(delay); + void window.electronAPI.setCountdownDelay(delay); + }, []); + useEffect(() => { let cleanup: (() => void) | undefined; @@ -548,16 +571,35 @@ export function useScreenRecorder(): UseScreenRecorderReturn { } }; - const toggleRecording = () => { - if (starting) { + const toggleRecording = async () => { + if (starting || countdownActive) { return; } - recording ? stopRecording.current() : startRecording(); + if (recording) { + stopRecording.current(); + return; + } + + // Start recording with optional countdown + if (countdownDelay > 0) { + setCountdownActive(true); + try { + const result = await window.electronAPI.startCountdown(countdownDelay); + if (!result.success || result.cancelled) { + return; + } + } finally { + setCountdownActive(false); + } + } + + startRecording(); }; return { recording, + countdownActive, toggleRecording, preparePermissions, isMacOS, @@ -567,6 +609,8 @@ export function useScreenRecorder(): UseScreenRecorderReturn { setMicrophoneDeviceId, systemAudioEnabled, setSystemAudioEnabled, + countdownDelay, + setCountdownDelay, }; } diff --git a/src/i18n/locales/en/launch.json b/src/i18n/locales/en/launch.json index 4829693f..bb709f98 100644 --- a/src/i18n/locales/en/launch.json +++ b/src/i18n/locales/en/launch.json @@ -4,6 +4,8 @@ "enableSystemAudio": "Enable system audio", "disableMicrophone": "Disable microphone", "enableMicrophone": "Enable microphone", + "countdownDelay": "Countdown delay", + "noDelay": "No delay", "record": "Record", "recordingFolder": "Recording folder: {{path}}", "chooseRecordingsFolder": "Choose recordings folder", diff --git a/src/i18n/locales/en/settings.json b/src/i18n/locales/en/settings.json index 46a76248..b0d7121d 100644 --- a/src/i18n/locales/en/settings.json +++ b/src/i18n/locales/en/settings.json @@ -48,7 +48,8 @@ "quality": { "low": "Low", "medium": "Medium", - "high": "High" + "high": "High", + "original": "Original" }, "loop": "Loop", "outputDimensions": "Output: {{dimensions}}px", diff --git a/src/i18n/locales/es/launch.json b/src/i18n/locales/es/launch.json index 0bd6d05e..e8390ad3 100644 --- a/src/i18n/locales/es/launch.json +++ b/src/i18n/locales/es/launch.json @@ -4,6 +4,8 @@ "enableSystemAudio": "Activar audio del sistema", "disableMicrophone": "Desactivar micrófono", "enableMicrophone": "Activar micrófono", + "countdownDelay": "Retraso de cuenta regresiva", + "noDelay": "Sin retraso", "record": "Grabar", "recordingFolder": "Carpeta de grabaciones: {{path}}", "chooseRecordingsFolder": "Elegir carpeta de grabaciones", diff --git a/src/i18n/locales/es/settings.json b/src/i18n/locales/es/settings.json index 1eb012fd..9f35964d 100644 --- a/src/i18n/locales/es/settings.json +++ b/src/i18n/locales/es/settings.json @@ -48,7 +48,8 @@ "quality": { "low": "Baja", "medium": "Media", - "high": "Alta" + "high": "Alta", + "original": "Original" }, "loop": "Bucle", "outputDimensions": "Salida: {{dimensions}}px", diff --git a/src/i18n/locales/zh-CN/launch.json b/src/i18n/locales/zh-CN/launch.json index 9ac43f3a..dd2aca81 100644 --- a/src/i18n/locales/zh-CN/launch.json +++ b/src/i18n/locales/zh-CN/launch.json @@ -4,6 +4,8 @@ "enableSystemAudio": "启用系统音频", "disableMicrophone": "禁用麦克风", "enableMicrophone": "启用麦克风", + "countdownDelay": "倒计时延迟", + "noDelay": "无延迟", "record": "录制", "recordingFolder": "录制文件夹：{{path}}", "chooseRecordingsFolder": "选择录制文件夹", diff --git a/src/i18n/locales/zh-CN/settings.json b/src/i18n/locales/zh-CN/settings.json index 310a8cc3..f3854608 100644 --- a/src/i18n/locales/zh-CN/settings.json +++ b/src/i18n/locales/zh-CN/settings.json @@ -48,7 +48,8 @@ "quality": { "low": "低", "medium": "中", - "high": "高" + "high": "高", + "original": "原始" }, "loop": "循环", "outputDimensions": "输出：{{dimensions}}px", diff --git a/src/lib/exporter/audioEncoder.ts b/src/lib/exporter/audioEncoder.ts index 09407801..bd1c20cd 100644 --- a/src/lib/exporter/audioEncoder.ts +++ b/src/lib/exporter/audioEncoder.ts @@ -1,5 +1,6 @@ import { WebDemuxer } from 'web-demuxer' -import type { SpeedRegion, TrimRegion } from '@/components/video-editor/types' +import type { SpeedRegion, TrimRegion, AudioRegion } from '@/components/video-editor/types' +import { toFileUrl } from '@/components/video-editor/projectPersistence' import type { VideoMuxer } from './muxer' const AUDIO_BITRATE = 128_000 @@ -21,6 +22,7 @@ export class AudioProcessor { trimRegions?: TrimRegion[], speedRegions?: SpeedRegion[], readEndSec?: number, + audioRegions?: AudioRegion[], ): Promise { const sortedTrims = trimRegions ? [...trimRegions].sort((a, b) => a.startMs - b.startMs) : [] const sortedSpeedRegions = speedRegions @@ -28,13 +30,17 @@ export class AudioProcessor { .filter((region) => region.endMs - region.startMs > MIN_SPEED_REGION_DELTA_MS) .sort((a, b) => a.startMs - b.startMs) : [] + const sortedAudioRegions = audioRegions + ? [...audioRegions].sort((a, b) => a.startMs - b.startMs) + : [] - // Speed edits must use timeline playback to preserve pitch. - if (sortedSpeedRegions.length > 0) { - const renderedAudioBlob = await this.renderPitchPreservedTimelineAudio( + // When audio regions or speed edits are present, use AudioContext mixing path. + if (sortedSpeedRegions.length > 0 || sortedAudioRegions.length > 0) { + const renderedAudioBlob = await this.renderMixedTimelineAudio( videoUrl, sortedTrims, sortedSpeedRegions, + sortedAudioRegions, ) if (!this.cancelled) { await this.muxRenderedAudioBlob(renderedAudioBlob, muxer) @@ -42,7 +48,7 @@ export class AudioProcessor { } } - // No speed edits: keep the original demux/decode/encode path with trim timestamp remap. + // No speed edits or audio regions: keep the original demux/decode/encode path with trim timestamp remap. await this.processTrimOnlyAudio(demuxer, muxer, sortedTrims, readEndSec) } @@ -158,12 +164,13 @@ export class AudioProcessor { } } - // Speed-aware path that mirrors preview semantics (trim skipping + playbackRate regions) - // and preserves pitch through browser media playback behavior. - private async renderPitchPreservedTimelineAudio( + // Renders mixed audio: original video audio (with speed/trim) + external audio regions. + // Uses AudioContext to mix all sources into a single recorded stream. + private async renderMixedTimelineAudio( videoUrl: string, trimRegions: TrimRegion[], speedRegions: SpeedRegion[], + audioRegions: AudioRegion[], ): Promise { const media = document.createElement('audio') media.src = videoUrl @@ -184,10 +191,41 @@ export class AudioProcessor { } const audioContext = new AudioContext() - const sourceNode = audioContext.createMediaElementSource(media) const destinationNode = audioContext.createMediaStreamDestination() + + // Connect original video audio + const sourceNode = audioContext.createMediaElementSource(media) sourceNode.connect(destinationNode) + // Prepare external audio region elements + const audioRegionElements: { + media: HTMLAudioElement + sourceNode: MediaElementAudioSourceNode + gainNode: GainNode + region: AudioRegion + }[] = [] + + for (const region of audioRegions) { + const audioEl = document.createElement('audio') + audioEl.src = toFileUrl(region.audioPath) + audioEl.preload = 'auto' + try { + await this.waitForLoadedMetadata(audioEl) + } catch { + console.warn('[AudioProcessor] Failed to load audio region:', region.audioPath) + continue + } + if (this.cancelled) throw new Error('Export cancelled') + + const regionSource = audioContext.createMediaElementSource(audioEl) + const gainNode = audioContext.createGain() + gainNode.gain.value = Math.max(0, Math.min(1, region.volume)) + regionSource.connect(gainNode) + gainNode.connect(destinationNode) + + audioRegionElements.push({ media: audioEl, sourceNode: regionSource, gainNode, region }) + } + const { recorder, recordedBlobPromise } = this.startAudioRecording(destinationNode.stream) let rafId: number | null = null @@ -211,7 +249,7 @@ export class AudioProcessor { const onError = () => { cleanup() - reject(new Error('Failed while rendering speed-adjusted audio timeline')) + reject(new Error('Failed while rendering mixed audio timeline')) } const onEnded = () => { @@ -246,6 +284,26 @@ export class AudioProcessor { } } + // Sync external audio regions with the video timeline position + for (const entry of audioRegionElements) { + const { media: audioEl, region } = entry + const isInRegion = currentTimeMs >= region.startMs && currentTimeMs < region.endMs + + if (isInRegion) { + const audioOffset = (currentTimeMs - region.startMs) / 1000 + if (audioEl.paused) { + audioEl.currentTime = audioOffset + audioEl.play().catch(() => {}) + } else if (Math.abs(audioEl.currentTime - audioOffset) > 0.3) { + audioEl.currentTime = audioOffset + } + } else { + if (!audioEl.paused) { + audioEl.pause() + } + } + } + if (!media.paused && !media.ended) { rafId = requestAnimationFrame(tick) } else { @@ -263,6 +321,13 @@ export class AudioProcessor { cancelAnimationFrame(rafId) } media.pause() + for (const entry of audioRegionElements) { + entry.media.pause() + entry.sourceNode.disconnect() + entry.gainNode.disconnect() + entry.media.src = '' + entry.media.load() + } if (recorder.state !== 'inactive') { recorder.stop() } diff --git a/src/lib/exporter/types.ts b/src/lib/exporter/types.ts index fa0d4eaa..812d850d 100644 --- a/src/lib/exporter/types.ts +++ b/src/lib/exporter/types.ts @@ -27,7 +27,7 @@ export interface VideoFrameData { duration: number; // in microseconds } -export type ExportQuality = 'medium' | 'good' | 'source'; +export type ExportQuality = 'medium' | 'good' | 'high' | 'source'; // GIF Export Types export type ExportFormat = 'mp4' | 'gif'; diff --git a/src/lib/exporter/videoExporter.ts b/src/lib/exporter/videoExporter.ts index 9eee91ea..ab2a053e 100644 --- a/src/lib/exporter/videoExporter.ts +++ b/src/lib/exporter/videoExporter.ts @@ -3,7 +3,7 @@ import { AudioProcessor } from './audioEncoder'; import { StreamingVideoDecoder } from './streamingDecoder'; import { FrameRenderer } from './frameRenderer'; import { VideoMuxer } from './muxer'; -import type { ZoomRegion, CropRegion, TrimRegion, AnnotationRegion, SpeedRegion, CursorTelemetryPoint } from '@/components/video-editor/types'; +import type { ZoomRegion, CropRegion, TrimRegion, AnnotationRegion, SpeedRegion, AudioRegion, CursorTelemetryPoint } from '@/components/video-editor/types'; interface VideoExporterConfig extends ExportConfig { videoUrl: string; @@ -27,6 +27,7 @@ interface VideoExporterConfig extends ExportConfig { cursorSmoothing?: number; cursorMotionBlur?: number; cursorClickBounce?: number; + audioRegions?: AudioRegion[]; previewWidth?: number; previewHeight?: number; onProgress?: (progress: ExportProgress) => void; @@ -94,7 +95,8 @@ export class VideoExporter { // Initialize video encoder await this.initializeEncoder(); - const hasAudio = videoInfo.hasAudio; + const hasAudioRegions = (this.config.audioRegions ?? []).length > 0; + const hasAudio = videoInfo.hasAudio || hasAudioRegions; // Initialize muxer this.muxer = new VideoMuxer(this.config, hasAudio); @@ -148,15 +150,17 @@ export class VideoExporter { if (hasAudio && !this.cancelled) { const demuxer = this.streamingDecoder.getDemuxer(); - if (demuxer) { + if (demuxer || hasAudioRegions) { this.audioProcessor = new AudioProcessor(); await this.awaitWithWindowsTimeout( this.audioProcessor.process( - demuxer, + demuxer!, this.muxer!, this.config.videoUrl, this.config.trimRegions, this.config.speedRegions, + undefined, + this.config.audioRegions, ), 'audio processing', ); @@ -254,6 +258,14 @@ export class VideoExporter { this.chunkCount = 0; let videoDescription: Uint8Array | undefined; + // Ordered from most capable to most compatible. avc1.PPCCLL where PP=profile, CC=constraints, LL=level. + // High 5.1 → Main 5.1 → Baseline 5.1 → Main 3.1 → Baseline 3.1 + const CODEC_FALLBACK_LIST = this.config.codec + ? [this.config.codec] + : ['avc1.640033', 'avc1.4d4033', 'avc1.420033', 'avc1.4d401f', 'avc1.42001f']; + + let resolvedCodec: string | null = null; + this.encoder = new VideoEncoder({ output: (chunk, meta) => { // Capture decoder config metadata from encoder output @@ -284,7 +296,7 @@ export class VideoExporter { const metadata: EncodedVideoChunkMetadata = { decoderConfig: { - codec: this.config.codec || 'avc1.640033', + codec: resolvedCodec ?? (this.config.codec || 'avc1.640033'), codedWidth: this.config.width, codedHeight: this.config.height, description: this.videoDescription, @@ -303,44 +315,52 @@ export class VideoExporter { this.encodeQueue--; }, error: (error) => { - console.error('[VideoExporter] Encoder error:', error); - // Stop export encoding failed + console.error( + `[VideoExporter] Encoder error (codec: ${resolvedCodec}, ${this.config.width}x${this.config.height}):`, + error, + ); + // Stop export — encoding failed this.cancelled = true; }, }); - const codec = this.config.codec || 'avc1.640033'; - - const encoderConfig: VideoEncoderConfig = { - codec, + const baseConfig: Omit = { width: this.config.width, height: this.config.height, bitrate: this.config.bitrate, framerate: this.config.frameRate, - latencyMode: 'quality', // Changed from 'realtime' to 'quality' for better throughput + latencyMode: 'quality', bitrateMode: 'variable', - hardwareAcceleration: 'prefer-hardware', }; - // Check hardware support first - const hardwareSupport = await VideoEncoder.isConfigSupported(encoderConfig); - - if (hardwareSupport.supported) { - // Use hardware encoding - console.log('[VideoExporter] Using hardware acceleration'); - this.encoder.configure(encoderConfig); - } else { - // Fall back to software encoding - console.log('[VideoExporter] Hardware not supported, using software encoding'); - encoderConfig.hardwareAcceleration = 'prefer-software'; - - const softwareSupport = await VideoEncoder.isConfigSupported(encoderConfig); - if (!softwareSupport.supported) { - throw new Error('Video encoding not supported on this system'); + for (const candidateCodec of CODEC_FALLBACK_LIST) { + const hwConfig: VideoEncoderConfig = { ...baseConfig, codec: candidateCodec, hardwareAcceleration: 'prefer-hardware' }; + const hwSupport = await VideoEncoder.isConfigSupported(hwConfig); + if (hwSupport.supported) { + resolvedCodec = candidateCodec; + console.log(`[VideoExporter] Using hardware acceleration with codec ${candidateCodec}`); + this.encoder.configure(hwConfig); + return; } - this.encoder.configure(encoderConfig); + const swConfig: VideoEncoderConfig = { ...baseConfig, codec: candidateCodec, hardwareAcceleration: 'prefer-software' }; + const swSupport = await VideoEncoder.isConfigSupported(swConfig); + if (swSupport.supported) { + resolvedCodec = candidateCodec; + console.log(`[VideoExporter] Using software encoding with codec ${candidateCodec}`); + this.encoder.configure(swConfig); + return; + } + + console.warn(`[VideoExporter] Codec ${candidateCodec} not supported (${this.config.width}x${this.config.height}), trying next…`); } + + throw new Error( + `Video encoding not supported on this system. ` + + `Tried codecs: ${CODEC_FALLBACK_LIST.join(', ')} at ${this.config.width}x${this.config.height}. ` + + `Your browser or hardware may not support H.264 encoding at this resolution. ` + + `Try exporting at a lower quality setting.`, + ); } cancel(): void {