Add documentation of network capture & replay workflow

docs/behind_scenes/opengl_support.rst

@@ -33,7 +33,7 @@ There are several places where OpenGL is not yet at feature parity with D3D11.
 Linux and OS X
 --------------
 
-Linux and OS X support follows naturally when thinking about OpenGL support. Currently there is full support for capturing and replaying on linux, however the Qt UI is still heavily work in progress. The recommended method is to capture on linux, and then replay on windows.
+Linux and OS X support follows naturally when thinking about OpenGL support. Currently there is full support for capturing and replaying on linux, however the Qt UI is still heavily work in progress. The recommended method is to capture on linux, and then replay on windows. For more information on this see :doc:`../how/how_network_capture_replay`.
 
 See Also
 --------

docs/behind_scenes/vulkan_support.rst

@@ -49,7 +49,7 @@ RenderDoc has initial support for Vulkan at the launch of version 1.0, but it co
 Non-windows platforms
 ---------------------
 
-Linux and other platform support follows naturally when thinking about Vulkan support. Currently there is full support for capturing and replaying on linux, however the Qt UI is still heavily work in progress.
+Linux and other platform support follows naturally when thinking about Vulkan support. Currently there is full support for capturing and replaying on linux, however the Qt UI is still heavily work in progress. For more information on this see :doc:`../how/how_network_capture_replay`.
 
 See Also
 --------

docs/credits_acknowledgements.rst

@@ -127,3 +127,10 @@ There have been many people who have helped in the creation of RenderDoc. Whethe
 * Dominik Witczak
 * Chia-I Wu
 * Cory Bloor
+* John McDonald
+* Pierre-Loup Griffais
+* Jason Mitchell
+* Michael Rennie
+* Ian Elliot
+* Callan McInally
+* Gordon Selley

docs/how/how_capture_log.rst

@@ -52,3 +52,4 @@ See Also
 --------
 
 * :doc:`../window/capture_log_attach`
+* :doc:`how_network_capture_replay`

docs/how/how_network_capture_replay.rst

@@ -0,0 +1,135 @@
+How do I capture and replay over a network?
+===========================================
+
+RenderDoc supports capture and replay over a network connection. While slightly more complicated than the default workflow of doing everything on one machine, there are a number of scenarios where this can be useful.
+
+If you can run the minimal RenderDoc support for capturing on a target machine such as an embedded device or devkit, the bulk of the replay work can be done on an existing machine where the UI is running.
+
+Likewise if you have a capture that you'd like to run on a different driver or hardware, you don't have to set up a full environment to run RenderDoc, you can just run the minimal remote server.
+
+Overview
+--------
+
+RenderDoc's network support has the core concept of a "Replay Context".
+
+The idea is that by default you are in the "local" replay context. All operations happen relative to your own machine - from executables that you launch to capture, to how replays are processed and instantiated. This is the familiar process, where you browse and launch executables on a machine and all the replaying happens locally.
+
+To use a remote machine, you define a replay context for that machine. Otherwise everything works exactly the same way, but executables are run on that machine and captures are replayed on that machine.
+
+Configuring Remote Hosts
+------------------------
+
+To configure a remote host, open the manager by opening the Tools menu and select Manage Remote Servers.
+
+.. figure:: ../imgs/Screenshots/RemoteHostManager.png
+
+	Remote Hosts: Configuring remote hosts for replay contexts.
+
+This manager allows you to configure both which remote hosts are available, as well as starting and stopping the "Remote Server" on that host.
+
+To add a new host, simply type its hostname in the hostname box and click ``Add``. It will be added to the list of known remote hosts, and immediately a network lookup will happen to see if the remote server or any running RenderDoc-injected applications are alive.
+
+At minimum, this is all you must configure. However it is recommended that you configure a command which can be run on the host machine which will remotely start the remote server on that host. The reason for this is that it allows easy capture each time with zero manual extra steps.
+
+.. figure:: ../imgs/Screenshots/RemoteHostConfigure.png
+
+	Remote Hosts: Configuring the properties of a remote host.
+
+To configure such a command, enter it under the "Run Command" box and click ``Update``. If you haven't added the host yet, you can fill out both boxes at once and click ``Add``. This command runs under the context of the host with the UI, and is an arbitrary command. When the remote server is needed, RenderDoc will run this command to attempt to launch it without needing any manual intervention.
+
+The remote server can be run with the command ``renderdoccmd remoteserver``. The exact syntax may depend on your operating system, see ``renderdoccmd remoteserver --help`` for more information, you might want to use the ``-d`` parameter to make the remote server run in the background instead of pausing interactively.
+
+.. note::
+
+  If you are running on linux, it might be worth configuring the DISPLAY environment variable here - that way it will be inherited by every executable that the server runs. Otherwise you would have to configure the environment variable each time when capturing.
+
+.. warning::
+
+	Note that by default the remote server listens to all private IPs, and will allow any connection from those IPs to run commands as the remote server's user without authentication. See later in this document for configuration options to limit the vulnerability this poses.
+
+If a run command is configured you can also manually run the server from this window, as well as shut it down. The remote server is not shut down even when the host UI closes, so the only way to shut it down are either manually from the host itself, or via this window.
+
+An example for this for linux would be to use ``plink.exe`` and passwordless key authentication to run the command on a linux machine:
+
+.. code::
+
+    plink.exe user@host DISPLAY=:0.0 renderdoccmd remoteserver -d
+
+Assuming that plink.exe is in ``PATH`` on the host machine, and ``renderdoccmd`` is on the host machine.    
+
+Switching to a Replay Context
+-----------------------------
+
+Once a remote context is configured, you can switch to it in the bottom left of the status bar. This is only possible when there is no capture currently open.
+
+.. figure:: ../imgs/Screenshots/RemoteContextChooser.png
+
+	Replay Context status: Switching to a remote replay context
+
+The drop-down will show you which remote hosts you have configured are currently already running a remote server, with periodic refreshes. When a remote host is selected, RenderDoc will attempt to connect. If it fails to find an active server but a run command is configured it will execute the command to try and launch the remote server.
+
+If you don't have a run command configured, this is where you would need to launch the server manually. The server can be left running indefinitely until you want to stop it, as it will continue to live on after the UI closes.
+
+.. figure:: ../imgs/Screenshots/RemoteContextStatus.png
+
+	Replay Context status: Status of a remote replay context
+
+The status bar will show the current status of the replay context - whether the remote server could be reached, or if it was busy (as it can only be used for one user's active replay context connection at a time). Likewise if the remote server unexpectedly goes away (e.g. because it was killed remotely, or due to network problems) then the status bar will show that too.
+
+Working in a remote replay context
+----------------------------------
+
+By and large, working in a remote replay context is designed to be transparent to the user. All the familiar operations and workflows will work as expected, perhaps with some small decrease in responsiveness that comes with a network connection's added latency and possible lower specifications of the target system.
+
+In the :doc:`../window/capture_log_attach` window the file and directory browsing is by definition relative to the replay context you are working in. For this reason, a custom dialog is used to display the contents of the remote filesystem instead of the default system dialog for browsing the local computer.
+
+.. figure:: ../imgs/Screenshots/RemoteFileBrowse.png
+
+	Remote Hosts: Browsing the file system within the remote replay context
+
+Likewise, any environment variables set will be relative to the target system's environment and will not inherit anything from the host's system. Specifically, the remote server is used to execute all target programs so the environment will be inherited from it.
+
+Capture files will all be kept on the target system by default. They will only be copied back to the host machine when you explicitly save the file to a path. Otherwise they will be owned by the remote server, and cleaned up as appropriate.
+
+.. note::
+
+  There is a case where temporary captures can be 'leaked' and not cleaned up. If you do not have a remote server run command configured and have captures left over when the program closes, there will be no way to either save or delete the temporary log files. This *doesn't* apply to deleting or saving captures while the program is still running.
+
+  The capture connection will warn you about this case and let you know when you are leaking temporary captures - you can delete them by hand if necessary, or you can switch to a replay context on that host and then you will be able to save and delete them as normal.
+
+  Note that this is the same as if a program is run locally without any connection to the UI made at all - the captures will leak with nothing left to take ownership of them. The difference is that if a connection is made, because the files are on the local machine they can be deleted or saved directly by the UI even after the program has closed.
+
+Capture files made with a recent version of RenderDoc will store a coarse type of machine that was used at capture time, such as 'Linux 64-bit' or 'Windows 32-bit'. If you have the local replay context active and the machine you are running on differs significantly from the machine that the capture was made on, the UI will prompt you to ask if you really want to replay it locally (which may or may not work depending on the API and contents of the log), or switch to a remote context.
+
+.. figure:: ../imgs/Screenshots/RemoteHostCapturePrompt.png
+
+	Remote Hosts: Prompting for remote replay of a notably different log
+
+Configuring the Remote Server
+-----------------------------
+
+The remote server can be configured by a file in ``~/.renderdoc`` or ``%APPDATA%/renderdoc`` called ``remoteserver.conf``. This allows you to restrict which IPs can connect to the server, as well as whether execution is allowed.
+
+To whitelist an IP range, add a line such as this:
+
+.. code::
+
+    whitelist 192.168.0.0/16
+
+Which will allow any IP 192.168.x.x to connect. When the remote server starts, it prints the IP ranges it will allow. If no IP ranges are configured, it will by default listen to all private ranges - ``10.0.0.0/24``, ``192.168.0.0/16``, and ``172.16.0.0/12``.
+
+To prevent the server from ever executing any commands regardless of whether the IP is allowed, add a line such as this:
+
+.. code::
+
+    noexec
+
+This will prevent any execution from happening under any circumstances. Note that if you do this, you will have to launch renderdoc-injected commands another way and the workflow described in this document will not work as-is.
+
+The file also allows blank lines and comments beginning with ``#``.
+
+See Also
+--------
+
+* :doc:`../window/capture_connection`
+* :doc:`../window/capture_log_attach`

docs/how/index.rst

@@ -11,3 +11,4 @@ How do I ...?
 	how_capture_callstack
 	how_custom_visualisation
 	how_edit_shader
+	how_network_capture_replay

docs/renderdoc.hhk

@@ -100,7 +100,7 @@
 	<UL>
 		<LI> <OBJECT type="text/sitemap">
 				<param name="Keyword" value="Capture across a network">
-				<param name="Local" value="window/capture_connection.html#attaching-to-an-existing-instance">
+				<param name="Local" value="how/how_network_capture_replay">
 		</OBJECT>
 		<LI> <OBJECT type="text/sitemap">
 				<param name="Keyword" value="Capture a Log">

docs/window/capture_connection.rst

@@ -6,19 +6,19 @@ The Live capture window opens up when you launch a capture of a program, as well
 Attaching to an existing instance
 ---------------------------------
 
-After you've launched a program through RenderDoc and its hooks are added you can freely disconnect (by closing the live capture window) or close the main UI. You can then connect to this again later, either from the same computer or another computer connecting over the network.
+After you've launched a program through RenderDoc and its hooks are added you can freely disconnect (by closing the live capture window) or close the main UI. You can then connect to this again later, either from the same computer or another computer connecting over the network. For more information, see the page about :doc:`network capture and replay <../how/how_network_capture_replay>`.
 
-To connect to an existing hooked program, select Attach to Running Instance from the File menu. This opens up the attach window that allows you to select a remote host to connect to. By default localhost is already in the list, but you can add and remove other hosts.
+To connect to an existing hooked program, select Attach to Running Instance from the File menu. This opens up the remote host management window that allows you to select a remote host to connect to. localhost is always in the list, but you can add and remove other hosts.
 
 .. warning::
 
-	Please note that none of the connections RenderDoc makes or uses are encrypted or protected, so if this is a concern you should look into securing the connections by hand.
+	Please note that none of the connections RenderDoc makes or uses are encrypted or authenticated/protected, so if this is a concern you should look into securing the connections by hand.
 
-.. figure:: ../imgs/Screenshots/AttachInstance.png
+.. figure:: ../imgs/Screenshots/RemoteHostManager.png
 
 	Remote Hosts: Attaching to a running instance either locally or remotely.
 
-When the window opens, when you add a new host or when you click refresh then the hosts will be queried across the network to see if a connection exists. While this is in progress the host will be listed in italics and with a busy icon.
+When the window opens, when you add a new host, or when you click refresh then the hosts will be queried across the network to see if a connection exists. While this is in progress the host will be listed in italics and with a busy icon.
 
 Once a host has been scanned, if any instances are found then that host can be expanded to see the list, and details are listed about each instance. The name is OS-dependent but is usually the executable name. The API name is listed, as well as the username of any user that is already connected.
 
@@ -28,13 +28,28 @@ When you click connect, a live capture window will be opened up - just the same
 
 	If you connect to a running instance, any existing user will be kicked off. Just seeing the instances running on a host will not.
 
+For more information about specifically a network capture-and-replay workflow, please see :doc:`../how/how_network_capture_replay`.
 
 Capture Connection window
 -------------------------
 
 When a capture is launched (or attached to) the connection window is opened in the main UI. If you end up only taking one capture and closing the program afterwards the connection window will automatically close - likewise if you take no captures at all. These cases don't need the management options the connection window provides.
 
-In addition to managing captures that have been taken, you can also trigger a capture (optionally with a countdown timer).
+While a program is running, a couple of tools are available for triggering specific captures, which can be used as well as the keystroke capture triggers in the application.
+
+.. figure:: ../imgs/Screenshots/LiveCaptureTools.png
+
+	Connection Window: Tools for triggering specific frame captures.
+
+Firstly you can trigger a capture, with an optional delay in seconds, and for an arbitrary number of frames. When you click ``Trigger Capture`` the delay will count down, and then the application will capture the selected number of sequential frames into separate captures. By default the delay is 0 seconds and only one frame is captured.
+
+Next you can trigger a specific frame number, if you know that a bug will happen at a given point.
+
+.. note::
+
+	Note, capturing a frame in RenderDoc will cause a noticable stall on any non-trivial application, so if you are investigating a timing issue often this can throw things off and hide the bug. However capturing multiple frames can be useful when you know a bug alternates every N frames, or you specifically want to debug how N sequential frames look and you know extreme frame times won't skew the results.
+
+During running or after the application has closed, all captures will appear as thumbnails here. As mentioned above, if only one capture is made and the application is closed the capture dialog will automatically begin to load it up in the UI. Otherwise you can use this window to browse the captures, save and delete any frame captures, and open them either in the currently running UI or in a new separate instance.
 
 .. figure:: ../imgs/Screenshots/MultipleCaptures.png
 
@@ -52,7 +67,7 @@ Double clicking on any capture will close any current open capture in the Render
 
 .. figure:: ../imgs/Screenshots/OpenCapNewInstance.png
 
-	New instance: Launch new RenderDoc instance to open this capture.<br />
+	New instance: Launch new RenderDoc instance to open this capture.
 
 Child Processes
 ---------------
@@ -64,3 +79,9 @@ RenderDoc has a particular handling of child processes to help you navigate to t
 
 
 If a process exits, instead of just closing the connection window if there have been no captures, instead RenderDoc looks at the child processes - if there is only one child process, it assume that process must be of interest and immediately switches to tracking that process. If there are *more* than one child process open, the capture connection window will stay open to give you a chance to double click on those child processes to open a new connection window.
+
+See Also
+--------
+
+* :doc:`../how/how_network_capture_replay`
+* :doc:`capture_log_attach`

docs/window/capture_log_attach.rst

@@ -14,13 +14,25 @@ Capturing
 
 To capture a program you simply need to provide the details of the application to be launched.
 
-The Program section of the dialog prompts for the executable to be launched, the working directory, and any command-line arguments to be passed to the program.
+The Program section of the dialog prompts for the executable to be launched, the working directory, any command-line arguments to be passed to the program, and any modifications to the environment to make.
 
 .. figure:: ../imgs/Screenshots/CapturePathCmdline.png
 
   Program Capture: Configuring and launching an exe directly from RenderDoc.
 
-The ``...`` buttons next to the executable path and working directory can be used to browse through the file system. By default if the working directory box is left empty then the directory containing the executable will be used as the working directory.
+The ``...`` buttons next to the executable path and working directory can be used to browse through the file system. If you are working in :doc:`a remote context <../how/how_network_capture_replay>` then the file and directory browser will be replaced by one that browses in the file system of the remote context. By default if the working directory box is left empty then the directory containing the executable will be used as the working directory.
+
+The ``...`` button next to the environment variables line will open up a custom editor that allows you to specify any changes to the environment varibales that you'd like to make when launching the program. This could be used for example to set a configuration option, or perform necessary setup for correct running like configuring a path or setting the ``DISPLAY`` variable.
+
+.. figure:: ../imgs/Screenshots/CaptureEnvVarEditor.png
+
+  Environment Variables: changing the environment that the executable will run under.
+
+In the environment variable editor you can choose the name of the variable to change, and the value to use. Then you can choose the function to use.
+
+* ``Set`` will overwrite any existing value or create the variable with the value you specify.
+* ``Prepend`` and ``Append`` will add the value you specify to the start or beginning of the existing variable. If the variable does not already exist it will be created with the value.
+  When prepending or appending you can also choose the separator to apply, for example if you are modifying the ``PATH`` variable. You can choose either colons (``:``), semi-colons (``;``), platform style (``;`` on windows and ``:`` on other platforms), or no separator at all. If the variable does not exist, the separator won't be added, but if the variable does exist the separator will be added in between the existing value and the value you chose to add.
 
 When you are ready to capture simply click the ``Capture`` button in the bottom right.
 
@@ -182,3 +194,4 @@ See Also
 * :doc:`../getting_started/quick_start`
 * :doc:`../how/how_capture_callstack`
 * :doc:`../how/how_capture_log`
+* :doc:`../how/how_network_capture_replay`